From solipsis at pitrou.net Fri Jan 1 03:41:19 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 01 Jan 2016 08:41:19 +0000 Subject: [Python-checkins] Daily reference leaks (1ab7bcd4e176): sum=49 Message-ID: <20160101084119.21384.73028@psf.io> results for 1ab7bcd4e176 on branch "default" -------------------------------------------- test_asyncio leaked [34, 0, 0] references, sum=34 test_asyncio leaked [11, 0, 0] memory blocks, sum=11 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog4AKyDU', '--timeout', '7200'] From python-checkins at python.org Fri Jan 1 11:25:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 16:25:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_2016_will_be_a?= =?utf-8?q?nother_year_of_writing_copyrighted_code?= Message-ID: <20160101162555.9644.73446@psf.io> https://hg.python.org/cpython/rev/ecaa4bcca590 changeset: 99730:ecaa4bcca590 branch: 3.3 parent: 99456:39a709f1ca3a user: Benjamin Peterson date: Fri Jan 01 10:23:45 2016 -0600 summary: 2016 will be another year of writing copyrighted code files: Doc/copyright.rst | 2 +- Doc/license.rst | 2 +- LICENSE | 2 +- PC/python_nt.rc | 2 +- Python/getcopyright.c | 2 +- README | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/copyright.rst b/Doc/copyright.rst --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Python and this documentation is: -Copyright ?? 2001-2015 Python Software Foundation. All rights reserved. +Copyright ?? 2001-2016 Python Software Foundation. All rights reserved. Copyright ?? 2000 BeOpen.com. All rights reserved. diff --git a/Doc/license.rst b/Doc/license.rst --- a/Doc/license.rst +++ b/Doc/license.rst @@ -84,7 +84,7 @@ analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright ?? 2001-2015 Python Software Foundation; All Rights + copyright, i.e., "Copyright ?? 2001-2016 Python Software Foundation; All Rights Reserved" are retained in Python |release| alone or in any derivative version prepared by Licensee. diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,7 +74,7 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" +2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on diff --git a/PC/python_nt.rc b/PC/python_nt.rc --- a/PC/python_nt.rc +++ b/PC/python_nt.rc @@ -61,7 +61,7 @@ VALUE "FileDescription", "Python Core\0" VALUE "FileVersion", PYTHON_VERSION VALUE "InternalName", "Python DLL\0" - VALUE "LegalCopyright", "Copyright ? 2001-2015 Python Software Foundation. Copyright ? 2000 BeOpen.com. Copyright ? 1995-2001 CNRI. Copyright ? 1991-1995 SMC.\0" + VALUE "LegalCopyright", "Copyright ? 2001-2016 Python Software Foundation. Copyright ? 2000 BeOpen.com. Copyright ? 1995-2001 CNRI. Copyright ? 1991-1995 SMC.\0" VALUE "OriginalFilename", PYTHON_DLL_NAME "\0" VALUE "ProductName", "Python\0" VALUE "ProductVersion", PYTHON_VERSION diff --git a/Python/getcopyright.c b/Python/getcopyright.c --- a/Python/getcopyright.c +++ b/Python/getcopyright.c @@ -4,7 +4,7 @@ static const char cprt[] = "\ -Copyright (c) 2001-2015 Python Software Foundation.\n\ +Copyright (c) 2001-2016 Python Software Foundation.\n\ All Rights Reserved.\n\ \n\ Copyright (c) 2000 BeOpen.com.\n\ diff --git a/README b/README --- a/README +++ b/README @@ -2,7 +2,7 @@ ============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Python 3.x is a new version of the language, which is incompatible with the 2.x line of releases. The language is mostly the same, but many details, especially -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 11:25:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 16:25:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <20160101162555.126311.28995@psf.io> https://hg.python.org/cpython/rev/a74d808a3d2d changeset: 99731:a74d808a3d2d branch: 3.4 parent: 99650:73ad9db1dce6 parent: 99730:ecaa4bcca590 user: Benjamin Peterson date: Fri Jan 01 10:24:21 2016 -0600 summary: merge 3.3 files: Doc/copyright.rst | 2 +- Doc/license.rst | 6 +++--- LICENSE | 2 +- PC/python_nt.rc | 2 +- Python/getcopyright.c | 2 +- README | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/copyright.rst b/Doc/copyright.rst --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Python and this documentation is: -Copyright ?? 2001-2015 Python Software Foundation. All rights reserved. +Copyright ?? 2001-2016 Python Software Foundation. All rights reserved. Copyright ?? 2000 BeOpen.com. All rights reserved. diff --git a/Doc/license.rst b/Doc/license.rst --- a/Doc/license.rst +++ b/Doc/license.rst @@ -84,9 +84,9 @@ analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright ?? 2001-2015 Python Software Foundation; All - Rights Reserved" are retained in Python |release| alone or in any derivative - version prepared by Licensee. + copyright, i.e., "Copyright ?? 2001-2016 Python Software Foundation; All Rights + Reserved" are retained in Python |release| alone or in any derivative version + prepared by Licensee. #. In the event Licensee prepares a derivative work that is based on or incorporates Python |release| or any part thereof, and wants to make the diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,7 +74,7 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" +2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on diff --git a/PC/python_nt.rc b/PC/python_nt.rc --- a/PC/python_nt.rc +++ b/PC/python_nt.rc @@ -61,7 +61,7 @@ VALUE "FileDescription", "Python Core\0" VALUE "FileVersion", PYTHON_VERSION VALUE "InternalName", "Python DLL\0" - VALUE "LegalCopyright", "Copyright ? 2001-2015 Python Software Foundation. Copyright ? 2000 BeOpen.com. Copyright ? 1995-2001 CNRI. Copyright ? 1991-1995 SMC.\0" + VALUE "LegalCopyright", "Copyright ? 2001-2016 Python Software Foundation. Copyright ? 2000 BeOpen.com. Copyright ? 1995-2001 CNRI. Copyright ? 1991-1995 SMC.\0" VALUE "OriginalFilename", PYTHON_DLL_NAME "\0" VALUE "ProductName", "Python\0" VALUE "ProductVersion", PYTHON_VERSION diff --git a/Python/getcopyright.c b/Python/getcopyright.c --- a/Python/getcopyright.c +++ b/Python/getcopyright.c @@ -4,7 +4,7 @@ static const char cprt[] = "\ -Copyright (c) 2001-2015 Python Software Foundation.\n\ +Copyright (c) 2001-2016 Python Software Foundation.\n\ All Rights Reserved.\n\ \n\ Copyright (c) 2000 BeOpen.com.\n\ diff --git a/README b/README --- a/README +++ b/README @@ -2,7 +2,7 @@ ============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Python 3.x is a new version of the language, which is incompatible with the 2.x line of releases. The language is mostly the same, but many details, especially -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 11:25:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 16:25:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_2016_will_be_a?= =?utf-8?q?nother_year_of_writing_copyrighted_code?= Message-ID: <20160101162556.124848.81421@psf.io> https://hg.python.org/cpython/rev/2c4d6dd4f5f8 changeset: 99734:2c4d6dd4f5f8 branch: 2.7 parent: 99729:57fea6f75ac2 user: Benjamin Peterson date: Fri Jan 01 10:23:45 2016 -0600 summary: 2016 will be another year of writing copyrighted code files: Doc/copyright.rst | 2 +- Doc/license.rst | 2 +- LICENSE | 2 +- PC/python_nt.rc | 2 +- Python/getcopyright.c | 2 +- README | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/copyright.rst b/Doc/copyright.rst --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Python and this documentation is: -Copyright ?? 2001-2015 Python Software Foundation. All rights reserved. +Copyright ?? 2001-2016 Python Software Foundation. All rights reserved. Copyright ?? 2000 BeOpen.com. All rights reserved. diff --git a/Doc/license.rst b/Doc/license.rst --- a/Doc/license.rst +++ b/Doc/license.rst @@ -84,7 +84,7 @@ analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright ?? 2001-2015 Python Software Foundation; All Rights + copyright, i.e., "Copyright ?? 2001-2016 Python Software Foundation; All Rights Reserved" are retained in Python |release| alone or in any derivative version prepared by Licensee. diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,7 +74,7 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" +2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on diff --git a/PC/python_nt.rc b/PC/python_nt.rc --- a/PC/python_nt.rc +++ b/PC/python_nt.rc @@ -61,7 +61,7 @@ VALUE "FileDescription", "Python Core\0" VALUE "FileVersion", PYTHON_VERSION VALUE "InternalName", "Python DLL\0" - VALUE "LegalCopyright", "Copyright ? 2001-2015 Python Software Foundation. Copyright ? 2000 BeOpen.com. Copyright ? 1995-2001 CNRI. Copyright ? 1991-1995 SMC.\0" + VALUE "LegalCopyright", "Copyright ? 2001-2016 Python Software Foundation. Copyright ? 2000 BeOpen.com. Copyright ? 1995-2001 CNRI. Copyright ? 1991-1995 SMC.\0" VALUE "OriginalFilename", PYTHON_DLL_NAME "\0" VALUE "ProductName", "Python\0" VALUE "ProductVersion", PYTHON_VERSION diff --git a/Python/getcopyright.c b/Python/getcopyright.c --- a/Python/getcopyright.c +++ b/Python/getcopyright.c @@ -4,7 +4,7 @@ static char cprt[] = "\ -Copyright (c) 2001-2015 Python Software Foundation.\n\ +Copyright (c) 2001-2016 Python Software Foundation.\n\ All Rights Reserved.\n\ \n\ Copyright (c) 2000 BeOpen.com.\n\ diff --git a/README b/README --- a/README +++ b/README @@ -2,7 +2,7 @@ ============================= Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. All rights reserved. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 11:25:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 16:25:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160101162555.70429.53759@psf.io> https://hg.python.org/cpython/rev/7d2364aa0189 changeset: 99732:7d2364aa0189 branch: 3.5 parent: 99727:d2417971c934 parent: 99731:a74d808a3d2d user: Benjamin Peterson date: Fri Jan 01 10:25:12 2016 -0600 summary: merge 3.4 files: Doc/copyright.rst | 2 +- Doc/license.rst | 6 +++--- LICENSE | 2 +- PC/python_ver_rc.h | 2 +- Python/getcopyright.c | 2 +- README | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/copyright.rst b/Doc/copyright.rst --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Python and this documentation is: -Copyright ? 2001-2015 Python Software Foundation. All rights reserved. +Copyright ? 2001-2016 Python Software Foundation. All rights reserved. Copyright ? 2000 BeOpen.com. All rights reserved. diff --git a/Doc/license.rst b/Doc/license.rst --- a/Doc/license.rst +++ b/Doc/license.rst @@ -84,9 +84,9 @@ analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright ? 2001-2015 Python Software Foundation; All - Rights Reserved" are retained in Python |release| alone or in any derivative - version prepared by Licensee. + copyright, i.e., "Copyright ? 2001-2016 Python Software Foundation; All Rights + Reserved" are retained in Python |release| alone or in any derivative version + prepared by Licensee. #. In the event Licensee prepares a derivative work that is based on or incorporates Python |release| or any part thereof, and wants to make the diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,7 +74,7 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" +2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on diff --git a/PC/python_ver_rc.h b/PC/python_ver_rc.h --- a/PC/python_ver_rc.h +++ b/PC/python_ver_rc.h @@ -4,7 +4,7 @@ #include "winver.h" #define PYTHON_COMPANY "Python Software Foundation" -#define PYTHON_COPYRIGHT "Copyright \xA9 2001-2015 Python Software Foundation. Copyright \xA9 2000 BeOpen.com. Copyright \xA9 1995-2001 CNRI. Copyright \xA9 1991-1995 SMC." +#define PYTHON_COPYRIGHT "Copyright \xA9 2001-2016 Python Software Foundation. Copyright \xA9 2000 BeOpen.com. Copyright \xA9 1995-2001 CNRI. Copyright \xA9 1991-1995 SMC." #define MS_WINDOWS #include "modsupport.h" diff --git a/Python/getcopyright.c b/Python/getcopyright.c --- a/Python/getcopyright.c +++ b/Python/getcopyright.c @@ -4,7 +4,7 @@ static const char cprt[] = "\ -Copyright (c) 2001-2015 Python Software Foundation.\n\ +Copyright (c) 2001-2016 Python Software Foundation.\n\ All Rights Reserved.\n\ \n\ Copyright (c) 2000 BeOpen.com.\n\ diff --git a/README b/README --- a/README +++ b/README @@ -2,7 +2,7 @@ ============================ Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Python 3.x is a new version of the language, which is incompatible with the 2.x line of releases. The language is mostly the same, but many details, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 11:25:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 16:25:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160101162555.77237.75437@psf.io> https://hg.python.org/cpython/rev/0b8ff5216100 changeset: 99733:0b8ff5216100 parent: 99728:1ab7bcd4e176 parent: 99732:7d2364aa0189 user: Benjamin Peterson date: Fri Jan 01 10:25:22 2016 -0600 summary: merge 3.5 files: Doc/copyright.rst | 2 +- Doc/license.rst | 6 +++--- LICENSE | 2 +- PC/python_ver_rc.h | 2 +- Python/getcopyright.c | 2 +- README | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/copyright.rst b/Doc/copyright.rst --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Python and this documentation is: -Copyright ? 2001-2015 Python Software Foundation. All rights reserved. +Copyright ? 2001-2016 Python Software Foundation. All rights reserved. Copyright ? 2000 BeOpen.com. All rights reserved. diff --git a/Doc/license.rst b/Doc/license.rst --- a/Doc/license.rst +++ b/Doc/license.rst @@ -84,9 +84,9 @@ analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright ? 2001-2015 Python Software Foundation; All - Rights Reserved" are retained in Python |release| alone or in any derivative - version prepared by Licensee. + copyright, i.e., "Copyright ? 2001-2016 Python Software Foundation; All Rights + Reserved" are retained in Python |release| alone or in any derivative version + prepared by Licensee. #. In the event Licensee prepares a derivative work that is based on or incorporates Python |release| or any part thereof, and wants to make the diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,7 +74,7 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" +2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on diff --git a/PC/python_ver_rc.h b/PC/python_ver_rc.h --- a/PC/python_ver_rc.h +++ b/PC/python_ver_rc.h @@ -4,7 +4,7 @@ #include "winver.h" #define PYTHON_COMPANY "Python Software Foundation" -#define PYTHON_COPYRIGHT "Copyright \xA9 2001-2015 Python Software Foundation. Copyright \xA9 2000 BeOpen.com. Copyright \xA9 1995-2001 CNRI. Copyright \xA9 1991-1995 SMC." +#define PYTHON_COPYRIGHT "Copyright \xA9 2001-2016 Python Software Foundation. Copyright \xA9 2000 BeOpen.com. Copyright \xA9 1995-2001 CNRI. Copyright \xA9 1991-1995 SMC." #define MS_WINDOWS #include "modsupport.h" diff --git a/Python/getcopyright.c b/Python/getcopyright.c --- a/Python/getcopyright.c +++ b/Python/getcopyright.c @@ -4,7 +4,7 @@ static const char cprt[] = "\ -Copyright (c) 2001-2015 Python Software Foundation.\n\ +Copyright (c) 2001-2016 Python Software Foundation.\n\ All Rights Reserved.\n\ \n\ Copyright (c) 2000 BeOpen.com.\n\ diff --git a/README b/README --- a/README +++ b/README @@ -2,7 +2,7 @@ ==================================== Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Python 3.x is a new version of the language, which is incompatible with the 2.x line of releases. The language is mostly the same, but many details, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:13:38 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:13:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogcmVmbG93?= Message-ID: <20160101171338.9630.70307@psf.io> https://hg.python.org/cpython/rev/a84a15eafb76 changeset: 99735:a84a15eafb76 branch: 3.3 parent: 99730:ecaa4bcca590 user: Benjamin Peterson date: Fri Jan 01 11:12:44 2016 -0600 summary: reflow files: LICENSE | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,8 +74,9 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" -are retained in Python alone or in any derivative version prepared by Licensee. +2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights +Reserved" are retained in Python alone or in any derivative version prepared by +Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:13:39 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:13:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <20160101171338.124872.12695@psf.io> https://hg.python.org/cpython/rev/4a8c40b32969 changeset: 99736:4a8c40b32969 branch: 3.4 parent: 99731:a74d808a3d2d parent: 99735:a84a15eafb76 user: Benjamin Peterson date: Fri Jan 01 11:12:49 2016 -0600 summary: merge 3.3 files: LICENSE | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,8 +74,9 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" -are retained in Python alone or in any derivative version prepared by Licensee. +2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights +Reserved" are retained in Python alone or in any derivative version prepared by +Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:13:39 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:13:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160101171339.70419.17389@psf.io> https://hg.python.org/cpython/rev/90ee10c40b4b changeset: 99737:90ee10c40b4b branch: 3.5 parent: 99732:7d2364aa0189 parent: 99736:4a8c40b32969 user: Benjamin Peterson date: Fri Jan 01 11:13:10 2016 -0600 summary: merge 3.4 files: LICENSE | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,8 +74,9 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" -are retained in Python alone or in any derivative version prepared by Licensee. +2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights +Reserved" are retained in Python alone or in any derivative version prepared by +Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:13:39 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:13:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogcmVmbG93?= Message-ID: <20160101171339.21241.9997@psf.io> https://hg.python.org/cpython/rev/a0fcd73b04fe changeset: 99739:a0fcd73b04fe branch: 2.7 parent: 99734:2c4d6dd4f5f8 user: Benjamin Peterson date: Fri Jan 01 11:12:44 2016 -0600 summary: reflow files: LICENSE | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,8 +74,9 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" -are retained in Python alone or in any derivative version prepared by Licensee. +2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights +Reserved" are retained in Python alone or in any derivative version prepared by +Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:13:39 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:13:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160101171339.8289.13653@psf.io> https://hg.python.org/cpython/rev/816d21b55c6f changeset: 99738:816d21b55c6f parent: 99733:0b8ff5216100 parent: 99737:90ee10c40b4b user: Benjamin Peterson date: Fri Jan 01 11:13:16 2016 -0600 summary: merge 3.5 files: LICENSE | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/LICENSE b/LICENSE --- a/LICENSE +++ b/LICENSE @@ -74,8 +74,9 @@ distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015,2016 Python Software Foundation; All Rights Reserved" -are retained in Python alone or in any derivative version prepared by Licensee. +2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights +Reserved" are retained in Python alone or in any derivative version prepared by +Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:58:01 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:58:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_remove_some_co?= =?utf-8?q?pyright_notices_supserseded_by_the_toplevel_ones?= Message-ID: <20160101175801.77233.55461@psf.io> https://hg.python.org/cpython/rev/cd8cbe513597 changeset: 99741:cd8cbe513597 branch: 3.3 user: Benjamin Peterson date: Fri Jan 01 11:53:47 2016 -0600 summary: remove some copyright notices supserseded by the toplevel ones files: Doc/README.txt | 24 ------------------------ Modules/_collectionsmodule.c | 2 -- Modules/itertoolsmodule.c | 2 -- Objects/setobject.c | 3 --- 4 files changed, 0 insertions(+), 31 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -108,27 +108,3 @@ If you want to help the Documentation Team, you are always welcome. Just send a mail to docs at python.org. - - -Copyright notice -================ - -The Python source is copyrighted, but you can freely use and copy it -as long as you don't change or remove the copyright notice: - ----------------------------------------------------------------------- -Copyright (c) 2000-2014 Python Software Foundation. -All rights reserved. - -Copyright (c) 2000 BeOpen.com. -All rights reserved. - -Copyright (c) 1995-2000 Corporation for National Research Initiatives. -All rights reserved. - -Copyright (c) 1991-1995 Stichting Mathematisch Centrum. -All rights reserved. - -See the file "license.rst" for information on usage and redistribution -of this file, and for a DISCLAIMER OF ALL WARRANTIES. ----------------------------------------------------------------------- diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -3,8 +3,6 @@ /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger - Copyright (c) 2004 Python Software Foundation. - All rights reserved. */ /* The block length may be set to any number over 1. Larger numbers diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -4,8 +4,6 @@ /* Itertools module written and maintained by Raymond D. Hettinger - Copyright (c) 2003-2013 Python Software Foundation. - All rights reserved. */ diff --git a/Objects/setobject.c b/Objects/setobject.c --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -2,9 +2,6 @@ /* set object implementation Written and maintained by Raymond D. Hettinger Derived from Lib/sets.py and Objects/dictobject.c. - - Copyright (c) 2003-2008 Python Software Foundation. - All rights reserved. */ #include "Python.h" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:58:01 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:58:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_add_2015_and_2?= =?utf-8?q?016?= Message-ID: <20160101175800.124848.81939@psf.io> https://hg.python.org/cpython/rev/7c9d87745348 changeset: 99740:7c9d87745348 branch: 3.3 parent: 99735:a84a15eafb76 user: Benjamin Peterson date: Fri Jan 01 11:53:14 2016 -0600 summary: add 2015 and 2016 files: README | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -175,7 +175,7 @@ --------------------------------- Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. All rights reserved. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:58:01 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:58:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <20160101175801.77243.97096@psf.io> https://hg.python.org/cpython/rev/71c7e62bef0b changeset: 99742:71c7e62bef0b branch: 3.4 parent: 99736:4a8c40b32969 parent: 99741:cd8cbe513597 user: Benjamin Peterson date: Fri Jan 01 11:55:47 2016 -0600 summary: merge 3.3 files: Doc/README.txt | 24 ------------------------ Modules/_collectionsmodule.c | 2 -- Modules/itertoolsmodule.c | 2 -- Objects/setobject.c | 3 --- README | 2 +- 5 files changed, 1 insertions(+), 32 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -119,27 +119,3 @@ If you want to help the Documentation Team, you are always welcome. Just send a mail to docs at python.org. - - -Copyright notice -================ - -The Python source is copyrighted, but you can freely use and copy it -as long as you don't change or remove the copyright notice: - ----------------------------------------------------------------------- -Copyright (c) 2000-2015 Python Software Foundation. -All rights reserved. - -Copyright (c) 2000 BeOpen.com. -All rights reserved. - -Copyright (c) 1995-2000 Corporation for National Research Initiatives. -All rights reserved. - -Copyright (c) 1991-1995 Stichting Mathematisch Centrum. -All rights reserved. - -See the file "license.rst" for information on usage and redistribution -of this file, and for a DISCLAIMER OF ALL WARRANTIES. ----------------------------------------------------------------------- diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -3,8 +3,6 @@ /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger - Copyright (c) 2004-2013 Python Software Foundation. - All rights reserved. */ /* The block length may be set to any number over 1. Larger numbers diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -4,8 +4,6 @@ /* Itertools module written and maintained by Raymond D. Hettinger - Copyright (c) 2003-2013 Python Software Foundation. - All rights reserved. */ diff --git a/Objects/setobject.c b/Objects/setobject.c --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -4,9 +4,6 @@ Written and maintained by Raymond D. Hettinger Derived from Lib/sets.py and Objects/dictobject.c. - Copyright (c) 2003-2013 Python Software Foundation. - All rights reserved. - The basic lookup function used by all operations. This is based on Algorithm D from Knuth Vol. 3, Sec. 6.4. diff --git a/README b/README --- a/README +++ b/README @@ -175,7 +175,7 @@ --------------------------------- Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. All rights reserved. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:58:01 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:58:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160101175801.3371.13287@psf.io> https://hg.python.org/cpython/rev/eca9896edadd changeset: 99744:eca9896edadd parent: 99738:816d21b55c6f parent: 99743:391a6b323c5e user: Benjamin Peterson date: Fri Jan 01 11:56:35 2016 -0600 summary: merge 3.5 files: Doc/README.txt | 24 ------------------------ Modules/_collectionsmodule.c | 2 -- Modules/itertoolsmodule.c | 2 -- Objects/setobject.c | 3 --- README | 2 +- 5 files changed, 1 insertions(+), 32 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -119,27 +119,3 @@ If you want to help the Documentation Team, you are always welcome. Just send a mail to docs at python.org. - - -Copyright notice -================ - -The Python source is copyrighted, but you can freely use and copy it -as long as you don't change or remove the copyright notice: - ----------------------------------------------------------------------- -Copyright (c) 2000-2015 Python Software Foundation. -All rights reserved. - -Copyright (c) 2000 BeOpen.com. -All rights reserved. - -Copyright (c) 1995-2000 Corporation for National Research Initiatives. -All rights reserved. - -Copyright (c) 1991-1995 Stichting Mathematisch Centrum. -All rights reserved. - -See the file "license.rst" for information on usage and redistribution -of this file, and for a DISCLAIMER OF ALL WARRANTIES. ----------------------------------------------------------------------- diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -9,8 +9,6 @@ /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger - Copyright (c) 2004-2015 Python Software Foundation. - All rights reserved. */ /* The block length may be set to any number over 1. Larger numbers diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -5,8 +5,6 @@ /* Itertools module written and maintained by Raymond D. Hettinger - Copyright (c) 2003-2015 Python Software Foundation. - All rights reserved. */ diff --git a/Objects/setobject.c b/Objects/setobject.c --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -4,9 +4,6 @@ Written and maintained by Raymond D. Hettinger Derived from Lib/sets.py and Objects/dictobject.c. - Copyright (c) 2003-2015 Python Software Foundation. - All rights reserved. - The basic lookup function used by all operations. This is based on Algorithm D from Knuth Vol. 3, Sec. 6.4. diff --git a/README b/README --- a/README +++ b/README @@ -203,7 +203,7 @@ --------------------------------- Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. All rights reserved. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:58:02 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:58:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_remove_some_co?= =?utf-8?q?pyright_notices_supserseded_by_the_toplevel_ones?= Message-ID: <20160101175801.9618.5760@psf.io> https://hg.python.org/cpython/rev/6096e9c0906b changeset: 99745:6096e9c0906b branch: 2.7 parent: 99739:a0fcd73b04fe user: Benjamin Peterson date: Fri Jan 01 11:53:47 2016 -0600 summary: remove some copyright notices supserseded by the toplevel ones files: Doc/README.txt | 24 ------------------------ Modules/_collectionsmodule.c | 2 -- Modules/itertoolsmodule.c | 2 -- Objects/setobject.c | 3 --- 4 files changed, 0 insertions(+), 31 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -119,27 +119,3 @@ If you want to help the Documentation Team, you are always welcome. Just send a mail to docs at python.org. - - -Copyright notice -================ - -The Python source is copyrighted, but you can freely use and copy it -as long as you don't change or remove the copyright notice: - ----------------------------------------------------------------------- -Copyright (c) 2000-2014 Python Software Foundation. -All rights reserved. - -Copyright (c) 2000 BeOpen.com. -All rights reserved. - -Copyright (c) 1995-2000 Corporation for National Research Initiatives. -All rights reserved. - -Copyright (c) 1991-1995 Stichting Mathematisch Centrum. -All rights reserved. - -See the file "license.rst" for information on usage and redistribution -of this file, and for a DISCLAIMER OF ALL WARRANTIES. ----------------------------------------------------------------------- diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -3,8 +3,6 @@ /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger - Copyright (c) 2004 Python Software Foundation. - All rights reserved. */ /* The block length may be set to any number over 1. Larger numbers diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -4,8 +4,6 @@ /* Itertools module written and maintained by Raymond D. Hettinger - Copyright (c) 2003 Python Software Foundation. - All rights reserved. */ diff --git a/Objects/setobject.c b/Objects/setobject.c --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -2,9 +2,6 @@ /* set object implementation Written and maintained by Raymond D. Hettinger Derived from Lib/sets.py and Objects/dictobject.c. - - Copyright (c) 2003-2007 Python Software Foundation. - All rights reserved. */ #include "Python.h" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 12:58:01 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 01 Jan 2016 17:58:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160101175801.126297.76242@psf.io> https://hg.python.org/cpython/rev/391a6b323c5e changeset: 99743:391a6b323c5e branch: 3.5 parent: 99737:90ee10c40b4b parent: 99742:71c7e62bef0b user: Benjamin Peterson date: Fri Jan 01 11:56:16 2016 -0600 summary: merge 3.4 files: Doc/README.txt | 24 ------------------------ Modules/_collectionsmodule.c | 2 -- Modules/itertoolsmodule.c | 2 -- Objects/setobject.c | 3 --- README | 2 +- 5 files changed, 1 insertions(+), 32 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -119,27 +119,3 @@ If you want to help the Documentation Team, you are always welcome. Just send a mail to docs at python.org. - - -Copyright notice -================ - -The Python source is copyrighted, but you can freely use and copy it -as long as you don't change or remove the copyright notice: - ----------------------------------------------------------------------- -Copyright (c) 2000-2015 Python Software Foundation. -All rights reserved. - -Copyright (c) 2000 BeOpen.com. -All rights reserved. - -Copyright (c) 1995-2000 Corporation for National Research Initiatives. -All rights reserved. - -Copyright (c) 1991-1995 Stichting Mathematisch Centrum. -All rights reserved. - -See the file "license.rst" for information on usage and redistribution -of this file, and for a DISCLAIMER OF ALL WARRANTIES. ----------------------------------------------------------------------- diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -9,8 +9,6 @@ /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger - Copyright (c) 2004-2015 Python Software Foundation. - All rights reserved. */ /* The block length may be set to any number over 1. Larger numbers diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -4,8 +4,6 @@ /* Itertools module written and maintained by Raymond D. Hettinger - Copyright (c) 2003-2013 Python Software Foundation. - All rights reserved. */ diff --git a/Objects/setobject.c b/Objects/setobject.c --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -4,9 +4,6 @@ Written and maintained by Raymond D. Hettinger Derived from Lib/sets.py and Objects/dictobject.c. - Copyright (c) 2003-2015 Python Software Foundation. - All rights reserved. - The basic lookup function used by all operations. This is based on Algorithm D from Knuth Vol. 3, Sec. 6.4. diff --git a/README b/README --- a/README +++ b/README @@ -203,7 +203,7 @@ --------------------------------- Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved. +2012, 2013, 2014, 2015, 2016 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. All rights reserved. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 13:24:21 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 01 Jan 2016 18:24:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4IGdyYW1tYXIu?= Message-ID: <20160101182421.21388.6109@psf.io> https://hg.python.org/cpython/rev/2727211587f2 changeset: 99747:2727211587f2 branch: 3.5 parent: 99743:391a6b323c5e user: Zachary Ware date: Fri Jan 01 12:22:16 2016 -0600 summary: Fix grammar. Reported by Anatoly Techtonik on docs@ files: Doc/library/ctypes.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -1291,7 +1291,7 @@ Loading shared libraries ^^^^^^^^^^^^^^^^^^^^^^^^ -There are several ways to loaded shared libraries into the Python process. One +There are several ways to load shared libraries into the Python process. One way is to instantiate one of the following classes: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 13:24:21 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 01 Jan 2016 18:24:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4IGdyYW1tYXIu?= Message-ID: <20160101182421.126303.43436@psf.io> https://hg.python.org/cpython/rev/25b485deb67a changeset: 99746:25b485deb67a branch: 2.7 user: Zachary Ware date: Fri Jan 01 12:22:16 2016 -0600 summary: Fix grammar. Reported by Anatoly Techtonik on docs@ files: Doc/library/ctypes.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -1348,7 +1348,7 @@ Loading shared libraries ^^^^^^^^^^^^^^^^^^^^^^^^ -There are several ways to loaded shared libraries into the Python process. One +There are several ways to load shared libraries into the Python process. One way is to instantiate one of the following classes: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 13:24:21 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 01 Jan 2016 18:24:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160101182421.21386.98291@psf.io> https://hg.python.org/cpython/rev/22f1ddac9cb7 changeset: 99748:22f1ddac9cb7 parent: 99744:eca9896edadd parent: 99747:2727211587f2 user: Zachary Ware date: Fri Jan 01 12:24:12 2016 -0600 summary: Merge with 3.5 files: Doc/library/ctypes.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -1291,7 +1291,7 @@ Loading shared libraries ^^^^^^^^^^^^^^^^^^^^^^^^ -There are several ways to loaded shared libraries into the Python process. One +There are several ways to load shared libraries into the Python process. One way is to instantiate one of the following classes: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 17:46:59 2016 From: python-checkins at python.org (ned.deily) Date: Fri, 01 Jan 2016 22:46:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Update_copyrig?= =?utf-8?q?hts_for_2016=2E?= Message-ID: <20160101224659.126323.78372@psf.io> https://hg.python.org/cpython/rev/d705db81864f changeset: 99749:d705db81864f branch: 2.7 parent: 99746:25b485deb67a user: Ned Deily date: Fri Jan 01 17:45:31 2016 -0500 summary: Update copyrights for 2016. files: Mac/BuildScript/resources/License.rtf | 2 +- Mac/IDLE/Info.plist.in | 2 +- Mac/PythonLauncher/Info.plist.in | 2 +- Mac/Resources/app/Info.plist.in | 6 +++--- Mac/Resources/framework/Info.plist.in | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -54,7 +54,7 @@ \b0 \ 1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation.\ \ -2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ +2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ \ 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python.\ \ diff --git a/Mac/IDLE/Info.plist.in b/Mac/IDLE/Info.plist.in --- a/Mac/IDLE/Info.plist.in +++ b/Mac/IDLE/Info.plist.in @@ -36,7 +36,7 @@ CFBundleExecutable IDLE CFBundleGetInfoString - %VERSION%, ? 2001-2015 Python Software Foundation + %VERSION%, ? 2001-2016 Python Software Foundation CFBundleIconFile IDLE.icns CFBundleIdentifier diff --git a/Mac/PythonLauncher/Info.plist.in b/Mac/PythonLauncher/Info.plist.in --- a/Mac/PythonLauncher/Info.plist.in +++ b/Mac/PythonLauncher/Info.plist.in @@ -40,7 +40,7 @@ CFBundleExecutable PythonLauncher CFBundleGetInfoString - %VERSION%, ? 2001-2015 Python Software Foundation + %VERSION%, ? 2001-2016 Python Software Foundation CFBundleIconFile PythonLauncher.icns CFBundleIdentifier diff --git a/Mac/Resources/app/Info.plist.in b/Mac/Resources/app/Info.plist.in --- a/Mac/Resources/app/Info.plist.in +++ b/Mac/Resources/app/Info.plist.in @@ -20,7 +20,7 @@ CFBundleExecutable Python CFBundleGetInfoString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleHelpBookFolder Documentation @@ -37,7 +37,7 @@ CFBundleInfoDictionaryVersion 6.0 CFBundleLongVersionString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleName Python CFBundlePackageType @@ -55,7 +55,7 @@ NSAppleScriptEnabled NSHumanReadableCopyright - (c) 2001-2015 Python Software Foundation. + (c) 2001-2016 Python Software Foundation. NSHighResolutionCapable diff --git a/Mac/Resources/framework/Info.plist.in b/Mac/Resources/framework/Info.plist.in --- a/Mac/Resources/framework/Info.plist.in +++ b/Mac/Resources/framework/Info.plist.in @@ -17,9 +17,9 @@ CFBundlePackageType FMWK CFBundleShortVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleLongVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleSignature ???? CFBundleVersion -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 17:47:00 2016 From: python-checkins at python.org (ned.deily) Date: Fri, 01 Jan 2016 22:47:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Update_copyrights_for_2016=2E?= Message-ID: <20160101224700.124872.71433@psf.io> https://hg.python.org/cpython/rev/c91417b4cd01 changeset: 99752:c91417b4cd01 parent: 99748:22f1ddac9cb7 parent: 99751:d2d288b80be3 user: Ned Deily date: Fri Jan 01 17:45:06 2016 -0500 summary: Update copyrights for 2016. files: Mac/BuildScript/resources/License.rtf | 2 +- Mac/IDLE/IDLE.app/Contents/Info.plist | 2 +- Mac/PythonLauncher/Info.plist.in | 2 +- Mac/Resources/app/Info.plist.in | 6 +++--- Mac/Resources/framework/Info.plist.in | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -54,7 +54,7 @@ \b0 \ 1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation.\ \ -2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ +2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ \ 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python.\ \ diff --git a/Mac/IDLE/IDLE.app/Contents/Info.plist b/Mac/IDLE/IDLE.app/Contents/Info.plist --- a/Mac/IDLE/IDLE.app/Contents/Info.plist +++ b/Mac/IDLE/IDLE.app/Contents/Info.plist @@ -36,7 +36,7 @@ CFBundleExecutable IDLE CFBundleGetInfoString - %version%, ? 2001-2015 Python Software Foundation + %version%, ? 2001-2016 Python Software Foundation CFBundleIconFile IDLE.icns CFBundleIdentifier diff --git a/Mac/PythonLauncher/Info.plist.in b/Mac/PythonLauncher/Info.plist.in --- a/Mac/PythonLauncher/Info.plist.in +++ b/Mac/PythonLauncher/Info.plist.in @@ -40,7 +40,7 @@ CFBundleExecutable Python Launcher CFBundleGetInfoString - %VERSION%, ? 2001-2015 Python Software Foundation + %VERSION%, ? 2001-2016 Python Software Foundation CFBundleIconFile PythonLauncher.icns CFBundleIdentifier diff --git a/Mac/Resources/app/Info.plist.in b/Mac/Resources/app/Info.plist.in --- a/Mac/Resources/app/Info.plist.in +++ b/Mac/Resources/app/Info.plist.in @@ -20,7 +20,7 @@ CFBundleExecutable Python CFBundleGetInfoString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleHelpBookFolder Documentation @@ -37,7 +37,7 @@ CFBundleInfoDictionaryVersion 6.0 CFBundleLongVersionString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleName Python CFBundlePackageType @@ -55,7 +55,7 @@ NSAppleScriptEnabled NSHumanReadableCopyright - (c) 2001-2015 Python Software Foundation. + (c) 2001-2016 Python Software Foundation. NSHighResolutionCapable diff --git a/Mac/Resources/framework/Info.plist.in b/Mac/Resources/framework/Info.plist.in --- a/Mac/Resources/framework/Info.plist.in +++ b/Mac/Resources/framework/Info.plist.in @@ -17,9 +17,9 @@ CFBundlePackageType FMWK CFBundleShortVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleLongVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleSignature ???? CFBundleVersion -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 17:46:59 2016 From: python-checkins at python.org (ned.deily) Date: Fri, 01 Jan 2016 22:46:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Update_copyrig?= =?utf-8?q?hts_for_2016=2E?= Message-ID: <20160101224659.21388.85400@psf.io> https://hg.python.org/cpython/rev/02678f5ee8d2 changeset: 99750:02678f5ee8d2 branch: 3.4 parent: 99742:71c7e62bef0b user: Ned Deily date: Fri Jan 01 17:42:39 2016 -0500 summary: Update copyrights for 2016. files: Mac/BuildScript/resources/License.rtf | 2 +- Mac/IDLE/IDLE.app/Contents/Info.plist | 2 +- Mac/PythonLauncher/Info.plist.in | 2 +- Mac/Resources/app/Info.plist.in | 6 +++--- Mac/Resources/framework/Info.plist.in | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -54,7 +54,7 @@ \b0 \ 1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation.\ \ -2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ +2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ \ 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python.\ \ diff --git a/Mac/IDLE/IDLE.app/Contents/Info.plist b/Mac/IDLE/IDLE.app/Contents/Info.plist --- a/Mac/IDLE/IDLE.app/Contents/Info.plist +++ b/Mac/IDLE/IDLE.app/Contents/Info.plist @@ -36,7 +36,7 @@ CFBundleExecutable IDLE CFBundleGetInfoString - %version%, ? 2001-2015 Python Software Foundation + %version%, ? 2001-2016 Python Software Foundation CFBundleIconFile IDLE.icns CFBundleIdentifier diff --git a/Mac/PythonLauncher/Info.plist.in b/Mac/PythonLauncher/Info.plist.in --- a/Mac/PythonLauncher/Info.plist.in +++ b/Mac/PythonLauncher/Info.plist.in @@ -40,7 +40,7 @@ CFBundleExecutable PythonLauncher CFBundleGetInfoString - %VERSION%, ? 2001-2015 Python Software Foundation + %VERSION%, ? 2001-2016 Python Software Foundation CFBundleIconFile PythonLauncher.icns CFBundleIdentifier diff --git a/Mac/Resources/app/Info.plist.in b/Mac/Resources/app/Info.plist.in --- a/Mac/Resources/app/Info.plist.in +++ b/Mac/Resources/app/Info.plist.in @@ -20,7 +20,7 @@ CFBundleExecutable Python CFBundleGetInfoString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleHelpBookFolder Documentation @@ -37,7 +37,7 @@ CFBundleInfoDictionaryVersion 6.0 CFBundleLongVersionString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleName Python CFBundlePackageType @@ -55,7 +55,7 @@ NSAppleScriptEnabled NSHumanReadableCopyright - (c) 2001-2015 Python Software Foundation. + (c) 2001-2016 Python Software Foundation. NSHighResolutionCapable diff --git a/Mac/Resources/framework/Info.plist.in b/Mac/Resources/framework/Info.plist.in --- a/Mac/Resources/framework/Info.plist.in +++ b/Mac/Resources/framework/Info.plist.in @@ -17,9 +17,9 @@ CFBundlePackageType FMWK CFBundleShortVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleLongVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleSignature ???? CFBundleVersion -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 17:46:59 2016 From: python-checkins at python.org (ned.deily) Date: Fri, 01 Jan 2016 22:46:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Update_copyrights_for_2016=2E?= Message-ID: <20160101224659.21392.86797@psf.io> https://hg.python.org/cpython/rev/d2d288b80be3 changeset: 99751:d2d288b80be3 branch: 3.5 parent: 99747:2727211587f2 parent: 99750:02678f5ee8d2 user: Ned Deily date: Fri Jan 01 17:44:30 2016 -0500 summary: Update copyrights for 2016. files: Mac/BuildScript/resources/License.rtf | 2 +- Mac/IDLE/IDLE.app/Contents/Info.plist | 2 +- Mac/PythonLauncher/Info.plist.in | 2 +- Mac/Resources/app/Info.plist.in | 6 +++--- Mac/Resources/framework/Info.plist.in | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -54,7 +54,7 @@ \b0 \ 1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation.\ \ -2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ +2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee.\ \ 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python.\ \ diff --git a/Mac/IDLE/IDLE.app/Contents/Info.plist b/Mac/IDLE/IDLE.app/Contents/Info.plist --- a/Mac/IDLE/IDLE.app/Contents/Info.plist +++ b/Mac/IDLE/IDLE.app/Contents/Info.plist @@ -36,7 +36,7 @@ CFBundleExecutable IDLE CFBundleGetInfoString - %version%, ? 2001-2015 Python Software Foundation + %version%, ? 2001-2016 Python Software Foundation CFBundleIconFile IDLE.icns CFBundleIdentifier diff --git a/Mac/PythonLauncher/Info.plist.in b/Mac/PythonLauncher/Info.plist.in --- a/Mac/PythonLauncher/Info.plist.in +++ b/Mac/PythonLauncher/Info.plist.in @@ -40,7 +40,7 @@ CFBundleExecutable Python Launcher CFBundleGetInfoString - %VERSION%, ? 2001-2015 Python Software Foundation + %VERSION%, ? 2001-2016 Python Software Foundation CFBundleIconFile PythonLauncher.icns CFBundleIdentifier diff --git a/Mac/Resources/app/Info.plist.in b/Mac/Resources/app/Info.plist.in --- a/Mac/Resources/app/Info.plist.in +++ b/Mac/Resources/app/Info.plist.in @@ -20,7 +20,7 @@ CFBundleExecutable Python CFBundleGetInfoString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleHelpBookFolder Documentation @@ -37,7 +37,7 @@ CFBundleInfoDictionaryVersion 6.0 CFBundleLongVersionString - %version%, (c) 2001-2015 Python Software Foundation. + %version%, (c) 2001-2016 Python Software Foundation. CFBundleName Python CFBundlePackageType @@ -55,7 +55,7 @@ NSAppleScriptEnabled NSHumanReadableCopyright - (c) 2001-2015 Python Software Foundation. + (c) 2001-2016 Python Software Foundation. NSHighResolutionCapable diff --git a/Mac/Resources/framework/Info.plist.in b/Mac/Resources/framework/Info.plist.in --- a/Mac/Resources/framework/Info.plist.in +++ b/Mac/Resources/framework/Info.plist.in @@ -17,9 +17,9 @@ CFBundlePackageType FMWK CFBundleShortVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleLongVersionString - %VERSION%, (c) 2001-2015 Python Software Foundation. + %VERSION%, (c) 2001-2016 Python Software Foundation. CFBundleSignature ???? CFBundleVersion -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 20:46:20 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 02 Jan 2016 01:46:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Remove_a_dupli?= =?utf-8?q?cate_test=5Faddresses_key=3Avalue_as_identified_by_Vincent?= Message-ID: <20160102014620.105552.69483@psf.io> https://hg.python.org/cpython/rev/eb279f45f829 changeset: 99753:eb279f45f829 branch: 3.5 parent: 99751:d2d288b80be3 user: Gregory P. Smith date: Fri Jan 01 17:41:46 2016 -0800 summary: Remove a duplicate test_addresses key:value as identified by Vincent Davis reviewing code. files: Lib/test/test_ipaddress.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -1740,7 +1740,6 @@ '2001:0:0:4:0:0:0:8': '2001:0:0:4::8/128', '2001:0:0:4:5:6:7:8': '2001::4:5:6:7:8/128', '2001:0:3:4:5:6:7:8': '2001:0:3:4:5:6:7:8/128', - '2001:0:3:4:5:6:7:8': '2001:0:3:4:5:6:7:8/128', '0:0:3:0:0:0:0:ffff': '0:0:3::ffff/128', '0:0:0:4:0:0:0:ffff': '::4:0:0:0:ffff/128', '0:0:0:0:5:0:0:ffff': '::5:0:0:ffff/128', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 1 20:46:20 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 02 Jan 2016 01:46:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Remove_a_duplicate_test=5Faddresses_key=3Avalue_as_ident?= =?utf-8?q?ified_by_Vincent?= Message-ID: <20160102014620.21219.81291@psf.io> https://hg.python.org/cpython/rev/6446bdf9cdfe changeset: 99754:6446bdf9cdfe parent: 99752:c91417b4cd01 parent: 99753:eb279f45f829 user: Gregory P. Smith date: Fri Jan 01 17:42:30 2016 -0800 summary: Remove a duplicate test_addresses key:value as identified by Vincent Davis reviewing code. files: Lib/test/test_ipaddress.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -1740,7 +1740,6 @@ '2001:0:0:4:0:0:0:8': '2001:0:0:4::8/128', '2001:0:0:4:5:6:7:8': '2001::4:5:6:7:8/128', '2001:0:3:4:5:6:7:8': '2001:0:3:4:5:6:7:8/128', - '2001:0:3:4:5:6:7:8': '2001:0:3:4:5:6:7:8/128', '0:0:3:0:0:0:0:ffff': '0:0:3::ffff/128', '0:0:0:4:0:0:0:ffff': '::4:0:0:0:ffff/128', '0:0:0:0:5:0:0:ffff': '::5:0:0:ffff/128', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 2 02:27:45 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 02 Jan 2016 07:27:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160102072745.124848.35767@psf.io> https://hg.python.org/cpython/rev/3a6b1186745f changeset: 99757:3a6b1186745f parent: 99754:6446bdf9cdfe parent: 99756:6b9d8957aeef user: Senthil Kumaran date: Fri Jan 01 23:27:38 2016 -0800 summary: merge from 3.5 Issue25917 : Fix howto links in docs. Point the reference documentation instead of wiki. files: Doc/faq/programming.rst | 3 +-- Doc/library/functions.rst | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1010,8 +1010,7 @@ may come up with. This is doubly true for primitives written in C, such as builtins and some extension types. For example, be sure to use either the :meth:`list.sort` built-in method or the related :func:`sorted` - function to do sorting (and see the - `sorting mini-HOWTO `_ for examples + function to do sorting (and see the :ref:`sortinghowto` for examples of moderately advanced usage). * Abstractions tend to create indirections and force the interpreter to work diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1306,8 +1306,7 @@ compare equal --- this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade). - For sorting examples and a brief sorting tutorial, see `Sorting HowTo - `_\. + For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. .. function:: staticmethod(function) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 2 02:27:45 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 02 Jan 2016 07:27:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_from_3=2E4?= Message-ID: <20160102072745.124872.42957@psf.io> https://hg.python.org/cpython/rev/6b9d8957aeef changeset: 99756:6b9d8957aeef branch: 3.5 parent: 99753:eb279f45f829 parent: 99755:521a402ae177 user: Senthil Kumaran date: Fri Jan 01 23:26:53 2016 -0800 summary: merge from 3.4 Issue25917 : Fix howto links in docs. Point the reference documentation instead of wiki. files: Doc/faq/programming.rst | 3 +-- Doc/library/functions.rst | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1010,8 +1010,7 @@ may come up with. This is doubly true for primitives written in C, such as builtins and some extension types. For example, be sure to use either the :meth:`list.sort` built-in method or the related :func:`sorted` - function to do sorting (and see the - `sorting mini-HOWTO `_ for examples + function to do sorting (and see the :ref:`sortinghowto` for examples of moderately advanced usage). * Abstractions tend to create indirections and force the interpreter to work diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1306,8 +1306,7 @@ compare equal --- this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade). - For sorting examples and a brief sorting tutorial, see `Sorting HowTo - `_\. + For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. .. function:: staticmethod(function) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 2 02:27:45 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 02 Jan 2016 07:27:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUyNTkxNyA6?= =?utf-8?q?_Fix_howto_links_in_docs=2E_Point_the_reference_documentation_i?= =?utf-8?q?nstead?= Message-ID: <20160102072745.77249.94580@psf.io> https://hg.python.org/cpython/rev/521a402ae177 changeset: 99755:521a402ae177 branch: 3.4 parent: 99750:02678f5ee8d2 user: Senthil Kumaran date: Fri Jan 01 23:25:58 2016 -0800 summary: Issue25917 : Fix howto links in docs. Point the reference documentation instead of wiki. files: Doc/faq/programming.rst | 3 +-- Doc/library/functions.rst | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1010,8 +1010,7 @@ may come up with. This is doubly true for primitives written in C, such as builtins and some extension types. For example, be sure to use either the :meth:`list.sort` built-in method or the related :func:`sorted` - function to do sorting (and see the - `sorting mini-HOWTO `_ for examples + function to do sorting (and see the :ref:`sortinghowto` for examples of moderately advanced usage). * Abstractions tend to create indirections and force the interpreter to work diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1296,8 +1296,7 @@ compare equal --- this is helpful for sorting in multiple passes (for example, sort by department, then by salary grade). - For sorting examples and a brief sorting tutorial, see `Sorting HowTo - `_\. + For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. .. function:: staticmethod(function) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sat Jan 2 03:42:54 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 02 Jan 2016 08:42:54 +0000 Subject: [Python-checkins] Daily reference leaks (6446bdf9cdfe): sum=7 Message-ID: <20160102084254.105566.91218@psf.io> results for 6446bdf9cdfe on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog2KyA0P', '--timeout', '7200'] From python-checkins at python.org Sat Jan 2 10:11:27 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 02 Jan 2016 15:11:27 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_document_Andrew_MacIntyre?= =?utf-8?q?=27s_drop_request?= Message-ID: <20160102151127.126301.49468@psf.io> https://hg.python.org/devguide/rev/78ad3da30e8a changeset: 777:78ad3da30e8a user: Benjamin Peterson date: Sat Jan 02 09:11:24 2016 -0600 summary: document Andrew MacIntyre's drop request files: developers.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/developers.rst b/developers.rst --- a/developers.rst +++ b/developers.rst @@ -410,6 +410,9 @@ Permissions Dropped on Request ------------------------------ +- Andrew MacIntyre's privileges were dropped on 2 January 2016 by BCP per his + request. + - Skip Montanaro's permissions were removed on 21 April 2015 by BCP per `his request `_. -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Sat Jan 2 15:45:07 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 02 Jan 2016 20:45:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzIyNzA5OiBVc2Ug?= =?utf-8?q?stdin_as-is_if_it_does_not_have_a_buffer_attribute=2E?= Message-ID: <20160102204506.70425.67781@psf.io> https://hg.python.org/cpython/rev/ded1336bff49 changeset: 99758:ded1336bff49 branch: 3.5 parent: 99756:6b9d8957aeef user: R David Murray date: Sat Jan 02 15:41:41 2016 -0500 summary: #22709: Use stdin as-is if it does not have a buffer attribute. This restores backward compatibility lost in the fix for #21075, and is better duck typing. Patch by Akira Li. files: Lib/fileinput.py | 2 +- Lib/test/test_fileinput.py | 11 +++++++++++ Misc/NEWS | 3 +++ 3 files changed, 15 insertions(+), 1 deletions(-) diff --git a/Lib/fileinput.py b/Lib/fileinput.py --- a/Lib/fileinput.py +++ b/Lib/fileinput.py @@ -328,7 +328,7 @@ if self._filename == '-': self._filename = '' if 'b' in self._mode: - self._file = sys.stdin.buffer + self._file = getattr(sys.stdin, 'buffer', sys.stdin) else: self._file = sys.stdin self._isstdin = True diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -240,6 +240,17 @@ lines = list(fi) self.assertEqual(lines, [b'spam, bacon, sausage, and spam']) + def test_detached_stdin_binary_mode(self): + orig_stdin = sys.stdin + try: + sys.stdin = BytesIO(b'spam, bacon, sausage, and spam') + self.assertFalse(hasattr(sys.stdin, 'buffer')) + fi = FileInput(files=['-'], mode='rb') + lines = list(fi) + self.assertEqual(lines, [b'spam, bacon, sausage, and spam']) + finally: + sys.stdin = orig_stdin + def test_file_opening_hook(self): try: # cannot use openhook and inplace mode diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,9 @@ Library ------- +- Issue #25447: fileinput now uses sys.stdin as-is if it does not have a + buffer attribute (restores backward compatibility). + - Issue #25447: Copying the lru_cache() wrapper object now always works, independedly from the type of the wrapped object (by returning the original object unchanged). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 2 15:45:06 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 02 Jan 2016 20:45:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2322709=3A_Use_stdin_as-is_if_it_does_not_have?= =?utf-8?q?_a_buffer_attribute=2E?= Message-ID: <20160102204506.21382.33532@psf.io> https://hg.python.org/cpython/rev/688d32cdbc0c changeset: 99759:688d32cdbc0c parent: 99757:3a6b1186745f parent: 99758:ded1336bff49 user: R David Murray date: Sat Jan 02 15:43:44 2016 -0500 summary: Merge: #22709: Use stdin as-is if it does not have a buffer attribute. files: Lib/fileinput.py | 2 +- Lib/test/test_fileinput.py | 11 +++++++++++ Misc/NEWS | 3 +++ 3 files changed, 15 insertions(+), 1 deletions(-) diff --git a/Lib/fileinput.py b/Lib/fileinput.py --- a/Lib/fileinput.py +++ b/Lib/fileinput.py @@ -328,7 +328,7 @@ if self._filename == '-': self._filename = '' if 'b' in self._mode: - self._file = sys.stdin.buffer + self._file = getattr(sys.stdin, 'buffer', sys.stdin) else: self._file = sys.stdin self._isstdin = True diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -240,6 +240,17 @@ lines = list(fi) self.assertEqual(lines, [b'spam, bacon, sausage, and spam']) + def test_detached_stdin_binary_mode(self): + orig_stdin = sys.stdin + try: + sys.stdin = BytesIO(b'spam, bacon, sausage, and spam') + self.assertFalse(hasattr(sys.stdin, 'buffer')) + fi = FileInput(files=['-'], mode='rb') + lines = list(fi) + self.assertEqual(lines, [b'spam, bacon, sausage, and spam']) + finally: + sys.stdin = orig_stdin + def test_file_opening_hook(self): try: # cannot use openhook and inplace mode diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #25447: fileinput now uses sys.stdin as-is if it does not have a + buffer attribute (restores backward compatibility). + - Issue #25971: Optimized creating Fractions from floats by 2 times and from Decimals by 3 times. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 2 17:20:01 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 02 Jan 2016 22:20:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2321815=3A_violate_IMAP_R?= =?utf-8?q?FC_to_be_compatible_with=2C_e=2Eg=2E=2C_gmail?= Message-ID: <20160102222001.126309.93405@psf.io> https://hg.python.org/cpython/rev/54b36229021a changeset: 99760:54b36229021a user: R David Murray date: Sat Jan 02 17:18:34 2016 -0500 summary: #21815: violate IMAP RFC to be compatible with, e.g., gmail and others, including imaplib's own behavior. I'm applying this only to 3.6 because there's a potential backward compatibility concern: if there are servers that include ] characters in the 'text' portion of their imap responses, this code change could introduce a new bug. Patch by Lita Cho, reviewed by Jessica McKellar, Berker Peksag, Maciej Szulik, silentghost, and me (I fleshed out the comments with the additional info/concerns.) files: Doc/library/imaplib.rst | 11 ++++++ Lib/imaplib.py | 10 +++++- Lib/test/test_imaplib.py | 49 ++++++++++++++++++++++++++++ Misc/NEWS | 4 ++ 4 files changed, 73 insertions(+), 1 deletions(-) diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -500,6 +500,17 @@ M.store(num, '+FLAGS', '\\Deleted') M.expunge() + .. note:: + + Creating flags containing ']' (for example: "[test]") violates + :rfc:`3501` (the IMAP protocol). However, imaplib has historically + allowed creation of such tags, and popular IMAP servers, such as Gmail, + accept and produce such flags. There are non-Python programs which also + create such tags. Although it is an RFC violation and IMAP clients and + servers are supposed to be strict, imaplib nontheless continues to allow + such tags to be created for backward compatibility reasons, and as of + python 3.5.2/3.6.0, handles them if they are sent from the server, since + this improves real-world compatibility. .. method:: IMAP4.subscribe(mailbox) diff --git a/Lib/imaplib.py b/Lib/imaplib.py --- a/Lib/imaplib.py +++ b/Lib/imaplib.py @@ -111,7 +111,15 @@ # Literal is no longer used; kept for backward compatibility. Literal = re.compile(br'.*{(?P\d+)}$', re.ASCII) MapCRLF = re.compile(br'\r\n|\r|\n') -Response_code = re.compile(br'\[(?P[A-Z-]+)( (?P[^\]]*))?\]') +# We no longer exclude the ']' character from the data portion of the response +# code, even though it violates the RFC. Popular IMAP servers such as Gmail +# allow flags with ']', and there are programs (including imaplib!) that can +# produce them. The problem with this is if the 'text' portion of the response +# includes a ']' we'll parse the response wrong (which is the point of the RFC +# restriction). However, that seems less likely to be a problem in practice +# than being unable to correctly parse flags that include ']' chars, which +# was reported as a real-world problem in issue #21815. +Response_code = re.compile(br'\[(?P[A-Z-]+)( (?P.*))?\]') Untagged_response = re.compile(br'\* (?P[A-Z-]+)( (?P.*))?') # Untagged_status is no longer used; kept for backward compatibility Untagged_status = re.compile( diff --git a/Lib/test/test_imaplib.py b/Lib/test/test_imaplib.py --- a/Lib/test/test_imaplib.py +++ b/Lib/test/test_imaplib.py @@ -243,6 +243,55 @@ client.shutdown() @reap_threads + def test_bracket_flags(self): + + # This violates RFC 3501, which disallows ']' characters in tag names, + # but imaplib has allowed producing such tags forever, other programs + # also produce them (eg: OtherInbox's Organizer app as of 20140716), + # and Gmail, for example, accepts them and produces them. So we + # support them. See issue #21815. + + class BracketFlagHandler(SimpleIMAPHandler): + + def handle(self): + self.flags = ['Answered', 'Flagged', 'Deleted', 'Seen', 'Draft'] + super().handle() + + def cmd_AUTHENTICATE(self, tag, args): + self._send_textline('+') + self.server.response = yield + self._send_tagged(tag, 'OK', 'FAKEAUTH successful') + + def cmd_SELECT(self, tag, args): + flag_msg = ' \\'.join(self.flags) + self._send_line(('* FLAGS (%s)' % flag_msg).encode('ascii')) + self._send_line(b'* 2 EXISTS') + self._send_line(b'* 0 RECENT') + msg = ('* OK [PERMANENTFLAGS %s \\*)] Flags permitted.' + % flag_msg) + self._send_line(msg.encode('ascii')) + self._send_tagged(tag, 'OK', '[READ-WRITE] SELECT completed.') + + def cmd_STORE(self, tag, args): + new_flags = args[2].strip('(').strip(')').split() + self.flags.extend(new_flags) + flags_msg = '(FLAGS (%s))' % ' \\'.join(self.flags) + msg = '* %s FETCH %s' % (args[0], flags_msg) + self._send_line(msg.encode('ascii')) + self._send_tagged(tag, 'OK', 'STORE completed.') + + with self.reaped_pair(BracketFlagHandler) as (server, client): + code, data = client.authenticate('MYAUTH', lambda x: b'fake') + self.assertEqual(code, 'OK') + self.assertEqual(server.response, b'ZmFrZQ==\r\n') + client.select('test') + typ, [data] = client.store(b'1', "+FLAGS", "[test]") + self.assertIn(b'[test]', data) + client.select('test') + typ, [data] = client.response('PERMANENTFLAGS') + self.assertIn(b'[test]', data) + + @reap_threads def test_issue5949(self): class EOFHandler(socketserver.StreamRequestHandler): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,10 @@ Library ------- +- Issue #21815: Accept ] characters in the data portion of imap responses, + in order to handle the flags with square brackets accepted and produced + by servers such as gmail. + - Issue #25447: fileinput now uses sys.stdin as-is if it does not have a buffer attribute (restores backward compatibility). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 2 17:26:13 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 02 Jan 2016 22:26:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2321815=3A_Make_the_doc_c?= =?utf-8?q?hange_match_what_I_actually_did=2E?= Message-ID: <20160102222613.8275.25572@psf.io> https://hg.python.org/cpython/rev/53271aa4d84c changeset: 99761:53271aa4d84c user: R David Murray date: Sat Jan 02 17:25:59 2016 -0500 summary: #21815: Make the doc change match what I actually did. files: Doc/library/imaplib.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -509,8 +509,8 @@ create such tags. Although it is an RFC violation and IMAP clients and servers are supposed to be strict, imaplib nontheless continues to allow such tags to be created for backward compatibility reasons, and as of - python 3.5.2/3.6.0, handles them if they are sent from the server, since - this improves real-world compatibility. + python 3.6, handles them if they are sent from the server, since this + improves real-world compatibility. .. method:: IMAP4.subscribe(mailbox) -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sun Jan 3 03:42:09 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 03 Jan 2016 08:42:09 +0000 Subject: [Python-checkins] Daily reference leaks (53271aa4d84c): sum=4 Message-ID: <20160103084209.70425.16185@psf.io> results for 53271aa4d84c on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogX95aiZ', '--timeout', '7200'] From python-checkins at python.org Sun Jan 3 04:04:12 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 03 Jan 2016 09:04:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Issue21221_-_E?= =?utf-8?q?xplain_the_usage_of_tm=5Fisdst_attribute_of_mktime=2C_with_vali?= =?utf-8?q?d?= Message-ID: <20160103090411.124864.98948@psf.io> https://hg.python.org/cpython/rev/fae779a811ec changeset: 99762:fae779a811ec branch: 3.4 parent: 99755:521a402ae177 user: Senthil Kumaran date: Sun Jan 03 00:40:03 2016 -0800 summary: Issue21221 - Explain the usage of tm_isdst attribute of mktime, with valid values and meaning. Patch contributed by Andrew Scheller. files: Doc/library/time.rst | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/time.rst b/Doc/library/time.rst --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -550,9 +550,11 @@ +-------+-------------------+---------------------------------+ Note that unlike the C structure, the month value is a range of [1, 12], not - [0, 11]. A ``-1`` argument as the daylight - savings flag, passed to :func:`mktime` will usually result in the correct - daylight savings state to be filled in. + [0, 11]. + + In calls to :func:`mktime`, :attr:`tm_isdst` may be set to 1 when daylight + savings time is in effect, and 0 when it is not. A value of -1 indicates that + this is not known, and will usually result in the correct state being filled in. When a tuple with an incorrect length is passed to a function expecting a :class:`struct_time`, or having elements of the wrong type, a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 04:04:12 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 03 Jan 2016 09:04:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160103090412.70429.90354@psf.io> https://hg.python.org/cpython/rev/b729e7ae2747 changeset: 99764:b729e7ae2747 parent: 99761:53271aa4d84c parent: 99763:3b7a7ea1214c user: Senthil Kumaran date: Sun Jan 03 00:45:05 2016 -0800 summary: merge from 3.5 Issue21221 - Explain the usage of tm_isdst attribute of mktime, with valid values and meaning. Patch contributed by Andrew Scheller. files: Doc/library/time.rst | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/time.rst b/Doc/library/time.rst --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -555,9 +555,11 @@ +-------+-------------------+---------------------------------+ Note that unlike the C structure, the month value is a range of [1, 12], not - [0, 11]. A ``-1`` argument as the daylight - savings flag, passed to :func:`mktime` will usually result in the correct - daylight savings state to be filled in. + [0, 11]. + + In calls to :func:`mktime`, :attr:`tm_isdst` may be set to 1 when daylight + savings time is in effect, and 0 when it is not. A value of -1 indicates that + this is not known, and will usually result in the correct state being filled in. When a tuple with an incorrect length is passed to a function expecting a :class:`struct_time`, or having elements of the wrong type, a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 04:04:12 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 03 Jan 2016 09:04:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_from_3=2E4?= Message-ID: <20160103090411.21250.27436@psf.io> https://hg.python.org/cpython/rev/3b7a7ea1214c changeset: 99763:3b7a7ea1214c branch: 3.5 parent: 99758:ded1336bff49 parent: 99762:fae779a811ec user: Senthil Kumaran date: Sun Jan 03 00:43:23 2016 -0800 summary: merge from 3.4 Issue21221 - Explain the usage of tm_isdst attribute of mktime, with valid values and meaning. Patch contributed by Andrew Scheller. files: Doc/library/time.rst | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/time.rst b/Doc/library/time.rst --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -555,9 +555,11 @@ +-------+-------------------+---------------------------------+ Note that unlike the C structure, the month value is a range of [1, 12], not - [0, 11]. A ``-1`` argument as the daylight - savings flag, passed to :func:`mktime` will usually result in the correct - daylight savings state to be filled in. + [0, 11]. + + In calls to :func:`mktime`, :attr:`tm_isdst` may be set to 1 when daylight + savings time is in effect, and 0 when it is not. A value of -1 indicates that + this is not known, and will usually result in the correct state being filled in. When a tuple with an incorrect length is passed to a function expecting a :class:`struct_time`, or having elements of the wrong type, a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 04:04:22 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 03 Jan 2016 09:04:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backport_doc_i?= =?utf-8?q?mprovements_for_Issue21221_-_Explain_the_usage_of_tm=5Fisdst?= Message-ID: <20160103090422.8265.78650@psf.io> https://hg.python.org/cpython/rev/f7324e5935f8 changeset: 99765:f7324e5935f8 branch: 2.7 parent: 99749:d705db81864f user: Senthil Kumaran date: Sun Jan 03 01:03:40 2016 -0800 summary: Backport doc improvements for Issue21221 - Explain the usage of tm_isdst attribute of mktime, with valid values and meaning. Patch contributed by Andrew Scheller. files: Doc/library/time.rst | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Doc/library/time.rst b/Doc/library/time.rst --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -427,9 +427,12 @@ Note that unlike the C structure, the month value is a range of [1, 12], not [0, 11]. A year value will be handled as described under :ref:`Year 2000 - (Y2K) issues ` above. A ``-1`` argument as the daylight - savings flag, passed to :func:`mktime` will usually result in the correct - daylight savings state to be filled in. + (Y2K) issues ` above. + + In calls to :func:`mktime`, :attr:`tm_isdst` may be set to 1 when daylight + savings time is in effect, and 0 when it is not. A value of -1 indicates + that this is not known, and will usually result in the correct state being + filled in. When a tuple with an incorrect length is passed to a function expecting a :class:`struct_time`, or having elements of the wrong type, a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 09:38:50 2016 From: python-checkins at python.org (barry.warsaw) Date: Sun, 03 Jan 2016 14:38:50 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Status_and_resolution_for_507?= =?utf-8?q?=2E?= Message-ID: <20160103143850.8277.10250@psf.io> https://hg.python.org/peps/rev/847f0333c236 changeset: 6145:847f0333c236 user: Barry Warsaw date: Sun Jan 03 09:38:47 2016 -0500 summary: Status and resolution for 507. files: pep-0507.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0507.txt b/pep-0507.txt --- a/pep-0507.txt +++ b/pep-0507.txt @@ -3,11 +3,12 @@ Version: $Revision$ Last-Modified: $Date$ Author: Barry Warsaw -Status: Draft +Status: Rejected Type: Process Content-Type: text/x-rst Created: 2015-09-30 Post-History: +Resolution: https://mail.python.org/pipermail/core-workflow/2016-January/000345.html Abstract -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 3 21:00:39 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 04 Jan 2016 02:00:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Issue24898_-_I?= =?utf-8?q?mprove_str=2Efind_documentation=2E?= Message-ID: <20160104020039.21229.18967@psf.io> https://hg.python.org/cpython/rev/7150e8a38c63 changeset: 99766:7150e8a38c63 branch: 3.4 parent: 99762:fae779a811ec user: Senthil Kumaran date: Sun Jan 03 17:57:10 2016 -0800 summary: Issue24898 - Improve str.find documentation. Simplify str.find explaination as per Georg Brandl's suggestion. files: Doc/library/stdtypes.rst | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1563,10 +1563,9 @@ .. method:: str.find(sub[, start[, end]]) - Return the lowest index in the string where substring *sub* is found, such - that *sub* is contained in the slice ``s[start:end]``. Optional arguments - *start* and *end* are interpreted as in slice notation. Return ``-1`` if - *sub* is not found. + Return the lowest index in the string where substring *sub* is found within + the slice ``s[start:end]``. Optional arguments *start* and *end* are + interpreted as in slice notation. Return ``-1`` if *sub* is not found. .. note:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 21:00:39 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 04 Jan 2016 02:00:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160104020039.8273.1818@psf.io> https://hg.python.org/cpython/rev/136adbcefa5f changeset: 99768:136adbcefa5f parent: 99764:b729e7ae2747 parent: 99767:164b564e3c1a user: Senthil Kumaran date: Sun Jan 03 18:00:31 2016 -0800 summary: merge from 3.5 Issue24898 - Improve str.find documentation. Simplify str.find explaination as per Georg Brandl's suggestion. files: Doc/library/stdtypes.rst | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1563,10 +1563,9 @@ .. method:: str.find(sub[, start[, end]]) - Return the lowest index in the string where substring *sub* is found, such - that *sub* is contained in the slice ``s[start:end]``. Optional arguments - *start* and *end* are interpreted as in slice notation. Return ``-1`` if - *sub* is not found. + Return the lowest index in the string where substring *sub* is found within + the slice ``s[start:end]``. Optional arguments *start* and *end* are + interpreted as in slice notation. Return ``-1`` if *sub* is not found. .. note:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 21:00:40 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 04 Jan 2016 02:00:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_from_3=2E4?= Message-ID: <20160104020039.70441.43079@psf.io> https://hg.python.org/cpython/rev/164b564e3c1a changeset: 99767:164b564e3c1a branch: 3.5 parent: 99763:3b7a7ea1214c parent: 99766:7150e8a38c63 user: Senthil Kumaran date: Sun Jan 03 17:58:24 2016 -0800 summary: merge from 3.4 Issue24898 - Improve str.find documentation. Simplify str.find explaination as per Georg Brandl's suggestion. files: Doc/library/stdtypes.rst | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1563,10 +1563,9 @@ .. method:: str.find(sub[, start[, end]]) - Return the lowest index in the string where substring *sub* is found, such - that *sub* is contained in the slice ``s[start:end]``. Optional arguments - *start* and *end* are interpreted as in slice notation. Return ``-1`` if - *sub* is not found. + Return the lowest index in the string where substring *sub* is found within + the slice ``s[start:end]``. Optional arguments *start* and *end* are + interpreted as in slice notation. Return ``-1`` if *sub* is not found. .. note:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 3 21:07:14 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 04 Jan 2016 02:07:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backport_docum?= =?utf-8?q?entation_improvement=2E?= Message-ID: <20160104020714.21398.50779@psf.io> https://hg.python.org/cpython/rev/da934a19855b changeset: 99769:da934a19855b branch: 2.7 parent: 99765:f7324e5935f8 user: Senthil Kumaran date: Sun Jan 03 18:07:06 2016 -0800 summary: Backport documentation improvement. Issue24898 - Improve str.find documentation. Simplify str.find explaination as per Georg Brandl's suggestion. files: Doc/library/stdtypes.rst | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -963,10 +963,9 @@ .. method:: str.find(sub[, start[, end]]) - Return the lowest index in the string where substring *sub* is found, such - that *sub* is contained in the slice ``s[start:end]``. Optional arguments - *start* and *end* are interpreted as in slice notation. Return ``-1`` if - *sub* is not found. + Return the lowest index in the string where substring *sub* is found within + the slice ``s[start:end]``. Optional arguments *start* and *end* are + interpreted as in slice notation. Return ``-1`` if *sub* is not found. .. note:: -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Mon Jan 4 03:42:20 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 04 Jan 2016 08:42:20 +0000 Subject: [Python-checkins] Daily reference leaks (136adbcefa5f): sum=4 Message-ID: <20160104084219.77233.36770@psf.io> results for 136adbcefa5f on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogPM2_CC', '--timeout', '7200'] From python-checkins at python.org Mon Jan 4 12:00:10 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 04 Jan 2016 17:00:10 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_PEP_8_to_recommend_aga?= =?utf-8?q?inst_trailing_whitespace=2E?= Message-ID: <20160104170005.126319.86431@psf.io> https://hg.python.org/peps/rev/a1841e3549a0 changeset: 6146:a1841e3549a0 user: Guido van Rossum date: Mon Jan 04 08:59:47 2016 -0800 summary: Update PEP 8 to recommend against trailing whitespace. files: pep-0008.txt | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -473,6 +473,12 @@ Other Recommendations --------------------- +- Avoid trailing whitespace anywhere. Because it's usually invisible, + it can be confusing: e.g. a backslash followed by a space and a + newline does not count as a line continuation marker. Some editors + don't preserve it and many projects (like CPython itself) have + pre-commit hooks that reject it. + - Always surround these binary operators with a single space on either side: assignment (``=``), augmented assignment (``+=``, ``-=`` etc.), comparisons (``==``, ``<``, ``>``, ``!=``, ``<>``, ``<=``, -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 01:41:31 2016 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 05 Jan 2016 06:41:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue24733_-_R?= =?utf-8?q?emove_unreachable_code_in_traceback=2Ec?= Message-ID: <20160105064131.21382.17723@psf.io> https://hg.python.org/cpython/rev/f2a846c1b593 changeset: 99770:f2a846c1b593 branch: 2.7 user: Senthil Kumaran date: Mon Jan 04 22:40:39 2016 -0800 summary: Issue24733 - Remove unreachable code in traceback.c files: Python/traceback.c | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/Python/traceback.c b/Python/traceback.c --- a/Python/traceback.c +++ b/Python/traceback.c @@ -170,10 +170,6 @@ if (xfp == NULL) return err; - if (err != 0) { - fclose(xfp); - return err; - } for (i = 0; i < lineno; i++) { char* pLastChar = &linebuf[sizeof(linebuf)-2]; -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Tue Jan 5 03:49:52 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 05 Jan 2016 08:49:52 +0000 Subject: [Python-checkins] Daily reference leaks (136adbcefa5f): sum=4 Message-ID: <20160105084952.17325.65745@psf.io> results for 136adbcefa5f on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogp0qMbD', '--timeout', '7200'] From lp_benchmark_robot at intel.com Tue Jan 5 11:01:13 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 5 Jan 2016 16:01:13 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-05 Message-ID: <36170164-095a-4c87-908c-6849297cbe86@irsmsx104.ger.corp.intel.com> Results for project Python default, build date 2016-01-05 03:02:09 +0000 commit: 136adbcefa5f1c803a5584e74152eace9cb43852 previous commit: 5b43d7984a63a5b3eb6139187bc7da3a3dc1b011 revision date: 2016-01-04 02:00:31 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.31% -0.36% 10.85% 14.88% :-| pybench 0.08% 0.56% -1.47% 5.12% :-( regex_v8 2.67% 1.19% -4.92% 5.04% :-| nbody 0.39% 0.35% 1.04% 7.68% :-| json_dump_v2 0.16% -0.05% -0.99% 11.22% :-| normal_startup 0.74% 0.23% -1.64% 4.82% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Jan 5 11:03:31 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 5 Jan 2016 16:03:31 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-05 Message-ID: <744ac447-8f8b-4833-87b0-08e70e2effee@irsmsx104.ger.corp.intel.com> Results for project Python 2.7, build date 2016-01-05 03:04:13 +0000 commit: da934a19855bbdc46dad918874216b9a64ac6c08 previous commit: 11670e4be1a92d0a36ea6d8d2039a500f2f577ec revision date: 2016-01-04 02:07:06 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.38% 0.32% 4.22% 3.55% :-) pybench 0.15% 0.43% 6.01% 4.10% :-( regex_v8 1.11% -0.42% -2.54% 10.69% :-) nbody 0.11% 0.66% 7.89% 2.29% :-) json_dump_v2 0.27% -0.29% 5.96% 9.04% :-( normal_startup 2.08% -0.08% -5.35% 2.62% :-| ssbench 0.15% 0.06% 1.08% 1.03% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Jan 5 13:33:38 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 05 Jan 2016 18:33:38 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_rules_for_function_ann?= =?utf-8?q?otations_in_the_light_of_PEP_484=2E?= Message-ID: <20160105183338.3391.45382@psf.io> https://hg.python.org/peps/rev/3f6f9839bbe1 changeset: 6147:3f6f9839bbe1 user: Guido van Rossum date: Tue Jan 05 10:33:30 2016 -0800 summary: Update rules for function annotations in the light of PEP 484. files: pep-0008.txt | 130 +++++++++++++++++++++++--------------- 1 files changed, 77 insertions(+), 53 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -520,22 +520,33 @@ def complex(real, imag = 0.0): return magic(r = real, i = imag) -- Do use spaces around the ``=`` sign of an annotated function definition. - Additionally, use a single space after the ``:``, as well as a single space - on either side of the ``->`` sign representing an annotated return value. +- Function annotations should use the normal rules for colons and + always have spaces around the ``->`` arrow if present. (See + `Function Annotations`_ below for more about function annotations.) Yes:: - def munge(input: AnyStr): - def munge(sep: AnyStr = None): - def munge() -> AnyStr: - def munge(input: AnyStr, sep: AnyStr = None, limit=1000): + def munge(input: AnyStr): ... + def munge() -> AnyStr: ... No:: - def munge(input: AnyStr=None): - def munge(input:AnyStr): - def munge(input: AnyStr)->PosInt: + def munge(input:AnyStr): ... + def munge()->PosInt: ... + +- When combining an argument annotation with a default value, use + spaces around the ``=`` sign (but only for those arguments that have + both an annotation and a default). + + Yes:: + + def munge(sep: AnyStr = None): ... + def munge(input: AnyStr, sep: AnyStr = None, limit=1000): ... + + No:: + + def munge(input: AnyStr=None): ... + def munge(input: AnyStr, limit = 1000): ... - Compound statements (multiple statements on the same line) are generally discouraged. @@ -1233,60 +1244,65 @@ No: if greeting == True: Worse: if greeting is True: -- The Python standard library will not use function annotations as - that would result in a premature commitment to a particular - annotation style. Instead, the annotations are left for users to - discover and experiment with useful annotation styles. +Function Annotations +-------------------- - It is recommended that third party experiments with annotations use an - associated decorator to indicate how the annotation should be - interpreted. +With the acceptance of PEP 484, the style rules for function +annotations are changing. - Early core developer attempts to use function annotations revealed - inconsistent, ad-hoc annotation styles. For example: +- In order to be forward compatible, function annotations in Python 3 + code should preferably use PEP 484 syntax. (There are some + formatting recommendations for annotations in the previous section.) - * ``[str]`` was ambiguous as to whether it represented a list of - strings or a value that could be either *str* or *None*. +- The experimentation with annotation styles that was recommended + previously in this PEP is no longer encouraged. - * The notation ``open(file:(str,bytes))`` was used for a value that - could be either *bytes* or *str* rather than a 2-tuple containing - a *str* value followed by a *bytes* value. +- However, experiments within the rules of PEP 484 are now encouraged. + For example, marking up a large library with PEP 484 style type + annotations, reviewing how easy it was to add those annotations, and + observing whether their presence increases code understandabilty. - * The annotation ``seek(whence:int)`` exhibited a mix of - over-specification and under-specification: *int* is too - restrictive (anything with ``__index__`` would be allowed) and it - is not restrictive enough (only the values 0, 1, and 2 are - allowed). Likewise, the annotation ``write(b: bytes)`` was also - too restrictive (anything supporting the buffer protocol would be - allowed). +- The Python standard library should be conservative in adopting such + annotations, but their use is allowed for new code and for big + refactorings. - * Annotations such as ``read1(n: int=None)`` were self-contradictory - since *None* is not an *int*. Annotations such as - ``source_path(self, fullname:str) -> object`` were confusing about - what the return type should be. +- For code that wants to make a different use of function annotations + it is recommended to put a comment of the form:: - * In addition to the above, annotations were inconsistent in the use - of concrete types versus abstract types: *int* versus *Integral* - and set/frozenset versus MutableSet/Set. + # type: ignore - * Some annotations in the abstract base classes were incorrect - specifications. For example, set-to-set operations require - *other* to be another instance of *Set* rather than just an - *Iterable*. + near the top of the file; this tells type checker to ignore all + annotations. (More fine-grained ways of disabling complaints from + type checkers can be found in PEP 484.) - * A further issue was that annotations become part of the - specification but weren't being tested. +- Like linters, type checkers are optional, separate tools. Python + interpreters by default should not issue any messages due to type + checking and should not alter their behavior based on annotations. - * In most cases, the docstrings already included the type - specifications and did so with greater clarity than the function - annotations. In the remaining cases, the docstrings were improved - once the annotations were removed. +- Users who don't want to use type checkers are free to ignore them. + However, it is expected that users of library packages may want to + run type checkers over those library packages. For this purpose PEP + 484 recommends the use of stub files: .pyi files that are read by + the type checker in preference of the corresponding .py files. Stub + files can be distributed with a library, or separately (with the + library author's permission) through the _typeshed_ repo [5]_. - * The observed function annotations were too ad-hoc and inconsistent - to work with a coherent system of automatic type checking or - argument validation. Leaving these annotations in the code would - have made it more difficult to make changes later so that - automated utilities could be supported. +- For code that needs to be backwards compatible, function annotations + can be added in the form of comments. Basically, this Python 3 annotation:: + + def embezzle(self, account: str, funds: int = 1000000, **fake_receipts: str) -> None: + """Embezzle funds from account using fake receipts.""" + + + is equivalent to the following:: + + def embezzle(self, account, funds=1000000, **fake_receipts): + # type: (str, int, **str) -> None + """Embezzle funds from account using fake receipts.""" + + + The _mypy_ type checker [6]_ currently support this syntax, and other + type checkers are encouraged to adopt it. .. rubric:: Footnotes @@ -1312,6 +1328,14 @@ .. [4] PEP 8 modernisation, July 2013 http://bugs.python.org/issue18472 +.. [5] Typeshed repo + https://github.com/python/typeshed + +.. [6] mypy type checker + http://mypy-lang.org + https://github.com/JukkaL/mypy + + Copyright ========= -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 14:28:45 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 05 Jan 2016 19:28:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320440=3A_Cleaning?= =?utf-8?q?_up_the_code_by_using_Py=5FSETREF=2E?= Message-ID: <20160105192844.17337.6589@psf.io> https://hg.python.org/cpython/rev/1118dfcbcc35 changeset: 99771:1118dfcbcc35 parent: 99768:136adbcefa5f user: Serhiy Storchaka date: Tue Jan 05 21:27:54 2016 +0200 summary: Issue #20440: Cleaning up the code by using Py_SETREF. files: Modules/_datetimemodule.c | 24 ++++------------- Modules/_elementtree.c | 4 +-- Modules/_lsprof.c | 7 +--- Modules/_pickle.c | 12 +------- Modules/readline.c | 4 +-- Objects/exceptions.c | 22 ++++++---------- Objects/frameobject.c | 6 +--- Objects/funcobject.c | 35 ++++---------------------- Objects/genobject.c | 12 +------- Objects/object.c | 8 ++--- Objects/typeobject.c | 6 +--- Python/import.c | 10 ++----- Python/peephole.c | 4 +-- Python/sysmodule.c | 5 +--- 14 files changed, 39 insertions(+), 120 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1057,10 +1057,8 @@ } /* Offset is normalized, so it is negative if days < 0 */ if (GET_TD_DAYS(offset) < 0) { - PyObject *temp = offset; sign = '-'; - offset = delta_negative((PyDateTime_Delta *)offset); - Py_DECREF(temp); + Py_SETREF(offset, delta_negative((PyDateTime_Delta *)offset)); if (offset == NULL) return -1; } @@ -3047,10 +3045,8 @@ if (dst == Py_None) goto Inconsistent; if (delta_bool((PyDateTime_Delta *)dst) != 0) { - PyObject *temp = result; - result = add_datetime_timedelta((PyDateTime_DateTime *)result, - (PyDateTime_Delta *)dst, 1); - Py_DECREF(temp); + Py_SETREF(result, add_datetime_timedelta((PyDateTime_DateTime *)result, + (PyDateTime_Delta *)dst, 1)); if (result == NULL) goto Fail; } @@ -4157,10 +4153,7 @@ tz); if (self != NULL && tz != Py_None) { /* Convert UTC to tzinfo's zone. */ - PyObject *temp = self; - - self = _PyObject_CallMethodId(tz, &PyId_fromutc, "O", self); - Py_DECREF(temp); + self = _PyObject_CallMethodId(tz, &PyId_fromutc, "N", self); } return self; } @@ -4195,10 +4188,7 @@ tzinfo); if (self != NULL && tzinfo != Py_None) { /* Convert UTC to tzinfo's zone. */ - PyObject *temp = self; - - self = _PyObject_CallMethodId(tzinfo, &PyId_fromutc, "O", self); - Py_DECREF(temp); + self = _PyObject_CallMethodId(tzinfo, &PyId_fromutc, "N", self); } return self; } @@ -4421,9 +4411,7 @@ return NULL; if (offdiff != NULL) { - PyObject *temp = result; - result = delta_subtract(result, offdiff); - Py_DECREF(temp); + Py_SETREF(result, delta_subtract(result, offdiff)); Py_DECREF(offdiff); } } diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -396,10 +396,8 @@ Py_XDECREF(attrib); /* Replace the objects already pointed to by tag, text and tail. */ - tmp = self_elem->tag; Py_INCREF(tag); - self_elem->tag = tag; - Py_DECREF(tmp); + Py_SETREF(self_elem->tag, tag); tmp = self_elem->text; Py_INCREF(Py_None); diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c --- a/Modules/_lsprof.c +++ b/Modules/_lsprof.c @@ -762,7 +762,6 @@ static int profiler_init(ProfilerObject *pObj, PyObject *args, PyObject *kw) { - PyObject *o; PyObject *timer = NULL; double timeunit = 0.0; int subcalls = 1; @@ -777,11 +776,9 @@ if (setSubcalls(pObj, subcalls) < 0 || setBuiltins(pObj, builtins) < 0) return -1; - o = pObj->externalTimer; - pObj->externalTimer = timer; + pObj->externalTimerUnit = timeunit; Py_XINCREF(timer); - Py_XDECREF(o); - pObj->externalTimerUnit = timeunit; + Py_SETREF(pObj->externalTimer, timer); return 0; } diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -4494,8 +4494,6 @@ static int Pickler_set_persid(PicklerObject *self, PyObject *value) { - PyObject *tmp; - if (value == NULL) { PyErr_SetString(PyExc_TypeError, "attribute deletion is not supported"); @@ -4507,10 +4505,8 @@ return -1; } - tmp = self->pers_func; Py_INCREF(value); - self->pers_func = value; - Py_XDECREF(tmp); /* self->pers_func can be NULL, so be careful. */ + Py_SETREF(self->pers_func, value); return 0; } @@ -6946,8 +6942,6 @@ static int Unpickler_set_persload(UnpicklerObject *self, PyObject *value) { - PyObject *tmp; - if (value == NULL) { PyErr_SetString(PyExc_TypeError, "attribute deletion is not supported"); @@ -6960,10 +6954,8 @@ return -1; } - tmp = self->pers_func; Py_INCREF(value); - self->pers_func = value; - Py_XDECREF(tmp); /* self->pers_func can be NULL, so be careful. */ + Py_SETREF(self->pers_func, value); return 0; } diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -321,10 +321,8 @@ Py_CLEAR(*hook_var); } else if (PyCallable_Check(function)) { - PyObject *tmp = *hook_var; Py_INCREF(function); - *hook_var = function; - Py_XDECREF(tmp); + Py_SETREF(*hook_var, function); } else { PyErr_Format(PyExc_TypeError, diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -59,15 +59,11 @@ static int BaseException_init(PyBaseExceptionObject *self, PyObject *args, PyObject *kwds) { - PyObject *tmp; - if (!_PyArg_NoKeywords(Py_TYPE(self)->tp_name, kwds)) return -1; - tmp = self->args; - self->args = args; - Py_INCREF(self->args); - Py_XDECREF(tmp); + Py_INCREF(args); + Py_SETREF(self->args, args); return 0; } @@ -328,11 +324,10 @@ /* Steals a reference to cause */ void -PyException_SetCause(PyObject *self, PyObject *cause) { - PyObject *old_cause = ((PyBaseExceptionObject *)self)->cause; - ((PyBaseExceptionObject *)self)->cause = cause; +PyException_SetCause(PyObject *self, PyObject *cause) +{ ((PyBaseExceptionObject *)self)->suppress_context = 1; - Py_XDECREF(old_cause); + Py_SETREF(((PyBaseExceptionObject *)self)->cause, cause); } PyObject * @@ -344,10 +339,9 @@ /* Steals a reference to context */ void -PyException_SetContext(PyObject *self, PyObject *context) { - PyObject *old_context = ((PyBaseExceptionObject *)self)->context; - ((PyBaseExceptionObject *)self)->context = context; - Py_XDECREF(old_context); +PyException_SetContext(PyObject *self, PyObject *context) +{ + Py_SETREF(((PyBaseExceptionObject *)self)->context, context); } diff --git a/Objects/frameobject.c b/Objects/frameobject.c --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -349,15 +349,11 @@ static int frame_settrace(PyFrameObject *f, PyObject* v, void *closure) { - PyObject* old_value; - /* We rely on f_lineno being accurate when f_trace is set. */ f->f_lineno = PyFrame_GetLineNumber(f); - old_value = f->f_trace; Py_XINCREF(v); - f->f_trace = v; - Py_XDECREF(old_value); + Py_SETREF(f->f_trace, v); return 0; } diff --git a/Objects/funcobject.c b/Objects/funcobject.c --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -249,7 +249,6 @@ static int func_set_code(PyFunctionObject *op, PyObject *value) { - PyObject *tmp; Py_ssize_t nfree, nclosure; /* Not legal to del f.func_code or to set it to anything @@ -270,10 +269,8 @@ nclosure, nfree); return -1; } - tmp = op->func_code; Py_INCREF(value); - op->func_code = value; - Py_DECREF(tmp); + Py_SETREF(op->func_code, value); return 0; } @@ -287,8 +284,6 @@ static int func_set_name(PyFunctionObject *op, PyObject *value) { - PyObject *tmp; - /* Not legal to del f.func_name or to set it to anything * other than a string object. */ if (value == NULL || !PyUnicode_Check(value)) { @@ -296,10 +291,8 @@ "__name__ must be set to a string object"); return -1; } - tmp = op->func_name; Py_INCREF(value); - op->func_name = value; - Py_DECREF(tmp); + Py_SETREF(op->func_name, value); return 0; } @@ -313,8 +306,6 @@ static int func_set_qualname(PyFunctionObject *op, PyObject *value) { - PyObject *tmp; - /* Not legal to del f.__qualname__ or to set it to anything * other than a string object. */ if (value == NULL || !PyUnicode_Check(value)) { @@ -322,10 +313,8 @@ "__qualname__ must be set to a string object"); return -1; } - tmp = op->func_qualname; Py_INCREF(value); - op->func_qualname = value; - Py_DECREF(tmp); + Py_SETREF(op->func_qualname, value); return 0; } @@ -343,8 +332,6 @@ static int func_set_defaults(PyFunctionObject *op, PyObject *value) { - PyObject *tmp; - /* Legal to del f.func_defaults. * Can only set func_defaults to NULL or a tuple. */ if (value == Py_None) @@ -354,10 +341,8 @@ "__defaults__ must be set to a tuple object"); return -1; } - tmp = op->func_defaults; Py_XINCREF(value); - op->func_defaults = value; - Py_XDECREF(tmp); + Py_SETREF(op->func_defaults, value); return 0; } @@ -375,8 +360,6 @@ static int func_set_kwdefaults(PyFunctionObject *op, PyObject *value) { - PyObject *tmp; - if (value == Py_None) value = NULL; /* Legal to del f.func_kwdefaults. @@ -386,10 +369,8 @@ "__kwdefaults__ must be set to a dict object"); return -1; } - tmp = op->func_kwdefaults; Py_XINCREF(value); - op->func_kwdefaults = value; - Py_XDECREF(tmp); + Py_SETREF(op->func_kwdefaults, value); return 0; } @@ -408,8 +389,6 @@ static int func_set_annotations(PyFunctionObject *op, PyObject *value) { - PyObject *tmp; - if (value == Py_None) value = NULL; /* Legal to del f.func_annotations. @@ -420,10 +399,8 @@ "__annotations__ must be set to a dict object"); return -1; } - tmp = op->func_annotations; Py_XINCREF(value); - op->func_annotations = value; - Py_XDECREF(tmp); + Py_SETREF(op->func_annotations, value); return 0; } diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -510,8 +510,6 @@ static int gen_set_name(PyGenObject *op, PyObject *value) { - PyObject *tmp; - /* Not legal to del gen.gi_name or to set it to anything * other than a string object. */ if (value == NULL || !PyUnicode_Check(value)) { @@ -519,10 +517,8 @@ "__name__ must be set to a string object"); return -1; } - tmp = op->gi_name; Py_INCREF(value); - op->gi_name = value; - Py_DECREF(tmp); + Py_SETREF(op->gi_name, value); return 0; } @@ -536,8 +532,6 @@ static int gen_set_qualname(PyGenObject *op, PyObject *value) { - PyObject *tmp; - /* Not legal to del gen.__qualname__ or to set it to anything * other than a string object. */ if (value == NULL || !PyUnicode_Check(value)) { @@ -545,10 +539,8 @@ "__qualname__ must be set to a string object"); return -1; } - tmp = op->gi_qualname; Py_INCREF(value); - op->gi_qualname = value; - Py_DECREF(tmp); + Py_SETREF(op->gi_qualname, value); return 0; } diff --git a/Objects/object.c b/Objects/object.c --- a/Objects/object.c +++ b/Objects/object.c @@ -1203,7 +1203,7 @@ int PyObject_GenericSetDict(PyObject *obj, PyObject *value, void *context) { - PyObject *dict, **dictptr = _PyObject_GetDictPtr(obj); + PyObject **dictptr = _PyObject_GetDictPtr(obj); if (dictptr == NULL) { PyErr_SetString(PyExc_AttributeError, "This object has no __dict__"); @@ -1219,10 +1219,8 @@ "not a '%.200s'", Py_TYPE(value)->tp_name); return -1; } - dict = *dictptr; - Py_XINCREF(value); - *dictptr = value; - Py_XDECREF(dict); + Py_INCREF(value); + Py_SETREF(*dictptr, value); return 0; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -2092,7 +2092,7 @@ static int subtype_setdict(PyObject *obj, PyObject *value, void *context) { - PyObject *dict, **dictptr; + PyObject **dictptr; PyTypeObject *base; base = get_builtin_base_with_dict(Py_TYPE(obj)); @@ -2123,10 +2123,8 @@ "not a '%.200s'", Py_TYPE(value)->tp_name); return -1; } - dict = *dictptr; Py_XINCREF(value); - *dictptr = value; - Py_XDECREF(dict); + Py_SETREF(*dictptr, value); return 0; } diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -879,10 +879,8 @@ if (PyUnicode_Compare(co->co_filename, oldname)) return; - tmp = co->co_filename; - co->co_filename = newname; - Py_INCREF(co->co_filename); - Py_DECREF(tmp); + Py_INCREF(newname); + Py_SETREF(co->co_filename, newname); constants = co->co_consts; n = PyTuple_GET_SIZE(constants); @@ -1327,10 +1325,8 @@ (always_trim || PyUnicode_CompareWithASCIIString(code->co_name, remove_frames) == 0)) { - PyObject *tmp = *outer_link; - *outer_link = next; Py_XINCREF(next); - Py_DECREF(tmp); + Py_SETREF(*outer_link, next); prev_link = outer_link; } else { diff --git a/Python/peephole.c b/Python/peephole.c --- a/Python/peephole.c +++ b/Python/peephole.c @@ -118,9 +118,7 @@ /* If it's a BUILD_SET, use the PyTuple we just built to create a PyFrozenSet, and use that as the constant instead: */ if (codestr[0] == BUILD_SET) { - PyObject *tuple = newconst; - newconst = PyFrozenSet_New(tuple); - Py_DECREF(tuple); + Py_SETREF(newconst, PyFrozenSet_New(newconst)); if (newconst == NULL) return 0; } diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -436,10 +436,7 @@ return -1; } if (result != Py_None) { - PyObject *temp = frame->f_trace; - frame->f_trace = NULL; - Py_XDECREF(temp); - frame->f_trace = result; + Py_SETREF(frame->f_trace, result); } else { Py_DECREF(result); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 5 15:12:35 2016 From: python-checkins at python.org (ethan.furman) Date: Tue, 05 Jan 2016 20:12:35 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_fix_grammar?= Message-ID: <20160105201235.21239.43583@psf.io> https://hg.python.org/peps/rev/2f85b091045e changeset: 6148:2f85b091045e user: Ethan Furman date: Tue Jan 05 12:12:27 2016 -0800 summary: fix grammar files: pep-0008.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -1301,7 +1301,7 @@ """Embezzle funds from account using fake receipts.""" - The _mypy_ type checker [6]_ currently support this syntax, and other + The _mypy_ type checker [6]_ currently supports this syntax, and other type checkers are encouraged to adopt it. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 18:25:49 2016 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 05 Jan 2016 23:25:49 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Clarify_that_stdlib_experimen?= =?utf-8?q?ts_with_annotations_are_not_encouraged=2E?= Message-ID: <20160105232549.21398.32969@psf.io> https://hg.python.org/peps/rev/19ef54ed6aaa changeset: 6149:19ef54ed6aaa user: Guido van Rossum date: Tue Jan 05 15:25:40 2016 -0800 summary: Clarify that stdlib experiments with annotations are not encouraged. files: pep-0008.txt | 23 ++++++++++++----------- 1 files changed, 12 insertions(+), 11 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -1257,10 +1257,11 @@ - The experimentation with annotation styles that was recommended previously in this PEP is no longer encouraged. -- However, experiments within the rules of PEP 484 are now encouraged. - For example, marking up a large library with PEP 484 style type - annotations, reviewing how easy it was to add those annotations, and - observing whether their presence increases code understandabilty. +- However, outside the stdlib, experiments within the rules of PEP 484 + are now encouraged. For example, marking up a large third party + library or application with PEP 484 style type annotations, + reviewing how easy it was to add those annotations, and observing + whether their presence increases code understandabilty. - The Python standard library should be conservative in adopting such annotations, but their use is allowed for new code and for big @@ -1280,12 +1281,12 @@ checking and should not alter their behavior based on annotations. - Users who don't want to use type checkers are free to ignore them. - However, it is expected that users of library packages may want to - run type checkers over those library packages. For this purpose PEP - 484 recommends the use of stub files: .pyi files that are read by - the type checker in preference of the corresponding .py files. Stub - files can be distributed with a library, or separately (with the - library author's permission) through the _typeshed_ repo [5]_. + However, it is expected that users of third party library packages + may want to run type checkers over those packages. For this purpose + PEP 484 recommends the use of stub files: .pyi files that are read + by the type checker in preference of the corresponding .py files. + Stub files can be distributed with a library, or separately (with + the library author's permission) through the typeshed repo [5]_. - For code that needs to be backwards compatible, function annotations can be added in the form of comments. Basically, this Python 3 annotation:: @@ -1301,7 +1302,7 @@ """Embezzle funds from account using fake receipts.""" - The _mypy_ type checker [6]_ currently supports this syntax, and other + The mypy type checker [6]_ currently supports this syntax, and other type checkers are encouraged to adopt it. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 18:35:47 2016 From: python-checkins at python.org (barry.warsaw) Date: Tue, 05 Jan 2016 23:35:47 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_header_order=2E?= Message-ID: <20160105233547.7758.13642@psf.io> https://hg.python.org/peps/rev/2d4328f39d25 changeset: 6151:2d4328f39d25 user: Barry Warsaw date: Tue Jan 05 18:35:21 2016 -0500 summary: Fix header order. files: pep-0009.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0009.txt b/pep-0009.txt --- a/pep-0009.txt +++ b/pep-0009.txt @@ -7,8 +7,8 @@ Type: Process Content-Type: text/plain Created: 14-Aug-2001 +Post-History: Resolution: https://mail.python.org/mailman/private/peps/2016-January/001165.html -Post-History: Update -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 18:35:48 2016 From: python-checkins at python.org (barry.warsaw) Date: Tue, 05 Jan 2016 23:35:48 +0000 Subject: [Python-checkins] =?utf-8?q?peps_=28merge_default_-=3E_default=29?= =?utf-8?q?=3A_Merge?= Message-ID: <20160105233547.30374.66431@psf.io> https://hg.python.org/peps/rev/d93f2932e2c4 changeset: 6152:d93f2932e2c4 parent: 6151:2d4328f39d25 parent: 6149:19ef54ed6aaa user: Barry Warsaw date: Tue Jan 05 18:35:42 2016 -0500 summary: Merge files: pep-0008.txt | 23 ++++++++++++----------- 1 files changed, 12 insertions(+), 11 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -1257,10 +1257,11 @@ - The experimentation with annotation styles that was recommended previously in this PEP is no longer encouraged. -- However, experiments within the rules of PEP 484 are now encouraged. - For example, marking up a large library with PEP 484 style type - annotations, reviewing how easy it was to add those annotations, and - observing whether their presence increases code understandabilty. +- However, outside the stdlib, experiments within the rules of PEP 484 + are now encouraged. For example, marking up a large third party + library or application with PEP 484 style type annotations, + reviewing how easy it was to add those annotations, and observing + whether their presence increases code understandabilty. - The Python standard library should be conservative in adopting such annotations, but their use is allowed for new code and for big @@ -1280,12 +1281,12 @@ checking and should not alter their behavior based on annotations. - Users who don't want to use type checkers are free to ignore them. - However, it is expected that users of library packages may want to - run type checkers over those library packages. For this purpose PEP - 484 recommends the use of stub files: .pyi files that are read by - the type checker in preference of the corresponding .py files. Stub - files can be distributed with a library, or separately (with the - library author's permission) through the _typeshed_ repo [5]_. + However, it is expected that users of third party library packages + may want to run type checkers over those packages. For this purpose + PEP 484 recommends the use of stub files: .pyi files that are read + by the type checker in preference of the corresponding .py files. + Stub files can be distributed with a library, or separately (with + the library author's permission) through the typeshed repo [5]_. - For code that needs to be backwards compatible, function annotations can be added in the form of comments. Basically, this Python 3 annotation:: @@ -1301,7 +1302,7 @@ """Embezzle funds from account using fake receipts.""" - The _mypy_ type checker [6]_ currently supports this syntax, and other + The mypy type checker [6]_ currently supports this syntax, and other type checkers are encouraged to adopt it. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 18:36:00 2016 From: python-checkins at python.org (barry.warsaw) Date: Tue, 05 Jan 2016 23:36:00 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_9_=28plaintext_PEPs=29_is?= =?utf-8?q?_deprecated=2E?= Message-ID: <20160105233546.69623.28327@psf.io> https://hg.python.org/peps/rev/756c8c349726 changeset: 6150:756c8c349726 parent: 6148:2f85b091045e user: Barry Warsaw date: Tue Jan 05 18:33:02 2016 -0500 summary: PEP 9 (plaintext PEPs) is deprecated. files: pep-0001.txt | 23 ++++++++--------------- pep-0009.txt | 11 ++++++++++- pep-0012.txt | 25 ++++++++----------------- 3 files changed, 26 insertions(+), 33 deletions(-) diff --git a/pep-0001.txt b/pep-0001.txt --- a/pep-0001.txt +++ b/pep-0001.txt @@ -340,22 +340,15 @@ PEP Formats and Templates ========================= -There are two PEP formats available to authors: plaintext and -reStructuredText_. Both are UTF-8-encoded text files. +PEPs are UTF-8 encoded text files using the reStructuredText_ format. +ReStructuredText_ allows for rich markup that is still quite easy to +read, but also results in good-looking and functional HTML. PEP 12 +contains instructions and a template [4]_ for reStructuredText PEPs. -Plaintext PEPs are written with minimal structural markup that adheres -to a rigid style. PEP 9 contains a instructions and a template [3]_ -you can use to get started writing your plaintext PEP. - -ReStructuredText_ PEPs allow for rich markup that is still quite easy -to read, but results in much better-looking and more functional HTML. -PEP 12 contains instructions and a template [4]_ for reStructuredText -PEPs. - -There is a Python script that converts both styles of PEPs to HTML for -viewing on the web [5]_. Parsing and conversion of plaintext PEPs is -self-contained within the script. reStructuredText PEPs are parsed -and converted by Docutils_ code called from the script. +A Python script automatically converts PEPs to HTML for viewing on +the web [5]_. The conversion of reStructuredText PEPs is handled by +the Docutils_ module; the same script also renders a legacy plain-text +format of PEP internally, to support pre-reST documents. PEP Header Preamble diff --git a/pep-0009.txt b/pep-0009.txt --- a/pep-0009.txt +++ b/pep-0009.txt @@ -3,13 +3,22 @@ Version: $Revision$ Last-Modified: $Date$ Author: Barry Warsaw -Status: Active +Status: Withdrawn Type: Process Content-Type: text/plain Created: 14-Aug-2001 +Resolution: https://mail.python.org/mailman/private/peps/2016-January/001165.html Post-History: +Update + + As of 05-Jan-2016, this PEP is officially deprecated and replaced + by PEP 12. All PEPs should now use the reStructuredText format + described by PEP 12, and plaintext PEPs will no longer be + accepted, however you may still see legacy PEPs written using the + plaintext style. + Abstract This PEP provides a boilerplate or sample template for creating diff --git a/pep-0012.txt b/pep-0012.txt --- a/pep-0012.txt +++ b/pep-0012.txt @@ -26,25 +26,19 @@ The source for this (or any) PEP can be found in the PEPs repository, viewable on the web at https://hg.python.org/peps/file/tip . -If you would prefer not to use markup in your PEP, please see PEP 9, -"Sample Plaintext PEP Template" [2]_. - Rationale ========= -PEP submissions come in a wide variety of forms, not all adhering -to the format guidelines set forth below. Use this template, in -conjunction with the format guidelines below, to ensure that your -PEP submission won't get automatically rejected because of form. +If you intend to submit a PEP, you MUST use this template, in +conjunction with the format guidelines below, to ensure that your PEP +submission won't get automatically rejected because of form. -ReStructuredText is offered as an alternative to plaintext PEPs, to -allow PEP authors more functionality and expressivity, while -maintaining easy readability in the source text. The processed HTML -form makes the functionality accessible to readers: live hyperlinks, -styled text, tables, images, and automatic tables of contents, among -other advantages. For an example of a PEP marked up with -reStructuredText, see PEP 287. +ReStructuredText provides PEP authors with useful functionality and +expressivity, while maintaining easy readability in the source text. +The processed HTML form makes the functionality accessible to readers: +live hyperlinks, styled text, tables, images, and automatic tables of +contents, among other advantages. How to Use This Template @@ -622,9 +616,6 @@ .. [1] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton (http://www.python.org/dev/peps/pep-0001) -.. [2] PEP 9, Sample Plaintext PEP Template, Warsaw - (http://www.python.org/dev/peps/pep-0009) - Copyright ========= -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 5 19:04:58 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 06 Jan 2016 00:04:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzY1MDA6?= =?utf-8?q?_Fix_=22maximum_recursion_depth_exceeded=22_error_caused_by?= Message-ID: <20160106000458.9642.13592@psf.io> https://hg.python.org/cpython/rev/fbea8ff8db5e changeset: 99772:fbea8ff8db5e branch: 2.7 parent: 99770:f2a846c1b593 user: Berker Peksag date: Wed Jan 06 02:04:52 2016 +0200 summary: Issue #6500: Fix "maximum recursion depth exceeded" error caused by Request.__getattr__() files: Lib/test/test_urllib2.py | 7 +++++++ Lib/urllib2.py | 3 +-- Misc/NEWS | 3 +++ 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1302,6 +1302,13 @@ self.assertEqual("POST", self.post.get_method()) self.assertEqual("GET", self.get.get_method()) + def test___getattr__(self): + self.assertEqual("GET", self.get._Request__r_method) + self.assertEqual("http://www.python.org/~jeremy/", + self.get._Request__r_full_url) + with self.assertRaises(AttributeError): + self.get._Request__r_invalid_attr + def test_add_data(self): self.assertTrue(not self.get.has_data()) self.assertEqual("GET", self.get.get_method()) diff --git a/Lib/urllib2.py b/Lib/urllib2.py --- a/Lib/urllib2.py +++ b/Lib/urllib2.py @@ -251,8 +251,7 @@ if attr[:12] == '_Request__r_': name = attr[12:] if hasattr(Request, 'get_' + name): - getattr(self, 'get_' + name)() - return getattr(self, attr) + return getattr(self, 'get_' + name)() raise AttributeError, attr def get_method(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,9 @@ Library ------- +- Issue #6500: Fix "maximum recursion depth exceeded" error caused + by urllib2.Request.__getattr__(). + - Issue #24103: Fixed possible use after free in ElementTree.iterparse(). - Issue #20954: _args_from_interpreter_flags used by multiprocessing and some -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 5 20:08:18 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 06 Jan 2016 01:08:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzY1MDA6?= =?utf-8?q?_Reverting_fbea8ff8db5e_since_it_broke_tests?= Message-ID: <20160106010817.3381.73906@psf.io> https://hg.python.org/cpython/rev/e2faa18802bb changeset: 99773:e2faa18802bb branch: 2.7 user: Berker Peksag date: Wed Jan 06 03:08:12 2016 +0200 summary: Issue #6500: Reverting fbea8ff8db5e since it broke tests files: Lib/test/test_urllib2.py | 7 ------- Lib/urllib2.py | 3 ++- Misc/NEWS | 3 --- 3 files changed, 2 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1302,13 +1302,6 @@ self.assertEqual("POST", self.post.get_method()) self.assertEqual("GET", self.get.get_method()) - def test___getattr__(self): - self.assertEqual("GET", self.get._Request__r_method) - self.assertEqual("http://www.python.org/~jeremy/", - self.get._Request__r_full_url) - with self.assertRaises(AttributeError): - self.get._Request__r_invalid_attr - def test_add_data(self): self.assertTrue(not self.get.has_data()) self.assertEqual("GET", self.get.get_method()) diff --git a/Lib/urllib2.py b/Lib/urllib2.py --- a/Lib/urllib2.py +++ b/Lib/urllib2.py @@ -251,7 +251,8 @@ if attr[:12] == '_Request__r_': name = attr[12:] if hasattr(Request, 'get_' + name): - return getattr(self, 'get_' + name)() + getattr(self, 'get_' + name)() + return getattr(self, attr) raise AttributeError, attr def get_method(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,9 +36,6 @@ Library ------- -- Issue #6500: Fix "maximum recursion depth exceeded" error caused - by urllib2.Request.__getattr__(). - - Issue #24103: Fixed possible use after free in ElementTree.iterparse(). - Issue #20954: _args_from_interpreter_flags used by multiprocessing and some -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Wed Jan 6 03:42:38 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 06 Jan 2016 08:42:38 +0000 Subject: [Python-checkins] Daily reference leaks (1118dfcbcc35): sum=25 Message-ID: <20160106084238.7774.82013@psf.io> results for 1118dfcbcc35 on branch "default" -------------------------------------------- test_asyncio leaked [17, 0, 0] references, sum=17 test_asyncio leaked [6, 0, 0] memory blocks, sum=6 test_collections leaked [0, 0, -2] references, sum=-2 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogt_9jbz', '--timeout', '7200'] From python-checkins at python.org Wed Jan 6 06:54:24 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 06 Jan 2016 11:54:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_issue18918?= =?utf-8?q?_=3A_Attach_the_pydoc_documentation_for_=27FILES=27_topic=2E?= Message-ID: <20160106115424.17327.78628@psf.io> https://hg.python.org/cpython/rev/0792a0ad7f22 changeset: 99774:0792a0ad7f22 branch: 2.7 user: Senthil Kumaran date: Wed Jan 06 03:54:18 2016 -0800 summary: Fix issue18918 : Attach the pydoc documentation for 'FILES' topic. Also rebuilt the pydoc_data with this change. files: Doc/tools/extensions/pyspecific.py | 2 +- Lib/pydoc_data/topics.py | 13396 +-------------- 2 files changed, 81 insertions(+), 13317 deletions(-) diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -131,7 +131,7 @@ 'assert', 'assignment', 'atom-identifiers', 'atom-literals', 'attribute-access', 'attribute-references', 'augassign', 'binary', 'bitwise', 'bltin-code-objects', 'bltin-ellipsis-object', - 'bltin-null-object', 'bltin-type-objects', 'booleans', + 'bltin-file-objects', 'bltin-null-object', 'bltin-type-objects', 'booleans', 'break', 'callable-types', 'calls', 'class', 'comparisons', 'compound', 'context-managers', 'continue', 'conversions', 'customization', 'debugger', 'del', 'dict', 'dynamic-features', 'else', 'exceptions', 'exec', 'execmodel', diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,13317 +1,81 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Sat Nov 21 13:35:13 2015 -topics = {'assert': '\n' - 'The "assert" statement\n' - '**********************\n' - '\n' - 'Assert statements are a convenient way to insert debugging ' - 'assertions\n' - 'into a program:\n' - '\n' - ' assert_stmt ::= "assert" expression ["," expression]\n' - '\n' - 'The simple form, "assert expression", is equivalent to\n' - '\n' - ' if __debug__:\n' - ' if not expression: raise AssertionError\n' - '\n' - 'The extended form, "assert expression1, expression2", is ' - 'equivalent to\n' - '\n' - ' if __debug__:\n' - ' if not expression1: raise AssertionError(expression2)\n' - '\n' - 'These equivalences assume that "__debug__" and "AssertionError" ' - 'refer\n' - 'to the built-in variables with those names. In the current\n' - 'implementation, the built-in variable "__debug__" is "True" ' - 'under\n' - 'normal circumstances, "False" when optimization is requested ' - '(command\n' - 'line option -O). The current code generator emits no code for ' - 'an\n' - 'assert statement when optimization is requested at compile ' - 'time. Note\n' - 'that it is unnecessary to include the source code for the ' - 'expression\n' - 'that failed in the error message; it will be displayed as part ' - 'of the\n' - 'stack trace.\n' - '\n' - 'Assignments to "__debug__" are illegal. The value for the ' - 'built-in\n' - 'variable is determined when the interpreter starts.\n', - 'assignment': '\n' - 'Assignment statements\n' - '*********************\n' - '\n' - 'Assignment statements are used to (re)bind names to values ' - 'and to\n' - 'modify attributes or items of mutable objects:\n' - '\n' - ' assignment_stmt ::= (target_list "=")+ (expression_list | ' - 'yield_expression)\n' - ' target_list ::= target ("," target)* [","]\n' - ' target ::= identifier\n' - ' | "(" target_list ")"\n' - ' | "[" target_list "]"\n' - ' | attributeref\n' - ' | subscription\n' - ' | slicing\n' - '\n' - '(See section Primaries for the syntax definitions for the ' - 'last three\n' - 'symbols.)\n' - '\n' - 'An assignment statement evaluates the expression list ' - '(remember that\n' - 'this can be a single expression or a comma-separated list, ' - 'the latter\n' - 'yielding a tuple) and assigns the single resulting object to ' - 'each of\n' - 'the target lists, from left to right.\n' - '\n' - 'Assignment is defined recursively depending on the form of ' - 'the target\n' - '(list). When a target is part of a mutable object (an ' - 'attribute\n' - 'reference, subscription or slicing), the mutable object ' - 'must\n' - 'ultimately perform the assignment and decide about its ' - 'validity, and\n' - 'may raise an exception if the assignment is unacceptable. ' - 'The rules\n' - 'observed by various types and the exceptions raised are ' - 'given with the\n' - 'definition of the object types (see section The standard ' - 'type\n' - 'hierarchy).\n' - '\n' - 'Assignment of an object to a target list is recursively ' - 'defined as\n' - 'follows.\n' - '\n' - '* If the target list is a single target: The object is ' - 'assigned to\n' - ' that target.\n' - '\n' - '* If the target list is a comma-separated list of targets: ' - 'The\n' - ' object must be an iterable with the same number of items ' - 'as there\n' - ' are targets in the target list, and the items are ' - 'assigned, from\n' - ' left to right, to the corresponding targets.\n' - '\n' - 'Assignment of an object to a single target is recursively ' - 'defined as\n' - 'follows.\n' - '\n' - '* If the target is an identifier (name):\n' - '\n' - ' * If the name does not occur in a "global" statement in ' - 'the\n' - ' current code block: the name is bound to the object in ' - 'the current\n' - ' local namespace.\n' - '\n' - ' * Otherwise: the name is bound to the object in the ' - 'current global\n' - ' namespace.\n' - '\n' - ' The name is rebound if it was already bound. This may ' - 'cause the\n' - ' reference count for the object previously bound to the ' - 'name to reach\n' - ' zero, causing the object to be deallocated and its ' - 'destructor (if it\n' - ' has one) to be called.\n' - '\n' - '* If the target is a target list enclosed in parentheses or ' - 'in\n' - ' square brackets: The object must be an iterable with the ' - 'same number\n' - ' of items as there are targets in the target list, and its ' - 'items are\n' - ' assigned, from left to right, to the corresponding ' - 'targets.\n' - '\n' - '* If the target is an attribute reference: The primary ' - 'expression in\n' - ' the reference is evaluated. It should yield an object ' - 'with\n' - ' assignable attributes; if this is not the case, ' - '"TypeError" is\n' - ' raised. That object is then asked to assign the assigned ' - 'object to\n' - ' the given attribute; if it cannot perform the assignment, ' - 'it raises\n' - ' an exception (usually but not necessarily ' - '"AttributeError").\n' - '\n' - ' Note: If the object is a class instance and the attribute ' - 'reference\n' - ' occurs on both sides of the assignment operator, the RHS ' - 'expression,\n' - ' "a.x" can access either an instance attribute or (if no ' - 'instance\n' - ' attribute exists) a class attribute. The LHS target "a.x" ' - 'is always\n' - ' set as an instance attribute, creating it if necessary. ' - 'Thus, the\n' - ' two occurrences of "a.x" do not necessarily refer to the ' - 'same\n' - ' attribute: if the RHS expression refers to a class ' - 'attribute, the\n' - ' LHS creates a new instance attribute as the target of the\n' - ' assignment:\n' - '\n' - ' class Cls:\n' - ' x = 3 # class variable\n' - ' inst = Cls()\n' - ' inst.x = inst.x + 1 # writes inst.x as 4 leaving ' - 'Cls.x as 3\n' - '\n' - ' This description does not necessarily apply to descriptor\n' - ' attributes, such as properties created with "property()".\n' - '\n' - '* If the target is a subscription: The primary expression in ' - 'the\n' - ' reference is evaluated. It should yield either a mutable ' - 'sequence\n' - ' object (such as a list) or a mapping object (such as a ' - 'dictionary).\n' - ' Next, the subscript expression is evaluated.\n' - '\n' - ' If the primary is a mutable sequence object (such as a ' - 'list), the\n' - ' subscript must yield a plain integer. If it is negative, ' - 'the\n' - " sequence's length is added to it. The resulting value must " - 'be a\n' - " nonnegative integer less than the sequence's length, and " - 'the\n' - ' sequence is asked to assign the assigned object to its ' - 'item with\n' - ' that index. If the index is out of range, "IndexError" is ' - 'raised\n' - ' (assignment to a subscripted sequence cannot add new items ' - 'to a\n' - ' list).\n' - '\n' - ' If the primary is a mapping object (such as a dictionary), ' - 'the\n' - " subscript must have a type compatible with the mapping's " - 'key type,\n' - ' and the mapping is then asked to create a key/datum pair ' - 'which maps\n' - ' the subscript to the assigned object. This can either ' - 'replace an\n' - ' existing key/value pair with the same key value, or insert ' - 'a new\n' - ' key/value pair (if no key with the same value existed).\n' - '\n' - '* If the target is a slicing: The primary expression in the\n' - ' reference is evaluated. It should yield a mutable ' - 'sequence object\n' - ' (such as a list). The assigned object should be a ' - 'sequence object\n' - ' of the same type. Next, the lower and upper bound ' - 'expressions are\n' - ' evaluated, insofar they are present; defaults are zero and ' - 'the\n' - " sequence's length. The bounds should evaluate to (small) " - 'integers.\n' - " If either bound is negative, the sequence's length is " - 'added to it.\n' - ' The resulting bounds are clipped to lie between zero and ' - 'the\n' - " sequence's length, inclusive. Finally, the sequence " - 'object is asked\n' - ' to replace the slice with the items of the assigned ' - 'sequence. The\n' - ' length of the slice may be different from the length of ' - 'the assigned\n' - ' sequence, thus changing the length of the target sequence, ' - 'if the\n' - ' object allows it.\n' - '\n' - '**CPython implementation detail:** In the current ' - 'implementation, the\n' - 'syntax for targets is taken to be the same as for ' - 'expressions, and\n' - 'invalid syntax is rejected during the code generation phase, ' - 'causing\n' - 'less detailed error messages.\n' - '\n' - 'WARNING: Although the definition of assignment implies that ' - 'overlaps\n' - 'between the left-hand side and the right-hand side are ' - "'safe' (for\n" - 'example "a, b = b, a" swaps two variables), overlaps ' - '*within* the\n' - 'collection of assigned-to variables are not safe! For ' - 'instance, the\n' - 'following program prints "[0, 2]":\n' - '\n' - ' x = [0, 1]\n' - ' i = 0\n' - ' i, x[i] = 1, 2\n' - ' print x\n' - '\n' - '\n' - 'Augmented assignment statements\n' - '===============================\n' - '\n' - 'Augmented assignment is the combination, in a single ' - 'statement, of a\n' - 'binary operation and an assignment statement:\n' - '\n' - ' augmented_assignment_stmt ::= augtarget augop ' - '(expression_list | yield_expression)\n' - ' augtarget ::= identifier | attributeref | ' - 'subscription | slicing\n' - ' augop ::= "+=" | "-=" | "*=" | "/=" | ' - '"//=" | "%=" | "**="\n' - ' | ">>=" | "<<=" | "&=" | "^=" | "|="\n' - '\n' - '(See section Primaries for the syntax definitions for the ' - 'last three\n' - 'symbols.)\n' - '\n' - 'An augmented assignment evaluates the target (which, unlike ' - 'normal\n' - 'assignment statements, cannot be an unpacking) and the ' - 'expression\n' - 'list, performs the binary operation specific to the type of ' - 'assignment\n' - 'on the two operands, and assigns the result to the original ' - 'target.\n' - 'The target is only evaluated once.\n' - '\n' - 'An augmented assignment expression like "x += 1" can be ' - 'rewritten as\n' - '"x = x + 1" to achieve a similar, but not exactly equal ' - 'effect. In the\n' - 'augmented version, "x" is only evaluated once. Also, when ' - 'possible,\n' - 'the actual operation is performed *in-place*, meaning that ' - 'rather than\n' - 'creating a new object and assigning that to the target, the ' - 'old object\n' - 'is modified instead.\n' - '\n' - 'With the exception of assigning to tuples and multiple ' - 'targets in a\n' - 'single statement, the assignment done by augmented ' - 'assignment\n' - 'statements is handled the same way as normal assignments. ' - 'Similarly,\n' - 'with the exception of the possible *in-place* behavior, the ' - 'binary\n' - 'operation performed by augmented assignment is the same as ' - 'the normal\n' - 'binary operations.\n' - '\n' - 'For targets which are attribute references, the same caveat ' - 'about\n' - 'class and instance attributes applies as for regular ' - 'assignments.\n', - 'atom-identifiers': '\n' - 'Identifiers (Names)\n' - '*******************\n' - '\n' - 'An identifier occurring as an atom is a name. See ' - 'section Identifiers\n' - 'and keywords for lexical definition and section Naming ' - 'and binding for\n' - 'documentation of naming and binding.\n' - '\n' - 'When the name is bound to an object, evaluation of the ' - 'atom yields\n' - 'that object. When a name is not bound, an attempt to ' - 'evaluate it\n' - 'raises a "NameError" exception.\n' - '\n' - '**Private name mangling:** When an identifier that ' - 'textually occurs in\n' - 'a class definition begins with two or more underscore ' - 'characters and\n' - 'does not end in two or more underscores, it is ' - 'considered a *private\n' - 'name* of that class. Private names are transformed to ' - 'a longer form\n' - 'before code is generated for them. The transformation ' - 'inserts the\n' - 'class name, with leading underscores removed and a ' - 'single underscore\n' - 'inserted, in front of the name. For example, the ' - 'identifier "__spam"\n' - 'occurring in a class named "Ham" will be transformed ' - 'to "_Ham__spam".\n' - 'This transformation is independent of the syntactical ' - 'context in which\n' - 'the identifier is used. If the transformed name is ' - 'extremely long\n' - '(longer than 255 characters), implementation defined ' - 'truncation may\n' - 'happen. If the class name consists only of ' - 'underscores, no\n' - 'transformation is done.\n', - 'atom-literals': '\n' - 'Literals\n' - '********\n' - '\n' - 'Python supports string literals and various numeric ' - 'literals:\n' - '\n' - ' literal ::= stringliteral | integer | longinteger\n' - ' | floatnumber | imagnumber\n' - '\n' - 'Evaluation of a literal yields an object of the given ' - 'type (string,\n' - 'integer, long integer, floating point number, complex ' - 'number) with the\n' - 'given value. The value may be approximated in the case ' - 'of floating\n' - 'point and imaginary (complex) literals. See section ' - 'Literals for\n' - 'details.\n' - '\n' - 'All literals correspond to immutable data types, and ' - 'hence the\n' - "object's identity is less important than its value. " - 'Multiple\n' - 'evaluations of literals with the same value (either the ' - 'same\n' - 'occurrence in the program text or a different occurrence) ' - 'may obtain\n' - 'the same object or a different object with the same ' - 'value.\n', - 'attribute-access': '\n' - 'Customizing attribute access\n' - '****************************\n' - '\n' - 'The following methods can be defined to customize the ' - 'meaning of\n' - 'attribute access (use of, assignment to, or deletion ' - 'of "x.name") for\n' - 'class instances.\n' - '\n' - 'object.__getattr__(self, name)\n' - '\n' - ' Called when an attribute lookup has not found the ' - 'attribute in the\n' - ' usual places (i.e. it is not an instance attribute ' - 'nor is it found\n' - ' in the class tree for "self"). "name" is the ' - 'attribute name. This\n' - ' method should return the (computed) attribute value ' - 'or raise an\n' - ' "AttributeError" exception.\n' - '\n' - ' Note that if the attribute is found through the ' - 'normal mechanism,\n' - ' "__getattr__()" is not called. (This is an ' - 'intentional asymmetry\n' - ' between "__getattr__()" and "__setattr__()".) This ' - 'is done both for\n' - ' efficiency reasons and because otherwise ' - '"__getattr__()" would have\n' - ' no way to access other attributes of the instance. ' - 'Note that at\n' - ' least for instance variables, you can fake total ' - 'control by not\n' - ' inserting any values in the instance attribute ' - 'dictionary (but\n' - ' instead inserting them in another object). See ' - 'the\n' - ' "__getattribute__()" method below for a way to ' - 'actually get total\n' - ' control in new-style classes.\n' - '\n' - 'object.__setattr__(self, name, value)\n' - '\n' - ' Called when an attribute assignment is attempted. ' - 'This is called\n' - ' instead of the normal mechanism (i.e. store the ' - 'value in the\n' - ' instance dictionary). *name* is the attribute ' - 'name, *value* is the\n' - ' value to be assigned to it.\n' - '\n' - ' If "__setattr__()" wants to assign to an instance ' - 'attribute, it\n' - ' should not simply execute "self.name = value" --- ' - 'this would cause\n' - ' a recursive call to itself. Instead, it should ' - 'insert the value in\n' - ' the dictionary of instance attributes, e.g., ' - '"self.__dict__[name] =\n' - ' value". For new-style classes, rather than ' - 'accessing the instance\n' - ' dictionary, it should call the base class method ' - 'with the same\n' - ' name, for example, "object.__setattr__(self, name, ' - 'value)".\n' - '\n' - 'object.__delattr__(self, name)\n' - '\n' - ' Like "__setattr__()" but for attribute deletion ' - 'instead of\n' - ' assignment. This should only be implemented if ' - '"del obj.name" is\n' - ' meaningful for the object.\n' - '\n' - '\n' - 'More attribute access for new-style classes\n' - '===========================================\n' - '\n' - 'The following methods only apply to new-style ' - 'classes.\n' - '\n' - 'object.__getattribute__(self, name)\n' - '\n' - ' Called unconditionally to implement attribute ' - 'accesses for\n' - ' instances of the class. If the class also defines ' - '"__getattr__()",\n' - ' the latter will not be called unless ' - '"__getattribute__()" either\n' - ' calls it explicitly or raises an "AttributeError". ' - 'This method\n' - ' should return the (computed) attribute value or ' - 'raise an\n' - ' "AttributeError" exception. In order to avoid ' - 'infinite recursion in\n' - ' this method, its implementation should always call ' - 'the base class\n' - ' method with the same name to access any attributes ' - 'it needs, for\n' - ' example, "object.__getattribute__(self, name)".\n' - '\n' - ' Note: This method may still be bypassed when ' - 'looking up special\n' - ' methods as the result of implicit invocation via ' - 'language syntax\n' - ' or built-in functions. See Special method lookup ' - 'for new-style\n' - ' classes.\n' - '\n' - '\n' - 'Implementing Descriptors\n' - '========================\n' - '\n' - 'The following methods only apply when an instance of ' - 'the class\n' - 'containing the method (a so-called *descriptor* class) ' - 'appears in an\n' - '*owner* class (the descriptor must be in either the ' - "owner's class\n" - 'dictionary or in the class dictionary for one of its ' - 'parents). In the\n' - 'examples below, "the attribute" refers to the ' - 'attribute whose name is\n' - "the key of the property in the owner class' " - '"__dict__".\n' - '\n' - 'object.__get__(self, instance, owner)\n' - '\n' - ' Called to get the attribute of the owner class ' - '(class attribute\n' - ' access) or of an instance of that class (instance ' - 'attribute\n' - ' access). *owner* is always the owner class, while ' - '*instance* is the\n' - ' instance that the attribute was accessed through, ' - 'or "None" when\n' - ' the attribute is accessed through the *owner*. ' - 'This method should\n' - ' return the (computed) attribute value or raise an ' - '"AttributeError"\n' - ' exception.\n' - '\n' - 'object.__set__(self, instance, value)\n' - '\n' - ' Called to set the attribute on an instance ' - '*instance* of the owner\n' - ' class to a new value, *value*.\n' - '\n' - 'object.__delete__(self, instance)\n' - '\n' - ' Called to delete the attribute on an instance ' - '*instance* of the\n' - ' owner class.\n' - '\n' - '\n' - 'Invoking Descriptors\n' - '====================\n' - '\n' - 'In general, a descriptor is an object attribute with ' - '"binding\n' - 'behavior", one whose attribute access has been ' - 'overridden by methods\n' - 'in the descriptor protocol: "__get__()", "__set__()", ' - 'and\n' - '"__delete__()". If any of those methods are defined ' - 'for an object, it\n' - 'is said to be a descriptor.\n' - '\n' - 'The default behavior for attribute access is to get, ' - 'set, or delete\n' - "the attribute from an object's dictionary. For " - 'instance, "a.x" has a\n' - 'lookup chain starting with "a.__dict__[\'x\']", then\n' - '"type(a).__dict__[\'x\']", and continuing through the ' - 'base classes of\n' - '"type(a)" excluding metaclasses.\n' - '\n' - 'However, if the looked-up value is an object defining ' - 'one of the\n' - 'descriptor methods, then Python may override the ' - 'default behavior and\n' - 'invoke the descriptor method instead. Where this ' - 'occurs in the\n' - 'precedence chain depends on which descriptor methods ' - 'were defined and\n' - 'how they were called. Note that descriptors are only ' - 'invoked for new\n' - 'style objects or classes (ones that subclass ' - '"object()" or "type()").\n' - '\n' - 'The starting point for descriptor invocation is a ' - 'binding, "a.x". How\n' - 'the arguments are assembled depends on "a":\n' - '\n' - 'Direct Call\n' - ' The simplest and least common call is when user ' - 'code directly\n' - ' invokes a descriptor method: "x.__get__(a)".\n' - '\n' - 'Instance Binding\n' - ' If binding to a new-style object instance, "a.x" is ' - 'transformed\n' - ' into the call: "type(a).__dict__[\'x\'].__get__(a, ' - 'type(a))".\n' - '\n' - 'Class Binding\n' - ' If binding to a new-style class, "A.x" is ' - 'transformed into the\n' - ' call: "A.__dict__[\'x\'].__get__(None, A)".\n' - '\n' - 'Super Binding\n' - ' If "a" is an instance of "super", then the binding ' - '"super(B,\n' - ' obj).m()" searches "obj.__class__.__mro__" for the ' - 'base class "A"\n' - ' immediately preceding "B" and then invokes the ' - 'descriptor with the\n' - ' call: "A.__dict__[\'m\'].__get__(obj, ' - 'obj.__class__)".\n' - '\n' - 'For instance bindings, the precedence of descriptor ' - 'invocation depends\n' - 'on the which descriptor methods are defined. A ' - 'descriptor can define\n' - 'any combination of "__get__()", "__set__()" and ' - '"__delete__()". If it\n' - 'does not define "__get__()", then accessing the ' - 'attribute will return\n' - 'the descriptor object itself unless there is a value ' - "in the object's\n" - 'instance dictionary. If the descriptor defines ' - '"__set__()" and/or\n' - '"__delete__()", it is a data descriptor; if it defines ' - 'neither, it is\n' - 'a non-data descriptor. Normally, data descriptors ' - 'define both\n' - '"__get__()" and "__set__()", while non-data ' - 'descriptors have just the\n' - '"__get__()" method. Data descriptors with "__set__()" ' - 'and "__get__()"\n' - 'defined always override a redefinition in an instance ' - 'dictionary. In\n' - 'contrast, non-data descriptors can be overridden by ' - 'instances.\n' - '\n' - 'Python methods (including "staticmethod()" and ' - '"classmethod()") are\n' - 'implemented as non-data descriptors. Accordingly, ' - 'instances can\n' - 'redefine and override methods. This allows individual ' - 'instances to\n' - 'acquire behaviors that differ from other instances of ' - 'the same class.\n' - '\n' - 'The "property()" function is implemented as a data ' - 'descriptor.\n' - 'Accordingly, instances cannot override the behavior of ' - 'a property.\n' - '\n' - '\n' - '__slots__\n' - '=========\n' - '\n' - 'By default, instances of both old and new-style ' - 'classes have a\n' - 'dictionary for attribute storage. This wastes space ' - 'for objects\n' - 'having very few instance variables. The space ' - 'consumption can become\n' - 'acute when creating large numbers of instances.\n' - '\n' - 'The default can be overridden by defining *__slots__* ' - 'in a new-style\n' - 'class definition. The *__slots__* declaration takes a ' - 'sequence of\n' - 'instance variables and reserves just enough space in ' - 'each instance to\n' - 'hold a value for each variable. Space is saved ' - 'because *__dict__* is\n' - 'not created for each instance.\n' - '\n' - '__slots__\n' - '\n' - ' This class variable can be assigned a string, ' - 'iterable, or sequence\n' - ' of strings with variable names used by instances. ' - 'If defined in a\n' - ' new-style class, *__slots__* reserves space for the ' - 'declared\n' - ' variables and prevents the automatic creation of ' - '*__dict__* and\n' - ' *__weakref__* for each instance.\n' - '\n' - ' New in version 2.2.\n' - '\n' - 'Notes on using *__slots__*\n' - '\n' - '* When inheriting from a class without *__slots__*, ' - 'the *__dict__*\n' - ' attribute of that class will always be accessible, ' - 'so a *__slots__*\n' - ' definition in the subclass is meaningless.\n' - '\n' - '* Without a *__dict__* variable, instances cannot be ' - 'assigned new\n' - ' variables not listed in the *__slots__* definition. ' - 'Attempts to\n' - ' assign to an unlisted variable name raises ' - '"AttributeError". If\n' - ' dynamic assignment of new variables is desired, then ' - 'add\n' - ' "\'__dict__\'" to the sequence of strings in the ' - '*__slots__*\n' - ' declaration.\n' - '\n' - ' Changed in version 2.3: Previously, adding ' - '"\'__dict__\'" to the\n' - ' *__slots__* declaration would not enable the ' - 'assignment of new\n' - ' attributes not specifically listed in the sequence ' - 'of instance\n' - ' variable names.\n' - '\n' - '* Without a *__weakref__* variable for each instance, ' - 'classes\n' - ' defining *__slots__* do not support weak references ' - 'to its\n' - ' instances. If weak reference support is needed, then ' - 'add\n' - ' "\'__weakref__\'" to the sequence of strings in the ' - '*__slots__*\n' - ' declaration.\n' - '\n' - ' Changed in version 2.3: Previously, adding ' - '"\'__weakref__\'" to the\n' - ' *__slots__* declaration would not enable support for ' - 'weak\n' - ' references.\n' - '\n' - '* *__slots__* are implemented at the class level by ' - 'creating\n' - ' descriptors (Implementing Descriptors) for each ' - 'variable name. As a\n' - ' result, class attributes cannot be used to set ' - 'default values for\n' - ' instance variables defined by *__slots__*; ' - 'otherwise, the class\n' - ' attribute would overwrite the descriptor ' - 'assignment.\n' - '\n' - '* The action of a *__slots__* declaration is limited ' - 'to the class\n' - ' where it is defined. As a result, subclasses will ' - 'have a *__dict__*\n' - ' unless they also define *__slots__* (which must only ' - 'contain names\n' - ' of any *additional* slots).\n' - '\n' - '* If a class defines a slot also defined in a base ' - 'class, the\n' - ' instance variable defined by the base class slot is ' - 'inaccessible\n' - ' (except by retrieving its descriptor directly from ' - 'the base class).\n' - ' This renders the meaning of the program undefined. ' - 'In the future, a\n' - ' check may be added to prevent this.\n' - '\n' - '* Nonempty *__slots__* does not work for classes ' - 'derived from\n' - ' "variable-length" built-in types such as "long", ' - '"str" and "tuple".\n' - '\n' - '* Any non-string iterable may be assigned to ' - '*__slots__*. Mappings\n' - ' may also be used; however, in the future, special ' - 'meaning may be\n' - ' assigned to the values corresponding to each key.\n' - '\n' - '* *__class__* assignment works only if both classes ' - 'have the same\n' - ' *__slots__*.\n' - '\n' - ' Changed in version 2.6: Previously, *__class__* ' - 'assignment raised an\n' - ' error if either new or old class had *__slots__*.\n', - 'attribute-references': '\n' - 'Attribute references\n' - '********************\n' - '\n' - 'An attribute reference is a primary followed by a ' - 'period and a name:\n' - '\n' - ' attributeref ::= primary "." identifier\n' - '\n' - 'The primary must evaluate to an object of a type ' - 'that supports\n' - 'attribute references, e.g., a module, list, or an ' - 'instance. This\n' - 'object is then asked to produce the attribute ' - 'whose name is the\n' - 'identifier. If this attribute is not available, ' - 'the exception\n' - '"AttributeError" is raised. Otherwise, the type ' - 'and value of the\n' - 'object produced is determined by the object. ' - 'Multiple evaluations of\n' - 'the same attribute reference may yield different ' - 'objects.\n', - 'augassign': '\n' - 'Augmented assignment statements\n' - '*******************************\n' - '\n' - 'Augmented assignment is the combination, in a single ' - 'statement, of a\n' - 'binary operation and an assignment statement:\n' - '\n' - ' augmented_assignment_stmt ::= augtarget augop ' - '(expression_list | yield_expression)\n' - ' augtarget ::= identifier | attributeref | ' - 'subscription | slicing\n' - ' augop ::= "+=" | "-=" | "*=" | "/=" | ' - '"//=" | "%=" | "**="\n' - ' | ">>=" | "<<=" | "&=" | "^=" | "|="\n' - '\n' - '(See section Primaries for the syntax definitions for the ' - 'last three\n' - 'symbols.)\n' - '\n' - 'An augmented assignment evaluates the target (which, unlike ' - 'normal\n' - 'assignment statements, cannot be an unpacking) and the ' - 'expression\n' - 'list, performs the binary operation specific to the type of ' - 'assignment\n' - 'on the two operands, and assigns the result to the original ' - 'target.\n' - 'The target is only evaluated once.\n' - '\n' - 'An augmented assignment expression like "x += 1" can be ' - 'rewritten as\n' - '"x = x + 1" to achieve a similar, but not exactly equal ' - 'effect. In the\n' - 'augmented version, "x" is only evaluated once. Also, when ' - 'possible,\n' - 'the actual operation is performed *in-place*, meaning that ' - 'rather than\n' - 'creating a new object and assigning that to the target, the ' - 'old object\n' - 'is modified instead.\n' - '\n' - 'With the exception of assigning to tuples and multiple ' - 'targets in a\n' - 'single statement, the assignment done by augmented ' - 'assignment\n' - 'statements is handled the same way as normal assignments. ' - 'Similarly,\n' - 'with the exception of the possible *in-place* behavior, the ' - 'binary\n' - 'operation performed by augmented assignment is the same as ' - 'the normal\n' - 'binary operations.\n' - '\n' - 'For targets which are attribute references, the same caveat ' - 'about\n' - 'class and instance attributes applies as for regular ' - 'assignments.\n', - 'binary': '\n' - 'Binary arithmetic operations\n' - '****************************\n' - '\n' - 'The binary arithmetic operations have the conventional priority\n' - 'levels. Note that some of these operations also apply to ' - 'certain non-\n' - 'numeric types. Apart from the power operator, there are only ' - 'two\n' - 'levels, one for multiplicative operators and one for additive\n' - 'operators:\n' - '\n' - ' m_expr ::= u_expr | m_expr "*" u_expr | m_expr "//" u_expr | ' - 'm_expr "/" u_expr\n' - ' | m_expr "%" u_expr\n' - ' a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr\n' - '\n' - 'The "*" (multiplication) operator yields the product of its ' - 'arguments.\n' - 'The arguments must either both be numbers, or one argument must ' - 'be an\n' - 'integer (plain or long) and the other must be a sequence. In ' - 'the\n' - 'former case, the numbers are converted to a common type and ' - 'then\n' - 'multiplied together. In the latter case, sequence repetition ' - 'is\n' - 'performed; a negative repetition factor yields an empty ' - 'sequence.\n' - '\n' - 'The "/" (division) and "//" (floor division) operators yield ' - 'the\n' - 'quotient of their arguments. The numeric arguments are first\n' - 'converted to a common type. Plain or long integer division ' - 'yields an\n' - 'integer of the same type; the result is that of mathematical ' - 'division\n' - "with the 'floor' function applied to the result. Division by " - 'zero\n' - 'raises the "ZeroDivisionError" exception.\n' - '\n' - 'The "%" (modulo) operator yields the remainder from the division ' - 'of\n' - 'the first argument by the second. The numeric arguments are ' - 'first\n' - 'converted to a common type. A zero right argument raises the\n' - '"ZeroDivisionError" exception. The arguments may be floating ' - 'point\n' - 'numbers, e.g., "3.14%0.7" equals "0.34" (since "3.14" equals ' - '"4*0.7 +\n' - '0.34".) The modulo operator always yields a result with the ' - 'same sign\n' - 'as its second operand (or zero); the absolute value of the ' - 'result is\n' - 'strictly smaller than the absolute value of the second operand ' - '[2].\n' - '\n' - 'The integer division and modulo operators are connected by the\n' - 'following identity: "x == (x/y)*y + (x%y)". Integer division ' - 'and\n' - 'modulo are also connected with the built-in function ' - '"divmod()":\n' - '"divmod(x, y) == (x/y, x%y)". These identities don\'t hold for\n' - 'floating point numbers; there similar identities hold ' - 'approximately\n' - 'where "x/y" is replaced by "floor(x/y)" or "floor(x/y) - 1" ' - '[3].\n' - '\n' - 'In addition to performing the modulo operation on numbers, the ' - '"%"\n' - 'operator is also overloaded by string and unicode objects to ' - 'perform\n' - 'string formatting (also known as interpolation). The syntax for ' - 'string\n' - 'formatting is described in the Python Library Reference, ' - 'section\n' - 'String Formatting Operations.\n' - '\n' - 'Deprecated since version 2.3: The floor division operator, the ' - 'modulo\n' - 'operator, and the "divmod()" function are no longer defined for\n' - 'complex numbers. Instead, convert to a floating point number ' - 'using\n' - 'the "abs()" function if appropriate.\n' - '\n' - 'The "+" (addition) operator yields the sum of its arguments. ' - 'The\n' - 'arguments must either both be numbers or both sequences of the ' - 'same\n' - 'type. In the former case, the numbers are converted to a common ' - 'type\n' - 'and then added together. In the latter case, the sequences are\n' - 'concatenated.\n' - '\n' - 'The "-" (subtraction) operator yields the difference of its ' - 'arguments.\n' - 'The numeric arguments are first converted to a common type.\n', - 'bitwise': '\n' - 'Binary bitwise operations\n' - '*************************\n' - '\n' - 'Each of the three bitwise operations has a different priority ' - 'level:\n' - '\n' - ' and_expr ::= shift_expr | and_expr "&" shift_expr\n' - ' xor_expr ::= and_expr | xor_expr "^" and_expr\n' - ' or_expr ::= xor_expr | or_expr "|" xor_expr\n' - '\n' - 'The "&" operator yields the bitwise AND of its arguments, which ' - 'must\n' - 'be plain or long integers. The arguments are converted to a ' - 'common\n' - 'type.\n' - '\n' - 'The "^" operator yields the bitwise XOR (exclusive OR) of its\n' - 'arguments, which must be plain or long integers. The arguments ' - 'are\n' - 'converted to a common type.\n' - '\n' - 'The "|" operator yields the bitwise (inclusive) OR of its ' - 'arguments,\n' - 'which must be plain or long integers. The arguments are ' - 'converted to\n' - 'a common type.\n', - 'bltin-code-objects': '\n' - 'Code Objects\n' - '************\n' - '\n' - 'Code objects are used by the implementation to ' - 'represent "pseudo-\n' - 'compiled" executable Python code such as a function ' - 'body. They differ\n' - "from function objects because they don't contain a " - 'reference to their\n' - 'global execution environment. Code objects are ' - 'returned by the built-\n' - 'in "compile()" function and can be extracted from ' - 'function objects\n' - 'through their "func_code" attribute. See also the ' - '"code" module.\n' - '\n' - 'A code object can be executed or evaluated by ' - 'passing it (instead of a\n' - 'source string) to the "exec" statement or the ' - 'built-in "eval()"\n' - 'function.\n' - '\n' - 'See The standard type hierarchy for more ' - 'information.\n', - 'bltin-ellipsis-object': '\n' - 'The Ellipsis Object\n' - '*******************\n' - '\n' - 'This object is used by extended slice notation ' - '(see Slicings). It\n' - 'supports no special operations. There is exactly ' - 'one ellipsis object,\n' - 'named "Ellipsis" (a built-in name).\n' - '\n' - 'It is written as "Ellipsis". When in a ' - 'subscript, it can also be\n' - 'written as "...", for example "seq[...]".\n', - 'bltin-null-object': '\n' - 'The Null Object\n' - '***************\n' - '\n' - "This object is returned by functions that don't " - 'explicitly return a\n' - 'value. It supports no special operations. There is ' - 'exactly one null\n' - 'object, named "None" (a built-in name).\n' - '\n' - 'It is written as "None".\n', - 'bltin-type-objects': '\n' - 'Type Objects\n' - '************\n' - '\n' - 'Type objects represent the various object types. An ' - "object's type is\n" - 'accessed by the built-in function "type()". There ' - 'are no special\n' - 'operations on types. The standard module "types" ' - 'defines names for\n' - 'all standard built-in types.\n' - '\n' - 'Types are written like this: "".\n', - 'booleans': '\n' - 'Boolean operations\n' - '******************\n' - '\n' - ' or_test ::= and_test | or_test "or" and_test\n' - ' and_test ::= not_test | and_test "and" not_test\n' - ' not_test ::= comparison | "not" not_test\n' - '\n' - 'In the context of Boolean operations, and also when ' - 'expressions are\n' - 'used by control flow statements, the following values are ' - 'interpreted\n' - 'as false: "False", "None", numeric zero of all types, and ' - 'empty\n' - 'strings and containers (including strings, tuples, lists,\n' - 'dictionaries, sets and frozensets). All other values are ' - 'interpreted\n' - 'as true. (See the "__nonzero__()" special method for a way to ' - 'change\n' - 'this.)\n' - '\n' - 'The operator "not" yields "True" if its argument is false, ' - '"False"\n' - 'otherwise.\n' - '\n' - 'The expression "x and y" first evaluates *x*; if *x* is false, ' - 'its\n' - 'value is returned; otherwise, *y* is evaluated and the ' - 'resulting value\n' - 'is returned.\n' - '\n' - 'The expression "x or y" first evaluates *x*; if *x* is true, ' - 'its value\n' - 'is returned; otherwise, *y* is evaluated and the resulting ' - 'value is\n' - 'returned.\n' - '\n' - '(Note that neither "and" nor "or" restrict the value and type ' - 'they\n' - 'return to "False" and "True", but rather return the last ' - 'evaluated\n' - 'argument. This is sometimes useful, e.g., if "s" is a string ' - 'that\n' - 'should be replaced by a default value if it is empty, the ' - 'expression\n' - '"s or \'foo\'" yields the desired value. Because "not" has to ' - 'invent a\n' - 'value anyway, it does not bother to return a value of the same ' - 'type as\n' - 'its argument, so e.g., "not \'foo\'" yields "False", not ' - '"\'\'".)\n', - 'break': '\n' - 'The "break" statement\n' - '*********************\n' - '\n' - ' break_stmt ::= "break"\n' - '\n' - '"break" may only occur syntactically nested in a "for" or ' - '"while"\n' - 'loop, but not nested in a function or class definition within ' - 'that\n' - 'loop.\n' - '\n' - 'It terminates the nearest enclosing loop, skipping the optional ' - '"else"\n' - 'clause if the loop has one.\n' - '\n' - 'If a "for" loop is terminated by "break", the loop control ' - 'target\n' - 'keeps its current value.\n' - '\n' - 'When "break" passes control out of a "try" statement with a ' - '"finally"\n' - 'clause, that "finally" clause is executed before really leaving ' - 'the\n' - 'loop.\n', - 'callable-types': '\n' - 'Emulating callable objects\n' - '**************************\n' - '\n' - 'object.__call__(self[, args...])\n' - '\n' - ' Called when the instance is "called" as a function; ' - 'if this method\n' - ' is defined, "x(arg1, arg2, ...)" is a shorthand for\n' - ' "x.__call__(arg1, arg2, ...)".\n', - 'calls': '\n' - 'Calls\n' - '*****\n' - '\n' - 'A call calls a callable object (e.g., a *function*) with a ' - 'possibly\n' - 'empty series of *arguments*:\n' - '\n' - ' call ::= primary "(" [argument_list [","]\n' - ' | expression genexpr_for] ")"\n' - ' argument_list ::= positional_arguments ["," ' - 'keyword_arguments]\n' - ' ["," "*" expression] ["," ' - 'keyword_arguments]\n' - ' ["," "**" expression]\n' - ' | keyword_arguments ["," "*" expression]\n' - ' ["," "**" expression]\n' - ' | "*" expression ["," keyword_arguments] ' - '["," "**" expression]\n' - ' | "**" expression\n' - ' positional_arguments ::= expression ("," expression)*\n' - ' keyword_arguments ::= keyword_item ("," keyword_item)*\n' - ' keyword_item ::= identifier "=" expression\n' - '\n' - 'A trailing comma may be present after the positional and keyword\n' - 'arguments but does not affect the semantics.\n' - '\n' - 'The primary must evaluate to a callable object (user-defined\n' - 'functions, built-in functions, methods of built-in objects, ' - 'class\n' - 'objects, methods of class instances, and certain class instances\n' - 'themselves are callable; extensions may define additional ' - 'callable\n' - 'object types). All argument expressions are evaluated before the ' - 'call\n' - 'is attempted. Please refer to section Function definitions for ' - 'the\n' - 'syntax of formal *parameter* lists.\n' - '\n' - 'If keyword arguments are present, they are first converted to\n' - 'positional arguments, as follows. First, a list of unfilled ' - 'slots is\n' - 'created for the formal parameters. If there are N positional\n' - 'arguments, they are placed in the first N slots. Next, for each\n' - 'keyword argument, the identifier is used to determine the\n' - 'corresponding slot (if the identifier is the same as the first ' - 'formal\n' - 'parameter name, the first slot is used, and so on). If the slot ' - 'is\n' - 'already filled, a "TypeError" exception is raised. Otherwise, ' - 'the\n' - 'value of the argument is placed in the slot, filling it (even if ' - 'the\n' - 'expression is "None", it fills the slot). When all arguments ' - 'have\n' - 'been processed, the slots that are still unfilled are filled with ' - 'the\n' - 'corresponding default value from the function definition. ' - '(Default\n' - 'values are calculated, once, when the function is defined; thus, ' - 'a\n' - 'mutable object such as a list or dictionary used as default value ' - 'will\n' - "be shared by all calls that don't specify an argument value for " - 'the\n' - 'corresponding slot; this should usually be avoided.) If there ' - 'are any\n' - 'unfilled slots for which no default value is specified, a ' - '"TypeError"\n' - 'exception is raised. Otherwise, the list of filled slots is used ' - 'as\n' - 'the argument list for the call.\n' - '\n' - '**CPython implementation detail:** An implementation may provide\n' - 'built-in functions whose positional parameters do not have names, ' - 'even\n' - "if they are 'named' for the purpose of documentation, and which\n" - 'therefore cannot be supplied by keyword. In CPython, this is the ' - 'case\n' - 'for functions implemented in C that use "PyArg_ParseTuple()" to ' - 'parse\n' - 'their arguments.\n' - '\n' - 'If there are more positional arguments than there are formal ' - 'parameter\n' - 'slots, a "TypeError" exception is raised, unless a formal ' - 'parameter\n' - 'using the syntax "*identifier" is present; in this case, that ' - 'formal\n' - 'parameter receives a tuple containing the excess positional ' - 'arguments\n' - '(or an empty tuple if there were no excess positional ' - 'arguments).\n' - '\n' - 'If any keyword argument does not correspond to a formal ' - 'parameter\n' - 'name, a "TypeError" exception is raised, unless a formal ' - 'parameter\n' - 'using the syntax "**identifier" is present; in this case, that ' - 'formal\n' - 'parameter receives a dictionary containing the excess keyword\n' - 'arguments (using the keywords as keys and the argument values as\n' - 'corresponding values), or a (new) empty dictionary if there were ' - 'no\n' - 'excess keyword arguments.\n' - '\n' - 'If the syntax "*expression" appears in the function call, ' - '"expression"\n' - 'must evaluate to an iterable. Elements from this iterable are ' - 'treated\n' - 'as if they were additional positional arguments; if there are\n' - 'positional arguments *x1*, ..., *xN*, and "expression" evaluates ' - 'to a\n' - 'sequence *y1*, ..., *yM*, this is equivalent to a call with M+N\n' - 'positional arguments *x1*, ..., *xN*, *y1*, ..., *yM*.\n' - '\n' - 'A consequence of this is that although the "*expression" syntax ' - 'may\n' - 'appear *after* some keyword arguments, it is processed *before* ' - 'the\n' - 'keyword arguments (and the "**expression" argument, if any -- ' - 'see\n' - 'below). So:\n' - '\n' - ' >>> def f(a, b):\n' - ' ... print a, b\n' - ' ...\n' - ' >>> f(b=1, *(2,))\n' - ' 2 1\n' - ' >>> f(a=1, *(2,))\n' - ' Traceback (most recent call last):\n' - ' File "", line 1, in ?\n' - " TypeError: f() got multiple values for keyword argument 'a'\n" - ' >>> f(1, *(2,))\n' - ' 1 2\n' - '\n' - 'It is unusual for both keyword arguments and the "*expression" ' - 'syntax\n' - 'to be used in the same call, so in practice this confusion does ' - 'not\n' - 'arise.\n' - '\n' - 'If the syntax "**expression" appears in the function call,\n' - '"expression" must evaluate to a mapping, the contents of which ' - 'are\n' - 'treated as additional keyword arguments. In the case of a ' - 'keyword\n' - 'appearing in both "expression" and as an explicit keyword ' - 'argument, a\n' - '"TypeError" exception is raised.\n' - '\n' - 'Formal parameters using the syntax "*identifier" or ' - '"**identifier"\n' - 'cannot be used as positional argument slots or as keyword ' - 'argument\n' - 'names. Formal parameters using the syntax "(sublist)" cannot be ' - 'used\n' - 'as keyword argument names; the outermost sublist corresponds to ' - 'a\n' - 'single unnamed argument slot, and the argument value is assigned ' - 'to\n' - 'the sublist using the usual tuple assignment rules after all ' - 'other\n' - 'parameter processing is done.\n' - '\n' - 'A call always returns some value, possibly "None", unless it ' - 'raises an\n' - 'exception. How this value is computed depends on the type of ' - 'the\n' - 'callable object.\n' - '\n' - 'If it is---\n' - '\n' - 'a user-defined function:\n' - ' The code block for the function is executed, passing it the\n' - ' argument list. The first thing the code block will do is bind ' - 'the\n' - ' formal parameters to the arguments; this is described in ' - 'section\n' - ' Function definitions. When the code block executes a ' - '"return"\n' - ' statement, this specifies the return value of the function ' - 'call.\n' - '\n' - 'a built-in function or method:\n' - ' The result is up to the interpreter; see Built-in Functions ' - 'for the\n' - ' descriptions of built-in functions and methods.\n' - '\n' - 'a class object:\n' - ' A new instance of that class is returned.\n' - '\n' - 'a class instance method:\n' - ' The corresponding user-defined function is called, with an ' - 'argument\n' - ' list that is one longer than the argument list of the call: ' - 'the\n' - ' instance becomes the first argument.\n' - '\n' - 'a class instance:\n' - ' The class must define a "__call__()" method; the effect is ' - 'then the\n' - ' same as if that method was called.\n', - 'class': '\n' - 'Class definitions\n' - '*****************\n' - '\n' - 'A class definition defines a class object (see section The ' - 'standard\n' - 'type hierarchy):\n' - '\n' - ' classdef ::= "class" classname [inheritance] ":" suite\n' - ' inheritance ::= "(" [expression_list] ")"\n' - ' classname ::= identifier\n' - '\n' - 'A class definition is an executable statement. It first ' - 'evaluates the\n' - 'inheritance list, if present. Each item in the inheritance list\n' - 'should evaluate to a class object or class type which allows\n' - "subclassing. The class's suite is then executed in a new " - 'execution\n' - 'frame (see section Naming and binding), using a newly created ' - 'local\n' - 'namespace and the original global namespace. (Usually, the suite\n' - "contains only function definitions.) When the class's suite " - 'finishes\n' - 'execution, its execution frame is discarded but its local ' - 'namespace is\n' - 'saved. [4] A class object is then created using the inheritance ' - 'list\n' - 'for the base classes and the saved local namespace for the ' - 'attribute\n' - 'dictionary. The class name is bound to this class object in the\n' - 'original local namespace.\n' - '\n' - "**Programmer's note:** Variables defined in the class definition " - 'are\n' - 'class variables; they are shared by all instances. To create ' - 'instance\n' - 'variables, they can be set in a method with "self.name = value". ' - 'Both\n' - 'class and instance variables are accessible through the notation\n' - '""self.name"", and an instance variable hides a class variable ' - 'with\n' - 'the same name when accessed in this way. Class variables can be ' - 'used\n' - 'as defaults for instance variables, but using mutable values ' - 'there can\n' - 'lead to unexpected results. For *new-style class*es, descriptors ' - 'can\n' - 'be used to create instance variables with different ' - 'implementation\n' - 'details.\n' - '\n' - 'Class definitions, like function definitions, may be wrapped by ' - 'one or\n' - 'more *decorator* expressions. The evaluation rules for the ' - 'decorator\n' - 'expressions are the same as for functions. The result must be a ' - 'class\n' - 'object, which is then bound to the class name.\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] The exception is propagated to the invocation stack unless\n' - ' there is a "finally" clause which happens to raise another\n' - ' exception. That new exception causes the old one to be lost.\n' - '\n' - '[2] Currently, control "flows off the end" except in the case of\n' - ' an exception or the execution of a "return", "continue", or\n' - ' "break" statement.\n' - '\n' - '[3] A string literal appearing as the first statement in the\n' - ' function body is transformed into the function\'s "__doc__"\n' - " attribute and therefore the function's *docstring*.\n" - '\n' - '[4] A string literal appearing as the first statement in the ' - 'class\n' - ' body is transformed into the namespace\'s "__doc__" item and\n' - " therefore the class's *docstring*.\n", - 'comparisons': '\n' - 'Comparisons\n' - '***********\n' - '\n' - 'Unlike C, all comparison operations in Python have the same ' - 'priority,\n' - 'which is lower than that of any arithmetic, shifting or ' - 'bitwise\n' - 'operation. Also unlike C, expressions like "a < b < c" ' - 'have the\n' - 'interpretation that is conventional in mathematics:\n' - '\n' - ' comparison ::= or_expr ( comp_operator or_expr )*\n' - ' comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "<>" ' - '| "!="\n' - ' | "is" ["not"] | ["not"] "in"\n' - '\n' - 'Comparisons yield boolean values: "True" or "False".\n' - '\n' - 'Comparisons can be chained arbitrarily, e.g., "x < y <= z" ' - 'is\n' - 'equivalent to "x < y and y <= z", except that "y" is ' - 'evaluated only\n' - 'once (but in both cases "z" is not evaluated at all when "x ' - '< y" is\n' - 'found to be false).\n' - '\n' - 'Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions ' - 'and *op1*,\n' - '*op2*, ..., *opN* are comparison operators, then "a op1 b ' - 'op2 c ... y\n' - 'opN z" is equivalent to "a op1 b and b op2 c and ... y opN ' - 'z", except\n' - 'that each expression is evaluated at most once.\n' - '\n' - 'Note that "a op1 b op2 c" doesn\'t imply any kind of ' - 'comparison between\n' - '*a* and *c*, so that, e.g., "x < y > z" is perfectly legal ' - '(though\n' - 'perhaps not pretty).\n' - '\n' - 'The forms "<>" and "!=" are equivalent; for consistency ' - 'with C, "!="\n' - 'is preferred; where "!=" is mentioned below "<>" is also ' - 'accepted.\n' - 'The "<>" spelling is considered obsolescent.\n' - '\n' - 'The operators "<", ">", "==", ">=", "<=", and "!=" compare ' - 'the values\n' - 'of two objects. The objects need not have the same type. ' - 'If both are\n' - 'numbers, they are converted to a common type. Otherwise, ' - 'objects of\n' - 'different types *always* compare unequal, and are ordered ' - 'consistently\n' - 'but arbitrarily. You can control comparison behavior of ' - 'objects of\n' - 'non-built-in types by defining a "__cmp__" method or rich ' - 'comparison\n' - 'methods like "__gt__", described in section Special method ' - 'names.\n' - '\n' - '(This unusual definition of comparison was used to simplify ' - 'the\n' - 'definition of operations like sorting and the "in" and "not ' - 'in"\n' - 'operators. In the future, the comparison rules for objects ' - 'of\n' - 'different types are likely to change.)\n' - '\n' - 'Comparison of objects of the same type depends on the ' - 'type:\n' - '\n' - '* Numbers are compared arithmetically.\n' - '\n' - '* Strings are compared lexicographically using the numeric\n' - ' equivalents (the result of the built-in function "ord()") ' - 'of their\n' - ' characters. Unicode and 8-bit strings are fully ' - 'interoperable in\n' - ' this behavior. [4]\n' - '\n' - '* Tuples and lists are compared lexicographically using ' - 'comparison\n' - ' of corresponding elements. This means that to compare ' - 'equal, each\n' - ' element must compare equal and the two sequences must be ' - 'of the same\n' - ' type and have the same length.\n' - '\n' - ' If not equal, the sequences are ordered the same as their ' - 'first\n' - ' differing elements. For example, "cmp([1,2,x], [1,2,y])" ' - 'returns\n' - ' the same as "cmp(x,y)". If the corresponding element ' - 'does not\n' - ' exist, the shorter sequence is ordered first (for ' - 'example, "[1,2] <\n' - ' [1,2,3]").\n' - '\n' - '* Mappings (dictionaries) compare equal if and only if ' - 'their sorted\n' - ' (key, value) lists compare equal. [5] Outcomes other than ' - 'equality\n' - ' are resolved consistently, but are not otherwise defined. ' - '[6]\n' - '\n' - '* Most other objects of built-in types compare unequal ' - 'unless they\n' - ' are the same object; the choice whether one object is ' - 'considered\n' - ' smaller or larger than another one is made arbitrarily ' - 'but\n' - ' consistently within one execution of a program.\n' - '\n' - 'The operators "in" and "not in" test for collection ' - 'membership. "x in\n' - 's" evaluates to true if *x* is a member of the collection ' - '*s*, and\n' - 'false otherwise. "x not in s" returns the negation of "x ' - 'in s". The\n' - 'collection membership test has traditionally been bound to ' - 'sequences;\n' - 'an object is a member of a collection if the collection is ' - 'a sequence\n' - 'and contains an element equal to that object. However, it ' - 'make sense\n' - 'for many other object types to support membership tests ' - 'without being\n' - 'a sequence. In particular, dictionaries (for keys) and ' - 'sets support\n' - 'membership testing.\n' - '\n' - 'For the list and tuple types, "x in y" is true if and only ' - 'if there\n' - 'exists an index *i* such that "x == y[i]" is true.\n' - '\n' - 'For the Unicode and string types, "x in y" is true if and ' - 'only if *x*\n' - 'is a substring of *y*. An equivalent test is "y.find(x) != ' - '-1".\n' - 'Note, *x* and *y* need not be the same type; consequently, ' - '"u\'ab\' in\n' - '\'abc\'" will return "True". Empty strings are always ' - 'considered to be a\n' - 'substring of any other string, so """ in "abc"" will return ' - '"True".\n' - '\n' - 'Changed in version 2.3: Previously, *x* was required to be ' - 'a string of\n' - 'length "1".\n' - '\n' - 'For user-defined classes which define the "__contains__()" ' - 'method, "x\n' - 'in y" is true if and only if "y.__contains__(x)" is true.\n' - '\n' - 'For user-defined classes which do not define ' - '"__contains__()" but do\n' - 'define "__iter__()", "x in y" is true if some value "z" ' - 'with "x == z"\n' - 'is produced while iterating over "y". If an exception is ' - 'raised\n' - 'during the iteration, it is as if "in" raised that ' - 'exception.\n' - '\n' - 'Lastly, the old-style iteration protocol is tried: if a ' - 'class defines\n' - '"__getitem__()", "x in y" is true if and only if there is a ' - 'non-\n' - 'negative integer index *i* such that "x == y[i]", and all ' - 'lower\n' - 'integer indices do not raise "IndexError" exception. (If ' - 'any other\n' - 'exception is raised, it is as if "in" raised that ' - 'exception).\n' - '\n' - 'The operator "not in" is defined to have the inverse true ' - 'value of\n' - '"in".\n' - '\n' - 'The operators "is" and "is not" test for object identity: ' - '"x is y" is\n' - 'true if and only if *x* and *y* are the same object. "x is ' - 'not y"\n' - 'yields the inverse truth value. [7]\n', - 'compound': '\n' - 'Compound statements\n' - '*******************\n' - '\n' - 'Compound statements contain (groups of) other statements; they ' - 'affect\n' - 'or control the execution of those other statements in some ' - 'way. In\n' - 'general, compound statements span multiple lines, although in ' - 'simple\n' - 'incarnations a whole compound statement may be contained in ' - 'one line.\n' - '\n' - 'The "if", "while" and "for" statements implement traditional ' - 'control\n' - 'flow constructs. "try" specifies exception handlers and/or ' - 'cleanup\n' - 'code for a group of statements. Function and class ' - 'definitions are\n' - 'also syntactically compound statements.\n' - '\n' - "Compound statements consist of one or more 'clauses.' A " - 'clause\n' - "consists of a header and a 'suite.' The clause headers of a\n" - 'particular compound statement are all at the same indentation ' - 'level.\n' - 'Each clause header begins with a uniquely identifying keyword ' - 'and ends\n' - 'with a colon. A suite is a group of statements controlled by ' - 'a\n' - 'clause. A suite can be one or more semicolon-separated ' - 'simple\n' - 'statements on the same line as the header, following the ' - "header's\n" - 'colon, or it can be one or more indented statements on ' - 'subsequent\n' - 'lines. Only the latter form of suite can contain nested ' - 'compound\n' - 'statements; the following is illegal, mostly because it ' - "wouldn't be\n" - 'clear to which "if" clause a following "else" clause would ' - 'belong:\n' - '\n' - ' if test1: if test2: print x\n' - '\n' - 'Also note that the semicolon binds tighter than the colon in ' - 'this\n' - 'context, so that in the following example, either all or none ' - 'of the\n' - '"print" statements are executed:\n' - '\n' - ' if x < y < z: print x; print y; print z\n' - '\n' - 'Summarizing:\n' - '\n' - ' compound_stmt ::= if_stmt\n' - ' | while_stmt\n' - ' | for_stmt\n' - ' | try_stmt\n' - ' | with_stmt\n' - ' | funcdef\n' - ' | classdef\n' - ' | decorated\n' - ' suite ::= stmt_list NEWLINE | NEWLINE INDENT ' - 'statement+ DEDENT\n' - ' statement ::= stmt_list NEWLINE | compound_stmt\n' - ' stmt_list ::= simple_stmt (";" simple_stmt)* [";"]\n' - '\n' - 'Note that statements always end in a "NEWLINE" possibly ' - 'followed by a\n' - '"DEDENT". Also note that optional continuation clauses always ' - 'begin\n' - 'with a keyword that cannot start a statement, thus there are ' - 'no\n' - 'ambiguities (the \'dangling "else"\' problem is solved in ' - 'Python by\n' - 'requiring nested "if" statements to be indented).\n' - '\n' - 'The formatting of the grammar rules in the following sections ' - 'places\n' - 'each clause on a separate line for clarity.\n' - '\n' - '\n' - 'The "if" statement\n' - '==================\n' - '\n' - 'The "if" statement is used for conditional execution:\n' - '\n' - ' if_stmt ::= "if" expression ":" suite\n' - ' ( "elif" expression ":" suite )*\n' - ' ["else" ":" suite]\n' - '\n' - 'It selects exactly one of the suites by evaluating the ' - 'expressions one\n' - 'by one until one is found to be true (see section Boolean ' - 'operations\n' - 'for the definition of true and false); then that suite is ' - 'executed\n' - '(and no other part of the "if" statement is executed or ' - 'evaluated).\n' - 'If all expressions are false, the suite of the "else" clause, ' - 'if\n' - 'present, is executed.\n' - '\n' - '\n' - 'The "while" statement\n' - '=====================\n' - '\n' - 'The "while" statement is used for repeated execution as long ' - 'as an\n' - 'expression is true:\n' - '\n' - ' while_stmt ::= "while" expression ":" suite\n' - ' ["else" ":" suite]\n' - '\n' - 'This repeatedly tests the expression and, if it is true, ' - 'executes the\n' - 'first suite; if the expression is false (which may be the ' - 'first time\n' - 'it is tested) the suite of the "else" clause, if present, is ' - 'executed\n' - 'and the loop terminates.\n' - '\n' - 'A "break" statement executed in the first suite terminates the ' - 'loop\n' - 'without executing the "else" clause\'s suite. A "continue" ' - 'statement\n' - 'executed in the first suite skips the rest of the suite and ' - 'goes back\n' - 'to testing the expression.\n' - '\n' - '\n' - 'The "for" statement\n' - '===================\n' - '\n' - 'The "for" statement is used to iterate over the elements of a ' - 'sequence\n' - '(such as a string, tuple or list) or other iterable object:\n' - '\n' - ' for_stmt ::= "for" target_list "in" expression_list ":" ' - 'suite\n' - ' ["else" ":" suite]\n' - '\n' - 'The expression list is evaluated once; it should yield an ' - 'iterable\n' - 'object. An iterator is created for the result of the\n' - '"expression_list". The suite is then executed once for each ' - 'item\n' - 'provided by the iterator, in the order of ascending indices. ' - 'Each\n' - 'item in turn is assigned to the target list using the standard ' - 'rules\n' - 'for assignments, and then the suite is executed. When the ' - 'items are\n' - 'exhausted (which is immediately when the sequence is empty), ' - 'the suite\n' - 'in the "else" clause, if present, is executed, and the loop\n' - 'terminates.\n' - '\n' - 'A "break" statement executed in the first suite terminates the ' - 'loop\n' - 'without executing the "else" clause\'s suite. A "continue" ' - 'statement\n' - 'executed in the first suite skips the rest of the suite and ' - 'continues\n' - 'with the next item, or with the "else" clause if there was no ' - 'next\n' - 'item.\n' - '\n' - 'The suite may assign to the variable(s) in the target list; ' - 'this does\n' - 'not affect the next item assigned to it.\n' - '\n' - 'The target list is not deleted when the loop is finished, but ' - 'if the\n' - 'sequence is empty, it will not have been assigned to at all by ' - 'the\n' - 'loop. Hint: the built-in function "range()" returns a ' - 'sequence of\n' - 'integers suitable to emulate the effect of Pascal\'s "for i := ' - 'a to b\n' - 'do"; e.g., "range(3)" returns the list "[0, 1, 2]".\n' - '\n' - 'Note: There is a subtlety when the sequence is being modified ' - 'by the\n' - ' loop (this can only occur for mutable sequences, i.e. ' - 'lists). An\n' - ' internal counter is used to keep track of which item is used ' - 'next,\n' - ' and this is incremented on each iteration. When this ' - 'counter has\n' - ' reached the length of the sequence the loop terminates. ' - 'This means\n' - ' that if the suite deletes the current (or a previous) item ' - 'from the\n' - ' sequence, the next item will be skipped (since it gets the ' - 'index of\n' - ' the current item which has already been treated). Likewise, ' - 'if the\n' - ' suite inserts an item in the sequence before the current ' - 'item, the\n' - ' current item will be treated again the next time through the ' - 'loop.\n' - ' This can lead to nasty bugs that can be avoided by making a\n' - ' temporary copy using a slice of the whole sequence, e.g.,\n' - '\n' - ' for x in a[:]:\n' - ' if x < 0: a.remove(x)\n' - '\n' - '\n' - 'The "try" statement\n' - '===================\n' - '\n' - 'The "try" statement specifies exception handlers and/or ' - 'cleanup code\n' - 'for a group of statements:\n' - '\n' - ' try_stmt ::= try1_stmt | try2_stmt\n' - ' try1_stmt ::= "try" ":" suite\n' - ' ("except" [expression [("as" | ",") ' - 'identifier]] ":" suite)+\n' - ' ["else" ":" suite]\n' - ' ["finally" ":" suite]\n' - ' try2_stmt ::= "try" ":" suite\n' - ' "finally" ":" suite\n' - '\n' - 'Changed in version 2.5: In previous versions of Python,\n' - '"try"..."except"..."finally" did not work. "try"..."except" ' - 'had to be\n' - 'nested in "try"..."finally".\n' - '\n' - 'The "except" clause(s) specify one or more exception handlers. ' - 'When no\n' - 'exception occurs in the "try" clause, no exception handler is\n' - 'executed. When an exception occurs in the "try" suite, a ' - 'search for an\n' - 'exception handler is started. This search inspects the except ' - 'clauses\n' - 'in turn until one is found that matches the exception. An ' - 'expression-\n' - 'less except clause, if present, must be last; it matches any\n' - 'exception. For an except clause with an expression, that ' - 'expression\n' - 'is evaluated, and the clause matches the exception if the ' - 'resulting\n' - 'object is "compatible" with the exception. An object is ' - 'compatible\n' - 'with an exception if it is the class or a base class of the ' - 'exception\n' - 'object, or a tuple containing an item compatible with the ' - 'exception.\n' - '\n' - 'If no except clause matches the exception, the search for an ' - 'exception\n' - 'handler continues in the surrounding code and on the ' - 'invocation stack.\n' - '[1]\n' - '\n' - 'If the evaluation of an expression in the header of an except ' - 'clause\n' - 'raises an exception, the original search for a handler is ' - 'canceled and\n' - 'a search starts for the new exception in the surrounding code ' - 'and on\n' - 'the call stack (it is treated as if the entire "try" statement ' - 'raised\n' - 'the exception).\n' - '\n' - 'When a matching except clause is found, the exception is ' - 'assigned to\n' - 'the target specified in that except clause, if present, and ' - 'the except\n' - "clause's suite is executed. All except clauses must have an\n" - 'executable block. When the end of this block is reached, ' - 'execution\n' - 'continues normally after the entire try statement. (This ' - 'means that\n' - 'if two nested handlers exist for the same exception, and the ' - 'exception\n' - 'occurs in the try clause of the inner handler, the outer ' - 'handler will\n' - 'not handle the exception.)\n' - '\n' - "Before an except clause's suite is executed, details about " - 'the\n' - 'exception are assigned to three variables in the "sys" ' - 'module:\n' - '"sys.exc_type" receives the object identifying the exception;\n' - '"sys.exc_value" receives the exception\'s parameter;\n' - '"sys.exc_traceback" receives a traceback object (see section ' - 'The\n' - 'standard type hierarchy) identifying the point in the program ' - 'where\n' - 'the exception occurred. These details are also available ' - 'through the\n' - '"sys.exc_info()" function, which returns a tuple "(exc_type,\n' - 'exc_value, exc_traceback)". Use of the corresponding ' - 'variables is\n' - 'deprecated in favor of this function, since their use is ' - 'unsafe in a\n' - 'threaded program. As of Python 1.5, the variables are ' - 'restored to\n' - 'their previous values (before the call) when returning from a ' - 'function\n' - 'that handled an exception.\n' - '\n' - 'The optional "else" clause is executed if and when control ' - 'flows off\n' - 'the end of the "try" clause. [2] Exceptions in the "else" ' - 'clause are\n' - 'not handled by the preceding "except" clauses.\n' - '\n' - 'If "finally" is present, it specifies a \'cleanup\' handler. ' - 'The "try"\n' - 'clause is executed, including any "except" and "else" ' - 'clauses. If an\n' - 'exception occurs in any of the clauses and is not handled, ' - 'the\n' - 'exception is temporarily saved. The "finally" clause is ' - 'executed. If\n' - 'there is a saved exception, it is re-raised at the end of the\n' - '"finally" clause. If the "finally" clause raises another ' - 'exception or\n' - 'executes a "return" or "break" statement, the saved exception ' - 'is\n' - 'discarded:\n' - '\n' - ' >>> def f():\n' - ' ... try:\n' - ' ... 1/0\n' - ' ... finally:\n' - ' ... return 42\n' - ' ...\n' - ' >>> f()\n' - ' 42\n' - '\n' - 'The exception information is not available to the program ' - 'during\n' - 'execution of the "finally" clause.\n' - '\n' - 'When a "return", "break" or "continue" statement is executed ' - 'in the\n' - '"try" suite of a "try"..."finally" statement, the "finally" ' - 'clause is\n' - 'also executed \'on the way out.\' A "continue" statement is ' - 'illegal in\n' - 'the "finally" clause. (The reason is a problem with the ' - 'current\n' - 'implementation --- this restriction may be lifted in the ' - 'future).\n' - '\n' - 'The return value of a function is determined by the last ' - '"return"\n' - 'statement executed. Since the "finally" clause always ' - 'executes, a\n' - '"return" statement executed in the "finally" clause will ' - 'always be the\n' - 'last one executed:\n' - '\n' - ' >>> def foo():\n' - ' ... try:\n' - " ... return 'try'\n" - ' ... finally:\n' - " ... return 'finally'\n" - ' ...\n' - ' >>> foo()\n' - " 'finally'\n" - '\n' - 'Additional information on exceptions can be found in section\n' - 'Exceptions, and information on using the "raise" statement to ' - 'generate\n' - 'exceptions may be found in section The raise statement.\n' - '\n' - '\n' - 'The "with" statement\n' - '====================\n' - '\n' - 'New in version 2.5.\n' - '\n' - 'The "with" statement is used to wrap the execution of a block ' - 'with\n' - 'methods defined by a context manager (see section With ' - 'Statement\n' - 'Context Managers). This allows common ' - '"try"..."except"..."finally"\n' - 'usage patterns to be encapsulated for convenient reuse.\n' - '\n' - ' with_stmt ::= "with" with_item ("," with_item)* ":" suite\n' - ' with_item ::= expression ["as" target]\n' - '\n' - 'The execution of the "with" statement with one "item" proceeds ' - 'as\n' - 'follows:\n' - '\n' - '1. The context expression (the expression given in the ' - '"with_item")\n' - ' is evaluated to obtain a context manager.\n' - '\n' - '2. The context manager\'s "__exit__()" is loaded for later ' - 'use.\n' - '\n' - '3. The context manager\'s "__enter__()" method is invoked.\n' - '\n' - '4. If a target was included in the "with" statement, the ' - 'return\n' - ' value from "__enter__()" is assigned to it.\n' - '\n' - ' Note: The "with" statement guarantees that if the ' - '"__enter__()"\n' - ' method returns without an error, then "__exit__()" will ' - 'always be\n' - ' called. Thus, if an error occurs during the assignment to ' - 'the\n' - ' target list, it will be treated the same as an error ' - 'occurring\n' - ' within the suite would be. See step 6 below.\n' - '\n' - '5. The suite is executed.\n' - '\n' - '6. The context manager\'s "__exit__()" method is invoked. If ' - 'an\n' - ' exception caused the suite to be exited, its type, value, ' - 'and\n' - ' traceback are passed as arguments to "__exit__()". ' - 'Otherwise, three\n' - ' "None" arguments are supplied.\n' - '\n' - ' If the suite was exited due to an exception, and the return ' - 'value\n' - ' from the "__exit__()" method was false, the exception is ' - 'reraised.\n' - ' If the return value was true, the exception is suppressed, ' - 'and\n' - ' execution continues with the statement following the ' - '"with"\n' - ' statement.\n' - '\n' - ' If the suite was exited for any reason other than an ' - 'exception, the\n' - ' return value from "__exit__()" is ignored, and execution ' - 'proceeds\n' - ' at the normal location for the kind of exit that was ' - 'taken.\n' - '\n' - 'With more than one item, the context managers are processed as ' - 'if\n' - 'multiple "with" statements were nested:\n' - '\n' - ' with A() as a, B() as b:\n' - ' suite\n' - '\n' - 'is equivalent to\n' - '\n' - ' with A() as a:\n' - ' with B() as b:\n' - ' suite\n' - '\n' - 'Note: In Python 2.5, the "with" statement is only allowed when ' - 'the\n' - ' "with_statement" feature has been enabled. It is always ' - 'enabled in\n' - ' Python 2.6.\n' - '\n' - 'Changed in version 2.7: Support for multiple context ' - 'expressions.\n' - '\n' - 'See also: **PEP 0343** - The "with" statement\n' - '\n' - ' The specification, background, and examples for the ' - 'Python "with"\n' - ' statement.\n' - '\n' - '\n' - 'Function definitions\n' - '====================\n' - '\n' - 'A function definition defines a user-defined function object ' - '(see\n' - 'section The standard type hierarchy):\n' - '\n' - ' decorated ::= decorators (classdef | funcdef)\n' - ' decorators ::= decorator+\n' - ' decorator ::= "@" dotted_name ["(" [argument_list ' - '[","]] ")"] NEWLINE\n' - ' funcdef ::= "def" funcname "(" [parameter_list] ")" ' - '":" suite\n' - ' dotted_name ::= identifier ("." identifier)*\n' - ' parameter_list ::= (defparameter ",")*\n' - ' ( "*" identifier ["," "**" identifier]\n' - ' | "**" identifier\n' - ' | defparameter [","] )\n' - ' defparameter ::= parameter ["=" expression]\n' - ' sublist ::= parameter ("," parameter)* [","]\n' - ' parameter ::= identifier | "(" sublist ")"\n' - ' funcname ::= identifier\n' - '\n' - 'A function definition is an executable statement. Its ' - 'execution binds\n' - 'the function name in the current local namespace to a function ' - 'object\n' - '(a wrapper around the executable code for the function). ' - 'This\n' - 'function object contains a reference to the current global ' - 'namespace\n' - 'as the global namespace to be used when the function is ' - 'called.\n' - '\n' - 'The function definition does not execute the function body; ' - 'this gets\n' - 'executed only when the function is called. [3]\n' - '\n' - 'A function definition may be wrapped by one or more ' - '*decorator*\n' - 'expressions. Decorator expressions are evaluated when the ' - 'function is\n' - 'defined, in the scope that contains the function definition. ' - 'The\n' - 'result must be a callable, which is invoked with the function ' - 'object\n' - 'as the only argument. The returned value is bound to the ' - 'function name\n' - 'instead of the function object. Multiple decorators are ' - 'applied in\n' - 'nested fashion. For example, the following code:\n' - '\n' - ' @f1(arg)\n' - ' @f2\n' - ' def func(): pass\n' - '\n' - 'is equivalent to:\n' - '\n' - ' def func(): pass\n' - ' func = f1(arg)(f2(func))\n' - '\n' - 'When one or more top-level *parameters* have the form ' - '*parameter* "="\n' - '*expression*, the function is said to have "default parameter ' - 'values."\n' - 'For a parameter with a default value, the corresponding ' - '*argument* may\n' - "be omitted from a call, in which case the parameter's default " - 'value is\n' - 'substituted. If a parameter has a default value, all ' - 'following\n' - 'parameters must also have a default value --- this is a ' - 'syntactic\n' - 'restriction that is not expressed by the grammar.\n' - '\n' - '**Default parameter values are evaluated when the function ' - 'definition\n' - 'is executed.** This means that the expression is evaluated ' - 'once, when\n' - 'the function is defined, and that the same "pre-computed" ' - 'value is\n' - 'used for each call. This is especially important to ' - 'understand when a\n' - 'default parameter is a mutable object, such as a list or a ' - 'dictionary:\n' - 'if the function modifies the object (e.g. by appending an item ' - 'to a\n' - 'list), the default value is in effect modified. This is ' - 'generally not\n' - 'what was intended. A way around this is to use "None" as ' - 'the\n' - 'default, and explicitly test for it in the body of the ' - 'function, e.g.:\n' - '\n' - ' def whats_on_the_telly(penguin=None):\n' - ' if penguin is None:\n' - ' penguin = []\n' - ' penguin.append("property of the zoo")\n' - ' return penguin\n' - '\n' - 'Function call semantics are described in more detail in ' - 'section Calls.\n' - 'A function call always assigns values to all parameters ' - 'mentioned in\n' - 'the parameter list, either from position arguments, from ' - 'keyword\n' - 'arguments, or from default values. If the form ' - '""*identifier"" is\n' - 'present, it is initialized to a tuple receiving any excess ' - 'positional\n' - 'parameters, defaulting to the empty tuple. If the form\n' - '""**identifier"" is present, it is initialized to a new ' - 'dictionary\n' - 'receiving any excess keyword arguments, defaulting to a new ' - 'empty\n' - 'dictionary.\n' - '\n' - 'It is also possible to create anonymous functions (functions ' - 'not bound\n' - 'to a name), for immediate use in expressions. This uses ' - 'lambda\n' - 'expressions, described in section Lambdas. Note that the ' - 'lambda\n' - 'expression is merely a shorthand for a simplified function ' - 'definition;\n' - 'a function defined in a ""def"" statement can be passed around ' - 'or\n' - 'assigned to another name just like a function defined by a ' - 'lambda\n' - 'expression. The ""def"" form is actually more powerful since ' - 'it\n' - 'allows the execution of multiple statements.\n' - '\n' - "**Programmer's note:** Functions are first-class objects. A " - '""def""\n' - 'form executed inside a function definition defines a local ' - 'function\n' - 'that can be returned or passed around. Free variables used in ' - 'the\n' - 'nested function can access the local variables of the ' - 'function\n' - 'containing the def. See section Naming and binding for ' - 'details.\n' - '\n' - '\n' - 'Class definitions\n' - '=================\n' - '\n' - 'A class definition defines a class object (see section The ' - 'standard\n' - 'type hierarchy):\n' - '\n' - ' classdef ::= "class" classname [inheritance] ":" suite\n' - ' inheritance ::= "(" [expression_list] ")"\n' - ' classname ::= identifier\n' - '\n' - 'A class definition is an executable statement. It first ' - 'evaluates the\n' - 'inheritance list, if present. Each item in the inheritance ' - 'list\n' - 'should evaluate to a class object or class type which allows\n' - "subclassing. The class's suite is then executed in a new " - 'execution\n' - 'frame (see section Naming and binding), using a newly created ' - 'local\n' - 'namespace and the original global namespace. (Usually, the ' - 'suite\n' - "contains only function definitions.) When the class's suite " - 'finishes\n' - 'execution, its execution frame is discarded but its local ' - 'namespace is\n' - 'saved. [4] A class object is then created using the ' - 'inheritance list\n' - 'for the base classes and the saved local namespace for the ' - 'attribute\n' - 'dictionary. The class name is bound to this class object in ' - 'the\n' - 'original local namespace.\n' - '\n' - "**Programmer's note:** Variables defined in the class " - 'definition are\n' - 'class variables; they are shared by all instances. To create ' - 'instance\n' - 'variables, they can be set in a method with "self.name = ' - 'value". Both\n' - 'class and instance variables are accessible through the ' - 'notation\n' - '""self.name"", and an instance variable hides a class variable ' - 'with\n' - 'the same name when accessed in this way. Class variables can ' - 'be used\n' - 'as defaults for instance variables, but using mutable values ' - 'there can\n' - 'lead to unexpected results. For *new-style class*es, ' - 'descriptors can\n' - 'be used to create instance variables with different ' - 'implementation\n' - 'details.\n' - '\n' - 'Class definitions, like function definitions, may be wrapped ' - 'by one or\n' - 'more *decorator* expressions. The evaluation rules for the ' - 'decorator\n' - 'expressions are the same as for functions. The result must be ' - 'a class\n' - 'object, which is then bound to the class name.\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] The exception is propagated to the invocation stack ' - 'unless\n' - ' there is a "finally" clause which happens to raise ' - 'another\n' - ' exception. That new exception causes the old one to be ' - 'lost.\n' - '\n' - '[2] Currently, control "flows off the end" except in the case ' - 'of\n' - ' an exception or the execution of a "return", "continue", ' - 'or\n' - ' "break" statement.\n' - '\n' - '[3] A string literal appearing as the first statement in the\n' - " function body is transformed into the function's " - '"__doc__"\n' - " attribute and therefore the function's *docstring*.\n" - '\n' - '[4] A string literal appearing as the first statement in the ' - 'class\n' - ' body is transformed into the namespace\'s "__doc__" item ' - 'and\n' - " therefore the class's *docstring*.\n", - 'context-managers': '\n' - 'With Statement Context Managers\n' - '*******************************\n' - '\n' - 'New in version 2.5.\n' - '\n' - 'A *context manager* is an object that defines the ' - 'runtime context to\n' - 'be established when executing a "with" statement. The ' - 'context manager\n' - 'handles the entry into, and the exit from, the desired ' - 'runtime context\n' - 'for the execution of the block of code. Context ' - 'managers are normally\n' - 'invoked using the "with" statement (described in ' - 'section The with\n' - 'statement), but can also be used by directly invoking ' - 'their methods.\n' - '\n' - 'Typical uses of context managers include saving and ' - 'restoring various\n' - 'kinds of global state, locking and unlocking ' - 'resources, closing opened\n' - 'files, etc.\n' - '\n' - 'For more information on context managers, see Context ' - 'Manager Types.\n' - '\n' - 'object.__enter__(self)\n' - '\n' - ' Enter the runtime context related to this object. ' - 'The "with"\n' - " statement will bind this method's return value to " - 'the target(s)\n' - ' specified in the "as" clause of the statement, if ' - 'any.\n' - '\n' - 'object.__exit__(self, exc_type, exc_value, traceback)\n' - '\n' - ' Exit the runtime context related to this object. ' - 'The parameters\n' - ' describe the exception that caused the context to ' - 'be exited. If the\n' - ' context was exited without an exception, all three ' - 'arguments will\n' - ' be "None".\n' - '\n' - ' If an exception is supplied, and the method wishes ' - 'to suppress the\n' - ' exception (i.e., prevent it from being propagated), ' - 'it should\n' - ' return a true value. Otherwise, the exception will ' - 'be processed\n' - ' normally upon exit from this method.\n' - '\n' - ' Note that "__exit__()" methods should not reraise ' - 'the passed-in\n' - " exception; this is the caller's responsibility.\n" - '\n' - 'See also: **PEP 0343** - The "with" statement\n' - '\n' - ' The specification, background, and examples for ' - 'the Python "with"\n' - ' statement.\n', - 'continue': '\n' - 'The "continue" statement\n' - '************************\n' - '\n' - ' continue_stmt ::= "continue"\n' - '\n' - '"continue" may only occur syntactically nested in a "for" or ' - '"while"\n' - 'loop, but not nested in a function or class definition or ' - '"finally"\n' - 'clause within that loop. It continues with the next cycle of ' - 'the\n' - 'nearest enclosing loop.\n' - '\n' - 'When "continue" passes control out of a "try" statement with ' - 'a\n' - '"finally" clause, that "finally" clause is executed before ' - 'really\n' - 'starting the next loop cycle.\n', - 'conversions': '\n' - 'Arithmetic conversions\n' - '**********************\n' - '\n' - 'When a description of an arithmetic operator below uses the ' - 'phrase\n' - '"the numeric arguments are converted to a common type," the ' - 'arguments\n' - 'are coerced using the coercion rules listed at Coercion ' - 'rules. If\n' - 'both arguments are standard numeric types, the following ' - 'coercions are\n' - 'applied:\n' - '\n' - '* If either argument is a complex number, the other is ' - 'converted to\n' - ' complex;\n' - '\n' - '* otherwise, if either argument is a floating point number, ' - 'the\n' - ' other is converted to floating point;\n' - '\n' - '* otherwise, if either argument is a long integer, the ' - 'other is\n' - ' converted to long integer;\n' - '\n' - '* otherwise, both must be plain integers and no conversion ' - 'is\n' - ' necessary.\n' - '\n' - 'Some additional rules apply for certain operators (e.g., a ' - 'string left\n' - "argument to the '%' operator). Extensions can define their " - 'own\n' - 'coercions.\n', - 'customization': '\n' - 'Basic customization\n' - '*******************\n' - '\n' - 'object.__new__(cls[, ...])\n' - '\n' - ' Called to create a new instance of class *cls*. ' - '"__new__()" is a\n' - ' static method (special-cased so you need not declare ' - 'it as such)\n' - ' that takes the class of which an instance was ' - 'requested as its\n' - ' first argument. The remaining arguments are those ' - 'passed to the\n' - ' object constructor expression (the call to the ' - 'class). The return\n' - ' value of "__new__()" should be the new object instance ' - '(usually an\n' - ' instance of *cls*).\n' - '\n' - ' Typical implementations create a new instance of the ' - 'class by\n' - ' invoking the superclass\'s "__new__()" method using\n' - ' "super(currentclass, cls).__new__(cls[, ...])" with ' - 'appropriate\n' - ' arguments and then modifying the newly-created ' - 'instance as\n' - ' necessary before returning it.\n' - '\n' - ' If "__new__()" returns an instance of *cls*, then the ' - 'new\n' - ' instance\'s "__init__()" method will be invoked like\n' - ' "__init__(self[, ...])", where *self* is the new ' - 'instance and the\n' - ' remaining arguments are the same as were passed to ' - '"__new__()".\n' - '\n' - ' If "__new__()" does not return an instance of *cls*, ' - 'then the new\n' - ' instance\'s "__init__()" method will not be invoked.\n' - '\n' - ' "__new__()" is intended mainly to allow subclasses of ' - 'immutable\n' - ' types (like int, str, or tuple) to customize instance ' - 'creation. It\n' - ' is also commonly overridden in custom metaclasses in ' - 'order to\n' - ' customize class creation.\n' - '\n' - 'object.__init__(self[, ...])\n' - '\n' - ' Called after the instance has been created (by ' - '"__new__()"), but\n' - ' before it is returned to the caller. The arguments ' - 'are those\n' - ' passed to the class constructor expression. If a base ' - 'class has an\n' - ' "__init__()" method, the derived class\'s "__init__()" ' - 'method, if\n' - ' any, must explicitly call it to ensure proper ' - 'initialization of the\n' - ' base class part of the instance; for example:\n' - ' "BaseClass.__init__(self, [args...])".\n' - '\n' - ' Because "__new__()" and "__init__()" work together in ' - 'constructing\n' - ' objects ("__new__()" to create it, and "__init__()" to ' - 'customise\n' - ' it), no non-"None" value may be returned by ' - '"__init__()"; doing so\n' - ' will cause a "TypeError" to be raised at runtime.\n' - '\n' - 'object.__del__(self)\n' - '\n' - ' Called when the instance is about to be destroyed. ' - 'This is also\n' - ' called a destructor. If a base class has a ' - '"__del__()" method, the\n' - ' derived class\'s "__del__()" method, if any, must ' - 'explicitly call it\n' - ' to ensure proper deletion of the base class part of ' - 'the instance.\n' - ' Note that it is possible (though not recommended!) for ' - 'the\n' - ' "__del__()" method to postpone destruction of the ' - 'instance by\n' - ' creating a new reference to it. It may then be called ' - 'at a later\n' - ' time when this new reference is deleted. It is not ' - 'guaranteed that\n' - ' "__del__()" methods are called for objects that still ' - 'exist when\n' - ' the interpreter exits.\n' - '\n' - ' Note: "del x" doesn\'t directly call "x.__del__()" --- ' - 'the former\n' - ' decrements the reference count for "x" by one, and ' - 'the latter is\n' - ' only called when "x"\'s reference count reaches ' - 'zero. Some common\n' - ' situations that may prevent the reference count of ' - 'an object from\n' - ' going to zero include: circular references between ' - 'objects (e.g.,\n' - ' a doubly-linked list or a tree data structure with ' - 'parent and\n' - ' child pointers); a reference to the object on the ' - 'stack frame of\n' - ' a function that caught an exception (the traceback ' - 'stored in\n' - ' "sys.exc_traceback" keeps the stack frame alive); or ' - 'a reference\n' - ' to the object on the stack frame that raised an ' - 'unhandled\n' - ' exception in interactive mode (the traceback stored ' - 'in\n' - ' "sys.last_traceback" keeps the stack frame alive). ' - 'The first\n' - ' situation can only be remedied by explicitly ' - 'breaking the cycles;\n' - ' the latter two situations can be resolved by storing ' - '"None" in\n' - ' "sys.exc_traceback" or "sys.last_traceback". ' - 'Circular references\n' - ' which are garbage are detected when the option cycle ' - 'detector is\n' - " enabled (it's on by default), but can only be " - 'cleaned up if there\n' - ' are no Python-level "__del__()" methods involved. ' - 'Refer to the\n' - ' documentation for the "gc" module for more ' - 'information about how\n' - ' "__del__()" methods are handled by the cycle ' - 'detector,\n' - ' particularly the description of the "garbage" ' - 'value.\n' - '\n' - ' Warning: Due to the precarious circumstances under ' - 'which\n' - ' "__del__()" methods are invoked, exceptions that ' - 'occur during\n' - ' their execution are ignored, and a warning is ' - 'printed to\n' - ' "sys.stderr" instead. Also, when "__del__()" is ' - 'invoked in\n' - ' response to a module being deleted (e.g., when ' - 'execution of the\n' - ' program is done), other globals referenced by the ' - '"__del__()"\n' - ' method may already have been deleted or in the ' - 'process of being\n' - ' torn down (e.g. the import machinery shutting ' - 'down). For this\n' - ' reason, "__del__()" methods should do the absolute ' - 'minimum needed\n' - ' to maintain external invariants. Starting with ' - 'version 1.5,\n' - ' Python guarantees that globals whose name begins ' - 'with a single\n' - ' underscore are deleted from their module before ' - 'other globals are\n' - ' deleted; if no other references to such globals ' - 'exist, this may\n' - ' help in assuring that imported modules are still ' - 'available at the\n' - ' time when the "__del__()" method is called.\n' - '\n' - ' See also the "-R" command-line option.\n' - '\n' - 'object.__repr__(self)\n' - '\n' - ' Called by the "repr()" built-in function and by string ' - 'conversions\n' - ' (reverse quotes) to compute the "official" string ' - 'representation of\n' - ' an object. If at all possible, this should look like ' - 'a valid\n' - ' Python expression that could be used to recreate an ' - 'object with the\n' - ' same value (given an appropriate environment). If ' - 'this is not\n' - ' possible, a string of the form "<...some useful ' - 'description...>"\n' - ' should be returned. The return value must be a string ' - 'object. If a\n' - ' class defines "__repr__()" but not "__str__()", then ' - '"__repr__()"\n' - ' is also used when an "informal" string representation ' - 'of instances\n' - ' of that class is required.\n' - '\n' - ' This is typically used for debugging, so it is ' - 'important that the\n' - ' representation is information-rich and unambiguous.\n' - '\n' - 'object.__str__(self)\n' - '\n' - ' Called by the "str()" built-in function and by the ' - '"print"\n' - ' statement to compute the "informal" string ' - 'representation of an\n' - ' object. This differs from "__repr__()" in that it ' - 'does not have to\n' - ' be a valid Python expression: a more convenient or ' - 'concise\n' - ' representation may be used instead. The return value ' - 'must be a\n' - ' string object.\n' - '\n' - 'object.__lt__(self, other)\n' - 'object.__le__(self, other)\n' - 'object.__eq__(self, other)\n' - 'object.__ne__(self, other)\n' - 'object.__gt__(self, other)\n' - 'object.__ge__(self, other)\n' - '\n' - ' New in version 2.1.\n' - '\n' - ' These are the so-called "rich comparison" methods, and ' - 'are called\n' - ' for comparison operators in preference to "__cmp__()" ' - 'below. The\n' - ' correspondence between operator symbols and method ' - 'names is as\n' - ' follows: "xy" call ' - '"x.__ne__(y)",\n' - ' "x>y" calls "x.__gt__(y)", and "x>=y" calls ' - '"x.__ge__(y)".\n' - '\n' - ' A rich comparison method may return the singleton ' - '"NotImplemented"\n' - ' if it does not implement the operation for a given ' - 'pair of\n' - ' arguments. By convention, "False" and "True" are ' - 'returned for a\n' - ' successful comparison. However, these methods can ' - 'return any value,\n' - ' so if the comparison operator is used in a Boolean ' - 'context (e.g.,\n' - ' in the condition of an "if" statement), Python will ' - 'call "bool()"\n' - ' on the value to determine if the result is true or ' - 'false.\n' - '\n' - ' There are no implied relationships among the ' - 'comparison operators.\n' - ' The truth of "x==y" does not imply that "x!=y" is ' - 'false.\n' - ' Accordingly, when defining "__eq__()", one should also ' - 'define\n' - ' "__ne__()" so that the operators will behave as ' - 'expected. See the\n' - ' paragraph on "__hash__()" for some important notes on ' - 'creating\n' - ' *hashable* objects which support custom comparison ' - 'operations and\n' - ' are usable as dictionary keys.\n' - '\n' - ' There are no swapped-argument versions of these ' - 'methods (to be used\n' - ' when the left argument does not support the operation ' - 'but the right\n' - ' argument does); rather, "__lt__()" and "__gt__()" are ' - "each other's\n" - ' reflection, "__le__()" and "__ge__()" are each ' - "other's reflection,\n" - ' and "__eq__()" and "__ne__()" are their own ' - 'reflection.\n' - '\n' - ' Arguments to rich comparison methods are never ' - 'coerced.\n' - '\n' - ' To automatically generate ordering operations from a ' - 'single root\n' - ' operation, see "functools.total_ordering()".\n' - '\n' - 'object.__cmp__(self, other)\n' - '\n' - ' Called by comparison operations if rich comparison ' - '(see above) is\n' - ' not defined. Should return a negative integer if ' - '"self < other",\n' - ' zero if "self == other", a positive integer if "self > ' - 'other". If\n' - ' no "__cmp__()", "__eq__()" or "__ne__()" operation is ' - 'defined,\n' - ' class instances are compared by object identity ' - '("address"). See\n' - ' also the description of "__hash__()" for some ' - 'important notes on\n' - ' creating *hashable* objects which support custom ' - 'comparison\n' - ' operations and are usable as dictionary keys. (Note: ' - 'the\n' - ' restriction that exceptions are not propagated by ' - '"__cmp__()" has\n' - ' been removed since Python 1.5.)\n' - '\n' - 'object.__rcmp__(self, other)\n' - '\n' - ' Changed in version 2.1: No longer supported.\n' - '\n' - 'object.__hash__(self)\n' - '\n' - ' Called by built-in function "hash()" and for ' - 'operations on members\n' - ' of hashed collections including "set", "frozenset", ' - 'and "dict".\n' - ' "__hash__()" should return an integer. The only ' - 'required property\n' - ' is that objects which compare equal have the same hash ' - 'value; it is\n' - ' advised to somehow mix together (e.g. using exclusive ' - 'or) the hash\n' - ' values for the components of the object that also play ' - 'a part in\n' - ' comparison of objects.\n' - '\n' - ' If a class does not define a "__cmp__()" or "__eq__()" ' - 'method it\n' - ' should not define a "__hash__()" operation either; if ' - 'it defines\n' - ' "__cmp__()" or "__eq__()" but not "__hash__()", its ' - 'instances will\n' - ' not be usable in hashed collections. If a class ' - 'defines mutable\n' - ' objects and implements a "__cmp__()" or "__eq__()" ' - 'method, it\n' - ' should not implement "__hash__()", since hashable ' - 'collection\n' - " implementations require that a object's hash value is " - 'immutable (if\n' - " the object's hash value changes, it will be in the " - 'wrong hash\n' - ' bucket).\n' - '\n' - ' User-defined classes have "__cmp__()" and "__hash__()" ' - 'methods by\n' - ' default; with them, all objects compare unequal ' - '(except with\n' - ' themselves) and "x.__hash__()" returns a result ' - 'derived from\n' - ' "id(x)".\n' - '\n' - ' Classes which inherit a "__hash__()" method from a ' - 'parent class but\n' - ' change the meaning of "__cmp__()" or "__eq__()" such ' - 'that the hash\n' - ' value returned is no longer appropriate (e.g. by ' - 'switching to a\n' - ' value-based concept of equality instead of the default ' - 'identity\n' - ' based equality) can explicitly flag themselves as ' - 'being unhashable\n' - ' by setting "__hash__ = None" in the class definition. ' - 'Doing so\n' - ' means that not only will instances of the class raise ' - 'an\n' - ' appropriate "TypeError" when a program attempts to ' - 'retrieve their\n' - ' hash value, but they will also be correctly identified ' - 'as\n' - ' unhashable when checking "isinstance(obj, ' - 'collections.Hashable)"\n' - ' (unlike classes which define their own "__hash__()" to ' - 'explicitly\n' - ' raise "TypeError").\n' - '\n' - ' Changed in version 2.5: "__hash__()" may now also ' - 'return a long\n' - ' integer object; the 32-bit integer is then derived ' - 'from the hash of\n' - ' that object.\n' - '\n' - ' Changed in version 2.6: "__hash__" may now be set to ' - '"None" to\n' - ' explicitly flag instances of a class as unhashable.\n' - '\n' - 'object.__nonzero__(self)\n' - '\n' - ' Called to implement truth value testing and the ' - 'built-in operation\n' - ' "bool()"; should return "False" or "True", or their ' - 'integer\n' - ' equivalents "0" or "1". When this method is not ' - 'defined,\n' - ' "__len__()" is called, if it is defined, and the ' - 'object is\n' - ' considered true if its result is nonzero. If a class ' - 'defines\n' - ' neither "__len__()" nor "__nonzero__()", all its ' - 'instances are\n' - ' considered true.\n' - '\n' - 'object.__unicode__(self)\n' - '\n' - ' Called to implement "unicode()" built-in; should ' - 'return a Unicode\n' - ' object. When this method is not defined, string ' - 'conversion is\n' - ' attempted, and the result of string conversion is ' - 'converted to\n' - ' Unicode using the system default encoding.\n', - 'debugger': '\n' - '"pdb" --- The Python Debugger\n' - '*****************************\n' - '\n' - '**Source code:** Lib/pdb.py\n' - '\n' - '======================================================================\n' - '\n' - 'The module "pdb" defines an interactive source code debugger ' - 'for\n' - 'Python programs. It supports setting (conditional) ' - 'breakpoints and\n' - 'single stepping at the source line level, inspection of stack ' - 'frames,\n' - 'source code listing, and evaluation of arbitrary Python code ' - 'in the\n' - 'context of any stack frame. It also supports post-mortem ' - 'debugging\n' - 'and can be called under program control.\n' - '\n' - 'The debugger is extensible --- it is actually defined as the ' - 'class\n' - '"Pdb". This is currently undocumented but easily understood by ' - 'reading\n' - 'the source. The extension interface uses the modules "bdb" ' - 'and "cmd".\n' - '\n' - 'The debugger\'s prompt is "(Pdb)". Typical usage to run a ' - 'program under\n' - 'control of the debugger is:\n' - '\n' - ' >>> import pdb\n' - ' >>> import mymodule\n' - " >>> pdb.run('mymodule.test()')\n" - ' > (0)?()\n' - ' (Pdb) continue\n' - ' > (1)?()\n' - ' (Pdb) continue\n' - " NameError: 'spam'\n" - ' > (1)?()\n' - ' (Pdb)\n' - '\n' - '"pdb.py" can also be invoked as a script to debug other ' - 'scripts. For\n' - 'example:\n' - '\n' - ' python -m pdb myscript.py\n' - '\n' - 'When invoked as a script, pdb will automatically enter ' - 'post-mortem\n' - 'debugging if the program being debugged exits abnormally. ' - 'After post-\n' - 'mortem debugging (or after normal exit of the program), pdb ' - 'will\n' - "restart the program. Automatic restarting preserves pdb's " - 'state (such\n' - 'as breakpoints) and in most cases is more useful than quitting ' - 'the\n' - "debugger upon program's exit.\n" - '\n' - 'New in version 2.4: Restarting post-mortem behavior added.\n' - '\n' - 'The typical usage to break into the debugger from a running ' - 'program is\n' - 'to insert\n' - '\n' - ' import pdb; pdb.set_trace()\n' - '\n' - 'at the location you want to break into the debugger. You can ' - 'then\n' - 'step through the code following this statement, and continue ' - 'running\n' - 'without the debugger using the "c" command.\n' - '\n' - 'The typical usage to inspect a crashed program is:\n' - '\n' - ' >>> import pdb\n' - ' >>> import mymodule\n' - ' >>> mymodule.test()\n' - ' Traceback (most recent call last):\n' - ' File "", line 1, in ?\n' - ' File "./mymodule.py", line 4, in test\n' - ' test2()\n' - ' File "./mymodule.py", line 3, in test2\n' - ' print spam\n' - ' NameError: spam\n' - ' >>> pdb.pm()\n' - ' > ./mymodule.py(3)test2()\n' - ' -> print spam\n' - ' (Pdb)\n' - '\n' - 'The module defines the following functions; each enters the ' - 'debugger\n' - 'in a slightly different way:\n' - '\n' - 'pdb.run(statement[, globals[, locals]])\n' - '\n' - ' Execute the *statement* (given as a string) under debugger ' - 'control.\n' - ' The debugger prompt appears before any code is executed; ' - 'you can\n' - ' set breakpoints and type "continue", or you can step ' - 'through the\n' - ' statement using "step" or "next" (all these commands are ' - 'explained\n' - ' below). The optional *globals* and *locals* arguments ' - 'specify the\n' - ' environment in which the code is executed; by default the\n' - ' dictionary of the module "__main__" is used. (See the ' - 'explanation\n' - ' of the "exec" statement or the "eval()" built-in ' - 'function.)\n' - '\n' - 'pdb.runeval(expression[, globals[, locals]])\n' - '\n' - ' Evaluate the *expression* (given as a string) under ' - 'debugger\n' - ' control. When "runeval()" returns, it returns the value of ' - 'the\n' - ' expression. Otherwise this function is similar to ' - '"run()".\n' - '\n' - 'pdb.runcall(function[, argument, ...])\n' - '\n' - ' Call the *function* (a function or method object, not a ' - 'string)\n' - ' with the given arguments. When "runcall()" returns, it ' - 'returns\n' - ' whatever the function call returned. The debugger prompt ' - 'appears\n' - ' as soon as the function is entered.\n' - '\n' - 'pdb.set_trace()\n' - '\n' - ' Enter the debugger at the calling stack frame. This is ' - 'useful to\n' - ' hard-code a breakpoint at a given point in a program, even ' - 'if the\n' - ' code is not otherwise being debugged (e.g. when an ' - 'assertion\n' - ' fails).\n' - '\n' - 'pdb.post_mortem([traceback])\n' - '\n' - ' Enter post-mortem debugging of the given *traceback* ' - 'object. If no\n' - ' *traceback* is given, it uses the one of the exception that ' - 'is\n' - ' currently being handled (an exception must be being handled ' - 'if the\n' - ' default is to be used).\n' - '\n' - 'pdb.pm()\n' - '\n' - ' Enter post-mortem debugging of the traceback found in\n' - ' "sys.last_traceback".\n' - '\n' - 'The "run*" functions and "set_trace()" are aliases for ' - 'instantiating\n' - 'the "Pdb" class and calling the method of the same name. If ' - 'you want\n' - 'to access further features, you have to do this yourself:\n' - '\n' - "class class pdb.Pdb(completekey='tab', stdin=None, " - 'stdout=None, skip=None)\n' - '\n' - ' "Pdb" is the debugger class.\n' - '\n' - ' The *completekey*, *stdin* and *stdout* arguments are ' - 'passed to the\n' - ' underlying "cmd.Cmd" class; see the description there.\n' - '\n' - ' The *skip* argument, if given, must be an iterable of ' - 'glob-style\n' - ' module name patterns. The debugger will not step into ' - 'frames that\n' - ' originate in a module that matches one of these patterns. ' - '[1]\n' - '\n' - ' Example call to enable tracing with *skip*:\n' - '\n' - " import pdb; pdb.Pdb(skip=['django.*']).set_trace()\n" - '\n' - ' New in version 2.7: The *skip* argument.\n' - '\n' - ' run(statement[, globals[, locals]])\n' - ' runeval(expression[, globals[, locals]])\n' - ' runcall(function[, argument, ...])\n' - ' set_trace()\n' - '\n' - ' See the documentation for the functions explained ' - 'above.\n', - 'del': '\n' - 'The "del" statement\n' - '*******************\n' - '\n' - ' del_stmt ::= "del" target_list\n' - '\n' - 'Deletion is recursively defined very similar to the way assignment ' - 'is\n' - 'defined. Rather than spelling it out in full details, here are ' - 'some\n' - 'hints.\n' - '\n' - 'Deletion of a target list recursively deletes each target, from ' - 'left\n' - 'to right.\n' - '\n' - 'Deletion of a name removes the binding of that name from the local ' - 'or\n' - 'global namespace, depending on whether the name occurs in a ' - '"global"\n' - 'statement in the same code block. If the name is unbound, a\n' - '"NameError" exception will be raised.\n' - '\n' - 'It is illegal to delete a name from the local namespace if it ' - 'occurs\n' - 'as a free variable in a nested block.\n' - '\n' - 'Deletion of attribute references, subscriptions and slicings is ' - 'passed\n' - 'to the primary object involved; deletion of a slicing is in ' - 'general\n' - 'equivalent to assignment of an empty slice of the right type (but ' - 'even\n' - 'this is determined by the sliced object).\n', - 'dict': '\n' - 'Dictionary displays\n' - '*******************\n' - '\n' - 'A dictionary display is a possibly empty series of key/datum ' - 'pairs\n' - 'enclosed in curly braces:\n' - '\n' - ' dict_display ::= "{" [key_datum_list | ' - 'dict_comprehension] "}"\n' - ' key_datum_list ::= key_datum ("," key_datum)* [","]\n' - ' key_datum ::= expression ":" expression\n' - ' dict_comprehension ::= expression ":" expression comp_for\n' - '\n' - 'A dictionary display yields a new dictionary object.\n' - '\n' - 'If a comma-separated sequence of key/datum pairs is given, they ' - 'are\n' - 'evaluated from left to right to define the entries of the ' - 'dictionary:\n' - 'each key object is used as a key into the dictionary to store the\n' - 'corresponding datum. This means that you can specify the same ' - 'key\n' - "multiple times in the key/datum list, and the final dictionary's " - 'value\n' - 'for that key will be the last one given.\n' - '\n' - 'A dict comprehension, in contrast to list and set comprehensions,\n' - 'needs two expressions separated with a colon followed by the ' - 'usual\n' - '"for" and "if" clauses. When the comprehension is run, the ' - 'resulting\n' - 'key and value elements are inserted in the new dictionary in the ' - 'order\n' - 'they are produced.\n' - '\n' - 'Restrictions on the types of the key values are listed earlier in\n' - 'section The standard type hierarchy. (To summarize, the key type\n' - 'should be *hashable*, which excludes all mutable objects.) ' - 'Clashes\n' - 'between duplicate keys are not detected; the last datum ' - '(textually\n' - 'rightmost in the display) stored for a given key value prevails.\n', - 'dynamic-features': '\n' - 'Interaction with dynamic features\n' - '*********************************\n' - '\n' - 'There are several cases where Python statements are ' - 'illegal when used\n' - 'in conjunction with nested scopes that contain free ' - 'variables.\n' - '\n' - 'If a variable is referenced in an enclosing scope, it ' - 'is illegal to\n' - 'delete the name. An error will be reported at compile ' - 'time.\n' - '\n' - 'If the wild card form of import --- "import *" --- is ' - 'used in a\n' - 'function and the function contains or is a nested ' - 'block with free\n' - 'variables, the compiler will raise a "SyntaxError".\n' - '\n' - 'If "exec" is used in a function and the function ' - 'contains or is a\n' - 'nested block with free variables, the compiler will ' - 'raise a\n' - '"SyntaxError" unless the exec explicitly specifies the ' - 'local namespace\n' - 'for the "exec". (In other words, "exec obj" would be ' - 'illegal, but\n' - '"exec obj in ns" would be legal.)\n' - '\n' - 'The "eval()", "execfile()", and "input()" functions ' - 'and the "exec"\n' - 'statement do not have access to the full environment ' - 'for resolving\n' - 'names. Names may be resolved in the local and global ' - 'namespaces of\n' - 'the caller. Free variables are not resolved in the ' - 'nearest enclosing\n' - 'namespace, but in the global namespace. [1] The "exec" ' - 'statement and\n' - 'the "eval()" and "execfile()" functions have optional ' - 'arguments to\n' - 'override the global and local namespace. If only one ' - 'namespace is\n' - 'specified, it is used for both.\n', - 'else': '\n' - 'The "if" statement\n' - '******************\n' - '\n' - 'The "if" statement is used for conditional execution:\n' - '\n' - ' if_stmt ::= "if" expression ":" suite\n' - ' ( "elif" expression ":" suite )*\n' - ' ["else" ":" suite]\n' - '\n' - 'It selects exactly one of the suites by evaluating the expressions ' - 'one\n' - 'by one until one is found to be true (see section Boolean ' - 'operations\n' - 'for the definition of true and false); then that suite is ' - 'executed\n' - '(and no other part of the "if" statement is executed or ' - 'evaluated).\n' - 'If all expressions are false, the suite of the "else" clause, if\n' - 'present, is executed.\n', - 'exceptions': '\n' - 'Exceptions\n' - '**********\n' - '\n' - 'Exceptions are a means of breaking out of the normal flow of ' - 'control\n' - 'of a code block in order to handle errors or other ' - 'exceptional\n' - 'conditions. An exception is *raised* at the point where the ' - 'error is\n' - 'detected; it may be *handled* by the surrounding code block ' - 'or by any\n' - 'code block that directly or indirectly invoked the code ' - 'block where\n' - 'the error occurred.\n' - '\n' - 'The Python interpreter raises an exception when it detects a ' - 'run-time\n' - 'error (such as division by zero). A Python program can ' - 'also\n' - 'explicitly raise an exception with the "raise" statement. ' - 'Exception\n' - 'handlers are specified with the "try" ... "except" ' - 'statement. The\n' - '"finally" clause of such a statement can be used to specify ' - 'cleanup\n' - 'code which does not handle the exception, but is executed ' - 'whether an\n' - 'exception occurred or not in the preceding code.\n' - '\n' - 'Python uses the "termination" model of error handling: an ' - 'exception\n' - 'handler can find out what happened and continue execution at ' - 'an outer\n' - 'level, but it cannot repair the cause of the error and retry ' - 'the\n' - 'failing operation (except by re-entering the offending piece ' - 'of code\n' - 'from the top).\n' - '\n' - 'When an exception is not handled at all, the interpreter ' - 'terminates\n' - 'execution of the program, or returns to its interactive main ' - 'loop. In\n' - 'either case, it prints a stack backtrace, except when the ' - 'exception is\n' - '"SystemExit".\n' - '\n' - 'Exceptions are identified by class instances. The "except" ' - 'clause is\n' - 'selected depending on the class of the instance: it must ' - 'reference the\n' - 'class of the instance or a base class thereof. The instance ' - 'can be\n' - 'received by the handler and can carry additional information ' - 'about the\n' - 'exceptional condition.\n' - '\n' - 'Exceptions can also be identified by strings, in which case ' - 'the\n' - '"except" clause is selected by object identity. An ' - 'arbitrary value\n' - 'can be raised along with the identifying string which can be ' - 'passed to\n' - 'the handler.\n' - '\n' - 'Note: Messages to exceptions are not part of the Python ' - 'API. Their\n' - ' contents may change from one version of Python to the next ' - 'without\n' - ' warning and should not be relied on by code which will run ' - 'under\n' - ' multiple versions of the interpreter.\n' - '\n' - 'See also the description of the "try" statement in section ' - 'The try\n' - 'statement and "raise" statement in section The raise ' - 'statement.\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] This limitation occurs because the code that is executed ' - 'by\n' - ' these operations is not available at the time the module ' - 'is\n' - ' compiled.\n', - 'exec': '\n' - 'The "exec" statement\n' - '********************\n' - '\n' - ' exec_stmt ::= "exec" or_expr ["in" expression ["," ' - 'expression]]\n' - '\n' - 'This statement supports dynamic execution of Python code. The ' - 'first\n' - 'expression should evaluate to either a Unicode string, a ' - '*Latin-1*\n' - 'encoded string, an open file object, a code object, or a tuple. ' - 'If it\n' - 'is a string, the string is parsed as a suite of Python statements\n' - 'which is then executed (unless a syntax error occurs). [1] If it ' - 'is an\n' - 'open file, the file is parsed until EOF and executed. If it is a ' - 'code\n' - 'object, it is simply executed. For the interpretation of a tuple, ' - 'see\n' - "below. In all cases, the code that's executed is expected to be " - 'valid\n' - 'as file input (see section File input). Be aware that the ' - '"return"\n' - 'and "yield" statements may not be used outside of function ' - 'definitions\n' - 'even within the context of code passed to the "exec" statement.\n' - '\n' - 'In all cases, if the optional parts are omitted, the code is ' - 'executed\n' - 'in the current scope. If only the first expression after "in" is\n' - 'specified, it should be a dictionary, which will be used for both ' - 'the\n' - 'global and the local variables. If two expressions are given, ' - 'they\n' - 'are used for the global and local variables, respectively. If\n' - 'provided, *locals* can be any mapping object. Remember that at ' - 'module\n' - 'level, globals and locals are the same dictionary. If two ' - 'separate\n' - 'objects are given as *globals* and *locals*, the code will be ' - 'executed\n' - 'as if it were embedded in a class definition.\n' - '\n' - 'The first expression may also be a tuple of length 2 or 3. In ' - 'this\n' - 'case, the optional parts must be omitted. The form "exec(expr,\n' - 'globals)" is equivalent to "exec expr in globals", while the form\n' - '"exec(expr, globals, locals)" is equivalent to "exec expr in ' - 'globals,\n' - 'locals". The tuple form of "exec" provides compatibility with ' - 'Python\n' - '3, where "exec" is a function rather than a statement.\n' - '\n' - 'Changed in version 2.4: Formerly, *locals* was required to be a\n' - 'dictionary.\n' - '\n' - 'As a side effect, an implementation may insert additional keys ' - 'into\n' - 'the dictionaries given besides those corresponding to variable ' - 'names\n' - 'set by the executed code. For example, the current implementation ' - 'may\n' - 'add a reference to the dictionary of the built-in module ' - '"__builtin__"\n' - 'under the key "__builtins__" (!).\n' - '\n' - "**Programmer's hints:** dynamic evaluation of expressions is " - 'supported\n' - 'by the built-in function "eval()". The built-in functions ' - '"globals()"\n' - 'and "locals()" return the current global and local dictionary,\n' - 'respectively, which may be useful to pass around for use by ' - '"exec".\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] Note that the parser only accepts the Unix-style end of line\n' - ' convention. If you are reading the code from a file, make sure ' - 'to\n' - ' use *universal newlines* mode to convert Windows or Mac-style\n' - ' newlines.\n', - 'execmodel': '\n' - 'Execution model\n' - '***************\n' - '\n' - '\n' - 'Naming and binding\n' - '==================\n' - '\n' - '*Names* refer to objects. Names are introduced by name ' - 'binding\n' - 'operations. Each occurrence of a name in the program text ' - 'refers to\n' - 'the *binding* of that name established in the innermost ' - 'function block\n' - 'containing the use.\n' - '\n' - 'A *block* is a piece of Python program text that is executed ' - 'as a\n' - 'unit. The following are blocks: a module, a function body, ' - 'and a class\n' - 'definition. Each command typed interactively is a block. A ' - 'script\n' - 'file (a file given as standard input to the interpreter or ' - 'specified\n' - 'on the interpreter command line the first argument) is a code ' - 'block.\n' - 'A script command (a command specified on the interpreter ' - 'command line\n' - "with the '**-c**' option) is a code block. The file read by " - 'the\n' - 'built-in function "execfile()" is a code block. The string ' - 'argument\n' - 'passed to the built-in function "eval()" and to the "exec" ' - 'statement\n' - 'is a code block. The expression read and evaluated by the ' - 'built-in\n' - 'function "input()" is a code block.\n' - '\n' - 'A code block is executed in an *execution frame*. A frame ' - 'contains\n' - 'some administrative information (used for debugging) and ' - 'determines\n' - "where and how execution continues after the code block's " - 'execution has\n' - 'completed.\n' - '\n' - 'A *scope* defines the visibility of a name within a block. ' - 'If a local\n' - 'variable is defined in a block, its scope includes that ' - 'block. If the\n' - 'definition occurs in a function block, the scope extends to ' - 'any blocks\n' - 'contained within the defining one, unless a contained block ' - 'introduces\n' - 'a different binding for the name. The scope of names defined ' - 'in a\n' - 'class block is limited to the class block; it does not extend ' - 'to the\n' - 'code blocks of methods -- this includes generator expressions ' - 'since\n' - 'they are implemented using a function scope. This means that ' - 'the\n' - 'following will fail:\n' - '\n' - ' class A:\n' - ' a = 42\n' - ' b = list(a + i for i in range(10))\n' - '\n' - 'When a name is used in a code block, it is resolved using the ' - 'nearest\n' - 'enclosing scope. The set of all such scopes visible to a ' - 'code block\n' - "is called the block's *environment*.\n" - '\n' - 'If a name is bound in a block, it is a local variable of that ' - 'block.\n' - 'If a name is bound at the module level, it is a global ' - 'variable. (The\n' - 'variables of the module code block are local and global.) If ' - 'a\n' - 'variable is used in a code block but not defined there, it is ' - 'a *free\n' - 'variable*.\n' - '\n' - 'When a name is not found at all, a "NameError" exception is ' - 'raised.\n' - 'If the name refers to a local variable that has not been ' - 'bound, a\n' - '"UnboundLocalError" exception is raised. "UnboundLocalError" ' - 'is a\n' - 'subclass of "NameError".\n' - '\n' - 'The following constructs bind names: formal parameters to ' - 'functions,\n' - '"import" statements, class and function definitions (these ' - 'bind the\n' - 'class or function name in the defining block), and targets ' - 'that are\n' - 'identifiers if occurring in an assignment, "for" loop header, ' - 'in the\n' - 'second position of an "except" clause header or after "as" in ' - 'a "with"\n' - 'statement. The "import" statement of the form "from ... ' - 'import *"\n' - 'binds all names defined in the imported module, except those ' - 'beginning\n' - 'with an underscore. This form may only be used at the module ' - 'level.\n' - '\n' - 'A target occurring in a "del" statement is also considered ' - 'bound for\n' - 'this purpose (though the actual semantics are to unbind the ' - 'name). It\n' - 'is illegal to unbind a name that is referenced by an ' - 'enclosing scope;\n' - 'the compiler will report a "SyntaxError".\n' - '\n' - 'Each assignment or import statement occurs within a block ' - 'defined by a\n' - 'class or function definition or at the module level (the ' - 'top-level\n' - 'code block).\n' - '\n' - 'If a name binding operation occurs anywhere within a code ' - 'block, all\n' - 'uses of the name within the block are treated as references ' - 'to the\n' - 'current block. This can lead to errors when a name is used ' - 'within a\n' - 'block before it is bound. This rule is subtle. Python lacks\n' - 'declarations and allows name binding operations to occur ' - 'anywhere\n' - 'within a code block. The local variables of a code block can ' - 'be\n' - 'determined by scanning the entire text of the block for name ' - 'binding\n' - 'operations.\n' - '\n' - 'If the global statement occurs within a block, all uses of ' - 'the name\n' - 'specified in the statement refer to the binding of that name ' - 'in the\n' - 'top-level namespace. Names are resolved in the top-level ' - 'namespace by\n' - 'searching the global namespace, i.e. the namespace of the ' - 'module\n' - 'containing the code block, and the builtins namespace, the ' - 'namespace\n' - 'of the module "__builtin__". The global namespace is ' - 'searched first.\n' - 'If the name is not found there, the builtins namespace is ' - 'searched.\n' - 'The global statement must precede all uses of the name.\n' - '\n' - 'The builtins namespace associated with the execution of a ' - 'code block\n' - 'is actually found by looking up the name "__builtins__" in ' - 'its global\n' - 'namespace; this should be a dictionary or a module (in the ' - 'latter case\n' - "the module's dictionary is used). By default, when in the " - '"__main__"\n' - 'module, "__builtins__" is the built-in module "__builtin__" ' - '(note: no\n' - '\'s\'); when in any other module, "__builtins__" is an alias ' - 'for the\n' - 'dictionary of the "__builtin__" module itself. ' - '"__builtins__" can be\n' - 'set to a user-created dictionary to create a weak form of ' - 'restricted\n' - 'execution.\n' - '\n' - '**CPython implementation detail:** Users should not touch\n' - '"__builtins__"; it is strictly an implementation detail. ' - 'Users\n' - 'wanting to override values in the builtins namespace should ' - '"import"\n' - 'the "__builtin__" (no \'s\') module and modify its ' - 'attributes\n' - 'appropriately.\n' - '\n' - 'The namespace for a module is automatically created the first ' - 'time a\n' - 'module is imported. The main module for a script is always ' - 'called\n' - '"__main__".\n' - '\n' - 'The "global" statement has the same scope as a name binding ' - 'operation\n' - 'in the same block. If the nearest enclosing scope for a free ' - 'variable\n' - 'contains a global statement, the free variable is treated as ' - 'a global.\n' - '\n' - 'A class definition is an executable statement that may use ' - 'and define\n' - 'names. These references follow the normal rules for name ' - 'resolution.\n' - 'The namespace of the class definition becomes the attribute ' - 'dictionary\n' - 'of the class. Names defined at the class scope are not ' - 'visible in\n' - 'methods.\n' - '\n' - '\n' - 'Interaction with dynamic features\n' - '---------------------------------\n' - '\n' - 'There are several cases where Python statements are illegal ' - 'when used\n' - 'in conjunction with nested scopes that contain free ' - 'variables.\n' - '\n' - 'If a variable is referenced in an enclosing scope, it is ' - 'illegal to\n' - 'delete the name. An error will be reported at compile time.\n' - '\n' - 'If the wild card form of import --- "import *" --- is used in ' - 'a\n' - 'function and the function contains or is a nested block with ' - 'free\n' - 'variables, the compiler will raise a "SyntaxError".\n' - '\n' - 'If "exec" is used in a function and the function contains or ' - 'is a\n' - 'nested block with free variables, the compiler will raise a\n' - '"SyntaxError" unless the exec explicitly specifies the local ' - 'namespace\n' - 'for the "exec". (In other words, "exec obj" would be ' - 'illegal, but\n' - '"exec obj in ns" would be legal.)\n' - '\n' - 'The "eval()", "execfile()", and "input()" functions and the ' - '"exec"\n' - 'statement do not have access to the full environment for ' - 'resolving\n' - 'names. Names may be resolved in the local and global ' - 'namespaces of\n' - 'the caller. Free variables are not resolved in the nearest ' - 'enclosing\n' - 'namespace, but in the global namespace. [1] The "exec" ' - 'statement and\n' - 'the "eval()" and "execfile()" functions have optional ' - 'arguments to\n' - 'override the global and local namespace. If only one ' - 'namespace is\n' - 'specified, it is used for both.\n' - '\n' - '\n' - 'Exceptions\n' - '==========\n' - '\n' - 'Exceptions are a means of breaking out of the normal flow of ' - 'control\n' - 'of a code block in order to handle errors or other ' - 'exceptional\n' - 'conditions. An exception is *raised* at the point where the ' - 'error is\n' - 'detected; it may be *handled* by the surrounding code block ' - 'or by any\n' - 'code block that directly or indirectly invoked the code block ' - 'where\n' - 'the error occurred.\n' - '\n' - 'The Python interpreter raises an exception when it detects a ' - 'run-time\n' - 'error (such as division by zero). A Python program can also\n' - 'explicitly raise an exception with the "raise" statement. ' - 'Exception\n' - 'handlers are specified with the "try" ... "except" ' - 'statement. The\n' - '"finally" clause of such a statement can be used to specify ' - 'cleanup\n' - 'code which does not handle the exception, but is executed ' - 'whether an\n' - 'exception occurred or not in the preceding code.\n' - '\n' - 'Python uses the "termination" model of error handling: an ' - 'exception\n' - 'handler can find out what happened and continue execution at ' - 'an outer\n' - 'level, but it cannot repair the cause of the error and retry ' - 'the\n' - 'failing operation (except by re-entering the offending piece ' - 'of code\n' - 'from the top).\n' - '\n' - 'When an exception is not handled at all, the interpreter ' - 'terminates\n' - 'execution of the program, or returns to its interactive main ' - 'loop. In\n' - 'either case, it prints a stack backtrace, except when the ' - 'exception is\n' - '"SystemExit".\n' - '\n' - 'Exceptions are identified by class instances. The "except" ' - 'clause is\n' - 'selected depending on the class of the instance: it must ' - 'reference the\n' - 'class of the instance or a base class thereof. The instance ' - 'can be\n' - 'received by the handler and can carry additional information ' - 'about the\n' - 'exceptional condition.\n' - '\n' - 'Exceptions can also be identified by strings, in which case ' - 'the\n' - '"except" clause is selected by object identity. An arbitrary ' - 'value\n' - 'can be raised along with the identifying string which can be ' - 'passed to\n' - 'the handler.\n' - '\n' - 'Note: Messages to exceptions are not part of the Python API. ' - 'Their\n' - ' contents may change from one version of Python to the next ' - 'without\n' - ' warning and should not be relied on by code which will run ' - 'under\n' - ' multiple versions of the interpreter.\n' - '\n' - 'See also the description of the "try" statement in section ' - 'The try\n' - 'statement and "raise" statement in section The raise ' - 'statement.\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] This limitation occurs because the code that is executed ' - 'by\n' - ' these operations is not available at the time the module ' - 'is\n' - ' compiled.\n', - 'exprlists': '\n' - 'Expression lists\n' - '****************\n' - '\n' - ' expression_list ::= expression ( "," expression )* [","]\n' - '\n' - 'An expression list containing at least one comma yields a ' - 'tuple. The\n' - 'length of the tuple is the number of expressions in the ' - 'list. The\n' - 'expressions are evaluated from left to right.\n' - '\n' - 'The trailing comma is required only to create a single tuple ' - '(a.k.a. a\n' - '*singleton*); it is optional in all other cases. A single ' - 'expression\n' - "without a trailing comma doesn't create a tuple, but rather " - 'yields the\n' - 'value of that expression. (To create an empty tuple, use an ' - 'empty pair\n' - 'of parentheses: "()".)\n', - 'floating': '\n' - 'Floating point literals\n' - '***********************\n' - '\n' - 'Floating point literals are described by the following ' - 'lexical\n' - 'definitions:\n' - '\n' - ' floatnumber ::= pointfloat | exponentfloat\n' - ' pointfloat ::= [intpart] fraction | intpart "."\n' - ' exponentfloat ::= (intpart | pointfloat) exponent\n' - ' intpart ::= digit+\n' - ' fraction ::= "." digit+\n' - ' exponent ::= ("e" | "E") ["+" | "-"] digit+\n' - '\n' - 'Note that the integer and exponent parts of floating point ' - 'numbers can\n' - 'look like octal integers, but are interpreted using radix 10. ' - 'For\n' - 'example, "077e010" is legal, and denotes the same number as ' - '"77e10".\n' - 'The allowed range of floating point literals is ' - 'implementation-\n' - 'dependent. Some examples of floating point literals:\n' - '\n' - ' 3.14 10. .001 1e100 3.14e-10 0e0\n' - '\n' - 'Note that numeric literals do not include a sign; a phrase ' - 'like "-1"\n' - 'is actually an expression composed of the unary operator "-" ' - 'and the\n' - 'literal "1".\n', - 'for': '\n' - 'The "for" statement\n' - '*******************\n' - '\n' - 'The "for" statement is used to iterate over the elements of a ' - 'sequence\n' - '(such as a string, tuple or list) or other iterable object:\n' - '\n' - ' for_stmt ::= "for" target_list "in" expression_list ":" suite\n' - ' ["else" ":" suite]\n' - '\n' - 'The expression list is evaluated once; it should yield an iterable\n' - 'object. An iterator is created for the result of the\n' - '"expression_list". The suite is then executed once for each item\n' - 'provided by the iterator, in the order of ascending indices. Each\n' - 'item in turn is assigned to the target list using the standard ' - 'rules\n' - 'for assignments, and then the suite is executed. When the items ' - 'are\n' - 'exhausted (which is immediately when the sequence is empty), the ' - 'suite\n' - 'in the "else" clause, if present, is executed, and the loop\n' - 'terminates.\n' - '\n' - 'A "break" statement executed in the first suite terminates the ' - 'loop\n' - 'without executing the "else" clause\'s suite. A "continue" ' - 'statement\n' - 'executed in the first suite skips the rest of the suite and ' - 'continues\n' - 'with the next item, or with the "else" clause if there was no next\n' - 'item.\n' - '\n' - 'The suite may assign to the variable(s) in the target list; this ' - 'does\n' - 'not affect the next item assigned to it.\n' - '\n' - 'The target list is not deleted when the loop is finished, but if ' - 'the\n' - 'sequence is empty, it will not have been assigned to at all by the\n' - 'loop. Hint: the built-in function "range()" returns a sequence of\n' - 'integers suitable to emulate the effect of Pascal\'s "for i := a to ' - 'b\n' - 'do"; e.g., "range(3)" returns the list "[0, 1, 2]".\n' - '\n' - 'Note: There is a subtlety when the sequence is being modified by ' - 'the\n' - ' loop (this can only occur for mutable sequences, i.e. lists). An\n' - ' internal counter is used to keep track of which item is used ' - 'next,\n' - ' and this is incremented on each iteration. When this counter ' - 'has\n' - ' reached the length of the sequence the loop terminates. This ' - 'means\n' - ' that if the suite deletes the current (or a previous) item from ' - 'the\n' - ' sequence, the next item will be skipped (since it gets the index ' - 'of\n' - ' the current item which has already been treated). Likewise, if ' - 'the\n' - ' suite inserts an item in the sequence before the current item, ' - 'the\n' - ' current item will be treated again the next time through the ' - 'loop.\n' - ' This can lead to nasty bugs that can be avoided by making a\n' - ' temporary copy using a slice of the whole sequence, e.g.,\n' - '\n' - ' for x in a[:]:\n' - ' if x < 0: a.remove(x)\n', - 'formatstrings': '\n' - 'Format String Syntax\n' - '********************\n' - '\n' - 'The "str.format()" method and the "Formatter" class share ' - 'the same\n' - 'syntax for format strings (although in the case of ' - '"Formatter",\n' - 'subclasses can define their own format string syntax).\n' - '\n' - 'Format strings contain "replacement fields" surrounded by ' - 'curly braces\n' - '"{}". Anything that is not contained in braces is ' - 'considered literal\n' - 'text, which is copied unchanged to the output. If you ' - 'need to include\n' - 'a brace character in the literal text, it can be escaped ' - 'by doubling:\n' - '"{{" and "}}".\n' - '\n' - 'The grammar for a replacement field is as follows:\n' - '\n' - ' replacement_field ::= "{" [field_name] ["!" ' - 'conversion] [":" format_spec] "}"\n' - ' field_name ::= arg_name ("." attribute_name ' - '| "[" element_index "]")*\n' - ' arg_name ::= [identifier | integer]\n' - ' attribute_name ::= identifier\n' - ' element_index ::= integer | index_string\n' - ' index_string ::= +\n' - ' conversion ::= "r" | "s"\n' - ' format_spec ::= \n' - '\n' - 'In less formal terms, the replacement field can start ' - 'with a\n' - '*field_name* that specifies the object whose value is to ' - 'be formatted\n' - 'and inserted into the output instead of the replacement ' - 'field. The\n' - '*field_name* is optionally followed by a *conversion* ' - 'field, which is\n' - 'preceded by an exclamation point "\'!\'", and a ' - '*format_spec*, which is\n' - 'preceded by a colon "\':\'". These specify a non-default ' - 'format for the\n' - 'replacement value.\n' - '\n' - 'See also the Format Specification Mini-Language section.\n' - '\n' - 'The *field_name* itself begins with an *arg_name* that is ' - 'either a\n' - "number or a keyword. If it's a number, it refers to a " - 'positional\n' - "argument, and if it's a keyword, it refers to a named " - 'keyword\n' - 'argument. If the numerical arg_names in a format string ' - 'are 0, 1, 2,\n' - '... in sequence, they can all be omitted (not just some) ' - 'and the\n' - 'numbers 0, 1, 2, ... will be automatically inserted in ' - 'that order.\n' - 'Because *arg_name* is not quote-delimited, it is not ' - 'possible to\n' - 'specify arbitrary dictionary keys (e.g., the strings ' - '"\'10\'" or\n' - '"\':-]\'") within a format string. The *arg_name* can be ' - 'followed by any\n' - 'number of index or attribute expressions. An expression ' - 'of the form\n' - '"\'.name\'" selects the named attribute using ' - '"getattr()", while an\n' - 'expression of the form "\'[index]\'" does an index lookup ' - 'using\n' - '"__getitem__()".\n' - '\n' - 'Changed in version 2.7: The positional argument ' - 'specifiers can be\n' - 'omitted, so "\'{} {}\'" is equivalent to "\'{0} {1}\'".\n' - '\n' - 'Some simple format string examples:\n' - '\n' - ' "First, thou shalt count to {0}" # References first ' - 'positional argument\n' - ' "Bring me a {}" # Implicitly ' - 'references the first positional argument\n' - ' "From {} to {}" # Same as "From {0} ' - 'to {1}"\n' - ' "My quest is {name}" # References keyword ' - "argument 'name'\n" - ' "Weight in tons {0.weight}" # \'weight\' ' - 'attribute of first positional arg\n' - ' "Units destroyed: {players[0]}" # First element of ' - "keyword argument 'players'.\n" - '\n' - 'The *conversion* field causes a type coercion before ' - 'formatting.\n' - 'Normally, the job of formatting a value is done by the ' - '"__format__()"\n' - 'method of the value itself. However, in some cases it is ' - 'desirable to\n' - 'force a type to be formatted as a string, overriding its ' - 'own\n' - 'definition of formatting. By converting the value to a ' - 'string before\n' - 'calling "__format__()", the normal formatting logic is ' - 'bypassed.\n' - '\n' - 'Two conversion flags are currently supported: "\'!s\'" ' - 'which calls\n' - '"str()" on the value, and "\'!r\'" which calls "repr()".\n' - '\n' - 'Some examples:\n' - '\n' - ' "Harold\'s a clever {0!s}" # Calls str() on the ' - 'argument first\n' - ' "Bring out the holy {name!r}" # Calls repr() on the ' - 'argument first\n' - '\n' - 'The *format_spec* field contains a specification of how ' - 'the value\n' - 'should be presented, including such details as field ' - 'width, alignment,\n' - 'padding, decimal precision and so on. Each value type ' - 'can define its\n' - 'own "formatting mini-language" or interpretation of the ' - '*format_spec*.\n' - '\n' - 'Most built-in types support a common formatting ' - 'mini-language, which\n' - 'is described in the next section.\n' - '\n' - 'A *format_spec* field can also include nested replacement ' - 'fields\n' - 'within it. These nested replacement fields can contain ' - 'only a field\n' - 'name; conversion flags and format specifications are not ' - 'allowed. The\n' - 'replacement fields within the format_spec are substituted ' - 'before the\n' - '*format_spec* string is interpreted. This allows the ' - 'formatting of a\n' - 'value to be dynamically specified.\n' - '\n' - 'See the Format examples section for some examples.\n' - '\n' - '\n' - 'Format Specification Mini-Language\n' - '==================================\n' - '\n' - '"Format specifications" are used within replacement ' - 'fields contained\n' - 'within a format string to define how individual values ' - 'are presented\n' - '(see Format String Syntax). They can also be passed ' - 'directly to the\n' - 'built-in "format()" function. Each formattable type may ' - 'define how\n' - 'the format specification is to be interpreted.\n' - '\n' - 'Most built-in types implement the following options for ' - 'format\n' - 'specifications, although some of the formatting options ' - 'are only\n' - 'supported by the numeric types.\n' - '\n' - 'A general convention is that an empty format string ' - '("""") produces\n' - 'the same result as if you had called "str()" on the ' - 'value. A non-empty\n' - 'format string typically modifies the result.\n' - '\n' - 'The general form of a *standard format specifier* is:\n' - '\n' - ' format_spec ::= ' - '[[fill]align][sign][#][0][width][,][.precision][type]\n' - ' fill ::= \n' - ' align ::= "<" | ">" | "=" | "^"\n' - ' sign ::= "+" | "-" | " "\n' - ' width ::= integer\n' - ' precision ::= integer\n' - ' type ::= "b" | "c" | "d" | "e" | "E" | "f" | ' - '"F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n' - '\n' - 'If a valid *align* value is specified, it can be preceded ' - 'by a *fill*\n' - 'character that can be any character and defaults to a ' - 'space if\n' - 'omitted. Note that it is not possible to use "{" and "}" ' - 'as *fill*\n' - 'char while using the "str.format()" method; this ' - 'limitation however\n' - 'doesn\'t affect the "format()" function.\n' - '\n' - 'The meaning of the various alignment options is as ' - 'follows:\n' - '\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | Option | ' - 'Meaning ' - '|\n' - ' ' - '+===========+============================================================+\n' - ' | "\'<\'" | Forces the field to be left-aligned ' - 'within the available |\n' - ' | | space (this is the default for most ' - 'objects). |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'>\'" | Forces the field to be right-aligned ' - 'within the available |\n' - ' | | space (this is the default for ' - 'numbers). |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'=\'" | Forces the padding to be placed after ' - 'the sign (if any) |\n' - ' | | but before the digits. This is used for ' - 'printing fields |\n' - " | | in the form '+000000120'. This alignment " - 'option is only |\n' - ' | | valid for numeric ' - 'types. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'^\'" | Forces the field to be centered within ' - 'the available |\n' - ' | | ' - 'space. ' - '|\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - '\n' - 'Note that unless a minimum field width is defined, the ' - 'field width\n' - 'will always be the same size as the data to fill it, so ' - 'that the\n' - 'alignment option has no meaning in this case.\n' - '\n' - 'The *sign* option is only valid for number types, and can ' - 'be one of\n' - 'the following:\n' - '\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | Option | ' - 'Meaning ' - '|\n' - ' ' - '+===========+============================================================+\n' - ' | "\'+\'" | indicates that a sign should be used ' - 'for both positive as |\n' - ' | | well as negative ' - 'numbers. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'-\'" | indicates that a sign should be used ' - 'only for negative |\n' - ' | | numbers (this is the default ' - 'behavior). |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | space | indicates that a leading space should be ' - 'used on positive |\n' - ' | | numbers, and a minus sign on negative ' - 'numbers. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - '\n' - 'The "\'#\'" option is only valid for integers, and only ' - 'for binary,\n' - 'octal, or hexadecimal output. If present, it specifies ' - 'that the\n' - 'output will be prefixed by "\'0b\'", "\'0o\'", or ' - '"\'0x\'", respectively.\n' - '\n' - 'The "\',\'" option signals the use of a comma for a ' - 'thousands separator.\n' - 'For a locale aware separator, use the "\'n\'" integer ' - 'presentation type\n' - 'instead.\n' - '\n' - 'Changed in version 2.7: Added the "\',\'" option (see ' - 'also **PEP 378**).\n' - '\n' - '*width* is a decimal integer defining the minimum field ' - 'width. If not\n' - 'specified, then the field width will be determined by the ' - 'content.\n' - '\n' - 'Preceding the *width* field by a zero ("\'0\'") character ' - 'enables sign-\n' - 'aware zero-padding for numeric types. This is equivalent ' - 'to a *fill*\n' - 'character of "\'0\'" with an *alignment* type of ' - '"\'=\'".\n' - '\n' - 'The *precision* is a decimal number indicating how many ' - 'digits should\n' - 'be displayed after the decimal point for a floating point ' - 'value\n' - 'formatted with "\'f\'" and "\'F\'", or before and after ' - 'the decimal point\n' - 'for a floating point value formatted with "\'g\'" or ' - '"\'G\'". For non-\n' - 'number types the field indicates the maximum field size - ' - 'in other\n' - 'words, how many characters will be used from the field ' - 'content. The\n' - '*precision* is not allowed for integer values.\n' - '\n' - 'Finally, the *type* determines how the data should be ' - 'presented.\n' - '\n' - 'The available string presentation types are:\n' - '\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | Type | ' - 'Meaning ' - '|\n' - ' ' - '+===========+============================================================+\n' - ' | "\'s\'" | String format. This is the default ' - 'type for strings and |\n' - ' | | may be ' - 'omitted. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | None | The same as ' - '"\'s\'". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - '\n' - 'The available integer presentation types are:\n' - '\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | Type | ' - 'Meaning ' - '|\n' - ' ' - '+===========+============================================================+\n' - ' | "\'b\'" | Binary format. Outputs the number in ' - 'base 2. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'c\'" | Character. Converts the integer to the ' - 'corresponding |\n' - ' | | unicode character before ' - 'printing. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'d\'" | Decimal Integer. Outputs the number in ' - 'base 10. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'o\'" | Octal format. Outputs the number in ' - 'base 8. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'x\'" | Hex format. Outputs the number in base ' - '16, using lower- |\n' - ' | | case letters for the digits above ' - '9. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'X\'" | Hex format. Outputs the number in base ' - '16, using upper- |\n' - ' | | case letters for the digits above ' - '9. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'n\'" | Number. This is the same as "\'d\'", ' - 'except that it uses the |\n' - ' | | current locale setting to insert the ' - 'appropriate number |\n' - ' | | separator ' - 'characters. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | None | The same as ' - '"\'d\'". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - '\n' - 'In addition to the above presentation types, integers can ' - 'be formatted\n' - 'with the floating point presentation types listed below ' - '(except "\'n\'"\n' - 'and None). When doing so, "float()" is used to convert ' - 'the integer to\n' - 'a floating point number before formatting.\n' - '\n' - 'The available presentation types for floating point and ' - 'decimal values\n' - 'are:\n' - '\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | Type | ' - 'Meaning ' - '|\n' - ' ' - '+===========+============================================================+\n' - ' | "\'e\'" | Exponent notation. Prints the number ' - 'in scientific |\n' - " | | notation using the letter 'e' to " - 'indicate the exponent. |\n' - ' | | The default precision is ' - '"6". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'E\'" | Exponent notation. Same as "\'e\'" ' - 'except it uses an upper |\n' - " | | case 'E' as the separator " - 'character. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'f\'" | Fixed point. Displays the number as a ' - 'fixed-point number. |\n' - ' | | The default precision is ' - '"6". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'F\'" | Fixed point. Same as ' - '"\'f\'". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'g\'" | General format. For a given precision ' - '"p >= 1", this |\n' - ' | | rounds the number to "p" significant ' - 'digits and then |\n' - ' | | formats the result in either fixed-point ' - 'format or in |\n' - ' | | scientific notation, depending on its ' - 'magnitude. The |\n' - ' | | precise rules are as follows: suppose ' - 'that the result |\n' - ' | | formatted with presentation type "\'e\'" ' - 'and precision "p-1" |\n' - ' | | would have exponent "exp". Then if "-4 ' - '<= exp < p", the |\n' - ' | | number is formatted with presentation ' - 'type "\'f\'" and |\n' - ' | | precision "p-1-exp". Otherwise, the ' - 'number is formatted |\n' - ' | | with presentation type "\'e\'" and ' - 'precision "p-1". In both |\n' - ' | | cases insignificant trailing zeros are ' - 'removed from the |\n' - ' | | significand, and the decimal point is ' - 'also removed if |\n' - ' | | there are no remaining digits following ' - 'it. Positive and |\n' - ' | | negative infinity, positive and negative ' - 'zero, and nans, |\n' - ' | | are formatted as "inf", "-inf", "0", ' - '"-0" and "nan" |\n' - ' | | respectively, regardless of the ' - 'precision. A precision of |\n' - ' | | "0" is treated as equivalent to a ' - 'precision of "1". The |\n' - ' | | default precision is ' - '"6". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'G\'" | General format. Same as "\'g\'" except ' - 'switches to "\'E\'" if |\n' - ' | | the number gets too large. The ' - 'representations of infinity |\n' - ' | | and NaN are uppercased, ' - 'too. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'n\'" | Number. This is the same as "\'g\'", ' - 'except that it uses the |\n' - ' | | current locale setting to insert the ' - 'appropriate number |\n' - ' | | separator ' - 'characters. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | "\'%\'" | Percentage. Multiplies the number by ' - '100 and displays in |\n' - ' | | fixed ("\'f\'") format, followed by a ' - 'percent sign. |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - ' | None | The same as ' - '"\'g\'". |\n' - ' ' - '+-----------+------------------------------------------------------------+\n' - '\n' - '\n' - 'Format examples\n' - '===============\n' - '\n' - 'This section contains examples of the new format syntax ' - 'and comparison\n' - 'with the old "%"-formatting.\n' - '\n' - 'In most of the cases the syntax is similar to the old ' - '"%"-formatting,\n' - 'with the addition of the "{}" and with ":" used instead ' - 'of "%". For\n' - 'example, "\'%03.2f\'" can be translated to ' - '"\'{:03.2f}\'".\n' - '\n' - 'The new format syntax also supports new and different ' - 'options, shown\n' - 'in the follow examples.\n' - '\n' - 'Accessing arguments by position:\n' - '\n' - " >>> '{0}, {1}, {2}'.format('a', 'b', 'c')\n" - " 'a, b, c'\n" - " >>> '{}, {}, {}'.format('a', 'b', 'c') # 2.7+ only\n" - " 'a, b, c'\n" - " >>> '{2}, {1}, {0}'.format('a', 'b', 'c')\n" - " 'c, b, a'\n" - " >>> '{2}, {1}, {0}'.format(*'abc') # unpacking " - 'argument sequence\n' - " 'c, b, a'\n" - " >>> '{0}{1}{0}'.format('abra', 'cad') # arguments' " - 'indices can be repeated\n' - " 'abracadabra'\n" - '\n' - 'Accessing arguments by name:\n' - '\n' - " >>> 'Coordinates: {latitude}, " - "{longitude}'.format(latitude='37.24N', " - "longitude='-115.81W')\n" - " 'Coordinates: 37.24N, -115.81W'\n" - " >>> coord = {'latitude': '37.24N', 'longitude': " - "'-115.81W'}\n" - " >>> 'Coordinates: {latitude}, " - "{longitude}'.format(**coord)\n" - " 'Coordinates: 37.24N, -115.81W'\n" - '\n' - "Accessing arguments' attributes:\n" - '\n' - ' >>> c = 3-5j\n' - " >>> ('The complex number {0} is formed from the real " - "part {0.real} '\n" - " ... 'and the imaginary part {0.imag}.').format(c)\n" - " 'The complex number (3-5j) is formed from the real " - "part 3.0 and the imaginary part -5.0.'\n" - ' >>> class Point(object):\n' - ' ... def __init__(self, x, y):\n' - ' ... self.x, self.y = x, y\n' - ' ... def __str__(self):\n' - " ... return 'Point({self.x}, " - "{self.y})'.format(self=self)\n" - ' ...\n' - ' >>> str(Point(4, 2))\n' - " 'Point(4, 2)'\n" - '\n' - "Accessing arguments' items:\n" - '\n' - ' >>> coord = (3, 5)\n' - " >>> 'X: {0[0]}; Y: {0[1]}'.format(coord)\n" - " 'X: 3; Y: 5'\n" - '\n' - 'Replacing "%s" and "%r":\n' - '\n' - ' >>> "repr() shows quotes: {!r}; str() doesn\'t: ' - '{!s}".format(\'test1\', \'test2\')\n' - ' "repr() shows quotes: \'test1\'; str() doesn\'t: ' - 'test2"\n' - '\n' - 'Aligning the text and specifying a width:\n' - '\n' - " >>> '{:<30}'.format('left aligned')\n" - " 'left aligned '\n" - " >>> '{:>30}'.format('right aligned')\n" - " ' right aligned'\n" - " >>> '{:^30}'.format('centered')\n" - " ' centered '\n" - " >>> '{:*^30}'.format('centered') # use '*' as a fill " - 'char\n' - " '***********centered***********'\n" - '\n' - 'Replacing "%+f", "%-f", and "% f" and specifying a sign:\n' - '\n' - " >>> '{:+f}; {:+f}'.format(3.14, -3.14) # show it " - 'always\n' - " '+3.140000; -3.140000'\n" - " >>> '{: f}; {: f}'.format(3.14, -3.14) # show a space " - 'for positive numbers\n' - " ' 3.140000; -3.140000'\n" - " >>> '{:-f}; {:-f}'.format(3.14, -3.14) # show only " - "the minus -- same as '{:f}; {:f}'\n" - " '3.140000; -3.140000'\n" - '\n' - 'Replacing "%x" and "%o" and converting the value to ' - 'different bases:\n' - '\n' - ' >>> # format also supports binary numbers\n' - ' >>> "int: {0:d}; hex: {0:x}; oct: {0:o}; bin: ' - '{0:b}".format(42)\n' - " 'int: 42; hex: 2a; oct: 52; bin: 101010'\n" - ' >>> # with 0x, 0o, or 0b as prefix:\n' - ' >>> "int: {0:d}; hex: {0:#x}; oct: {0:#o}; bin: ' - '{0:#b}".format(42)\n' - " 'int: 42; hex: 0x2a; oct: 0o52; bin: 0b101010'\n" - '\n' - 'Using the comma as a thousands separator:\n' - '\n' - " >>> '{:,}'.format(1234567890)\n" - " '1,234,567,890'\n" - '\n' - 'Expressing a percentage:\n' - '\n' - ' >>> points = 19.5\n' - ' >>> total = 22\n' - " >>> 'Correct answers: {:.2%}'.format(points/total)\n" - " 'Correct answers: 88.64%'\n" - '\n' - 'Using type-specific formatting:\n' - '\n' - ' >>> import datetime\n' - ' >>> d = datetime.datetime(2010, 7, 4, 12, 15, 58)\n' - " >>> '{:%Y-%m-%d %H:%M:%S}'.format(d)\n" - " '2010-07-04 12:15:58'\n" - '\n' - 'Nesting arguments and more complex examples:\n' - '\n' - " >>> for align, text in zip('<^>', ['left', 'center', " - "'right']):\n" - " ... '{0:{fill}{align}16}'.format(text, fill=align, " - 'align=align)\n' - ' ...\n' - " 'left<<<<<<<<<<<<'\n" - " '^^^^^center^^^^^'\n" - " '>>>>>>>>>>>right'\n" - ' >>>\n' - ' >>> octets = [192, 168, 0, 1]\n' - " >>> '{:02X}{:02X}{:02X}{:02X}'.format(*octets)\n" - " 'C0A80001'\n" - ' >>> int(_, 16)\n' - ' 3232235521\n' - ' >>>\n' - ' >>> width = 5\n' - ' >>> for num in range(5,12):\n' - " ... for base in 'dXob':\n" - " ... print '{0:{width}{base}}'.format(num, " - 'base=base, width=width),\n' - ' ... print\n' - ' ...\n' - ' 5 5 5 101\n' - ' 6 6 6 110\n' - ' 7 7 7 111\n' - ' 8 8 10 1000\n' - ' 9 9 11 1001\n' - ' 10 A 12 1010\n' - ' 11 B 13 1011\n', - 'function': '\n' - 'Function definitions\n' - '********************\n' - '\n' - 'A function definition defines a user-defined function object ' - '(see\n' - 'section The standard type hierarchy):\n' - '\n' - ' decorated ::= decorators (classdef | funcdef)\n' - ' decorators ::= decorator+\n' - ' decorator ::= "@" dotted_name ["(" [argument_list ' - '[","]] ")"] NEWLINE\n' - ' funcdef ::= "def" funcname "(" [parameter_list] ")" ' - '":" suite\n' - ' dotted_name ::= identifier ("." identifier)*\n' - ' parameter_list ::= (defparameter ",")*\n' - ' ( "*" identifier ["," "**" identifier]\n' - ' | "**" identifier\n' - ' | defparameter [","] )\n' - ' defparameter ::= parameter ["=" expression]\n' - ' sublist ::= parameter ("," parameter)* [","]\n' - ' parameter ::= identifier | "(" sublist ")"\n' - ' funcname ::= identifier\n' - '\n' - 'A function definition is an executable statement. Its ' - 'execution binds\n' - 'the function name in the current local namespace to a function ' - 'object\n' - '(a wrapper around the executable code for the function). ' - 'This\n' - 'function object contains a reference to the current global ' - 'namespace\n' - 'as the global namespace to be used when the function is ' - 'called.\n' - '\n' - 'The function definition does not execute the function body; ' - 'this gets\n' - 'executed only when the function is called. [3]\n' - '\n' - 'A function definition may be wrapped by one or more ' - '*decorator*\n' - 'expressions. Decorator expressions are evaluated when the ' - 'function is\n' - 'defined, in the scope that contains the function definition. ' - 'The\n' - 'result must be a callable, which is invoked with the function ' - 'object\n' - 'as the only argument. The returned value is bound to the ' - 'function name\n' - 'instead of the function object. Multiple decorators are ' - 'applied in\n' - 'nested fashion. For example, the following code:\n' - '\n' - ' @f1(arg)\n' - ' @f2\n' - ' def func(): pass\n' - '\n' - 'is equivalent to:\n' - '\n' - ' def func(): pass\n' - ' func = f1(arg)(f2(func))\n' - '\n' - 'When one or more top-level *parameters* have the form ' - '*parameter* "="\n' - '*expression*, the function is said to have "default parameter ' - 'values."\n' - 'For a parameter with a default value, the corresponding ' - '*argument* may\n' - "be omitted from a call, in which case the parameter's default " - 'value is\n' - 'substituted. If a parameter has a default value, all ' - 'following\n' - 'parameters must also have a default value --- this is a ' - 'syntactic\n' - 'restriction that is not expressed by the grammar.\n' - '\n' - '**Default parameter values are evaluated when the function ' - 'definition\n' - 'is executed.** This means that the expression is evaluated ' - 'once, when\n' - 'the function is defined, and that the same "pre-computed" ' - 'value is\n' - 'used for each call. This is especially important to ' - 'understand when a\n' - 'default parameter is a mutable object, such as a list or a ' - 'dictionary:\n' - 'if the function modifies the object (e.g. by appending an item ' - 'to a\n' - 'list), the default value is in effect modified. This is ' - 'generally not\n' - 'what was intended. A way around this is to use "None" as ' - 'the\n' - 'default, and explicitly test for it in the body of the ' - 'function, e.g.:\n' - '\n' - ' def whats_on_the_telly(penguin=None):\n' - ' if penguin is None:\n' - ' penguin = []\n' - ' penguin.append("property of the zoo")\n' - ' return penguin\n' - '\n' - 'Function call semantics are described in more detail in ' - 'section Calls.\n' - 'A function call always assigns values to all parameters ' - 'mentioned in\n' - 'the parameter list, either from position arguments, from ' - 'keyword\n' - 'arguments, or from default values. If the form ' - '""*identifier"" is\n' - 'present, it is initialized to a tuple receiving any excess ' - 'positional\n' - 'parameters, defaulting to the empty tuple. If the form\n' - '""**identifier"" is present, it is initialized to a new ' - 'dictionary\n' - 'receiving any excess keyword arguments, defaulting to a new ' - 'empty\n' - 'dictionary.\n' - '\n' - 'It is also possible to create anonymous functions (functions ' - 'not bound\n' - 'to a name), for immediate use in expressions. This uses ' - 'lambda\n' - 'expressions, described in section Lambdas. Note that the ' - 'lambda\n' - 'expression is merely a shorthand for a simplified function ' - 'definition;\n' - 'a function defined in a ""def"" statement can be passed around ' - 'or\n' - 'assigned to another name just like a function defined by a ' - 'lambda\n' - 'expression. The ""def"" form is actually more powerful since ' - 'it\n' - 'allows the execution of multiple statements.\n' - '\n' - "**Programmer's note:** Functions are first-class objects. A " - '""def""\n' - 'form executed inside a function definition defines a local ' - 'function\n' - 'that can be returned or passed around. Free variables used in ' - 'the\n' - 'nested function can access the local variables of the ' - 'function\n' - 'containing the def. See section Naming and binding for ' - 'details.\n', - 'global': '\n' - 'The "global" statement\n' - '**********************\n' - '\n' - ' global_stmt ::= "global" identifier ("," identifier)*\n' - '\n' - 'The "global" statement is a declaration which holds for the ' - 'entire\n' - 'current code block. It means that the listed identifiers are to ' - 'be\n' - 'interpreted as globals. It would be impossible to assign to a ' - 'global\n' - 'variable without "global", although free variables may refer to\n' - 'globals without being declared global.\n' - '\n' - 'Names listed in a "global" statement must not be used in the ' - 'same code\n' - 'block textually preceding that "global" statement.\n' - '\n' - 'Names listed in a "global" statement must not be defined as ' - 'formal\n' - 'parameters or in a "for" loop control target, "class" ' - 'definition,\n' - 'function definition, or "import" statement.\n' - '\n' - '**CPython implementation detail:** The current implementation ' - 'does not\n' - 'enforce the latter two restrictions, but programs should not ' - 'abuse\n' - 'this freedom, as future implementations may enforce them or ' - 'silently\n' - 'change the meaning of the program.\n' - '\n' - '**Programmer\'s note:** the "global" is a directive to the ' - 'parser. It\n' - 'applies only to code parsed at the same time as the "global"\n' - 'statement. In particular, a "global" statement contained in an ' - '"exec"\n' - 'statement does not affect the code block *containing* the ' - '"exec"\n' - 'statement, and code contained in an "exec" statement is ' - 'unaffected by\n' - '"global" statements in the code containing the "exec" ' - 'statement. The\n' - 'same applies to the "eval()", "execfile()" and "compile()" ' - 'functions.\n', - 'id-classes': '\n' - 'Reserved classes of identifiers\n' - '*******************************\n' - '\n' - 'Certain classes of identifiers (besides keywords) have ' - 'special\n' - 'meanings. These classes are identified by the patterns of ' - 'leading and\n' - 'trailing underscore characters:\n' - '\n' - '"_*"\n' - ' Not imported by "from module import *". The special ' - 'identifier "_"\n' - ' is used in the interactive interpreter to store the ' - 'result of the\n' - ' last evaluation; it is stored in the "__builtin__" ' - 'module. When\n' - ' not in interactive mode, "_" has no special meaning and ' - 'is not\n' - ' defined. See section The import statement.\n' - '\n' - ' Note: The name "_" is often used in conjunction with\n' - ' internationalization; refer to the documentation for ' - 'the\n' - ' "gettext" module for more information on this ' - 'convention.\n' - '\n' - '"__*__"\n' - ' System-defined names. These names are defined by the ' - 'interpreter\n' - ' and its implementation (including the standard library). ' - 'Current\n' - ' system names are discussed in the Special method names ' - 'section and\n' - ' elsewhere. More will likely be defined in future ' - 'versions of\n' - ' Python. *Any* use of "__*__" names, in any context, that ' - 'does not\n' - ' follow explicitly documented use, is subject to breakage ' - 'without\n' - ' warning.\n' - '\n' - '"__*"\n' - ' Class-private names. Names in this category, when used ' - 'within the\n' - ' context of a class definition, are re-written to use a ' - 'mangled form\n' - ' to help avoid name clashes between "private" attributes ' - 'of base and\n' - ' derived classes. See section Identifiers (Names).\n', - 'identifiers': '\n' - 'Identifiers and keywords\n' - '************************\n' - '\n' - 'Identifiers (also referred to as *names*) are described by ' - 'the\n' - 'following lexical definitions:\n' - '\n' - ' identifier ::= (letter|"_") (letter | digit | "_")*\n' - ' letter ::= lowercase | uppercase\n' - ' lowercase ::= "a"..."z"\n' - ' uppercase ::= "A"..."Z"\n' - ' digit ::= "0"..."9"\n' - '\n' - 'Identifiers are unlimited in length. Case is significant.\n' - '\n' - '\n' - 'Keywords\n' - '========\n' - '\n' - 'The following identifiers are used as reserved words, or ' - '*keywords* of\n' - 'the language, and cannot be used as ordinary identifiers. ' - 'They must\n' - 'be spelled exactly as written here:\n' - '\n' - ' and del from not while\n' - ' as elif global or with\n' - ' assert else if pass yield\n' - ' break except import print\n' - ' class exec in raise\n' - ' continue finally is return\n' - ' def for lambda try\n' - '\n' - 'Changed in version 2.4: "None" became a constant and is now ' - 'recognized\n' - 'by the compiler as a name for the built-in object "None". ' - 'Although it\n' - 'is not a keyword, you cannot assign a different object to ' - 'it.\n' - '\n' - 'Changed in version 2.5: Using "as" and "with" as ' - 'identifiers triggers\n' - 'a warning. To use them as keywords, enable the ' - '"with_statement"\n' - 'future feature .\n' - '\n' - 'Changed in version 2.6: "as" and "with" are full keywords.\n' - '\n' - '\n' - 'Reserved classes of identifiers\n' - '===============================\n' - '\n' - 'Certain classes of identifiers (besides keywords) have ' - 'special\n' - 'meanings. These classes are identified by the patterns of ' - 'leading and\n' - 'trailing underscore characters:\n' - '\n' - '"_*"\n' - ' Not imported by "from module import *". The special ' - 'identifier "_"\n' - ' is used in the interactive interpreter to store the ' - 'result of the\n' - ' last evaluation; it is stored in the "__builtin__" ' - 'module. When\n' - ' not in interactive mode, "_" has no special meaning and ' - 'is not\n' - ' defined. See section The import statement.\n' - '\n' - ' Note: The name "_" is often used in conjunction with\n' - ' internationalization; refer to the documentation for ' - 'the\n' - ' "gettext" module for more information on this ' - 'convention.\n' - '\n' - '"__*__"\n' - ' System-defined names. These names are defined by the ' - 'interpreter\n' - ' and its implementation (including the standard ' - 'library). Current\n' - ' system names are discussed in the Special method names ' - 'section and\n' - ' elsewhere. More will likely be defined in future ' - 'versions of\n' - ' Python. *Any* use of "__*__" names, in any context, ' - 'that does not\n' - ' follow explicitly documented use, is subject to breakage ' - 'without\n' - ' warning.\n' - '\n' - '"__*"\n' - ' Class-private names. Names in this category, when used ' - 'within the\n' - ' context of a class definition, are re-written to use a ' - 'mangled form\n' - ' to help avoid name clashes between "private" attributes ' - 'of base and\n' - ' derived classes. See section Identifiers (Names).\n', - 'if': '\n' - 'The "if" statement\n' - '******************\n' - '\n' - 'The "if" statement is used for conditional execution:\n' - '\n' - ' if_stmt ::= "if" expression ":" suite\n' - ' ( "elif" expression ":" suite )*\n' - ' ["else" ":" suite]\n' - '\n' - 'It selects exactly one of the suites by evaluating the expressions ' - 'one\n' - 'by one until one is found to be true (see section Boolean ' - 'operations\n' - 'for the definition of true and false); then that suite is executed\n' - '(and no other part of the "if" statement is executed or evaluated).\n' - 'If all expressions are false, the suite of the "else" clause, if\n' - 'present, is executed.\n', - 'imaginary': '\n' - 'Imaginary literals\n' - '******************\n' - '\n' - 'Imaginary literals are described by the following lexical ' - 'definitions:\n' - '\n' - ' imagnumber ::= (floatnumber | intpart) ("j" | "J")\n' - '\n' - 'An imaginary literal yields a complex number with a real part ' - 'of 0.0.\n' - 'Complex numbers are represented as a pair of floating point ' - 'numbers\n' - 'and have the same restrictions on their range. To create a ' - 'complex\n' - 'number with a nonzero real part, add a floating point number ' - 'to it,\n' - 'e.g., "(3+4j)". Some examples of imaginary literals:\n' - '\n' - ' 3.14j 10.j 10j .001j 1e100j 3.14e-10j\n', - 'import': '\n' - 'The "import" statement\n' - '**********************\n' - '\n' - ' import_stmt ::= "import" module ["as" name] ( "," module ' - '["as" name] )*\n' - ' | "from" relative_module "import" identifier ' - '["as" name]\n' - ' ( "," identifier ["as" name] )*\n' - ' | "from" relative_module "import" "(" ' - 'identifier ["as" name]\n' - ' ( "," identifier ["as" name] )* [","] ")"\n' - ' | "from" module "import" "*"\n' - ' module ::= (identifier ".")* identifier\n' - ' relative_module ::= "."* module | "."+\n' - ' name ::= identifier\n' - '\n' - 'Import statements are executed in two steps: (1) find a module, ' - 'and\n' - 'initialize it if necessary; (2) define a name or names in the ' - 'local\n' - 'namespace (of the scope where the "import" statement occurs). ' - 'The\n' - 'statement comes in two forms differing on whether it uses the ' - '"from"\n' - 'keyword. The first form (without "from") repeats these steps for ' - 'each\n' - 'identifier in the list. The form with "from" performs step (1) ' - 'once,\n' - 'and then performs step (2) repeatedly.\n' - '\n' - 'To understand how step (1) occurs, one must first understand ' - 'how\n' - 'Python handles hierarchical naming of modules. To help organize\n' - 'modules and provide a hierarchy in naming, Python has a concept ' - 'of\n' - 'packages. A package can contain other packages and modules ' - 'while\n' - 'modules cannot contain other modules or packages. From a file ' - 'system\n' - 'perspective, packages are directories and modules are files.\n' - '\n' - 'Once the name of the module is known (unless otherwise ' - 'specified, the\n' - 'term "module" will refer to both packages and modules), ' - 'searching for\n' - 'the module or package can begin. The first place checked is\n' - '"sys.modules", the cache of all modules that have been imported\n' - 'previously. If the module is found there then it is used in step ' - '(2)\n' - 'of import.\n' - '\n' - 'If the module is not found in the cache, then "sys.meta_path" ' - 'is\n' - 'searched (the specification for "sys.meta_path" can be found in ' - '**PEP\n' - '302**). The object is a list of *finder* objects which are ' - 'queried in\n' - 'order as to whether they know how to load the module by calling ' - 'their\n' - '"find_module()" method with the name of the module. If the ' - 'module\n' - 'happens to be contained within a package (as denoted by the ' - 'existence\n' - 'of a dot in the name), then a second argument to "find_module()" ' - 'is\n' - 'given as the value of the "__path__" attribute from the parent ' - 'package\n' - '(everything up to the last dot in the name of the module being\n' - 'imported). If a finder can find the module it returns a ' - '*loader*\n' - '(discussed later) or returns "None".\n' - '\n' - 'If none of the finders on "sys.meta_path" are able to find the ' - 'module\n' - 'then some implicitly defined finders are queried. ' - 'Implementations of\n' - 'Python vary in what implicit meta path finders are defined. The ' - 'one\n' - 'they all do define, though, is one that handles ' - '"sys.path_hooks",\n' - '"sys.path_importer_cache", and "sys.path".\n' - '\n' - 'The implicit finder searches for the requested module in the ' - '"paths"\n' - 'specified in one of two places ("paths" do not have to be file ' - 'system\n' - 'paths). If the module being imported is supposed to be ' - 'contained\n' - 'within a package then the second argument passed to ' - '"find_module()",\n' - '"__path__" on the parent package, is used as the source of ' - 'paths. If\n' - 'the module is not contained in a package then "sys.path" is used ' - 'as\n' - 'the source of paths.\n' - '\n' - 'Once the source of paths is chosen it is iterated over to find ' - 'a\n' - 'finder that can handle that path. The dict at\n' - '"sys.path_importer_cache" caches finders for paths and is ' - 'checked for\n' - 'a finder. If the path does not have a finder cached then\n' - '"sys.path_hooks" is searched by calling each object in the list ' - 'with a\n' - 'single argument of the path, returning a finder or raises\n' - '"ImportError". If a finder is returned then it is cached in\n' - '"sys.path_importer_cache" and then used for that path entry. If ' - 'no\n' - 'finder can be found but the path exists then a value of "None" ' - 'is\n' - 'stored in "sys.path_importer_cache" to signify that an implicit, ' - 'file-\n' - 'based finder that handles modules stored as individual files ' - 'should be\n' - 'used for that path. If the path does not exist then a finder ' - 'which\n' - 'always returns "None" is placed in the cache for the path.\n' - '\n' - 'If no finder can find the module then "ImportError" is raised.\n' - 'Otherwise some finder returned a loader whose "load_module()" ' - 'method\n' - 'is called with the name of the module to load (see **PEP 302** ' - 'for the\n' - 'original definition of loaders). A loader has several ' - 'responsibilities\n' - 'to perform on a module it loads. First, if the module already ' - 'exists\n' - 'in "sys.modules" (a possibility if the loader is called outside ' - 'of the\n' - 'import machinery) then it is to use that module for ' - 'initialization and\n' - 'not a new module. But if the module does not exist in ' - '"sys.modules"\n' - 'then it is to be added to that dict before initialization ' - 'begins. If\n' - 'an error occurs during loading of the module and it was added ' - 'to\n' - '"sys.modules" it is to be removed from the dict. If an error ' - 'occurs\n' - 'but the module was already in "sys.modules" it is left in the ' - 'dict.\n' - '\n' - 'The loader must set several attributes on the module. "__name__" ' - 'is to\n' - 'be set to the name of the module. "__file__" is to be the "path" ' - 'to\n' - 'the file unless the module is built-in (and thus listed in\n' - '"sys.builtin_module_names") in which case the attribute is not ' - 'set. If\n' - 'what is being imported is a package then "__path__" is to be set ' - 'to a\n' - 'list of paths to be searched when looking for modules and ' - 'packages\n' - 'contained within the package being imported. "__package__" is ' - 'optional\n' - 'but should be set to the name of package that contains the ' - 'module or\n' - 'package (the empty string is used for module not contained in a\n' - 'package). "__loader__" is also optional but should be set to ' - 'the\n' - 'loader object that is loading the module.\n' - '\n' - 'If an error occurs during loading then the loader raises ' - '"ImportError"\n' - 'if some other exception is not already being propagated. ' - 'Otherwise the\n' - 'loader returns the module that was loaded and initialized.\n' - '\n' - 'When step (1) finishes without raising an exception, step (2) ' - 'can\n' - 'begin.\n' - '\n' - 'The first form of "import" statement binds the module name in ' - 'the\n' - 'local namespace to the module object, and then goes on to import ' - 'the\n' - 'next identifier, if any. If the module name is followed by ' - '"as", the\n' - 'name following "as" is used as the local name for the module.\n' - '\n' - 'The "from" form does not bind the module name: it goes through ' - 'the\n' - 'list of identifiers, looks each one of them up in the module ' - 'found in\n' - 'step (1), and binds the name in the local namespace to the ' - 'object thus\n' - 'found. As with the first form of "import", an alternate local ' - 'name\n' - 'can be supplied by specifying ""as" localname". If a name is ' - 'not\n' - 'found, "ImportError" is raised. If the list of identifiers is\n' - 'replaced by a star ("\'*\'"), all public names defined in the ' - 'module are\n' - 'bound in the local namespace of the "import" statement..\n' - '\n' - 'The *public names* defined by a module are determined by ' - 'checking the\n' - 'module\'s namespace for a variable named "__all__"; if defined, ' - 'it must\n' - 'be a sequence of strings which are names defined or imported by ' - 'that\n' - 'module. The names given in "__all__" are all considered public ' - 'and\n' - 'are required to exist. If "__all__" is not defined, the set of ' - 'public\n' - "names includes all names found in the module's namespace which " - 'do not\n' - 'begin with an underscore character ("\'_\'"). "__all__" should ' - 'contain\n' - 'the entire public API. It is intended to avoid accidentally ' - 'exporting\n' - 'items that are not part of the API (such as library modules ' - 'which were\n' - 'imported and used within the module).\n' - '\n' - 'The "from" form with "*" may only occur in a module scope. If ' - 'the\n' - 'wild card form of import --- "import *" --- is used in a ' - 'function and\n' - 'the function contains or is a nested block with free variables, ' - 'the\n' - 'compiler will raise a "SyntaxError".\n' - '\n' - 'When specifying what module to import you do not have to specify ' - 'the\n' - 'absolute name of the module. When a module or package is ' - 'contained\n' - 'within another package it is possible to make a relative import ' - 'within\n' - 'the same top package without having to mention the package name. ' - 'By\n' - 'using leading dots in the specified module or package after ' - '"from" you\n' - 'can specify how high to traverse up the current package ' - 'hierarchy\n' - 'without specifying exact names. One leading dot means the ' - 'current\n' - 'package where the module making the import exists. Two dots ' - 'means up\n' - 'one package level. Three dots is up two levels, etc. So if you ' - 'execute\n' - '"from . import mod" from a module in the "pkg" package then you ' - 'will\n' - 'end up importing "pkg.mod". If you execute "from ..subpkg2 ' - 'import mod"\n' - 'from within "pkg.subpkg1" you will import "pkg.subpkg2.mod". ' - 'The\n' - 'specification for relative imports is contained within **PEP ' - '328**.\n' - '\n' - '"importlib.import_module()" is provided to support applications ' - 'that\n' - 'determine which modules need to be loaded dynamically.\n' - '\n' - '\n' - 'Future statements\n' - '=================\n' - '\n' - 'A *future statement* is a directive to the compiler that a ' - 'particular\n' - 'module should be compiled using syntax or semantics that will ' - 'be\n' - 'available in a specified future release of Python. The future\n' - 'statement is intended to ease migration to future versions of ' - 'Python\n' - 'that introduce incompatible changes to the language. It allows ' - 'use of\n' - 'the new features on a per-module basis before the release in ' - 'which the\n' - 'feature becomes standard.\n' - '\n' - ' future_statement ::= "from" "__future__" "import" feature ' - '["as" name]\n' - ' ("," feature ["as" name])*\n' - ' | "from" "__future__" "import" "(" ' - 'feature ["as" name]\n' - ' ("," feature ["as" name])* [","] ")"\n' - ' feature ::= identifier\n' - ' name ::= identifier\n' - '\n' - 'A future statement must appear near the top of the module. The ' - 'only\n' - 'lines that can appear before a future statement are:\n' - '\n' - '* the module docstring (if any),\n' - '\n' - '* comments,\n' - '\n' - '* blank lines, and\n' - '\n' - '* other future statements.\n' - '\n' - 'The features recognized by Python 2.6 are "unicode_literals",\n' - '"print_function", "absolute_import", "division", "generators",\n' - '"nested_scopes" and "with_statement". "generators", ' - '"with_statement",\n' - '"nested_scopes" are redundant in Python version 2.6 and above ' - 'because\n' - 'they are always enabled.\n' - '\n' - 'A future statement is recognized and treated specially at ' - 'compile\n' - 'time: Changes to the semantics of core constructs are often\n' - 'implemented by generating different code. It may even be the ' - 'case\n' - 'that a new feature introduces new incompatible syntax (such as a ' - 'new\n' - 'reserved word), in which case the compiler may need to parse ' - 'the\n' - 'module differently. Such decisions cannot be pushed off until\n' - 'runtime.\n' - '\n' - 'For any given release, the compiler knows which feature names ' - 'have\n' - 'been defined, and raises a compile-time error if a future ' - 'statement\n' - 'contains a feature not known to it.\n' - '\n' - 'The direct runtime semantics are the same as for any import ' - 'statement:\n' - 'there is a standard module "__future__", described later, and it ' - 'will\n' - 'be imported in the usual way at the time the future statement ' - 'is\n' - 'executed.\n' - '\n' - 'The interesting runtime semantics depend on the specific ' - 'feature\n' - 'enabled by the future statement.\n' - '\n' - 'Note that there is nothing special about the statement:\n' - '\n' - ' import __future__ [as name]\n' - '\n' - "That is not a future statement; it's an ordinary import " - 'statement with\n' - 'no special semantics or syntax restrictions.\n' - '\n' - 'Code compiled by an "exec" statement or calls to the built-in\n' - 'functions "compile()" and "execfile()" that occur in a module ' - '"M"\n' - 'containing a future statement will, by default, use the new ' - 'syntax or\n' - 'semantics associated with the future statement. This can, ' - 'starting\n' - 'with Python 2.2 be controlled by optional arguments to ' - '"compile()" ---\n' - 'see the documentation of that function for details.\n' - '\n' - 'A future statement typed at an interactive interpreter prompt ' - 'will\n' - 'take effect for the rest of the interpreter session. If an\n' - 'interpreter is started with the "-i" option, is passed a script ' - 'name\n' - 'to execute, and the script includes a future statement, it will ' - 'be in\n' - 'effect in the interactive session started after the script is\n' - 'executed.\n' - '\n' - 'See also: **PEP 236** - Back to the __future__\n' - '\n' - ' The original proposal for the __future__ mechanism.\n', - 'in': '\n' - 'Comparisons\n' - '***********\n' - '\n' - 'Unlike C, all comparison operations in Python have the same ' - 'priority,\n' - 'which is lower than that of any arithmetic, shifting or bitwise\n' - 'operation. Also unlike C, expressions like "a < b < c" have the\n' - 'interpretation that is conventional in mathematics:\n' - '\n' - ' comparison ::= or_expr ( comp_operator or_expr )*\n' - ' comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "<>" | "!="\n' - ' | "is" ["not"] | ["not"] "in"\n' - '\n' - 'Comparisons yield boolean values: "True" or "False".\n' - '\n' - 'Comparisons can be chained arbitrarily, e.g., "x < y <= z" is\n' - 'equivalent to "x < y and y <= z", except that "y" is evaluated only\n' - 'once (but in both cases "z" is not evaluated at all when "x < y" is\n' - 'found to be false).\n' - '\n' - 'Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and ' - '*op1*,\n' - '*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... ' - 'y\n' - 'opN z" is equivalent to "a op1 b and b op2 c and ... y opN z", ' - 'except\n' - 'that each expression is evaluated at most once.\n' - '\n' - 'Note that "a op1 b op2 c" doesn\'t imply any kind of comparison ' - 'between\n' - '*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\n' - 'perhaps not pretty).\n' - '\n' - 'The forms "<>" and "!=" are equivalent; for consistency with C, ' - '"!="\n' - 'is preferred; where "!=" is mentioned below "<>" is also accepted.\n' - 'The "<>" spelling is considered obsolescent.\n' - '\n' - 'The operators "<", ">", "==", ">=", "<=", and "!=" compare the ' - 'values\n' - 'of two objects. The objects need not have the same type. If both ' - 'are\n' - 'numbers, they are converted to a common type. Otherwise, objects ' - 'of\n' - 'different types *always* compare unequal, and are ordered ' - 'consistently\n' - 'but arbitrarily. You can control comparison behavior of objects of\n' - 'non-built-in types by defining a "__cmp__" method or rich ' - 'comparison\n' - 'methods like "__gt__", described in section Special method names.\n' - '\n' - '(This unusual definition of comparison was used to simplify the\n' - 'definition of operations like sorting and the "in" and "not in"\n' - 'operators. In the future, the comparison rules for objects of\n' - 'different types are likely to change.)\n' - '\n' - 'Comparison of objects of the same type depends on the type:\n' - '\n' - '* Numbers are compared arithmetically.\n' - '\n' - '* Strings are compared lexicographically using the numeric\n' - ' equivalents (the result of the built-in function "ord()") of ' - 'their\n' - ' characters. Unicode and 8-bit strings are fully interoperable in\n' - ' this behavior. [4]\n' - '\n' - '* Tuples and lists are compared lexicographically using comparison\n' - ' of corresponding elements. This means that to compare equal, ' - 'each\n' - ' element must compare equal and the two sequences must be of the ' - 'same\n' - ' type and have the same length.\n' - '\n' - ' If not equal, the sequences are ordered the same as their first\n' - ' differing elements. For example, "cmp([1,2,x], [1,2,y])" returns\n' - ' the same as "cmp(x,y)". If the corresponding element does not\n' - ' exist, the shorter sequence is ordered first (for example, "[1,2] ' - '<\n' - ' [1,2,3]").\n' - '\n' - '* Mappings (dictionaries) compare equal if and only if their sorted\n' - ' (key, value) lists compare equal. [5] Outcomes other than ' - 'equality\n' - ' are resolved consistently, but are not otherwise defined. [6]\n' - '\n' - '* Most other objects of built-in types compare unequal unless they\n' - ' are the same object; the choice whether one object is considered\n' - ' smaller or larger than another one is made arbitrarily but\n' - ' consistently within one execution of a program.\n' - '\n' - 'The operators "in" and "not in" test for collection membership. "x ' - 'in\n' - 's" evaluates to true if *x* is a member of the collection *s*, and\n' - 'false otherwise. "x not in s" returns the negation of "x in s". ' - 'The\n' - 'collection membership test has traditionally been bound to ' - 'sequences;\n' - 'an object is a member of a collection if the collection is a ' - 'sequence\n' - 'and contains an element equal to that object. However, it make ' - 'sense\n' - 'for many other object types to support membership tests without ' - 'being\n' - 'a sequence. In particular, dictionaries (for keys) and sets ' - 'support\n' - 'membership testing.\n' - '\n' - 'For the list and tuple types, "x in y" is true if and only if there\n' - 'exists an index *i* such that "x == y[i]" is true.\n' - '\n' - 'For the Unicode and string types, "x in y" is true if and only if ' - '*x*\n' - 'is a substring of *y*. An equivalent test is "y.find(x) != -1".\n' - 'Note, *x* and *y* need not be the same type; consequently, "u\'ab\' ' - 'in\n' - '\'abc\'" will return "True". Empty strings are always considered to ' - 'be a\n' - 'substring of any other string, so """ in "abc"" will return "True".\n' - '\n' - 'Changed in version 2.3: Previously, *x* was required to be a string ' - 'of\n' - 'length "1".\n' - '\n' - 'For user-defined classes which define the "__contains__()" method, ' - '"x\n' - 'in y" is true if and only if "y.__contains__(x)" is true.\n' - '\n' - 'For user-defined classes which do not define "__contains__()" but ' - 'do\n' - 'define "__iter__()", "x in y" is true if some value "z" with "x == ' - 'z"\n' - 'is produced while iterating over "y". If an exception is raised\n' - 'during the iteration, it is as if "in" raised that exception.\n' - '\n' - 'Lastly, the old-style iteration protocol is tried: if a class ' - 'defines\n' - '"__getitem__()", "x in y" is true if and only if there is a non-\n' - 'negative integer index *i* such that "x == y[i]", and all lower\n' - 'integer indices do not raise "IndexError" exception. (If any other\n' - 'exception is raised, it is as if "in" raised that exception).\n' - '\n' - 'The operator "not in" is defined to have the inverse true value of\n' - '"in".\n' - '\n' - 'The operators "is" and "is not" test for object identity: "x is y" ' - 'is\n' - 'true if and only if *x* and *y* are the same object. "x is not y"\n' - 'yields the inverse truth value. [7]\n', - 'integers': '\n' - 'Integer and long integer literals\n' - '*********************************\n' - '\n' - 'Integer and long integer literals are described by the ' - 'following\n' - 'lexical definitions:\n' - '\n' - ' longinteger ::= integer ("l" | "L")\n' - ' integer ::= decimalinteger | octinteger | hexinteger ' - '| bininteger\n' - ' decimalinteger ::= nonzerodigit digit* | "0"\n' - ' octinteger ::= "0" ("o" | "O") octdigit+ | "0" ' - 'octdigit+\n' - ' hexinteger ::= "0" ("x" | "X") hexdigit+\n' - ' bininteger ::= "0" ("b" | "B") bindigit+\n' - ' nonzerodigit ::= "1"..."9"\n' - ' octdigit ::= "0"..."7"\n' - ' bindigit ::= "0" | "1"\n' - ' hexdigit ::= digit | "a"..."f" | "A"..."F"\n' - '\n' - 'Although both lower case "\'l\'" and upper case "\'L\'" are ' - 'allowed as\n' - 'suffix for long integers, it is strongly recommended to always ' - 'use\n' - '"\'L\'", since the letter "\'l\'" looks too much like the ' - 'digit "\'1\'".\n' - '\n' - 'Plain integer literals that are above the largest ' - 'representable plain\n' - 'integer (e.g., 2147483647 when using 32-bit arithmetic) are ' - 'accepted\n' - 'as if they were long integers instead. [1] There is no limit ' - 'for long\n' - 'integer literals apart from what can be stored in available ' - 'memory.\n' - '\n' - 'Some examples of plain integer literals (first row) and long ' - 'integer\n' - 'literals (second and third rows):\n' - '\n' - ' 7 2147483647 0177\n' - ' 3L 79228162514264337593543950336L 0377L ' - '0x100000000L\n' - ' 79228162514264337593543950336 0xdeadbeef\n', - 'lambda': '\n' - 'Lambdas\n' - '*******\n' - '\n' - ' lambda_expr ::= "lambda" [parameter_list]: expression\n' - ' old_lambda_expr ::= "lambda" [parameter_list]: ' - 'old_expression\n' - '\n' - 'Lambda expressions (sometimes called lambda forms) have the ' - 'same\n' - 'syntactic position as expressions. They are a shorthand to ' - 'create\n' - 'anonymous functions; the expression "lambda arguments: ' - 'expression"\n' - 'yields a function object. The unnamed object behaves like a ' - 'function\n' - 'object defined with\n' - '\n' - ' def name(arguments):\n' - ' return expression\n' - '\n' - 'See section Function definitions for the syntax of parameter ' - 'lists.\n' - 'Note that functions created with lambda expressions cannot ' - 'contain\n' - 'statements.\n', - 'lists': '\n' - 'List displays\n' - '*************\n' - '\n' - 'A list display is a possibly empty series of expressions enclosed ' - 'in\n' - 'square brackets:\n' - '\n' - ' list_display ::= "[" [expression_list | ' - 'list_comprehension] "]"\n' - ' list_comprehension ::= expression list_for\n' - ' list_for ::= "for" target_list "in" ' - 'old_expression_list [list_iter]\n' - ' old_expression_list ::= old_expression [("," old_expression)+ ' - '[","]]\n' - ' old_expression ::= or_test | old_lambda_expr\n' - ' list_iter ::= list_for | list_if\n' - ' list_if ::= "if" old_expression [list_iter]\n' - '\n' - 'A list display yields a new list object. Its contents are ' - 'specified\n' - 'by providing either a list of expressions or a list ' - 'comprehension.\n' - 'When a comma-separated list of expressions is supplied, its ' - 'elements\n' - 'are evaluated from left to right and placed into the list object ' - 'in\n' - 'that order. When a list comprehension is supplied, it consists ' - 'of a\n' - 'single expression followed by at least one "for" clause and zero ' - 'or\n' - 'more "for" or "if" clauses. In this case, the elements of the ' - 'new\n' - 'list are those that would be produced by considering each of the ' - '"for"\n' - 'or "if" clauses a block, nesting from left to right, and ' - 'evaluating\n' - 'the expression to produce a list element each time the innermost ' - 'block\n' - 'is reached [1].\n', - 'naming': '\n' - 'Naming and binding\n' - '******************\n' - '\n' - '*Names* refer to objects. Names are introduced by name binding\n' - 'operations. Each occurrence of a name in the program text refers ' - 'to\n' - 'the *binding* of that name established in the innermost function ' - 'block\n' - 'containing the use.\n' - '\n' - 'A *block* is a piece of Python program text that is executed as ' - 'a\n' - 'unit. The following are blocks: a module, a function body, and a ' - 'class\n' - 'definition. Each command typed interactively is a block. A ' - 'script\n' - 'file (a file given as standard input to the interpreter or ' - 'specified\n' - 'on the interpreter command line the first argument) is a code ' - 'block.\n' - 'A script command (a command specified on the interpreter command ' - 'line\n' - "with the '**-c**' option) is a code block. The file read by " - 'the\n' - 'built-in function "execfile()" is a code block. The string ' - 'argument\n' - 'passed to the built-in function "eval()" and to the "exec" ' - 'statement\n' - 'is a code block. The expression read and evaluated by the ' - 'built-in\n' - 'function "input()" is a code block.\n' - '\n' - 'A code block is executed in an *execution frame*. A frame ' - 'contains\n' - 'some administrative information (used for debugging) and ' - 'determines\n' - "where and how execution continues after the code block's " - 'execution has\n' - 'completed.\n' - '\n' - 'A *scope* defines the visibility of a name within a block. If a ' - 'local\n' - 'variable is defined in a block, its scope includes that block. ' - 'If the\n' - 'definition occurs in a function block, the scope extends to any ' - 'blocks\n' - 'contained within the defining one, unless a contained block ' - 'introduces\n' - 'a different binding for the name. The scope of names defined in ' - 'a\n' - 'class block is limited to the class block; it does not extend to ' - 'the\n' - 'code blocks of methods -- this includes generator expressions ' - 'since\n' - 'they are implemented using a function scope. This means that ' - 'the\n' - 'following will fail:\n' - '\n' - ' class A:\n' - ' a = 42\n' - ' b = list(a + i for i in range(10))\n' - '\n' - 'When a name is used in a code block, it is resolved using the ' - 'nearest\n' - 'enclosing scope. The set of all such scopes visible to a code ' - 'block\n' - "is called the block's *environment*.\n" - '\n' - 'If a name is bound in a block, it is a local variable of that ' - 'block.\n' - 'If a name is bound at the module level, it is a global ' - 'variable. (The\n' - 'variables of the module code block are local and global.) If a\n' - 'variable is used in a code block but not defined there, it is a ' - '*free\n' - 'variable*.\n' - '\n' - 'When a name is not found at all, a "NameError" exception is ' - 'raised.\n' - 'If the name refers to a local variable that has not been bound, ' - 'a\n' - '"UnboundLocalError" exception is raised. "UnboundLocalError" is ' - 'a\n' - 'subclass of "NameError".\n' - '\n' - 'The following constructs bind names: formal parameters to ' - 'functions,\n' - '"import" statements, class and function definitions (these bind ' - 'the\n' - 'class or function name in the defining block), and targets that ' - 'are\n' - 'identifiers if occurring in an assignment, "for" loop header, in ' - 'the\n' - 'second position of an "except" clause header or after "as" in a ' - '"with"\n' - 'statement. The "import" statement of the form "from ... import ' - '*"\n' - 'binds all names defined in the imported module, except those ' - 'beginning\n' - 'with an underscore. This form may only be used at the module ' - 'level.\n' - '\n' - 'A target occurring in a "del" statement is also considered bound ' - 'for\n' - 'this purpose (though the actual semantics are to unbind the ' - 'name). It\n' - 'is illegal to unbind a name that is referenced by an enclosing ' - 'scope;\n' - 'the compiler will report a "SyntaxError".\n' - '\n' - 'Each assignment or import statement occurs within a block ' - 'defined by a\n' - 'class or function definition or at the module level (the ' - 'top-level\n' - 'code block).\n' - '\n' - 'If a name binding operation occurs anywhere within a code block, ' - 'all\n' - 'uses of the name within the block are treated as references to ' - 'the\n' - 'current block. This can lead to errors when a name is used ' - 'within a\n' - 'block before it is bound. This rule is subtle. Python lacks\n' - 'declarations and allows name binding operations to occur ' - 'anywhere\n' - 'within a code block. The local variables of a code block can ' - 'be\n' - 'determined by scanning the entire text of the block for name ' - 'binding\n' - 'operations.\n' - '\n' - 'If the global statement occurs within a block, all uses of the ' - 'name\n' - 'specified in the statement refer to the binding of that name in ' - 'the\n' - 'top-level namespace. Names are resolved in the top-level ' - 'namespace by\n' - 'searching the global namespace, i.e. the namespace of the ' - 'module\n' - 'containing the code block, and the builtins namespace, the ' - 'namespace\n' - 'of the module "__builtin__". The global namespace is searched ' - 'first.\n' - 'If the name is not found there, the builtins namespace is ' - 'searched.\n' - 'The global statement must precede all uses of the name.\n' - '\n' - 'The builtins namespace associated with the execution of a code ' - 'block\n' - 'is actually found by looking up the name "__builtins__" in its ' - 'global\n' - 'namespace; this should be a dictionary or a module (in the ' - 'latter case\n' - "the module's dictionary is used). By default, when in the " - '"__main__"\n' - 'module, "__builtins__" is the built-in module "__builtin__" ' - '(note: no\n' - '\'s\'); when in any other module, "__builtins__" is an alias for ' - 'the\n' - 'dictionary of the "__builtin__" module itself. "__builtins__" ' - 'can be\n' - 'set to a user-created dictionary to create a weak form of ' - 'restricted\n' - 'execution.\n' - '\n' - '**CPython implementation detail:** Users should not touch\n' - '"__builtins__"; it is strictly an implementation detail. Users\n' - 'wanting to override values in the builtins namespace should ' - '"import"\n' - 'the "__builtin__" (no \'s\') module and modify its attributes\n' - 'appropriately.\n' - '\n' - 'The namespace for a module is automatically created the first ' - 'time a\n' - 'module is imported. The main module for a script is always ' - 'called\n' - '"__main__".\n' - '\n' - 'The "global" statement has the same scope as a name binding ' - 'operation\n' - 'in the same block. If the nearest enclosing scope for a free ' - 'variable\n' - 'contains a global statement, the free variable is treated as a ' - 'global.\n' - '\n' - 'A class definition is an executable statement that may use and ' - 'define\n' - 'names. These references follow the normal rules for name ' - 'resolution.\n' - 'The namespace of the class definition becomes the attribute ' - 'dictionary\n' - 'of the class. Names defined at the class scope are not visible ' - 'in\n' - 'methods.\n' - '\n' - '\n' - 'Interaction with dynamic features\n' - '=================================\n' - '\n' - 'There are several cases where Python statements are illegal when ' - 'used\n' - 'in conjunction with nested scopes that contain free variables.\n' - '\n' - 'If a variable is referenced in an enclosing scope, it is illegal ' - 'to\n' - 'delete the name. An error will be reported at compile time.\n' - '\n' - 'If the wild card form of import --- "import *" --- is used in a\n' - 'function and the function contains or is a nested block with ' - 'free\n' - 'variables, the compiler will raise a "SyntaxError".\n' - '\n' - 'If "exec" is used in a function and the function contains or is ' - 'a\n' - 'nested block with free variables, the compiler will raise a\n' - '"SyntaxError" unless the exec explicitly specifies the local ' - 'namespace\n' - 'for the "exec". (In other words, "exec obj" would be illegal, ' - 'but\n' - '"exec obj in ns" would be legal.)\n' - '\n' - 'The "eval()", "execfile()", and "input()" functions and the ' - '"exec"\n' - 'statement do not have access to the full environment for ' - 'resolving\n' - 'names. Names may be resolved in the local and global namespaces ' - 'of\n' - 'the caller. Free variables are not resolved in the nearest ' - 'enclosing\n' - 'namespace, but in the global namespace. [1] The "exec" statement ' - 'and\n' - 'the "eval()" and "execfile()" functions have optional arguments ' - 'to\n' - 'override the global and local namespace. If only one namespace ' - 'is\n' - 'specified, it is used for both.\n', - 'numbers': '\n' - 'Numeric literals\n' - '****************\n' - '\n' - 'There are four types of numeric literals: plain integers, long\n' - 'integers, floating point numbers, and imaginary numbers. There ' - 'are no\n' - 'complex literals (complex numbers can be formed by adding a ' - 'real\n' - 'number and an imaginary number).\n' - '\n' - 'Note that numeric literals do not include a sign; a phrase like ' - '"-1"\n' - 'is actually an expression composed of the unary operator ' - '\'"-"\' and the\n' - 'literal "1".\n', - 'numeric-types': '\n' - 'Emulating numeric types\n' - '***********************\n' - '\n' - 'The following methods can be defined to emulate numeric ' - 'objects.\n' - 'Methods corresponding to operations that are not ' - 'supported by the\n' - 'particular kind of number implemented (e.g., bitwise ' - 'operations for\n' - 'non-integral numbers) should be left undefined.\n' - '\n' - 'object.__add__(self, other)\n' - 'object.__sub__(self, other)\n' - 'object.__mul__(self, other)\n' - 'object.__floordiv__(self, other)\n' - 'object.__mod__(self, other)\n' - 'object.__divmod__(self, other)\n' - 'object.__pow__(self, other[, modulo])\n' - 'object.__lshift__(self, other)\n' - 'object.__rshift__(self, other)\n' - 'object.__and__(self, other)\n' - 'object.__xor__(self, other)\n' - 'object.__or__(self, other)\n' - '\n' - ' These methods are called to implement the binary ' - 'arithmetic\n' - ' operations ("+", "-", "*", "//", "%", "divmod()", ' - '"pow()", "**",\n' - ' "<<", ">>", "&", "^", "|"). For instance, to evaluate ' - 'the\n' - ' expression "x + y", where *x* is an instance of a ' - 'class that has an\n' - ' "__add__()" method, "x.__add__(y)" is called. The ' - '"__divmod__()"\n' - ' method should be the equivalent to using ' - '"__floordiv__()" and\n' - ' "__mod__()"; it should not be related to ' - '"__truediv__()" (described\n' - ' below). Note that "__pow__()" should be defined to ' - 'accept an\n' - ' optional third argument if the ternary version of the ' - 'built-in\n' - ' "pow()" function is to be supported.\n' - '\n' - ' If one of those methods does not support the operation ' - 'with the\n' - ' supplied arguments, it should return ' - '"NotImplemented".\n' - '\n' - 'object.__div__(self, other)\n' - 'object.__truediv__(self, other)\n' - '\n' - ' The division operator ("/") is implemented by these ' - 'methods. The\n' - ' "__truediv__()" method is used when ' - '"__future__.division" is in\n' - ' effect, otherwise "__div__()" is used. If only one of ' - 'these two\n' - ' methods is defined, the object will not support ' - 'division in the\n' - ' alternate context; "TypeError" will be raised ' - 'instead.\n' - '\n' - 'object.__radd__(self, other)\n' - 'object.__rsub__(self, other)\n' - 'object.__rmul__(self, other)\n' - 'object.__rdiv__(self, other)\n' - 'object.__rtruediv__(self, other)\n' - 'object.__rfloordiv__(self, other)\n' - 'object.__rmod__(self, other)\n' - 'object.__rdivmod__(self, other)\n' - 'object.__rpow__(self, other)\n' - 'object.__rlshift__(self, other)\n' - 'object.__rrshift__(self, other)\n' - 'object.__rand__(self, other)\n' - 'object.__rxor__(self, other)\n' - 'object.__ror__(self, other)\n' - '\n' - ' These methods are called to implement the binary ' - 'arithmetic\n' - ' operations ("+", "-", "*", "/", "%", "divmod()", ' - '"pow()", "**",\n' - ' "<<", ">>", "&", "^", "|") with reflected (swapped) ' - 'operands.\n' - ' These functions are only called if the left operand ' - 'does not\n' - ' support the corresponding operation and the operands ' - 'are of\n' - ' different types. [2] For instance, to evaluate the ' - 'expression "x -\n' - ' y", where *y* is an instance of a class that has an ' - '"__rsub__()"\n' - ' method, "y.__rsub__(x)" is called if "x.__sub__(y)" ' - 'returns\n' - ' *NotImplemented*.\n' - '\n' - ' Note that ternary "pow()" will not try calling ' - '"__rpow__()" (the\n' - ' coercion rules would become too complicated).\n' - '\n' - " Note: If the right operand's type is a subclass of the " - 'left\n' - " operand's type and that subclass provides the " - 'reflected method\n' - ' for the operation, this method will be called before ' - 'the left\n' - " operand's non-reflected method. This behavior " - 'allows subclasses\n' - " to override their ancestors' operations.\n" - '\n' - 'object.__iadd__(self, other)\n' - 'object.__isub__(self, other)\n' - 'object.__imul__(self, other)\n' - 'object.__idiv__(self, other)\n' - 'object.__itruediv__(self, other)\n' - 'object.__ifloordiv__(self, other)\n' - 'object.__imod__(self, other)\n' - 'object.__ipow__(self, other[, modulo])\n' - 'object.__ilshift__(self, other)\n' - 'object.__irshift__(self, other)\n' - 'object.__iand__(self, other)\n' - 'object.__ixor__(self, other)\n' - 'object.__ior__(self, other)\n' - '\n' - ' These methods are called to implement the augmented ' - 'arithmetic\n' - ' assignments ("+=", "-=", "*=", "/=", "//=", "%=", ' - '"**=", "<<=",\n' - ' ">>=", "&=", "^=", "|="). These methods should ' - 'attempt to do the\n' - ' operation in-place (modifying *self*) and return the ' - 'result (which\n' - ' could be, but does not have to be, *self*). If a ' - 'specific method\n' - ' is not defined, the augmented assignment falls back to ' - 'the normal\n' - ' methods. For instance, to execute the statement "x += ' - 'y", where\n' - ' *x* is an instance of a class that has an "__iadd__()" ' - 'method,\n' - ' "x.__iadd__(y)" is called. If *x* is an instance of a ' - 'class that\n' - ' does not define a "__iadd__()" method, "x.__add__(y)" ' - 'and\n' - ' "y.__radd__(x)" are considered, as with the evaluation ' - 'of "x + y".\n' - '\n' - 'object.__neg__(self)\n' - 'object.__pos__(self)\n' - 'object.__abs__(self)\n' - 'object.__invert__(self)\n' - '\n' - ' Called to implement the unary arithmetic operations ' - '("-", "+",\n' - ' "abs()" and "~").\n' - '\n' - 'object.__complex__(self)\n' - 'object.__int__(self)\n' - 'object.__long__(self)\n' - 'object.__float__(self)\n' - '\n' - ' Called to implement the built-in functions ' - '"complex()", "int()",\n' - ' "long()", and "float()". Should return a value of the ' - 'appropriate\n' - ' type.\n' - '\n' - 'object.__oct__(self)\n' - 'object.__hex__(self)\n' - '\n' - ' Called to implement the built-in functions "oct()" and ' - '"hex()".\n' - ' Should return a string value.\n' - '\n' - 'object.__index__(self)\n' - '\n' - ' Called to implement "operator.index()". Also called ' - 'whenever\n' - ' Python needs an integer object (such as in slicing). ' - 'Must return\n' - ' an integer (int or long).\n' - '\n' - ' New in version 2.5.\n' - '\n' - 'object.__coerce__(self, other)\n' - '\n' - ' Called to implement "mixed-mode" numeric arithmetic. ' - 'Should either\n' - ' return a 2-tuple containing *self* and *other* ' - 'converted to a\n' - ' common numeric type, or "None" if conversion is ' - 'impossible. When\n' - ' the common type would be the type of "other", it is ' - 'sufficient to\n' - ' return "None", since the interpreter will also ask the ' - 'other object\n' - ' to attempt a coercion (but sometimes, if the ' - 'implementation of the\n' - ' other type cannot be changed, it is useful to do the ' - 'conversion to\n' - ' the other type here). A return value of ' - '"NotImplemented" is\n' - ' equivalent to returning "None".\n', - 'objects': '\n' - 'Objects, values and types\n' - '*************************\n' - '\n' - "*Objects* are Python's abstraction for data. All data in a " - 'Python\n' - 'program is represented by objects or by relations between ' - 'objects. (In\n' - "a sense, and in conformance to Von Neumann's model of a " - '"stored\n' - 'program computer," code is also represented by objects.)\n' - '\n' - "Every object has an identity, a type and a value. An object's\n" - '*identity* never changes once it has been created; you may ' - 'think of it\n' - 'as the object\'s address in memory. The \'"is"\' operator ' - 'compares the\n' - 'identity of two objects; the "id()" function returns an ' - 'integer\n' - 'representing its identity (currently implemented as its ' - 'address). An\n' - "object's *type* is also unchangeable. [1] An object's type " - 'determines\n' - 'the operations that the object supports (e.g., "does it have a\n' - 'length?") and also defines the possible values for objects of ' - 'that\n' - 'type. The "type()" function returns an object\'s type (which ' - 'is an\n' - 'object itself). The *value* of some objects can change. ' - 'Objects\n' - 'whose value can change are said to be *mutable*; objects whose ' - 'value\n' - 'is unchangeable once they are created are called *immutable*. ' - '(The\n' - 'value of an immutable container object that contains a ' - 'reference to a\n' - "mutable object can change when the latter's value is changed; " - 'however\n' - 'the container is still considered immutable, because the ' - 'collection of\n' - 'objects it contains cannot be changed. So, immutability is ' - 'not\n' - 'strictly the same as having an unchangeable value, it is more ' - 'subtle.)\n' - "An object's mutability is determined by its type; for " - 'instance,\n' - 'numbers, strings and tuples are immutable, while dictionaries ' - 'and\n' - 'lists are mutable.\n' - '\n' - 'Objects are never explicitly destroyed; however, when they ' - 'become\n' - 'unreachable they may be garbage-collected. An implementation ' - 'is\n' - 'allowed to postpone garbage collection or omit it altogether ' - '--- it is\n' - 'a matter of implementation quality how garbage collection is\n' - 'implemented, as long as no objects are collected that are ' - 'still\n' - 'reachable.\n' - '\n' - '**CPython implementation detail:** CPython currently uses a ' - 'reference-\n' - 'counting scheme with (optional) delayed detection of cyclically ' - 'linked\n' - 'garbage, which collects most objects as soon as they become\n' - 'unreachable, but is not guaranteed to collect garbage ' - 'containing\n' - 'circular references. See the documentation of the "gc" module ' - 'for\n' - 'information on controlling the collection of cyclic garbage. ' - 'Other\n' - 'implementations act differently and CPython may change. Do not ' - 'depend\n' - 'on immediate finalization of objects when they become ' - 'unreachable (ex:\n' - 'always close files).\n' - '\n' - "Note that the use of the implementation's tracing or debugging\n" - 'facilities may keep objects alive that would normally be ' - 'collectable.\n' - 'Also note that catching an exception with a ' - '\'"try"..."except"\'\n' - 'statement may keep objects alive.\n' - '\n' - 'Some objects contain references to "external" resources such as ' - 'open\n' - 'files or windows. It is understood that these resources are ' - 'freed\n' - 'when the object is garbage-collected, but since garbage ' - 'collection is\n' - 'not guaranteed to happen, such objects also provide an explicit ' - 'way to\n' - 'release the external resource, usually a "close()" method. ' - 'Programs\n' - 'are strongly recommended to explicitly close such objects. ' - 'The\n' - '\'"try"..."finally"\' statement provides a convenient way to do ' - 'this.\n' - '\n' - 'Some objects contain references to other objects; these are ' - 'called\n' - '*containers*. Examples of containers are tuples, lists and\n' - "dictionaries. The references are part of a container's value. " - 'In\n' - 'most cases, when we talk about the value of a container, we ' - 'imply the\n' - 'values, not the identities of the contained objects; however, ' - 'when we\n' - 'talk about the mutability of a container, only the identities ' - 'of the\n' - 'immediately contained objects are implied. So, if an ' - 'immutable\n' - 'container (like a tuple) contains a reference to a mutable ' - 'object, its\n' - 'value changes if that mutable object is changed.\n' - '\n' - 'Types affect almost all aspects of object behavior. Even the\n' - 'importance of object identity is affected in some sense: for ' - 'immutable\n' - 'types, operations that compute new values may actually return ' - 'a\n' - 'reference to any existing object with the same type and value, ' - 'while\n' - 'for mutable objects this is not allowed. E.g., after "a = 1; b ' - '= 1",\n' - '"a" and "b" may or may not refer to the same object with the ' - 'value\n' - 'one, depending on the implementation, but after "c = []; d = ' - '[]", "c"\n' - 'and "d" are guaranteed to refer to two different, unique, ' - 'newly\n' - 'created empty lists. (Note that "c = d = []" assigns the same ' - 'object\n' - 'to both "c" and "d".)\n', - 'operator-summary': '\n' - 'Operator precedence\n' - '*******************\n' - '\n' - 'The following table summarizes the operator ' - 'precedences in Python,\n' - 'from lowest precedence (least binding) to highest ' - 'precedence (most\n' - 'binding). Operators in the same box have the same ' - 'precedence. Unless\n' - 'the syntax is explicitly given, operators are binary. ' - 'Operators in\n' - 'the same box group left to right (except for ' - 'comparisons, including\n' - 'tests, which all have the same precedence and chain ' - 'from left to right\n' - '--- see section Comparisons --- and exponentiation, ' - 'which groups from\n' - 'right to left).\n' - '\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| Operator | ' - 'Description |\n' - '+=================================================+=======================================+\n' - '| "lambda" | ' - 'Lambda expression |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "if" -- "else" | ' - 'Conditional expression |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "or" | ' - 'Boolean OR |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "and" | ' - 'Boolean AND |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "not" "x" | ' - 'Boolean NOT |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "in", "not in", "is", "is not", "<", "<=", ">", | ' - 'Comparisons, including membership |\n' - '| ">=", "<>", "!=", "==" | ' - 'tests and identity tests |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "|" | ' - 'Bitwise OR |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "^" | ' - 'Bitwise XOR |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "&" | ' - 'Bitwise AND |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "<<", ">>" | ' - 'Shifts |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "+", "-" | ' - 'Addition and subtraction |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "*", "/", "//", "%" | ' - 'Multiplication, division, remainder |\n' - '| | ' - '[8] |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "+x", "-x", "~x" | ' - 'Positive, negative, bitwise NOT |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "**" | ' - 'Exponentiation [9] |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "x[index]", "x[index:index]", | ' - 'Subscription, slicing, call, |\n' - '| "x(arguments...)", "x.attribute" | ' - 'attribute reference |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '| "(expressions...)", "[expressions...]", "{key: | ' - 'Binding or tuple display, list |\n' - '| value...}", "`expressions...`" | ' - 'display, dictionary display, string |\n' - '| | ' - 'conversion |\n' - '+-------------------------------------------------+---------------------------------------+\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] In Python 2.3 and later releases, a list ' - 'comprehension "leaks"\n' - ' the control variables of each "for" it contains ' - 'into the\n' - ' containing scope. However, this behavior is ' - 'deprecated, and\n' - ' relying on it will not work in Python 3.\n' - '\n' - '[2] While "abs(x%y) < abs(y)" is true mathematically, ' - 'for floats\n' - ' it may not be true numerically due to roundoff. ' - 'For example, and\n' - ' assuming a platform on which a Python float is an ' - 'IEEE 754 double-\n' - ' precision number, in order that "-1e-100 % 1e100" ' - 'have the same\n' - ' sign as "1e100", the computed result is "-1e-100 + ' - '1e100", which\n' - ' is numerically exactly equal to "1e100". The ' - 'function\n' - ' "math.fmod()" returns a result whose sign matches ' - 'the sign of the\n' - ' first argument instead, and so returns "-1e-100" ' - 'in this case.\n' - ' Which approach is more appropriate depends on the ' - 'application.\n' - '\n' - '[3] If x is very close to an exact integer multiple of ' - "y, it's\n" - ' possible for "floor(x/y)" to be one larger than ' - '"(x-x%y)/y" due to\n' - ' rounding. In such cases, Python returns the ' - 'latter result, in\n' - ' order to preserve that "divmod(x,y)[0] * y + x % ' - 'y" be very close\n' - ' to "x".\n' - '\n' - '[4] While comparisons between unicode strings make ' - 'sense at the\n' - ' byte level, they may be counter-intuitive to ' - 'users. For example,\n' - ' the strings "u"\\u00C7"" and "u"\\u0043\\u0327"" ' - 'compare differently,\n' - ' even though they both represent the same unicode ' - 'character (LATIN\n' - ' CAPITAL LETTER C WITH CEDILLA). To compare strings ' - 'in a human\n' - ' recognizable way, compare using ' - '"unicodedata.normalize()".\n' - '\n' - '[5] The implementation computes this efficiently, ' - 'without\n' - ' constructing lists or sorting.\n' - '\n' - '[6] Earlier versions of Python used lexicographic ' - 'comparison of\n' - ' the sorted (key, value) lists, but this was very ' - 'expensive for the\n' - ' common case of comparing for equality. An even ' - 'earlier version of\n' - ' Python compared dictionaries by identity only, but ' - 'this caused\n' - ' surprises because people expected to be able to ' - 'test a dictionary\n' - ' for emptiness by comparing it to "{}".\n' - '\n' - '[7] Due to automatic garbage-collection, free lists, ' - 'and the\n' - ' dynamic nature of descriptors, you may notice ' - 'seemingly unusual\n' - ' behaviour in certain uses of the "is" operator, ' - 'like those\n' - ' involving comparisons between instance methods, or ' - 'constants.\n' - ' Check their documentation for more info.\n' - '\n' - '[8] The "%" operator is also used for string ' - 'formatting; the same\n' - ' precedence applies.\n' - '\n' - '[9] The power operator "**" binds less tightly than an ' - 'arithmetic\n' - ' or bitwise unary operator on its right, that is, ' - '"2**-1" is "0.5".\n', - 'pass': '\n' - 'The "pass" statement\n' - '********************\n' - '\n' - ' pass_stmt ::= "pass"\n' - '\n' - '"pass" is a null operation --- when it is executed, nothing ' - 'happens.\n' - 'It is useful as a placeholder when a statement is required\n' - 'syntactically, but no code needs to be executed, for example:\n' - '\n' - ' def f(arg): pass # a function that does nothing (yet)\n' - '\n' - ' class C: pass # a class with no methods (yet)\n', - 'power': '\n' - 'The power operator\n' - '******************\n' - '\n' - 'The power operator binds more tightly than unary operators on ' - 'its\n' - 'left; it binds less tightly than unary operators on its right. ' - 'The\n' - 'syntax is:\n' - '\n' - ' power ::= primary ["**" u_expr]\n' - '\n' - 'Thus, in an unparenthesized sequence of power and unary ' - 'operators, the\n' - 'operators are evaluated from right to left (this does not ' - 'constrain\n' - 'the evaluation order for the operands): "-1**2" results in "-1".\n' - '\n' - 'The power operator has the same semantics as the built-in ' - '"pow()"\n' - 'function, when called with two arguments: it yields its left ' - 'argument\n' - 'raised to the power of its right argument. The numeric arguments ' - 'are\n' - 'first converted to a common type. The result type is that of ' - 'the\n' - 'arguments after coercion.\n' - '\n' - 'With mixed operand types, the coercion rules for binary ' - 'arithmetic\n' - 'operators apply. For int and long int operands, the result has ' - 'the\n' - 'same type as the operands (after coercion) unless the second ' - 'argument\n' - 'is negative; in that case, all arguments are converted to float ' - 'and a\n' - 'float result is delivered. For example, "10**2" returns "100", ' - 'but\n' - '"10**-2" returns "0.01". (This last feature was added in Python ' - '2.2.\n' - 'In Python 2.1 and before, if both arguments were of integer types ' - 'and\n' - 'the second argument was negative, an exception was raised).\n' - '\n' - 'Raising "0.0" to a negative power results in a ' - '"ZeroDivisionError".\n' - 'Raising a negative number to a fractional power results in a\n' - '"ValueError".\n', - 'print': '\n' - 'The "print" statement\n' - '*********************\n' - '\n' - ' print_stmt ::= "print" ([expression ("," expression)* [","]]\n' - ' | ">>" expression [("," expression)+ [","]])\n' - '\n' - '"print" evaluates each expression in turn and writes the ' - 'resulting\n' - 'object to standard output (see below). If an object is not a ' - 'string,\n' - 'it is first converted to a string using the rules for string\n' - 'conversions. The (resulting or original) string is then ' - 'written. A\n' - 'space is written before each object is (converted and) written, ' - 'unless\n' - 'the output system believes it is positioned at the beginning of ' - 'a\n' - 'line. This is the case (1) when no characters have yet been ' - 'written\n' - 'to standard output, (2) when the last character written to ' - 'standard\n' - 'output is a whitespace character except "\' \'", or (3) when the ' - 'last\n' - 'write operation on standard output was not a "print" statement. ' - '(In\n' - 'some cases it may be functional to write an empty string to ' - 'standard\n' - 'output for this reason.)\n' - '\n' - 'Note: Objects which act like file objects but which are not the\n' - ' built-in file objects often do not properly emulate this aspect ' - 'of\n' - " the file object's behavior, so it is best not to rely on this.\n" - '\n' - 'A "\'\\n\'" character is written at the end, unless the "print" ' - 'statement\n' - 'ends with a comma. This is the only action if the statement ' - 'contains\n' - 'just the keyword "print".\n' - '\n' - 'Standard output is defined as the file object named "stdout" in ' - 'the\n' - 'built-in module "sys". If no such object exists, or if it does ' - 'not\n' - 'have a "write()" method, a "RuntimeError" exception is raised.\n' - '\n' - '"print" also has an extended form, defined by the second portion ' - 'of\n' - 'the syntax described above. This form is sometimes referred to ' - 'as\n' - '""print" chevron." In this form, the first expression after the ' - '">>"\n' - 'must evaluate to a "file-like" object, specifically an object ' - 'that has\n' - 'a "write()" method as described above. With this extended form, ' - 'the\n' - 'subsequent expressions are printed to this file object. If the ' - 'first\n' - 'expression evaluates to "None", then "sys.stdout" is used as the ' - 'file\n' - 'for output.\n', - 'raise': '\n' - 'The "raise" statement\n' - '*********************\n' - '\n' - ' raise_stmt ::= "raise" [expression ["," expression ["," ' - 'expression]]]\n' - '\n' - 'If no expressions are present, "raise" re-raises the last ' - 'exception\n' - 'that was active in the current scope. If no exception is active ' - 'in\n' - 'the current scope, a "TypeError" exception is raised indicating ' - 'that\n' - 'this is an error (if running under IDLE, a "Queue.Empty" ' - 'exception is\n' - 'raised instead).\n' - '\n' - 'Otherwise, "raise" evaluates the expressions to get three ' - 'objects,\n' - 'using "None" as the value of omitted expressions. The first two\n' - 'objects are used to determine the *type* and *value* of the ' - 'exception.\n' - '\n' - 'If the first object is an instance, the type of the exception is ' - 'the\n' - 'class of the instance, the instance itself is the value, and the\n' - 'second object must be "None".\n' - '\n' - 'If the first object is a class, it becomes the type of the ' - 'exception.\n' - 'The second object is used to determine the exception value: If it ' - 'is\n' - 'an instance of the class, the instance becomes the exception ' - 'value. If\n' - 'the second object is a tuple, it is used as the argument list for ' - 'the\n' - 'class constructor; if it is "None", an empty argument list is ' - 'used,\n' - 'and any other object is treated as a single argument to the\n' - 'constructor. The instance so created by calling the constructor ' - 'is\n' - 'used as the exception value.\n' - '\n' - 'If a third object is present and not "None", it must be a ' - 'traceback\n' - 'object (see section The standard type hierarchy), and it is\n' - 'substituted instead of the current location as the place where ' - 'the\n' - 'exception occurred. If the third object is present and not a\n' - 'traceback object or "None", a "TypeError" exception is raised. ' - 'The\n' - 'three-expression form of "raise" is useful to re-raise an ' - 'exception\n' - 'transparently in an except clause, but "raise" with no ' - 'expressions\n' - 'should be preferred if the exception to be re-raised was the ' - 'most\n' - 'recently active exception in the current scope.\n' - '\n' - 'Additional information on exceptions can be found in section\n' - 'Exceptions, and information about handling exceptions is in ' - 'section\n' - 'The try statement.\n', - 'return': '\n' - 'The "return" statement\n' - '**********************\n' - '\n' - ' return_stmt ::= "return" [expression_list]\n' - '\n' - '"return" may only occur syntactically nested in a function ' - 'definition,\n' - 'not within a nested class definition.\n' - '\n' - 'If an expression list is present, it is evaluated, else "None" ' - 'is\n' - 'substituted.\n' - '\n' - '"return" leaves the current function call with the expression ' - 'list (or\n' - '"None") as return value.\n' - '\n' - 'When "return" passes control out of a "try" statement with a ' - '"finally"\n' - 'clause, that "finally" clause is executed before really leaving ' - 'the\n' - 'function.\n' - '\n' - 'In a generator function, the "return" statement is not allowed ' - 'to\n' - 'include an "expression_list". In that context, a bare "return"\n' - 'indicates that the generator is done and will cause ' - '"StopIteration" to\n' - 'be raised.\n', - 'sequence-types': '\n' - 'Emulating container types\n' - '*************************\n' - '\n' - 'The following methods can be defined to implement ' - 'container objects.\n' - 'Containers usually are sequences (such as lists or ' - 'tuples) or mappings\n' - '(like dictionaries), but can represent other containers ' - 'as well. The\n' - 'first set of methods is used either to emulate a ' - 'sequence or to\n' - 'emulate a mapping; the difference is that for a ' - 'sequence, the\n' - 'allowable keys should be the integers *k* for which "0 ' - '<= k < N" where\n' - '*N* is the length of the sequence, or slice objects, ' - 'which define a\n' - 'range of items. (For backwards compatibility, the ' - 'method\n' - '"__getslice__()" (see below) can also be defined to ' - 'handle simple, but\n' - 'not extended slices.) It is also recommended that ' - 'mappings provide the\n' - 'methods "keys()", "values()", "items()", "has_key()", ' - '"get()",\n' - '"clear()", "setdefault()", "iterkeys()", ' - '"itervalues()",\n' - '"iteritems()", "pop()", "popitem()", "copy()", and ' - '"update()" behaving\n' - "similar to those for Python's standard dictionary " - 'objects. The\n' - '"UserDict" module provides a "DictMixin" class to help ' - 'create those\n' - 'methods from a base set of "__getitem__()", ' - '"__setitem__()",\n' - '"__delitem__()", and "keys()". Mutable sequences should ' - 'provide\n' - 'methods "append()", "count()", "index()", "extend()", ' - '"insert()",\n' - '"pop()", "remove()", "reverse()" and "sort()", like ' - 'Python standard\n' - 'list objects. Finally, sequence types should implement ' - 'addition\n' - '(meaning concatenation) and multiplication (meaning ' - 'repetition) by\n' - 'defining the methods "__add__()", "__radd__()", ' - '"__iadd__()",\n' - '"__mul__()", "__rmul__()" and "__imul__()" described ' - 'below; they\n' - 'should not define "__coerce__()" or other numerical ' - 'operators. It is\n' - 'recommended that both mappings and sequences implement ' - 'the\n' - '"__contains__()" method to allow efficient use of the ' - '"in" operator;\n' - 'for mappings, "in" should be equivalent of "has_key()"; ' - 'for sequences,\n' - 'it should search through the values. It is further ' - 'recommended that\n' - 'both mappings and sequences implement the "__iter__()" ' - 'method to allow\n' - 'efficient iteration through the container; for mappings, ' - '"__iter__()"\n' - 'should be the same as "iterkeys()"; for sequences, it ' - 'should iterate\n' - 'through the values.\n' - '\n' - 'object.__len__(self)\n' - '\n' - ' Called to implement the built-in function "len()". ' - 'Should return\n' - ' the length of the object, an integer ">=" 0. Also, ' - 'an object that\n' - ' doesn\'t define a "__nonzero__()" method and whose ' - '"__len__()"\n' - ' method returns zero is considered to be false in a ' - 'Boolean context.\n' - '\n' - 'object.__getitem__(self, key)\n' - '\n' - ' Called to implement evaluation of "self[key]". For ' - 'sequence types,\n' - ' the accepted keys should be integers and slice ' - 'objects. Note that\n' - ' the special interpretation of negative indexes (if ' - 'the class wishes\n' - ' to emulate a sequence type) is up to the ' - '"__getitem__()" method. If\n' - ' *key* is of an inappropriate type, "TypeError" may be ' - 'raised; if of\n' - ' a value outside the set of indexes for the sequence ' - '(after any\n' - ' special interpretation of negative values), ' - '"IndexError" should be\n' - ' raised. For mapping types, if *key* is missing (not ' - 'in the\n' - ' container), "KeyError" should be raised.\n' - '\n' - ' Note: "for" loops expect that an "IndexError" will be ' - 'raised for\n' - ' illegal indexes to allow proper detection of the ' - 'end of the\n' - ' sequence.\n' - '\n' - 'object.__missing__(self, key)\n' - '\n' - ' Called by "dict"."__getitem__()" to implement ' - '"self[key]" for dict\n' - ' subclasses when key is not in the dictionary.\n' - '\n' - 'object.__setitem__(self, key, value)\n' - '\n' - ' Called to implement assignment to "self[key]". Same ' - 'note as for\n' - ' "__getitem__()". This should only be implemented for ' - 'mappings if\n' - ' the objects support changes to the values for keys, ' - 'or if new keys\n' - ' can be added, or for sequences if elements can be ' - 'replaced. The\n' - ' same exceptions should be raised for improper *key* ' - 'values as for\n' - ' the "__getitem__()" method.\n' - '\n' - 'object.__delitem__(self, key)\n' - '\n' - ' Called to implement deletion of "self[key]". Same ' - 'note as for\n' - ' "__getitem__()". This should only be implemented for ' - 'mappings if\n' - ' the objects support removal of keys, or for sequences ' - 'if elements\n' - ' can be removed from the sequence. The same ' - 'exceptions should be\n' - ' raised for improper *key* values as for the ' - '"__getitem__()" method.\n' - '\n' - 'object.__iter__(self)\n' - '\n' - ' This method is called when an iterator is required ' - 'for a container.\n' - ' This method should return a new iterator object that ' - 'can iterate\n' - ' over all the objects in the container. For mappings, ' - 'it should\n' - ' iterate over the keys of the container, and should ' - 'also be made\n' - ' available as the method "iterkeys()".\n' - '\n' - ' Iterator objects also need to implement this method; ' - 'they are\n' - ' required to return themselves. For more information ' - 'on iterator\n' - ' objects, see Iterator Types.\n' - '\n' - 'object.__reversed__(self)\n' - '\n' - ' Called (if present) by the "reversed()" built-in to ' - 'implement\n' - ' reverse iteration. It should return a new iterator ' - 'object that\n' - ' iterates over all the objects in the container in ' - 'reverse order.\n' - '\n' - ' If the "__reversed__()" method is not provided, the ' - '"reversed()"\n' - ' built-in will fall back to using the sequence ' - 'protocol ("__len__()"\n' - ' and "__getitem__()"). Objects that support the ' - 'sequence protocol\n' - ' should only provide "__reversed__()" if they can ' - 'provide an\n' - ' implementation that is more efficient than the one ' - 'provided by\n' - ' "reversed()".\n' - '\n' - ' New in version 2.6.\n' - '\n' - 'The membership test operators ("in" and "not in") are ' - 'normally\n' - 'implemented as an iteration through a sequence. ' - 'However, container\n' - 'objects can supply the following special method with a ' - 'more efficient\n' - 'implementation, which also does not require the object ' - 'be a sequence.\n' - '\n' - 'object.__contains__(self, item)\n' - '\n' - ' Called to implement membership test operators. ' - 'Should return true\n' - ' if *item* is in *self*, false otherwise. For mapping ' - 'objects, this\n' - ' should consider the keys of the mapping rather than ' - 'the values or\n' - ' the key-item pairs.\n' - '\n' - ' For objects that don\'t define "__contains__()", the ' - 'membership test\n' - ' first tries iteration via "__iter__()", then the old ' - 'sequence\n' - ' iteration protocol via "__getitem__()", see this ' - 'section in the\n' - ' language reference.\n', - 'shifting': '\n' - 'Shifting operations\n' - '*******************\n' - '\n' - 'The shifting operations have lower priority than the ' - 'arithmetic\n' - 'operations:\n' - '\n' - ' shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr\n' - '\n' - 'These operators accept plain or long integers as arguments. ' - 'The\n' - 'arguments are converted to a common type. They shift the ' - 'first\n' - 'argument to the left or right by the number of bits given by ' - 'the\n' - 'second argument.\n' - '\n' - 'A right shift by *n* bits is defined as division by "pow(2, ' - 'n)". A\n' - 'left shift by *n* bits is defined as multiplication with ' - '"pow(2, n)".\n' - 'Negative shift counts raise a "ValueError" exception.\n' - '\n' - 'Note: In the current implementation, the right-hand operand ' - 'is\n' - ' required to be at most "sys.maxsize". If the right-hand ' - 'operand is\n' - ' larger than "sys.maxsize" an "OverflowError" exception is ' - 'raised.\n', - 'slicings': '\n' - 'Slicings\n' - '********\n' - '\n' - 'A slicing selects a range of items in a sequence object (e.g., ' - 'a\n' - 'string, tuple or list). Slicings may be used as expressions ' - 'or as\n' - 'targets in assignment or "del" statements. The syntax for a ' - 'slicing:\n' - '\n' - ' slicing ::= simple_slicing | extended_slicing\n' - ' simple_slicing ::= primary "[" short_slice "]"\n' - ' extended_slicing ::= primary "[" slice_list "]"\n' - ' slice_list ::= slice_item ("," slice_item)* [","]\n' - ' slice_item ::= expression | proper_slice | ellipsis\n' - ' proper_slice ::= short_slice | long_slice\n' - ' short_slice ::= [lower_bound] ":" [upper_bound]\n' - ' long_slice ::= short_slice ":" [stride]\n' - ' lower_bound ::= expression\n' - ' upper_bound ::= expression\n' - ' stride ::= expression\n' - ' ellipsis ::= "..."\n' - '\n' - 'There is ambiguity in the formal syntax here: anything that ' - 'looks like\n' - 'an expression list also looks like a slice list, so any ' - 'subscription\n' - 'can be interpreted as a slicing. Rather than further ' - 'complicating the\n' - 'syntax, this is disambiguated by defining that in this case ' - 'the\n' - 'interpretation as a subscription takes priority over the\n' - 'interpretation as a slicing (this is the case if the slice ' - 'list\n' - 'contains no proper slice nor ellipses). Similarly, when the ' - 'slice\n' - 'list has exactly one short slice and no trailing comma, the\n' - 'interpretation as a simple slicing takes priority over that as ' - 'an\n' - 'extended slicing.\n' - '\n' - 'The semantics for a simple slicing are as follows. The ' - 'primary must\n' - 'evaluate to a sequence object. The lower and upper bound ' - 'expressions,\n' - 'if present, must evaluate to plain integers; defaults are zero ' - 'and the\n' - '"sys.maxint", respectively. If either bound is negative, the\n' - "sequence's length is added to it. The slicing now selects all " - 'items\n' - 'with index *k* such that "i <= k < j" where *i* and *j* are ' - 'the\n' - 'specified lower and upper bounds. This may be an empty ' - 'sequence. It\n' - 'is not an error if *i* or *j* lie outside the range of valid ' - 'indexes\n' - "(such items don't exist so they aren't selected).\n" - '\n' - 'The semantics for an extended slicing are as follows. The ' - 'primary\n' - 'must evaluate to a mapping object, and it is indexed with a ' - 'key that\n' - 'is constructed from the slice list, as follows. If the slice ' - 'list\n' - 'contains at least one comma, the key is a tuple containing ' - 'the\n' - 'conversion of the slice items; otherwise, the conversion of ' - 'the lone\n' - 'slice item is the key. The conversion of a slice item that is ' - 'an\n' - 'expression is that expression. The conversion of an ellipsis ' - 'slice\n' - 'item is the built-in "Ellipsis" object. The conversion of a ' - 'proper\n' - 'slice is a slice object (see section The standard type ' - 'hierarchy)\n' - 'whose "start", "stop" and "step" attributes are the values of ' - 'the\n' - 'expressions given as lower bound, upper bound and stride,\n' - 'respectively, substituting "None" for missing expressions.\n', - 'specialattrs': '\n' - 'Special Attributes\n' - '******************\n' - '\n' - 'The implementation adds a few special read-only attributes ' - 'to several\n' - 'object types, where they are relevant. Some of these are ' - 'not reported\n' - 'by the "dir()" built-in function.\n' - '\n' - 'object.__dict__\n' - '\n' - ' A dictionary or other mapping object used to store an ' - "object's\n" - ' (writable) attributes.\n' - '\n' - 'object.__methods__\n' - '\n' - ' Deprecated since version 2.2: Use the built-in function ' - '"dir()" to\n' - " get a list of an object's attributes. This attribute is " - 'no longer\n' - ' available.\n' - '\n' - 'object.__members__\n' - '\n' - ' Deprecated since version 2.2: Use the built-in function ' - '"dir()" to\n' - " get a list of an object's attributes. This attribute is " - 'no longer\n' - ' available.\n' - '\n' - 'instance.__class__\n' - '\n' - ' The class to which a class instance belongs.\n' - '\n' - 'class.__bases__\n' - '\n' - ' The tuple of base classes of a class object.\n' - '\n' - 'class.__name__\n' - '\n' - ' The name of the class or type.\n' - '\n' - 'The following attributes are only supported by *new-style ' - 'class*es.\n' - '\n' - 'class.__mro__\n' - '\n' - ' This attribute is a tuple of classes that are ' - 'considered when\n' - ' looking for base classes during method resolution.\n' - '\n' - 'class.mro()\n' - '\n' - ' This method can be overridden by a metaclass to ' - 'customize the\n' - ' method resolution order for its instances. It is ' - 'called at class\n' - ' instantiation, and its result is stored in "__mro__".\n' - '\n' - 'class.__subclasses__()\n' - '\n' - ' Each new-style class keeps a list of weak references to ' - 'its\n' - ' immediate subclasses. This method returns a list of ' - 'all those\n' - ' references still alive. Example:\n' - '\n' - ' >>> int.__subclasses__()\n' - " []\n" - '\n' - '-[ Footnotes ]-\n' - '\n' - '[1] Additional information on these special methods may be ' - 'found\n' - ' in the Python Reference Manual (Basic customization).\n' - '\n' - '[2] As a consequence, the list "[1, 2]" is considered ' - 'equal to\n' - ' "[1.0, 2.0]", and similarly for tuples.\n' - '\n' - "[3] They must have since the parser can't tell the type of " - 'the\n' - ' operands.\n' - '\n' - '[4] Cased characters are those with general category ' - 'property\n' - ' being one of "Lu" (Letter, uppercase), "Ll" (Letter, ' - 'lowercase),\n' - ' or "Lt" (Letter, titlecase).\n' - '\n' - '[5] To format only a tuple you should therefore provide a\n' - ' singleton tuple whose only element is the tuple to be ' - 'formatted.\n' - '\n' - '[6] The advantage of leaving the newline on is that ' - 'returning an\n' - ' empty string is then an unambiguous EOF indication. ' - 'It is also\n' - ' possible (in cases where it might matter, for example, ' - 'if you want\n' - ' to make an exact copy of a file while scanning its ' - 'lines) to tell\n' - ' whether the last line of a file ended in a newline or ' - 'not (yes\n' - ' this happens!).\n', - 'specialnames': '\n' - 'Special method names\n' - '********************\n' - '\n' - 'A class can implement certain operations that are invoked ' - 'by special\n' - 'syntax (such as arithmetic operations or subscripting and ' - 'slicing) by\n' - "defining methods with special names. This is Python's " - 'approach to\n' - '*operator overloading*, allowing classes to define their ' - 'own behavior\n' - 'with respect to language operators. For instance, if a ' - 'class defines\n' - 'a method named "__getitem__()", and "x" is an instance of ' - 'this class,\n' - 'then "x[i]" is roughly equivalent to "x.__getitem__(i)" ' - 'for old-style\n' - 'classes and "type(x).__getitem__(x, i)" for new-style ' - 'classes. Except\n' - 'where mentioned, attempts to execute an operation raise an ' - 'exception\n' - 'when no appropriate method is defined (typically ' - '"AttributeError" or\n' - '"TypeError").\n' - '\n' - 'When implementing a class that emulates any built-in type, ' - 'it is\n' - 'important that the emulation only be implemented to the ' - 'degree that it\n' - 'makes sense for the object being modelled. For example, ' - 'some\n' - 'sequences may work well with retrieval of individual ' - 'elements, but\n' - 'extracting a slice may not make sense. (One example of ' - 'this is the\n' - '"NodeList" interface in the W3C\'s Document Object ' - 'Model.)\n' - '\n' - '\n' - 'Basic customization\n' - '===================\n' - '\n' - 'object.__new__(cls[, ...])\n' - '\n' - ' Called to create a new instance of class *cls*. ' - '"__new__()" is a\n' - ' static method (special-cased so you need not declare it ' - 'as such)\n' - ' that takes the class of which an instance was requested ' - 'as its\n' - ' first argument. The remaining arguments are those ' - 'passed to the\n' - ' object constructor expression (the call to the class). ' - 'The return\n' - ' value of "__new__()" should be the new object instance ' - '(usually an\n' - ' instance of *cls*).\n' - '\n' - ' Typical implementations create a new instance of the ' - 'class by\n' - ' invoking the superclass\'s "__new__()" method using\n' - ' "super(currentclass, cls).__new__(cls[, ...])" with ' - 'appropriate\n' - ' arguments and then modifying the newly-created instance ' - 'as\n' - ' necessary before returning it.\n' - '\n' - ' If "__new__()" returns an instance of *cls*, then the ' - 'new\n' - ' instance\'s "__init__()" method will be invoked like\n' - ' "__init__(self[, ...])", where *self* is the new ' - 'instance and the\n' - ' remaining arguments are the same as were passed to ' - '"__new__()".\n' - '\n' - ' If "__new__()" does not return an instance of *cls*, ' - 'then the new\n' - ' instance\'s "__init__()" method will not be invoked.\n' - '\n' - ' "__new__()" is intended mainly to allow subclasses of ' - 'immutable\n' - ' types (like int, str, or tuple) to customize instance ' - 'creation. It\n' - ' is also commonly overridden in custom metaclasses in ' - 'order to\n' - ' customize class creation.\n' - '\n' - 'object.__init__(self[, ...])\n' - '\n' - ' Called after the instance has been created (by ' - '"__new__()"), but\n' - ' before it is returned to the caller. The arguments are ' - 'those\n' - ' passed to the class constructor expression. If a base ' - 'class has an\n' - ' "__init__()" method, the derived class\'s "__init__()" ' - 'method, if\n' - ' any, must explicitly call it to ensure proper ' - 'initialization of the\n' - ' base class part of the instance; for example:\n' - ' "BaseClass.__init__(self, [args...])".\n' - '\n' - ' Because "__new__()" and "__init__()" work together in ' - 'constructing\n' - ' objects ("__new__()" to create it, and "__init__()" to ' - 'customise\n' - ' it), no non-"None" value may be returned by ' - '"__init__()"; doing so\n' - ' will cause a "TypeError" to be raised at runtime.\n' - '\n' - 'object.__del__(self)\n' - '\n' - ' Called when the instance is about to be destroyed. ' - 'This is also\n' - ' called a destructor. If a base class has a "__del__()" ' - 'method, the\n' - ' derived class\'s "__del__()" method, if any, must ' - 'explicitly call it\n' - ' to ensure proper deletion of the base class part of the ' - 'instance.\n' - ' Note that it is possible (though not recommended!) for ' - 'the\n' - ' "__del__()" method to postpone destruction of the ' - 'instance by\n' - ' creating a new reference to it. It may then be called ' - 'at a later\n' - ' time when this new reference is deleted. It is not ' - 'guaranteed that\n' - ' "__del__()" methods are called for objects that still ' - 'exist when\n' - ' the interpreter exits.\n' - '\n' - ' Note: "del x" doesn\'t directly call "x.__del__()" --- ' - 'the former\n' - ' decrements the reference count for "x" by one, and ' - 'the latter is\n' - ' only called when "x"\'s reference count reaches ' - 'zero. Some common\n' - ' situations that may prevent the reference count of an ' - 'object from\n' - ' going to zero include: circular references between ' - 'objects (e.g.,\n' - ' a doubly-linked list or a tree data structure with ' - 'parent and\n' - ' child pointers); a reference to the object on the ' - 'stack frame of\n' - ' a function that caught an exception (the traceback ' - 'stored in\n' - ' "sys.exc_traceback" keeps the stack frame alive); or ' - 'a reference\n' - ' to the object on the stack frame that raised an ' - 'unhandled\n' - ' exception in interactive mode (the traceback stored ' - 'in\n' - ' "sys.last_traceback" keeps the stack frame alive). ' - 'The first\n' - ' situation can only be remedied by explicitly breaking ' - 'the cycles;\n' - ' the latter two situations can be resolved by storing ' - '"None" in\n' - ' "sys.exc_traceback" or "sys.last_traceback". ' - 'Circular references\n' - ' which are garbage are detected when the option cycle ' - 'detector is\n' - " enabled (it's on by default), but can only be cleaned " - 'up if there\n' - ' are no Python-level "__del__()" methods involved. ' - 'Refer to the\n' - ' documentation for the "gc" module for more ' - 'information about how\n' - ' "__del__()" methods are handled by the cycle ' - 'detector,\n' - ' particularly the description of the "garbage" value.\n' - '\n' - ' Warning: Due to the precarious circumstances under ' - 'which\n' - ' "__del__()" methods are invoked, exceptions that ' - 'occur during\n' - ' their execution are ignored, and a warning is printed ' - 'to\n' - ' "sys.stderr" instead. Also, when "__del__()" is ' - 'invoked in\n' - ' response to a module being deleted (e.g., when ' - 'execution of the\n' - ' program is done), other globals referenced by the ' - '"__del__()"\n' - ' method may already have been deleted or in the ' - 'process of being\n' - ' torn down (e.g. the import machinery shutting down). ' - 'For this\n' - ' reason, "__del__()" methods should do the absolute ' - 'minimum needed\n' - ' to maintain external invariants. Starting with ' - 'version 1.5,\n' - ' Python guarantees that globals whose name begins with ' - 'a single\n' - ' underscore are deleted from their module before other ' - 'globals are\n' - ' deleted; if no other references to such globals ' - 'exist, this may\n' - ' help in assuring that imported modules are still ' - 'available at the\n' - ' time when the "__del__()" method is called.\n' - '\n' - ' See also the "-R" command-line option.\n' - '\n' - 'object.__repr__(self)\n' - '\n' - ' Called by the "repr()" built-in function and by string ' - 'conversions\n' - ' (reverse quotes) to compute the "official" string ' - 'representation of\n' - ' an object. If at all possible, this should look like a ' - 'valid\n' - ' Python expression that could be used to recreate an ' - 'object with the\n' - ' same value (given an appropriate environment). If this ' - 'is not\n' - ' possible, a string of the form "<...some useful ' - 'description...>"\n' - ' should be returned. The return value must be a string ' - 'object. If a\n' - ' class defines "__repr__()" but not "__str__()", then ' - '"__repr__()"\n' - ' is also used when an "informal" string representation ' - 'of instances\n' - ' of that class is required.\n' - '\n' - ' This is typically used for debugging, so it is ' - 'important that the\n' - ' representation is information-rich and unambiguous.\n' - '\n' - 'object.__str__(self)\n' - '\n' - ' Called by the "str()" built-in function and by the ' - '"print"\n' - ' statement to compute the "informal" string ' - 'representation of an\n' - ' object. This differs from "__repr__()" in that it does ' - 'not have to\n' - ' be a valid Python expression: a more convenient or ' - 'concise\n' - ' representation may be used instead. The return value ' - 'must be a\n' - ' string object.\n' - '\n' - 'object.__lt__(self, other)\n' - 'object.__le__(self, other)\n' - 'object.__eq__(self, other)\n' - 'object.__ne__(self, other)\n' - 'object.__gt__(self, other)\n' - 'object.__ge__(self, other)\n' - '\n' - ' New in version 2.1.\n' - '\n' - ' These are the so-called "rich comparison" methods, and ' - 'are called\n' - ' for comparison operators in preference to "__cmp__()" ' - 'below. The\n' - ' correspondence between operator symbols and method ' - 'names is as\n' - ' follows: "xy" call ' - '"x.__ne__(y)",\n' - ' "x>y" calls "x.__gt__(y)", and "x>=y" calls ' - '"x.__ge__(y)".\n' - '\n' - ' A rich comparison method may return the singleton ' - '"NotImplemented"\n' - ' if it does not implement the operation for a given pair ' - 'of\n' - ' arguments. By convention, "False" and "True" are ' - 'returned for a\n' - ' successful comparison. However, these methods can ' - 'return any value,\n' - ' so if the comparison operator is used in a Boolean ' - 'context (e.g.,\n' - ' in the condition of an "if" statement), Python will ' - 'call "bool()"\n' - ' on the value to determine if the result is true or ' - 'false.\n' - '\n' - ' There are no implied relationships among the comparison ' - 'operators.\n' - ' The truth of "x==y" does not imply that "x!=y" is ' - 'false.\n' - ' Accordingly, when defining "__eq__()", one should also ' - 'define\n' - ' "__ne__()" so that the operators will behave as ' - 'expected. See the\n' - ' paragraph on "__hash__()" for some important notes on ' - 'creating\n' - ' *hashable* objects which support custom comparison ' - 'operations and\n' - ' are usable as dictionary keys.\n' - '\n' - ' There are no swapped-argument versions of these methods ' - '(to be used\n' - ' when the left argument does not support the operation ' - 'but the right\n' - ' argument does); rather, "__lt__()" and "__gt__()" are ' - "each other's\n" - ' reflection, "__le__()" and "__ge__()" are each other\'s ' - 'reflection,\n' - ' and "__eq__()" and "__ne__()" are their own ' - 'reflection.\n' - '\n' - ' Arguments to rich comparison methods are never ' - 'coerced.\n' - '\n' - ' To automatically generate ordering operations from a ' - 'single root\n' - ' operation, see "functools.total_ordering()".\n' - '\n' - 'object.__cmp__(self, other)\n' - '\n' - ' Called by comparison operations if rich comparison (see ' - 'above) is\n' - ' not defined. Should return a negative integer if "self ' - '< other",\n' - ' zero if "self == other", a positive integer if "self > ' - 'other". If\n' - ' no "__cmp__()", "__eq__()" or "__ne__()" operation is ' - 'defined,\n' - ' class instances are compared by object identity ' - '("address"). See\n' - ' also the description of "__hash__()" for some important ' - 'notes on\n' - ' creating *hashable* objects which support custom ' - 'comparison\n' - ' operations and are usable as dictionary keys. (Note: ' - 'the\n' - ' restriction that exceptions are not propagated by ' - '"__cmp__()" has\n' - ' been removed since Python 1.5.)\n' - '\n' - 'object.__rcmp__(self, other)\n' - '\n' - ' Changed in version 2.1: No longer supported.\n' - '\n' - 'object.__hash__(self)\n' - '\n' - ' Called by built-in function "hash()" and for operations ' - 'on members\n' - ' of hashed collections including "set", "frozenset", and ' - '"dict".\n' - ' "__hash__()" should return an integer. The only ' - 'required property\n' - ' is that objects which compare equal have the same hash ' - 'value; it is\n' - ' advised to somehow mix together (e.g. using exclusive ' - 'or) the hash\n' - ' values for the components of the object that also play ' - 'a part in\n' - ' comparison of objects.\n' - '\n' - ' If a class does not define a "__cmp__()" or "__eq__()" ' - 'method it\n' - ' should not define a "__hash__()" operation either; if ' - 'it defines\n' - ' "__cmp__()" or "__eq__()" but not "__hash__()", its ' - 'instances will\n' - ' not be usable in hashed collections. If a class ' - 'defines mutable\n' - ' objects and implements a "__cmp__()" or "__eq__()" ' - 'method, it\n' - ' should not implement "__hash__()", since hashable ' - 'collection\n' - " implementations require that a object's hash value is " - 'immutable (if\n' - " the object's hash value changes, it will be in the " - 'wrong hash\n' - ' bucket).\n' - '\n' - ' User-defined classes have "__cmp__()" and "__hash__()" ' - 'methods by\n' - ' default; with them, all objects compare unequal (except ' - 'with\n' - ' themselves) and "x.__hash__()" returns a result derived ' - 'from\n' - ' "id(x)".\n' - '\n' - ' Classes which inherit a "__hash__()" method from a ' - 'parent class but\n' - ' change the meaning of "__cmp__()" or "__eq__()" such ' - 'that the hash\n' - ' value returned is no longer appropriate (e.g. by ' - 'switching to a\n' - ' value-based concept of equality instead of the default ' - 'identity\n' - ' based equality) can explicitly flag themselves as being ' - 'unhashable\n' - ' by setting "__hash__ = None" in the class definition. ' - 'Doing so\n' - ' means that not only will instances of the class raise ' - 'an\n' - ' appropriate "TypeError" when a program attempts to ' - 'retrieve their\n' - ' hash value, but they will also be correctly identified ' - 'as\n' - ' unhashable when checking "isinstance(obj, ' - 'collections.Hashable)"\n' - ' (unlike classes which define their own "__hash__()" to ' - 'explicitly\n' - ' raise "TypeError").\n' - '\n' - ' Changed in version 2.5: "__hash__()" may now also ' - 'return a long\n' - ' integer object; the 32-bit integer is then derived from ' - 'the hash of\n' - ' that object.\n' - '\n' - ' Changed in version 2.6: "__hash__" may now be set to ' - '"None" to\n' - ' explicitly flag instances of a class as unhashable.\n' - '\n' - 'object.__nonzero__(self)\n' - '\n' - ' Called to implement truth value testing and the ' - 'built-in operation\n' - ' "bool()"; should return "False" or "True", or their ' - 'integer\n' - ' equivalents "0" or "1". When this method is not ' - 'defined,\n' - ' "__len__()" is called, if it is defined, and the object ' - 'is\n' - ' considered true if its result is nonzero. If a class ' - 'defines\n' - ' neither "__len__()" nor "__nonzero__()", all its ' - 'instances are\n' - ' considered true.\n' - '\n' - 'object.__unicode__(self)\n' - '\n' - ' Called to implement "unicode()" built-in; should return ' - 'a Unicode\n' - ' object. When this method is not defined, string ' - 'conversion is\n' - ' attempted, and the result of string conversion is ' - 'converted to\n' - ' Unicode using the system default encoding.\n' - '\n' - '\n' - 'Customizing attribute access\n' - '============================\n' - '\n' - 'The following methods can be defined to customize the ' - 'meaning of\n' - 'attribute access (use of, assignment to, or deletion of ' - '"x.name") for\n' - 'class instances.\n' - '\n' - 'object.__getattr__(self, name)\n' - '\n' - ' Called when an attribute lookup has not found the ' - 'attribute in the\n' - ' usual places (i.e. it is not an instance attribute nor ' - 'is it found\n' - ' in the class tree for "self"). "name" is the attribute ' - 'name. This\n' - ' method should return the (computed) attribute value or ' - 'raise an\n' - ' "AttributeError" exception.\n' - '\n' - ' Note that if the attribute is found through the normal ' - 'mechanism,\n' - ' "__getattr__()" is not called. (This is an intentional ' - 'asymmetry\n' - ' between "__getattr__()" and "__setattr__()".) This is ' - 'done both for\n' - ' efficiency reasons and because otherwise ' - '"__getattr__()" would have\n' - ' no way to access other attributes of the instance. ' - 'Note that at\n' - ' least for instance variables, you can fake total ' - 'control by not\n' - ' inserting any values in the instance attribute ' - 'dictionary (but\n' - ' instead inserting them in another object). See the\n' - ' "__getattribute__()" method below for a way to actually ' - 'get total\n' - ' control in new-style classes.\n' - '\n' - 'object.__setattr__(self, name, value)\n' - '\n' - ' Called when an attribute assignment is attempted. This ' - 'is called\n' - ' instead of the normal mechanism (i.e. store the value ' - 'in the\n' - ' instance dictionary). *name* is the attribute name, ' - '*value* is the\n' - ' value to be assigned to it.\n' - '\n' - ' If "__setattr__()" wants to assign to an instance ' - 'attribute, it\n' - ' should not simply execute "self.name = value" --- this ' - 'would cause\n' - ' a recursive call to itself. Instead, it should insert ' - 'the value in\n' - ' the dictionary of instance attributes, e.g., ' - '"self.__dict__[name] =\n' - ' value". For new-style classes, rather than accessing ' - 'the instance\n' - ' dictionary, it should call the base class method with ' - 'the same\n' - ' name, for example, "object.__setattr__(self, name, ' - 'value)".\n' - '\n' - 'object.__delattr__(self, name)\n' - '\n' - ' Like "__setattr__()" but for attribute deletion instead ' - 'of\n' - ' assignment. This should only be implemented if "del ' - 'obj.name" is\n' - ' meaningful for the object.\n' - '\n' - '\n' - 'More attribute access for new-style classes\n' - '-------------------------------------------\n' - '\n' - 'The following methods only apply to new-style classes.\n' - '\n' - 'object.__getattribute__(self, name)\n' - '\n' - ' Called unconditionally to implement attribute accesses ' - 'for\n' - ' instances of the class. If the class also defines ' - '"__getattr__()",\n' - ' the latter will not be called unless ' - '"__getattribute__()" either\n' - ' calls it explicitly or raises an "AttributeError". This ' - 'method\n' - ' should return the (computed) attribute value or raise ' - 'an\n' - ' "AttributeError" exception. In order to avoid infinite ' - 'recursion in\n' - ' this method, its implementation should always call the ' - 'base class\n' - ' method with the same name to access any attributes it ' - 'needs, for\n' - ' example, "object.__getattribute__(self, name)".\n' - '\n' - ' Note: This method may still be bypassed when looking up ' - 'special\n' - ' methods as the result of implicit invocation via ' - 'language syntax\n' - ' or built-in functions. See Special method lookup for ' - 'new-style\n' - ' classes.\n' - '\n' - '\n' - 'Implementing Descriptors\n' - '------------------------\n' - '\n' - 'The following methods only apply when an instance of the ' - 'class\n' - 'containing the method (a so-called *descriptor* class) ' - 'appears in an\n' - '*owner* class (the descriptor must be in either the ' - "owner's class\n" - 'dictionary or in the class dictionary for one of its ' - 'parents). In the\n' - 'examples below, "the attribute" refers to the attribute ' - 'whose name is\n' - 'the key of the property in the owner class\' "__dict__".\n' - '\n' - 'object.__get__(self, instance, owner)\n' - '\n' - ' Called to get the attribute of the owner class (class ' - 'attribute\n' - ' access) or of an instance of that class (instance ' - 'attribute\n' - ' access). *owner* is always the owner class, while ' - '*instance* is the\n' - ' instance that the attribute was accessed through, or ' - '"None" when\n' - ' the attribute is accessed through the *owner*. This ' - 'method should\n' - ' return the (computed) attribute value or raise an ' - '"AttributeError"\n' - ' exception.\n' - '\n' - 'object.__set__(self, instance, value)\n' - '\n' - ' Called to set the attribute on an instance *instance* ' - 'of the owner\n' - ' class to a new value, *value*.\n' - '\n' - 'object.__delete__(self, instance)\n' - '\n' - ' Called to delete the attribute on an instance ' - '*instance* of the\n' - ' owner class.\n' - '\n' - '\n' - 'Invoking Descriptors\n' - '--------------------\n' - '\n' - 'In general, a descriptor is an object attribute with ' - '"binding\n' - 'behavior", one whose attribute access has been overridden ' - 'by methods\n' - 'in the descriptor protocol: "__get__()", "__set__()", ' - 'and\n' - '"__delete__()". If any of those methods are defined for an ' - 'object, it\n' - 'is said to be a descriptor.\n' - '\n' - 'The default behavior for attribute access is to get, set, ' - 'or delete\n' - "the attribute from an object's dictionary. For instance, " - '"a.x" has a\n' - 'lookup chain starting with "a.__dict__[\'x\']", then\n' - '"type(a).__dict__[\'x\']", and continuing through the base ' - 'classes of\n' - '"type(a)" excluding metaclasses.\n' - '\n' - 'However, if the looked-up value is an object defining one ' - 'of the\n' - 'descriptor methods, then Python may override the default ' - 'behavior and\n' - 'invoke the descriptor method instead. Where this occurs ' - 'in the\n' - 'precedence chain depends on which descriptor methods were ' - 'defined and\n' - 'how they were called. Note that descriptors are only ' - 'invoked for new\n' - 'style objects or classes (ones that subclass "object()" or ' - '"type()").\n' - '\n' - 'The starting point for descriptor invocation is a binding, ' - '"a.x". How\n' - 'the arguments are assembled depends on "a":\n' - '\n' - 'Direct Call\n' - ' The simplest and least common call is when user code ' - 'directly\n' - ' invokes a descriptor method: "x.__get__(a)".\n' - '\n' - 'Instance Binding\n' - ' If binding to a new-style object instance, "a.x" is ' - 'transformed\n' - ' into the call: "type(a).__dict__[\'x\'].__get__(a, ' - 'type(a))".\n' - '\n' - 'Class Binding\n' - ' If binding to a new-style class, "A.x" is transformed ' - 'into the\n' - ' call: "A.__dict__[\'x\'].__get__(None, A)".\n' - '\n' - 'Super Binding\n' - ' If "a" is an instance of "super", then the binding ' - '"super(B,\n' - ' obj).m()" searches "obj.__class__.__mro__" for the base ' - 'class "A"\n' - ' immediately preceding "B" and then invokes the ' - 'descriptor with the\n' - ' call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n' - '\n' - 'For instance bindings, the precedence of descriptor ' - 'invocation depends\n' - 'on the which descriptor methods are defined. A descriptor ' - 'can define\n' - 'any combination of "__get__()", "__set__()" and ' - '"__delete__()". If it\n' - 'does not define "__get__()", then accessing the attribute ' - 'will return\n' - 'the descriptor object itself unless there is a value in ' - "the object's\n" - 'instance dictionary. If the descriptor defines ' - '"__set__()" and/or\n' - '"__delete__()", it is a data descriptor; if it defines ' - 'neither, it is\n' - 'a non-data descriptor. Normally, data descriptors define ' - 'both\n' - '"__get__()" and "__set__()", while non-data descriptors ' - 'have just the\n' - '"__get__()" method. Data descriptors with "__set__()" and ' - '"__get__()"\n' - 'defined always override a redefinition in an instance ' - 'dictionary. In\n' - 'contrast, non-data descriptors can be overridden by ' - 'instances.\n' - '\n' - 'Python methods (including "staticmethod()" and ' - '"classmethod()") are\n' - 'implemented as non-data descriptors. Accordingly, ' - 'instances can\n' - 'redefine and override methods. This allows individual ' - 'instances to\n' - 'acquire behaviors that differ from other instances of the ' - 'same class.\n' - '\n' - 'The "property()" function is implemented as a data ' - 'descriptor.\n' - 'Accordingly, instances cannot override the behavior of a ' - 'property.\n' - '\n' - '\n' - '__slots__\n' - '---------\n' - '\n' - 'By default, instances of both old and new-style classes ' - 'have a\n' - 'dictionary for attribute storage. This wastes space for ' - 'objects\n' - 'having very few instance variables. The space consumption ' - 'can become\n' - 'acute when creating large numbers of instances.\n' - '\n' - 'The default can be overridden by defining *__slots__* in a ' - 'new-style\n' - 'class definition. The *__slots__* declaration takes a ' - 'sequence of\n' - 'instance variables and reserves just enough space in each ' - 'instance to\n' - 'hold a value for each variable. Space is saved because ' - '*__dict__* is\n' - 'not created for each instance.\n' - '\n' - '__slots__\n' - '\n' - ' This class variable can be assigned a string, iterable, ' - 'or sequence\n' - ' of strings with variable names used by instances. If ' - 'defined in a\n' - ' new-style class, *__slots__* reserves space for the ' - 'declared\n' - ' variables and prevents the automatic creation of ' - '*__dict__* and\n' - ' *__weakref__* for each instance.\n' - '\n' - ' New in version 2.2.\n' - '\n' - 'Notes on using *__slots__*\n' - '\n' - '* When inheriting from a class without *__slots__*, the ' - '*__dict__*\n' - ' attribute of that class will always be accessible, so a ' - '*__slots__*\n' - ' definition in the subclass is meaningless.\n' - '\n' - '* Without a *__dict__* variable, instances cannot be ' - 'assigned new\n' - ' variables not listed in the *__slots__* definition. ' - 'Attempts to\n' - ' assign to an unlisted variable name raises ' - '"AttributeError". If\n' - ' dynamic assignment of new variables is desired, then ' - 'add\n' - ' "\'__dict__\'" to the sequence of strings in the ' - '*__slots__*\n' - ' declaration.\n' - '\n' - ' Changed in version 2.3: Previously, adding ' - '"\'__dict__\'" to the\n' - ' *__slots__* declaration would not enable the assignment ' - 'of new\n' - ' attributes not specifically listed in the sequence of ' - 'instance\n' - ' variable names.\n' - '\n' - '* Without a *__weakref__* variable for each instance, ' - 'classes\n' - ' defining *__slots__* do not support weak references to ' - 'its\n' - ' instances. If weak reference support is needed, then ' - 'add\n' - ' "\'__weakref__\'" to the sequence of strings in the ' - '*__slots__*\n' - ' declaration.\n' - '\n' - ' Changed in version 2.3: Previously, adding ' - '"\'__weakref__\'" to the\n' - ' *__slots__* declaration would not enable support for ' - 'weak\n' - ' references.\n' - '\n' - '* *__slots__* are implemented at the class level by ' - 'creating\n' - ' descriptors (Implementing Descriptors) for each variable ' - 'name. As a\n' - ' result, class attributes cannot be used to set default ' - 'values for\n' - ' instance variables defined by *__slots__*; otherwise, ' - 'the class\n' - ' attribute would overwrite the descriptor assignment.\n' - '\n' - '* The action of a *__slots__* declaration is limited to ' - 'the class\n' - ' where it is defined. As a result, subclasses will have ' - 'a *__dict__*\n' - ' unless they also define *__slots__* (which must only ' - 'contain names\n' - ' of any *additional* slots).\n' - '\n' - '* If a class defines a slot also defined in a base class, ' - 'the\n' - ' instance variable defined by the base class slot is ' - 'inaccessible\n' - ' (except by retrieving its descriptor directly from the ' - 'base class).\n' - ' This renders the meaning of the program undefined. In ' - 'the future, a\n' - ' check may be added to prevent this.\n' - '\n' - '* Nonempty *__slots__* does not work for classes derived ' - 'from\n' - ' "variable-length" built-in types such as "long", "str" ' - 'and "tuple".\n' - '\n' - '* Any non-string iterable may be assigned to *__slots__*. ' - 'Mappings\n' - ' may also be used; however, in the future, special ' - 'meaning may be\n' - ' assigned to the values corresponding to each key.\n' - '\n' - '* *__class__* assignment works only if both classes have ' - 'the same\n' - ' *__slots__*.\n' - '\n' - ' Changed in version 2.6: Previously, *__class__* ' - 'assignment raised an\n' - ' error if either new or old class had *__slots__*.\n' - '\n' - '\n' - 'Customizing class creation\n' - '==========================\n' - '\n' - 'By default, new-style classes are constructed using ' - '"type()". A class\n' - 'definition is read into a separate namespace and the value ' - 'of class\n' - 'name is bound to the result of "type(name, bases, dict)".\n' - '\n' - 'When the class definition is read, if *__metaclass__* is ' - 'defined then\n' - 'the callable assigned to it will be called instead of ' - '"type()". This\n' - 'allows classes or functions to be written which monitor or ' - 'alter the\n' - 'class creation process:\n' - '\n' - '* Modifying the class dictionary prior to the class being ' - 'created.\n' - '\n' - '* Returning an instance of another class -- essentially ' - 'performing\n' - ' the role of a factory function.\n' - '\n' - "These steps will have to be performed in the metaclass's " - '"__new__()"\n' - 'method -- "type.__new__()" can then be called from this ' - 'method to\n' - 'create a class with different properties. This example ' - 'adds a new\n' - 'element to the class dictionary before creating the ' - 'class:\n' - '\n' - ' class metacls(type):\n' - ' def __new__(mcs, name, bases, dict):\n' - " dict['foo'] = 'metacls was here'\n" - ' return type.__new__(mcs, name, bases, dict)\n' - '\n' - 'You can of course also override other class methods (or ' - 'add new\n' - 'methods); for example defining a custom "__call__()" ' - 'method in the\n' - 'metaclass allows custom behavior when the class is called, ' - 'e.g. not\n' - 'always creating a new instance.\n' - '\n' - '__metaclass__\n' - '\n' - ' This variable can be any callable accepting arguments ' - 'for "name",\n' - ' "bases", and "dict". Upon class creation, the callable ' - 'is used\n' - ' instead of the built-in "type()".\n' - '\n' - ' New in version 2.2.\n' - '\n' - 'The appropriate metaclass is determined by the following ' - 'precedence\n' - 'rules:\n' - '\n' - '* If "dict[\'__metaclass__\']" exists, it is used.\n' - '\n' - '* Otherwise, if there is at least one base class, its ' - 'metaclass is\n' - ' used (this looks for a *__class__* attribute first and ' - 'if not found,\n' - ' uses its type).\n' - '\n' - '* Otherwise, if a global variable named __metaclass__ ' - 'exists, it is\n' - ' used.\n' - '\n' - '* Otherwise, the old-style, classic metaclass ' - '(types.ClassType) is\n' - ' used.\n' - '\n' - 'The potential uses for metaclasses are boundless. Some ' - 'ideas that have\n' - 'been explored including logging, interface checking, ' - 'automatic\n' - 'delegation, automatic property creation, proxies, ' - 'frameworks, and\n' - 'automatic resource locking/synchronization.\n' - '\n' - '\n' - 'Customizing instance and subclass checks\n' - '========================================\n' - '\n' - 'New in version 2.6.\n' - '\n' - 'The following methods are used to override the default ' - 'behavior of the\n' - '"isinstance()" and "issubclass()" built-in functions.\n' - '\n' - 'In particular, the metaclass "abc.ABCMeta" implements ' - 'these methods in\n' - 'order to allow the addition of Abstract Base Classes ' - '(ABCs) as\n' - '"virtual base classes" to any class or type (including ' - 'built-in\n' - 'types), including other ABCs.\n' - '\n' - 'class.__instancecheck__(self, instance)\n' - '\n' - ' Return true if *instance* should be considered a ' - '(direct or\n' - ' indirect) instance of *class*. If defined, called to ' - 'implement\n' - ' "isinstance(instance, class)".\n' - '\n' - 'class.__subclasscheck__(self, subclass)\n' - '\n' - ' Return true if *subclass* should be considered a ' - '(direct or\n' - ' indirect) subclass of *class*. If defined, called to ' - 'implement\n' - ' "issubclass(subclass, class)".\n' - '\n' - 'Note that these methods are looked up on the type ' - '(metaclass) of a\n' - 'class. They cannot be defined as class methods in the ' - 'actual class.\n' - 'This is consistent with the lookup of special methods that ' - 'are called\n' - 'on instances, only in this case the instance is itself a ' - 'class.\n' - '\n' - 'See also: **PEP 3119** - Introducing Abstract Base ' - 'Classes\n' - '\n' - ' Includes the specification for customizing ' - '"isinstance()" and\n' - ' "issubclass()" behavior through "__instancecheck__()" ' - 'and\n' - ' "__subclasscheck__()", with motivation for this ' - 'functionality in\n' - ' the context of adding Abstract Base Classes (see the ' - '"abc"\n' - ' module) to the language.\n' - '\n' - '\n' - 'Emulating callable objects\n' - '==========================\n' - '\n' - 'object.__call__(self[, args...])\n' - '\n' - ' Called when the instance is "called" as a function; if ' - 'this method\n' - ' is defined, "x(arg1, arg2, ...)" is a shorthand for\n' - ' "x.__call__(arg1, arg2, ...)".\n' - '\n' - '\n' - 'Emulating container types\n' - '=========================\n' - '\n' - 'The following methods can be defined to implement ' - 'container objects.\n' - 'Containers usually are sequences (such as lists or tuples) ' - 'or mappings\n' - '(like dictionaries), but can represent other containers as ' - 'well. The\n' - 'first set of methods is used either to emulate a sequence ' - 'or to\n' - 'emulate a mapping; the difference is that for a sequence, ' - 'the\n' - 'allowable keys should be the integers *k* for which "0 <= ' - 'k < N" where\n' - '*N* is the length of the sequence, or slice objects, which ' - 'define a\n' - 'range of items. (For backwards compatibility, the method\n' - '"__getslice__()" (see below) can also be defined to handle ' - 'simple, but\n' - 'not extended slices.) It is also recommended that mappings ' - 'provide the\n' - 'methods "keys()", "values()", "items()", "has_key()", ' - '"get()",\n' - '"clear()", "setdefault()", "iterkeys()", "itervalues()",\n' - '"iteritems()", "pop()", "popitem()", "copy()", and ' - '"update()" behaving\n' - "similar to those for Python's standard dictionary " - 'objects. The\n' - '"UserDict" module provides a "DictMixin" class to help ' - 'create those\n' - 'methods from a base set of "__getitem__()", ' - '"__setitem__()",\n' - '"__delitem__()", and "keys()". Mutable sequences should ' - 'provide\n' - 'methods "append()", "count()", "index()", "extend()", ' - '"insert()",\n' - '"pop()", "remove()", "reverse()" and "sort()", like Python ' - 'standard\n' - 'list objects. Finally, sequence types should implement ' - 'addition\n' - '(meaning concatenation) and multiplication (meaning ' - 'repetition) by\n' - 'defining the methods "__add__()", "__radd__()", ' - '"__iadd__()",\n' - '"__mul__()", "__rmul__()" and "__imul__()" described ' - 'below; they\n' - 'should not define "__coerce__()" or other numerical ' - 'operators. It is\n' - 'recommended that both mappings and sequences implement ' - 'the\n' - '"__contains__()" method to allow efficient use of the "in" ' - 'operator;\n' - 'for mappings, "in" should be equivalent of "has_key()"; ' - 'for sequences,\n' - 'it should search through the values. It is further ' - 'recommended that\n' - 'both mappings and sequences implement the "__iter__()" ' - 'method to allow\n' - 'efficient iteration through the container; for mappings, ' - '"__iter__()"\n' - 'should be the same as "iterkeys()"; for sequences, it ' - 'should iterate\n' - 'through the values.\n' - '\n' - 'object.__len__(self)\n' - '\n' - ' Called to implement the built-in function "len()". ' - 'Should return\n' - ' the length of the object, an integer ">=" 0. Also, an ' - 'object that\n' - ' doesn\'t define a "__nonzero__()" method and whose ' - '"__len__()"\n' - ' method returns zero is considered to be false in a ' - 'Boolean context.\n' - '\n' - 'object.__getitem__(self, key)\n' - '\n' - ' Called to implement evaluation of "self[key]". For ' - 'sequence types,\n' - ' the accepted keys should be integers and slice ' - 'objects. Note that\n' - ' the special interpretation of negative indexes (if the ' - 'class wishes\n' - ' to emulate a sequence type) is up to the ' - '"__getitem__()" method. If\n' - ' *key* is of an inappropriate type, "TypeError" may be ' - 'raised; if of\n' - ' a value outside the set of indexes for the sequence ' - '(after any\n' - ' special interpretation of negative values), ' - '"IndexError" should be\n' - ' raised. For mapping types, if *key* is missing (not in ' - 'the\n' - ' container), "KeyError" should be raised.\n' - '\n' - ' Note: "for" loops expect that an "IndexError" will be ' - 'raised for\n' - ' illegal indexes to allow proper detection of the end ' - 'of the\n' - ' sequence.\n' - '\n' - 'object.__missing__(self, key)\n' - '\n' - ' Called by "dict"."__getitem__()" to implement ' - '"self[key]" for dict\n' - ' subclasses when key is not in the dictionary.\n' - '\n' - 'object.__setitem__(self, key, value)\n' - '\n' - ' Called to implement assignment to "self[key]". Same ' - 'note as for\n' - ' "__getitem__()". This should only be implemented for ' - 'mappings if\n' - ' the objects support changes to the values for keys, or ' - 'if new keys\n' - ' can be added, or for sequences if elements can be ' - 'replaced. The\n' - ' same exceptions should be raised for improper *key* ' - 'values as for\n' - ' the "__getitem__()" method.\n' - '\n' - 'object.__delitem__(self, key)\n' - '\n' - ' Called to implement deletion of "self[key]". Same note ' - 'as for\n' - ' "__getitem__()". This should only be implemented for ' - 'mappings if\n' - ' the objects support removal of keys, or for sequences ' - 'if elements\n' - ' can be removed from the sequence. The same exceptions ' - 'should be\n' - ' raised for improper *key* values as for the ' - '"__getitem__()" method.\n' - '\n' - 'object.__iter__(self)\n' - '\n' - ' This method is called when an iterator is required for ' - 'a container.\n' - ' This method should return a new iterator object that ' - 'can iterate\n' - ' over all the objects in the container. For mappings, ' - 'it should\n' - ' iterate over the keys of the container, and should also ' - 'be made\n' - ' available as the method "iterkeys()".\n' - '\n' - ' Iterator objects also need to implement this method; ' - 'they are\n' - ' required to return themselves. For more information on ' - 'iterator\n' - ' objects, see Iterator Types.\n' - '\n' - 'object.__reversed__(self)\n' - '\n' - ' Called (if present) by the "reversed()" built-in to ' - 'implement\n' - ' reverse iteration. It should return a new iterator ' - 'object that\n' - ' iterates over all the objects in the container in ' - 'reverse order.\n' - '\n' - ' If the "__reversed__()" method is not provided, the ' - '"reversed()"\n' - ' built-in will fall back to using the sequence protocol ' - '("__len__()"\n' - ' and "__getitem__()"). Objects that support the ' - 'sequence protocol\n' - ' should only provide "__reversed__()" if they can ' - 'provide an\n' - ' implementation that is more efficient than the one ' - 'provided by\n' - ' "reversed()".\n' - '\n' - ' New in version 2.6.\n' - '\n' - 'The membership test operators ("in" and "not in") are ' - 'normally\n' - 'implemented as an iteration through a sequence. However, ' - 'container\n' - 'objects can supply the following special method with a ' - 'more efficient\n' - 'implementation, which also does not require the object be ' - 'a sequence.\n' - '\n' - 'object.__contains__(self, item)\n' - '\n' - ' Called to implement membership test operators. Should ' - 'return true\n' - ' if *item* is in *self*, false otherwise. For mapping ' - 'objects, this\n' - ' should consider the keys of the mapping rather than the ' - 'values or\n' - ' the key-item pairs.\n' - '\n' - ' For objects that don\'t define "__contains__()", the ' - 'membership test\n' - ' first tries iteration via "__iter__()", then the old ' - 'sequence\n' - ' iteration protocol via "__getitem__()", see this ' - 'section in the\n' - ' language reference.\n' - '\n' - '\n' - 'Additional methods for emulation of sequence types\n' - '==================================================\n' - '\n' - 'The following optional methods can be defined to further ' - 'emulate\n' - 'sequence objects. Immutable sequences methods should at ' - 'most only\n' - 'define "__getslice__()"; mutable sequences might define ' - 'all three\n' - 'methods.\n' - '\n' - 'object.__getslice__(self, i, j)\n' - '\n' - ' Deprecated since version 2.0: Support slice objects as ' - 'parameters\n' - ' to the "__getitem__()" method. (However, built-in types ' - 'in CPython\n' - ' currently still implement "__getslice__()". Therefore, ' - 'you have to\n' - ' override it in derived classes when implementing ' - 'slicing.)\n' - '\n' - ' Called to implement evaluation of "self[i:j]". The ' - 'returned object\n' - ' should be of the same type as *self*. Note that ' - 'missing *i* or *j*\n' - ' in the slice expression are replaced by zero or ' - '"sys.maxsize",\n' - ' respectively. If negative indexes are used in the ' - 'slice, the\n' - ' length of the sequence is added to that index. If the ' - 'instance does\n' - ' not implement the "__len__()" method, an ' - '"AttributeError" is\n' - ' raised. No guarantee is made that indexes adjusted this ' - 'way are not\n' - ' still negative. Indexes which are greater than the ' - 'length of the\n' - ' sequence are not modified. If no "__getslice__()" is ' - 'found, a slice\n' - ' object is created instead, and passed to ' - '"__getitem__()" instead.\n' - '\n' - 'object.__setslice__(self, i, j, sequence)\n' - '\n' - ' Called to implement assignment to "self[i:j]". Same ' - 'notes for *i*\n' - ' and *j* as for "__getslice__()".\n' - '\n' - ' This method is deprecated. If no "__setslice__()" is ' - 'found, or for\n' - ' extended slicing of the form "self[i:j:k]", a slice ' - 'object is\n' - ' created, and passed to "__setitem__()", instead of ' - '"__setslice__()"\n' - ' being called.\n' - '\n' - 'object.__delslice__(self, i, j)\n' - '\n' - ' Called to implement deletion of "self[i:j]". Same notes ' - 'for *i* and\n' - ' *j* as for "__getslice__()". This method is deprecated. ' - 'If no\n' - ' "__delslice__()" is found, or for extended slicing of ' - 'the form\n' - ' "self[i:j:k]", a slice object is created, and passed ' - 'to\n' - ' "__delitem__()", instead of "__delslice__()" being ' - 'called.\n' - '\n' - 'Notice that these methods are only invoked when a single ' - 'slice with a\n' - 'single colon is used, and the slice method is available. ' - 'For slice\n' - 'operations involving extended slice notation, or in ' - 'absence of the\n' - 'slice methods, "__getitem__()", "__setitem__()" or ' - '"__delitem__()" is\n' - 'called with a slice object as argument.\n' - '\n' - 'The following example demonstrate how to make your program ' - 'or module\n' - 'compatible with earlier versions of Python (assuming that ' - 'methods\n' - '"__getitem__()", "__setitem__()" and "__delitem__()" ' - 'support slice\n' - 'objects as arguments):\n' - '\n' - ' class MyClass:\n' - ' ...\n' - ' def __getitem__(self, index):\n' - ' ...\n' - ' def __setitem__(self, index, value):\n' - ' ...\n' - ' def __delitem__(self, index):\n' - ' ...\n' - '\n' - ' if sys.version_info < (2, 0):\n' - " # They won't be defined if version is at least " - '2.0 final\n' - '\n' - ' def __getslice__(self, i, j):\n' - ' return self[max(0, i):max(0, j):]\n' - ' def __setslice__(self, i, j, seq):\n' - ' self[max(0, i):max(0, j):] = seq\n' - ' def __delslice__(self, i, j):\n' - ' del self[max(0, i):max(0, j):]\n' - ' ...\n' - '\n' - 'Note the calls to "max()"; these are necessary because of ' - 'the handling\n' - 'of negative indices before the "__*slice__()" methods are ' - 'called.\n' - 'When negative indexes are used, the "__*item__()" methods ' - 'receive them\n' - 'as provided, but the "__*slice__()" methods get a "cooked" ' - 'form of the\n' - 'index values. For each negative index value, the length ' - 'of the\n' - 'sequence is added to the index before calling the method ' - '(which may\n' - 'still result in a negative index); this is the customary ' - 'handling of\n' - 'negative indexes by the built-in sequence types, and the ' - '"__*item__()"\n' - 'methods are expected to do this as well. However, since ' - 'they should\n' - 'already be doing that, negative indexes cannot be passed ' - 'in; they must\n' - 'be constrained to the bounds of the sequence before being ' - 'passed to\n' - 'the "__*item__()" methods. Calling "max(0, i)" ' - 'conveniently returns\n' - 'the proper value.\n' - '\n' - '\n' - 'Emulating numeric types\n' - '=======================\n' - '\n' - 'The following methods can be defined to emulate numeric ' - 'objects.\n' - 'Methods corresponding to operations that are not supported ' - 'by the\n' - 'particular kind of number implemented (e.g., bitwise ' - 'operations for\n' - 'non-integral numbers) should be left undefined.\n' - '\n' - 'object.__add__(self, other)\n' - 'object.__sub__(self, other)\n' - 'object.__mul__(self, other)\n' - 'object.__floordiv__(self, other)\n' - 'object.__mod__(self, other)\n' - 'object.__divmod__(self, other)\n' - 'object.__pow__(self, other[, modulo])\n' - 'object.__lshift__(self, other)\n' - 'object.__rshift__(self, other)\n' - 'object.__and__(self, other)\n' - 'object.__xor__(self, other)\n' - 'object.__or__(self, other)\n' - '\n' - ' These methods are called to implement the binary ' - 'arithmetic\n' - ' operations ("+", "-", "*", "//", "%", "divmod()", ' - '"pow()", "**",\n' - ' "<<", ">>", "&", "^", "|"). For instance, to evaluate ' - 'the\n' - ' expression "x + y", where *x* is an instance of a class ' - 'that has an\n' - ' "__add__()" method, "x.__add__(y)" is called. The ' - '"__divmod__()"\n' - ' method should be the equivalent to using ' - '"__floordiv__()" and\n' - ' "__mod__()"; it should not be related to ' - '"__truediv__()" (described\n' - ' below). Note that "__pow__()" should be defined to ' - 'accept an\n' - ' optional third argument if the ternary version of the ' - 'built-in\n' - ' "pow()" function is to be supported.\n' - '\n' - ' If one of those methods does not support the operation ' - 'with the\n' - ' supplied arguments, it should return "NotImplemented".\n' - '\n' - 'object.__div__(self, other)\n' - 'object.__truediv__(self, other)\n' - '\n' - ' The division operator ("/") is implemented by these ' - 'methods. The\n' - ' "__truediv__()" method is used when ' - '"__future__.division" is in\n' - ' effect, otherwise "__div__()" is used. If only one of ' - 'these two\n' - ' methods is defined, the object will not support ' - 'division in the\n' - ' alternate context; "TypeError" will be raised instead.\n' - '\n' - 'object.__radd__(self, other)\n' - 'object.__rsub__(self, other)\n' - 'object.__rmul__(self, other)\n' - 'object.__rdiv__(self, other)\n' - 'object.__rtruediv__(self, other)\n' - 'object.__rfloordiv__(self, other)\n' - 'object.__rmod__(self, other)\n' - 'object.__rdivmod__(self, other)\n' - 'object.__rpow__(self, other)\n' - 'object.__rlshift__(self, other)\n' - 'object.__rrshift__(self, other)\n' - 'object.__rand__(self, other)\n' - 'object.__rxor__(self, other)\n' - 'object.__ror__(self, other)\n' - '\n' - ' These methods are called to implement the binary ' - 'arithmetic\n' - ' operations ("+", "-", "*", "/", "%", "divmod()", ' - '"pow()", "**",\n' - ' "<<", ">>", "&", "^", "|") with reflected (swapped) ' - 'operands.\n' - ' These functions are only called if the left operand ' - 'does not\n' - ' support the corresponding operation and the operands ' - 'are of\n' - ' different types. [2] For instance, to evaluate the ' - 'expression "x -\n' - ' y", where *y* is an instance of a class that has an ' - '"__rsub__()"\n' - ' method, "y.__rsub__(x)" is called if "x.__sub__(y)" ' - 'returns\n' - ' *NotImplemented*.\n' - '\n' - ' Note that ternary "pow()" will not try calling ' - '"__rpow__()" (the\n' - ' coercion rules would become too complicated).\n' - '\n' - " Note: If the right operand's type is a subclass of the " - 'left\n' - " operand's type and that subclass provides the " - 'reflected method\n' - ' for the operation, this method will be called before ' - 'the left\n' - " operand's non-reflected method. This behavior allows " - 'subclasses\n' - " to override their ancestors' operations.\n" - '\n' - 'object.__iadd__(self, other)\n' - 'object.__isub__(self, other)\n' - 'object.__imul__(self, other)\n' - 'object.__idiv__(self, other)\n' - 'object.__itruediv__(self, other)\n' - 'object.__ifloordiv__(self, other)\n' - 'object.__imod__(self, other)\n' - 'object.__ipow__(self, other[, modulo])\n' - 'object.__ilshift__(self, other)\n' - 'object.__irshift__(self, other)\n' - 'object.__iand__(self, other)\n' - 'object.__ixor__(self, other)\n' - 'object.__ior__(self, other)\n' - '\n' - ' These methods are called to implement the augmented ' - 'arithmetic\n' - ' assignments ("+=", "-=", "*=", "/=", "//=", "%=", ' - '"**=", "<<=",\n' - ' ">>=", "&=", "^=", "|="). These methods should attempt ' - 'to do the\n' - ' operation in-place (modifying *self*) and return the ' - 'result (which\n' - ' could be, but does not have to be, *self*). If a ' - 'specific method\n' - ' is not defined, the augmented assignment falls back to ' - 'the normal\n' - ' methods. For instance, to execute the statement "x += ' - 'y", where\n' - ' *x* is an instance of a class that has an "__iadd__()" ' - 'method,\n' - ' "x.__iadd__(y)" is called. If *x* is an instance of a ' - 'class that\n' - ' does not define a "__iadd__()" method, "x.__add__(y)" ' - 'and\n' - ' "y.__radd__(x)" are considered, as with the evaluation ' - 'of "x + y".\n' - '\n' - 'object.__neg__(self)\n' - 'object.__pos__(self)\n' - 'object.__abs__(self)\n' - 'object.__invert__(self)\n' - '\n' - ' Called to implement the unary arithmetic operations ' - '("-", "+",\n' - ' "abs()" and "~").\n' - '\n' - 'object.__complex__(self)\n' - 'object.__int__(self)\n' - 'object.__long__(self)\n' - 'object.__float__(self)\n' - '\n' - ' Called to implement the built-in functions "complex()", ' - '"int()",\n' - ' "long()", and "float()". Should return a value of the ' - 'appropriate\n' - ' type.\n' - '\n' - 'object.__oct__(self)\n' - 'object.__hex__(self)\n' - '\n' - ' Called to implement the built-in functions "oct()" and ' - '"hex()".\n' - ' Should return a string value.\n' - '\n' - 'object.__index__(self)\n' - '\n' - ' Called to implement "operator.index()". Also called ' - 'whenever\n' - ' Python needs an integer object (such as in slicing). ' - 'Must return\n' - ' an integer (int or long).\n' - '\n' - ' New in version 2.5.\n' - '\n' - 'object.__coerce__(self, other)\n' - '\n' - ' Called to implement "mixed-mode" numeric arithmetic. ' - 'Should either\n' - ' return a 2-tuple containing *self* and *other* ' - 'converted to a\n' - ' common numeric type, or "None" if conversion is ' - 'impossible. When\n' - ' the common type would be the type of "other", it is ' - 'sufficient to\n' - ' return "None", since the interpreter will also ask the ' - 'other object\n' - ' to attempt a coercion (but sometimes, if the ' - 'implementation of the\n' - ' other type cannot be changed, it is useful to do the ' - 'conversion to\n' - ' the other type here). A return value of ' - '"NotImplemented" is\n' - ' equivalent to returning "None".\n' - '\n' - '\n' - 'Coercion rules\n' - '==============\n' - '\n' - 'This section used to document the rules for coercion. As ' - 'the language\n' - 'has evolved, the coercion rules have become hard to ' - 'document\n' - 'precisely; documenting what one version of one particular\n' - 'implementation does is undesirable. Instead, here are ' - 'some informal\n' - 'guidelines regarding coercion. In Python 3, coercion will ' - 'not be\n' - 'supported.\n' - '\n' - '* If the left operand of a % operator is a string or ' - 'Unicode object,\n' - ' no coercion takes place and the string formatting ' - 'operation is\n' - ' invoked instead.\n' - '\n' - '* It is no longer recommended to define a coercion ' - 'operation. Mixed-\n' - " mode operations on types that don't define coercion pass " - 'the\n' - ' original arguments to the operation.\n' - '\n' - '* New-style classes (those derived from "object") never ' - 'invoke the\n' - ' "__coerce__()" method in response to a binary operator; ' - 'the only\n' - ' time "__coerce__()" is invoked is when the built-in ' - 'function\n' - ' "coerce()" is called.\n' - '\n' - '* For most intents and purposes, an operator that returns\n' - ' "NotImplemented" is treated the same as one that is not ' - 'implemented\n' - ' at all.\n' - '\n' - '* Below, "__op__()" and "__rop__()" are used to signify ' - 'the generic\n' - ' method names corresponding to an operator; "__iop__()" ' - 'is used for\n' - ' the corresponding in-place operator. For example, for ' - 'the operator\n' - ' \'"+"\', "__add__()" and "__radd__()" are used for the ' - 'left and right\n' - ' variant of the binary operator, and "__iadd__()" for the ' - 'in-place\n' - ' variant.\n' - '\n' - '* For objects *x* and *y*, first "x.__op__(y)" is tried. ' - 'If this is\n' - ' not implemented or returns "NotImplemented", ' - '"y.__rop__(x)" is\n' - ' tried. If this is also not implemented or returns ' - '"NotImplemented",\n' - ' a "TypeError" exception is raised. But see the ' - 'following exception:\n' - '\n' - '* Exception to the previous item: if the left operand is ' - 'an instance\n' - ' of a built-in type or a new-style class, and the right ' - 'operand is an\n' - ' instance of a proper subclass of that type or class and ' - 'overrides\n' - ' the base\'s "__rop__()" method, the right operand\'s ' - '"__rop__()"\n' - ' method is tried *before* the left operand\'s "__op__()" ' - 'method.\n' - '\n' - ' This is done so that a subclass can completely override ' - 'binary\n' - ' operators. Otherwise, the left operand\'s "__op__()" ' - 'method would\n' - ' always accept the right operand: when an instance of a ' - 'given class\n' - ' is expected, an instance of a subclass of that class is ' - 'always\n' - ' acceptable.\n' - '\n' - '* When either operand type defines a coercion, this ' - 'coercion is\n' - ' called before that type\'s "__op__()" or "__rop__()" ' - 'method is\n' - ' called, but no sooner. If the coercion returns an ' - 'object of a\n' - ' different type for the operand whose coercion is ' - 'invoked, part of\n' - ' the process is redone using the new object.\n' - '\n' - '* When an in-place operator (like \'"+="\') is used, if ' - 'the left\n' - ' operand implements "__iop__()", it is invoked without ' - 'any coercion.\n' - ' When the operation falls back to "__op__()" and/or ' - '"__rop__()", the\n' - ' normal coercion rules apply.\n' - '\n' - '* In "x + y", if *x* is a sequence that implements ' - 'sequence\n' - ' concatenation, sequence concatenation is invoked.\n' - '\n' - '* In "x * y", if one operand is a sequence that implements ' - 'sequence\n' - ' repetition, and the other is an integer ("int" or ' - '"long"), sequence\n' - ' repetition is invoked.\n' - '\n' - '* Rich comparisons (implemented by methods "__eq__()" and ' - 'so on)\n' - ' never use coercion. Three-way comparison (implemented ' - 'by\n' - ' "__cmp__()") does use coercion under the same conditions ' - 'as other\n' - ' binary operations use it.\n' - '\n' - '* In the current implementation, the built-in numeric ' - 'types "int",\n' - ' "long", "float", and "complex" do not use coercion. All ' - 'these types\n' - ' implement a "__coerce__()" method, for use by the ' - 'built-in\n' - ' "coerce()" function.\n' - '\n' - ' Changed in version 2.7: The complex type no longer makes ' - 'implicit\n' - ' calls to the "__coerce__()" method for mixed-type binary ' - 'arithmetic\n' - ' operations.\n' - '\n' - '\n' - 'With Statement Context Managers\n' - '===============================\n' - '\n' - 'New in version 2.5.\n' - '\n' - 'A *context manager* is an object that defines the runtime ' - 'context to\n' - 'be established when executing a "with" statement. The ' - 'context manager\n' - 'handles the entry into, and the exit from, the desired ' - 'runtime context\n' - 'for the execution of the block of code. Context managers ' - 'are normally\n' - 'invoked using the "with" statement (described in section ' - 'The with\n' - 'statement), but can also be used by directly invoking ' - 'their methods.\n' - '\n' - 'Typical uses of context managers include saving and ' - 'restoring various\n' - 'kinds of global state, locking and unlocking resources, ' - 'closing opened\n' - 'files, etc.\n' - '\n' - 'For more information on context managers, see Context ' - 'Manager Types.\n' - '\n' - 'object.__enter__(self)\n' - '\n' - ' Enter the runtime context related to this object. The ' - '"with"\n' - " statement will bind this method's return value to the " - 'target(s)\n' - ' specified in the "as" clause of the statement, if any.\n' - '\n' - 'object.__exit__(self, exc_type, exc_value, traceback)\n' - '\n' - ' Exit the runtime context related to this object. The ' - 'parameters\n' - ' describe the exception that caused the context to be ' - 'exited. If the\n' - ' context was exited without an exception, all three ' - 'arguments will\n' - ' be "None".\n' - '\n' - ' If an exception is supplied, and the method wishes to ' - 'suppress the\n' - ' exception (i.e., prevent it from being propagated), it ' - 'should\n' - ' return a true value. Otherwise, the exception will be ' - 'processed\n' - ' normally upon exit from this method.\n' - '\n' - ' Note that "__exit__()" methods should not reraise the ' - 'passed-in\n' - " exception; this is the caller's responsibility.\n" - '\n' - 'See also: **PEP 0343** - The "with" statement\n' - '\n' - ' The specification, background, and examples for the ' - 'Python "with"\n' - ' statement.\n' - '\n' - '\n' - 'Special method lookup for old-style classes\n' - '===========================================\n' - '\n' - 'For old-style classes, special methods are always looked ' - 'up in exactly\n' - 'the same way as any other method or attribute. This is the ' - 'case\n' - 'regardless of whether the method is being looked up ' - 'explicitly as in\n' - '"x.__getitem__(i)" or implicitly as in "x[i]".\n' - '\n' - 'This behaviour means that special methods may exhibit ' - 'different\n' - 'behaviour for different instances of a single old-style ' - 'class if the\n' - 'appropriate special attributes are set differently:\n' - '\n' - ' >>> class C:\n' - ' ... pass\n' - ' ...\n' - ' >>> c1 = C()\n' - ' >>> c2 = C()\n' - ' >>> c1.__len__ = lambda: 5\n' - ' >>> c2.__len__ = lambda: 9\n' - ' >>> len(c1)\n' - ' 5\n' - ' >>> len(c2)\n' - ' 9\n' - '\n' - '\n' - 'Special method lookup for new-style classes\n' - '===========================================\n' - '\n' - 'For new-style classes, implicit invocations of special ' - 'methods are\n' - 'only guaranteed to work correctly if defined on an ' - "object's type, not\n" - "in the object's instance dictionary. That behaviour is " - 'the reason why\n' - 'the following code raises an exception (unlike the ' - 'equivalent example\n' - 'with old-style classes):\n' - '\n' - ' >>> class C(object):\n' - ' ... pass\n' - ' ...\n' - ' >>> c = C()\n' - ' >>> c.__len__ = lambda: 5\n' - ' >>> len(c)\n' - ' Traceback (most recent call last):\n' - ' File "", line 1, in \n' - " TypeError: object of type 'C' has no len()\n" - '\n' - 'The rationale behind this behaviour lies with a number of ' - 'special\n' - 'methods such as "__hash__()" and "__repr__()" that are ' - 'implemented by\n' - 'all objects, including type objects. If the implicit ' - 'lookup of these\n' - 'methods used the conventional lookup process, they would ' - 'fail when\n' - 'invoked on the type object itself:\n' - '\n' - ' >>> 1 .__hash__() == hash(1)\n' - ' True\n' - ' >>> int.__hash__() == hash(int)\n' - ' Traceback (most recent call last):\n' - ' File "", line 1, in \n' - " TypeError: descriptor '__hash__' of 'int' object needs " - 'an argument\n' - '\n' - 'Incorrectly attempting to invoke an unbound method of a ' - 'class in this\n' - "way is sometimes referred to as 'metaclass confusion', and " - 'is avoided\n' - 'by bypassing the instance when looking up special ' - 'methods:\n' - '\n' - ' >>> type(1).__hash__(1) == hash(1)\n' - ' True\n' - ' >>> type(int).__hash__(int) == hash(int)\n' - ' True\n' - '\n' - 'In addition to bypassing any instance attributes in the ' - 'interest of\n' - 'correctness, implicit special method lookup generally also ' - 'bypasses\n' - 'the "__getattribute__()" method even of the object\'s ' - 'metaclass:\n' - '\n' - ' >>> class Meta(type):\n' - ' ... def __getattribute__(*args):\n' - ' ... print "Metaclass getattribute invoked"\n' - ' ... return type.__getattribute__(*args)\n' - ' ...\n' - ' >>> class C(object):\n' - ' ... __metaclass__ = Meta\n' - ' ... def __len__(self):\n' - ' ... return 10\n' - ' ... def __getattribute__(*args):\n' - ' ... print "Class getattribute invoked"\n' - ' ... return object.__getattribute__(*args)\n' - ' ...\n' - ' >>> c = C()\n' - ' >>> c.__len__() # Explicit lookup via ' - 'instance\n' - ' Class getattribute invoked\n' - ' 10\n' - ' >>> type(c).__len__(c) # Explicit lookup via ' - 'type\n' - ' Metaclass getattribute invoked\n' - ' 10\n' - ' >>> len(c) # Implicit lookup\n' - ' 10\n' - '\n' - 'Bypassing the "__getattribute__()" machinery in this ' - 'fashion provides\n' - 'significant scope for speed optimisations within the ' - 'interpreter, at\n' - 'the cost of some flexibility in the handling of special ' - 'methods (the\n' - 'special method *must* be set on the class object itself in ' - 'order to be\n' - 'consistently invoked by the interpreter).\n' - '\n' - '-[ Footnotes ]-\n' - '\n' - "[1] It *is* possible in some cases to change an object's " - 'type,\n' - ' under certain controlled conditions. It generally ' - "isn't a good\n" - ' idea though, since it can lead to some very strange ' - 'behaviour if\n' - ' it is handled incorrectly.\n' - '\n' - '[2] For operands of the same type, it is assumed that if ' - 'the non-\n' - ' reflected method (such as "__add__()") fails the ' - 'operation is not\n' - ' supported, which is why the reflected method is not ' - 'called.\n', - 'string-methods': '\n' - 'String Methods\n' - '**************\n' - '\n' - 'Below are listed the string methods which both 8-bit ' - 'strings and\n' - 'Unicode objects support. Some of them are also ' - 'available on\n' - '"bytearray" objects.\n' - '\n' - "In addition, Python's strings support the sequence type " - 'methods\n' - 'described in the Sequence Types --- str, unicode, list, ' - 'tuple,\n' - 'bytearray, buffer, xrange section. To output formatted ' - 'strings use\n' - 'template strings or the "%" operator described in the ' - 'String\n' - 'Formatting Operations section. Also, see the "re" module ' - 'for string\n' - 'functions based on regular expressions.\n' - '\n' - 'str.capitalize()\n' - '\n' - ' Return a copy of the string with its first character ' - 'capitalized\n' - ' and the rest lowercased.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.center(width[, fillchar])\n' - '\n' - ' Return centered in a string of length *width*. ' - 'Padding is done\n' - ' using the specified *fillchar* (default is a space).\n' - '\n' - ' Changed in version 2.4: Support for the *fillchar* ' - 'argument.\n' - '\n' - 'str.count(sub[, start[, end]])\n' - '\n' - ' Return the number of non-overlapping occurrences of ' - 'substring *sub*\n' - ' in the range [*start*, *end*]. Optional arguments ' - '*start* and\n' - ' *end* are interpreted as in slice notation.\n' - '\n' - 'str.decode([encoding[, errors]])\n' - '\n' - ' Decodes the string using the codec registered for ' - '*encoding*.\n' - ' *encoding* defaults to the default string encoding. ' - '*errors* may\n' - ' be given to set a different error handling scheme. ' - 'The default is\n' - ' "\'strict\'", meaning that encoding errors raise ' - '"UnicodeError".\n' - ' Other possible values are "\'ignore\'", "\'replace\'" ' - 'and any other\n' - ' name registered via "codecs.register_error()", see ' - 'section Codec\n' - ' Base Classes.\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' Changed in version 2.3: Support for other error ' - 'handling schemes\n' - ' added.\n' - '\n' - ' Changed in version 2.7: Support for keyword arguments ' - 'added.\n' - '\n' - 'str.encode([encoding[, errors]])\n' - '\n' - ' Return an encoded version of the string. Default ' - 'encoding is the\n' - ' current default string encoding. *errors* may be ' - 'given to set a\n' - ' different error handling scheme. The default for ' - '*errors* is\n' - ' "\'strict\'", meaning that encoding errors raise a ' - '"UnicodeError".\n' - ' Other possible values are "\'ignore\'", ' - '"\'replace\'",\n' - ' "\'xmlcharrefreplace\'", "\'backslashreplace\'" and ' - 'any other name\n' - ' registered via "codecs.register_error()", see section ' - 'Codec Base\n' - ' Classes. For a list of possible encodings, see ' - 'section Standard\n' - ' Encodings.\n' - '\n' - ' New in version 2.0.\n' - '\n' - ' Changed in version 2.3: Support for ' - '"\'xmlcharrefreplace\'" and\n' - ' "\'backslashreplace\'" and other error handling ' - 'schemes added.\n' - '\n' - ' Changed in version 2.7: Support for keyword arguments ' - 'added.\n' - '\n' - 'str.endswith(suffix[, start[, end]])\n' - '\n' - ' Return "True" if the string ends with the specified ' - '*suffix*,\n' - ' otherwise return "False". *suffix* can also be a ' - 'tuple of suffixes\n' - ' to look for. With optional *start*, test beginning ' - 'at that\n' - ' position. With optional *end*, stop comparing at ' - 'that position.\n' - '\n' - ' Changed in version 2.5: Accept tuples as *suffix*.\n' - '\n' - 'str.expandtabs([tabsize])\n' - '\n' - ' Return a copy of the string where all tab characters ' - 'are replaced\n' - ' by one or more spaces, depending on the current ' - 'column and the\n' - ' given tab size. Tab positions occur every *tabsize* ' - 'characters\n' - ' (default is 8, giving tab positions at columns 0, 8, ' - '16 and so on).\n' - ' To expand the string, the current column is set to ' - 'zero and the\n' - ' string is examined character by character. If the ' - 'character is a\n' - ' tab ("\\t"), one or more space characters are ' - 'inserted in the result\n' - ' until the current column is equal to the next tab ' - 'position. (The\n' - ' tab character itself is not copied.) If the ' - 'character is a newline\n' - ' ("\\n") or return ("\\r"), it is copied and the ' - 'current column is\n' - ' reset to zero. Any other character is copied ' - 'unchanged and the\n' - ' current column is incremented by one regardless of ' - 'how the\n' - ' character is represented when printed.\n' - '\n' - " >>> '01\\t012\\t0123\\t01234'.expandtabs()\n" - " '01 012 0123 01234'\n" - " >>> '01\\t012\\t0123\\t01234'.expandtabs(4)\n" - " '01 012 0123 01234'\n" - '\n' - 'str.find(sub[, start[, end]])\n' - '\n' - ' Return the lowest index in the string where substring ' - '*sub* is\n' - ' found, such that *sub* is contained in the slice ' - '"s[start:end]".\n' - ' Optional arguments *start* and *end* are interpreted ' - 'as in slice\n' - ' notation. Return "-1" if *sub* is not found.\n' - '\n' - ' Note: The "find()" method should be used only if you ' - 'need to know\n' - ' the position of *sub*. To check if *sub* is a ' - 'substring or not,\n' - ' use the "in" operator:\n' - '\n' - " >>> 'Py' in 'Python'\n" - ' True\n' - '\n' - 'str.format(*args, **kwargs)\n' - '\n' - ' Perform a string formatting operation. The string on ' - 'which this\n' - ' method is called can contain literal text or ' - 'replacement fields\n' - ' delimited by braces "{}". Each replacement field ' - 'contains either\n' - ' the numeric index of a positional argument, or the ' - 'name of a\n' - ' keyword argument. Returns a copy of the string where ' - 'each\n' - ' replacement field is replaced with the string value ' - 'of the\n' - ' corresponding argument.\n' - '\n' - ' >>> "The sum of 1 + 2 is {0}".format(1+2)\n' - " 'The sum of 1 + 2 is 3'\n" - '\n' - ' See Format String Syntax for a description of the ' - 'various\n' - ' formatting options that can be specified in format ' - 'strings.\n' - '\n' - ' This method of string formatting is the new standard ' - 'in Python 3,\n' - ' and should be preferred to the "%" formatting ' - 'described in String\n' - ' Formatting Operations in new code.\n' - '\n' - ' New in version 2.6.\n' - '\n' - 'str.index(sub[, start[, end]])\n' - '\n' - ' Like "find()", but raise "ValueError" when the ' - 'substring is not\n' - ' found.\n' - '\n' - 'str.isalnum()\n' - '\n' - ' Return true if all characters in the string are ' - 'alphanumeric and\n' - ' there is at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isalpha()\n' - '\n' - ' Return true if all characters in the string are ' - 'alphabetic and\n' - ' there is at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isdigit()\n' - '\n' - ' Return true if all characters in the string are ' - 'digits and there is\n' - ' at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.islower()\n' - '\n' - ' Return true if all cased characters [4] in the string ' - 'are lowercase\n' - ' and there is at least one cased character, false ' - 'otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isspace()\n' - '\n' - ' Return true if there are only whitespace characters ' - 'in the string\n' - ' and there is at least one character, false ' - 'otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.istitle()\n' - '\n' - ' Return true if the string is a titlecased string and ' - 'there is at\n' - ' least one character, for example uppercase characters ' - 'may only\n' - ' follow uncased characters and lowercase characters ' - 'only cased ones.\n' - ' Return false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isupper()\n' - '\n' - ' Return true if all cased characters [4] in the string ' - 'are uppercase\n' - ' and there is at least one cased character, false ' - 'otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.join(iterable)\n' - '\n' - ' Return a string which is the concatenation of the ' - 'strings in the\n' - ' *iterable* *iterable*. The separator between ' - 'elements is the\n' - ' string providing this method.\n' - '\n' - 'str.ljust(width[, fillchar])\n' - '\n' - ' Return the string left justified in a string of ' - 'length *width*.\n' - ' Padding is done using the specified *fillchar* ' - '(default is a\n' - ' space). The original string is returned if *width* ' - 'is less than or\n' - ' equal to "len(s)".\n' - '\n' - ' Changed in version 2.4: Support for the *fillchar* ' - 'argument.\n' - '\n' - 'str.lower()\n' - '\n' - ' Return a copy of the string with all the cased ' - 'characters [4]\n' - ' converted to lowercase.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.lstrip([chars])\n' - '\n' - ' Return a copy of the string with leading characters ' - 'removed. The\n' - ' *chars* argument is a string specifying the set of ' - 'characters to be\n' - ' removed. If omitted or "None", the *chars* argument ' - 'defaults to\n' - ' removing whitespace. The *chars* argument is not a ' - 'prefix; rather,\n' - ' all combinations of its values are stripped:\n' - '\n' - " >>> ' spacious '.lstrip()\n" - " 'spacious '\n" - " >>> 'www.example.com'.lstrip('cmowz.')\n" - " 'example.com'\n" - '\n' - ' Changed in version 2.2.2: Support for the *chars* ' - 'argument.\n' - '\n' - 'str.partition(sep)\n' - '\n' - ' Split the string at the first occurrence of *sep*, ' - 'and return a\n' - ' 3-tuple containing the part before the separator, the ' - 'separator\n' - ' itself, and the part after the separator. If the ' - 'separator is not\n' - ' found, return a 3-tuple containing the string itself, ' - 'followed by\n' - ' two empty strings.\n' - '\n' - ' New in version 2.5.\n' - '\n' - 'str.replace(old, new[, count])\n' - '\n' - ' Return a copy of the string with all occurrences of ' - 'substring *old*\n' - ' replaced by *new*. If the optional argument *count* ' - 'is given, only\n' - ' the first *count* occurrences are replaced.\n' - '\n' - 'str.rfind(sub[, start[, end]])\n' - '\n' - ' Return the highest index in the string where ' - 'substring *sub* is\n' - ' found, such that *sub* is contained within ' - '"s[start:end]".\n' - ' Optional arguments *start* and *end* are interpreted ' - 'as in slice\n' - ' notation. Return "-1" on failure.\n' - '\n' - 'str.rindex(sub[, start[, end]])\n' - '\n' - ' Like "rfind()" but raises "ValueError" when the ' - 'substring *sub* is\n' - ' not found.\n' - '\n' - 'str.rjust(width[, fillchar])\n' - '\n' - ' Return the string right justified in a string of ' - 'length *width*.\n' - ' Padding is done using the specified *fillchar* ' - '(default is a\n' - ' space). The original string is returned if *width* is ' - 'less than or\n' - ' equal to "len(s)".\n' - '\n' - ' Changed in version 2.4: Support for the *fillchar* ' - 'argument.\n' - '\n' - 'str.rpartition(sep)\n' - '\n' - ' Split the string at the last occurrence of *sep*, and ' - 'return a\n' - ' 3-tuple containing the part before the separator, the ' - 'separator\n' - ' itself, and the part after the separator. If the ' - 'separator is not\n' - ' found, return a 3-tuple containing two empty strings, ' - 'followed by\n' - ' the string itself.\n' - '\n' - ' New in version 2.5.\n' - '\n' - 'str.rsplit([sep[, maxsplit]])\n' - '\n' - ' Return a list of the words in the string, using *sep* ' - 'as the\n' - ' delimiter string. If *maxsplit* is given, at most ' - '*maxsplit* splits\n' - ' are done, the *rightmost* ones. If *sep* is not ' - 'specified or\n' - ' "None", any whitespace string is a separator. Except ' - 'for splitting\n' - ' from the right, "rsplit()" behaves like "split()" ' - 'which is\n' - ' described in detail below.\n' - '\n' - ' New in version 2.4.\n' - '\n' - 'str.rstrip([chars])\n' - '\n' - ' Return a copy of the string with trailing characters ' - 'removed. The\n' - ' *chars* argument is a string specifying the set of ' - 'characters to be\n' - ' removed. If omitted or "None", the *chars* argument ' - 'defaults to\n' - ' removing whitespace. The *chars* argument is not a ' - 'suffix; rather,\n' - ' all combinations of its values are stripped:\n' - '\n' - " >>> ' spacious '.rstrip()\n" - " ' spacious'\n" - " >>> 'mississippi'.rstrip('ipz')\n" - " 'mississ'\n" - '\n' - ' Changed in version 2.2.2: Support for the *chars* ' - 'argument.\n' - '\n' - 'str.split([sep[, maxsplit]])\n' - '\n' - ' Return a list of the words in the string, using *sep* ' - 'as the\n' - ' delimiter string. If *maxsplit* is given, at most ' - '*maxsplit*\n' - ' splits are done (thus, the list will have at most ' - '"maxsplit+1"\n' - ' elements). If *maxsplit* is not specified or "-1", ' - 'then there is\n' - ' no limit on the number of splits (all possible splits ' - 'are made).\n' - '\n' - ' If *sep* is given, consecutive delimiters are not ' - 'grouped together\n' - ' and are deemed to delimit empty strings (for ' - 'example,\n' - ' "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', ' - '\'2\']"). The *sep* argument\n' - ' may consist of multiple characters (for example,\n' - ' "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', ' - '\'3\']"). Splitting an\n' - ' empty string with a specified separator returns ' - '"[\'\']".\n' - '\n' - ' If *sep* is not specified or is "None", a different ' - 'splitting\n' - ' algorithm is applied: runs of consecutive whitespace ' - 'are regarded\n' - ' as a single separator, and the result will contain no ' - 'empty strings\n' - ' at the start or end if the string has leading or ' - 'trailing\n' - ' whitespace. Consequently, splitting an empty string ' - 'or a string\n' - ' consisting of just whitespace with a "None" separator ' - 'returns "[]".\n' - '\n' - ' For example, "\' 1 2 3 \'.split()" returns ' - '"[\'1\', \'2\', \'3\']", and\n' - ' "\' 1 2 3 \'.split(None, 1)" returns "[\'1\', ' - '\'2 3 \']".\n' - '\n' - 'str.splitlines([keepends])\n' - '\n' - ' Return a list of the lines in the string, breaking at ' - 'line\n' - ' boundaries. This method uses the *universal newlines* ' - 'approach to\n' - ' splitting lines. Line breaks are not included in the ' - 'resulting list\n' - ' unless *keepends* is given and true.\n' - '\n' - ' For example, "\'ab c\\n\\nde ' - 'fg\\rkl\\r\\n\'.splitlines()" returns "[\'ab\n' - ' c\', \'\', \'de fg\', \'kl\']", while the same call ' - 'with\n' - ' "splitlines(True)" returns "[\'ab c\\n\', \'\\n\', ' - '\'de fg\\r\', \'kl\\r\\n\']".\n' - '\n' - ' Unlike "split()" when a delimiter string *sep* is ' - 'given, this\n' - ' method returns an empty list for the empty string, ' - 'and a terminal\n' - ' line break does not result in an extra line.\n' - '\n' - 'str.startswith(prefix[, start[, end]])\n' - '\n' - ' Return "True" if string starts with the *prefix*, ' - 'otherwise return\n' - ' "False". *prefix* can also be a tuple of prefixes to ' - 'look for.\n' - ' With optional *start*, test string beginning at that ' - 'position.\n' - ' With optional *end*, stop comparing string at that ' - 'position.\n' - '\n' - ' Changed in version 2.5: Accept tuples as *prefix*.\n' - '\n' - 'str.strip([chars])\n' - '\n' - ' Return a copy of the string with the leading and ' - 'trailing\n' - ' characters removed. The *chars* argument is a string ' - 'specifying the\n' - ' set of characters to be removed. If omitted or ' - '"None", the *chars*\n' - ' argument defaults to removing whitespace. The *chars* ' - 'argument is\n' - ' not a prefix or suffix; rather, all combinations of ' - 'its values are\n' - ' stripped:\n' - '\n' - " >>> ' spacious '.strip()\n" - " 'spacious'\n" - " >>> 'www.example.com'.strip('cmowz.')\n" - " 'example'\n" - '\n' - ' Changed in version 2.2.2: Support for the *chars* ' - 'argument.\n' - '\n' - 'str.swapcase()\n' - '\n' - ' Return a copy of the string with uppercase characters ' - 'converted to\n' - ' lowercase and vice versa.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.title()\n' - '\n' - ' Return a titlecased version of the string where words ' - 'start with an\n' - ' uppercase character and the remaining characters are ' - 'lowercase.\n' - '\n' - ' The algorithm uses a simple language-independent ' - 'definition of a\n' - ' word as groups of consecutive letters. The ' - 'definition works in\n' - ' many contexts but it means that apostrophes in ' - 'contractions and\n' - ' possessives form word boundaries, which may not be ' - 'the desired\n' - ' result:\n' - '\n' - ' >>> "they\'re bill\'s friends from the ' - 'UK".title()\n' - ' "They\'Re Bill\'S Friends From The Uk"\n' - '\n' - ' A workaround for apostrophes can be constructed using ' - 'regular\n' - ' expressions:\n' - '\n' - ' >>> import re\n' - ' >>> def titlecase(s):\n' - ' ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n' - ' ... lambda mo: ' - 'mo.group(0)[0].upper() +\n' - ' ... ' - 'mo.group(0)[1:].lower(),\n' - ' ... s)\n' - ' ...\n' - ' >>> titlecase("they\'re bill\'s friends.")\n' - ' "They\'re Bill\'s Friends."\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.translate(table[, deletechars])\n' - '\n' - ' Return a copy of the string where all characters ' - 'occurring in the\n' - ' optional argument *deletechars* are removed, and the ' - 'remaining\n' - ' characters have been mapped through the given ' - 'translation table,\n' - ' which must be a string of length 256.\n' - '\n' - ' You can use the "maketrans()" helper function in the ' - '"string"\n' - ' module to create a translation table. For string ' - 'objects, set the\n' - ' *table* argument to "None" for translations that only ' - 'delete\n' - ' characters:\n' - '\n' - " >>> 'read this short text'.translate(None, 'aeiou')\n" - " 'rd ths shrt txt'\n" - '\n' - ' New in version 2.6: Support for a "None" *table* ' - 'argument.\n' - '\n' - ' For Unicode objects, the "translate()" method does ' - 'not accept the\n' - ' optional *deletechars* argument. Instead, it returns ' - 'a copy of the\n' - ' *s* where all characters have been mapped through the ' - 'given\n' - ' translation table which must be a mapping of Unicode ' - 'ordinals to\n' - ' Unicode ordinals, Unicode strings or "None". Unmapped ' - 'characters\n' - ' are left untouched. Characters mapped to "None" are ' - 'deleted. Note,\n' - ' a more flexible approach is to create a custom ' - 'character mapping\n' - ' codec using the "codecs" module (see ' - '"encodings.cp1251" for an\n' - ' example).\n' - '\n' - 'str.upper()\n' - '\n' - ' Return a copy of the string with all the cased ' - 'characters [4]\n' - ' converted to uppercase. Note that ' - '"str.upper().isupper()" might be\n' - ' "False" if "s" contains uncased characters or if the ' - 'Unicode\n' - ' category of the resulting character(s) is not "Lu" ' - '(Letter,\n' - ' uppercase), but e.g. "Lt" (Letter, titlecase).\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.zfill(width)\n' - '\n' - ' Return the numeric string left filled with zeros in a ' - 'string of\n' - ' length *width*. A sign prefix is handled correctly. ' - 'The original\n' - ' string is returned if *width* is less than or equal ' - 'to "len(s)".\n' - '\n' - ' New in version 2.2.2.\n' - '\n' - 'The following methods are present only on unicode ' - 'objects:\n' - '\n' - 'unicode.isnumeric()\n' - '\n' - ' Return "True" if there are only numeric characters in ' - 'S, "False"\n' - ' otherwise. Numeric characters include digit ' - 'characters, and all\n' - ' characters that have the Unicode numeric value ' - 'property, e.g.\n' - ' U+2155, VULGAR FRACTION ONE FIFTH.\n' - '\n' - 'unicode.isdecimal()\n' - '\n' - ' Return "True" if there are only decimal characters in ' - 'S, "False"\n' - ' otherwise. Decimal characters include digit ' - 'characters, and all\n' - ' characters that can be used to form decimal-radix ' - 'numbers, e.g.\n' - ' U+0660, ARABIC-INDIC DIGIT ZERO.\n', - 'strings': '\n' - 'String literals\n' - '***************\n' - '\n' - 'String literals are described by the following lexical ' - 'definitions:\n' - '\n' - ' stringliteral ::= [stringprefix](shortstring | ' - 'longstring)\n' - ' stringprefix ::= "r" | "u" | "ur" | "R" | "U" | "UR" | ' - '"Ur" | "uR"\n' - ' | "b" | "B" | "br" | "Br" | "bR" | "BR"\n' - ' shortstring ::= "\'" shortstringitem* "\'" | \'"\' ' - 'shortstringitem* \'"\'\n' - ' longstring ::= "\'\'\'" longstringitem* "\'\'\'"\n' - ' | \'"""\' longstringitem* \'"""\'\n' - ' shortstringitem ::= shortstringchar | escapeseq\n' - ' longstringitem ::= longstringchar | escapeseq\n' - ' shortstringchar ::= \n' - ' longstringchar ::= \n' - ' escapeseq ::= "\\" \n' - '\n' - 'One syntactic restriction not indicated by these productions is ' - 'that\n' - 'whitespace is not allowed between the "stringprefix" and the ' - 'rest of\n' - 'the string literal. The source character set is defined by the\n' - 'encoding declaration; it is ASCII if no encoding declaration is ' - 'given\n' - 'in the source file; see section Encoding declarations.\n' - '\n' - 'In plain English: String literals can be enclosed in matching ' - 'single\n' - 'quotes ("\'") or double quotes ("""). They can also be ' - 'enclosed in\n' - 'matching groups of three single or double quotes (these are ' - 'generally\n' - 'referred to as *triple-quoted strings*). The backslash ("\\")\n' - 'character is used to escape characters that otherwise have a ' - 'special\n' - 'meaning, such as newline, backslash itself, or the quote ' - 'character.\n' - 'String literals may optionally be prefixed with a letter ' - '"\'r\'" or\n' - '"\'R\'"; such strings are called *raw strings* and use ' - 'different rules\n' - 'for interpreting backslash escape sequences. A prefix of ' - '"\'u\'" or\n' - '"\'U\'" makes the string a Unicode string. Unicode strings use ' - 'the\n' - 'Unicode character set as defined by the Unicode Consortium and ' - 'ISO\n' - '10646. Some additional escape sequences, described below, are\n' - 'available in Unicode strings. A prefix of "\'b\'" or "\'B\'" is ' - 'ignored in\n' - 'Python 2; it indicates that the literal should become a bytes ' - 'literal\n' - 'in Python 3 (e.g. when code is automatically converted with ' - '2to3). A\n' - '"\'u\'" or "\'b\'" prefix may be followed by an "\'r\'" ' - 'prefix.\n' - '\n' - 'In triple-quoted strings, unescaped newlines and quotes are ' - 'allowed\n' - '(and are retained), except that three unescaped quotes in a ' - 'row\n' - 'terminate the string. (A "quote" is the character used to open ' - 'the\n' - 'string, i.e. either "\'" or """.)\n' - '\n' - 'Unless an "\'r\'" or "\'R\'" prefix is present, escape ' - 'sequences in\n' - 'strings are interpreted according to rules similar to those ' - 'used by\n' - 'Standard C. The recognized escape sequences are:\n' - '\n' - '+-------------------+-----------------------------------+---------+\n' - '| Escape Sequence | Meaning | ' - 'Notes |\n' - '+===================+===================================+=========+\n' - '| "\\newline" | Ignored ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\\\" | Backslash ("\\") ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\\'" | Single quote ("\'") ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\"" | Double quote (""") ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\a" | ASCII Bell (BEL) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\b" | ASCII Backspace (BS) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\f" | ASCII Formfeed (FF) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\n" | ASCII Linefeed (LF) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\N{name}" | Character named *name* in the ' - '| |\n' - '| | Unicode database (Unicode only) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\r" | ASCII Carriage Return (CR) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\t" | ASCII Horizontal Tab (TAB) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\uxxxx" | Character with 16-bit hex value | ' - '(1) |\n' - '| | *xxxx* (Unicode only) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\Uxxxxxxxx" | Character with 32-bit hex value | ' - '(2) |\n' - '| | *xxxxxxxx* (Unicode only) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\v" | ASCII Vertical Tab (VT) ' - '| |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\ooo" | Character with octal value *ooo* | ' - '(3,5) |\n' - '+-------------------+-----------------------------------+---------+\n' - '| "\\xhh" | Character with hex value *hh* | ' - '(4,5) |\n' - '+-------------------+-----------------------------------+---------+\n' - '\n' - 'Notes:\n' - '\n' - '1. Individual code units which form parts of a surrogate pair ' - 'can\n' - ' be encoded using this escape sequence.\n' - '\n' - '2. Any Unicode character can be encoded this way, but ' - 'characters\n' - ' outside the Basic Multilingual Plane (BMP) will be encoded ' - 'using a\n' - ' surrogate pair if Python is compiled to use 16-bit code ' - 'units (the\n' - ' default).\n' - '\n' - '3. As in Standard C, up to three octal digits are accepted.\n' - '\n' - '4. Unlike in Standard C, exactly two hex digits are required.\n' - '\n' - '5. In a string literal, hexadecimal and octal escapes denote ' - 'the\n' - ' byte with the given value; it is not necessary that the ' - 'byte\n' - ' encodes a character in the source character set. In a ' - 'Unicode\n' - ' literal, these escapes denote a Unicode character with the ' - 'given\n' - ' value.\n' - '\n' - 'Unlike Standard C, all unrecognized escape sequences are left ' - 'in the\n' - 'string unchanged, i.e., *the backslash is left in the string*. ' - '(This\n' - 'behavior is useful when debugging: if an escape sequence is ' - 'mistyped,\n' - 'the resulting output is more easily recognized as broken.) It ' - 'is also\n' - 'important to note that the escape sequences marked as "(Unicode ' - 'only)"\n' - 'in the table above fall into the category of unrecognized ' - 'escapes for\n' - 'non-Unicode string literals.\n' - '\n' - 'When an "\'r\'" or "\'R\'" prefix is present, a character ' - 'following a\n' - 'backslash is included in the string without change, and *all\n' - 'backslashes are left in the string*. For example, the string ' - 'literal\n' - '"r"\\n"" consists of two characters: a backslash and a ' - 'lowercase "\'n\'".\n' - 'String quotes can be escaped with a backslash, but the ' - 'backslash\n' - 'remains in the string; for example, "r"\\""" is a valid string ' - 'literal\n' - 'consisting of two characters: a backslash and a double quote; ' - '"r"\\""\n' - 'is not a valid string literal (even a raw string cannot end in ' - 'an odd\n' - 'number of backslashes). Specifically, *a raw string cannot end ' - 'in a\n' - 'single backslash* (since the backslash would escape the ' - 'following\n' - 'quote character). Note also that a single backslash followed ' - 'by a\n' - 'newline is interpreted as those two characters as part of the ' - 'string,\n' - '*not* as a line continuation.\n' - '\n' - 'When an "\'r\'" or "\'R\'" prefix is used in conjunction with a ' - '"\'u\'" or\n' - '"\'U\'" prefix, then the "\\uXXXX" and "\\UXXXXXXXX" escape ' - 'sequences are\n' - 'processed while *all other backslashes are left in the ' - 'string*. For\n' - 'example, the string literal "ur"\\u0062\\n"" consists of three ' - 'Unicode\n' - "characters: 'LATIN SMALL LETTER B', 'REVERSE SOLIDUS', and " - "'LATIN\n" - "SMALL LETTER N'. Backslashes can be escaped with a preceding\n" - 'backslash; however, both remain in the string. As a result, ' - '"\\uXXXX"\n' - 'escape sequences are only recognized when there are an odd ' - 'number of\n' - 'backslashes.\n', - 'subscriptions': '\n' - 'Subscriptions\n' - '*************\n' - '\n' - 'A subscription selects an item of a sequence (string, ' - 'tuple or list)\n' - 'or mapping (dictionary) object:\n' - '\n' - ' subscription ::= primary "[" expression_list "]"\n' - '\n' - 'The primary must evaluate to an object of a sequence or ' - 'mapping type.\n' - '\n' - 'If the primary is a mapping, the expression list must ' - 'evaluate to an\n' - 'object whose value is one of the keys of the mapping, and ' - 'the\n' - 'subscription selects the value in the mapping that ' - 'corresponds to that\n' - 'key. (The expression list is a tuple except if it has ' - 'exactly one\n' - 'item.)\n' - '\n' - 'If the primary is a sequence, the expression (list) must ' - 'evaluate to a\n' - 'plain integer. If this value is negative, the length of ' - 'the sequence\n' - 'is added to it (so that, e.g., "x[-1]" selects the last ' - 'item of "x".)\n' - 'The resulting value must be a nonnegative integer less ' - 'than the number\n' - 'of items in the sequence, and the subscription selects ' - 'the item whose\n' - 'index is that value (counting from zero).\n' - '\n' - "A string's items are characters. A character is not a " - 'separate data\n' - 'type but a string of exactly one character.\n', - 'truth': '\n' - 'Truth Value Testing\n' - '*******************\n' - '\n' - 'Any object can be tested for truth value, for use in an "if" or\n' - '"while" condition or as operand of the Boolean operations below. ' - 'The\n' - 'following values are considered false:\n' - '\n' - '* "None"\n' - '\n' - '* "False"\n' - '\n' - '* zero of any numeric type, for example, "0", "0L", "0.0", "0j".\n' - '\n' - '* any empty sequence, for example, "\'\'", "()", "[]".\n' - '\n' - '* any empty mapping, for example, "{}".\n' - '\n' - '* instances of user-defined classes, if the class defines a\n' - ' "__nonzero__()" or "__len__()" method, when that method returns ' - 'the\n' - ' integer zero or "bool" value "False". [1]\n' - '\n' - 'All other values are considered true --- so objects of many types ' - 'are\n' - 'always true.\n' - '\n' - 'Operations and built-in functions that have a Boolean result ' - 'always\n' - 'return "0" or "False" for false and "1" or "True" for true, ' - 'unless\n' - 'otherwise stated. (Important exception: the Boolean operations ' - '"or"\n' - 'and "and" always return one of their operands.)\n', - 'try': '\n' - 'The "try" statement\n' - '*******************\n' - '\n' - 'The "try" statement specifies exception handlers and/or cleanup ' - 'code\n' - 'for a group of statements:\n' - '\n' - ' try_stmt ::= try1_stmt | try2_stmt\n' - ' try1_stmt ::= "try" ":" suite\n' - ' ("except" [expression [("as" | ",") identifier]] ' - '":" suite)+\n' - ' ["else" ":" suite]\n' - ' ["finally" ":" suite]\n' - ' try2_stmt ::= "try" ":" suite\n' - ' "finally" ":" suite\n' - '\n' - 'Changed in version 2.5: In previous versions of Python,\n' - '"try"..."except"..."finally" did not work. "try"..."except" had to ' - 'be\n' - 'nested in "try"..."finally".\n' - '\n' - 'The "except" clause(s) specify one or more exception handlers. When ' - 'no\n' - 'exception occurs in the "try" clause, no exception handler is\n' - 'executed. When an exception occurs in the "try" suite, a search for ' - 'an\n' - 'exception handler is started. This search inspects the except ' - 'clauses\n' - 'in turn until one is found that matches the exception. An ' - 'expression-\n' - 'less except clause, if present, must be last; it matches any\n' - 'exception. For an except clause with an expression, that ' - 'expression\n' - 'is evaluated, and the clause matches the exception if the ' - 'resulting\n' - 'object is "compatible" with the exception. An object is ' - 'compatible\n' - 'with an exception if it is the class or a base class of the ' - 'exception\n' - 'object, or a tuple containing an item compatible with the ' - 'exception.\n' - '\n' - 'If no except clause matches the exception, the search for an ' - 'exception\n' - 'handler continues in the surrounding code and on the invocation ' - 'stack.\n' - '[1]\n' - '\n' - 'If the evaluation of an expression in the header of an except ' - 'clause\n' - 'raises an exception, the original search for a handler is canceled ' - 'and\n' - 'a search starts for the new exception in the surrounding code and ' - 'on\n' - 'the call stack (it is treated as if the entire "try" statement ' - 'raised\n' - 'the exception).\n' - '\n' - 'When a matching except clause is found, the exception is assigned ' - 'to\n' - 'the target specified in that except clause, if present, and the ' - 'except\n' - "clause's suite is executed. All except clauses must have an\n" - 'executable block. When the end of this block is reached, ' - 'execution\n' - 'continues normally after the entire try statement. (This means ' - 'that\n' - 'if two nested handlers exist for the same exception, and the ' - 'exception\n' - 'occurs in the try clause of the inner handler, the outer handler ' - 'will\n' - 'not handle the exception.)\n' - '\n' - "Before an except clause's suite is executed, details about the\n" - 'exception are assigned to three variables in the "sys" module:\n' - '"sys.exc_type" receives the object identifying the exception;\n' - '"sys.exc_value" receives the exception\'s parameter;\n' - '"sys.exc_traceback" receives a traceback object (see section The\n' - 'standard type hierarchy) identifying the point in the program ' - 'where\n' - 'the exception occurred. These details are also available through ' - 'the\n' - '"sys.exc_info()" function, which returns a tuple "(exc_type,\n' - 'exc_value, exc_traceback)". Use of the corresponding variables is\n' - 'deprecated in favor of this function, since their use is unsafe in ' - 'a\n' - 'threaded program. As of Python 1.5, the variables are restored to\n' - 'their previous values (before the call) when returning from a ' - 'function\n' - 'that handled an exception.\n' - '\n' - 'The optional "else" clause is executed if and when control flows ' - 'off\n' - 'the end of the "try" clause. [2] Exceptions in the "else" clause ' - 'are\n' - 'not handled by the preceding "except" clauses.\n' - '\n' - 'If "finally" is present, it specifies a \'cleanup\' handler. The ' - '"try"\n' - 'clause is executed, including any "except" and "else" clauses. If ' - 'an\n' - 'exception occurs in any of the clauses and is not handled, the\n' - 'exception is temporarily saved. The "finally" clause is executed. ' - 'If\n' - 'there is a saved exception, it is re-raised at the end of the\n' - '"finally" clause. If the "finally" clause raises another exception ' - 'or\n' - 'executes a "return" or "break" statement, the saved exception is\n' - 'discarded:\n' - '\n' - ' >>> def f():\n' - ' ... try:\n' - ' ... 1/0\n' - ' ... finally:\n' - ' ... return 42\n' - ' ...\n' - ' >>> f()\n' - ' 42\n' - '\n' - 'The exception information is not available to the program during\n' - 'execution of the "finally" clause.\n' - '\n' - 'When a "return", "break" or "continue" statement is executed in ' - 'the\n' - '"try" suite of a "try"..."finally" statement, the "finally" clause ' - 'is\n' - 'also executed \'on the way out.\' A "continue" statement is illegal ' - 'in\n' - 'the "finally" clause. (The reason is a problem with the current\n' - 'implementation --- this restriction may be lifted in the future).\n' - '\n' - 'The return value of a function is determined by the last "return"\n' - 'statement executed. Since the "finally" clause always executes, a\n' - '"return" statement executed in the "finally" clause will always be ' - 'the\n' - 'last one executed:\n' - '\n' - ' >>> def foo():\n' - ' ... try:\n' - " ... return 'try'\n" - ' ... finally:\n' - " ... return 'finally'\n" - ' ...\n' - ' >>> foo()\n' - " 'finally'\n" - '\n' - 'Additional information on exceptions can be found in section\n' - 'Exceptions, and information on using the "raise" statement to ' - 'generate\n' - 'exceptions may be found in section The raise statement.\n', - 'types': '\n' - 'The standard type hierarchy\n' - '***************************\n' - '\n' - 'Below is a list of the types that are built into Python. ' - 'Extension\n' - 'modules (written in C, Java, or other languages, depending on ' - 'the\n' - 'implementation) can define additional types. Future versions of\n' - 'Python may add types to the type hierarchy (e.g., rational ' - 'numbers,\n' - 'efficiently stored arrays of integers, etc.).\n' - '\n' - 'Some of the type descriptions below contain a paragraph listing\n' - "'special attributes.' These are attributes that provide access " - 'to the\n' - 'implementation and are not intended for general use. Their ' - 'definition\n' - 'may change in the future.\n' - '\n' - 'None\n' - ' This type has a single value. There is a single object with ' - 'this\n' - ' value. This object is accessed through the built-in name ' - '"None". It\n' - ' is used to signify the absence of a value in many situations, ' - 'e.g.,\n' - " it is returned from functions that don't explicitly return\n" - ' anything. Its truth value is false.\n' - '\n' - 'NotImplemented\n' - ' This type has a single value. There is a single object with ' - 'this\n' - ' value. This object is accessed through the built-in name\n' - ' "NotImplemented". Numeric methods and rich comparison methods ' - 'may\n' - ' return this value if they do not implement the operation for ' - 'the\n' - ' operands provided. (The interpreter will then try the ' - 'reflected\n' - ' operation, or some other fallback, depending on the ' - 'operator.) Its\n' - ' truth value is true.\n' - '\n' - 'Ellipsis\n' - ' This type has a single value. There is a single object with ' - 'this\n' - ' value. This object is accessed through the built-in name\n' - ' "Ellipsis". It is used to indicate the presence of the "..." ' - 'syntax\n' - ' in a slice. Its truth value is true.\n' - '\n' - '"numbers.Number"\n' - ' These are created by numeric literals and returned as results ' - 'by\n' - ' arithmetic operators and arithmetic built-in functions. ' - 'Numeric\n' - ' objects are immutable; once created their value never ' - 'changes.\n' - ' Python numbers are of course strongly related to mathematical\n' - ' numbers, but subject to the limitations of numerical ' - 'representation\n' - ' in computers.\n' - '\n' - ' Python distinguishes between integers, floating point numbers, ' - 'and\n' - ' complex numbers:\n' - '\n' - ' "numbers.Integral"\n' - ' These represent elements from the mathematical set of ' - 'integers\n' - ' (positive and negative).\n' - '\n' - ' There are three types of integers:\n' - '\n' - ' Plain integers\n' - ' These represent numbers in the range -2147483648 ' - 'through\n' - ' 2147483647. (The range may be larger on machines with a\n' - ' larger natural word size, but not smaller.) When the ' - 'result\n' - ' of an operation would fall outside this range, the ' - 'result is\n' - ' normally returned as a long integer (in some cases, the\n' - ' exception "OverflowError" is raised instead). For the\n' - ' purpose of shift and mask operations, integers are ' - 'assumed to\n' - " have a binary, 2's complement notation using 32 or more " - 'bits,\n' - ' and hiding no bits from the user (i.e., all 4294967296\n' - ' different bit patterns correspond to different values).\n' - '\n' - ' Long integers\n' - ' These represent numbers in an unlimited range, subject ' - 'to\n' - ' available (virtual) memory only. For the purpose of ' - 'shift\n' - ' and mask operations, a binary representation is assumed, ' - 'and\n' - " negative numbers are represented in a variant of 2's\n" - ' complement which gives the illusion of an infinite ' - 'string of\n' - ' sign bits extending to the left.\n' - '\n' - ' Booleans\n' - ' These represent the truth values False and True. The ' - 'two\n' - ' objects representing the values "False" and "True" are ' - 'the\n' - ' only Boolean objects. The Boolean type is a subtype of ' - 'plain\n' - ' integers, and Boolean values behave like the values 0 ' - 'and 1,\n' - ' respectively, in almost all contexts, the exception ' - 'being\n' - ' that when converted to a string, the strings ""False"" ' - 'or\n' - ' ""True"" are returned, respectively.\n' - '\n' - ' The rules for integer representation are intended to give ' - 'the\n' - ' most meaningful interpretation of shift and mask ' - 'operations\n' - ' involving negative integers and the least surprises when\n' - ' switching between the plain and long integer domains. Any\n' - ' operation, if it yields a result in the plain integer ' - 'domain,\n' - ' will yield the same result in the long integer domain or ' - 'when\n' - ' using mixed operands. The switch between domains is ' - 'transparent\n' - ' to the programmer.\n' - '\n' - ' "numbers.Real" ("float")\n' - ' These represent machine-level double precision floating ' - 'point\n' - ' numbers. You are at the mercy of the underlying machine\n' - ' architecture (and C or Java implementation) for the ' - 'accepted\n' - ' range and handling of overflow. Python does not support ' - 'single-\n' - ' precision floating point numbers; the savings in processor ' - 'and\n' - ' memory usage that are usually the reason for using these ' - 'are\n' - ' dwarfed by the overhead of using objects in Python, so ' - 'there is\n' - ' no reason to complicate the language with two kinds of ' - 'floating\n' - ' point numbers.\n' - '\n' - ' "numbers.Complex"\n' - ' These represent complex numbers as a pair of machine-level\n' - ' double precision floating point numbers. The same caveats ' - 'apply\n' - ' as for floating point numbers. The real and imaginary parts ' - 'of a\n' - ' complex number "z" can be retrieved through the read-only\n' - ' attributes "z.real" and "z.imag".\n' - '\n' - 'Sequences\n' - ' These represent finite ordered sets indexed by non-negative\n' - ' numbers. The built-in function "len()" returns the number of ' - 'items\n' - ' of a sequence. When the length of a sequence is *n*, the index ' - 'set\n' - ' contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence ' - '*a* is\n' - ' selected by "a[i]".\n' - '\n' - ' Sequences also support slicing: "a[i:j]" selects all items ' - 'with\n' - ' index *k* such that *i* "<=" *k* "<" *j*. When used as an\n' - ' expression, a slice is a sequence of the same type. This ' - 'implies\n' - ' that the index set is renumbered so that it starts at 0.\n' - '\n' - ' Some sequences also support "extended slicing" with a third ' - '"step"\n' - ' parameter: "a[i:j:k]" selects all items of *a* with index *x* ' - 'where\n' - ' "x = i + n*k", *n* ">=" "0" and *i* "<=" *x* "<" *j*.\n' - '\n' - ' Sequences are distinguished according to their mutability:\n' - '\n' - ' Immutable sequences\n' - ' An object of an immutable sequence type cannot change once ' - 'it is\n' - ' created. (If the object contains references to other ' - 'objects,\n' - ' these other objects may be mutable and may be changed; ' - 'however,\n' - ' the collection of objects directly referenced by an ' - 'immutable\n' - ' object cannot change.)\n' - '\n' - ' The following types are immutable sequences:\n' - '\n' - ' Strings\n' - ' The items of a string are characters. There is no ' - 'separate\n' - ' character type; a character is represented by a string ' - 'of one\n' - ' item. Characters represent (at least) 8-bit bytes. The\n' - ' built-in functions "chr()" and "ord()" convert between\n' - ' characters and nonnegative integers representing the ' - 'byte\n' - ' values. Bytes with the values 0-127 usually represent ' - 'the\n' - ' corresponding ASCII values, but the interpretation of ' - 'values\n' - ' is up to the program. The string data type is also used ' - 'to\n' - ' represent arrays of bytes, e.g., to hold data read from ' - 'a\n' - ' file.\n' - '\n' - ' (On systems whose native character set is not ASCII, ' - 'strings\n' - ' may use EBCDIC in their internal representation, ' - 'provided the\n' - ' functions "chr()" and "ord()" implement a mapping ' - 'between\n' - ' ASCII and EBCDIC, and string comparison preserves the ' - 'ASCII\n' - ' order. Or perhaps someone can propose a better rule?)\n' - '\n' - ' Unicode\n' - ' The items of a Unicode object are Unicode code units. ' - 'A\n' - ' Unicode code unit is represented by a Unicode object of ' - 'one\n' - ' item and can hold either a 16-bit or 32-bit value\n' - ' representing a Unicode ordinal (the maximum value for ' - 'the\n' - ' ordinal is given in "sys.maxunicode", and depends on ' - 'how\n' - ' Python is configured at compile time). Surrogate pairs ' - 'may\n' - ' be present in the Unicode object, and will be reported ' - 'as two\n' - ' separate items. The built-in functions "unichr()" and\n' - ' "ord()" convert between code units and nonnegative ' - 'integers\n' - ' representing the Unicode ordinals as defined in the ' - 'Unicode\n' - ' Standard 3.0. Conversion from and to other encodings ' - 'are\n' - ' possible through the Unicode method "encode()" and the ' - 'built-\n' - ' in function "unicode()".\n' - '\n' - ' Tuples\n' - ' The items of a tuple are arbitrary Python objects. ' - 'Tuples of\n' - ' two or more items are formed by comma-separated lists ' - 'of\n' - " expressions. A tuple of one item (a 'singleton') can " - 'be\n' - ' formed by affixing a comma to an expression (an ' - 'expression by\n' - ' itself does not create a tuple, since parentheses must ' - 'be\n' - ' usable for grouping of expressions). An empty tuple can ' - 'be\n' - ' formed by an empty pair of parentheses.\n' - '\n' - ' Mutable sequences\n' - ' Mutable sequences can be changed after they are created. ' - 'The\n' - ' subscription and slicing notations can be used as the ' - 'target of\n' - ' assignment and "del" (delete) statements.\n' - '\n' - ' There are currently two intrinsic mutable sequence types:\n' - '\n' - ' Lists\n' - ' The items of a list are arbitrary Python objects. Lists ' - 'are\n' - ' formed by placing a comma-separated list of expressions ' - 'in\n' - ' square brackets. (Note that there are no special cases ' - 'needed\n' - ' to form lists of length 0 or 1.)\n' - '\n' - ' Byte Arrays\n' - ' A bytearray object is a mutable array. They are created ' - 'by\n' - ' the built-in "bytearray()" constructor. Aside from ' - 'being\n' - ' mutable (and hence unhashable), byte arrays otherwise ' - 'provide\n' - ' the same interface and functionality as immutable bytes\n' - ' objects.\n' - '\n' - ' The extension module "array" provides an additional example ' - 'of a\n' - ' mutable sequence type.\n' - '\n' - 'Set types\n' - ' These represent unordered, finite sets of unique, immutable\n' - ' objects. As such, they cannot be indexed by any subscript. ' - 'However,\n' - ' they can be iterated over, and the built-in function "len()"\n' - ' returns the number of items in a set. Common uses for sets are ' - 'fast\n' - ' membership testing, removing duplicates from a sequence, and\n' - ' computing mathematical operations such as intersection, ' - 'union,\n' - ' difference, and symmetric difference.\n' - '\n' - ' For set elements, the same immutability rules apply as for\n' - ' dictionary keys. Note that numeric types obey the normal rules ' - 'for\n' - ' numeric comparison: if two numbers compare equal (e.g., "1" ' - 'and\n' - ' "1.0"), only one of them can be contained in a set.\n' - '\n' - ' There are currently two intrinsic set types:\n' - '\n' - ' Sets\n' - ' These represent a mutable set. They are created by the ' - 'built-in\n' - ' "set()" constructor and can be modified afterwards by ' - 'several\n' - ' methods, such as "add()".\n' - '\n' - ' Frozen sets\n' - ' These represent an immutable set. They are created by the\n' - ' built-in "frozenset()" constructor. As a frozenset is ' - 'immutable\n' - ' and *hashable*, it can be used again as an element of ' - 'another\n' - ' set, or as a dictionary key.\n' - '\n' - 'Mappings\n' - ' These represent finite sets of objects indexed by arbitrary ' - 'index\n' - ' sets. The subscript notation "a[k]" selects the item indexed ' - 'by "k"\n' - ' from the mapping "a"; this can be used in expressions and as ' - 'the\n' - ' target of assignments or "del" statements. The built-in ' - 'function\n' - ' "len()" returns the number of items in a mapping.\n' - '\n' - ' There is currently a single intrinsic mapping type:\n' - '\n' - ' Dictionaries\n' - ' These represent finite sets of objects indexed by nearly\n' - ' arbitrary values. The only types of values not acceptable ' - 'as\n' - ' keys are values containing lists or dictionaries or other\n' - ' mutable types that are compared by value rather than by ' - 'object\n' - ' identity, the reason being that the efficient ' - 'implementation of\n' - " dictionaries requires a key's hash value to remain " - 'constant.\n' - ' Numeric types used for keys obey the normal rules for ' - 'numeric\n' - ' comparison: if two numbers compare equal (e.g., "1" and ' - '"1.0")\n' - ' then they can be used interchangeably to index the same\n' - ' dictionary entry.\n' - '\n' - ' Dictionaries are mutable; they can be created by the ' - '"{...}"\n' - ' notation (see section Dictionary displays).\n' - '\n' - ' The extension modules "dbm", "gdbm", and "bsddb" provide\n' - ' additional examples of mapping types.\n' - '\n' - 'Callable types\n' - ' These are the types to which the function call operation (see\n' - ' section Calls) can be applied:\n' - '\n' - ' User-defined functions\n' - ' A user-defined function object is created by a function\n' - ' definition (see section Function definitions). It should ' - 'be\n' - ' called with an argument list containing the same number of ' - 'items\n' - " as the function's formal parameter list.\n" - '\n' - ' Special attributes:\n' - '\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | Attribute | Meaning ' - '| |\n' - ' ' - '+=========================+=================================+=============+\n' - ' | "__doc__" "func_doc" | The function\'s ' - 'documentation | Writable |\n' - ' | | string, or "None" if ' - '| |\n' - ' | | unavailable. ' - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__name__" "func_name" | The function\'s ' - 'name. | Writable |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__module__" | The name of the module the ' - '| Writable |\n' - ' | | function was defined in, or ' - '| |\n' - ' | | "None" if unavailable. ' - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__defaults__" | A tuple containing default ' - '| Writable |\n' - ' | "func_defaults" | argument values for those ' - '| |\n' - ' | | arguments that have defaults, ' - '| |\n' - ' | | or "None" if no arguments have ' - '| |\n' - ' | | a default value. ' - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__code__" "func_code" | The code object representing ' - '| Writable |\n' - ' | | the compiled function body. ' - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__globals__" | A reference to the dictionary ' - '| Read-only |\n' - ' | "func_globals" | that holds the ' - "function's | |\n" - ' | | global variables --- the global ' - '| |\n' - ' | | namespace of the module in ' - '| |\n' - ' | | which the function was defined. ' - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__dict__" "func_dict" | The namespace supporting ' - '| Writable |\n' - ' | | arbitrary function attributes. ' - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - ' | "__closure__" | "None" or a tuple of cells that ' - '| Read-only |\n' - ' | "func_closure" | contain bindings for the ' - '| |\n' - " | | function's free variables. " - '| |\n' - ' ' - '+-------------------------+---------------------------------+-------------+\n' - '\n' - ' Most of the attributes labelled "Writable" check the type ' - 'of the\n' - ' assigned value.\n' - '\n' - ' Changed in version 2.4: "func_name" is now writable.\n' - '\n' - ' Changed in version 2.6: The double-underscore attributes\n' - ' "__closure__", "__code__", "__defaults__", and ' - '"__globals__"\n' - ' were introduced as aliases for the corresponding "func_*"\n' - ' attributes for forwards compatibility with Python 3.\n' - '\n' - ' Function objects also support getting and setting ' - 'arbitrary\n' - ' attributes, which can be used, for example, to attach ' - 'metadata\n' - ' to functions. Regular attribute dot-notation is used to ' - 'get and\n' - ' set such attributes. *Note that the current implementation ' - 'only\n' - ' supports function attributes on user-defined functions. ' - 'Function\n' - ' attributes on built-in functions may be supported in the\n' - ' future.*\n' - '\n' - " Additional information about a function's definition can " - 'be\n' - ' retrieved from its code object; see the description of ' - 'internal\n' - ' types below.\n' - '\n' - ' User-defined methods\n' - ' A user-defined method object combines a class, a class ' - 'instance\n' - ' (or "None") and any callable object (normally a ' - 'user-defined\n' - ' function).\n' - '\n' - ' Special read-only attributes: "im_self" is the class ' - 'instance\n' - ' object, "im_func" is the function object; "im_class" is ' - 'the\n' - ' class of "im_self" for bound methods or the class that ' - 'asked for\n' - ' the method for unbound methods; "__doc__" is the method\'s\n' - ' documentation (same as "im_func.__doc__"); "__name__" is ' - 'the\n' - ' method name (same as "im_func.__name__"); "__module__" is ' - 'the\n' - ' name of the module the method was defined in, or "None" if\n' - ' unavailable.\n' - '\n' - ' Changed in version 2.2: "im_self" used to refer to the ' - 'class\n' - ' that defined the method.\n' - '\n' - ' Changed in version 2.6: For Python 3 ' - 'forward-compatibility,\n' - ' "im_func" is also available as "__func__", and "im_self" ' - 'as\n' - ' "__self__".\n' - '\n' - ' Methods also support accessing (but not setting) the ' - 'arbitrary\n' - ' function attributes on the underlying function object.\n' - '\n' - ' User-defined method objects may be created when getting an\n' - ' attribute of a class (perhaps via an instance of that ' - 'class), if\n' - ' that attribute is a user-defined function object, an ' - 'unbound\n' - ' user-defined method object, or a class method object. When ' - 'the\n' - ' attribute is a user-defined method object, a new method ' - 'object\n' - ' is only created if the class from which it is being ' - 'retrieved is\n' - ' the same as, or a derived class of, the class stored in ' - 'the\n' - ' original method object; otherwise, the original method ' - 'object is\n' - ' used as it is.\n' - '\n' - ' When a user-defined method object is created by retrieving ' - 'a\n' - ' user-defined function object from a class, its "im_self"\n' - ' attribute is "None" and the method object is said to be ' - 'unbound.\n' - ' When one is created by retrieving a user-defined function ' - 'object\n' - ' from a class via one of its instances, its "im_self" ' - 'attribute\n' - ' is the instance, and the method object is said to be bound. ' - 'In\n' - ' either case, the new method\'s "im_class" attribute is the ' - 'class\n' - ' from which the retrieval takes place, and its "im_func"\n' - ' attribute is the original function object.\n' - '\n' - ' When a user-defined method object is created by retrieving\n' - ' another method object from a class or instance, the ' - 'behaviour is\n' - ' the same as for a function object, except that the ' - '"im_func"\n' - ' attribute of the new instance is not the original method ' - 'object\n' - ' but its "im_func" attribute.\n' - '\n' - ' When a user-defined method object is created by retrieving ' - 'a\n' - ' class method object from a class or instance, its ' - '"im_self"\n' - ' attribute is the class itself, and its "im_func" attribute ' - 'is\n' - ' the function object underlying the class method.\n' - '\n' - ' When an unbound user-defined method object is called, the\n' - ' underlying function ("im_func") is called, with the ' - 'restriction\n' - ' that the first argument must be an instance of the proper ' - 'class\n' - ' ("im_class") or of a derived class thereof.\n' - '\n' - ' When a bound user-defined method object is called, the\n' - ' underlying function ("im_func") is called, inserting the ' - 'class\n' - ' instance ("im_self") in front of the argument list. For\n' - ' instance, when "C" is a class which contains a definition ' - 'for a\n' - ' function "f()", and "x" is an instance of "C", calling ' - '"x.f(1)"\n' - ' is equivalent to calling "C.f(x, 1)".\n' - '\n' - ' When a user-defined method object is derived from a class ' - 'method\n' - ' object, the "class instance" stored in "im_self" will ' - 'actually\n' - ' be the class itself, so that calling either "x.f(1)" or ' - '"C.f(1)"\n' - ' is equivalent to calling "f(C,1)" where "f" is the ' - 'underlying\n' - ' function.\n' - '\n' - ' Note that the transformation from function object to ' - '(unbound or\n' - ' bound) method object happens each time the attribute is\n' - ' retrieved from the class or instance. In some cases, a ' - 'fruitful\n' - ' optimization is to assign the attribute to a local variable ' - 'and\n' - ' call that local variable. Also notice that this ' - 'transformation\n' - ' only happens for user-defined functions; other callable ' - 'objects\n' - ' (and all non-callable objects) are retrieved without\n' - ' transformation. It is also important to note that ' - 'user-defined\n' - ' functions which are attributes of a class instance are not\n' - ' converted to bound methods; this *only* happens when the\n' - ' function is an attribute of the class.\n' - '\n' - ' Generator functions\n' - ' A function or method which uses the "yield" statement (see\n' - ' section The yield statement) is called a *generator ' - 'function*.\n' - ' Such a function, when called, always returns an iterator ' - 'object\n' - ' which can be used to execute the body of the function: ' - 'calling\n' - ' the iterator\'s "next()" method will cause the function to\n' - ' execute until it provides a value using the "yield" ' - 'statement.\n' - ' When the function executes a "return" statement or falls ' - 'off the\n' - ' end, a "StopIteration" exception is raised and the iterator ' - 'will\n' - ' have reached the end of the set of values to be returned.\n' - '\n' - ' Built-in functions\n' - ' A built-in function object is a wrapper around a C ' - 'function.\n' - ' Examples of built-in functions are "len()" and ' - '"math.sin()"\n' - ' ("math" is a standard built-in module). The number and type ' - 'of\n' - ' the arguments are determined by the C function. Special ' - 'read-\n' - ' only attributes: "__doc__" is the function\'s ' - 'documentation\n' - ' string, or "None" if unavailable; "__name__" is the ' - "function's\n" - ' name; "__self__" is set to "None" (but see the next item);\n' - ' "__module__" is the name of the module the function was ' - 'defined\n' - ' in or "None" if unavailable.\n' - '\n' - ' Built-in methods\n' - ' This is really a different disguise of a built-in function, ' - 'this\n' - ' time containing an object passed to the C function as an\n' - ' implicit extra argument. An example of a built-in method ' - 'is\n' - ' "alist.append()", assuming *alist* is a list object. In ' - 'this\n' - ' case, the special read-only attribute "__self__" is set to ' - 'the\n' - ' object denoted by *alist*.\n' - '\n' - ' Class Types\n' - ' Class types, or "new-style classes," are callable. These\n' - ' objects normally act as factories for new instances of\n' - ' themselves, but variations are possible for class types ' - 'that\n' - ' override "__new__()". The arguments of the call are passed ' - 'to\n' - ' "__new__()" and, in the typical case, to "__init__()" to\n' - ' initialize the new instance.\n' - '\n' - ' Classic Classes\n' - ' Class objects are described below. When a class object is\n' - ' called, a new class instance (also described below) is ' - 'created\n' - " and returned. This implies a call to the class's " - '"__init__()"\n' - ' method if it has one. Any arguments are passed on to the\n' - ' "__init__()" method. If there is no "__init__()" method, ' - 'the\n' - ' class must be called without arguments.\n' - '\n' - ' Class instances\n' - ' Class instances are described below. Class instances are\n' - ' callable only when the class has a "__call__()" method;\n' - ' "x(arguments)" is a shorthand for "x.__call__(arguments)".\n' - '\n' - 'Modules\n' - ' Modules are imported by the "import" statement (see section ' - 'The\n' - ' import statement). A module object has a namespace implemented ' - 'by a\n' - ' dictionary object (this is the dictionary referenced by the\n' - ' func_globals attribute of functions defined in the module).\n' - ' Attribute references are translated to lookups in this ' - 'dictionary,\n' - ' e.g., "m.x" is equivalent to "m.__dict__["x"]". A module ' - 'object\n' - ' does not contain the code object used to initialize the ' - 'module\n' - " (since it isn't needed once the initialization is done).\n" - '\n' - " Attribute assignment updates the module's namespace " - 'dictionary,\n' - ' e.g., "m.x = 1" is equivalent to "m.__dict__["x"] = 1".\n' - '\n' - ' Special read-only attribute: "__dict__" is the module\'s ' - 'namespace\n' - ' as a dictionary object.\n' - '\n' - ' **CPython implementation detail:** Because of the way CPython\n' - ' clears module dictionaries, the module dictionary will be ' - 'cleared\n' - ' when the module falls out of scope even if the dictionary ' - 'still has\n' - ' live references. To avoid this, copy the dictionary or keep ' - 'the\n' - ' module around while using its dictionary directly.\n' - '\n' - ' Predefined (writable) attributes: "__name__" is the module\'s ' - 'name;\n' - ' "__doc__" is the module\'s documentation string, or "None" if\n' - ' unavailable; "__file__" is the pathname of the file from which ' - 'the\n' - ' module was loaded, if it was loaded from a file. The ' - '"__file__"\n' - ' attribute is not present for C modules that are statically ' - 'linked\n' - ' into the interpreter; for extension modules loaded dynamically ' - 'from\n' - ' a shared library, it is the pathname of the shared library ' - 'file.\n' - '\n' - 'Classes\n' - ' Both class types (new-style classes) and class objects (old-\n' - ' style/classic classes) are typically created by class ' - 'definitions\n' - ' (see section Class definitions). A class has a namespace\n' - ' implemented by a dictionary object. Class attribute references ' - 'are\n' - ' translated to lookups in this dictionary, e.g., "C.x" is ' - 'translated\n' - ' to "C.__dict__["x"]" (although for new-style classes in ' - 'particular\n' - ' there are a number of hooks which allow for other means of ' - 'locating\n' - ' attributes). When the attribute name is not found there, the\n' - ' attribute search continues in the base classes. For ' - 'old-style\n' - ' classes, the search is depth-first, left-to-right in the order ' - 'of\n' - ' occurrence in the base class list. New-style classes use the ' - 'more\n' - ' complex C3 method resolution order which behaves correctly ' - 'even in\n' - " the presence of 'diamond' inheritance structures where there " - 'are\n' - ' multiple inheritance paths leading back to a common ancestor.\n' - ' Additional details on the C3 MRO used by new-style classes can ' - 'be\n' - ' found in the documentation accompanying the 2.3 release at\n' - ' https://www.python.org/download/releases/2.3/mro/.\n' - '\n' - ' When a class attribute reference (for class "C", say) would ' - 'yield a\n' - ' user-defined function object or an unbound user-defined ' - 'method\n' - ' object whose associated class is either "C" or one of its ' - 'base\n' - ' classes, it is transformed into an unbound user-defined ' - 'method\n' - ' object whose "im_class" attribute is "C". When it would yield ' - 'a\n' - ' class method object, it is transformed into a bound ' - 'user-defined\n' - ' method object whose "im_self" attribute is "C". When it ' - 'would\n' - ' yield a static method object, it is transformed into the ' - 'object\n' - ' wrapped by the static method object. See section Implementing\n' - ' Descriptors for another way in which attributes retrieved from ' - 'a\n' - ' class may differ from those actually contained in its ' - '"__dict__"\n' - ' (note that only new-style classes support descriptors).\n' - '\n' - " Class attribute assignments update the class's dictionary, " - 'never\n' - ' the dictionary of a base class.\n' - '\n' - ' A class object can be called (see above) to yield a class ' - 'instance\n' - ' (see below).\n' - '\n' - ' Special attributes: "__name__" is the class name; "__module__" ' - 'is\n' - ' the module name in which the class was defined; "__dict__" is ' - 'the\n' - ' dictionary containing the class\'s namespace; "__bases__" is a ' - 'tuple\n' - ' (possibly empty or a singleton) containing the base classes, ' - 'in the\n' - ' order of their occurrence in the base class list; "__doc__" is ' - 'the\n' - " class's documentation string, or None if undefined.\n" - '\n' - 'Class instances\n' - ' A class instance is created by calling a class object (see ' - 'above).\n' - ' A class instance has a namespace implemented as a dictionary ' - 'which\n' - ' is the first place in which attribute references are ' - 'searched.\n' - " When an attribute is not found there, and the instance's class " - 'has\n' - ' an attribute by that name, the search continues with the ' - 'class\n' - ' attributes. If a class attribute is found that is a ' - 'user-defined\n' - ' function object or an unbound user-defined method object ' - 'whose\n' - ' associated class is the class (call it "C") of the instance ' - 'for\n' - ' which the attribute reference was initiated or one of its ' - 'bases, it\n' - ' is transformed into a bound user-defined method object whose\n' - ' "im_class" attribute is "C" and whose "im_self" attribute is ' - 'the\n' - ' instance. Static method and class method objects are also\n' - ' transformed, as if they had been retrieved from class "C"; ' - 'see\n' - ' above under "Classes". See section Implementing Descriptors ' - 'for\n' - ' another way in which attributes of a class retrieved via its\n' - ' instances may differ from the objects actually stored in the\n' - ' class\'s "__dict__". If no class attribute is found, and the\n' - ' object\'s class has a "__getattr__()" method, that is called ' - 'to\n' - ' satisfy the lookup.\n' - '\n' - " Attribute assignments and deletions update the instance's\n" - " dictionary, never a class's dictionary. If the class has a\n" - ' "__setattr__()" or "__delattr__()" method, this is called ' - 'instead\n' - ' of updating the instance dictionary directly.\n' - '\n' - ' Class instances can pretend to be numbers, sequences, or ' - 'mappings\n' - ' if they have methods with certain special names. See section\n' - ' Special method names.\n' - '\n' - ' Special attributes: "__dict__" is the attribute dictionary;\n' - ' "__class__" is the instance\'s class.\n' - '\n' - 'Files\n' - ' A file object represents an open file. File objects are ' - 'created by\n' - ' the "open()" built-in function, and also by "os.popen()",\n' - ' "os.fdopen()", and the "makefile()" method of socket objects ' - '(and\n' - ' perhaps by other functions or methods provided by extension\n' - ' modules). The objects "sys.stdin", "sys.stdout" and ' - '"sys.stderr"\n' - ' are initialized to file objects corresponding to the ' - "interpreter's\n" - ' standard input, output and error streams. See File Objects ' - 'for\n' - ' complete documentation of file objects.\n' - '\n' - 'Internal types\n' - ' A few types used internally by the interpreter are exposed to ' - 'the\n' - ' user. Their definitions may change with future versions of ' - 'the\n' - ' interpreter, but they are mentioned here for completeness.\n' - '\n' - ' Code objects\n' - ' Code objects represent *byte-compiled* executable Python ' - 'code,\n' - ' or *bytecode*. The difference between a code object and a\n' - ' function object is that the function object contains an ' - 'explicit\n' - " reference to the function's globals (the module in which it " - 'was\n' - ' defined), while a code object contains no context; also ' - 'the\n' - ' default argument values are stored in the function object, ' - 'not\n' - ' in the code object (because they represent values ' - 'calculated at\n' - ' run-time). Unlike function objects, code objects are ' - 'immutable\n' - ' and contain no references (directly or indirectly) to ' - 'mutable\n' - ' objects.\n' - '\n' - ' Special read-only attributes: "co_name" gives the function ' - 'name;\n' - ' "co_argcount" is the number of positional arguments ' - '(including\n' - ' arguments with default values); "co_nlocals" is the number ' - 'of\n' - ' local variables used by the function (including ' - 'arguments);\n' - ' "co_varnames" is a tuple containing the names of the local\n' - ' variables (starting with the argument names); "co_cellvars" ' - 'is a\n' - ' tuple containing the names of local variables that are\n' - ' referenced by nested functions; "co_freevars" is a tuple\n' - ' containing the names of free variables; "co_code" is a ' - 'string\n' - ' representing the sequence of bytecode instructions; ' - '"co_consts"\n' - ' is a tuple containing the literals used by the bytecode;\n' - ' "co_names" is a tuple containing the names used by the ' - 'bytecode;\n' - ' "co_filename" is the filename from which the code was ' - 'compiled;\n' - ' "co_firstlineno" is the first line number of the function;\n' - ' "co_lnotab" is a string encoding the mapping from bytecode\n' - ' offsets to line numbers (for details see the source code of ' - 'the\n' - ' interpreter); "co_stacksize" is the required stack size\n' - ' (including local variables); "co_flags" is an integer ' - 'encoding a\n' - ' number of flags for the interpreter.\n' - '\n' - ' The following flag bits are defined for "co_flags": bit ' - '"0x04"\n' - ' is set if the function uses the "*arguments" syntax to ' - 'accept an\n' - ' arbitrary number of positional arguments; bit "0x08" is set ' - 'if\n' - ' the function uses the "**keywords" syntax to accept ' - 'arbitrary\n' - ' keyword arguments; bit "0x20" is set if the function is a\n' - ' generator.\n' - '\n' - ' Future feature declarations ("from __future__ import ' - 'division")\n' - ' also use bits in "co_flags" to indicate whether a code ' - 'object\n' - ' was compiled with a particular feature enabled: bit ' - '"0x2000" is\n' - ' set if the function was compiled with future division ' - 'enabled;\n' - ' bits "0x10" and "0x1000" were used in earlier versions of\n' - ' Python.\n' - '\n' - ' Other bits in "co_flags" are reserved for internal use.\n' - '\n' - ' If a code object represents a function, the first item in\n' - ' "co_consts" is the documentation string of the function, ' - 'or\n' - ' "None" if undefined.\n' - '\n' - ' Frame objects\n' - ' Frame objects represent execution frames. They may occur ' - 'in\n' - ' traceback objects (see below).\n' - '\n' - ' Special read-only attributes: "f_back" is to the previous ' - 'stack\n' - ' frame (towards the caller), or "None" if this is the ' - 'bottom\n' - ' stack frame; "f_code" is the code object being executed in ' - 'this\n' - ' frame; "f_locals" is the dictionary used to look up local\n' - ' variables; "f_globals" is used for global variables;\n' - ' "f_builtins" is used for built-in (intrinsic) names;\n' - ' "f_restricted" is a flag indicating whether the function ' - 'is\n' - ' executing in restricted execution mode; "f_lasti" gives ' - 'the\n' - ' precise instruction (this is an index into the bytecode ' - 'string\n' - ' of the code object).\n' - '\n' - ' Special writable attributes: "f_trace", if not "None", is ' - 'a\n' - ' function called at the start of each source code line (this ' - 'is\n' - ' used by the debugger); "f_exc_type", "f_exc_value",\n' - ' "f_exc_traceback" represent the last exception raised in ' - 'the\n' - ' parent frame provided another exception was ever raised in ' - 'the\n' - ' current frame (in all other cases they are None); ' - '"f_lineno" is\n' - ' the current line number of the frame --- writing to this ' - 'from\n' - ' within a trace function jumps to the given line (only for ' - 'the\n' - ' bottom-most frame). A debugger can implement a Jump ' - 'command\n' - ' (aka Set Next Statement) by writing to f_lineno.\n' - '\n' - ' Traceback objects\n' - ' Traceback objects represent a stack trace of an exception. ' - 'A\n' - ' traceback object is created when an exception occurs. When ' - 'the\n' - ' search for an exception handler unwinds the execution ' - 'stack, at\n' - ' each unwound level a traceback object is inserted in front ' - 'of\n' - ' the current traceback. When an exception handler is ' - 'entered,\n' - ' the stack trace is made available to the program. (See ' - 'section\n' - ' The try statement.) It is accessible as ' - '"sys.exc_traceback", and\n' - ' also as the third item of the tuple returned by\n' - ' "sys.exc_info()". The latter is the preferred interface, ' - 'since\n' - ' it works correctly when the program is using multiple ' - 'threads.\n' - ' When the program contains no suitable handler, the stack ' - 'trace\n' - ' is written (nicely formatted) to the standard error stream; ' - 'if\n' - ' the interpreter is interactive, it is also made available ' - 'to the\n' - ' user as "sys.last_traceback".\n' - '\n' - ' Special read-only attributes: "tb_next" is the next level ' - 'in the\n' - ' stack trace (towards the frame where the exception ' - 'occurred), or\n' - ' "None" if there is no next level; "tb_frame" points to the\n' - ' execution frame of the current level; "tb_lineno" gives the ' - 'line\n' - ' number where the exception occurred; "tb_lasti" indicates ' - 'the\n' - ' precise instruction. The line number and last instruction ' - 'in\n' - ' the traceback may differ from the line number of its frame\n' - ' object if the exception occurred in a "try" statement with ' - 'no\n' - ' matching except clause or with a finally clause.\n' - '\n' - ' Slice objects\n' - ' Slice objects are used to represent slices when *extended ' - 'slice\n' - ' syntax* is used. This is a slice using two colons, or ' - 'multiple\n' - ' slices or ellipses separated by commas, e.g., ' - '"a[i:j:step]",\n' - ' "a[i:j, k:l]", or "a[..., i:j]". They are also created by ' - 'the\n' - ' built-in "slice()" function.\n' - '\n' - ' Special read-only attributes: "start" is the lower bound; ' - '"stop"\n' - ' is the upper bound; "step" is the step value; each is ' - '"None" if\n' - ' omitted. These attributes can have any type.\n' - '\n' - ' Slice objects support one method:\n' - '\n' - ' slice.indices(self, length)\n' - '\n' - ' This method takes a single integer argument *length* ' - 'and\n' - ' computes information about the extended slice that the ' - 'slice\n' - ' object would describe if applied to a sequence of ' - '*length*\n' - ' items. It returns a tuple of three integers; ' - 'respectively\n' - ' these are the *start* and *stop* indices and the *step* ' - 'or\n' - ' stride length of the slice. Missing or out-of-bounds ' - 'indices\n' - ' are handled in a manner consistent with regular slices.\n' - '\n' - ' New in version 2.3.\n' - '\n' - ' Static method objects\n' - ' Static method objects provide a way of defeating the\n' - ' transformation of function objects to method objects ' - 'described\n' - ' above. A static method object is a wrapper around any ' - 'other\n' - ' object, usually a user-defined method object. When a ' - 'static\n' - ' method object is retrieved from a class or a class ' - 'instance, the\n' - ' object actually returned is the wrapped object, which is ' - 'not\n' - ' subject to any further transformation. Static method ' - 'objects are\n' - ' not themselves callable, although the objects they wrap ' - 'usually\n' - ' are. Static method objects are created by the built-in\n' - ' "staticmethod()" constructor.\n' - '\n' - ' Class method objects\n' - ' A class method object, like a static method object, is a ' - 'wrapper\n' - ' around another object that alters the way in which that ' - 'object\n' - ' is retrieved from classes and class instances. The ' - 'behaviour of\n' - ' class method objects upon such retrieval is described ' - 'above,\n' - ' under "User-defined methods". Class method objects are ' - 'created\n' - ' by the built-in "classmethod()" constructor.\n', - 'typesfunctions': '\n' - 'Functions\n' - '*********\n' - '\n' - 'Function objects are created by function definitions. ' - 'The only\n' - 'operation on a function object is to call it: ' - '"func(argument-list)".\n' - '\n' - 'There are really two flavors of function objects: ' - 'built-in functions\n' - 'and user-defined functions. Both support the same ' - 'operation (to call\n' - 'the function), but the implementation is different, ' - 'hence the\n' - 'different object types.\n' - '\n' - 'See Function definitions for more information.\n', - 'typesmapping': '\n' - 'Mapping Types --- "dict"\n' - '************************\n' - '\n' - 'A *mapping* object maps *hashable* values to arbitrary ' - 'objects.\n' - 'Mappings are mutable objects. There is currently only one ' - 'standard\n' - 'mapping type, the *dictionary*. (For other containers see ' - 'the built\n' - 'in "list", "set", and "tuple" classes, and the ' - '"collections" module.)\n' - '\n' - "A dictionary's keys are *almost* arbitrary values. Values " - 'that are\n' - 'not *hashable*, that is, values containing lists, ' - 'dictionaries or\n' - 'other mutable types (that are compared by value rather ' - 'than by object\n' - 'identity) may not be used as keys. Numeric types used for ' - 'keys obey\n' - 'the normal rules for numeric comparison: if two numbers ' - 'compare equal\n' - '(such as "1" and "1.0") then they can be used ' - 'interchangeably to index\n' - 'the same dictionary entry. (Note however, that since ' - 'computers store\n' - 'floating-point numbers as approximations it is usually ' - 'unwise to use\n' - 'them as dictionary keys.)\n' - '\n' - 'Dictionaries can be created by placing a comma-separated ' - 'list of "key:\n' - 'value" pairs within braces, for example: "{\'jack\': 4098, ' - "'sjoerd':\n" - '4127}" or "{4098: \'jack\', 4127: \'sjoerd\'}", or by the ' - '"dict"\n' - 'constructor.\n' - '\n' - 'class class dict(**kwarg)\n' - 'class class dict(mapping, **kwarg)\n' - 'class class dict(iterable, **kwarg)\n' - '\n' - ' Return a new dictionary initialized from an optional ' - 'positional\n' - ' argument and a possibly empty set of keyword ' - 'arguments.\n' - '\n' - ' If no positional argument is given, an empty dictionary ' - 'is created.\n' - ' If a positional argument is given and it is a mapping ' - 'object, a\n' - ' dictionary is created with the same key-value pairs as ' - 'the mapping\n' - ' object. Otherwise, the positional argument must be an ' - '*iterable*\n' - ' object. Each item in the iterable must itself be an ' - 'iterable with\n' - ' exactly two objects. The first object of each item ' - 'becomes a key\n' - ' in the new dictionary, and the second object the ' - 'corresponding\n' - ' value. If a key occurs more than once, the last value ' - 'for that key\n' - ' becomes the corresponding value in the new dictionary.\n' - '\n' - ' If keyword arguments are given, the keyword arguments ' - 'and their\n' - ' values are added to the dictionary created from the ' - 'positional\n' - ' argument. If a key being added is already present, the ' - 'value from\n' - ' the keyword argument replaces the value from the ' - 'positional\n' - ' argument.\n' - '\n' - ' To illustrate, the following examples all return a ' - 'dictionary equal\n' - ' to "{"one": 1, "two": 2, "three": 3}":\n' - '\n' - ' >>> a = dict(one=1, two=2, three=3)\n' - " >>> b = {'one': 1, 'two': 2, 'three': 3}\n" - " >>> c = dict(zip(['one', 'two', 'three'], [1, 2, " - '3]))\n' - " >>> d = dict([('two', 2), ('one', 1), ('three', " - '3)])\n' - " >>> e = dict({'three': 3, 'one': 1, 'two': 2})\n" - ' >>> a == b == c == d == e\n' - ' True\n' - '\n' - ' Providing keyword arguments as in the first example ' - 'only works for\n' - ' keys that are valid Python identifiers. Otherwise, any ' - 'valid keys\n' - ' can be used.\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' Changed in version 2.3: Support for building a ' - 'dictionary from\n' - ' keyword arguments added.\n' - '\n' - ' These are the operations that dictionaries support (and ' - 'therefore,\n' - ' custom mapping types should support too):\n' - '\n' - ' len(d)\n' - '\n' - ' Return the number of items in the dictionary *d*.\n' - '\n' - ' d[key]\n' - '\n' - ' Return the item of *d* with key *key*. Raises a ' - '"KeyError" if\n' - ' *key* is not in the map.\n' - '\n' - ' If a subclass of dict defines a method ' - '"__missing__()" and *key*\n' - ' is not present, the "d[key]" operation calls that ' - 'method with\n' - ' the key *key* as argument. The "d[key]" operation ' - 'then returns\n' - ' or raises whatever is returned or raised by the\n' - ' "__missing__(key)" call. No other operations or ' - 'methods invoke\n' - ' "__missing__()". If "__missing__()" is not defined, ' - '"KeyError"\n' - ' is raised. "__missing__()" must be a method; it ' - 'cannot be an\n' - ' instance variable:\n' - '\n' - ' >>> class Counter(dict):\n' - ' ... def __missing__(self, key):\n' - ' ... return 0\n' - ' >>> c = Counter()\n' - " >>> c['red']\n" - ' 0\n' - " >>> c['red'] += 1\n" - " >>> c['red']\n" - ' 1\n' - '\n' - ' The example above shows part of the implementation ' - 'of\n' - ' "collections.Counter". A different "__missing__" ' - 'method is used\n' - ' by "collections.defaultdict".\n' - '\n' - ' New in version 2.5: Recognition of __missing__ ' - 'methods of dict\n' - ' subclasses.\n' - '\n' - ' d[key] = value\n' - '\n' - ' Set "d[key]" to *value*.\n' - '\n' - ' del d[key]\n' - '\n' - ' Remove "d[key]" from *d*. Raises a "KeyError" if ' - '*key* is not\n' - ' in the map.\n' - '\n' - ' key in d\n' - '\n' - ' Return "True" if *d* has a key *key*, else "False".\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' key not in d\n' - '\n' - ' Equivalent to "not key in d".\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' iter(d)\n' - '\n' - ' Return an iterator over the keys of the dictionary. ' - 'This is a\n' - ' shortcut for "iterkeys()".\n' - '\n' - ' clear()\n' - '\n' - ' Remove all items from the dictionary.\n' - '\n' - ' copy()\n' - '\n' - ' Return a shallow copy of the dictionary.\n' - '\n' - ' fromkeys(seq[, value])\n' - '\n' - ' Create a new dictionary with keys from *seq* and ' - 'values set to\n' - ' *value*.\n' - '\n' - ' "fromkeys()" is a class method that returns a new ' - 'dictionary.\n' - ' *value* defaults to "None".\n' - '\n' - ' New in version 2.3.\n' - '\n' - ' get(key[, default])\n' - '\n' - ' Return the value for *key* if *key* is in the ' - 'dictionary, else\n' - ' *default*. If *default* is not given, it defaults to ' - '"None", so\n' - ' that this method never raises a "KeyError".\n' - '\n' - ' has_key(key)\n' - '\n' - ' Test for the presence of *key* in the dictionary. ' - '"has_key()"\n' - ' is deprecated in favor of "key in d".\n' - '\n' - ' items()\n' - '\n' - ' Return a copy of the dictionary\'s list of "(key, ' - 'value)" pairs.\n' - '\n' - ' **CPython implementation detail:** Keys and values ' - 'are listed in\n' - ' an arbitrary order which is non-random, varies ' - 'across Python\n' - " implementations, and depends on the dictionary's " - 'history of\n' - ' insertions and deletions.\n' - '\n' - ' If "items()", "keys()", "values()", "iteritems()", ' - '"iterkeys()",\n' - ' and "itervalues()" are called with no intervening ' - 'modifications\n' - ' to the dictionary, the lists will directly ' - 'correspond. This\n' - ' allows the creation of "(value, key)" pairs using ' - '"zip()":\n' - ' "pairs = zip(d.values(), d.keys())". The same ' - 'relationship\n' - ' holds for the "iterkeys()" and "itervalues()" ' - 'methods: "pairs =\n' - ' zip(d.itervalues(), d.iterkeys())" provides the same ' - 'value for\n' - ' "pairs". Another way to create the same list is ' - '"pairs = [(v, k)\n' - ' for (k, v) in d.iteritems()]".\n' - '\n' - ' iteritems()\n' - '\n' - ' Return an iterator over the dictionary\'s "(key, ' - 'value)" pairs.\n' - ' See the note for "dict.items()".\n' - '\n' - ' Using "iteritems()" while adding or deleting entries ' - 'in the\n' - ' dictionary may raise a "RuntimeError" or fail to ' - 'iterate over\n' - ' all entries.\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' iterkeys()\n' - '\n' - " Return an iterator over the dictionary's keys. See " - 'the note for\n' - ' "dict.items()".\n' - '\n' - ' Using "iterkeys()" while adding or deleting entries ' - 'in the\n' - ' dictionary may raise a "RuntimeError" or fail to ' - 'iterate over\n' - ' all entries.\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' itervalues()\n' - '\n' - " Return an iterator over the dictionary's values. " - 'See the note\n' - ' for "dict.items()".\n' - '\n' - ' Using "itervalues()" while adding or deleting ' - 'entries in the\n' - ' dictionary may raise a "RuntimeError" or fail to ' - 'iterate over\n' - ' all entries.\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' keys()\n' - '\n' - " Return a copy of the dictionary's list of keys. See " - 'the note\n' - ' for "dict.items()".\n' - '\n' - ' pop(key[, default])\n' - '\n' - ' If *key* is in the dictionary, remove it and return ' - 'its value,\n' - ' else return *default*. If *default* is not given ' - 'and *key* is\n' - ' not in the dictionary, a "KeyError" is raised.\n' - '\n' - ' New in version 2.3.\n' - '\n' - ' popitem()\n' - '\n' - ' Remove and return an arbitrary "(key, value)" pair ' - 'from the\n' - ' dictionary.\n' - '\n' - ' "popitem()" is useful to destructively iterate over ' - 'a\n' - ' dictionary, as often used in set algorithms. If the ' - 'dictionary\n' - ' is empty, calling "popitem()" raises a "KeyError".\n' - '\n' - ' setdefault(key[, default])\n' - '\n' - ' If *key* is in the dictionary, return its value. If ' - 'not, insert\n' - ' *key* with a value of *default* and return ' - '*default*. *default*\n' - ' defaults to "None".\n' - '\n' - ' update([other])\n' - '\n' - ' Update the dictionary with the key/value pairs from ' - '*other*,\n' - ' overwriting existing keys. Return "None".\n' - '\n' - ' "update()" accepts either another dictionary object ' - 'or an\n' - ' iterable of key/value pairs (as tuples or other ' - 'iterables of\n' - ' length two). If keyword arguments are specified, ' - 'the dictionary\n' - ' is then updated with those key/value pairs: ' - '"d.update(red=1,\n' - ' blue=2)".\n' - '\n' - ' Changed in version 2.4: Allowed the argument to be ' - 'an iterable\n' - ' of key/value pairs and allowed keyword arguments.\n' - '\n' - ' values()\n' - '\n' - " Return a copy of the dictionary's list of values. " - 'See the note\n' - ' for "dict.items()".\n' - '\n' - ' viewitems()\n' - '\n' - ' Return a new view of the dictionary\'s items ("(key, ' - 'value)"\n' - ' pairs). See below for documentation of view ' - 'objects.\n' - '\n' - ' New in version 2.7.\n' - '\n' - ' viewkeys()\n' - '\n' - " Return a new view of the dictionary's keys. See " - 'below for\n' - ' documentation of view objects.\n' - '\n' - ' New in version 2.7.\n' - '\n' - ' viewvalues()\n' - '\n' - " Return a new view of the dictionary's values. See " - 'below for\n' - ' documentation of view objects.\n' - '\n' - ' New in version 2.7.\n' - '\n' - ' Dictionaries compare equal if and only if they have the ' - 'same "(key,\n' - ' value)" pairs.\n' - '\n' - '\n' - 'Dictionary view objects\n' - '=======================\n' - '\n' - 'The objects returned by "dict.viewkeys()", ' - '"dict.viewvalues()" and\n' - '"dict.viewitems()" are *view objects*. They provide a ' - 'dynamic view on\n' - "the dictionary's entries, which means that when the " - 'dictionary\n' - 'changes, the view reflects these changes.\n' - '\n' - 'Dictionary views can be iterated over to yield their ' - 'respective data,\n' - 'and support membership tests:\n' - '\n' - 'len(dictview)\n' - '\n' - ' Return the number of entries in the dictionary.\n' - '\n' - 'iter(dictview)\n' - '\n' - ' Return an iterator over the keys, values or items ' - '(represented as\n' - ' tuples of "(key, value)") in the dictionary.\n' - '\n' - ' Keys and values are iterated over in an arbitrary order ' - 'which is\n' - ' non-random, varies across Python implementations, and ' - 'depends on\n' - " the dictionary's history of insertions and deletions. " - 'If keys,\n' - ' values and items views are iterated over with no ' - 'intervening\n' - ' modifications to the dictionary, the order of items ' - 'will directly\n' - ' correspond. This allows the creation of "(value, key)" ' - 'pairs using\n' - ' "zip()": "pairs = zip(d.values(), d.keys())". Another ' - 'way to\n' - ' create the same list is "pairs = [(v, k) for (k, v) in ' - 'd.items()]".\n' - '\n' - ' Iterating views while adding or deleting entries in the ' - 'dictionary\n' - ' may raise a "RuntimeError" or fail to iterate over all ' - 'entries.\n' - '\n' - 'x in dictview\n' - '\n' - ' Return "True" if *x* is in the underlying dictionary\'s ' - 'keys, values\n' - ' or items (in the latter case, *x* should be a "(key, ' - 'value)"\n' - ' tuple).\n' - '\n' - 'Keys views are set-like since their entries are unique and ' - 'hashable.\n' - 'If all values are hashable, so that (key, value) pairs are ' - 'unique and\n' - 'hashable, then the items view is also set-like. (Values ' - 'views are not\n' - 'treated as set-like since the entries are generally not ' - 'unique.) Then\n' - 'these set operations are available ("other" refers either ' - 'to another\n' - 'view or a set):\n' - '\n' - 'dictview & other\n' - '\n' - ' Return the intersection of the dictview and the other ' - 'object as a\n' - ' new set.\n' - '\n' - 'dictview | other\n' - '\n' - ' Return the union of the dictview and the other object ' - 'as a new set.\n' - '\n' - 'dictview - other\n' - '\n' - ' Return the difference between the dictview and the ' - 'other object\n' - " (all elements in *dictview* that aren't in *other*) as " - 'a new set.\n' - '\n' - 'dictview ^ other\n' - '\n' - ' Return the symmetric difference (all elements either in ' - '*dictview*\n' - ' or *other*, but not in both) of the dictview and the ' - 'other object\n' - ' as a new set.\n' - '\n' - 'An example of dictionary view usage:\n' - '\n' - " >>> dishes = {'eggs': 2, 'sausage': 1, 'bacon': 1, " - "'spam': 500}\n" - ' >>> keys = dishes.viewkeys()\n' - ' >>> values = dishes.viewvalues()\n' - '\n' - ' >>> # iteration\n' - ' >>> n = 0\n' - ' >>> for val in values:\n' - ' ... n += val\n' - ' >>> print(n)\n' - ' 504\n' - '\n' - ' >>> # keys and values are iterated over in the same ' - 'order\n' - ' >>> list(keys)\n' - " ['eggs', 'bacon', 'sausage', 'spam']\n" - ' >>> list(values)\n' - ' [2, 1, 1, 500]\n' - '\n' - ' >>> # view objects are dynamic and reflect dict ' - 'changes\n' - " >>> del dishes['eggs']\n" - " >>> del dishes['sausage']\n" - ' >>> list(keys)\n' - " ['spam', 'bacon']\n" - '\n' - ' >>> # set operations\n' - " >>> keys & {'eggs', 'bacon', 'salad'}\n" - " {'bacon'}\n", - 'typesmethods': '\n' - 'Methods\n' - '*******\n' - '\n' - 'Methods are functions that are called using the attribute ' - 'notation.\n' - 'There are two flavors: built-in methods (such as ' - '"append()" on lists)\n' - 'and class instance methods. Built-in methods are ' - 'described with the\n' - 'types that support them.\n' - '\n' - 'The implementation adds two special read-only attributes ' - 'to class\n' - 'instance methods: "m.im_self" is the object on which the ' - 'method\n' - 'operates, and "m.im_func" is the function implementing the ' - 'method.\n' - 'Calling "m(arg-1, arg-2, ..., arg-n)" is completely ' - 'equivalent to\n' - 'calling "m.im_func(m.im_self, arg-1, arg-2, ..., arg-n)".\n' - '\n' - 'Class instance methods are either *bound* or *unbound*, ' - 'referring to\n' - 'whether the method was accessed through an instance or a ' - 'class,\n' - 'respectively. When a method is unbound, its "im_self" ' - 'attribute will\n' - 'be "None" and if called, an explicit "self" object must be ' - 'passed as\n' - 'the first argument. In this case, "self" must be an ' - 'instance of the\n' - "unbound method's class (or a subclass of that class), " - 'otherwise a\n' - '"TypeError" is raised.\n' - '\n' - 'Like function objects, methods objects support getting ' - 'arbitrary\n' - 'attributes. However, since method attributes are actually ' - 'stored on\n' - 'the underlying function object ("meth.im_func"), setting ' - 'method\n' - 'attributes on either bound or unbound methods is ' - 'disallowed.\n' - 'Attempting to set an attribute on a method results in an\n' - '"AttributeError" being raised. In order to set a method ' - 'attribute,\n' - 'you need to explicitly set it on the underlying function ' - 'object:\n' - '\n' - ' >>> class C:\n' - ' ... def method(self):\n' - ' ... pass\n' - ' ...\n' - ' >>> c = C()\n' - " >>> c.method.whoami = 'my name is method' # can't set " - 'on the method\n' - ' Traceback (most recent call last):\n' - ' File "", line 1, in \n' - " AttributeError: 'instancemethod' object has no " - "attribute 'whoami'\n" - " >>> c.method.im_func.whoami = 'my name is method'\n" - ' >>> c.method.whoami\n' - " 'my name is method'\n" - '\n' - 'See The standard type hierarchy for more information.\n', - 'typesmodules': '\n' - 'Modules\n' - '*******\n' - '\n' - 'The only special operation on a module is attribute ' - 'access: "m.name",\n' - 'where *m* is a module and *name* accesses a name defined ' - "in *m*'s\n" - 'symbol table. Module attributes can be assigned to. (Note ' - 'that the\n' - '"import" statement is not, strictly speaking, an operation ' - 'on a module\n' - 'object; "import foo" does not require a module object ' - 'named *foo* to\n' - 'exist, rather it requires an (external) *definition* for a ' - 'module\n' - 'named *foo* somewhere.)\n' - '\n' - 'A special attribute of every module is "__dict__". This is ' - 'the\n' - "dictionary containing the module's symbol table. Modifying " - 'this\n' - "dictionary will actually change the module's symbol table, " - 'but direct\n' - 'assignment to the "__dict__" attribute is not possible ' - '(you can write\n' - '"m.__dict__[\'a\'] = 1", which defines "m.a" to be "1", ' - "but you can't\n" - 'write "m.__dict__ = {}"). Modifying "__dict__" directly ' - 'is not\n' - 'recommended.\n' - '\n' - 'Modules built into the interpreter are written like this: ' - '"". If loaded from a file, they are ' - 'written as\n' - '"".\n', - 'typesseq': '\n' - 'Sequence Types --- "str", "unicode", "list", "tuple", ' - '"bytearray", "buffer", "xrange"\n' - '*************************************************************************************\n' - '\n' - 'There are seven sequence types: strings, Unicode strings, ' - 'lists,\n' - 'tuples, bytearrays, buffers, and xrange objects.\n' - '\n' - 'For other containers see the built in "dict" and "set" ' - 'classes, and\n' - 'the "collections" module.\n' - '\n' - 'String literals are written in single or double quotes: ' - '"\'xyzzy\'",\n' - '""frobozz"". See String literals for more about string ' - 'literals.\n' - 'Unicode strings are much like strings, but are specified in ' - 'the syntax\n' - 'using a preceding "\'u\'" character: "u\'abc\'", "u"def"". In ' - 'addition to\n' - 'the functionality described here, there are also ' - 'string-specific\n' - 'methods described in the String Methods section. Lists are ' - 'constructed\n' - 'with square brackets, separating items with commas: "[a, b, ' - 'c]".\n' - 'Tuples are constructed by the comma operator (not within ' - 'square\n' - 'brackets), with or without enclosing parentheses, but an empty ' - 'tuple\n' - 'must have the enclosing parentheses, such as "a, b, c" or ' - '"()". A\n' - 'single item tuple must have a trailing comma, such as "(d,)".\n' - '\n' - 'Bytearray objects are created with the built-in function\n' - '"bytearray()".\n' - '\n' - 'Buffer objects are not directly supported by Python syntax, ' - 'but can be\n' - 'created by calling the built-in function "buffer()". They ' - "don't\n" - 'support concatenation or repetition.\n' - '\n' - 'Objects of type xrange are similar to buffers in that there is ' - 'no\n' - 'specific syntax to create them, but they are created using ' - 'the\n' - '"xrange()" function. They don\'t support slicing, ' - 'concatenation or\n' - 'repetition, and using "in", "not in", "min()" or "max()" on ' - 'them is\n' - 'inefficient.\n' - '\n' - 'Most sequence types support the following operations. The ' - '"in" and\n' - '"not in" operations have the same priorities as the ' - 'comparison\n' - 'operations. The "+" and "*" operations have the same priority ' - 'as the\n' - 'corresponding numeric operations. [3] Additional methods are ' - 'provided\n' - 'for Mutable Sequence Types.\n' - '\n' - 'This table lists the sequence operations sorted in ascending ' - 'priority.\n' - 'In the table, *s* and *t* are sequences of the same type; *n*, ' - '*i* and\n' - '*j* are integers:\n' - '\n' - '+--------------------+----------------------------------+------------+\n' - '| Operation | Result | ' - 'Notes |\n' - '+====================+==================================+============+\n' - '| "x in s" | "True" if an item of *s* is | ' - '(1) |\n' - '| | equal to *x*, else "False" ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "x not in s" | "False" if an item of *s* is | ' - '(1) |\n' - '| | equal to *x*, else "True" ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s + t" | the concatenation of *s* and *t* | ' - '(6) |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s * n, n * s" | equivalent to adding *s* to | ' - '(2) |\n' - '| | itself *n* times ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s[i]" | *i*th item of *s*, origin 0 | ' - '(3) |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s[i:j]" | slice of *s* from *i* to *j* | ' - '(3)(4) |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s[i:j:k]" | slice of *s* from *i* to *j* | ' - '(3)(5) |\n' - '| | with step *k* ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "len(s)" | length of *s* ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "min(s)" | smallest item of *s* ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "max(s)" | largest item of *s* ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s.index(x)" | index of the first occurrence of ' - '| |\n' - '| | *x* in *s* ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '| "s.count(x)" | total number of occurrences of ' - '| |\n' - '| | *x* in *s* ' - '| |\n' - '+--------------------+----------------------------------+------------+\n' - '\n' - 'Sequence types also support comparisons. In particular, tuples ' - 'and\n' - 'lists are compared lexicographically by comparing ' - 'corresponding\n' - 'elements. This means that to compare equal, every element must ' - 'compare\n' - 'equal and the two sequences must be of the same type and have ' - 'the same\n' - 'length. (For full details see Comparisons in the language ' - 'reference.)\n' - '\n' - 'Notes:\n' - '\n' - '1. When *s* is a string or Unicode string object the "in" and ' - '"not\n' - ' in" operations act like a substring test. In Python ' - 'versions\n' - ' before 2.3, *x* had to be a string of length 1. In Python ' - '2.3 and\n' - ' beyond, *x* may be a string of any length.\n' - '\n' - '2. Values of *n* less than "0" are treated as "0" (which ' - 'yields an\n' - ' empty sequence of the same type as *s*). Note that items ' - 'in the\n' - ' sequence *s* are not copied; they are referenced multiple ' - 'times.\n' - ' This often haunts new Python programmers; consider:\n' - '\n' - ' >>> lists = [[]] * 3\n' - ' >>> lists\n' - ' [[], [], []]\n' - ' >>> lists[0].append(3)\n' - ' >>> lists\n' - ' [[3], [3], [3]]\n' - '\n' - ' What has happened is that "[[]]" is a one-element list ' - 'containing\n' - ' an empty list, so all three elements of "[[]] * 3" are ' - 'references\n' - ' to this single empty list. Modifying any of the elements ' - 'of\n' - ' "lists" modifies this single list. You can create a list ' - 'of\n' - ' different lists this way:\n' - '\n' - ' >>> lists = [[] for i in range(3)]\n' - ' >>> lists[0].append(3)\n' - ' >>> lists[1].append(5)\n' - ' >>> lists[2].append(7)\n' - ' >>> lists\n' - ' [[3], [5], [7]]\n' - '\n' - ' Further explanation is available in the FAQ entry How do I ' - 'create a\n' - ' multidimensional list?.\n' - '\n' - '3. If *i* or *j* is negative, the index is relative to the end ' - 'of\n' - ' the string: "len(s) + i" or "len(s) + j" is substituted. ' - 'But note\n' - ' that "-0" is still "0".\n' - '\n' - '4. The slice of *s* from *i* to *j* is defined as the sequence ' - 'of\n' - ' items with index *k* such that "i <= k < j". If *i* or *j* ' - 'is\n' - ' greater than "len(s)", use "len(s)". If *i* is omitted or ' - '"None",\n' - ' use "0". If *j* is omitted or "None", use "len(s)". If ' - '*i* is\n' - ' greater than or equal to *j*, the slice is empty.\n' - '\n' - '5. The slice of *s* from *i* to *j* with step *k* is defined ' - 'as the\n' - ' sequence of items with index "x = i + n*k" such that "0 <= ' - 'n <\n' - ' (j-i)/k". In other words, the indices are "i", "i+k", ' - '"i+2*k",\n' - ' "i+3*k" and so on, stopping when *j* is reached (but never\n' - ' including *j*). If *i* or *j* is greater than "len(s)", ' - 'use\n' - ' "len(s)". If *i* or *j* are omitted or "None", they become ' - '"end"\n' - ' values (which end depends on the sign of *k*). Note, *k* ' - 'cannot be\n' - ' zero. If *k* is "None", it is treated like "1".\n' - '\n' - '6. **CPython implementation detail:** If *s* and *t* are both\n' - ' strings, some Python implementations such as CPython can ' - 'usually\n' - ' perform an in-place optimization for assignments of the ' - 'form "s = s\n' - ' + t" or "s += t". When applicable, this optimization ' - 'makes\n' - ' quadratic run-time much less likely. This optimization is ' - 'both\n' - ' version and implementation dependent. For performance ' - 'sensitive\n' - ' code, it is preferable to use the "str.join()" method which ' - 'assures\n' - ' consistent linear concatenation performance across versions ' - 'and\n' - ' implementations.\n' - '\n' - ' Changed in version 2.4: Formerly, string concatenation ' - 'never\n' - ' occurred in-place.\n' - '\n' - '\n' - 'String Methods\n' - '==============\n' - '\n' - 'Below are listed the string methods which both 8-bit strings ' - 'and\n' - 'Unicode objects support. Some of them are also available on\n' - '"bytearray" objects.\n' - '\n' - "In addition, Python's strings support the sequence type " - 'methods\n' - 'described in the Sequence Types --- str, unicode, list, ' - 'tuple,\n' - 'bytearray, buffer, xrange section. To output formatted strings ' - 'use\n' - 'template strings or the "%" operator described in the String\n' - 'Formatting Operations section. Also, see the "re" module for ' - 'string\n' - 'functions based on regular expressions.\n' - '\n' - 'str.capitalize()\n' - '\n' - ' Return a copy of the string with its first character ' - 'capitalized\n' - ' and the rest lowercased.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.center(width[, fillchar])\n' - '\n' - ' Return centered in a string of length *width*. Padding is ' - 'done\n' - ' using the specified *fillchar* (default is a space).\n' - '\n' - ' Changed in version 2.4: Support for the *fillchar* ' - 'argument.\n' - '\n' - 'str.count(sub[, start[, end]])\n' - '\n' - ' Return the number of non-overlapping occurrences of ' - 'substring *sub*\n' - ' in the range [*start*, *end*]. Optional arguments *start* ' - 'and\n' - ' *end* are interpreted as in slice notation.\n' - '\n' - 'str.decode([encoding[, errors]])\n' - '\n' - ' Decodes the string using the codec registered for ' - '*encoding*.\n' - ' *encoding* defaults to the default string encoding. ' - '*errors* may\n' - ' be given to set a different error handling scheme. The ' - 'default is\n' - ' "\'strict\'", meaning that encoding errors raise ' - '"UnicodeError".\n' - ' Other possible values are "\'ignore\'", "\'replace\'" and ' - 'any other\n' - ' name registered via "codecs.register_error()", see section ' - 'Codec\n' - ' Base Classes.\n' - '\n' - ' New in version 2.2.\n' - '\n' - ' Changed in version 2.3: Support for other error handling ' - 'schemes\n' - ' added.\n' - '\n' - ' Changed in version 2.7: Support for keyword arguments ' - 'added.\n' - '\n' - 'str.encode([encoding[, errors]])\n' - '\n' - ' Return an encoded version of the string. Default encoding ' - 'is the\n' - ' current default string encoding. *errors* may be given to ' - 'set a\n' - ' different error handling scheme. The default for *errors* ' - 'is\n' - ' "\'strict\'", meaning that encoding errors raise a ' - '"UnicodeError".\n' - ' Other possible values are "\'ignore\'", "\'replace\'",\n' - ' "\'xmlcharrefreplace\'", "\'backslashreplace\'" and any ' - 'other name\n' - ' registered via "codecs.register_error()", see section Codec ' - 'Base\n' - ' Classes. For a list of possible encodings, see section ' - 'Standard\n' - ' Encodings.\n' - '\n' - ' New in version 2.0.\n' - '\n' - ' Changed in version 2.3: Support for "\'xmlcharrefreplace\'" ' - 'and\n' - ' "\'backslashreplace\'" and other error handling schemes ' - 'added.\n' - '\n' - ' Changed in version 2.7: Support for keyword arguments ' - 'added.\n' - '\n' - 'str.endswith(suffix[, start[, end]])\n' - '\n' - ' Return "True" if the string ends with the specified ' - '*suffix*,\n' - ' otherwise return "False". *suffix* can also be a tuple of ' - 'suffixes\n' - ' to look for. With optional *start*, test beginning at ' - 'that\n' - ' position. With optional *end*, stop comparing at that ' - 'position.\n' - '\n' - ' Changed in version 2.5: Accept tuples as *suffix*.\n' - '\n' - 'str.expandtabs([tabsize])\n' - '\n' - ' Return a copy of the string where all tab characters are ' - 'replaced\n' - ' by one or more spaces, depending on the current column and ' - 'the\n' - ' given tab size. Tab positions occur every *tabsize* ' - 'characters\n' - ' (default is 8, giving tab positions at columns 0, 8, 16 and ' - 'so on).\n' - ' To expand the string, the current column is set to zero and ' - 'the\n' - ' string is examined character by character. If the ' - 'character is a\n' - ' tab ("\\t"), one or more space characters are inserted in ' - 'the result\n' - ' until the current column is equal to the next tab position. ' - '(The\n' - ' tab character itself is not copied.) If the character is a ' - 'newline\n' - ' ("\\n") or return ("\\r"), it is copied and the current ' - 'column is\n' - ' reset to zero. Any other character is copied unchanged and ' - 'the\n' - ' current column is incremented by one regardless of how the\n' - ' character is represented when printed.\n' - '\n' - " >>> '01\\t012\\t0123\\t01234'.expandtabs()\n" - " '01 012 0123 01234'\n" - " >>> '01\\t012\\t0123\\t01234'.expandtabs(4)\n" - " '01 012 0123 01234'\n" - '\n' - 'str.find(sub[, start[, end]])\n' - '\n' - ' Return the lowest index in the string where substring *sub* ' - 'is\n' - ' found, such that *sub* is contained in the slice ' - '"s[start:end]".\n' - ' Optional arguments *start* and *end* are interpreted as in ' - 'slice\n' - ' notation. Return "-1" if *sub* is not found.\n' - '\n' - ' Note: The "find()" method should be used only if you need ' - 'to know\n' - ' the position of *sub*. To check if *sub* is a substring ' - 'or not,\n' - ' use the "in" operator:\n' - '\n' - " >>> 'Py' in 'Python'\n" - ' True\n' - '\n' - 'str.format(*args, **kwargs)\n' - '\n' - ' Perform a string formatting operation. The string on which ' - 'this\n' - ' method is called can contain literal text or replacement ' - 'fields\n' - ' delimited by braces "{}". Each replacement field contains ' - 'either\n' - ' the numeric index of a positional argument, or the name of ' - 'a\n' - ' keyword argument. Returns a copy of the string where each\n' - ' replacement field is replaced with the string value of the\n' - ' corresponding argument.\n' - '\n' - ' >>> "The sum of 1 + 2 is {0}".format(1+2)\n' - " 'The sum of 1 + 2 is 3'\n" - '\n' - ' See Format String Syntax for a description of the various\n' - ' formatting options that can be specified in format ' - 'strings.\n' - '\n' - ' This method of string formatting is the new standard in ' - 'Python 3,\n' - ' and should be preferred to the "%" formatting described in ' - 'String\n' - ' Formatting Operations in new code.\n' - '\n' - ' New in version 2.6.\n' - '\n' - 'str.index(sub[, start[, end]])\n' - '\n' - ' Like "find()", but raise "ValueError" when the substring is ' - 'not\n' - ' found.\n' - '\n' - 'str.isalnum()\n' - '\n' - ' Return true if all characters in the string are ' - 'alphanumeric and\n' - ' there is at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isalpha()\n' - '\n' - ' Return true if all characters in the string are alphabetic ' - 'and\n' - ' there is at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isdigit()\n' - '\n' - ' Return true if all characters in the string are digits and ' - 'there is\n' - ' at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.islower()\n' - '\n' - ' Return true if all cased characters [4] in the string are ' - 'lowercase\n' - ' and there is at least one cased character, false ' - 'otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isspace()\n' - '\n' - ' Return true if there are only whitespace characters in the ' - 'string\n' - ' and there is at least one character, false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.istitle()\n' - '\n' - ' Return true if the string is a titlecased string and there ' - 'is at\n' - ' least one character, for example uppercase characters may ' - 'only\n' - ' follow uncased characters and lowercase characters only ' - 'cased ones.\n' - ' Return false otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.isupper()\n' - '\n' - ' Return true if all cased characters [4] in the string are ' - 'uppercase\n' - ' and there is at least one cased character, false ' - 'otherwise.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.join(iterable)\n' - '\n' - ' Return a string which is the concatenation of the strings ' - 'in the\n' - ' *iterable* *iterable*. The separator between elements is ' - 'the\n' - ' string providing this method.\n' - '\n' - 'str.ljust(width[, fillchar])\n' - '\n' - ' Return the string left justified in a string of length ' - '*width*.\n' - ' Padding is done using the specified *fillchar* (default is ' - 'a\n' - ' space). The original string is returned if *width* is less ' - 'than or\n' - ' equal to "len(s)".\n' - '\n' - ' Changed in version 2.4: Support for the *fillchar* ' - 'argument.\n' - '\n' - 'str.lower()\n' - '\n' - ' Return a copy of the string with all the cased characters ' - '[4]\n' - ' converted to lowercase.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.lstrip([chars])\n' - '\n' - ' Return a copy of the string with leading characters ' - 'removed. The\n' - ' *chars* argument is a string specifying the set of ' - 'characters to be\n' - ' removed. If omitted or "None", the *chars* argument ' - 'defaults to\n' - ' removing whitespace. The *chars* argument is not a prefix; ' - 'rather,\n' - ' all combinations of its values are stripped:\n' - '\n' - " >>> ' spacious '.lstrip()\n" - " 'spacious '\n" - " >>> 'www.example.com'.lstrip('cmowz.')\n" - " 'example.com'\n" - '\n' - ' Changed in version 2.2.2: Support for the *chars* ' - 'argument.\n' - '\n' - 'str.partition(sep)\n' - '\n' - ' Split the string at the first occurrence of *sep*, and ' - 'return a\n' - ' 3-tuple containing the part before the separator, the ' - 'separator\n' - ' itself, and the part after the separator. If the separator ' - 'is not\n' - ' found, return a 3-tuple containing the string itself, ' - 'followed by\n' - ' two empty strings.\n' - '\n' - ' New in version 2.5.\n' - '\n' - 'str.replace(old, new[, count])\n' - '\n' - ' Return a copy of the string with all occurrences of ' - 'substring *old*\n' - ' replaced by *new*. If the optional argument *count* is ' - 'given, only\n' - ' the first *count* occurrences are replaced.\n' - '\n' - 'str.rfind(sub[, start[, end]])\n' - '\n' - ' Return the highest index in the string where substring ' - '*sub* is\n' - ' found, such that *sub* is contained within "s[start:end]".\n' - ' Optional arguments *start* and *end* are interpreted as in ' - 'slice\n' - ' notation. Return "-1" on failure.\n' - '\n' - 'str.rindex(sub[, start[, end]])\n' - '\n' - ' Like "rfind()" but raises "ValueError" when the substring ' - '*sub* is\n' - ' not found.\n' - '\n' - 'str.rjust(width[, fillchar])\n' - '\n' - ' Return the string right justified in a string of length ' - '*width*.\n' - ' Padding is done using the specified *fillchar* (default is ' - 'a\n' - ' space). The original string is returned if *width* is less ' - 'than or\n' - ' equal to "len(s)".\n' - '\n' - ' Changed in version 2.4: Support for the *fillchar* ' - 'argument.\n' - '\n' - 'str.rpartition(sep)\n' - '\n' - ' Split the string at the last occurrence of *sep*, and ' - 'return a\n' - ' 3-tuple containing the part before the separator, the ' - 'separator\n' - ' itself, and the part after the separator. If the separator ' - 'is not\n' - ' found, return a 3-tuple containing two empty strings, ' - 'followed by\n' - ' the string itself.\n' - '\n' - ' New in version 2.5.\n' - '\n' - 'str.rsplit([sep[, maxsplit]])\n' - '\n' - ' Return a list of the words in the string, using *sep* as ' - 'the\n' - ' delimiter string. If *maxsplit* is given, at most ' - '*maxsplit* splits\n' - ' are done, the *rightmost* ones. If *sep* is not specified ' - 'or\n' - ' "None", any whitespace string is a separator. Except for ' - 'splitting\n' - ' from the right, "rsplit()" behaves like "split()" which is\n' - ' described in detail below.\n' - '\n' - ' New in version 2.4.\n' - '\n' - 'str.rstrip([chars])\n' - '\n' - ' Return a copy of the string with trailing characters ' - 'removed. The\n' - ' *chars* argument is a string specifying the set of ' - 'characters to be\n' - ' removed. If omitted or "None", the *chars* argument ' - 'defaults to\n' - ' removing whitespace. The *chars* argument is not a suffix; ' - 'rather,\n' - ' all combinations of its values are stripped:\n' - '\n' - " >>> ' spacious '.rstrip()\n" - " ' spacious'\n" - " >>> 'mississippi'.rstrip('ipz')\n" - " 'mississ'\n" - '\n' - ' Changed in version 2.2.2: Support for the *chars* ' - 'argument.\n' - '\n' - 'str.split([sep[, maxsplit]])\n' - '\n' - ' Return a list of the words in the string, using *sep* as ' - 'the\n' - ' delimiter string. If *maxsplit* is given, at most ' - '*maxsplit*\n' - ' splits are done (thus, the list will have at most ' - '"maxsplit+1"\n' - ' elements). If *maxsplit* is not specified or "-1", then ' - 'there is\n' - ' no limit on the number of splits (all possible splits are ' - 'made).\n' - '\n' - ' If *sep* is given, consecutive delimiters are not grouped ' - 'together\n' - ' and are deemed to delimit empty strings (for example,\n' - ' "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', \'2\']"). ' - 'The *sep* argument\n' - ' may consist of multiple characters (for example,\n' - ' "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', ' - '\'3\']"). Splitting an\n' - ' empty string with a specified separator returns "[\'\']".\n' - '\n' - ' If *sep* is not specified or is "None", a different ' - 'splitting\n' - ' algorithm is applied: runs of consecutive whitespace are ' - 'regarded\n' - ' as a single separator, and the result will contain no empty ' - 'strings\n' - ' at the start or end if the string has leading or trailing\n' - ' whitespace. Consequently, splitting an empty string or a ' - 'string\n' - ' consisting of just whitespace with a "None" separator ' - 'returns "[]".\n' - '\n' - ' For example, "\' 1 2 3 \'.split()" returns "[\'1\', ' - '\'2\', \'3\']", and\n' - ' "\' 1 2 3 \'.split(None, 1)" returns "[\'1\', \'2 3 ' - '\']".\n' - '\n' - 'str.splitlines([keepends])\n' - '\n' - ' Return a list of the lines in the string, breaking at line\n' - ' boundaries. This method uses the *universal newlines* ' - 'approach to\n' - ' splitting lines. Line breaks are not included in the ' - 'resulting list\n' - ' unless *keepends* is given and true.\n' - '\n' - ' For example, "\'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines()" ' - 'returns "[\'ab\n' - ' c\', \'\', \'de fg\', \'kl\']", while the same call with\n' - ' "splitlines(True)" returns "[\'ab c\\n\', \'\\n\', \'de ' - 'fg\\r\', \'kl\\r\\n\']".\n' - '\n' - ' Unlike "split()" when a delimiter string *sep* is given, ' - 'this\n' - ' method returns an empty list for the empty string, and a ' - 'terminal\n' - ' line break does not result in an extra line.\n' - '\n' - 'str.startswith(prefix[, start[, end]])\n' - '\n' - ' Return "True" if string starts with the *prefix*, otherwise ' - 'return\n' - ' "False". *prefix* can also be a tuple of prefixes to look ' - 'for.\n' - ' With optional *start*, test string beginning at that ' - 'position.\n' - ' With optional *end*, stop comparing string at that ' - 'position.\n' - '\n' - ' Changed in version 2.5: Accept tuples as *prefix*.\n' - '\n' - 'str.strip([chars])\n' - '\n' - ' Return a copy of the string with the leading and trailing\n' - ' characters removed. The *chars* argument is a string ' - 'specifying the\n' - ' set of characters to be removed. If omitted or "None", the ' - '*chars*\n' - ' argument defaults to removing whitespace. The *chars* ' - 'argument is\n' - ' not a prefix or suffix; rather, all combinations of its ' - 'values are\n' - ' stripped:\n' - '\n' - " >>> ' spacious '.strip()\n" - " 'spacious'\n" - " >>> 'www.example.com'.strip('cmowz.')\n" - " 'example'\n" - '\n' - ' Changed in version 2.2.2: Support for the *chars* ' - 'argument.\n' - '\n' - 'str.swapcase()\n' - '\n' - ' Return a copy of the string with uppercase characters ' - 'converted to\n' - ' lowercase and vice versa.\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.title()\n' - '\n' - ' Return a titlecased version of the string where words start ' - 'with an\n' - ' uppercase character and the remaining characters are ' - 'lowercase.\n' - '\n' - ' The algorithm uses a simple language-independent definition ' - 'of a\n' - ' word as groups of consecutive letters. The definition ' - 'works in\n' - ' many contexts but it means that apostrophes in contractions ' - 'and\n' - ' possessives form word boundaries, which may not be the ' - 'desired\n' - ' result:\n' - '\n' - ' >>> "they\'re bill\'s friends from the UK".title()\n' - ' "They\'Re Bill\'S Friends From The Uk"\n' - '\n' - ' A workaround for apostrophes can be constructed using ' - 'regular\n' - ' expressions:\n' - '\n' - ' >>> import re\n' - ' >>> def titlecase(s):\n' - ' ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n' - ' ... lambda mo: mo.group(0)[0].upper() ' - '+\n' - ' ... ' - 'mo.group(0)[1:].lower(),\n' - ' ... s)\n' - ' ...\n' - ' >>> titlecase("they\'re bill\'s friends.")\n' - ' "They\'re Bill\'s Friends."\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.translate(table[, deletechars])\n' - '\n' - ' Return a copy of the string where all characters occurring ' - 'in the\n' - ' optional argument *deletechars* are removed, and the ' - 'remaining\n' - ' characters have been mapped through the given translation ' - 'table,\n' - ' which must be a string of length 256.\n' - '\n' - ' You can use the "maketrans()" helper function in the ' - '"string"\n' - ' module to create a translation table. For string objects, ' - 'set the\n' - ' *table* argument to "None" for translations that only ' - 'delete\n' - ' characters:\n' - '\n' - " >>> 'read this short text'.translate(None, 'aeiou')\n" - " 'rd ths shrt txt'\n" - '\n' - ' New in version 2.6: Support for a "None" *table* argument.\n' - '\n' - ' For Unicode objects, the "translate()" method does not ' - 'accept the\n' - ' optional *deletechars* argument. Instead, it returns a ' - 'copy of the\n' - ' *s* where all characters have been mapped through the ' - 'given\n' - ' translation table which must be a mapping of Unicode ' - 'ordinals to\n' - ' Unicode ordinals, Unicode strings or "None". Unmapped ' - 'characters\n' - ' are left untouched. Characters mapped to "None" are ' - 'deleted. Note,\n' - ' a more flexible approach is to create a custom character ' - 'mapping\n' - ' codec using the "codecs" module (see "encodings.cp1251" for ' - 'an\n' - ' example).\n' - '\n' - 'str.upper()\n' - '\n' - ' Return a copy of the string with all the cased characters ' - '[4]\n' - ' converted to uppercase. Note that "str.upper().isupper()" ' - 'might be\n' - ' "False" if "s" contains uncased characters or if the ' - 'Unicode\n' - ' category of the resulting character(s) is not "Lu" ' - '(Letter,\n' - ' uppercase), but e.g. "Lt" (Letter, titlecase).\n' - '\n' - ' For 8-bit strings, this method is locale-dependent.\n' - '\n' - 'str.zfill(width)\n' - '\n' - ' Return the numeric string left filled with zeros in a ' - 'string of\n' - ' length *width*. A sign prefix is handled correctly. The ' - 'original\n' - ' string is returned if *width* is less than or equal to ' - '"len(s)".\n' - '\n' - ' New in version 2.2.2.\n' - '\n' - 'The following methods are present only on unicode objects:\n' - '\n' - 'unicode.isnumeric()\n' - '\n' - ' Return "True" if there are only numeric characters in S, ' - '"False"\n' - ' otherwise. Numeric characters include digit characters, and ' - 'all\n' - ' characters that have the Unicode numeric value property, ' - 'e.g.\n' - ' U+2155, VULGAR FRACTION ONE FIFTH.\n' - '\n' - 'unicode.isdecimal()\n' - '\n' - ' Return "True" if there are only decimal characters in S, ' - '"False"\n' - ' otherwise. Decimal characters include digit characters, and ' - 'all\n' - ' characters that can be used to form decimal-radix numbers, ' - 'e.g.\n' - ' U+0660, ARABIC-INDIC DIGIT ZERO.\n' - '\n' - '\n' - 'String Formatting Operations\n' - '============================\n' - '\n' - 'String and Unicode objects have one unique built-in operation: ' - 'the "%"\n' - 'operator (modulo). This is also known as the string ' - '*formatting* or\n' - '*interpolation* operator. Given "format % values" (where ' - '*format* is\n' - 'a string or Unicode object), "%" conversion specifications in ' - '*format*\n' - 'are replaced with zero or more elements of *values*. The ' - 'effect is\n' - 'similar to the using "sprintf()" in the C language. If ' - '*format* is a\n' - 'Unicode object, or if any of the objects being converted using ' - 'the\n' - '"%s" conversion are Unicode objects, the result will also be a ' - 'Unicode\n' - 'object.\n' - '\n' - 'If *format* requires a single argument, *values* may be a ' - 'single non-\n' - 'tuple object. [5] Otherwise, *values* must be a tuple with ' - 'exactly\n' - 'the number of items specified by the format string, or a ' - 'single\n' - 'mapping object (for example, a dictionary).\n' - '\n' - 'A conversion specifier contains two or more characters and has ' - 'the\n' - 'following components, which must occur in this order:\n' - '\n' - '1. The "\'%\'" character, which marks the start of the ' - 'specifier.\n' - '\n' - '2. Mapping key (optional), consisting of a parenthesised ' - 'sequence\n' - ' of characters (for example, "(somename)").\n' - '\n' - '3. Conversion flags (optional), which affect the result of ' - 'some\n' - ' conversion types.\n' - '\n' - '4. Minimum field width (optional). If specified as an ' - '"\'*\'"\n' - ' (asterisk), the actual width is read from the next element ' - 'of the\n' - ' tuple in *values*, and the object to convert comes after ' - 'the\n' - ' minimum field width and optional precision.\n' - '\n' - '5. Precision (optional), given as a "\'.\'" (dot) followed by ' - 'the\n' - ' precision. If specified as "\'*\'" (an asterisk), the ' - 'actual width\n' - ' is read from the next element of the tuple in *values*, and ' - 'the\n' - ' value to convert comes after the precision.\n' - '\n' - '6. Length modifier (optional).\n' - '\n' - '7. Conversion type.\n' - '\n' - 'When the right argument is a dictionary (or other mapping ' - 'type), then\n' - 'the formats in the string *must* include a parenthesised ' - 'mapping key\n' - 'into that dictionary inserted immediately after the "\'%\'" ' - 'character.\n' - 'The mapping key selects the value to be formatted from the ' - 'mapping.\n' - 'For example:\n' - '\n' - ">>> print '%(language)s has %(number)03d quote types.' % \\\n" - '... {"language": "Python", "number": 2}\n' - 'Python has 002 quote types.\n' - '\n' - 'In this case no "*" specifiers may occur in a format (since ' - 'they\n' - 'require a sequential parameter list).\n' - '\n' - 'The conversion flag characters are:\n' - '\n' - '+-----------+-----------------------------------------------------------------------+\n' - '| Flag | ' - 'Meaning ' - '|\n' - '+===========+=======================================================================+\n' - '| "\'#\'" | The value conversion will use the "alternate ' - 'form" (where defined |\n' - '| | ' - 'below). ' - '|\n' - '+-----------+-----------------------------------------------------------------------+\n' - '| "\'0\'" | The conversion will be zero padded for numeric ' - 'values. |\n' - '+-----------+-----------------------------------------------------------------------+\n' - '| "\'-\'" | The converted value is left adjusted ' - '(overrides the "\'0\'" conversion |\n' - '| | if both are ' - 'given). |\n' - '+-----------+-----------------------------------------------------------------------+\n' - '| "\' \'" | (a space) A blank should be left before a ' - 'positive number (or empty |\n' - '| | string) produced by a signed ' - 'conversion. |\n' - '+-----------+-----------------------------------------------------------------------+\n' - '| "\'+\'" | A sign character ("\'+\'" or "\'-\'") will ' - 'precede the conversion |\n' - '| | (overrides a "space" ' - 'flag). |\n' - '+-----------+-----------------------------------------------------------------------+\n' - '\n' - 'A length modifier ("h", "l", or "L") may be present, but is ' - 'ignored as\n' - 'it is not necessary for Python -- so e.g. "%ld" is identical ' - 'to "%d".\n' - '\n' - 'The conversion types are:\n' - '\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| Conversion | ' - 'Meaning | ' - 'Notes |\n' - '+==============+=======================================================+=========+\n' - '| "\'d\'" | Signed integer ' - 'decimal. | |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'i\'" | Signed integer ' - 'decimal. | |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'o\'" | Signed octal ' - 'value. | (1) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'u\'" | Obsolete type -- it is identical to ' - '"\'d\'". | (7) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'x\'" | Signed hexadecimal ' - '(lowercase). | (2) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'X\'" | Signed hexadecimal ' - '(uppercase). | (2) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'e\'" | Floating point exponential format ' - '(lowercase). | (3) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'E\'" | Floating point exponential format ' - '(uppercase). | (3) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'f\'" | Floating point decimal ' - 'format. | (3) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'F\'" | Floating point decimal ' - 'format. | (3) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'g\'" | Floating point format. Uses lowercase ' - 'exponential | (4) |\n' - '| | format if exponent is less than -4 or not ' - 'less than | |\n' - '| | precision, decimal format ' - 'otherwise. | |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'G\'" | Floating point format. Uses uppercase ' - 'exponential | (4) |\n' - '| | format if exponent is less than -4 or not ' - 'less than | |\n' - '| | precision, decimal format ' - 'otherwise. | |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'c\'" | Single character (accepts integer or single ' - 'character | |\n' - '| | ' - 'string). ' - '| |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'r\'" | String (converts any Python object using ' - 'repr()). | (5) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'s\'" | String (converts any Python object using ' - '"str()"). | (6) |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '| "\'%\'" | No argument is converted, results in a ' - '"\'%\'" | |\n' - '| | character in the ' - 'result. | |\n' - '+--------------+-------------------------------------------------------+---------+\n' - '\n' - 'Notes:\n' - '\n' - '1. The alternate form causes a leading zero ("\'0\'") to be ' - 'inserted\n' - ' between left-hand padding and the formatting of the number ' - 'if the\n' - ' leading character of the result is not already a zero.\n' - '\n' - '2. The alternate form causes a leading "\'0x\'" or "\'0X\'" ' - '(depending\n' - ' on whether the "\'x\'" or "\'X\'" format was used) to be ' - 'inserted\n' - ' between left-hand padding and the formatting of the number ' - 'if the\n' - ' leading character of the result is not already a zero.\n' - '\n' - '3. The alternate form causes the result to always contain a ' - 'decimal\n' - ' point, even if no digits follow it.\n' - '\n' - ' The precision determines the number of digits after the ' - 'decimal\n' - ' point and defaults to 6.\n' - '\n' - '4. The alternate form causes the result to always contain a ' - 'decimal\n' - ' point, and trailing zeroes are not removed as they would ' - 'otherwise\n' - ' be.\n' - '\n' - ' The precision determines the number of significant digits ' - 'before\n' - ' and after the decimal point and defaults to 6.\n' - '\n' - '5. The "%r" conversion was added in Python 2.0.\n' - '\n' - ' The precision determines the maximal number of characters ' - 'used.\n' - '\n' - '6. If the object or format provided is a "unicode" string, ' - 'the\n' - ' resulting string will also be "unicode".\n' - '\n' - ' The precision determines the maximal number of characters ' - 'used.\n' - '\n' - '7. See **PEP 237**.\n' - '\n' - 'Since Python strings have an explicit length, "%s" conversions ' - 'do not\n' - 'assume that "\'\\0\'" is the end of the string.\n' - '\n' - 'Changed in version 2.7: "%f" conversions for numbers whose ' - 'absolute\n' - 'value is over 1e50 are no longer replaced by "%g" ' - 'conversions.\n' - '\n' - 'Additional string operations are defined in standard modules ' - '"string"\n' - 'and "re".\n' - '\n' - '\n' - 'XRange Type\n' - '===========\n' - '\n' - 'The "xrange" type is an immutable sequence which is commonly ' - 'used for\n' - 'looping. The advantage of the "xrange" type is that an ' - '"xrange"\n' - 'object will always take the same amount of memory, no matter ' - 'the size\n' - 'of the range it represents. There are no consistent ' - 'performance\n' - 'advantages.\n' - '\n' - 'XRange objects have very little behavior: they only support ' - 'indexing,\n' - 'iteration, and the "len()" function.\n' - '\n' - '\n' - 'Mutable Sequence Types\n' - '======================\n' - '\n' - 'List and "bytearray" objects support additional operations ' - 'that allow\n' - 'in-place modification of the object. Other mutable sequence ' - 'types\n' - '(when added to the language) should also support these ' - 'operations.\n' - 'Strings and tuples are immutable sequence types: such objects ' - 'cannot\n' - 'be modified once created. The following operations are defined ' - 'on\n' - 'mutable sequence types (where *x* is an arbitrary object):\n' - '\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| Operation | ' - 'Result | Notes |\n' - '+================================+==================================+=======================+\n' - '| "s[i] = x" | item *i* of *s* is replaced ' - 'by | |\n' - '| | ' - '*x* | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s[i:j] = t" | slice of *s* from *i* to ' - '*j* is | |\n' - '| | replaced by the contents of ' - 'the | |\n' - '| | iterable ' - '*t* | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "del s[i:j]" | same as "s[i:j] = ' - '[]" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s[i:j:k] = t" | the elements of "s[i:j:k]" ' - 'are | (1) |\n' - '| | replaced by those of ' - '*t* | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "del s[i:j:k]" | removes the elements ' - 'of | |\n' - '| | "s[i:j:k]" from the ' - 'list | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.append(x)" | same as "s[len(s):len(s)] = ' - '[x]" | (2) |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.extend(x)" or "s += t" | for the most part the same ' - 'as | (3) |\n' - '| | "s[len(s):len(s)] = ' - 'x" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s *= n" | updates *s* with its ' - 'contents | (11) |\n' - '| | repeated *n* ' - 'times | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.count(x)" | return number of *i*\'s for ' - 'which | |\n' - '| | "s[i] == ' - 'x" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.index(x[, i[, j]])" | return smallest *k* such ' - 'that | (4) |\n' - '| | "s[k] == x" and "i <= k < ' - 'j" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.insert(i, x)" | same as "s[i:i] = ' - '[x]" | (5) |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.pop([i])" | same as "x = s[i]; del ' - 's[i]; | (6) |\n' - '| | return ' - 'x" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.remove(x)" | same as "del ' - 's[s.index(x)]" | (4) |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.reverse()" | reverses the items of *s* ' - 'in | (7) |\n' - '| | ' - 'place | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.sort([cmp[, key[, | sort the items of *s* in ' - 'place | (7)(8)(9)(10) |\n' - '| reverse]]])" ' - '| | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '\n' - 'Notes:\n' - '\n' - '1. *t* must have the same length as the slice it is ' - 'replacing.\n' - '\n' - '2. The C implementation of Python has historically accepted\n' - ' multiple parameters and implicitly joined them into a ' - 'tuple; this\n' - ' no longer works in Python 2.0. Use of this misfeature has ' - 'been\n' - ' deprecated since Python 1.4.\n' - '\n' - '3. *x* can be any iterable object.\n' - '\n' - '4. Raises "ValueError" when *x* is not found in *s*. When a\n' - ' negative index is passed as the second or third parameter ' - 'to the\n' - ' "index()" method, the list length is added, as for slice ' - 'indices.\n' - ' If it is still negative, it is truncated to zero, as for ' - 'slice\n' - ' indices.\n' - '\n' - ' Changed in version 2.3: Previously, "index()" didn\'t have ' - 'arguments\n' - ' for specifying start and stop positions.\n' - '\n' - '5. When a negative index is passed as the first parameter to ' - 'the\n' - ' "insert()" method, the list length is added, as for slice ' - 'indices.\n' - ' If it is still negative, it is truncated to zero, as for ' - 'slice\n' - ' indices.\n' - '\n' - ' Changed in version 2.3: Previously, all negative indices ' - 'were\n' - ' truncated to zero.\n' - '\n' - '6. The "pop()" method\'s optional argument *i* defaults to ' - '"-1", so\n' - ' that by default the last item is removed and returned.\n' - '\n' - '7. The "sort()" and "reverse()" methods modify the list in ' - 'place\n' - ' for economy of space when sorting or reversing a large ' - 'list. To\n' - " remind you that they operate by side effect, they don't " - 'return the\n' - ' sorted or reversed list.\n' - '\n' - '8. The "sort()" method takes optional arguments for ' - 'controlling the\n' - ' comparisons.\n' - '\n' - ' *cmp* specifies a custom comparison function of two ' - 'arguments (list\n' - ' items) which should return a negative, zero or positive ' - 'number\n' - ' depending on whether the first argument is considered ' - 'smaller than,\n' - ' equal to, or larger than the second argument: "cmp=lambda ' - 'x,y:\n' - ' cmp(x.lower(), y.lower())". The default value is "None".\n' - '\n' - ' *key* specifies a function of one argument that is used to ' - 'extract\n' - ' a comparison key from each list element: "key=str.lower". ' - 'The\n' - ' default value is "None".\n' - '\n' - ' *reverse* is a boolean value. If set to "True", then the ' - 'list\n' - ' elements are sorted as if each comparison were reversed.\n' - '\n' - ' In general, the *key* and *reverse* conversion processes ' - 'are much\n' - ' faster than specifying an equivalent *cmp* function. This ' - 'is\n' - ' because *cmp* is called multiple times for each list ' - 'element while\n' - ' *key* and *reverse* touch each element only once. Use\n' - ' "functools.cmp_to_key()" to convert an old-style *cmp* ' - 'function to\n' - ' a *key* function.\n' - '\n' - ' Changed in version 2.3: Support for "None" as an equivalent ' - 'to\n' - ' omitting *cmp* was added.\n' - '\n' - ' Changed in version 2.4: Support for *key* and *reverse* was ' - 'added.\n' - '\n' - '9. Starting with Python 2.3, the "sort()" method is guaranteed ' - 'to\n' - ' be stable. A sort is stable if it guarantees not to change ' - 'the\n' - ' relative order of elements that compare equal --- this is ' - 'helpful\n' - ' for sorting in multiple passes (for example, sort by ' - 'department,\n' - ' then by salary grade).\n' - '\n' - '10. **CPython implementation detail:** While a list is being\n' - ' sorted, the effect of attempting to mutate, or even ' - 'inspect, the\n' - ' list is undefined. The C implementation of Python 2.3 and ' - 'newer\n' - ' makes the list appear empty for the duration, and raises\n' - ' "ValueError" if it can detect that the list has been ' - 'mutated\n' - ' during a sort.\n' - '\n' - '11. The value *n* is an integer, or an object implementing\n' - ' "__index__()". Zero and negative values of *n* clear the\n' - ' sequence. Items in the sequence are not copied; they are\n' - ' referenced multiple times, as explained for "s * n" under ' - 'Sequence\n' - ' Types --- str, unicode, list, tuple, bytearray, buffer, ' - 'xrange.\n', - 'typesseq-mutable': '\n' - 'Mutable Sequence Types\n' - '**********************\n' - '\n' - 'List and "bytearray" objects support additional ' - 'operations that allow\n' - 'in-place modification of the object. Other mutable ' - 'sequence types\n' - '(when added to the language) should also support these ' - 'operations.\n' - 'Strings and tuples are immutable sequence types: such ' - 'objects cannot\n' - 'be modified once created. The following operations are ' - 'defined on\n' - 'mutable sequence types (where *x* is an arbitrary ' - 'object):\n' - '\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| Operation | ' - 'Result | ' - 'Notes |\n' - '+================================+==================================+=======================+\n' - '| "s[i] = x" | item *i* of *s* is ' - 'replaced by | |\n' - '| | ' - '*x* ' - '| |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s[i:j] = t" | slice of *s* from ' - '*i* to *j* is | |\n' - '| | replaced by the ' - 'contents of the | |\n' - '| | iterable ' - '*t* | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "del s[i:j]" | same as "s[i:j] = ' - '[]" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s[i:j:k] = t" | the elements of ' - '"s[i:j:k]" are | (1) |\n' - '| | replaced by those ' - 'of *t* | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "del s[i:j:k]" | removes the ' - 'elements of | |\n' - '| | "s[i:j:k]" from the ' - 'list | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.append(x)" | same as ' - '"s[len(s):len(s)] = [x]" | (2) |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.extend(x)" or "s += t" | for the most part ' - 'the same as | (3) |\n' - '| | "s[len(s):len(s)] = ' - 'x" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s *= n" | updates *s* with ' - 'its contents | (11) |\n' - '| | repeated *n* ' - 'times | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.count(x)" | return number of ' - "*i*'s for which | |\n" - '| | "s[i] == ' - 'x" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.index(x[, i[, j]])" | return smallest *k* ' - 'such that | (4) |\n' - '| | "s[k] == x" and "i ' - '<= k < j" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.insert(i, x)" | same as "s[i:i] = ' - '[x]" | (5) |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.pop([i])" | same as "x = s[i]; ' - 'del s[i]; | (6) |\n' - '| | return ' - 'x" | |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.remove(x)" | same as "del ' - 's[s.index(x)]" | (4) |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.reverse()" | reverses the items ' - 'of *s* in | (7) |\n' - '| | ' - 'place ' - '| |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '| "s.sort([cmp[, key[, | sort the items of ' - '*s* in place | (7)(8)(9)(10) |\n' - '| reverse]]])" ' - '| ' - '| |\n' - '+--------------------------------+----------------------------------+-----------------------+\n' - '\n' - 'Notes:\n' - '\n' - '1. *t* must have the same length as the slice it is ' - 'replacing.\n' - '\n' - '2. The C implementation of Python has historically ' - 'accepted\n' - ' multiple parameters and implicitly joined them into ' - 'a tuple; this\n' - ' no longer works in Python 2.0. Use of this ' - 'misfeature has been\n' - ' deprecated since Python 1.4.\n' - '\n' - '3. *x* can be any iterable object.\n' - '\n' - '4. Raises "ValueError" when *x* is not found in *s*. ' - 'When a\n' - ' negative index is passed as the second or third ' - 'parameter to the\n' - ' "index()" method, the list length is added, as for ' - 'slice indices.\n' - ' If it is still negative, it is truncated to zero, ' - 'as for slice\n' - ' indices.\n' - '\n' - ' Changed in version 2.3: Previously, "index()" ' - "didn't have arguments\n" - ' for specifying start and stop positions.\n' - '\n' - '5. When a negative index is passed as the first ' - 'parameter to the\n' - ' "insert()" method, the list length is added, as for ' - 'slice indices.\n' - ' If it is still negative, it is truncated to zero, ' - 'as for slice\n' - ' indices.\n' - '\n' - ' Changed in version 2.3: Previously, all negative ' - 'indices were\n' - ' truncated to zero.\n' - '\n' - '6. The "pop()" method\'s optional argument *i* ' - 'defaults to "-1", so\n' - ' that by default the last item is removed and ' - 'returned.\n' - '\n' - '7. The "sort()" and "reverse()" methods modify the ' - 'list in place\n' - ' for economy of space when sorting or reversing a ' - 'large list. To\n' - ' remind you that they operate by side effect, they ' - "don't return the\n" - ' sorted or reversed list.\n' - '\n' - '8. The "sort()" method takes optional arguments for ' - 'controlling the\n' - ' comparisons.\n' - '\n' - ' *cmp* specifies a custom comparison function of two ' - 'arguments (list\n' - ' items) which should return a negative, zero or ' - 'positive number\n' - ' depending on whether the first argument is ' - 'considered smaller than,\n' - ' equal to, or larger than the second argument: ' - '"cmp=lambda x,y:\n' - ' cmp(x.lower(), y.lower())". The default value is ' - '"None".\n' - '\n' - ' *key* specifies a function of one argument that is ' - 'used to extract\n' - ' a comparison key from each list element: ' - '"key=str.lower". The\n' - ' default value is "None".\n' - '\n' - ' *reverse* is a boolean value. If set to "True", ' - 'then the list\n' - ' elements are sorted as if each comparison were ' - 'reversed.\n' - '\n' - ' In general, the *key* and *reverse* conversion ' - 'processes are much\n' - ' faster than specifying an equivalent *cmp* ' - 'function. This is\n' - ' because *cmp* is called multiple times for each ' - 'list element while\n' - ' *key* and *reverse* touch each element only once. ' - 'Use\n' - ' "functools.cmp_to_key()" to convert an old-style ' - '*cmp* function to\n' - ' a *key* function.\n' - '\n' - ' Changed in version 2.3: Support for "None" as an ' - 'equivalent to\n' - ' omitting *cmp* was added.\n' - '\n' - ' Changed in version 2.4: Support for *key* and ' - '*reverse* was added.\n' - '\n' - '9. Starting with Python 2.3, the "sort()" method is ' - 'guaranteed to\n' - ' be stable. A sort is stable if it guarantees not ' - 'to change the\n' - ' relative order of elements that compare equal --- ' - 'this is helpful\n' - ' for sorting in multiple passes (for example, sort ' - 'by department,\n' - ' then by salary grade).\n' - '\n' - '10. **CPython implementation detail:** While a list is ' - 'being\n' - ' sorted, the effect of attempting to mutate, or ' - 'even inspect, the\n' - ' list is undefined. The C implementation of Python ' - '2.3 and newer\n' - ' makes the list appear empty for the duration, and ' - 'raises\n' - ' "ValueError" if it can detect that the list has ' - 'been mutated\n' - ' during a sort.\n' - '\n' - '11. The value *n* is an integer, or an object ' - 'implementing\n' - ' "__index__()". Zero and negative values of *n* ' - 'clear the\n' - ' sequence. Items in the sequence are not copied; ' - 'they are\n' - ' referenced multiple times, as explained for "s * ' - 'n" under Sequence\n' - ' Types --- str, unicode, list, tuple, bytearray, ' - 'buffer, xrange.\n', - 'unary': '\n' - 'Unary arithmetic and bitwise operations\n' - '***************************************\n' - '\n' - 'All unary arithmetic and bitwise operations have the same ' - 'priority:\n' - '\n' - ' u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr\n' - '\n' - 'The unary "-" (minus) operator yields the negation of its ' - 'numeric\n' - 'argument.\n' - '\n' - 'The unary "+" (plus) operator yields its numeric argument ' - 'unchanged.\n' - '\n' - 'The unary "~" (invert) operator yields the bitwise inversion of ' - 'its\n' - 'plain or long integer argument. The bitwise inversion of "x" is\n' - 'defined as "-(x+1)". It only applies to integral numbers.\n' - '\n' - 'In all three cases, if the argument does not have the proper ' - 'type, a\n' - '"TypeError" exception is raised.\n', - 'while': '\n' - 'The "while" statement\n' - '*********************\n' - '\n' - 'The "while" statement is used for repeated execution as long as ' - 'an\n' - 'expression is true:\n' - '\n' - ' while_stmt ::= "while" expression ":" suite\n' - ' ["else" ":" suite]\n' - '\n' - 'This repeatedly tests the expression and, if it is true, executes ' - 'the\n' - 'first suite; if the expression is false (which may be the first ' - 'time\n' - 'it is tested) the suite of the "else" clause, if present, is ' - 'executed\n' - 'and the loop terminates.\n' - '\n' - 'A "break" statement executed in the first suite terminates the ' - 'loop\n' - 'without executing the "else" clause\'s suite. A "continue" ' - 'statement\n' - 'executed in the first suite skips the rest of the suite and goes ' - 'back\n' - 'to testing the expression.\n', - 'with': '\n' - 'The "with" statement\n' - '********************\n' - '\n' - 'New in version 2.5.\n' - '\n' - 'The "with" statement is used to wrap the execution of a block ' - 'with\n' - 'methods defined by a context manager (see section With Statement\n' - 'Context Managers). This allows common ' - '"try"..."except"..."finally"\n' - 'usage patterns to be encapsulated for convenient reuse.\n' - '\n' - ' with_stmt ::= "with" with_item ("," with_item)* ":" suite\n' - ' with_item ::= expression ["as" target]\n' - '\n' - 'The execution of the "with" statement with one "item" proceeds as\n' - 'follows:\n' - '\n' - '1. The context expression (the expression given in the ' - '"with_item")\n' - ' is evaluated to obtain a context manager.\n' - '\n' - '2. The context manager\'s "__exit__()" is loaded for later use.\n' - '\n' - '3. The context manager\'s "__enter__()" method is invoked.\n' - '\n' - '4. If a target was included in the "with" statement, the return\n' - ' value from "__enter__()" is assigned to it.\n' - '\n' - ' Note: The "with" statement guarantees that if the ' - '"__enter__()"\n' - ' method returns without an error, then "__exit__()" will ' - 'always be\n' - ' called. Thus, if an error occurs during the assignment to ' - 'the\n' - ' target list, it will be treated the same as an error ' - 'occurring\n' - ' within the suite would be. See step 6 below.\n' - '\n' - '5. The suite is executed.\n' - '\n' - '6. The context manager\'s "__exit__()" method is invoked. If an\n' - ' exception caused the suite to be exited, its type, value, and\n' - ' traceback are passed as arguments to "__exit__()". Otherwise, ' - 'three\n' - ' "None" arguments are supplied.\n' - '\n' - ' If the suite was exited due to an exception, and the return ' - 'value\n' - ' from the "__exit__()" method was false, the exception is ' - 'reraised.\n' - ' If the return value was true, the exception is suppressed, and\n' - ' execution continues with the statement following the "with"\n' - ' statement.\n' - '\n' - ' If the suite was exited for any reason other than an exception, ' - 'the\n' - ' return value from "__exit__()" is ignored, and execution ' - 'proceeds\n' - ' at the normal location for the kind of exit that was taken.\n' - '\n' - 'With more than one item, the context managers are processed as if\n' - 'multiple "with" statements were nested:\n' - '\n' - ' with A() as a, B() as b:\n' - ' suite\n' - '\n' - 'is equivalent to\n' - '\n' - ' with A() as a:\n' - ' with B() as b:\n' - ' suite\n' - '\n' - 'Note: In Python 2.5, the "with" statement is only allowed when ' - 'the\n' - ' "with_statement" feature has been enabled. It is always enabled ' - 'in\n' - ' Python 2.6.\n' - '\n' - 'Changed in version 2.7: Support for multiple context expressions.\n' - '\n' - 'See also: **PEP 0343** - The "with" statement\n' - '\n' - ' The specification, background, and examples for the Python ' - '"with"\n' - ' statement.\n', - 'yield': '\n' - 'The "yield" statement\n' - '*********************\n' - '\n' - ' yield_stmt ::= yield_expression\n' - '\n' - 'The "yield" statement is only used when defining a generator ' - 'function,\n' - 'and is only used in the body of the generator function. Using a\n' - '"yield" statement in a function definition is sufficient to cause ' - 'that\n' - 'definition to create a generator function instead of a normal\n' - 'function.\n' - '\n' - 'When a generator function is called, it returns an iterator known ' - 'as a\n' - 'generator iterator, or more commonly, a generator. The body of ' - 'the\n' - "generator function is executed by calling the generator's " - '"next()"\n' - 'method repeatedly until it raises an exception.\n' - '\n' - 'When a "yield" statement is executed, the state of the generator ' - 'is\n' - 'frozen and the value of "expression_list" is returned to ' - '"next()"\'s\n' - 'caller. By "frozen" we mean that all local state is retained,\n' - 'including the current bindings of local variables, the ' - 'instruction\n' - 'pointer, and the internal evaluation stack: enough information ' - 'is\n' - 'saved so that the next time "next()" is invoked, the function ' - 'can\n' - 'proceed exactly as if the "yield" statement were just another ' - 'external\n' - 'call.\n' - '\n' - 'As of Python version 2.5, the "yield" statement is now allowed in ' - 'the\n' - '"try" clause of a "try" ... "finally" construct. If the ' - 'generator is\n' - 'not resumed before it is finalized (by reaching a zero reference ' - 'count\n' - "or by being garbage collected), the generator-iterator's " - '"close()"\n' - 'method will be called, allowing any pending "finally" clauses to\n' - 'execute.\n' - '\n' - 'For full details of "yield" semantics, refer to the Yield ' - 'expressions\n' - 'section.\n' - '\n' - 'Note: In Python 2.2, the "yield" statement was only allowed when ' - 'the\n' - ' "generators" feature has been enabled. This "__future__" ' - 'import\n' - ' statement was used to enable the feature:\n' - '\n' - ' from __future__ import generators\n' - '\n' - 'See also: **PEP 0255** - Simple Generators\n' - '\n' - ' The proposal for adding generators and the "yield" statement ' - 'to\n' - ' Python.\n' - '\n' - ' **PEP 0342** - Coroutines via Enhanced Generators\n' - ' The proposal that, among other generator enhancements, ' - 'proposed\n' - ' allowing "yield" to appear inside a "try" ... "finally" ' - 'block.\n'} +# Autogenerated by Sphinx on Wed Jan 6 03:48:54 2016 +topics = {'assert': u'\nThe "assert" statement\n**********************\n\nAssert statements are a convenient way to insert debugging assertions\ninto a program:\n\n assert_stmt ::= "assert" expression ["," expression]\n\nThe simple form, "assert expression", is equivalent to\n\n if __debug__:\n if not expression: raise AssertionError\n\nThe extended form, "assert expression1, expression2", is equivalent to\n\n if __debug__:\n if not expression1: raise AssertionError(expression2)\n\nThese equivalences assume that "__debug__" and "AssertionError" refer\nto the built-in variables with those names. In the current\nimplementation, the built-in variable "__debug__" is "True" under\nnormal circumstances, "False" when optimization is requested (command\nline option -O). The current code generator emits no code for an\nassert statement when optimization is requested at compile time. Note\nthat it is unnecessary to include the source code for the expression\nthat failed in the error message; it will be displayed as part of the\nstack trace.\n\nAssignments to "__debug__" are illegal. The value for the built-in\nvariable is determined when the interpreter starts.\n', + 'assignment': u'\nAssignment statements\n*********************\n\nAssignment statements are used to (re)bind names to values and to\nmodify attributes or items of mutable objects:\n\n assignment_stmt ::= (target_list "=")+ (expression_list | yield_expression)\n target_list ::= target ("," target)* [","]\n target ::= identifier\n | "(" target_list ")"\n | "[" target_list "]"\n | attributeref\n | subscription\n | slicing\n\n(See section Primaries for the syntax definitions for the last three\nsymbols.)\n\nAn assignment statement evaluates the expression list (remember that\nthis can be a single expression or a comma-separated list, the latter\nyielding a tuple) and assigns the single resulting object to each of\nthe target lists, from left to right.\n\nAssignment is defined recursively depending on the form of the target\n(list). When a target is part of a mutable object (an attribute\nreference, subscription or slicing), the mutable object must\nultimately perform the assignment and decide about its validity, and\nmay raise an exception if the assignment is unacceptable. The rules\nobserved by various types and the exceptions raised are given with the\ndefinition of the object types (see section The standard type\nhierarchy).\n\nAssignment of an object to a target list is recursively defined as\nfollows.\n\n* If the target list is a single target: The object is assigned to\n that target.\n\n* If the target list is a comma-separated list of targets: The\n object must be an iterable with the same number of items as there\n are targets in the target list, and the items are assigned, from\n left to right, to the corresponding targets.\n\nAssignment of an object to a single target is recursively defined as\nfollows.\n\n* If the target is an identifier (name):\n\n * If the name does not occur in a "global" statement in the\n current code block: the name is bound to the object in the current\n local namespace.\n\n * Otherwise: the name is bound to the object in the current global\n namespace.\n\n The name is rebound if it was already bound. This may cause the\n reference count for the object previously bound to the name to reach\n zero, causing the object to be deallocated and its destructor (if it\n has one) to be called.\n\n* If the target is a target list enclosed in parentheses or in\n square brackets: The object must be an iterable with the same number\n of items as there are targets in the target list, and its items are\n assigned, from left to right, to the corresponding targets.\n\n* If the target is an attribute reference: The primary expression in\n the reference is evaluated. It should yield an object with\n assignable attributes; if this is not the case, "TypeError" is\n raised. That object is then asked to assign the assigned object to\n the given attribute; if it cannot perform the assignment, it raises\n an exception (usually but not necessarily "AttributeError").\n\n Note: If the object is a class instance and the attribute reference\n occurs on both sides of the assignment operator, the RHS expression,\n "a.x" can access either an instance attribute or (if no instance\n attribute exists) a class attribute. The LHS target "a.x" is always\n set as an instance attribute, creating it if necessary. Thus, the\n two occurrences of "a.x" do not necessarily refer to the same\n attribute: if the RHS expression refers to a class attribute, the\n LHS creates a new instance attribute as the target of the\n assignment:\n\n class Cls:\n x = 3 # class variable\n inst = Cls()\n inst.x = inst.x + 1 # writes inst.x as 4 leaving Cls.x as 3\n\n This description does not necessarily apply to descriptor\n attributes, such as properties created with "property()".\n\n* If the target is a subscription: The primary expression in the\n reference is evaluated. It should yield either a mutable sequence\n object (such as a list) or a mapping object (such as a dictionary).\n Next, the subscript expression is evaluated.\n\n If the primary is a mutable sequence object (such as a list), the\n subscript must yield a plain integer. If it is negative, the\n sequence\'s length is added to it. The resulting value must be a\n nonnegative integer less than the sequence\'s length, and the\n sequence is asked to assign the assigned object to its item with\n that index. If the index is out of range, "IndexError" is raised\n (assignment to a subscripted sequence cannot add new items to a\n list).\n\n If the primary is a mapping object (such as a dictionary), the\n subscript must have a type compatible with the mapping\'s key type,\n and the mapping is then asked to create a key/datum pair which maps\n the subscript to the assigned object. This can either replace an\n existing key/value pair with the same key value, or insert a new\n key/value pair (if no key with the same value existed).\n\n* If the target is a slicing: The primary expression in the\n reference is evaluated. It should yield a mutable sequence object\n (such as a list). The assigned object should be a sequence object\n of the same type. Next, the lower and upper bound expressions are\n evaluated, insofar they are present; defaults are zero and the\n sequence\'s length. The bounds should evaluate to (small) integers.\n If either bound is negative, the sequence\'s length is added to it.\n The resulting bounds are clipped to lie between zero and the\n sequence\'s length, inclusive. Finally, the sequence object is asked\n to replace the slice with the items of the assigned sequence. The\n length of the slice may be different from the length of the assigned\n sequence, thus changing the length of the target sequence, if the\n object allows it.\n\n**CPython implementation detail:** In the current implementation, the\nsyntax for targets is taken to be the same as for expressions, and\ninvalid syntax is rejected during the code generation phase, causing\nless detailed error messages.\n\nWARNING: Although the definition of assignment implies that overlaps\nbetween the left-hand side and the right-hand side are \'safe\' (for\nexample "a, b = b, a" swaps two variables), overlaps *within* the\ncollection of assigned-to variables are not safe! For instance, the\nfollowing program prints "[0, 2]":\n\n x = [0, 1]\n i = 0\n i, x[i] = 1, 2\n print x\n\n\nAugmented assignment statements\n===============================\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section Primaries for the syntax definitions for the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same caveat about\nclass and instance attributes applies as for regular assignments.\n', + 'atom-identifiers': u'\nIdentifiers (Names)\n*******************\n\nAn identifier occurring as an atom is a name. See section Identifiers\nand keywords for lexical definition and section Naming and binding for\ndocumentation of naming and binding.\n\nWhen the name is bound to an object, evaluation of the atom yields\nthat object. When a name is not bound, an attempt to evaluate it\nraises a "NameError" exception.\n\n**Private name mangling:** When an identifier that textually occurs in\na class definition begins with two or more underscore characters and\ndoes not end in two or more underscores, it is considered a *private\nname* of that class. Private names are transformed to a longer form\nbefore code is generated for them. The transformation inserts the\nclass name, with leading underscores removed and a single underscore\ninserted, in front of the name. For example, the identifier "__spam"\noccurring in a class named "Ham" will be transformed to "_Ham__spam".\nThis transformation is independent of the syntactical context in which\nthe identifier is used. If the transformed name is extremely long\n(longer than 255 characters), implementation defined truncation may\nhappen. If the class name consists only of underscores, no\ntransformation is done.\n', + 'atom-literals': u"\nLiterals\n********\n\nPython supports string literals and various numeric literals:\n\n literal ::= stringliteral | integer | longinteger\n | floatnumber | imagnumber\n\nEvaluation of a literal yields an object of the given type (string,\ninteger, long integer, floating point number, complex number) with the\ngiven value. The value may be approximated in the case of floating\npoint and imaginary (complex) literals. See section Literals for\ndetails.\n\nAll literals correspond to immutable data types, and hence the\nobject's identity is less important than its value. Multiple\nevaluations of literals with the same value (either the same\noccurrence in the program text or a different occurrence) may obtain\nthe same object or a different object with the same value.\n", + 'attribute-access': u'\nCustomizing attribute access\n****************************\n\nThe following methods can be defined to customize the meaning of\nattribute access (use of, assignment to, or deletion of "x.name") for\nclass instances.\n\nobject.__getattr__(self, name)\n\n Called when an attribute lookup has not found the attribute in the\n usual places (i.e. it is not an instance attribute nor is it found\n in the class tree for "self"). "name" is the attribute name. This\n method should return the (computed) attribute value or raise an\n "AttributeError" exception.\n\n Note that if the attribute is found through the normal mechanism,\n "__getattr__()" is not called. (This is an intentional asymmetry\n between "__getattr__()" and "__setattr__()".) This is done both for\n efficiency reasons and because otherwise "__getattr__()" would have\n no way to access other attributes of the instance. Note that at\n least for instance variables, you can fake total control by not\n inserting any values in the instance attribute dictionary (but\n instead inserting them in another object). See the\n "__getattribute__()" method below for a way to actually get total\n control in new-style classes.\n\nobject.__setattr__(self, name, value)\n\n Called when an attribute assignment is attempted. This is called\n instead of the normal mechanism (i.e. store the value in the\n instance dictionary). *name* is the attribute name, *value* is the\n value to be assigned to it.\n\n If "__setattr__()" wants to assign to an instance attribute, it\n should not simply execute "self.name = value" --- this would cause\n a recursive call to itself. Instead, it should insert the value in\n the dictionary of instance attributes, e.g., "self.__dict__[name] =\n value". For new-style classes, rather than accessing the instance\n dictionary, it should call the base class method with the same\n name, for example, "object.__setattr__(self, name, value)".\n\nobject.__delattr__(self, name)\n\n Like "__setattr__()" but for attribute deletion instead of\n assignment. This should only be implemented if "del obj.name" is\n meaningful for the object.\n\n\nMore attribute access for new-style classes\n===========================================\n\nThe following methods only apply to new-style classes.\n\nobject.__getattribute__(self, name)\n\n Called unconditionally to implement attribute accesses for\n instances of the class. If the class also defines "__getattr__()",\n the latter will not be called unless "__getattribute__()" either\n calls it explicitly or raises an "AttributeError". This method\n should return the (computed) attribute value or raise an\n "AttributeError" exception. In order to avoid infinite recursion in\n this method, its implementation should always call the base class\n method with the same name to access any attributes it needs, for\n example, "object.__getattribute__(self, name)".\n\n Note: This method may still be bypassed when looking up special\n methods as the result of implicit invocation via language syntax\n or built-in functions. See Special method lookup for new-style\n classes.\n\n\nImplementing Descriptors\n========================\n\nThe following methods only apply when an instance of the class\ncontaining the method (a so-called *descriptor* class) appears in an\n*owner* class (the descriptor must be in either the owner\'s class\ndictionary or in the class dictionary for one of its parents). In the\nexamples below, "the attribute" refers to the attribute whose name is\nthe key of the property in the owner class\' "__dict__".\n\nobject.__get__(self, instance, owner)\n\n Called to get the attribute of the owner class (class attribute\n access) or of an instance of that class (instance attribute\n access). *owner* is always the owner class, while *instance* is the\n instance that the attribute was accessed through, or "None" when\n the attribute is accessed through the *owner*. This method should\n return the (computed) attribute value or raise an "AttributeError"\n exception.\n\nobject.__set__(self, instance, value)\n\n Called to set the attribute on an instance *instance* of the owner\n class to a new value, *value*.\n\nobject.__delete__(self, instance)\n\n Called to delete the attribute on an instance *instance* of the\n owner class.\n\n\nInvoking Descriptors\n====================\n\nIn general, a descriptor is an object attribute with "binding\nbehavior", one whose attribute access has been overridden by methods\nin the descriptor protocol: "__get__()", "__set__()", and\n"__delete__()". If any of those methods are defined for an object, it\nis said to be a descriptor.\n\nThe default behavior for attribute access is to get, set, or delete\nthe attribute from an object\'s dictionary. For instance, "a.x" has a\nlookup chain starting with "a.__dict__[\'x\']", then\n"type(a).__dict__[\'x\']", and continuing through the base classes of\n"type(a)" excluding metaclasses.\n\nHowever, if the looked-up value is an object defining one of the\ndescriptor methods, then Python may override the default behavior and\ninvoke the descriptor method instead. Where this occurs in the\nprecedence chain depends on which descriptor methods were defined and\nhow they were called. Note that descriptors are only invoked for new\nstyle objects or classes (ones that subclass "object()" or "type()").\n\nThe starting point for descriptor invocation is a binding, "a.x". How\nthe arguments are assembled depends on "a":\n\nDirect Call\n The simplest and least common call is when user code directly\n invokes a descriptor method: "x.__get__(a)".\n\nInstance Binding\n If binding to a new-style object instance, "a.x" is transformed\n into the call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n\nClass Binding\n If binding to a new-style class, "A.x" is transformed into the\n call: "A.__dict__[\'x\'].__get__(None, A)".\n\nSuper Binding\n If "a" is an instance of "super", then the binding "super(B,\n obj).m()" searches "obj.__class__.__mro__" for the base class "A"\n immediately preceding "B" and then invokes the descriptor with the\n call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n\nFor instance bindings, the precedence of descriptor invocation depends\non the which descriptor methods are defined. A descriptor can define\nany combination of "__get__()", "__set__()" and "__delete__()". If it\ndoes not define "__get__()", then accessing the attribute will return\nthe descriptor object itself unless there is a value in the object\'s\ninstance dictionary. If the descriptor defines "__set__()" and/or\n"__delete__()", it is a data descriptor; if it defines neither, it is\na non-data descriptor. Normally, data descriptors define both\n"__get__()" and "__set__()", while non-data descriptors have just the\n"__get__()" method. Data descriptors with "__set__()" and "__get__()"\ndefined always override a redefinition in an instance dictionary. In\ncontrast, non-data descriptors can be overridden by instances.\n\nPython methods (including "staticmethod()" and "classmethod()") are\nimplemented as non-data descriptors. Accordingly, instances can\nredefine and override methods. This allows individual instances to\nacquire behaviors that differ from other instances of the same class.\n\nThe "property()" function is implemented as a data descriptor.\nAccordingly, instances cannot override the behavior of a property.\n\n\n__slots__\n=========\n\nBy default, instances of both old and new-style classes have a\ndictionary for attribute storage. This wastes space for objects\nhaving very few instance variables. The space consumption can become\nacute when creating large numbers of instances.\n\nThe default can be overridden by defining *__slots__* in a new-style\nclass definition. The *__slots__* declaration takes a sequence of\ninstance variables and reserves just enough space in each instance to\nhold a value for each variable. Space is saved because *__dict__* is\nnot created for each instance.\n\n__slots__\n\n This class variable can be assigned a string, iterable, or sequence\n of strings with variable names used by instances. If defined in a\n new-style class, *__slots__* reserves space for the declared\n variables and prevents the automatic creation of *__dict__* and\n *__weakref__* for each instance.\n\n New in version 2.2.\n\nNotes on using *__slots__*\n\n* When inheriting from a class without *__slots__*, the *__dict__*\n attribute of that class will always be accessible, so a *__slots__*\n definition in the subclass is meaningless.\n\n* Without a *__dict__* variable, instances cannot be assigned new\n variables not listed in the *__slots__* definition. Attempts to\n assign to an unlisted variable name raises "AttributeError". If\n dynamic assignment of new variables is desired, then add\n "\'__dict__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n Changed in version 2.3: Previously, adding "\'__dict__\'" to the\n *__slots__* declaration would not enable the assignment of new\n attributes not specifically listed in the sequence of instance\n variable names.\n\n* Without a *__weakref__* variable for each instance, classes\n defining *__slots__* do not support weak references to its\n instances. If weak reference support is needed, then add\n "\'__weakref__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n Changed in version 2.3: Previously, adding "\'__weakref__\'" to the\n *__slots__* declaration would not enable support for weak\n references.\n\n* *__slots__* are implemented at the class level by creating\n descriptors (Implementing Descriptors) for each variable name. As a\n result, class attributes cannot be used to set default values for\n instance variables defined by *__slots__*; otherwise, the class\n attribute would overwrite the descriptor assignment.\n\n* The action of a *__slots__* declaration is limited to the class\n where it is defined. As a result, subclasses will have a *__dict__*\n unless they also define *__slots__* (which must only contain names\n of any *additional* slots).\n\n* If a class defines a slot also defined in a base class, the\n instance variable defined by the base class slot is inaccessible\n (except by retrieving its descriptor directly from the base class).\n This renders the meaning of the program undefined. In the future, a\n check may be added to prevent this.\n\n* Nonempty *__slots__* does not work for classes derived from\n "variable-length" built-in types such as "long", "str" and "tuple".\n\n* Any non-string iterable may be assigned to *__slots__*. Mappings\n may also be used; however, in the future, special meaning may be\n assigned to the values corresponding to each key.\n\n* *__class__* assignment works only if both classes have the same\n *__slots__*.\n\n Changed in version 2.6: Previously, *__class__* assignment raised an\n error if either new or old class had *__slots__*.\n', + 'attribute-references': u'\nAttribute references\n********************\n\nAn attribute reference is a primary followed by a period and a name:\n\n attributeref ::= primary "." identifier\n\nThe primary must evaluate to an object of a type that supports\nattribute references, e.g., a module, list, or an instance. This\nobject is then asked to produce the attribute whose name is the\nidentifier. If this attribute is not available, the exception\n"AttributeError" is raised. Otherwise, the type and value of the\nobject produced is determined by the object. Multiple evaluations of\nthe same attribute reference may yield different objects.\n', + 'augassign': u'\nAugmented assignment statements\n*******************************\n\nAugmented assignment is the combination, in a single statement, of a\nbinary operation and an assignment statement:\n\n augmented_assignment_stmt ::= augtarget augop (expression_list | yield_expression)\n augtarget ::= identifier | attributeref | subscription | slicing\n augop ::= "+=" | "-=" | "*=" | "/=" | "//=" | "%=" | "**="\n | ">>=" | "<<=" | "&=" | "^=" | "|="\n\n(See section Primaries for the syntax definitions for the last three\nsymbols.)\n\nAn augmented assignment evaluates the target (which, unlike normal\nassignment statements, cannot be an unpacking) and the expression\nlist, performs the binary operation specific to the type of assignment\non the two operands, and assigns the result to the original target.\nThe target is only evaluated once.\n\nAn augmented assignment expression like "x += 1" can be rewritten as\n"x = x + 1" to achieve a similar, but not exactly equal effect. In the\naugmented version, "x" is only evaluated once. Also, when possible,\nthe actual operation is performed *in-place*, meaning that rather than\ncreating a new object and assigning that to the target, the old object\nis modified instead.\n\nWith the exception of assigning to tuples and multiple targets in a\nsingle statement, the assignment done by augmented assignment\nstatements is handled the same way as normal assignments. Similarly,\nwith the exception of the possible *in-place* behavior, the binary\noperation performed by augmented assignment is the same as the normal\nbinary operations.\n\nFor targets which are attribute references, the same caveat about\nclass and instance attributes applies as for regular assignments.\n', + 'binary': u'\nBinary arithmetic operations\n****************************\n\nThe binary arithmetic operations have the conventional priority\nlevels. Note that some of these operations also apply to certain non-\nnumeric types. Apart from the power operator, there are only two\nlevels, one for multiplicative operators and one for additive\noperators:\n\n m_expr ::= u_expr | m_expr "*" u_expr | m_expr "//" u_expr | m_expr "/" u_expr\n | m_expr "%" u_expr\n a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr\n\nThe "*" (multiplication) operator yields the product of its arguments.\nThe arguments must either both be numbers, or one argument must be an\ninteger (plain or long) and the other must be a sequence. In the\nformer case, the numbers are converted to a common type and then\nmultiplied together. In the latter case, sequence repetition is\nperformed; a negative repetition factor yields an empty sequence.\n\nThe "/" (division) and "//" (floor division) operators yield the\nquotient of their arguments. The numeric arguments are first\nconverted to a common type. Plain or long integer division yields an\ninteger of the same type; the result is that of mathematical division\nwith the \'floor\' function applied to the result. Division by zero\nraises the "ZeroDivisionError" exception.\n\nThe "%" (modulo) operator yields the remainder from the division of\nthe first argument by the second. The numeric arguments are first\nconverted to a common type. A zero right argument raises the\n"ZeroDivisionError" exception. The arguments may be floating point\nnumbers, e.g., "3.14%0.7" equals "0.34" (since "3.14" equals "4*0.7 +\n0.34".) The modulo operator always yields a result with the same sign\nas its second operand (or zero); the absolute value of the result is\nstrictly smaller than the absolute value of the second operand [2].\n\nThe integer division and modulo operators are connected by the\nfollowing identity: "x == (x/y)*y + (x%y)". Integer division and\nmodulo are also connected with the built-in function "divmod()":\n"divmod(x, y) == (x/y, x%y)". These identities don\'t hold for\nfloating point numbers; there similar identities hold approximately\nwhere "x/y" is replaced by "floor(x/y)" or "floor(x/y) - 1" [3].\n\nIn addition to performing the modulo operation on numbers, the "%"\noperator is also overloaded by string and unicode objects to perform\nstring formatting (also known as interpolation). The syntax for string\nformatting is described in the Python Library Reference, section\nString Formatting Operations.\n\nDeprecated since version 2.3: The floor division operator, the modulo\noperator, and the "divmod()" function are no longer defined for\ncomplex numbers. Instead, convert to a floating point number using\nthe "abs()" function if appropriate.\n\nThe "+" (addition) operator yields the sum of its arguments. The\narguments must either both be numbers or both sequences of the same\ntype. In the former case, the numbers are converted to a common type\nand then added together. In the latter case, the sequences are\nconcatenated.\n\nThe "-" (subtraction) operator yields the difference of its arguments.\nThe numeric arguments are first converted to a common type.\n', + 'bitwise': u'\nBinary bitwise operations\n*************************\n\nEach of the three bitwise operations has a different priority level:\n\n and_expr ::= shift_expr | and_expr "&" shift_expr\n xor_expr ::= and_expr | xor_expr "^" and_expr\n or_expr ::= xor_expr | or_expr "|" xor_expr\n\nThe "&" operator yields the bitwise AND of its arguments, which must\nbe plain or long integers. The arguments are converted to a common\ntype.\n\nThe "^" operator yields the bitwise XOR (exclusive OR) of its\narguments, which must be plain or long integers. The arguments are\nconverted to a common type.\n\nThe "|" operator yields the bitwise (inclusive) OR of its arguments,\nwhich must be plain or long integers. The arguments are converted to\na common type.\n', + 'bltin-code-objects': u'\nCode Objects\n************\n\nCode objects are used by the implementation to represent "pseudo-\ncompiled" executable Python code such as a function body. They differ\nfrom function objects because they don\'t contain a reference to their\nglobal execution environment. Code objects are returned by the built-\nin "compile()" function and can be extracted from function objects\nthrough their "func_code" attribute. See also the "code" module.\n\nA code object can be executed or evaluated by passing it (instead of a\nsource string) to the "exec" statement or the built-in "eval()"\nfunction.\n\nSee The standard type hierarchy for more information.\n', + 'bltin-ellipsis-object': u'\nThe Ellipsis Object\n*******************\n\nThis object is used by extended slice notation (see Slicings). It\nsupports no special operations. There is exactly one ellipsis object,\nnamed "Ellipsis" (a built-in name).\n\nIt is written as "Ellipsis". When in a subscript, it can also be\nwritten as "...", for example "seq[...]".\n', + 'bltin-file-objects': u'\nFile Objects\n************\n\nFile objects are implemented using C\'s "stdio" package and can be\ncreated with the built-in "open()" function. File objects are also\nreturned by some other built-in functions and methods, such as\n"os.popen()" and "os.fdopen()" and the "makefile()" method of socket\nobjects. Temporary files can be created using the "tempfile" module,\nand high-level file operations such as copying, moving, and deleting\nfiles and directories can be achieved with the "shutil" module.\n\nWhen a file operation fails for an I/O-related reason, the exception\n"IOError" is raised. This includes situations where the operation is\nnot defined for some reason, like "seek()" on a tty device or writing\na file opened for reading.\n\nFiles have the following methods:\n\nfile.close()\n\n Close the file. A closed file cannot be read or written any more.\n Any operation which requires that the file be open will raise a\n "ValueError" after the file has been closed. Calling "close()"\n more than once is allowed.\n\n As of Python 2.5, you can avoid having to call this method\n explicitly if you use the "with" statement. For example, the\n following code will automatically close *f* when the "with" block\n is exited:\n\n from __future__ import with_statement # This isn\'t required in Python 2.6\n\n with open("hello.txt") as f:\n for line in f:\n print line,\n\n In older versions of Python, you would have needed to do this to\n get the same effect:\n\n f = open("hello.txt")\n try:\n for line in f:\n print line,\n finally:\n f.close()\n\n Note: Not all "file-like" types in Python support use as a\n context manager for the "with" statement. If your code is\n intended to work with any file-like object, you can use the\n function "contextlib.closing()" instead of using the object\n directly.\n\nfile.flush()\n\n Flush the internal buffer, like "stdio"\'s "fflush()". This may be\n a no-op on some file-like objects.\n\n Note: "flush()" does not necessarily write the file\'s data to\n disk. Use "flush()" followed by "os.fsync()" to ensure this\n behavior.\n\nfile.fileno()\n\n Return the integer "file descriptor" that is used by the underlying\n implementation to request I/O operations from the operating system.\n This can be useful for other, lower level interfaces that use file\n descriptors, such as the "fcntl" module or "os.read()" and friends.\n\n Note: File-like objects which do not have a real file descriptor\n should *not* provide this method!\n\nfile.isatty()\n\n Return "True" if the file is connected to a tty(-like) device, else\n "False".\n\n Note: If a file-like object is not associated with a real file,\n this method should *not* be implemented.\n\nfile.next()\n\n A file object is its own iterator, for example "iter(f)" returns\n *f* (unless *f* is closed). When a file is used as an iterator,\n typically in a "for" loop (for example, "for line in f: print\n line.strip()"), the "next()" method is called repeatedly. This\n method returns the next input line, or raises "StopIteration" when\n EOF is hit when the file is open for reading (behavior is undefined\n when the file is open for writing). In order to make a "for" loop\n the most efficient way of looping over the lines of a file (a very\n common operation), the "next()" method uses a hidden read-ahead\n buffer. As a consequence of using a read-ahead buffer, combining\n "next()" with other file methods (like "readline()") does not work\n right. However, using "seek()" to reposition the file to an\n absolute position will flush the read-ahead buffer.\n\n New in version 2.3.\n\nfile.read([size])\n\n Read at most *size* bytes from the file (less if the read hits EOF\n before obtaining *size* bytes). If the *size* argument is negative\n or omitted, read all data until EOF is reached. The bytes are\n returned as a string object. An empty string is returned when EOF\n is encountered immediately. (For certain files, like ttys, it\n makes sense to continue reading after an EOF is hit.) Note that\n this method may call the underlying C function "fread()" more than\n once in an effort to acquire as close to *size* bytes as possible.\n Also note that when in non-blocking mode, less data than was\n requested may be returned, even if no *size* parameter was given.\n\n Note: This function is simply a wrapper for the underlying\n "fread()" C function, and will behave the same in corner cases,\n such as whether the EOF value is cached.\n\nfile.readline([size])\n\n Read one entire line from the file. A trailing newline character\n is kept in the string (but may be absent when a file ends with an\n incomplete line). [6] If the *size* argument is present and non-\n negative, it is a maximum byte count (including the trailing\n newline) and an incomplete line may be returned. When *size* is not\n 0, an empty string is returned *only* when EOF is encountered\n immediately.\n\n Note: Unlike "stdio"\'s "fgets()", the returned string contains\n null characters ("\'\\0\'") if they occurred in the input.\n\nfile.readlines([sizehint])\n\n Read until EOF using "readline()" and return a list containing the\n lines thus read. If the optional *sizehint* argument is present,\n instead of reading up to EOF, whole lines totalling approximately\n *sizehint* bytes (possibly after rounding up to an internal buffer\n size) are read. Objects implementing a file-like interface may\n choose to ignore *sizehint* if it cannot be implemented, or cannot\n be implemented efficiently.\n\nfile.xreadlines()\n\n This method returns the same thing as "iter(f)".\n\n New in version 2.1.\n\n Deprecated since version 2.3: Use "for line in file" instead.\n\nfile.seek(offset[, whence])\n\n Set the file\'s current position, like "stdio"\'s "fseek()". The\n *whence* argument is optional and defaults to "os.SEEK_SET" or "0"\n (absolute file positioning); other values are "os.SEEK_CUR" or "1"\n (seek relative to the current position) and "os.SEEK_END" or "2"\n (seek relative to the file\'s end). There is no return value.\n\n For example, "f.seek(2, os.SEEK_CUR)" advances the position by two\n and "f.seek(-3, os.SEEK_END)" sets the position to the third to\n last.\n\n Note that if the file is opened for appending (mode "\'a\'" or\n "\'a+\'"), any "seek()" operations will be undone at the next write.\n If the file is only opened for writing in append mode (mode "\'a\'"),\n this method is essentially a no-op, but it remains useful for files\n opened in append mode with reading enabled (mode "\'a+\'"). If the\n file is opened in text mode (without "\'b\'"), only offsets returned\n by "tell()" are legal. Use of other offsets causes undefined\n behavior.\n\n Note that not all file objects are seekable.\n\n Changed in version 2.6: Passing float values as offset has been\n deprecated.\n\nfile.tell()\n\n Return the file\'s current position, like "stdio"\'s "ftell()".\n\n Note: On Windows, "tell()" can return illegal values (after an\n "fgets()") when reading files with Unix-style line-endings. Use\n binary mode ("\'rb\'") to circumvent this problem.\n\nfile.truncate([size])\n\n Truncate the file\'s size. If the optional *size* argument is\n present, the file is truncated to (at most) that size. The size\n defaults to the current position. The current file position is not\n changed. Note that if a specified size exceeds the file\'s current\n size, the result is platform-dependent: possibilities include that\n the file may remain unchanged, increase to the specified size as if\n zero-filled, or increase to the specified size with undefined new\n content. Availability: Windows, many Unix variants.\n\nfile.write(str)\n\n Write a string to the file. There is no return value. Due to\n buffering, the string may not actually show up in the file until\n the "flush()" or "close()" method is called.\n\nfile.writelines(sequence)\n\n Write a sequence of strings to the file. The sequence can be any\n iterable object producing strings, typically a list of strings.\n There is no return value. (The name is intended to match\n "readlines()"; "writelines()" does not add line separators.)\n\nFiles support the iterator protocol. Each iteration returns the same\nresult as "readline()", and iteration ends when the "readline()"\nmethod returns an empty string.\n\nFile objects also offer a number of other interesting attributes.\nThese are not required for file-like objects, but should be\nimplemented if they make sense for the particular object.\n\nfile.closed\n\n bool indicating the current state of the file object. This is a\n read-only attribute; the "close()" method changes the value. It may\n not be available on all file-like objects.\n\nfile.encoding\n\n The encoding that this file uses. When Unicode strings are written\n to a file, they will be converted to byte strings using this\n encoding. In addition, when the file is connected to a terminal,\n the attribute gives the encoding that the terminal is likely to use\n (that information might be incorrect if the user has misconfigured\n the terminal). The attribute is read-only and may not be present\n on all file-like objects. It may also be "None", in which case the\n file uses the system default encoding for converting Unicode\n strings.\n\n New in version 2.3.\n\nfile.errors\n\n The Unicode error handler used along with the encoding.\n\n New in version 2.6.\n\nfile.mode\n\n The I/O mode for the file. If the file was created using the\n "open()" built-in function, this will be the value of the *mode*\n parameter. This is a read-only attribute and may not be present on\n all file-like objects.\n\nfile.name\n\n If the file object was created using "open()", the name of the\n file. Otherwise, some string that indicates the source of the file\n object, of the form "<...>". This is a read-only attribute and may\n not be present on all file-like objects.\n\nfile.newlines\n\n If Python was built with *universal newlines* enabled (the default)\n this read-only attribute exists, and for files opened in universal\n newline read mode it keeps track of the types of newlines\n encountered while reading the file. The values it can take are\n "\'\\r\'", "\'\\n\'", "\'\\r\\n\'", "None" (unknown, no newlines read yet) or\n a tuple containing all the newline types seen, to indicate that\n multiple newline conventions were encountered. For files not opened\n in universal newlines read mode the value of this attribute will be\n "None".\n\nfile.softspace\n\n Boolean that indicates whether a space character needs to be\n printed before another value when using the "print" statement.\n Classes that are trying to simulate a file object should also have\n a writable "softspace" attribute, which should be initialized to\n zero. This will be automatic for most classes implemented in\n Python (care may be needed for objects that override attribute\n access); types implemented in C will have to provide a writable\n "softspace" attribute.\n\n Note: This attribute is not used to control the "print"\n statement, but to allow the implementation of "print" to keep\n track of its internal state.\n', + 'bltin-null-object': u'\nThe Null Object\n***************\n\nThis object is returned by functions that don\'t explicitly return a\nvalue. It supports no special operations. There is exactly one null\nobject, named "None" (a built-in name).\n\nIt is written as "None".\n', + 'bltin-type-objects': u'\nType Objects\n************\n\nType objects represent the various object types. An object\'s type is\naccessed by the built-in function "type()". There are no special\noperations on types. The standard module "types" defines names for\nall standard built-in types.\n\nTypes are written like this: "".\n', + 'booleans': u'\nBoolean operations\n******************\n\n or_test ::= and_test | or_test "or" and_test\n and_test ::= not_test | and_test "and" not_test\n not_test ::= comparison | "not" not_test\n\nIn the context of Boolean operations, and also when expressions are\nused by control flow statements, the following values are interpreted\nas false: "False", "None", numeric zero of all types, and empty\nstrings and containers (including strings, tuples, lists,\ndictionaries, sets and frozensets). All other values are interpreted\nas true. (See the "__nonzero__()" special method for a way to change\nthis.)\n\nThe operator "not" yields "True" if its argument is false, "False"\notherwise.\n\nThe expression "x and y" first evaluates *x*; if *x* is false, its\nvalue is returned; otherwise, *y* is evaluated and the resulting value\nis returned.\n\nThe expression "x or y" first evaluates *x*; if *x* is true, its value\nis returned; otherwise, *y* is evaluated and the resulting value is\nreturned.\n\n(Note that neither "and" nor "or" restrict the value and type they\nreturn to "False" and "True", but rather return the last evaluated\nargument. This is sometimes useful, e.g., if "s" is a string that\nshould be replaced by a default value if it is empty, the expression\n"s or \'foo\'" yields the desired value. Because "not" has to invent a\nvalue anyway, it does not bother to return a value of the same type as\nits argument, so e.g., "not \'foo\'" yields "False", not "\'\'".)\n', + 'break': u'\nThe "break" statement\n*********************\n\n break_stmt ::= "break"\n\n"break" may only occur syntactically nested in a "for" or "while"\nloop, but not nested in a function or class definition within that\nloop.\n\nIt terminates the nearest enclosing loop, skipping the optional "else"\nclause if the loop has one.\n\nIf a "for" loop is terminated by "break", the loop control target\nkeeps its current value.\n\nWhen "break" passes control out of a "try" statement with a "finally"\nclause, that "finally" clause is executed before really leaving the\nloop.\n', + 'callable-types': u'\nEmulating callable objects\n**************************\n\nobject.__call__(self[, args...])\n\n Called when the instance is "called" as a function; if this method\n is defined, "x(arg1, arg2, ...)" is a shorthand for\n "x.__call__(arg1, arg2, ...)".\n', + 'calls': u'\nCalls\n*****\n\nA call calls a callable object (e.g., a *function*) with a possibly\nempty series of *arguments*:\n\n call ::= primary "(" [argument_list [","]\n | expression genexpr_for] ")"\n argument_list ::= positional_arguments ["," keyword_arguments]\n ["," "*" expression] ["," keyword_arguments]\n ["," "**" expression]\n | keyword_arguments ["," "*" expression]\n ["," "**" expression]\n | "*" expression ["," keyword_arguments] ["," "**" expression]\n | "**" expression\n positional_arguments ::= expression ("," expression)*\n keyword_arguments ::= keyword_item ("," keyword_item)*\n keyword_item ::= identifier "=" expression\n\nA trailing comma may be present after the positional and keyword\narguments but does not affect the semantics.\n\nThe primary must evaluate to a callable object (user-defined\nfunctions, built-in functions, methods of built-in objects, class\nobjects, methods of class instances, and certain class instances\nthemselves are callable; extensions may define additional callable\nobject types). All argument expressions are evaluated before the call\nis attempted. Please refer to section Function definitions for the\nsyntax of formal *parameter* lists.\n\nIf keyword arguments are present, they are first converted to\npositional arguments, as follows. First, a list of unfilled slots is\ncreated for the formal parameters. If there are N positional\narguments, they are placed in the first N slots. Next, for each\nkeyword argument, the identifier is used to determine the\ncorresponding slot (if the identifier is the same as the first formal\nparameter name, the first slot is used, and so on). If the slot is\nalready filled, a "TypeError" exception is raised. Otherwise, the\nvalue of the argument is placed in the slot, filling it (even if the\nexpression is "None", it fills the slot). When all arguments have\nbeen processed, the slots that are still unfilled are filled with the\ncorresponding default value from the function definition. (Default\nvalues are calculated, once, when the function is defined; thus, a\nmutable object such as a list or dictionary used as default value will\nbe shared by all calls that don\'t specify an argument value for the\ncorresponding slot; this should usually be avoided.) If there are any\nunfilled slots for which no default value is specified, a "TypeError"\nexception is raised. Otherwise, the list of filled slots is used as\nthe argument list for the call.\n\n**CPython implementation detail:** An implementation may provide\nbuilt-in functions whose positional parameters do not have names, even\nif they are \'named\' for the purpose of documentation, and which\ntherefore cannot be supplied by keyword. In CPython, this is the case\nfor functions implemented in C that use "PyArg_ParseTuple()" to parse\ntheir arguments.\n\nIf there are more positional arguments than there are formal parameter\nslots, a "TypeError" exception is raised, unless a formal parameter\nusing the syntax "*identifier" is present; in this case, that formal\nparameter receives a tuple containing the excess positional arguments\n(or an empty tuple if there were no excess positional arguments).\n\nIf any keyword argument does not correspond to a formal parameter\nname, a "TypeError" exception is raised, unless a formal parameter\nusing the syntax "**identifier" is present; in this case, that formal\nparameter receives a dictionary containing the excess keyword\narguments (using the keywords as keys and the argument values as\ncorresponding values), or a (new) empty dictionary if there were no\nexcess keyword arguments.\n\nIf the syntax "*expression" appears in the function call, "expression"\nmust evaluate to an iterable. Elements from this iterable are treated\nas if they were additional positional arguments; if there are\npositional arguments *x1*, ..., *xN*, and "expression" evaluates to a\nsequence *y1*, ..., *yM*, this is equivalent to a call with M+N\npositional arguments *x1*, ..., *xN*, *y1*, ..., *yM*.\n\nA consequence of this is that although the "*expression" syntax may\nappear *after* some keyword arguments, it is processed *before* the\nkeyword arguments (and the "**expression" argument, if any -- see\nbelow). So:\n\n >>> def f(a, b):\n ... print a, b\n ...\n >>> f(b=1, *(2,))\n 2 1\n >>> f(a=1, *(2,))\n Traceback (most recent call last):\n File "", line 1, in ?\n TypeError: f() got multiple values for keyword argument \'a\'\n >>> f(1, *(2,))\n 1 2\n\nIt is unusual for both keyword arguments and the "*expression" syntax\nto be used in the same call, so in practice this confusion does not\narise.\n\nIf the syntax "**expression" appears in the function call,\n"expression" must evaluate to a mapping, the contents of which are\ntreated as additional keyword arguments. In the case of a keyword\nappearing in both "expression" and as an explicit keyword argument, a\n"TypeError" exception is raised.\n\nFormal parameters using the syntax "*identifier" or "**identifier"\ncannot be used as positional argument slots or as keyword argument\nnames. Formal parameters using the syntax "(sublist)" cannot be used\nas keyword argument names; the outermost sublist corresponds to a\nsingle unnamed argument slot, and the argument value is assigned to\nthe sublist using the usual tuple assignment rules after all other\nparameter processing is done.\n\nA call always returns some value, possibly "None", unless it raises an\nexception. How this value is computed depends on the type of the\ncallable object.\n\nIf it is---\n\na user-defined function:\n The code block for the function is executed, passing it the\n argument list. The first thing the code block will do is bind the\n formal parameters to the arguments; this is described in section\n Function definitions. When the code block executes a "return"\n statement, this specifies the return value of the function call.\n\na built-in function or method:\n The result is up to the interpreter; see Built-in Functions for the\n descriptions of built-in functions and methods.\n\na class object:\n A new instance of that class is returned.\n\na class instance method:\n The corresponding user-defined function is called, with an argument\n list that is one longer than the argument list of the call: the\n instance becomes the first argument.\n\na class instance:\n The class must define a "__call__()" method; the effect is then the\n same as if that method was called.\n', + 'class': u'\nClass definitions\n*****************\n\nA class definition defines a class object (see section The standard\ntype hierarchy):\n\n classdef ::= "class" classname [inheritance] ":" suite\n inheritance ::= "(" [expression_list] ")"\n classname ::= identifier\n\nA class definition is an executable statement. It first evaluates the\ninheritance list, if present. Each item in the inheritance list\nshould evaluate to a class object or class type which allows\nsubclassing. The class\'s suite is then executed in a new execution\nframe (see section Naming and binding), using a newly created local\nnamespace and the original global namespace. (Usually, the suite\ncontains only function definitions.) When the class\'s suite finishes\nexecution, its execution frame is discarded but its local namespace is\nsaved. [4] A class object is then created using the inheritance list\nfor the base classes and the saved local namespace for the attribute\ndictionary. The class name is bound to this class object in the\noriginal local namespace.\n\n**Programmer\'s note:** Variables defined in the class definition are\nclass variables; they are shared by all instances. To create instance\nvariables, they can be set in a method with "self.name = value". Both\nclass and instance variables are accessible through the notation\n""self.name"", and an instance variable hides a class variable with\nthe same name when accessed in this way. Class variables can be used\nas defaults for instance variables, but using mutable values there can\nlead to unexpected results. For *new-style class*es, descriptors can\nbe used to create instance variables with different implementation\ndetails.\n\nClass definitions, like function definitions, may be wrapped by one or\nmore *decorator* expressions. The evaluation rules for the decorator\nexpressions are the same as for functions. The result must be a class\nobject, which is then bound to the class name.\n\n-[ Footnotes ]-\n\n[1] The exception is propagated to the invocation stack unless\n there is a "finally" clause which happens to raise another\n exception. That new exception causes the old one to be lost.\n\n[2] Currently, control "flows off the end" except in the case of\n an exception or the execution of a "return", "continue", or\n "break" statement.\n\n[3] A string literal appearing as the first statement in the\n function body is transformed into the function\'s "__doc__"\n attribute and therefore the function\'s *docstring*.\n\n[4] A string literal appearing as the first statement in the class\n body is transformed into the namespace\'s "__doc__" item and\n therefore the class\'s *docstring*.\n', + 'comparisons': u'\nComparisons\n***********\n\nUnlike C, all comparison operations in Python have the same priority,\nwhich is lower than that of any arithmetic, shifting or bitwise\noperation. Also unlike C, expressions like "a < b < c" have the\ninterpretation that is conventional in mathematics:\n\n comparison ::= or_expr ( comp_operator or_expr )*\n comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "<>" | "!="\n | "is" ["not"] | ["not"] "in"\n\nComparisons yield boolean values: "True" or "False".\n\nComparisons can be chained arbitrarily, e.g., "x < y <= z" is\nequivalent to "x < y and y <= z", except that "y" is evaluated only\nonce (but in both cases "z" is not evaluated at all when "x < y" is\nfound to be false).\n\nFormally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*,\n*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... y\nopN z" is equivalent to "a op1 b and b op2 c and ... y opN z", except\nthat each expression is evaluated at most once.\n\nNote that "a op1 b op2 c" doesn\'t imply any kind of comparison between\n*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\nperhaps not pretty).\n\nThe forms "<>" and "!=" are equivalent; for consistency with C, "!="\nis preferred; where "!=" is mentioned below "<>" is also accepted.\nThe "<>" spelling is considered obsolescent.\n\nThe operators "<", ">", "==", ">=", "<=", and "!=" compare the values\nof two objects. The objects need not have the same type. If both are\nnumbers, they are converted to a common type. Otherwise, objects of\ndifferent types *always* compare unequal, and are ordered consistently\nbut arbitrarily. You can control comparison behavior of objects of\nnon-built-in types by defining a "__cmp__" method or rich comparison\nmethods like "__gt__", described in section Special method names.\n\n(This unusual definition of comparison was used to simplify the\ndefinition of operations like sorting and the "in" and "not in"\noperators. In the future, the comparison rules for objects of\ndifferent types are likely to change.)\n\nComparison of objects of the same type depends on the type:\n\n* Numbers are compared arithmetically.\n\n* Strings are compared lexicographically using the numeric\n equivalents (the result of the built-in function "ord()") of their\n characters. Unicode and 8-bit strings are fully interoperable in\n this behavior. [4]\n\n* Tuples and lists are compared lexicographically using comparison\n of corresponding elements. This means that to compare equal, each\n element must compare equal and the two sequences must be of the same\n type and have the same length.\n\n If not equal, the sequences are ordered the same as their first\n differing elements. For example, "cmp([1,2,x], [1,2,y])" returns\n the same as "cmp(x,y)". If the corresponding element does not\n exist, the shorter sequence is ordered first (for example, "[1,2] <\n [1,2,3]").\n\n* Mappings (dictionaries) compare equal if and only if their sorted\n (key, value) lists compare equal. [5] Outcomes other than equality\n are resolved consistently, but are not otherwise defined. [6]\n\n* Most other objects of built-in types compare unequal unless they\n are the same object; the choice whether one object is considered\n smaller or larger than another one is made arbitrarily but\n consistently within one execution of a program.\n\nThe operators "in" and "not in" test for collection membership. "x in\ns" evaluates to true if *x* is a member of the collection *s*, and\nfalse otherwise. "x not in s" returns the negation of "x in s". The\ncollection membership test has traditionally been bound to sequences;\nan object is a member of a collection if the collection is a sequence\nand contains an element equal to that object. However, it make sense\nfor many other object types to support membership tests without being\na sequence. In particular, dictionaries (for keys) and sets support\nmembership testing.\n\nFor the list and tuple types, "x in y" is true if and only if there\nexists an index *i* such that "x == y[i]" is true.\n\nFor the Unicode and string types, "x in y" is true if and only if *x*\nis a substring of *y*. An equivalent test is "y.find(x) != -1".\nNote, *x* and *y* need not be the same type; consequently, "u\'ab\' in\n\'abc\'" will return "True". Empty strings are always considered to be a\nsubstring of any other string, so """ in "abc"" will return "True".\n\nChanged in version 2.3: Previously, *x* was required to be a string of\nlength "1".\n\nFor user-defined classes which define the "__contains__()" method, "x\nin y" is true if and only if "y.__contains__(x)" is true.\n\nFor user-defined classes which do not define "__contains__()" but do\ndefine "__iter__()", "x in y" is true if some value "z" with "x == z"\nis produced while iterating over "y". If an exception is raised\nduring the iteration, it is as if "in" raised that exception.\n\nLastly, the old-style iteration protocol is tried: if a class defines\n"__getitem__()", "x in y" is true if and only if there is a non-\nnegative integer index *i* such that "x == y[i]", and all lower\ninteger indices do not raise "IndexError" exception. (If any other\nexception is raised, it is as if "in" raised that exception).\n\nThe operator "not in" is defined to have the inverse true value of\n"in".\n\nThe operators "is" and "is not" test for object identity: "x is y" is\ntrue if and only if *x* and *y* are the same object. "x is not y"\nyields the inverse truth value. [7]\n', + 'compound': u'\nCompound statements\n*******************\n\nCompound statements contain (groups of) other statements; they affect\nor control the execution of those other statements in some way. In\ngeneral, compound statements span multiple lines, although in simple\nincarnations a whole compound statement may be contained in one line.\n\nThe "if", "while" and "for" statements implement traditional control\nflow constructs. "try" specifies exception handlers and/or cleanup\ncode for a group of statements. Function and class definitions are\nalso syntactically compound statements.\n\nCompound statements consist of one or more \'clauses.\' A clause\nconsists of a header and a \'suite.\' The clause headers of a\nparticular compound statement are all at the same indentation level.\nEach clause header begins with a uniquely identifying keyword and ends\nwith a colon. A suite is a group of statements controlled by a\nclause. A suite can be one or more semicolon-separated simple\nstatements on the same line as the header, following the header\'s\ncolon, or it can be one or more indented statements on subsequent\nlines. Only the latter form of suite can contain nested compound\nstatements; the following is illegal, mostly because it wouldn\'t be\nclear to which "if" clause a following "else" clause would belong:\n\n if test1: if test2: print x\n\nAlso note that the semicolon binds tighter than the colon in this\ncontext, so that in the following example, either all or none of the\n"print" statements are executed:\n\n if x < y < z: print x; print y; print z\n\nSummarizing:\n\n compound_stmt ::= if_stmt\n | while_stmt\n | for_stmt\n | try_stmt\n | with_stmt\n | funcdef\n | classdef\n | decorated\n suite ::= stmt_list NEWLINE | NEWLINE INDENT statement+ DEDENT\n statement ::= stmt_list NEWLINE | compound_stmt\n stmt_list ::= simple_stmt (";" simple_stmt)* [";"]\n\nNote that statements always end in a "NEWLINE" possibly followed by a\n"DEDENT". Also note that optional continuation clauses always begin\nwith a keyword that cannot start a statement, thus there are no\nambiguities (the \'dangling "else"\' problem is solved in Python by\nrequiring nested "if" statements to be indented).\n\nThe formatting of the grammar rules in the following sections places\neach clause on a separate line for clarity.\n\n\nThe "if" statement\n==================\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section Boolean operations\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n\n\nThe "while" statement\n=====================\n\nThe "while" statement is used for repeated execution as long as an\nexpression is true:\n\n while_stmt ::= "while" expression ":" suite\n ["else" ":" suite]\n\nThis repeatedly tests the expression and, if it is true, executes the\nfirst suite; if the expression is false (which may be the first time\nit is tested) the suite of the "else" clause, if present, is executed\nand the loop terminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and goes back\nto testing the expression.\n\n\nThe "for" statement\n===================\n\nThe "for" statement is used to iterate over the elements of a sequence\n(such as a string, tuple or list) or other iterable object:\n\n for_stmt ::= "for" target_list "in" expression_list ":" suite\n ["else" ":" suite]\n\nThe expression list is evaluated once; it should yield an iterable\nobject. An iterator is created for the result of the\n"expression_list". The suite is then executed once for each item\nprovided by the iterator, in the order of ascending indices. Each\nitem in turn is assigned to the target list using the standard rules\nfor assignments, and then the suite is executed. When the items are\nexhausted (which is immediately when the sequence is empty), the suite\nin the "else" clause, if present, is executed, and the loop\nterminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and continues\nwith the next item, or with the "else" clause if there was no next\nitem.\n\nThe suite may assign to the variable(s) in the target list; this does\nnot affect the next item assigned to it.\n\nThe target list is not deleted when the loop is finished, but if the\nsequence is empty, it will not have been assigned to at all by the\nloop. Hint: the built-in function "range()" returns a sequence of\nintegers suitable to emulate the effect of Pascal\'s "for i := a to b\ndo"; e.g., "range(3)" returns the list "[0, 1, 2]".\n\nNote: There is a subtlety when the sequence is being modified by the\n loop (this can only occur for mutable sequences, i.e. lists). An\n internal counter is used to keep track of which item is used next,\n and this is incremented on each iteration. When this counter has\n reached the length of the sequence the loop terminates. This means\n that if the suite deletes the current (or a previous) item from the\n sequence, the next item will be skipped (since it gets the index of\n the current item which has already been treated). Likewise, if the\n suite inserts an item in the sequence before the current item, the\n current item will be treated again the next time through the loop.\n This can lead to nasty bugs that can be avoided by making a\n temporary copy using a slice of the whole sequence, e.g.,\n\n for x in a[:]:\n if x < 0: a.remove(x)\n\n\nThe "try" statement\n===================\n\nThe "try" statement specifies exception handlers and/or cleanup code\nfor a group of statements:\n\n try_stmt ::= try1_stmt | try2_stmt\n try1_stmt ::= "try" ":" suite\n ("except" [expression [("as" | ",") identifier]] ":" suite)+\n ["else" ":" suite]\n ["finally" ":" suite]\n try2_stmt ::= "try" ":" suite\n "finally" ":" suite\n\nChanged in version 2.5: In previous versions of Python,\n"try"..."except"..."finally" did not work. "try"..."except" had to be\nnested in "try"..."finally".\n\nThe "except" clause(s) specify one or more exception handlers. When no\nexception occurs in the "try" clause, no exception handler is\nexecuted. When an exception occurs in the "try" suite, a search for an\nexception handler is started. This search inspects the except clauses\nin turn until one is found that matches the exception. An expression-\nless except clause, if present, must be last; it matches any\nexception. For an except clause with an expression, that expression\nis evaluated, and the clause matches the exception if the resulting\nobject is "compatible" with the exception. An object is compatible\nwith an exception if it is the class or a base class of the exception\nobject, or a tuple containing an item compatible with the exception.\n\nIf no except clause matches the exception, the search for an exception\nhandler continues in the surrounding code and on the invocation stack.\n[1]\n\nIf the evaluation of an expression in the header of an except clause\nraises an exception, the original search for a handler is canceled and\na search starts for the new exception in the surrounding code and on\nthe call stack (it is treated as if the entire "try" statement raised\nthe exception).\n\nWhen a matching except clause is found, the exception is assigned to\nthe target specified in that except clause, if present, and the except\nclause\'s suite is executed. All except clauses must have an\nexecutable block. When the end of this block is reached, execution\ncontinues normally after the entire try statement. (This means that\nif two nested handlers exist for the same exception, and the exception\noccurs in the try clause of the inner handler, the outer handler will\nnot handle the exception.)\n\nBefore an except clause\'s suite is executed, details about the\nexception are assigned to three variables in the "sys" module:\n"sys.exc_type" receives the object identifying the exception;\n"sys.exc_value" receives the exception\'s parameter;\n"sys.exc_traceback" receives a traceback object (see section The\nstandard type hierarchy) identifying the point in the program where\nthe exception occurred. These details are also available through the\n"sys.exc_info()" function, which returns a tuple "(exc_type,\nexc_value, exc_traceback)". Use of the corresponding variables is\ndeprecated in favor of this function, since their use is unsafe in a\nthreaded program. As of Python 1.5, the variables are restored to\ntheir previous values (before the call) when returning from a function\nthat handled an exception.\n\nThe optional "else" clause is executed if and when control flows off\nthe end of the "try" clause. [2] Exceptions in the "else" clause are\nnot handled by the preceding "except" clauses.\n\nIf "finally" is present, it specifies a \'cleanup\' handler. The "try"\nclause is executed, including any "except" and "else" clauses. If an\nexception occurs in any of the clauses and is not handled, the\nexception is temporarily saved. The "finally" clause is executed. If\nthere is a saved exception, it is re-raised at the end of the\n"finally" clause. If the "finally" clause raises another exception or\nexecutes a "return" or "break" statement, the saved exception is\ndiscarded:\n\n >>> def f():\n ... try:\n ... 1/0\n ... finally:\n ... return 42\n ...\n >>> f()\n 42\n\nThe exception information is not available to the program during\nexecution of the "finally" clause.\n\nWhen a "return", "break" or "continue" statement is executed in the\n"try" suite of a "try"..."finally" statement, the "finally" clause is\nalso executed \'on the way out.\' A "continue" statement is illegal in\nthe "finally" clause. (The reason is a problem with the current\nimplementation --- this restriction may be lifted in the future).\n\nThe return value of a function is determined by the last "return"\nstatement executed. Since the "finally" clause always executes, a\n"return" statement executed in the "finally" clause will always be the\nlast one executed:\n\n >>> def foo():\n ... try:\n ... return \'try\'\n ... finally:\n ... return \'finally\'\n ...\n >>> foo()\n \'finally\'\n\nAdditional information on exceptions can be found in section\nExceptions, and information on using the "raise" statement to generate\nexceptions may be found in section The raise statement.\n\n\nThe "with" statement\n====================\n\nNew in version 2.5.\n\nThe "with" statement is used to wrap the execution of a block with\nmethods defined by a context manager (see section With Statement\nContext Managers). This allows common "try"..."except"..."finally"\nusage patterns to be encapsulated for convenient reuse.\n\n with_stmt ::= "with" with_item ("," with_item)* ":" suite\n with_item ::= expression ["as" target]\n\nThe execution of the "with" statement with one "item" proceeds as\nfollows:\n\n1. The context expression (the expression given in the "with_item")\n is evaluated to obtain a context manager.\n\n2. The context manager\'s "__exit__()" is loaded for later use.\n\n3. The context manager\'s "__enter__()" method is invoked.\n\n4. If a target was included in the "with" statement, the return\n value from "__enter__()" is assigned to it.\n\n Note: The "with" statement guarantees that if the "__enter__()"\n method returns without an error, then "__exit__()" will always be\n called. Thus, if an error occurs during the assignment to the\n target list, it will be treated the same as an error occurring\n within the suite would be. See step 6 below.\n\n5. The suite is executed.\n\n6. The context manager\'s "__exit__()" method is invoked. If an\n exception caused the suite to be exited, its type, value, and\n traceback are passed as arguments to "__exit__()". Otherwise, three\n "None" arguments are supplied.\n\n If the suite was exited due to an exception, and the return value\n from the "__exit__()" method was false, the exception is reraised.\n If the return value was true, the exception is suppressed, and\n execution continues with the statement following the "with"\n statement.\n\n If the suite was exited for any reason other than an exception, the\n return value from "__exit__()" is ignored, and execution proceeds\n at the normal location for the kind of exit that was taken.\n\nWith more than one item, the context managers are processed as if\nmultiple "with" statements were nested:\n\n with A() as a, B() as b:\n suite\n\nis equivalent to\n\n with A() as a:\n with B() as b:\n suite\n\nNote: In Python 2.5, the "with" statement is only allowed when the\n "with_statement" feature has been enabled. It is always enabled in\n Python 2.6.\n\nChanged in version 2.7: Support for multiple context expressions.\n\nSee also:\n\n **PEP 0343** - The "with" statement\n The specification, background, and examples for the Python "with"\n statement.\n\n\nFunction definitions\n====================\n\nA function definition defines a user-defined function object (see\nsection The standard type hierarchy):\n\n decorated ::= decorators (classdef | funcdef)\n decorators ::= decorator+\n decorator ::= "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE\n funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite\n dotted_name ::= identifier ("." identifier)*\n parameter_list ::= (defparameter ",")*\n ( "*" identifier ["," "**" identifier]\n | "**" identifier\n | defparameter [","] )\n defparameter ::= parameter ["=" expression]\n sublist ::= parameter ("," parameter)* [","]\n parameter ::= identifier | "(" sublist ")"\n funcname ::= identifier\n\nA function definition is an executable statement. Its execution binds\nthe function name in the current local namespace to a function object\n(a wrapper around the executable code for the function). This\nfunction object contains a reference to the current global namespace\nas the global namespace to be used when the function is called.\n\nThe function definition does not execute the function body; this gets\nexecuted only when the function is called. [3]\n\nA function definition may be wrapped by one or more *decorator*\nexpressions. Decorator expressions are evaluated when the function is\ndefined, in the scope that contains the function definition. The\nresult must be a callable, which is invoked with the function object\nas the only argument. The returned value is bound to the function name\ninstead of the function object. Multiple decorators are applied in\nnested fashion. For example, the following code:\n\n @f1(arg)\n @f2\n def func(): pass\n\nis equivalent to:\n\n def func(): pass\n func = f1(arg)(f2(func))\n\nWhen one or more top-level *parameters* have the form *parameter* "="\n*expression*, the function is said to have "default parameter values."\nFor a parameter with a default value, the corresponding *argument* may\nbe omitted from a call, in which case the parameter\'s default value is\nsubstituted. If a parameter has a default value, all following\nparameters must also have a default value --- this is a syntactic\nrestriction that is not expressed by the grammar.\n\n**Default parameter values are evaluated when the function definition\nis executed.** This means that the expression is evaluated once, when\nthe function is defined, and that the same "pre-computed" value is\nused for each call. This is especially important to understand when a\ndefault parameter is a mutable object, such as a list or a dictionary:\nif the function modifies the object (e.g. by appending an item to a\nlist), the default value is in effect modified. This is generally not\nwhat was intended. A way around this is to use "None" as the\ndefault, and explicitly test for it in the body of the function, e.g.:\n\n def whats_on_the_telly(penguin=None):\n if penguin is None:\n penguin = []\n penguin.append("property of the zoo")\n return penguin\n\nFunction call semantics are described in more detail in section Calls.\nA function call always assigns values to all parameters mentioned in\nthe parameter list, either from position arguments, from keyword\narguments, or from default values. If the form ""*identifier"" is\npresent, it is initialized to a tuple receiving any excess positional\nparameters, defaulting to the empty tuple. If the form\n""**identifier"" is present, it is initialized to a new dictionary\nreceiving any excess keyword arguments, defaulting to a new empty\ndictionary.\n\nIt is also possible to create anonymous functions (functions not bound\nto a name), for immediate use in expressions. This uses lambda\nexpressions, described in section Lambdas. Note that the lambda\nexpression is merely a shorthand for a simplified function definition;\na function defined in a ""def"" statement can be passed around or\nassigned to another name just like a function defined by a lambda\nexpression. The ""def"" form is actually more powerful since it\nallows the execution of multiple statements.\n\n**Programmer\'s note:** Functions are first-class objects. A ""def""\nform executed inside a function definition defines a local function\nthat can be returned or passed around. Free variables used in the\nnested function can access the local variables of the function\ncontaining the def. See section Naming and binding for details.\n\n\nClass definitions\n=================\n\nA class definition defines a class object (see section The standard\ntype hierarchy):\n\n classdef ::= "class" classname [inheritance] ":" suite\n inheritance ::= "(" [expression_list] ")"\n classname ::= identifier\n\nA class definition is an executable statement. It first evaluates the\ninheritance list, if present. Each item in the inheritance list\nshould evaluate to a class object or class type which allows\nsubclassing. The class\'s suite is then executed in a new execution\nframe (see section Naming and binding), using a newly created local\nnamespace and the original global namespace. (Usually, the suite\ncontains only function definitions.) When the class\'s suite finishes\nexecution, its execution frame is discarded but its local namespace is\nsaved. [4] A class object is then created using the inheritance list\nfor the base classes and the saved local namespace for the attribute\ndictionary. The class name is bound to this class object in the\noriginal local namespace.\n\n**Programmer\'s note:** Variables defined in the class definition are\nclass variables; they are shared by all instances. To create instance\nvariables, they can be set in a method with "self.name = value". Both\nclass and instance variables are accessible through the notation\n""self.name"", and an instance variable hides a class variable with\nthe same name when accessed in this way. Class variables can be used\nas defaults for instance variables, but using mutable values there can\nlead to unexpected results. For *new-style class*es, descriptors can\nbe used to create instance variables with different implementation\ndetails.\n\nClass definitions, like function definitions, may be wrapped by one or\nmore *decorator* expressions. The evaluation rules for the decorator\nexpressions are the same as for functions. The result must be a class\nobject, which is then bound to the class name.\n\n-[ Footnotes ]-\n\n[1] The exception is propagated to the invocation stack unless\n there is a "finally" clause which happens to raise another\n exception. That new exception causes the old one to be lost.\n\n[2] Currently, control "flows off the end" except in the case of\n an exception or the execution of a "return", "continue", or\n "break" statement.\n\n[3] A string literal appearing as the first statement in the\n function body is transformed into the function\'s "__doc__"\n attribute and therefore the function\'s *docstring*.\n\n[4] A string literal appearing as the first statement in the class\n body is transformed into the namespace\'s "__doc__" item and\n therefore the class\'s *docstring*.\n', + 'context-managers': u'\nWith Statement Context Managers\n*******************************\n\nNew in version 2.5.\n\nA *context manager* is an object that defines the runtime context to\nbe established when executing a "with" statement. The context manager\nhandles the entry into, and the exit from, the desired runtime context\nfor the execution of the block of code. Context managers are normally\ninvoked using the "with" statement (described in section The with\nstatement), but can also be used by directly invoking their methods.\n\nTypical uses of context managers include saving and restoring various\nkinds of global state, locking and unlocking resources, closing opened\nfiles, etc.\n\nFor more information on context managers, see Context Manager Types.\n\nobject.__enter__(self)\n\n Enter the runtime context related to this object. The "with"\n statement will bind this method\'s return value to the target(s)\n specified in the "as" clause of the statement, if any.\n\nobject.__exit__(self, exc_type, exc_value, traceback)\n\n Exit the runtime context related to this object. The parameters\n describe the exception that caused the context to be exited. If the\n context was exited without an exception, all three arguments will\n be "None".\n\n If an exception is supplied, and the method wishes to suppress the\n exception (i.e., prevent it from being propagated), it should\n return a true value. Otherwise, the exception will be processed\n normally upon exit from this method.\n\n Note that "__exit__()" methods should not reraise the passed-in\n exception; this is the caller\'s responsibility.\n\nSee also:\n\n **PEP 0343** - The "with" statement\n The specification, background, and examples for the Python "with"\n statement.\n', + 'continue': u'\nThe "continue" statement\n************************\n\n continue_stmt ::= "continue"\n\n"continue" may only occur syntactically nested in a "for" or "while"\nloop, but not nested in a function or class definition or "finally"\nclause within that loop. It continues with the next cycle of the\nnearest enclosing loop.\n\nWhen "continue" passes control out of a "try" statement with a\n"finally" clause, that "finally" clause is executed before really\nstarting the next loop cycle.\n', + 'conversions': u'\nArithmetic conversions\n**********************\n\nWhen a description of an arithmetic operator below uses the phrase\n"the numeric arguments are converted to a common type," the arguments\nare coerced using the coercion rules listed at Coercion rules. If\nboth arguments are standard numeric types, the following coercions are\napplied:\n\n* If either argument is a complex number, the other is converted to\n complex;\n\n* otherwise, if either argument is a floating point number, the\n other is converted to floating point;\n\n* otherwise, if either argument is a long integer, the other is\n converted to long integer;\n\n* otherwise, both must be plain integers and no conversion is\n necessary.\n\nSome additional rules apply for certain operators (e.g., a string left\nargument to the \'%\' operator). Extensions can define their own\ncoercions.\n', + 'customization': u'\nBasic customization\n*******************\n\nobject.__new__(cls[, ...])\n\n Called to create a new instance of class *cls*. "__new__()" is a\n static method (special-cased so you need not declare it as such)\n that takes the class of which an instance was requested as its\n first argument. The remaining arguments are those passed to the\n object constructor expression (the call to the class). The return\n value of "__new__()" should be the new object instance (usually an\n instance of *cls*).\n\n Typical implementations create a new instance of the class by\n invoking the superclass\'s "__new__()" method using\n "super(currentclass, cls).__new__(cls[, ...])" with appropriate\n arguments and then modifying the newly-created instance as\n necessary before returning it.\n\n If "__new__()" returns an instance of *cls*, then the new\n instance\'s "__init__()" method will be invoked like\n "__init__(self[, ...])", where *self* is the new instance and the\n remaining arguments are the same as were passed to "__new__()".\n\n If "__new__()" does not return an instance of *cls*, then the new\n instance\'s "__init__()" method will not be invoked.\n\n "__new__()" is intended mainly to allow subclasses of immutable\n types (like int, str, or tuple) to customize instance creation. It\n is also commonly overridden in custom metaclasses in order to\n customize class creation.\n\nobject.__init__(self[, ...])\n\n Called after the instance has been created (by "__new__()"), but\n before it is returned to the caller. The arguments are those\n passed to the class constructor expression. If a base class has an\n "__init__()" method, the derived class\'s "__init__()" method, if\n any, must explicitly call it to ensure proper initialization of the\n base class part of the instance; for example:\n "BaseClass.__init__(self, [args...])".\n\n Because "__new__()" and "__init__()" work together in constructing\n objects ("__new__()" to create it, and "__init__()" to customise\n it), no non-"None" value may be returned by "__init__()"; doing so\n will cause a "TypeError" to be raised at runtime.\n\nobject.__del__(self)\n\n Called when the instance is about to be destroyed. This is also\n called a destructor. If a base class has a "__del__()" method, the\n derived class\'s "__del__()" method, if any, must explicitly call it\n to ensure proper deletion of the base class part of the instance.\n Note that it is possible (though not recommended!) for the\n "__del__()" method to postpone destruction of the instance by\n creating a new reference to it. It may then be called at a later\n time when this new reference is deleted. It is not guaranteed that\n "__del__()" methods are called for objects that still exist when\n the interpreter exits.\n\n Note: "del x" doesn\'t directly call "x.__del__()" --- the former\n decrements the reference count for "x" by one, and the latter is\n only called when "x"\'s reference count reaches zero. Some common\n situations that may prevent the reference count of an object from\n going to zero include: circular references between objects (e.g.,\n a doubly-linked list or a tree data structure with parent and\n child pointers); a reference to the object on the stack frame of\n a function that caught an exception (the traceback stored in\n "sys.exc_traceback" keeps the stack frame alive); or a reference\n to the object on the stack frame that raised an unhandled\n exception in interactive mode (the traceback stored in\n "sys.last_traceback" keeps the stack frame alive). The first\n situation can only be remedied by explicitly breaking the cycles;\n the latter two situations can be resolved by storing "None" in\n "sys.exc_traceback" or "sys.last_traceback". Circular references\n which are garbage are detected when the option cycle detector is\n enabled (it\'s on by default), but can only be cleaned up if there\n are no Python-level "__del__()" methods involved. Refer to the\n documentation for the "gc" module for more information about how\n "__del__()" methods are handled by the cycle detector,\n particularly the description of the "garbage" value.\n\n Warning: Due to the precarious circumstances under which\n "__del__()" methods are invoked, exceptions that occur during\n their execution are ignored, and a warning is printed to\n "sys.stderr" instead. Also, when "__del__()" is invoked in\n response to a module being deleted (e.g., when execution of the\n program is done), other globals referenced by the "__del__()"\n method may already have been deleted or in the process of being\n torn down (e.g. the import machinery shutting down). For this\n reason, "__del__()" methods should do the absolute minimum needed\n to maintain external invariants. Starting with version 1.5,\n Python guarantees that globals whose name begins with a single\n underscore are deleted from their module before other globals are\n deleted; if no other references to such globals exist, this may\n help in assuring that imported modules are still available at the\n time when the "__del__()" method is called.\n\n See also the "-R" command-line option.\n\nobject.__repr__(self)\n\n Called by the "repr()" built-in function and by string conversions\n (reverse quotes) to compute the "official" string representation of\n an object. If at all possible, this should look like a valid\n Python expression that could be used to recreate an object with the\n same value (given an appropriate environment). If this is not\n possible, a string of the form "<...some useful description...>"\n should be returned. The return value must be a string object. If a\n class defines "__repr__()" but not "__str__()", then "__repr__()"\n is also used when an "informal" string representation of instances\n of that class is required.\n\n This is typically used for debugging, so it is important that the\n representation is information-rich and unambiguous.\n\nobject.__str__(self)\n\n Called by the "str()" built-in function and by the "print"\n statement to compute the "informal" string representation of an\n object. This differs from "__repr__()" in that it does not have to\n be a valid Python expression: a more convenient or concise\n representation may be used instead. The return value must be a\n string object.\n\nobject.__lt__(self, other)\nobject.__le__(self, other)\nobject.__eq__(self, other)\nobject.__ne__(self, other)\nobject.__gt__(self, other)\nobject.__ge__(self, other)\n\n New in version 2.1.\n\n These are the so-called "rich comparison" methods, and are called\n for comparison operators in preference to "__cmp__()" below. The\n correspondence between operator symbols and method names is as\n follows: "xy" call "x.__ne__(y)",\n "x>y" calls "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n\n A rich comparison method may return the singleton "NotImplemented"\n if it does not implement the operation for a given pair of\n arguments. By convention, "False" and "True" are returned for a\n successful comparison. However, these methods can return any value,\n so if the comparison operator is used in a Boolean context (e.g.,\n in the condition of an "if" statement), Python will call "bool()"\n on the value to determine if the result is true or false.\n\n There are no implied relationships among the comparison operators.\n The truth of "x==y" does not imply that "x!=y" is false.\n Accordingly, when defining "__eq__()", one should also define\n "__ne__()" so that the operators will behave as expected. See the\n paragraph on "__hash__()" for some important notes on creating\n *hashable* objects which support custom comparison operations and\n are usable as dictionary keys.\n\n There are no swapped-argument versions of these methods (to be used\n when the left argument does not support the operation but the right\n argument does); rather, "__lt__()" and "__gt__()" are each other\'s\n reflection, "__le__()" and "__ge__()" are each other\'s reflection,\n and "__eq__()" and "__ne__()" are their own reflection.\n\n Arguments to rich comparison methods are never coerced.\n\n To automatically generate ordering operations from a single root\n operation, see "functools.total_ordering()".\n\nobject.__cmp__(self, other)\n\n Called by comparison operations if rich comparison (see above) is\n not defined. Should return a negative integer if "self < other",\n zero if "self == other", a positive integer if "self > other". If\n no "__cmp__()", "__eq__()" or "__ne__()" operation is defined,\n class instances are compared by object identity ("address"). See\n also the description of "__hash__()" for some important notes on\n creating *hashable* objects which support custom comparison\n operations and are usable as dictionary keys. (Note: the\n restriction that exceptions are not propagated by "__cmp__()" has\n been removed since Python 1.5.)\n\nobject.__rcmp__(self, other)\n\n Changed in version 2.1: No longer supported.\n\nobject.__hash__(self)\n\n Called by built-in function "hash()" and for operations on members\n of hashed collections including "set", "frozenset", and "dict".\n "__hash__()" should return an integer. The only required property\n is that objects which compare equal have the same hash value; it is\n advised to somehow mix together (e.g. using exclusive or) the hash\n values for the components of the object that also play a part in\n comparison of objects.\n\n If a class does not define a "__cmp__()" or "__eq__()" method it\n should not define a "__hash__()" operation either; if it defines\n "__cmp__()" or "__eq__()" but not "__hash__()", its instances will\n not be usable in hashed collections. If a class defines mutable\n objects and implements a "__cmp__()" or "__eq__()" method, it\n should not implement "__hash__()", since hashable collection\n implementations require that a object\'s hash value is immutable (if\n the object\'s hash value changes, it will be in the wrong hash\n bucket).\n\n User-defined classes have "__cmp__()" and "__hash__()" methods by\n default; with them, all objects compare unequal (except with\n themselves) and "x.__hash__()" returns a result derived from\n "id(x)".\n\n Classes which inherit a "__hash__()" method from a parent class but\n change the meaning of "__cmp__()" or "__eq__()" such that the hash\n value returned is no longer appropriate (e.g. by switching to a\n value-based concept of equality instead of the default identity\n based equality) can explicitly flag themselves as being unhashable\n by setting "__hash__ = None" in the class definition. Doing so\n means that not only will instances of the class raise an\n appropriate "TypeError" when a program attempts to retrieve their\n hash value, but they will also be correctly identified as\n unhashable when checking "isinstance(obj, collections.Hashable)"\n (unlike classes which define their own "__hash__()" to explicitly\n raise "TypeError").\n\n Changed in version 2.5: "__hash__()" may now also return a long\n integer object; the 32-bit integer is then derived from the hash of\n that object.\n\n Changed in version 2.6: "__hash__" may now be set to "None" to\n explicitly flag instances of a class as unhashable.\n\nobject.__nonzero__(self)\n\n Called to implement truth value testing and the built-in operation\n "bool()"; should return "False" or "True", or their integer\n equivalents "0" or "1". When this method is not defined,\n "__len__()" is called, if it is defined, and the object is\n considered true if its result is nonzero. If a class defines\n neither "__len__()" nor "__nonzero__()", all its instances are\n considered true.\n\nobject.__unicode__(self)\n\n Called to implement "unicode()" built-in; should return a Unicode\n object. When this method is not defined, string conversion is\n attempted, and the result of string conversion is converted to\n Unicode using the system default encoding.\n', + 'debugger': u'\n"pdb" --- The Python Debugger\n*****************************\n\n**Source code:** Lib/pdb.py\n\n======================================================================\n\nThe module "pdb" defines an interactive source code debugger for\nPython programs. It supports setting (conditional) breakpoints and\nsingle stepping at the source line level, inspection of stack frames,\nsource code listing, and evaluation of arbitrary Python code in the\ncontext of any stack frame. It also supports post-mortem debugging\nand can be called under program control.\n\nThe debugger is extensible --- it is actually defined as the class\n"Pdb". This is currently undocumented but easily understood by reading\nthe source. The extension interface uses the modules "bdb" and "cmd".\n\nThe debugger\'s prompt is "(Pdb)". Typical usage to run a program under\ncontrol of the debugger is:\n\n >>> import pdb\n >>> import mymodule\n >>> pdb.run(\'mymodule.test()\')\n > (0)?()\n (Pdb) continue\n > (1)?()\n (Pdb) continue\n NameError: \'spam\'\n > (1)?()\n (Pdb)\n\n"pdb.py" can also be invoked as a script to debug other scripts. For\nexample:\n\n python -m pdb myscript.py\n\nWhen invoked as a script, pdb will automatically enter post-mortem\ndebugging if the program being debugged exits abnormally. After post-\nmortem debugging (or after normal exit of the program), pdb will\nrestart the program. Automatic restarting preserves pdb\'s state (such\nas breakpoints) and in most cases is more useful than quitting the\ndebugger upon program\'s exit.\n\nNew in version 2.4: Restarting post-mortem behavior added.\n\nThe typical usage to break into the debugger from a running program is\nto insert\n\n import pdb; pdb.set_trace()\n\nat the location you want to break into the debugger. You can then\nstep through the code following this statement, and continue running\nwithout the debugger using the "c" command.\n\nThe typical usage to inspect a crashed program is:\n\n >>> import pdb\n >>> import mymodule\n >>> mymodule.test()\n Traceback (most recent call last):\n File "", line 1, in ?\n File "./mymodule.py", line 4, in test\n test2()\n File "./mymodule.py", line 3, in test2\n print spam\n NameError: spam\n >>> pdb.pm()\n > ./mymodule.py(3)test2()\n -> print spam\n (Pdb)\n\nThe module defines the following functions; each enters the debugger\nin a slightly different way:\n\npdb.run(statement[, globals[, locals]])\n\n Execute the *statement* (given as a string) under debugger control.\n The debugger prompt appears before any code is executed; you can\n set breakpoints and type "continue", or you can step through the\n statement using "step" or "next" (all these commands are explained\n below). The optional *globals* and *locals* arguments specify the\n environment in which the code is executed; by default the\n dictionary of the module "__main__" is used. (See the explanation\n of the "exec" statement or the "eval()" built-in function.)\n\npdb.runeval(expression[, globals[, locals]])\n\n Evaluate the *expression* (given as a string) under debugger\n control. When "runeval()" returns, it returns the value of the\n expression. Otherwise this function is similar to "run()".\n\npdb.runcall(function[, argument, ...])\n\n Call the *function* (a function or method object, not a string)\n with the given arguments. When "runcall()" returns, it returns\n whatever the function call returned. The debugger prompt appears\n as soon as the function is entered.\n\npdb.set_trace()\n\n Enter the debugger at the calling stack frame. This is useful to\n hard-code a breakpoint at a given point in a program, even if the\n code is not otherwise being debugged (e.g. when an assertion\n fails).\n\npdb.post_mortem([traceback])\n\n Enter post-mortem debugging of the given *traceback* object. If no\n *traceback* is given, it uses the one of the exception that is\n currently being handled (an exception must be being handled if the\n default is to be used).\n\npdb.pm()\n\n Enter post-mortem debugging of the traceback found in\n "sys.last_traceback".\n\nThe "run*" functions and "set_trace()" are aliases for instantiating\nthe "Pdb" class and calling the method of the same name. If you want\nto access further features, you have to do this yourself:\n\nclass pdb.Pdb(completekey=\'tab\', stdin=None, stdout=None, skip=None)\n\n "Pdb" is the debugger class.\n\n The *completekey*, *stdin* and *stdout* arguments are passed to the\n underlying "cmd.Cmd" class; see the description there.\n\n The *skip* argument, if given, must be an iterable of glob-style\n module name patterns. The debugger will not step into frames that\n originate in a module that matches one of these patterns. [1]\n\n Example call to enable tracing with *skip*:\n\n import pdb; pdb.Pdb(skip=[\'django.*\']).set_trace()\n\n New in version 2.7: The *skip* argument.\n\n run(statement[, globals[, locals]])\n runeval(expression[, globals[, locals]])\n runcall(function[, argument, ...])\n set_trace()\n\n See the documentation for the functions explained above.\n', + 'del': u'\nThe "del" statement\n*******************\n\n del_stmt ::= "del" target_list\n\nDeletion is recursively defined very similar to the way assignment is\ndefined. Rather than spelling it out in full details, here are some\nhints.\n\nDeletion of a target list recursively deletes each target, from left\nto right.\n\nDeletion of a name removes the binding of that name from the local or\nglobal namespace, depending on whether the name occurs in a "global"\nstatement in the same code block. If the name is unbound, a\n"NameError" exception will be raised.\n\nIt is illegal to delete a name from the local namespace if it occurs\nas a free variable in a nested block.\n\nDeletion of attribute references, subscriptions and slicings is passed\nto the primary object involved; deletion of a slicing is in general\nequivalent to assignment of an empty slice of the right type (but even\nthis is determined by the sliced object).\n', + 'dict': u'\nDictionary displays\n*******************\n\nA dictionary display is a possibly empty series of key/datum pairs\nenclosed in curly braces:\n\n dict_display ::= "{" [key_datum_list | dict_comprehension] "}"\n key_datum_list ::= key_datum ("," key_datum)* [","]\n key_datum ::= expression ":" expression\n dict_comprehension ::= expression ":" expression comp_for\n\nA dictionary display yields a new dictionary object.\n\nIf a comma-separated sequence of key/datum pairs is given, they are\nevaluated from left to right to define the entries of the dictionary:\neach key object is used as a key into the dictionary to store the\ncorresponding datum. This means that you can specify the same key\nmultiple times in the key/datum list, and the final dictionary\'s value\nfor that key will be the last one given.\n\nA dict comprehension, in contrast to list and set comprehensions,\nneeds two expressions separated with a colon followed by the usual\n"for" and "if" clauses. When the comprehension is run, the resulting\nkey and value elements are inserted in the new dictionary in the order\nthey are produced.\n\nRestrictions on the types of the key values are listed earlier in\nsection The standard type hierarchy. (To summarize, the key type\nshould be *hashable*, which excludes all mutable objects.) Clashes\nbetween duplicate keys are not detected; the last datum (textually\nrightmost in the display) stored for a given key value prevails.\n', + 'dynamic-features': u'\nInteraction with dynamic features\n*********************************\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nIf the wild card form of import --- "import *" --- is used in a\nfunction and the function contains or is a nested block with free\nvariables, the compiler will raise a "SyntaxError".\n\nIf "exec" is used in a function and the function contains or is a\nnested block with free variables, the compiler will raise a\n"SyntaxError" unless the exec explicitly specifies the local namespace\nfor the "exec". (In other words, "exec obj" would be illegal, but\n"exec obj in ns" would be legal.)\n\nThe "eval()", "execfile()", and "input()" functions and the "exec"\nstatement do not have access to the full environment for resolving\nnames. Names may be resolved in the local and global namespaces of\nthe caller. Free variables are not resolved in the nearest enclosing\nnamespace, but in the global namespace. [1] The "exec" statement and\nthe "eval()" and "execfile()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n', + 'else': u'\nThe "if" statement\n******************\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section Boolean operations\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n', + 'exceptions': u'\nExceptions\n**********\n\nExceptions are a means of breaking out of the normal flow of control\nof a code block in order to handle errors or other exceptional\nconditions. An exception is *raised* at the point where the error is\ndetected; it may be *handled* by the surrounding code block or by any\ncode block that directly or indirectly invoked the code block where\nthe error occurred.\n\nThe Python interpreter raises an exception when it detects a run-time\nerror (such as division by zero). A Python program can also\nexplicitly raise an exception with the "raise" statement. Exception\nhandlers are specified with the "try" ... "except" statement. The\n"finally" clause of such a statement can be used to specify cleanup\ncode which does not handle the exception, but is executed whether an\nexception occurred or not in the preceding code.\n\nPython uses the "termination" model of error handling: an exception\nhandler can find out what happened and continue execution at an outer\nlevel, but it cannot repair the cause of the error and retry the\nfailing operation (except by re-entering the offending piece of code\nfrom the top).\n\nWhen an exception is not handled at all, the interpreter terminates\nexecution of the program, or returns to its interactive main loop. In\neither case, it prints a stack backtrace, except when the exception is\n"SystemExit".\n\nExceptions are identified by class instances. The "except" clause is\nselected depending on the class of the instance: it must reference the\nclass of the instance or a base class thereof. The instance can be\nreceived by the handler and can carry additional information about the\nexceptional condition.\n\nExceptions can also be identified by strings, in which case the\n"except" clause is selected by object identity. An arbitrary value\ncan be raised along with the identifying string which can be passed to\nthe handler.\n\nNote: Messages to exceptions are not part of the Python API. Their\n contents may change from one version of Python to the next without\n warning and should not be relied on by code which will run under\n multiple versions of the interpreter.\n\nSee also the description of the "try" statement in section The try\nstatement and "raise" statement in section The raise statement.\n\n-[ Footnotes ]-\n\n[1] This limitation occurs because the code that is executed by\n these operations is not available at the time the module is\n compiled.\n', + 'exec': u'\nThe "exec" statement\n********************\n\n exec_stmt ::= "exec" or_expr ["in" expression ["," expression]]\n\nThis statement supports dynamic execution of Python code. The first\nexpression should evaluate to either a Unicode string, a *Latin-1*\nencoded string, an open file object, a code object, or a tuple. If it\nis a string, the string is parsed as a suite of Python statements\nwhich is then executed (unless a syntax error occurs). [1] If it is an\nopen file, the file is parsed until EOF and executed. If it is a code\nobject, it is simply executed. For the interpretation of a tuple, see\nbelow. In all cases, the code that\'s executed is expected to be valid\nas file input (see section File input). Be aware that the "return"\nand "yield" statements may not be used outside of function definitions\neven within the context of code passed to the "exec" statement.\n\nIn all cases, if the optional parts are omitted, the code is executed\nin the current scope. If only the first expression after "in" is\nspecified, it should be a dictionary, which will be used for both the\nglobal and the local variables. If two expressions are given, they\nare used for the global and local variables, respectively. If\nprovided, *locals* can be any mapping object. Remember that at module\nlevel, globals and locals are the same dictionary. If two separate\nobjects are given as *globals* and *locals*, the code will be executed\nas if it were embedded in a class definition.\n\nThe first expression may also be a tuple of length 2 or 3. In this\ncase, the optional parts must be omitted. The form "exec(expr,\nglobals)" is equivalent to "exec expr in globals", while the form\n"exec(expr, globals, locals)" is equivalent to "exec expr in globals,\nlocals". The tuple form of "exec" provides compatibility with Python\n3, where "exec" is a function rather than a statement.\n\nChanged in version 2.4: Formerly, *locals* was required to be a\ndictionary.\n\nAs a side effect, an implementation may insert additional keys into\nthe dictionaries given besides those corresponding to variable names\nset by the executed code. For example, the current implementation may\nadd a reference to the dictionary of the built-in module "__builtin__"\nunder the key "__builtins__" (!).\n\n**Programmer\'s hints:** dynamic evaluation of expressions is supported\nby the built-in function "eval()". The built-in functions "globals()"\nand "locals()" return the current global and local dictionary,\nrespectively, which may be useful to pass around for use by "exec".\n\n-[ Footnotes ]-\n\n[1] Note that the parser only accepts the Unix-style end of line\n convention. If you are reading the code from a file, make sure to\n use *universal newlines* mode to convert Windows or Mac-style\n newlines.\n', + 'execmodel': u'\nExecution model\n***************\n\n\nNaming and binding\n==================\n\n*Names* refer to objects. Names are introduced by name binding\noperations. Each occurrence of a name in the program text refers to\nthe *binding* of that name established in the innermost function block\ncontaining the use.\n\nA *block* is a piece of Python program text that is executed as a\nunit. The following are blocks: a module, a function body, and a class\ndefinition. Each command typed interactively is a block. A script\nfile (a file given as standard input to the interpreter or specified\non the interpreter command line the first argument) is a code block.\nA script command (a command specified on the interpreter command line\nwith the \'**-c**\' option) is a code block. The file read by the\nbuilt-in function "execfile()" is a code block. The string argument\npassed to the built-in function "eval()" and to the "exec" statement\nis a code block. The expression read and evaluated by the built-in\nfunction "input()" is a code block.\n\nA code block is executed in an *execution frame*. A frame contains\nsome administrative information (used for debugging) and determines\nwhere and how execution continues after the code block\'s execution has\ncompleted.\n\nA *scope* defines the visibility of a name within a block. If a local\nvariable is defined in a block, its scope includes that block. If the\ndefinition occurs in a function block, the scope extends to any blocks\ncontained within the defining one, unless a contained block introduces\na different binding for the name. The scope of names defined in a\nclass block is limited to the class block; it does not extend to the\ncode blocks of methods -- this includes generator expressions since\nthey are implemented using a function scope. This means that the\nfollowing will fail:\n\n class A:\n a = 42\n b = list(a + i for i in range(10))\n\nWhen a name is used in a code block, it is resolved using the nearest\nenclosing scope. The set of all such scopes visible to a code block\nis called the block\'s *environment*.\n\nIf a name is bound in a block, it is a local variable of that block.\nIf a name is bound at the module level, it is a global variable. (The\nvariables of the module code block are local and global.) If a\nvariable is used in a code block but not defined there, it is a *free\nvariable*.\n\nWhen a name is not found at all, a "NameError" exception is raised.\nIf the name refers to a local variable that has not been bound, a\n"UnboundLocalError" exception is raised. "UnboundLocalError" is a\nsubclass of "NameError".\n\nThe following constructs bind names: formal parameters to functions,\n"import" statements, class and function definitions (these bind the\nclass or function name in the defining block), and targets that are\nidentifiers if occurring in an assignment, "for" loop header, in the\nsecond position of an "except" clause header or after "as" in a "with"\nstatement. The "import" statement of the form "from ... import *"\nbinds all names defined in the imported module, except those beginning\nwith an underscore. This form may only be used at the module level.\n\nA target occurring in a "del" statement is also considered bound for\nthis purpose (though the actual semantics are to unbind the name). It\nis illegal to unbind a name that is referenced by an enclosing scope;\nthe compiler will report a "SyntaxError".\n\nEach assignment or import statement occurs within a block defined by a\nclass or function definition or at the module level (the top-level\ncode block).\n\nIf a name binding operation occurs anywhere within a code block, all\nuses of the name within the block are treated as references to the\ncurrent block. This can lead to errors when a name is used within a\nblock before it is bound. This rule is subtle. Python lacks\ndeclarations and allows name binding operations to occur anywhere\nwithin a code block. The local variables of a code block can be\ndetermined by scanning the entire text of the block for name binding\noperations.\n\nIf the global statement occurs within a block, all uses of the name\nspecified in the statement refer to the binding of that name in the\ntop-level namespace. Names are resolved in the top-level namespace by\nsearching the global namespace, i.e. the namespace of the module\ncontaining the code block, and the builtins namespace, the namespace\nof the module "__builtin__". The global namespace is searched first.\nIf the name is not found there, the builtins namespace is searched.\nThe global statement must precede all uses of the name.\n\nThe builtins namespace associated with the execution of a code block\nis actually found by looking up the name "__builtins__" in its global\nnamespace; this should be a dictionary or a module (in the latter case\nthe module\'s dictionary is used). By default, when in the "__main__"\nmodule, "__builtins__" is the built-in module "__builtin__" (note: no\n\'s\'); when in any other module, "__builtins__" is an alias for the\ndictionary of the "__builtin__" module itself. "__builtins__" can be\nset to a user-created dictionary to create a weak form of restricted\nexecution.\n\n**CPython implementation detail:** Users should not touch\n"__builtins__"; it is strictly an implementation detail. Users\nwanting to override values in the builtins namespace should "import"\nthe "__builtin__" (no \'s\') module and modify its attributes\nappropriately.\n\nThe namespace for a module is automatically created the first time a\nmodule is imported. The main module for a script is always called\n"__main__".\n\nThe "global" statement has the same scope as a name binding operation\nin the same block. If the nearest enclosing scope for a free variable\ncontains a global statement, the free variable is treated as a global.\n\nA class definition is an executable statement that may use and define\nnames. These references follow the normal rules for name resolution.\nThe namespace of the class definition becomes the attribute dictionary\nof the class. Names defined at the class scope are not visible in\nmethods.\n\n\nInteraction with dynamic features\n---------------------------------\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nIf the wild card form of import --- "import *" --- is used in a\nfunction and the function contains or is a nested block with free\nvariables, the compiler will raise a "SyntaxError".\n\nIf "exec" is used in a function and the function contains or is a\nnested block with free variables, the compiler will raise a\n"SyntaxError" unless the exec explicitly specifies the local namespace\nfor the "exec". (In other words, "exec obj" would be illegal, but\n"exec obj in ns" would be legal.)\n\nThe "eval()", "execfile()", and "input()" functions and the "exec"\nstatement do not have access to the full environment for resolving\nnames. Names may be resolved in the local and global namespaces of\nthe caller. Free variables are not resolved in the nearest enclosing\nnamespace, but in the global namespace. [1] The "exec" statement and\nthe "eval()" and "execfile()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n\n\nExceptions\n==========\n\nExceptions are a means of breaking out of the normal flow of control\nof a code block in order to handle errors or other exceptional\nconditions. An exception is *raised* at the point where the error is\ndetected; it may be *handled* by the surrounding code block or by any\ncode block that directly or indirectly invoked the code block where\nthe error occurred.\n\nThe Python interpreter raises an exception when it detects a run-time\nerror (such as division by zero). A Python program can also\nexplicitly raise an exception with the "raise" statement. Exception\nhandlers are specified with the "try" ... "except" statement. The\n"finally" clause of such a statement can be used to specify cleanup\ncode which does not handle the exception, but is executed whether an\nexception occurred or not in the preceding code.\n\nPython uses the "termination" model of error handling: an exception\nhandler can find out what happened and continue execution at an outer\nlevel, but it cannot repair the cause of the error and retry the\nfailing operation (except by re-entering the offending piece of code\nfrom the top).\n\nWhen an exception is not handled at all, the interpreter terminates\nexecution of the program, or returns to its interactive main loop. In\neither case, it prints a stack backtrace, except when the exception is\n"SystemExit".\n\nExceptions are identified by class instances. The "except" clause is\nselected depending on the class of the instance: it must reference the\nclass of the instance or a base class thereof. The instance can be\nreceived by the handler and can carry additional information about the\nexceptional condition.\n\nExceptions can also be identified by strings, in which case the\n"except" clause is selected by object identity. An arbitrary value\ncan be raised along with the identifying string which can be passed to\nthe handler.\n\nNote: Messages to exceptions are not part of the Python API. Their\n contents may change from one version of Python to the next without\n warning and should not be relied on by code which will run under\n multiple versions of the interpreter.\n\nSee also the description of the "try" statement in section The try\nstatement and "raise" statement in section The raise statement.\n\n-[ Footnotes ]-\n\n[1] This limitation occurs because the code that is executed by\n these operations is not available at the time the module is\n compiled.\n', + 'exprlists': u'\nExpression lists\n****************\n\n expression_list ::= expression ( "," expression )* [","]\n\nAn expression list containing at least one comma yields a tuple. The\nlength of the tuple is the number of expressions in the list. The\nexpressions are evaluated from left to right.\n\nThe trailing comma is required only to create a single tuple (a.k.a. a\n*singleton*); it is optional in all other cases. A single expression\nwithout a trailing comma doesn\'t create a tuple, but rather yields the\nvalue of that expression. (To create an empty tuple, use an empty pair\nof parentheses: "()".)\n', + 'floating': u'\nFloating point literals\n***********************\n\nFloating point literals are described by the following lexical\ndefinitions:\n\n floatnumber ::= pointfloat | exponentfloat\n pointfloat ::= [intpart] fraction | intpart "."\n exponentfloat ::= (intpart | pointfloat) exponent\n intpart ::= digit+\n fraction ::= "." digit+\n exponent ::= ("e" | "E") ["+" | "-"] digit+\n\nNote that the integer and exponent parts of floating point numbers can\nlook like octal integers, but are interpreted using radix 10. For\nexample, "077e010" is legal, and denotes the same number as "77e10".\nThe allowed range of floating point literals is implementation-\ndependent. Some examples of floating point literals:\n\n 3.14 10. .001 1e100 3.14e-10 0e0\n\nNote that numeric literals do not include a sign; a phrase like "-1"\nis actually an expression composed of the unary operator "-" and the\nliteral "1".\n', + 'for': u'\nThe "for" statement\n*******************\n\nThe "for" statement is used to iterate over the elements of a sequence\n(such as a string, tuple or list) or other iterable object:\n\n for_stmt ::= "for" target_list "in" expression_list ":" suite\n ["else" ":" suite]\n\nThe expression list is evaluated once; it should yield an iterable\nobject. An iterator is created for the result of the\n"expression_list". The suite is then executed once for each item\nprovided by the iterator, in the order of ascending indices. Each\nitem in turn is assigned to the target list using the standard rules\nfor assignments, and then the suite is executed. When the items are\nexhausted (which is immediately when the sequence is empty), the suite\nin the "else" clause, if present, is executed, and the loop\nterminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and continues\nwith the next item, or with the "else" clause if there was no next\nitem.\n\nThe suite may assign to the variable(s) in the target list; this does\nnot affect the next item assigned to it.\n\nThe target list is not deleted when the loop is finished, but if the\nsequence is empty, it will not have been assigned to at all by the\nloop. Hint: the built-in function "range()" returns a sequence of\nintegers suitable to emulate the effect of Pascal\'s "for i := a to b\ndo"; e.g., "range(3)" returns the list "[0, 1, 2]".\n\nNote: There is a subtlety when the sequence is being modified by the\n loop (this can only occur for mutable sequences, i.e. lists). An\n internal counter is used to keep track of which item is used next,\n and this is incremented on each iteration. When this counter has\n reached the length of the sequence the loop terminates. This means\n that if the suite deletes the current (or a previous) item from the\n sequence, the next item will be skipped (since it gets the index of\n the current item which has already been treated). Likewise, if the\n suite inserts an item in the sequence before the current item, the\n current item will be treated again the next time through the loop.\n This can lead to nasty bugs that can be avoided by making a\n temporary copy using a slice of the whole sequence, e.g.,\n\n for x in a[:]:\n if x < 0: a.remove(x)\n', + 'formatstrings': u'\nFormat String Syntax\n********************\n\nThe "str.format()" method and the "Formatter" class share the same\nsyntax for format strings (although in the case of "Formatter",\nsubclasses can define their own format string syntax).\n\nFormat strings contain "replacement fields" surrounded by curly braces\n"{}". Anything that is not contained in braces is considered literal\ntext, which is copied unchanged to the output. If you need to include\na brace character in the literal text, it can be escaped by doubling:\n"{{" and "}}".\n\nThe grammar for a replacement field is as follows:\n\n replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}"\n field_name ::= arg_name ("." attribute_name | "[" element_index "]")*\n arg_name ::= [identifier | integer]\n attribute_name ::= identifier\n element_index ::= integer | index_string\n index_string ::= +\n conversion ::= "r" | "s"\n format_spec ::= \n\nIn less formal terms, the replacement field can start with a\n*field_name* that specifies the object whose value is to be formatted\nand inserted into the output instead of the replacement field. The\n*field_name* is optionally followed by a *conversion* field, which is\npreceded by an exclamation point "\'!\'", and a *format_spec*, which is\npreceded by a colon "\':\'". These specify a non-default format for the\nreplacement value.\n\nSee also the Format Specification Mini-Language section.\n\nThe *field_name* itself begins with an *arg_name* that is either a\nnumber or a keyword. If it\'s a number, it refers to a positional\nargument, and if it\'s a keyword, it refers to a named keyword\nargument. If the numerical arg_names in a format string are 0, 1, 2,\n... in sequence, they can all be omitted (not just some) and the\nnumbers 0, 1, 2, ... will be automatically inserted in that order.\nBecause *arg_name* is not quote-delimited, it is not possible to\nspecify arbitrary dictionary keys (e.g., the strings "\'10\'" or\n"\':-]\'") within a format string. The *arg_name* can be followed by any\nnumber of index or attribute expressions. An expression of the form\n"\'.name\'" selects the named attribute using "getattr()", while an\nexpression of the form "\'[index]\'" does an index lookup using\n"__getitem__()".\n\nChanged in version 2.7: The positional argument specifiers can be\nomitted, so "\'{} {}\'" is equivalent to "\'{0} {1}\'".\n\nSome simple format string examples:\n\n "First, thou shalt count to {0}" # References first positional argument\n "Bring me a {}" # Implicitly references the first positional argument\n "From {} to {}" # Same as "From {0} to {1}"\n "My quest is {name}" # References keyword argument \'name\'\n "Weight in tons {0.weight}" # \'weight\' attribute of first positional arg\n "Units destroyed: {players[0]}" # First element of keyword argument \'players\'.\n\nThe *conversion* field causes a type coercion before formatting.\nNormally, the job of formatting a value is done by the "__format__()"\nmethod of the value itself. However, in some cases it is desirable to\nforce a type to be formatted as a string, overriding its own\ndefinition of formatting. By converting the value to a string before\ncalling "__format__()", the normal formatting logic is bypassed.\n\nTwo conversion flags are currently supported: "\'!s\'" which calls\n"str()" on the value, and "\'!r\'" which calls "repr()".\n\nSome examples:\n\n "Harold\'s a clever {0!s}" # Calls str() on the argument first\n "Bring out the holy {name!r}" # Calls repr() on the argument first\n\nThe *format_spec* field contains a specification of how the value\nshould be presented, including such details as field width, alignment,\npadding, decimal precision and so on. Each value type can define its\nown "formatting mini-language" or interpretation of the *format_spec*.\n\nMost built-in types support a common formatting mini-language, which\nis described in the next section.\n\nA *format_spec* field can also include nested replacement fields\nwithin it. These nested replacement fields can contain only a field\nname; conversion flags and format specifications are not allowed. The\nreplacement fields within the format_spec are substituted before the\n*format_spec* string is interpreted. This allows the formatting of a\nvalue to be dynamically specified.\n\nSee the Format examples section for some examples.\n\n\nFormat Specification Mini-Language\n==================================\n\n"Format specifications" are used within replacement fields contained\nwithin a format string to define how individual values are presented\n(see Format String Syntax). They can also be passed directly to the\nbuilt-in "format()" function. Each formattable type may define how\nthe format specification is to be interpreted.\n\nMost built-in types implement the following options for format\nspecifications, although some of the formatting options are only\nsupported by the numeric types.\n\nA general convention is that an empty format string ("""") produces\nthe same result as if you had called "str()" on the value. A non-empty\nformat string typically modifies the result.\n\nThe general form of a *standard format specifier* is:\n\n format_spec ::= [[fill]align][sign][#][0][width][,][.precision][type]\n fill ::= \n align ::= "<" | ">" | "=" | "^"\n sign ::= "+" | "-" | " "\n width ::= integer\n precision ::= integer\n type ::= "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%"\n\nIf a valid *align* value is specified, it can be preceded by a *fill*\ncharacter that can be any character and defaults to a space if\nomitted. Note that it is not possible to use "{" and "}" as *fill*\nchar while using the "str.format()" method; this limitation however\ndoesn\'t affect the "format()" function.\n\nThe meaning of the various alignment options is as follows:\n\n +-----------+------------------------------------------------------------+\n | Option | Meaning |\n +===========+============================================================+\n | "\'<\'" | Forces the field to be left-aligned within the available |\n | | space (this is the default for most objects). |\n +-----------+------------------------------------------------------------+\n | "\'>\'" | Forces the field to be right-aligned within the available |\n | | space (this is the default for numbers). |\n +-----------+------------------------------------------------------------+\n | "\'=\'" | Forces the padding to be placed after the sign (if any) |\n | | but before the digits. This is used for printing fields |\n | | in the form \'+000000120\'. This alignment option is only |\n | | valid for numeric types. |\n +-----------+------------------------------------------------------------+\n | "\'^\'" | Forces the field to be centered within the available |\n | | space. |\n +-----------+------------------------------------------------------------+\n\nNote that unless a minimum field width is defined, the field width\nwill always be the same size as the data to fill it, so that the\nalignment option has no meaning in this case.\n\nThe *sign* option is only valid for number types, and can be one of\nthe following:\n\n +-----------+------------------------------------------------------------+\n | Option | Meaning |\n +===========+============================================================+\n | "\'+\'" | indicates that a sign should be used for both positive as |\n | | well as negative numbers. |\n +-----------+------------------------------------------------------------+\n | "\'-\'" | indicates that a sign should be used only for negative |\n | | numbers (this is the default behavior). |\n +-----------+------------------------------------------------------------+\n | space | indicates that a leading space should be used on positive |\n | | numbers, and a minus sign on negative numbers. |\n +-----------+------------------------------------------------------------+\n\nThe "\'#\'" option is only valid for integers, and only for binary,\noctal, or hexadecimal output. If present, it specifies that the\noutput will be prefixed by "\'0b\'", "\'0o\'", or "\'0x\'", respectively.\n\nThe "\',\'" option signals the use of a comma for a thousands separator.\nFor a locale aware separator, use the "\'n\'" integer presentation type\ninstead.\n\nChanged in version 2.7: Added the "\',\'" option (see also **PEP 378**).\n\n*width* is a decimal integer defining the minimum field width. If not\nspecified, then the field width will be determined by the content.\n\nPreceding the *width* field by a zero ("\'0\'") character enables sign-\naware zero-padding for numeric types. This is equivalent to a *fill*\ncharacter of "\'0\'" with an *alignment* type of "\'=\'".\n\nThe *precision* is a decimal number indicating how many digits should\nbe displayed after the decimal point for a floating point value\nformatted with "\'f\'" and "\'F\'", or before and after the decimal point\nfor a floating point value formatted with "\'g\'" or "\'G\'". For non-\nnumber types the field indicates the maximum field size - in other\nwords, how many characters will be used from the field content. The\n*precision* is not allowed for integer values.\n\nFinally, the *type* determines how the data should be presented.\n\nThe available string presentation types are:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'s\'" | String format. This is the default type for strings and |\n | | may be omitted. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'s\'". |\n +-----------+------------------------------------------------------------+\n\nThe available integer presentation types are:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'b\'" | Binary format. Outputs the number in base 2. |\n +-----------+------------------------------------------------------------+\n | "\'c\'" | Character. Converts the integer to the corresponding |\n | | unicode character before printing. |\n +-----------+------------------------------------------------------------+\n | "\'d\'" | Decimal Integer. Outputs the number in base 10. |\n +-----------+------------------------------------------------------------+\n | "\'o\'" | Octal format. Outputs the number in base 8. |\n +-----------+------------------------------------------------------------+\n | "\'x\'" | Hex format. Outputs the number in base 16, using lower- |\n | | case letters for the digits above 9. |\n +-----------+------------------------------------------------------------+\n | "\'X\'" | Hex format. Outputs the number in base 16, using upper- |\n | | case letters for the digits above 9. |\n +-----------+------------------------------------------------------------+\n | "\'n\'" | Number. This is the same as "\'d\'", except that it uses the |\n | | current locale setting to insert the appropriate number |\n | | separator characters. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'d\'". |\n +-----------+------------------------------------------------------------+\n\nIn addition to the above presentation types, integers can be formatted\nwith the floating point presentation types listed below (except "\'n\'"\nand None). When doing so, "float()" is used to convert the integer to\na floating point number before formatting.\n\nThe available presentation types for floating point and decimal values\nare:\n\n +-----------+------------------------------------------------------------+\n | Type | Meaning |\n +===========+============================================================+\n | "\'e\'" | Exponent notation. Prints the number in scientific |\n | | notation using the letter \'e\' to indicate the exponent. |\n | | The default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'E\'" | Exponent notation. Same as "\'e\'" except it uses an upper |\n | | case \'E\' as the separator character. |\n +-----------+------------------------------------------------------------+\n | "\'f\'" | Fixed point. Displays the number as a fixed-point number. |\n | | The default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'F\'" | Fixed point. Same as "\'f\'". |\n +-----------+------------------------------------------------------------+\n | "\'g\'" | General format. For a given precision "p >= 1", this |\n | | rounds the number to "p" significant digits and then |\n | | formats the result in either fixed-point format or in |\n | | scientific notation, depending on its magnitude. The |\n | | precise rules are as follows: suppose that the result |\n | | formatted with presentation type "\'e\'" and precision "p-1" |\n | | would have exponent "exp". Then if "-4 <= exp < p", the |\n | | number is formatted with presentation type "\'f\'" and |\n | | precision "p-1-exp". Otherwise, the number is formatted |\n | | with presentation type "\'e\'" and precision "p-1". In both |\n | | cases insignificant trailing zeros are removed from the |\n | | significand, and the decimal point is also removed if |\n | | there are no remaining digits following it. Positive and |\n | | negative infinity, positive and negative zero, and nans, |\n | | are formatted as "inf", "-inf", "0", "-0" and "nan" |\n | | respectively, regardless of the precision. A precision of |\n | | "0" is treated as equivalent to a precision of "1". The |\n | | default precision is "6". |\n +-----------+------------------------------------------------------------+\n | "\'G\'" | General format. Same as "\'g\'" except switches to "\'E\'" if |\n | | the number gets too large. The representations of infinity |\n | | and NaN are uppercased, too. |\n +-----------+------------------------------------------------------------+\n | "\'n\'" | Number. This is the same as "\'g\'", except that it uses the |\n | | current locale setting to insert the appropriate number |\n | | separator characters. |\n +-----------+------------------------------------------------------------+\n | "\'%\'" | Percentage. Multiplies the number by 100 and displays in |\n | | fixed ("\'f\'") format, followed by a percent sign. |\n +-----------+------------------------------------------------------------+\n | None | The same as "\'g\'". |\n +-----------+------------------------------------------------------------+\n\n\nFormat examples\n===============\n\nThis section contains examples of the new format syntax and comparison\nwith the old "%"-formatting.\n\nIn most of the cases the syntax is similar to the old "%"-formatting,\nwith the addition of the "{}" and with ":" used instead of "%". For\nexample, "\'%03.2f\'" can be translated to "\'{:03.2f}\'".\n\nThe new format syntax also supports new and different options, shown\nin the follow examples.\n\nAccessing arguments by position:\n\n >>> \'{0}, {1}, {2}\'.format(\'a\', \'b\', \'c\')\n \'a, b, c\'\n >>> \'{}, {}, {}\'.format(\'a\', \'b\', \'c\') # 2.7+ only\n \'a, b, c\'\n >>> \'{2}, {1}, {0}\'.format(\'a\', \'b\', \'c\')\n \'c, b, a\'\n >>> \'{2}, {1}, {0}\'.format(*\'abc\') # unpacking argument sequence\n \'c, b, a\'\n >>> \'{0}{1}{0}\'.format(\'abra\', \'cad\') # arguments\' indices can be repeated\n \'abracadabra\'\n\nAccessing arguments by name:\n\n >>> \'Coordinates: {latitude}, {longitude}\'.format(latitude=\'37.24N\', longitude=\'-115.81W\')\n \'Coordinates: 37.24N, -115.81W\'\n >>> coord = {\'latitude\': \'37.24N\', \'longitude\': \'-115.81W\'}\n >>> \'Coordinates: {latitude}, {longitude}\'.format(**coord)\n \'Coordinates: 37.24N, -115.81W\'\n\nAccessing arguments\' attributes:\n\n >>> c = 3-5j\n >>> (\'The complex number {0} is formed from the real part {0.real} \'\n ... \'and the imaginary part {0.imag}.\').format(c)\n \'The complex number (3-5j) is formed from the real part 3.0 and the imaginary part -5.0.\'\n >>> class Point(object):\n ... def __init__(self, x, y):\n ... self.x, self.y = x, y\n ... def __str__(self):\n ... return \'Point({self.x}, {self.y})\'.format(self=self)\n ...\n >>> str(Point(4, 2))\n \'Point(4, 2)\'\n\nAccessing arguments\' items:\n\n >>> coord = (3, 5)\n >>> \'X: {0[0]}; Y: {0[1]}\'.format(coord)\n \'X: 3; Y: 5\'\n\nReplacing "%s" and "%r":\n\n >>> "repr() shows quotes: {!r}; str() doesn\'t: {!s}".format(\'test1\', \'test2\')\n "repr() shows quotes: \'test1\'; str() doesn\'t: test2"\n\nAligning the text and specifying a width:\n\n >>> \'{:<30}\'.format(\'left aligned\')\n \'left aligned \'\n >>> \'{:>30}\'.format(\'right aligned\')\n \' right aligned\'\n >>> \'{:^30}\'.format(\'centered\')\n \' centered \'\n >>> \'{:*^30}\'.format(\'centered\') # use \'*\' as a fill char\n \'***********centered***********\'\n\nReplacing "%+f", "%-f", and "% f" and specifying a sign:\n\n >>> \'{:+f}; {:+f}\'.format(3.14, -3.14) # show it always\n \'+3.140000; -3.140000\'\n >>> \'{: f}; {: f}\'.format(3.14, -3.14) # show a space for positive numbers\n \' 3.140000; -3.140000\'\n >>> \'{:-f}; {:-f}\'.format(3.14, -3.14) # show only the minus -- same as \'{:f}; {:f}\'\n \'3.140000; -3.140000\'\n\nReplacing "%x" and "%o" and converting the value to different bases:\n\n >>> # format also supports binary numbers\n >>> "int: {0:d}; hex: {0:x}; oct: {0:o}; bin: {0:b}".format(42)\n \'int: 42; hex: 2a; oct: 52; bin: 101010\'\n >>> # with 0x, 0o, or 0b as prefix:\n >>> "int: {0:d}; hex: {0:#x}; oct: {0:#o}; bin: {0:#b}".format(42)\n \'int: 42; hex: 0x2a; oct: 0o52; bin: 0b101010\'\n\nUsing the comma as a thousands separator:\n\n >>> \'{:,}\'.format(1234567890)\n \'1,234,567,890\'\n\nExpressing a percentage:\n\n >>> points = 19.5\n >>> total = 22\n >>> \'Correct answers: {:.2%}\'.format(points/total)\n \'Correct answers: 88.64%\'\n\nUsing type-specific formatting:\n\n >>> import datetime\n >>> d = datetime.datetime(2010, 7, 4, 12, 15, 58)\n >>> \'{:%Y-%m-%d %H:%M:%S}\'.format(d)\n \'2010-07-04 12:15:58\'\n\nNesting arguments and more complex examples:\n\n >>> for align, text in zip(\'<^>\', [\'left\', \'center\', \'right\']):\n ... \'{0:{fill}{align}16}\'.format(text, fill=align, align=align)\n ...\n \'left<<<<<<<<<<<<\'\n \'^^^^^center^^^^^\'\n \'>>>>>>>>>>>right\'\n >>>\n >>> octets = [192, 168, 0, 1]\n >>> \'{:02X}{:02X}{:02X}{:02X}\'.format(*octets)\n \'C0A80001\'\n >>> int(_, 16)\n 3232235521\n >>>\n >>> width = 5\n >>> for num in range(5,12):\n ... for base in \'dXob\':\n ... print \'{0:{width}{base}}\'.format(num, base=base, width=width),\n ... print\n ...\n 5 5 5 101\n 6 6 6 110\n 7 7 7 111\n 8 8 10 1000\n 9 9 11 1001\n 10 A 12 1010\n 11 B 13 1011\n', + 'function': u'\nFunction definitions\n********************\n\nA function definition defines a user-defined function object (see\nsection The standard type hierarchy):\n\n decorated ::= decorators (classdef | funcdef)\n decorators ::= decorator+\n decorator ::= "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE\n funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite\n dotted_name ::= identifier ("." identifier)*\n parameter_list ::= (defparameter ",")*\n ( "*" identifier ["," "**" identifier]\n | "**" identifier\n | defparameter [","] )\n defparameter ::= parameter ["=" expression]\n sublist ::= parameter ("," parameter)* [","]\n parameter ::= identifier | "(" sublist ")"\n funcname ::= identifier\n\nA function definition is an executable statement. Its execution binds\nthe function name in the current local namespace to a function object\n(a wrapper around the executable code for the function). This\nfunction object contains a reference to the current global namespace\nas the global namespace to be used when the function is called.\n\nThe function definition does not execute the function body; this gets\nexecuted only when the function is called. [3]\n\nA function definition may be wrapped by one or more *decorator*\nexpressions. Decorator expressions are evaluated when the function is\ndefined, in the scope that contains the function definition. The\nresult must be a callable, which is invoked with the function object\nas the only argument. The returned value is bound to the function name\ninstead of the function object. Multiple decorators are applied in\nnested fashion. For example, the following code:\n\n @f1(arg)\n @f2\n def func(): pass\n\nis equivalent to:\n\n def func(): pass\n func = f1(arg)(f2(func))\n\nWhen one or more top-level *parameters* have the form *parameter* "="\n*expression*, the function is said to have "default parameter values."\nFor a parameter with a default value, the corresponding *argument* may\nbe omitted from a call, in which case the parameter\'s default value is\nsubstituted. If a parameter has a default value, all following\nparameters must also have a default value --- this is a syntactic\nrestriction that is not expressed by the grammar.\n\n**Default parameter values are evaluated when the function definition\nis executed.** This means that the expression is evaluated once, when\nthe function is defined, and that the same "pre-computed" value is\nused for each call. This is especially important to understand when a\ndefault parameter is a mutable object, such as a list or a dictionary:\nif the function modifies the object (e.g. by appending an item to a\nlist), the default value is in effect modified. This is generally not\nwhat was intended. A way around this is to use "None" as the\ndefault, and explicitly test for it in the body of the function, e.g.:\n\n def whats_on_the_telly(penguin=None):\n if penguin is None:\n penguin = []\n penguin.append("property of the zoo")\n return penguin\n\nFunction call semantics are described in more detail in section Calls.\nA function call always assigns values to all parameters mentioned in\nthe parameter list, either from position arguments, from keyword\narguments, or from default values. If the form ""*identifier"" is\npresent, it is initialized to a tuple receiving any excess positional\nparameters, defaulting to the empty tuple. If the form\n""**identifier"" is present, it is initialized to a new dictionary\nreceiving any excess keyword arguments, defaulting to a new empty\ndictionary.\n\nIt is also possible to create anonymous functions (functions not bound\nto a name), for immediate use in expressions. This uses lambda\nexpressions, described in section Lambdas. Note that the lambda\nexpression is merely a shorthand for a simplified function definition;\na function defined in a ""def"" statement can be passed around or\nassigned to another name just like a function defined by a lambda\nexpression. The ""def"" form is actually more powerful since it\nallows the execution of multiple statements.\n\n**Programmer\'s note:** Functions are first-class objects. A ""def""\nform executed inside a function definition defines a local function\nthat can be returned or passed around. Free variables used in the\nnested function can access the local variables of the function\ncontaining the def. See section Naming and binding for details.\n', + 'global': u'\nThe "global" statement\n**********************\n\n global_stmt ::= "global" identifier ("," identifier)*\n\nThe "global" statement is a declaration which holds for the entire\ncurrent code block. It means that the listed identifiers are to be\ninterpreted as globals. It would be impossible to assign to a global\nvariable without "global", although free variables may refer to\nglobals without being declared global.\n\nNames listed in a "global" statement must not be used in the same code\nblock textually preceding that "global" statement.\n\nNames listed in a "global" statement must not be defined as formal\nparameters or in a "for" loop control target, "class" definition,\nfunction definition, or "import" statement.\n\n**CPython implementation detail:** The current implementation does not\nenforce the latter two restrictions, but programs should not abuse\nthis freedom, as future implementations may enforce them or silently\nchange the meaning of the program.\n\n**Programmer\'s note:** the "global" is a directive to the parser. It\napplies only to code parsed at the same time as the "global"\nstatement. In particular, a "global" statement contained in an "exec"\nstatement does not affect the code block *containing* the "exec"\nstatement, and code contained in an "exec" statement is unaffected by\n"global" statements in the code containing the "exec" statement. The\nsame applies to the "eval()", "execfile()" and "compile()" functions.\n', + 'id-classes': u'\nReserved classes of identifiers\n*******************************\n\nCertain classes of identifiers (besides keywords) have special\nmeanings. These classes are identified by the patterns of leading and\ntrailing underscore characters:\n\n"_*"\n Not imported by "from module import *". The special identifier "_"\n is used in the interactive interpreter to store the result of the\n last evaluation; it is stored in the "__builtin__" module. When\n not in interactive mode, "_" has no special meaning and is not\n defined. See section The import statement.\n\n Note: The name "_" is often used in conjunction with\n internationalization; refer to the documentation for the\n "gettext" module for more information on this convention.\n\n"__*__"\n System-defined names. These names are defined by the interpreter\n and its implementation (including the standard library). Current\n system names are discussed in the Special method names section and\n elsewhere. More will likely be defined in future versions of\n Python. *Any* use of "__*__" names, in any context, that does not\n follow explicitly documented use, is subject to breakage without\n warning.\n\n"__*"\n Class-private names. Names in this category, when used within the\n context of a class definition, are re-written to use a mangled form\n to help avoid name clashes between "private" attributes of base and\n derived classes. See section Identifiers (Names).\n', + 'identifiers': u'\nIdentifiers and keywords\n************************\n\nIdentifiers (also referred to as *names*) are described by the\nfollowing lexical definitions:\n\n identifier ::= (letter|"_") (letter | digit | "_")*\n letter ::= lowercase | uppercase\n lowercase ::= "a"..."z"\n uppercase ::= "A"..."Z"\n digit ::= "0"..."9"\n\nIdentifiers are unlimited in length. Case is significant.\n\n\nKeywords\n========\n\nThe following identifiers are used as reserved words, or *keywords* of\nthe language, and cannot be used as ordinary identifiers. They must\nbe spelled exactly as written here:\n\n and del from not while\n as elif global or with\n assert else if pass yield\n break except import print\n class exec in raise\n continue finally is return\n def for lambda try\n\nChanged in version 2.4: "None" became a constant and is now recognized\nby the compiler as a name for the built-in object "None". Although it\nis not a keyword, you cannot assign a different object to it.\n\nChanged in version 2.5: Using "as" and "with" as identifiers triggers\na warning. To use them as keywords, enable the "with_statement"\nfuture feature .\n\nChanged in version 2.6: "as" and "with" are full keywords.\n\n\nReserved classes of identifiers\n===============================\n\nCertain classes of identifiers (besides keywords) have special\nmeanings. These classes are identified by the patterns of leading and\ntrailing underscore characters:\n\n"_*"\n Not imported by "from module import *". The special identifier "_"\n is used in the interactive interpreter to store the result of the\n last evaluation; it is stored in the "__builtin__" module. When\n not in interactive mode, "_" has no special meaning and is not\n defined. See section The import statement.\n\n Note: The name "_" is often used in conjunction with\n internationalization; refer to the documentation for the\n "gettext" module for more information on this convention.\n\n"__*__"\n System-defined names. These names are defined by the interpreter\n and its implementation (including the standard library). Current\n system names are discussed in the Special method names section and\n elsewhere. More will likely be defined in future versions of\n Python. *Any* use of "__*__" names, in any context, that does not\n follow explicitly documented use, is subject to breakage without\n warning.\n\n"__*"\n Class-private names. Names in this category, when used within the\n context of a class definition, are re-written to use a mangled form\n to help avoid name clashes between "private" attributes of base and\n derived classes. See section Identifiers (Names).\n', + 'if': u'\nThe "if" statement\n******************\n\nThe "if" statement is used for conditional execution:\n\n if_stmt ::= "if" expression ":" suite\n ( "elif" expression ":" suite )*\n ["else" ":" suite]\n\nIt selects exactly one of the suites by evaluating the expressions one\nby one until one is found to be true (see section Boolean operations\nfor the definition of true and false); then that suite is executed\n(and no other part of the "if" statement is executed or evaluated).\nIf all expressions are false, the suite of the "else" clause, if\npresent, is executed.\n', + 'imaginary': u'\nImaginary literals\n******************\n\nImaginary literals are described by the following lexical definitions:\n\n imagnumber ::= (floatnumber | intpart) ("j" | "J")\n\nAn imaginary literal yields a complex number with a real part of 0.0.\nComplex numbers are represented as a pair of floating point numbers\nand have the same restrictions on their range. To create a complex\nnumber with a nonzero real part, add a floating point number to it,\ne.g., "(3+4j)". Some examples of imaginary literals:\n\n 3.14j 10.j 10j .001j 1e100j 3.14e-10j\n', + 'import': u'\nThe "import" statement\n**********************\n\n import_stmt ::= "import" module ["as" name] ( "," module ["as" name] )*\n | "from" relative_module "import" identifier ["as" name]\n ( "," identifier ["as" name] )*\n | "from" relative_module "import" "(" identifier ["as" name]\n ( "," identifier ["as" name] )* [","] ")"\n | "from" module "import" "*"\n module ::= (identifier ".")* identifier\n relative_module ::= "."* module | "."+\n name ::= identifier\n\nImport statements are executed in two steps: (1) find a module, and\ninitialize it if necessary; (2) define a name or names in the local\nnamespace (of the scope where the "import" statement occurs). The\nstatement comes in two forms differing on whether it uses the "from"\nkeyword. The first form (without "from") repeats these steps for each\nidentifier in the list. The form with "from" performs step (1) once,\nand then performs step (2) repeatedly.\n\nTo understand how step (1) occurs, one must first understand how\nPython handles hierarchical naming of modules. To help organize\nmodules and provide a hierarchy in naming, Python has a concept of\npackages. A package can contain other packages and modules while\nmodules cannot contain other modules or packages. From a file system\nperspective, packages are directories and modules are files.\n\nOnce the name of the module is known (unless otherwise specified, the\nterm "module" will refer to both packages and modules), searching for\nthe module or package can begin. The first place checked is\n"sys.modules", the cache of all modules that have been imported\npreviously. If the module is found there then it is used in step (2)\nof import.\n\nIf the module is not found in the cache, then "sys.meta_path" is\nsearched (the specification for "sys.meta_path" can be found in **PEP\n302**). The object is a list of *finder* objects which are queried in\norder as to whether they know how to load the module by calling their\n"find_module()" method with the name of the module. If the module\nhappens to be contained within a package (as denoted by the existence\nof a dot in the name), then a second argument to "find_module()" is\ngiven as the value of the "__path__" attribute from the parent package\n(everything up to the last dot in the name of the module being\nimported). If a finder can find the module it returns a *loader*\n(discussed later) or returns "None".\n\nIf none of the finders on "sys.meta_path" are able to find the module\nthen some implicitly defined finders are queried. Implementations of\nPython vary in what implicit meta path finders are defined. The one\nthey all do define, though, is one that handles "sys.path_hooks",\n"sys.path_importer_cache", and "sys.path".\n\nThe implicit finder searches for the requested module in the "paths"\nspecified in one of two places ("paths" do not have to be file system\npaths). If the module being imported is supposed to be contained\nwithin a package then the second argument passed to "find_module()",\n"__path__" on the parent package, is used as the source of paths. If\nthe module is not contained in a package then "sys.path" is used as\nthe source of paths.\n\nOnce the source of paths is chosen it is iterated over to find a\nfinder that can handle that path. The dict at\n"sys.path_importer_cache" caches finders for paths and is checked for\na finder. If the path does not have a finder cached then\n"sys.path_hooks" is searched by calling each object in the list with a\nsingle argument of the path, returning a finder or raises\n"ImportError". If a finder is returned then it is cached in\n"sys.path_importer_cache" and then used for that path entry. If no\nfinder can be found but the path exists then a value of "None" is\nstored in "sys.path_importer_cache" to signify that an implicit, file-\nbased finder that handles modules stored as individual files should be\nused for that path. If the path does not exist then a finder which\nalways returns "None" is placed in the cache for the path.\n\nIf no finder can find the module then "ImportError" is raised.\nOtherwise some finder returned a loader whose "load_module()" method\nis called with the name of the module to load (see **PEP 302** for the\noriginal definition of loaders). A loader has several responsibilities\nto perform on a module it loads. First, if the module already exists\nin "sys.modules" (a possibility if the loader is called outside of the\nimport machinery) then it is to use that module for initialization and\nnot a new module. But if the module does not exist in "sys.modules"\nthen it is to be added to that dict before initialization begins. If\nan error occurs during loading of the module and it was added to\n"sys.modules" it is to be removed from the dict. If an error occurs\nbut the module was already in "sys.modules" it is left in the dict.\n\nThe loader must set several attributes on the module. "__name__" is to\nbe set to the name of the module. "__file__" is to be the "path" to\nthe file unless the module is built-in (and thus listed in\n"sys.builtin_module_names") in which case the attribute is not set. If\nwhat is being imported is a package then "__path__" is to be set to a\nlist of paths to be searched when looking for modules and packages\ncontained within the package being imported. "__package__" is optional\nbut should be set to the name of package that contains the module or\npackage (the empty string is used for module not contained in a\npackage). "__loader__" is also optional but should be set to the\nloader object that is loading the module.\n\nIf an error occurs during loading then the loader raises "ImportError"\nif some other exception is not already being propagated. Otherwise the\nloader returns the module that was loaded and initialized.\n\nWhen step (1) finishes without raising an exception, step (2) can\nbegin.\n\nThe first form of "import" statement binds the module name in the\nlocal namespace to the module object, and then goes on to import the\nnext identifier, if any. If the module name is followed by "as", the\nname following "as" is used as the local name for the module.\n\nThe "from" form does not bind the module name: it goes through the\nlist of identifiers, looks each one of them up in the module found in\nstep (1), and binds the name in the local namespace to the object thus\nfound. As with the first form of "import", an alternate local name\ncan be supplied by specifying ""as" localname". If a name is not\nfound, "ImportError" is raised. If the list of identifiers is\nreplaced by a star ("\'*\'"), all public names defined in the module are\nbound in the local namespace of the "import" statement..\n\nThe *public names* defined by a module are determined by checking the\nmodule\'s namespace for a variable named "__all__"; if defined, it must\nbe a sequence of strings which are names defined or imported by that\nmodule. The names given in "__all__" are all considered public and\nare required to exist. If "__all__" is not defined, the set of public\nnames includes all names found in the module\'s namespace which do not\nbegin with an underscore character ("\'_\'"). "__all__" should contain\nthe entire public API. It is intended to avoid accidentally exporting\nitems that are not part of the API (such as library modules which were\nimported and used within the module).\n\nThe "from" form with "*" may only occur in a module scope. If the\nwild card form of import --- "import *" --- is used in a function and\nthe function contains or is a nested block with free variables, the\ncompiler will raise a "SyntaxError".\n\nWhen specifying what module to import you do not have to specify the\nabsolute name of the module. When a module or package is contained\nwithin another package it is possible to make a relative import within\nthe same top package without having to mention the package name. By\nusing leading dots in the specified module or package after "from" you\ncan specify how high to traverse up the current package hierarchy\nwithout specifying exact names. One leading dot means the current\npackage where the module making the import exists. Two dots means up\none package level. Three dots is up two levels, etc. So if you execute\n"from . import mod" from a module in the "pkg" package then you will\nend up importing "pkg.mod". If you execute "from ..subpkg2 import mod"\nfrom within "pkg.subpkg1" you will import "pkg.subpkg2.mod". The\nspecification for relative imports is contained within **PEP 328**.\n\n"importlib.import_module()" is provided to support applications that\ndetermine which modules need to be loaded dynamically.\n\n\nFuture statements\n=================\n\nA *future statement* is a directive to the compiler that a particular\nmodule should be compiled using syntax or semantics that will be\navailable in a specified future release of Python. The future\nstatement is intended to ease migration to future versions of Python\nthat introduce incompatible changes to the language. It allows use of\nthe new features on a per-module basis before the release in which the\nfeature becomes standard.\n\n future_statement ::= "from" "__future__" "import" feature ["as" name]\n ("," feature ["as" name])*\n | "from" "__future__" "import" "(" feature ["as" name]\n ("," feature ["as" name])* [","] ")"\n feature ::= identifier\n name ::= identifier\n\nA future statement must appear near the top of the module. The only\nlines that can appear before a future statement are:\n\n* the module docstring (if any),\n\n* comments,\n\n* blank lines, and\n\n* other future statements.\n\nThe features recognized by Python 2.6 are "unicode_literals",\n"print_function", "absolute_import", "division", "generators",\n"nested_scopes" and "with_statement". "generators", "with_statement",\n"nested_scopes" are redundant in Python version 2.6 and above because\nthey are always enabled.\n\nA future statement is recognized and treated specially at compile\ntime: Changes to the semantics of core constructs are often\nimplemented by generating different code. It may even be the case\nthat a new feature introduces new incompatible syntax (such as a new\nreserved word), in which case the compiler may need to parse the\nmodule differently. Such decisions cannot be pushed off until\nruntime.\n\nFor any given release, the compiler knows which feature names have\nbeen defined, and raises a compile-time error if a future statement\ncontains a feature not known to it.\n\nThe direct runtime semantics are the same as for any import statement:\nthere is a standard module "__future__", described later, and it will\nbe imported in the usual way at the time the future statement is\nexecuted.\n\nThe interesting runtime semantics depend on the specific feature\nenabled by the future statement.\n\nNote that there is nothing special about the statement:\n\n import __future__ [as name]\n\nThat is not a future statement; it\'s an ordinary import statement with\nno special semantics or syntax restrictions.\n\nCode compiled by an "exec" statement or calls to the built-in\nfunctions "compile()" and "execfile()" that occur in a module "M"\ncontaining a future statement will, by default, use the new syntax or\nsemantics associated with the future statement. This can, starting\nwith Python 2.2 be controlled by optional arguments to "compile()" ---\nsee the documentation of that function for details.\n\nA future statement typed at an interactive interpreter prompt will\ntake effect for the rest of the interpreter session. If an\ninterpreter is started with the "-i" option, is passed a script name\nto execute, and the script includes a future statement, it will be in\neffect in the interactive session started after the script is\nexecuted.\n\nSee also:\n\n **PEP 236** - Back to the __future__\n The original proposal for the __future__ mechanism.\n', + 'in': u'\nComparisons\n***********\n\nUnlike C, all comparison operations in Python have the same priority,\nwhich is lower than that of any arithmetic, shifting or bitwise\noperation. Also unlike C, expressions like "a < b < c" have the\ninterpretation that is conventional in mathematics:\n\n comparison ::= or_expr ( comp_operator or_expr )*\n comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "<>" | "!="\n | "is" ["not"] | ["not"] "in"\n\nComparisons yield boolean values: "True" or "False".\n\nComparisons can be chained arbitrarily, e.g., "x < y <= z" is\nequivalent to "x < y and y <= z", except that "y" is evaluated only\nonce (but in both cases "z" is not evaluated at all when "x < y" is\nfound to be false).\n\nFormally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*,\n*op2*, ..., *opN* are comparison operators, then "a op1 b op2 c ... y\nopN z" is equivalent to "a op1 b and b op2 c and ... y opN z", except\nthat each expression is evaluated at most once.\n\nNote that "a op1 b op2 c" doesn\'t imply any kind of comparison between\n*a* and *c*, so that, e.g., "x < y > z" is perfectly legal (though\nperhaps not pretty).\n\nThe forms "<>" and "!=" are equivalent; for consistency with C, "!="\nis preferred; where "!=" is mentioned below "<>" is also accepted.\nThe "<>" spelling is considered obsolescent.\n\nThe operators "<", ">", "==", ">=", "<=", and "!=" compare the values\nof two objects. The objects need not have the same type. If both are\nnumbers, they are converted to a common type. Otherwise, objects of\ndifferent types *always* compare unequal, and are ordered consistently\nbut arbitrarily. You can control comparison behavior of objects of\nnon-built-in types by defining a "__cmp__" method or rich comparison\nmethods like "__gt__", described in section Special method names.\n\n(This unusual definition of comparison was used to simplify the\ndefinition of operations like sorting and the "in" and "not in"\noperators. In the future, the comparison rules for objects of\ndifferent types are likely to change.)\n\nComparison of objects of the same type depends on the type:\n\n* Numbers are compared arithmetically.\n\n* Strings are compared lexicographically using the numeric\n equivalents (the result of the built-in function "ord()") of their\n characters. Unicode and 8-bit strings are fully interoperable in\n this behavior. [4]\n\n* Tuples and lists are compared lexicographically using comparison\n of corresponding elements. This means that to compare equal, each\n element must compare equal and the two sequences must be of the same\n type and have the same length.\n\n If not equal, the sequences are ordered the same as their first\n differing elements. For example, "cmp([1,2,x], [1,2,y])" returns\n the same as "cmp(x,y)". If the corresponding element does not\n exist, the shorter sequence is ordered first (for example, "[1,2] <\n [1,2,3]").\n\n* Mappings (dictionaries) compare equal if and only if their sorted\n (key, value) lists compare equal. [5] Outcomes other than equality\n are resolved consistently, but are not otherwise defined. [6]\n\n* Most other objects of built-in types compare unequal unless they\n are the same object; the choice whether one object is considered\n smaller or larger than another one is made arbitrarily but\n consistently within one execution of a program.\n\nThe operators "in" and "not in" test for collection membership. "x in\ns" evaluates to true if *x* is a member of the collection *s*, and\nfalse otherwise. "x not in s" returns the negation of "x in s". The\ncollection membership test has traditionally been bound to sequences;\nan object is a member of a collection if the collection is a sequence\nand contains an element equal to that object. However, it make sense\nfor many other object types to support membership tests without being\na sequence. In particular, dictionaries (for keys) and sets support\nmembership testing.\n\nFor the list and tuple types, "x in y" is true if and only if there\nexists an index *i* such that "x == y[i]" is true.\n\nFor the Unicode and string types, "x in y" is true if and only if *x*\nis a substring of *y*. An equivalent test is "y.find(x) != -1".\nNote, *x* and *y* need not be the same type; consequently, "u\'ab\' in\n\'abc\'" will return "True". Empty strings are always considered to be a\nsubstring of any other string, so """ in "abc"" will return "True".\n\nChanged in version 2.3: Previously, *x* was required to be a string of\nlength "1".\n\nFor user-defined classes which define the "__contains__()" method, "x\nin y" is true if and only if "y.__contains__(x)" is true.\n\nFor user-defined classes which do not define "__contains__()" but do\ndefine "__iter__()", "x in y" is true if some value "z" with "x == z"\nis produced while iterating over "y". If an exception is raised\nduring the iteration, it is as if "in" raised that exception.\n\nLastly, the old-style iteration protocol is tried: if a class defines\n"__getitem__()", "x in y" is true if and only if there is a non-\nnegative integer index *i* such that "x == y[i]", and all lower\ninteger indices do not raise "IndexError" exception. (If any other\nexception is raised, it is as if "in" raised that exception).\n\nThe operator "not in" is defined to have the inverse true value of\n"in".\n\nThe operators "is" and "is not" test for object identity: "x is y" is\ntrue if and only if *x* and *y* are the same object. "x is not y"\nyields the inverse truth value. [7]\n', + 'integers': u'\nInteger and long integer literals\n*********************************\n\nInteger and long integer literals are described by the following\nlexical definitions:\n\n longinteger ::= integer ("l" | "L")\n integer ::= decimalinteger | octinteger | hexinteger | bininteger\n decimalinteger ::= nonzerodigit digit* | "0"\n octinteger ::= "0" ("o" | "O") octdigit+ | "0" octdigit+\n hexinteger ::= "0" ("x" | "X") hexdigit+\n bininteger ::= "0" ("b" | "B") bindigit+\n nonzerodigit ::= "1"..."9"\n octdigit ::= "0"..."7"\n bindigit ::= "0" | "1"\n hexdigit ::= digit | "a"..."f" | "A"..."F"\n\nAlthough both lower case "\'l\'" and upper case "\'L\'" are allowed as\nsuffix for long integers, it is strongly recommended to always use\n"\'L\'", since the letter "\'l\'" looks too much like the digit "\'1\'".\n\nPlain integer literals that are above the largest representable plain\ninteger (e.g., 2147483647 when using 32-bit arithmetic) are accepted\nas if they were long integers instead. [1] There is no limit for long\ninteger literals apart from what can be stored in available memory.\n\nSome examples of plain integer literals (first row) and long integer\nliterals (second and third rows):\n\n 7 2147483647 0177\n 3L 79228162514264337593543950336L 0377L 0x100000000L\n 79228162514264337593543950336 0xdeadbeef\n', + 'lambda': u'\nLambdas\n*******\n\n lambda_expr ::= "lambda" [parameter_list]: expression\n old_lambda_expr ::= "lambda" [parameter_list]: old_expression\n\nLambda expressions (sometimes called lambda forms) have the same\nsyntactic position as expressions. They are a shorthand to create\nanonymous functions; the expression "lambda arguments: expression"\nyields a function object. The unnamed object behaves like a function\nobject defined with\n\n def name(arguments):\n return expression\n\nSee section Function definitions for the syntax of parameter lists.\nNote that functions created with lambda expressions cannot contain\nstatements.\n', + 'lists': u'\nList displays\n*************\n\nA list display is a possibly empty series of expressions enclosed in\nsquare brackets:\n\n list_display ::= "[" [expression_list | list_comprehension] "]"\n list_comprehension ::= expression list_for\n list_for ::= "for" target_list "in" old_expression_list [list_iter]\n old_expression_list ::= old_expression [("," old_expression)+ [","]]\n old_expression ::= or_test | old_lambda_expr\n list_iter ::= list_for | list_if\n list_if ::= "if" old_expression [list_iter]\n\nA list display yields a new list object. Its contents are specified\nby providing either a list of expressions or a list comprehension.\nWhen a comma-separated list of expressions is supplied, its elements\nare evaluated from left to right and placed into the list object in\nthat order. When a list comprehension is supplied, it consists of a\nsingle expression followed by at least one "for" clause and zero or\nmore "for" or "if" clauses. In this case, the elements of the new\nlist are those that would be produced by considering each of the "for"\nor "if" clauses a block, nesting from left to right, and evaluating\nthe expression to produce a list element each time the innermost block\nis reached [1].\n', + 'naming': u'\nNaming and binding\n******************\n\n*Names* refer to objects. Names are introduced by name binding\noperations. Each occurrence of a name in the program text refers to\nthe *binding* of that name established in the innermost function block\ncontaining the use.\n\nA *block* is a piece of Python program text that is executed as a\nunit. The following are blocks: a module, a function body, and a class\ndefinition. Each command typed interactively is a block. A script\nfile (a file given as standard input to the interpreter or specified\non the interpreter command line the first argument) is a code block.\nA script command (a command specified on the interpreter command line\nwith the \'**-c**\' option) is a code block. The file read by the\nbuilt-in function "execfile()" is a code block. The string argument\npassed to the built-in function "eval()" and to the "exec" statement\nis a code block. The expression read and evaluated by the built-in\nfunction "input()" is a code block.\n\nA code block is executed in an *execution frame*. A frame contains\nsome administrative information (used for debugging) and determines\nwhere and how execution continues after the code block\'s execution has\ncompleted.\n\nA *scope* defines the visibility of a name within a block. If a local\nvariable is defined in a block, its scope includes that block. If the\ndefinition occurs in a function block, the scope extends to any blocks\ncontained within the defining one, unless a contained block introduces\na different binding for the name. The scope of names defined in a\nclass block is limited to the class block; it does not extend to the\ncode blocks of methods -- this includes generator expressions since\nthey are implemented using a function scope. This means that the\nfollowing will fail:\n\n class A:\n a = 42\n b = list(a + i for i in range(10))\n\nWhen a name is used in a code block, it is resolved using the nearest\nenclosing scope. The set of all such scopes visible to a code block\nis called the block\'s *environment*.\n\nIf a name is bound in a block, it is a local variable of that block.\nIf a name is bound at the module level, it is a global variable. (The\nvariables of the module code block are local and global.) If a\nvariable is used in a code block but not defined there, it is a *free\nvariable*.\n\nWhen a name is not found at all, a "NameError" exception is raised.\nIf the name refers to a local variable that has not been bound, a\n"UnboundLocalError" exception is raised. "UnboundLocalError" is a\nsubclass of "NameError".\n\nThe following constructs bind names: formal parameters to functions,\n"import" statements, class and function definitions (these bind the\nclass or function name in the defining block), and targets that are\nidentifiers if occurring in an assignment, "for" loop header, in the\nsecond position of an "except" clause header or after "as" in a "with"\nstatement. The "import" statement of the form "from ... import *"\nbinds all names defined in the imported module, except those beginning\nwith an underscore. This form may only be used at the module level.\n\nA target occurring in a "del" statement is also considered bound for\nthis purpose (though the actual semantics are to unbind the name). It\nis illegal to unbind a name that is referenced by an enclosing scope;\nthe compiler will report a "SyntaxError".\n\nEach assignment or import statement occurs within a block defined by a\nclass or function definition or at the module level (the top-level\ncode block).\n\nIf a name binding operation occurs anywhere within a code block, all\nuses of the name within the block are treated as references to the\ncurrent block. This can lead to errors when a name is used within a\nblock before it is bound. This rule is subtle. Python lacks\ndeclarations and allows name binding operations to occur anywhere\nwithin a code block. The local variables of a code block can be\ndetermined by scanning the entire text of the block for name binding\noperations.\n\nIf the global statement occurs within a block, all uses of the name\nspecified in the statement refer to the binding of that name in the\ntop-level namespace. Names are resolved in the top-level namespace by\nsearching the global namespace, i.e. the namespace of the module\ncontaining the code block, and the builtins namespace, the namespace\nof the module "__builtin__". The global namespace is searched first.\nIf the name is not found there, the builtins namespace is searched.\nThe global statement must precede all uses of the name.\n\nThe builtins namespace associated with the execution of a code block\nis actually found by looking up the name "__builtins__" in its global\nnamespace; this should be a dictionary or a module (in the latter case\nthe module\'s dictionary is used). By default, when in the "__main__"\nmodule, "__builtins__" is the built-in module "__builtin__" (note: no\n\'s\'); when in any other module, "__builtins__" is an alias for the\ndictionary of the "__builtin__" module itself. "__builtins__" can be\nset to a user-created dictionary to create a weak form of restricted\nexecution.\n\n**CPython implementation detail:** Users should not touch\n"__builtins__"; it is strictly an implementation detail. Users\nwanting to override values in the builtins namespace should "import"\nthe "__builtin__" (no \'s\') module and modify its attributes\nappropriately.\n\nThe namespace for a module is automatically created the first time a\nmodule is imported. The main module for a script is always called\n"__main__".\n\nThe "global" statement has the same scope as a name binding operation\nin the same block. If the nearest enclosing scope for a free variable\ncontains a global statement, the free variable is treated as a global.\n\nA class definition is an executable statement that may use and define\nnames. These references follow the normal rules for name resolution.\nThe namespace of the class definition becomes the attribute dictionary\nof the class. Names defined at the class scope are not visible in\nmethods.\n\n\nInteraction with dynamic features\n=================================\n\nThere are several cases where Python statements are illegal when used\nin conjunction with nested scopes that contain free variables.\n\nIf a variable is referenced in an enclosing scope, it is illegal to\ndelete the name. An error will be reported at compile time.\n\nIf the wild card form of import --- "import *" --- is used in a\nfunction and the function contains or is a nested block with free\nvariables, the compiler will raise a "SyntaxError".\n\nIf "exec" is used in a function and the function contains or is a\nnested block with free variables, the compiler will raise a\n"SyntaxError" unless the exec explicitly specifies the local namespace\nfor the "exec". (In other words, "exec obj" would be illegal, but\n"exec obj in ns" would be legal.)\n\nThe "eval()", "execfile()", and "input()" functions and the "exec"\nstatement do not have access to the full environment for resolving\nnames. Names may be resolved in the local and global namespaces of\nthe caller. Free variables are not resolved in the nearest enclosing\nnamespace, but in the global namespace. [1] The "exec" statement and\nthe "eval()" and "execfile()" functions have optional arguments to\noverride the global and local namespace. If only one namespace is\nspecified, it is used for both.\n', + 'numbers': u'\nNumeric literals\n****************\n\nThere are four types of numeric literals: plain integers, long\nintegers, floating point numbers, and imaginary numbers. There are no\ncomplex literals (complex numbers can be formed by adding a real\nnumber and an imaginary number).\n\nNote that numeric literals do not include a sign; a phrase like "-1"\nis actually an expression composed of the unary operator \'"-"\' and the\nliteral "1".\n', + 'numeric-types': u'\nEmulating numeric types\n***********************\n\nThe following methods can be defined to emulate numeric objects.\nMethods corresponding to operations that are not supported by the\nparticular kind of number implemented (e.g., bitwise operations for\nnon-integral numbers) should be left undefined.\n\nobject.__add__(self, other)\nobject.__sub__(self, other)\nobject.__mul__(self, other)\nobject.__floordiv__(self, other)\nobject.__mod__(self, other)\nobject.__divmod__(self, other)\nobject.__pow__(self, other[, modulo])\nobject.__lshift__(self, other)\nobject.__rshift__(self, other)\nobject.__and__(self, other)\nobject.__xor__(self, other)\nobject.__or__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "//", "%", "divmod()", "pow()", "**",\n "<<", ">>", "&", "^", "|"). For instance, to evaluate the\n expression "x + y", where *x* is an instance of a class that has an\n "__add__()" method, "x.__add__(y)" is called. The "__divmod__()"\n method should be the equivalent to using "__floordiv__()" and\n "__mod__()"; it should not be related to "__truediv__()" (described\n below). Note that "__pow__()" should be defined to accept an\n optional third argument if the ternary version of the built-in\n "pow()" function is to be supported.\n\n If one of those methods does not support the operation with the\n supplied arguments, it should return "NotImplemented".\n\nobject.__div__(self, other)\nobject.__truediv__(self, other)\n\n The division operator ("/") is implemented by these methods. The\n "__truediv__()" method is used when "__future__.division" is in\n effect, otherwise "__div__()" is used. If only one of these two\n methods is defined, the object will not support division in the\n alternate context; "TypeError" will be raised instead.\n\nobject.__radd__(self, other)\nobject.__rsub__(self, other)\nobject.__rmul__(self, other)\nobject.__rdiv__(self, other)\nobject.__rtruediv__(self, other)\nobject.__rfloordiv__(self, other)\nobject.__rmod__(self, other)\nobject.__rdivmod__(self, other)\nobject.__rpow__(self, other)\nobject.__rlshift__(self, other)\nobject.__rrshift__(self, other)\nobject.__rand__(self, other)\nobject.__rxor__(self, other)\nobject.__ror__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "/", "%", "divmod()", "pow()", "**",\n "<<", ">>", "&", "^", "|") with reflected (swapped) operands.\n These functions are only called if the left operand does not\n support the corresponding operation and the operands are of\n different types. [2] For instance, to evaluate the expression "x -\n y", where *y* is an instance of a class that has an "__rsub__()"\n method, "y.__rsub__(x)" is called if "x.__sub__(y)" returns\n *NotImplemented*.\n\n Note that ternary "pow()" will not try calling "__rpow__()" (the\n coercion rules would become too complicated).\n\n Note: If the right operand\'s type is a subclass of the left\n operand\'s type and that subclass provides the reflected method\n for the operation, this method will be called before the left\n operand\'s non-reflected method. This behavior allows subclasses\n to override their ancestors\' operations.\n\nobject.__iadd__(self, other)\nobject.__isub__(self, other)\nobject.__imul__(self, other)\nobject.__idiv__(self, other)\nobject.__itruediv__(self, other)\nobject.__ifloordiv__(self, other)\nobject.__imod__(self, other)\nobject.__ipow__(self, other[, modulo])\nobject.__ilshift__(self, other)\nobject.__irshift__(self, other)\nobject.__iand__(self, other)\nobject.__ixor__(self, other)\nobject.__ior__(self, other)\n\n These methods are called to implement the augmented arithmetic\n assignments ("+=", "-=", "*=", "/=", "//=", "%=", "**=", "<<=",\n ">>=", "&=", "^=", "|="). These methods should attempt to do the\n operation in-place (modifying *self*) and return the result (which\n could be, but does not have to be, *self*). If a specific method\n is not defined, the augmented assignment falls back to the normal\n methods. For instance, to execute the statement "x += y", where\n *x* is an instance of a class that has an "__iadd__()" method,\n "x.__iadd__(y)" is called. If *x* is an instance of a class that\n does not define a "__iadd__()" method, "x.__add__(y)" and\n "y.__radd__(x)" are considered, as with the evaluation of "x + y".\n\nobject.__neg__(self)\nobject.__pos__(self)\nobject.__abs__(self)\nobject.__invert__(self)\n\n Called to implement the unary arithmetic operations ("-", "+",\n "abs()" and "~").\n\nobject.__complex__(self)\nobject.__int__(self)\nobject.__long__(self)\nobject.__float__(self)\n\n Called to implement the built-in functions "complex()", "int()",\n "long()", and "float()". Should return a value of the appropriate\n type.\n\nobject.__oct__(self)\nobject.__hex__(self)\n\n Called to implement the built-in functions "oct()" and "hex()".\n Should return a string value.\n\nobject.__index__(self)\n\n Called to implement "operator.index()". Also called whenever\n Python needs an integer object (such as in slicing). Must return\n an integer (int or long).\n\n New in version 2.5.\n\nobject.__coerce__(self, other)\n\n Called to implement "mixed-mode" numeric arithmetic. Should either\n return a 2-tuple containing *self* and *other* converted to a\n common numeric type, or "None" if conversion is impossible. When\n the common type would be the type of "other", it is sufficient to\n return "None", since the interpreter will also ask the other object\n to attempt a coercion (but sometimes, if the implementation of the\n other type cannot be changed, it is useful to do the conversion to\n the other type here). A return value of "NotImplemented" is\n equivalent to returning "None".\n', + 'objects': u'\nObjects, values and types\n*************************\n\n*Objects* are Python\'s abstraction for data. All data in a Python\nprogram is represented by objects or by relations between objects. (In\na sense, and in conformance to Von Neumann\'s model of a "stored\nprogram computer," code is also represented by objects.)\n\nEvery object has an identity, a type and a value. An object\'s\n*identity* never changes once it has been created; you may think of it\nas the object\'s address in memory. The \'"is"\' operator compares the\nidentity of two objects; the "id()" function returns an integer\nrepresenting its identity (currently implemented as its address). An\nobject\'s *type* is also unchangeable. [1] An object\'s type determines\nthe operations that the object supports (e.g., "does it have a\nlength?") and also defines the possible values for objects of that\ntype. The "type()" function returns an object\'s type (which is an\nobject itself). The *value* of some objects can change. Objects\nwhose value can change are said to be *mutable*; objects whose value\nis unchangeable once they are created are called *immutable*. (The\nvalue of an immutable container object that contains a reference to a\nmutable object can change when the latter\'s value is changed; however\nthe container is still considered immutable, because the collection of\nobjects it contains cannot be changed. So, immutability is not\nstrictly the same as having an unchangeable value, it is more subtle.)\nAn object\'s mutability is determined by its type; for instance,\nnumbers, strings and tuples are immutable, while dictionaries and\nlists are mutable.\n\nObjects are never explicitly destroyed; however, when they become\nunreachable they may be garbage-collected. An implementation is\nallowed to postpone garbage collection or omit it altogether --- it is\na matter of implementation quality how garbage collection is\nimplemented, as long as no objects are collected that are still\nreachable.\n\n**CPython implementation detail:** CPython currently uses a reference-\ncounting scheme with (optional) delayed detection of cyclically linked\ngarbage, which collects most objects as soon as they become\nunreachable, but is not guaranteed to collect garbage containing\ncircular references. See the documentation of the "gc" module for\ninformation on controlling the collection of cyclic garbage. Other\nimplementations act differently and CPython may change. Do not depend\non immediate finalization of objects when they become unreachable (ex:\nalways close files).\n\nNote that the use of the implementation\'s tracing or debugging\nfacilities may keep objects alive that would normally be collectable.\nAlso note that catching an exception with a \'"try"..."except"\'\nstatement may keep objects alive.\n\nSome objects contain references to "external" resources such as open\nfiles or windows. It is understood that these resources are freed\nwhen the object is garbage-collected, but since garbage collection is\nnot guaranteed to happen, such objects also provide an explicit way to\nrelease the external resource, usually a "close()" method. Programs\nare strongly recommended to explicitly close such objects. The\n\'"try"..."finally"\' statement provides a convenient way to do this.\n\nSome objects contain references to other objects; these are called\n*containers*. Examples of containers are tuples, lists and\ndictionaries. The references are part of a container\'s value. In\nmost cases, when we talk about the value of a container, we imply the\nvalues, not the identities of the contained objects; however, when we\ntalk about the mutability of a container, only the identities of the\nimmediately contained objects are implied. So, if an immutable\ncontainer (like a tuple) contains a reference to a mutable object, its\nvalue changes if that mutable object is changed.\n\nTypes affect almost all aspects of object behavior. Even the\nimportance of object identity is affected in some sense: for immutable\ntypes, operations that compute new values may actually return a\nreference to any existing object with the same type and value, while\nfor mutable objects this is not allowed. E.g., after "a = 1; b = 1",\n"a" and "b" may or may not refer to the same object with the value\none, depending on the implementation, but after "c = []; d = []", "c"\nand "d" are guaranteed to refer to two different, unique, newly\ncreated empty lists. (Note that "c = d = []" assigns the same object\nto both "c" and "d".)\n', + 'operator-summary': u'\nOperator precedence\n*******************\n\nThe following table summarizes the operator precedences in Python,\nfrom lowest precedence (least binding) to highest precedence (most\nbinding). Operators in the same box have the same precedence. Unless\nthe syntax is explicitly given, operators are binary. Operators in\nthe same box group left to right (except for comparisons, including\ntests, which all have the same precedence and chain from left to right\n--- see section Comparisons --- and exponentiation, which groups from\nright to left).\n\n+-------------------------------------------------+---------------------------------------+\n| Operator | Description |\n+=================================================+=======================================+\n| "lambda" | Lambda expression |\n+-------------------------------------------------+---------------------------------------+\n| "if" -- "else" | Conditional expression |\n+-------------------------------------------------+---------------------------------------+\n| "or" | Boolean OR |\n+-------------------------------------------------+---------------------------------------+\n| "and" | Boolean AND |\n+-------------------------------------------------+---------------------------------------+\n| "not" "x" | Boolean NOT |\n+-------------------------------------------------+---------------------------------------+\n| "in", "not in", "is", "is not", "<", "<=", ">", | Comparisons, including membership |\n| ">=", "<>", "!=", "==" | tests and identity tests |\n+-------------------------------------------------+---------------------------------------+\n| "|" | Bitwise OR |\n+-------------------------------------------------+---------------------------------------+\n| "^" | Bitwise XOR |\n+-------------------------------------------------+---------------------------------------+\n| "&" | Bitwise AND |\n+-------------------------------------------------+---------------------------------------+\n| "<<", ">>" | Shifts |\n+-------------------------------------------------+---------------------------------------+\n| "+", "-" | Addition and subtraction |\n+-------------------------------------------------+---------------------------------------+\n| "*", "/", "//", "%" | Multiplication, division, remainder |\n| | [8] |\n+-------------------------------------------------+---------------------------------------+\n| "+x", "-x", "~x" | Positive, negative, bitwise NOT |\n+-------------------------------------------------+---------------------------------------+\n| "**" | Exponentiation [9] |\n+-------------------------------------------------+---------------------------------------+\n| "x[index]", "x[index:index]", | Subscription, slicing, call, |\n| "x(arguments...)", "x.attribute" | attribute reference |\n+-------------------------------------------------+---------------------------------------+\n| "(expressions...)", "[expressions...]", "{key: | Binding or tuple display, list |\n| value...}", "`expressions...`" | display, dictionary display, string |\n| | conversion |\n+-------------------------------------------------+---------------------------------------+\n\n-[ Footnotes ]-\n\n[1] In Python 2.3 and later releases, a list comprehension "leaks"\n the control variables of each "for" it contains into the\n containing scope. However, this behavior is deprecated, and\n relying on it will not work in Python 3.\n\n[2] While "abs(x%y) < abs(y)" is true mathematically, for floats\n it may not be true numerically due to roundoff. For example, and\n assuming a platform on which a Python float is an IEEE 754 double-\n precision number, in order that "-1e-100 % 1e100" have the same\n sign as "1e100", the computed result is "-1e-100 + 1e100", which\n is numerically exactly equal to "1e100". The function\n "math.fmod()" returns a result whose sign matches the sign of the\n first argument instead, and so returns "-1e-100" in this case.\n Which approach is more appropriate depends on the application.\n\n[3] If x is very close to an exact integer multiple of y, it\'s\n possible for "floor(x/y)" to be one larger than "(x-x%y)/y" due to\n rounding. In such cases, Python returns the latter result, in\n order to preserve that "divmod(x,y)[0] * y + x % y" be very close\n to "x".\n\n[4] While comparisons between unicode strings make sense at the\n byte level, they may be counter-intuitive to users. For example,\n the strings "u"\\u00C7"" and "u"\\u0043\\u0327"" compare differently,\n even though they both represent the same unicode character (LATIN\n CAPITAL LETTER C WITH CEDILLA). To compare strings in a human\n recognizable way, compare using "unicodedata.normalize()".\n\n[5] The implementation computes this efficiently, without\n constructing lists or sorting.\n\n[6] Earlier versions of Python used lexicographic comparison of\n the sorted (key, value) lists, but this was very expensive for the\n common case of comparing for equality. An even earlier version of\n Python compared dictionaries by identity only, but this caused\n surprises because people expected to be able to test a dictionary\n for emptiness by comparing it to "{}".\n\n[7] Due to automatic garbage-collection, free lists, and the\n dynamic nature of descriptors, you may notice seemingly unusual\n behaviour in certain uses of the "is" operator, like those\n involving comparisons between instance methods, or constants.\n Check their documentation for more info.\n\n[8] The "%" operator is also used for string formatting; the same\n precedence applies.\n\n[9] The power operator "**" binds less tightly than an arithmetic\n or bitwise unary operator on its right, that is, "2**-1" is "0.5".\n', + 'pass': u'\nThe "pass" statement\n********************\n\n pass_stmt ::= "pass"\n\n"pass" is a null operation --- when it is executed, nothing happens.\nIt is useful as a placeholder when a statement is required\nsyntactically, but no code needs to be executed, for example:\n\n def f(arg): pass # a function that does nothing (yet)\n\n class C: pass # a class with no methods (yet)\n', + 'power': u'\nThe power operator\n******************\n\nThe power operator binds more tightly than unary operators on its\nleft; it binds less tightly than unary operators on its right. The\nsyntax is:\n\n power ::= primary ["**" u_expr]\n\nThus, in an unparenthesized sequence of power and unary operators, the\noperators are evaluated from right to left (this does not constrain\nthe evaluation order for the operands): "-1**2" results in "-1".\n\nThe power operator has the same semantics as the built-in "pow()"\nfunction, when called with two arguments: it yields its left argument\nraised to the power of its right argument. The numeric arguments are\nfirst converted to a common type. The result type is that of the\narguments after coercion.\n\nWith mixed operand types, the coercion rules for binary arithmetic\noperators apply. For int and long int operands, the result has the\nsame type as the operands (after coercion) unless the second argument\nis negative; in that case, all arguments are converted to float and a\nfloat result is delivered. For example, "10**2" returns "100", but\n"10**-2" returns "0.01". (This last feature was added in Python 2.2.\nIn Python 2.1 and before, if both arguments were of integer types and\nthe second argument was negative, an exception was raised).\n\nRaising "0.0" to a negative power results in a "ZeroDivisionError".\nRaising a negative number to a fractional power results in a\n"ValueError".\n', + 'print': u'\nThe "print" statement\n*********************\n\n print_stmt ::= "print" ([expression ("," expression)* [","]]\n | ">>" expression [("," expression)+ [","]])\n\n"print" evaluates each expression in turn and writes the resulting\nobject to standard output (see below). If an object is not a string,\nit is first converted to a string using the rules for string\nconversions. The (resulting or original) string is then written. A\nspace is written before each object is (converted and) written, unless\nthe output system believes it is positioned at the beginning of a\nline. This is the case (1) when no characters have yet been written\nto standard output, (2) when the last character written to standard\noutput is a whitespace character except "\' \'", or (3) when the last\nwrite operation on standard output was not a "print" statement. (In\nsome cases it may be functional to write an empty string to standard\noutput for this reason.)\n\nNote: Objects which act like file objects but which are not the\n built-in file objects often do not properly emulate this aspect of\n the file object\'s behavior, so it is best not to rely on this.\n\nA "\'\\n\'" character is written at the end, unless the "print" statement\nends with a comma. This is the only action if the statement contains\njust the keyword "print".\n\nStandard output is defined as the file object named "stdout" in the\nbuilt-in module "sys". If no such object exists, or if it does not\nhave a "write()" method, a "RuntimeError" exception is raised.\n\n"print" also has an extended form, defined by the second portion of\nthe syntax described above. This form is sometimes referred to as\n""print" chevron." In this form, the first expression after the ">>"\nmust evaluate to a "file-like" object, specifically an object that has\na "write()" method as described above. With this extended form, the\nsubsequent expressions are printed to this file object. If the first\nexpression evaluates to "None", then "sys.stdout" is used as the file\nfor output.\n', + 'raise': u'\nThe "raise" statement\n*********************\n\n raise_stmt ::= "raise" [expression ["," expression ["," expression]]]\n\nIf no expressions are present, "raise" re-raises the last exception\nthat was active in the current scope. If no exception is active in\nthe current scope, a "TypeError" exception is raised indicating that\nthis is an error (if running under IDLE, a "Queue.Empty" exception is\nraised instead).\n\nOtherwise, "raise" evaluates the expressions to get three objects,\nusing "None" as the value of omitted expressions. The first two\nobjects are used to determine the *type* and *value* of the exception.\n\nIf the first object is an instance, the type of the exception is the\nclass of the instance, the instance itself is the value, and the\nsecond object must be "None".\n\nIf the first object is a class, it becomes the type of the exception.\nThe second object is used to determine the exception value: If it is\nan instance of the class, the instance becomes the exception value. If\nthe second object is a tuple, it is used as the argument list for the\nclass constructor; if it is "None", an empty argument list is used,\nand any other object is treated as a single argument to the\nconstructor. The instance so created by calling the constructor is\nused as the exception value.\n\nIf a third object is present and not "None", it must be a traceback\nobject (see section The standard type hierarchy), and it is\nsubstituted instead of the current location as the place where the\nexception occurred. If the third object is present and not a\ntraceback object or "None", a "TypeError" exception is raised. The\nthree-expression form of "raise" is useful to re-raise an exception\ntransparently in an except clause, but "raise" with no expressions\nshould be preferred if the exception to be re-raised was the most\nrecently active exception in the current scope.\n\nAdditional information on exceptions can be found in section\nExceptions, and information about handling exceptions is in section\nThe try statement.\n', + 'return': u'\nThe "return" statement\n**********************\n\n return_stmt ::= "return" [expression_list]\n\n"return" may only occur syntactically nested in a function definition,\nnot within a nested class definition.\n\nIf an expression list is present, it is evaluated, else "None" is\nsubstituted.\n\n"return" leaves the current function call with the expression list (or\n"None") as return value.\n\nWhen "return" passes control out of a "try" statement with a "finally"\nclause, that "finally" clause is executed before really leaving the\nfunction.\n\nIn a generator function, the "return" statement is not allowed to\ninclude an "expression_list". In that context, a bare "return"\nindicates that the generator is done and will cause "StopIteration" to\nbe raised.\n', + 'sequence-types': u'\nEmulating container types\n*************************\n\nThe following methods can be defined to implement container objects.\nContainers usually are sequences (such as lists or tuples) or mappings\n(like dictionaries), but can represent other containers as well. The\nfirst set of methods is used either to emulate a sequence or to\nemulate a mapping; the difference is that for a sequence, the\nallowable keys should be the integers *k* for which "0 <= k < N" where\n*N* is the length of the sequence, or slice objects, which define a\nrange of items. (For backwards compatibility, the method\n"__getslice__()" (see below) can also be defined to handle simple, but\nnot extended slices.) It is also recommended that mappings provide the\nmethods "keys()", "values()", "items()", "has_key()", "get()",\n"clear()", "setdefault()", "iterkeys()", "itervalues()",\n"iteritems()", "pop()", "popitem()", "copy()", and "update()" behaving\nsimilar to those for Python\'s standard dictionary objects. The\n"UserDict" module provides a "DictMixin" class to help create those\nmethods from a base set of "__getitem__()", "__setitem__()",\n"__delitem__()", and "keys()". Mutable sequences should provide\nmethods "append()", "count()", "index()", "extend()", "insert()",\n"pop()", "remove()", "reverse()" and "sort()", like Python standard\nlist objects. Finally, sequence types should implement addition\n(meaning concatenation) and multiplication (meaning repetition) by\ndefining the methods "__add__()", "__radd__()", "__iadd__()",\n"__mul__()", "__rmul__()" and "__imul__()" described below; they\nshould not define "__coerce__()" or other numerical operators. It is\nrecommended that both mappings and sequences implement the\n"__contains__()" method to allow efficient use of the "in" operator;\nfor mappings, "in" should be equivalent of "has_key()"; for sequences,\nit should search through the values. It is further recommended that\nboth mappings and sequences implement the "__iter__()" method to allow\nefficient iteration through the container; for mappings, "__iter__()"\nshould be the same as "iterkeys()"; for sequences, it should iterate\nthrough the values.\n\nobject.__len__(self)\n\n Called to implement the built-in function "len()". Should return\n the length of the object, an integer ">=" 0. Also, an object that\n doesn\'t define a "__nonzero__()" method and whose "__len__()"\n method returns zero is considered to be false in a Boolean context.\n\nobject.__getitem__(self, key)\n\n Called to implement evaluation of "self[key]". For sequence types,\n the accepted keys should be integers and slice objects. Note that\n the special interpretation of negative indexes (if the class wishes\n to emulate a sequence type) is up to the "__getitem__()" method. If\n *key* is of an inappropriate type, "TypeError" may be raised; if of\n a value outside the set of indexes for the sequence (after any\n special interpretation of negative values), "IndexError" should be\n raised. For mapping types, if *key* is missing (not in the\n container), "KeyError" should be raised.\n\n Note: "for" loops expect that an "IndexError" will be raised for\n illegal indexes to allow proper detection of the end of the\n sequence.\n\nobject.__missing__(self, key)\n\n Called by "dict"."__getitem__()" to implement "self[key]" for dict\n subclasses when key is not in the dictionary.\n\nobject.__setitem__(self, key, value)\n\n Called to implement assignment to "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support changes to the values for keys, or if new keys\n can be added, or for sequences if elements can be replaced. The\n same exceptions should be raised for improper *key* values as for\n the "__getitem__()" method.\n\nobject.__delitem__(self, key)\n\n Called to implement deletion of "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support removal of keys, or for sequences if elements\n can be removed from the sequence. The same exceptions should be\n raised for improper *key* values as for the "__getitem__()" method.\n\nobject.__iter__(self)\n\n This method is called when an iterator is required for a container.\n This method should return a new iterator object that can iterate\n over all the objects in the container. For mappings, it should\n iterate over the keys of the container, and should also be made\n available as the method "iterkeys()".\n\n Iterator objects also need to implement this method; they are\n required to return themselves. For more information on iterator\n objects, see Iterator Types.\n\nobject.__reversed__(self)\n\n Called (if present) by the "reversed()" built-in to implement\n reverse iteration. It should return a new iterator object that\n iterates over all the objects in the container in reverse order.\n\n If the "__reversed__()" method is not provided, the "reversed()"\n built-in will fall back to using the sequence protocol ("__len__()"\n and "__getitem__()"). Objects that support the sequence protocol\n should only provide "__reversed__()" if they can provide an\n implementation that is more efficient than the one provided by\n "reversed()".\n\n New in version 2.6.\n\nThe membership test operators ("in" and "not in") are normally\nimplemented as an iteration through a sequence. However, container\nobjects can supply the following special method with a more efficient\nimplementation, which also does not require the object be a sequence.\n\nobject.__contains__(self, item)\n\n Called to implement membership test operators. Should return true\n if *item* is in *self*, false otherwise. For mapping objects, this\n should consider the keys of the mapping rather than the values or\n the key-item pairs.\n\n For objects that don\'t define "__contains__()", the membership test\n first tries iteration via "__iter__()", then the old sequence\n iteration protocol via "__getitem__()", see this section in the\n language reference.\n', + 'shifting': u'\nShifting operations\n*******************\n\nThe shifting operations have lower priority than the arithmetic\noperations:\n\n shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr\n\nThese operators accept plain or long integers as arguments. The\narguments are converted to a common type. They shift the first\nargument to the left or right by the number of bits given by the\nsecond argument.\n\nA right shift by *n* bits is defined as division by "pow(2, n)". A\nleft shift by *n* bits is defined as multiplication with "pow(2, n)".\nNegative shift counts raise a "ValueError" exception.\n\nNote: In the current implementation, the right-hand operand is\n required to be at most "sys.maxsize". If the right-hand operand is\n larger than "sys.maxsize" an "OverflowError" exception is raised.\n', + 'slicings': u'\nSlicings\n********\n\nA slicing selects a range of items in a sequence object (e.g., a\nstring, tuple or list). Slicings may be used as expressions or as\ntargets in assignment or "del" statements. The syntax for a slicing:\n\n slicing ::= simple_slicing | extended_slicing\n simple_slicing ::= primary "[" short_slice "]"\n extended_slicing ::= primary "[" slice_list "]"\n slice_list ::= slice_item ("," slice_item)* [","]\n slice_item ::= expression | proper_slice | ellipsis\n proper_slice ::= short_slice | long_slice\n short_slice ::= [lower_bound] ":" [upper_bound]\n long_slice ::= short_slice ":" [stride]\n lower_bound ::= expression\n upper_bound ::= expression\n stride ::= expression\n ellipsis ::= "..."\n\nThere is ambiguity in the formal syntax here: anything that looks like\nan expression list also looks like a slice list, so any subscription\ncan be interpreted as a slicing. Rather than further complicating the\nsyntax, this is disambiguated by defining that in this case the\ninterpretation as a subscription takes priority over the\ninterpretation as a slicing (this is the case if the slice list\ncontains no proper slice nor ellipses). Similarly, when the slice\nlist has exactly one short slice and no trailing comma, the\ninterpretation as a simple slicing takes priority over that as an\nextended slicing.\n\nThe semantics for a simple slicing are as follows. The primary must\nevaluate to a sequence object. The lower and upper bound expressions,\nif present, must evaluate to plain integers; defaults are zero and the\n"sys.maxint", respectively. If either bound is negative, the\nsequence\'s length is added to it. The slicing now selects all items\nwith index *k* such that "i <= k < j" where *i* and *j* are the\nspecified lower and upper bounds. This may be an empty sequence. It\nis not an error if *i* or *j* lie outside the range of valid indexes\n(such items don\'t exist so they aren\'t selected).\n\nThe semantics for an extended slicing are as follows. The primary\nmust evaluate to a mapping object, and it is indexed with a key that\nis constructed from the slice list, as follows. If the slice list\ncontains at least one comma, the key is a tuple containing the\nconversion of the slice items; otherwise, the conversion of the lone\nslice item is the key. The conversion of a slice item that is an\nexpression is that expression. The conversion of an ellipsis slice\nitem is the built-in "Ellipsis" object. The conversion of a proper\nslice is a slice object (see section The standard type hierarchy)\nwhose "start", "stop" and "step" attributes are the values of the\nexpressions given as lower bound, upper bound and stride,\nrespectively, substituting "None" for missing expressions.\n', + 'specialattrs': u'\nSpecial Attributes\n******************\n\nThe implementation adds a few special read-only attributes to several\nobject types, where they are relevant. Some of these are not reported\nby the "dir()" built-in function.\n\nobject.__dict__\n\n A dictionary or other mapping object used to store an object\'s\n (writable) attributes.\n\nobject.__methods__\n\n Deprecated since version 2.2: Use the built-in function "dir()" to\n get a list of an object\'s attributes. This attribute is no longer\n available.\n\nobject.__members__\n\n Deprecated since version 2.2: Use the built-in function "dir()" to\n get a list of an object\'s attributes. This attribute is no longer\n available.\n\ninstance.__class__\n\n The class to which a class instance belongs.\n\nclass.__bases__\n\n The tuple of base classes of a class object.\n\nclass.__name__\n\n The name of the class or type.\n\nThe following attributes are only supported by *new-style class*es.\n\nclass.__mro__\n\n This attribute is a tuple of classes that are considered when\n looking for base classes during method resolution.\n\nclass.mro()\n\n This method can be overridden by a metaclass to customize the\n method resolution order for its instances. It is called at class\n instantiation, and its result is stored in "__mro__".\n\nclass.__subclasses__()\n\n Each new-style class keeps a list of weak references to its\n immediate subclasses. This method returns a list of all those\n references still alive. Example:\n\n >>> int.__subclasses__()\n []\n\n-[ Footnotes ]-\n\n[1] Additional information on these special methods may be found\n in the Python Reference Manual (Basic customization).\n\n[2] As a consequence, the list "[1, 2]" is considered equal to\n "[1.0, 2.0]", and similarly for tuples.\n\n[3] They must have since the parser can\'t tell the type of the\n operands.\n\n[4] Cased characters are those with general category property\n being one of "Lu" (Letter, uppercase), "Ll" (Letter, lowercase),\n or "Lt" (Letter, titlecase).\n\n[5] To format only a tuple you should therefore provide a\n singleton tuple whose only element is the tuple to be formatted.\n\n[6] The advantage of leaving the newline on is that returning an\n empty string is then an unambiguous EOF indication. It is also\n possible (in cases where it might matter, for example, if you want\n to make an exact copy of a file while scanning its lines) to tell\n whether the last line of a file ended in a newline or not (yes\n this happens!).\n', + 'specialnames': u'\nSpecial method names\n********************\n\nA class can implement certain operations that are invoked by special\nsyntax (such as arithmetic operations or subscripting and slicing) by\ndefining methods with special names. This is Python\'s approach to\n*operator overloading*, allowing classes to define their own behavior\nwith respect to language operators. For instance, if a class defines\na method named "__getitem__()", and "x" is an instance of this class,\nthen "x[i]" is roughly equivalent to "x.__getitem__(i)" for old-style\nclasses and "type(x).__getitem__(x, i)" for new-style classes. Except\nwhere mentioned, attempts to execute an operation raise an exception\nwhen no appropriate method is defined (typically "AttributeError" or\n"TypeError").\n\nWhen implementing a class that emulates any built-in type, it is\nimportant that the emulation only be implemented to the degree that it\nmakes sense for the object being modelled. For example, some\nsequences may work well with retrieval of individual elements, but\nextracting a slice may not make sense. (One example of this is the\n"NodeList" interface in the W3C\'s Document Object Model.)\n\n\nBasic customization\n===================\n\nobject.__new__(cls[, ...])\n\n Called to create a new instance of class *cls*. "__new__()" is a\n static method (special-cased so you need not declare it as such)\n that takes the class of which an instance was requested as its\n first argument. The remaining arguments are those passed to the\n object constructor expression (the call to the class). The return\n value of "__new__()" should be the new object instance (usually an\n instance of *cls*).\n\n Typical implementations create a new instance of the class by\n invoking the superclass\'s "__new__()" method using\n "super(currentclass, cls).__new__(cls[, ...])" with appropriate\n arguments and then modifying the newly-created instance as\n necessary before returning it.\n\n If "__new__()" returns an instance of *cls*, then the new\n instance\'s "__init__()" method will be invoked like\n "__init__(self[, ...])", where *self* is the new instance and the\n remaining arguments are the same as were passed to "__new__()".\n\n If "__new__()" does not return an instance of *cls*, then the new\n instance\'s "__init__()" method will not be invoked.\n\n "__new__()" is intended mainly to allow subclasses of immutable\n types (like int, str, or tuple) to customize instance creation. It\n is also commonly overridden in custom metaclasses in order to\n customize class creation.\n\nobject.__init__(self[, ...])\n\n Called after the instance has been created (by "__new__()"), but\n before it is returned to the caller. The arguments are those\n passed to the class constructor expression. If a base class has an\n "__init__()" method, the derived class\'s "__init__()" method, if\n any, must explicitly call it to ensure proper initialization of the\n base class part of the instance; for example:\n "BaseClass.__init__(self, [args...])".\n\n Because "__new__()" and "__init__()" work together in constructing\n objects ("__new__()" to create it, and "__init__()" to customise\n it), no non-"None" value may be returned by "__init__()"; doing so\n will cause a "TypeError" to be raised at runtime.\n\nobject.__del__(self)\n\n Called when the instance is about to be destroyed. This is also\n called a destructor. If a base class has a "__del__()" method, the\n derived class\'s "__del__()" method, if any, must explicitly call it\n to ensure proper deletion of the base class part of the instance.\n Note that it is possible (though not recommended!) for the\n "__del__()" method to postpone destruction of the instance by\n creating a new reference to it. It may then be called at a later\n time when this new reference is deleted. It is not guaranteed that\n "__del__()" methods are called for objects that still exist when\n the interpreter exits.\n\n Note: "del x" doesn\'t directly call "x.__del__()" --- the former\n decrements the reference count for "x" by one, and the latter is\n only called when "x"\'s reference count reaches zero. Some common\n situations that may prevent the reference count of an object from\n going to zero include: circular references between objects (e.g.,\n a doubly-linked list or a tree data structure with parent and\n child pointers); a reference to the object on the stack frame of\n a function that caught an exception (the traceback stored in\n "sys.exc_traceback" keeps the stack frame alive); or a reference\n to the object on the stack frame that raised an unhandled\n exception in interactive mode (the traceback stored in\n "sys.last_traceback" keeps the stack frame alive). The first\n situation can only be remedied by explicitly breaking the cycles;\n the latter two situations can be resolved by storing "None" in\n "sys.exc_traceback" or "sys.last_traceback". Circular references\n which are garbage are detected when the option cycle detector is\n enabled (it\'s on by default), but can only be cleaned up if there\n are no Python-level "__del__()" methods involved. Refer to the\n documentation for the "gc" module for more information about how\n "__del__()" methods are handled by the cycle detector,\n particularly the description of the "garbage" value.\n\n Warning: Due to the precarious circumstances under which\n "__del__()" methods are invoked, exceptions that occur during\n their execution are ignored, and a warning is printed to\n "sys.stderr" instead. Also, when "__del__()" is invoked in\n response to a module being deleted (e.g., when execution of the\n program is done), other globals referenced by the "__del__()"\n method may already have been deleted or in the process of being\n torn down (e.g. the import machinery shutting down). For this\n reason, "__del__()" methods should do the absolute minimum needed\n to maintain external invariants. Starting with version 1.5,\n Python guarantees that globals whose name begins with a single\n underscore are deleted from their module before other globals are\n deleted; if no other references to such globals exist, this may\n help in assuring that imported modules are still available at the\n time when the "__del__()" method is called.\n\n See also the "-R" command-line option.\n\nobject.__repr__(self)\n\n Called by the "repr()" built-in function and by string conversions\n (reverse quotes) to compute the "official" string representation of\n an object. If at all possible, this should look like a valid\n Python expression that could be used to recreate an object with the\n same value (given an appropriate environment). If this is not\n possible, a string of the form "<...some useful description...>"\n should be returned. The return value must be a string object. If a\n class defines "__repr__()" but not "__str__()", then "__repr__()"\n is also used when an "informal" string representation of instances\n of that class is required.\n\n This is typically used for debugging, so it is important that the\n representation is information-rich and unambiguous.\n\nobject.__str__(self)\n\n Called by the "str()" built-in function and by the "print"\n statement to compute the "informal" string representation of an\n object. This differs from "__repr__()" in that it does not have to\n be a valid Python expression: a more convenient or concise\n representation may be used instead. The return value must be a\n string object.\n\nobject.__lt__(self, other)\nobject.__le__(self, other)\nobject.__eq__(self, other)\nobject.__ne__(self, other)\nobject.__gt__(self, other)\nobject.__ge__(self, other)\n\n New in version 2.1.\n\n These are the so-called "rich comparison" methods, and are called\n for comparison operators in preference to "__cmp__()" below. The\n correspondence between operator symbols and method names is as\n follows: "xy" call "x.__ne__(y)",\n "x>y" calls "x.__gt__(y)", and "x>=y" calls "x.__ge__(y)".\n\n A rich comparison method may return the singleton "NotImplemented"\n if it does not implement the operation for a given pair of\n arguments. By convention, "False" and "True" are returned for a\n successful comparison. However, these methods can return any value,\n so if the comparison operator is used in a Boolean context (e.g.,\n in the condition of an "if" statement), Python will call "bool()"\n on the value to determine if the result is true or false.\n\n There are no implied relationships among the comparison operators.\n The truth of "x==y" does not imply that "x!=y" is false.\n Accordingly, when defining "__eq__()", one should also define\n "__ne__()" so that the operators will behave as expected. See the\n paragraph on "__hash__()" for some important notes on creating\n *hashable* objects which support custom comparison operations and\n are usable as dictionary keys.\n\n There are no swapped-argument versions of these methods (to be used\n when the left argument does not support the operation but the right\n argument does); rather, "__lt__()" and "__gt__()" are each other\'s\n reflection, "__le__()" and "__ge__()" are each other\'s reflection,\n and "__eq__()" and "__ne__()" are their own reflection.\n\n Arguments to rich comparison methods are never coerced.\n\n To automatically generate ordering operations from a single root\n operation, see "functools.total_ordering()".\n\nobject.__cmp__(self, other)\n\n Called by comparison operations if rich comparison (see above) is\n not defined. Should return a negative integer if "self < other",\n zero if "self == other", a positive integer if "self > other". If\n no "__cmp__()", "__eq__()" or "__ne__()" operation is defined,\n class instances are compared by object identity ("address"). See\n also the description of "__hash__()" for some important notes on\n creating *hashable* objects which support custom comparison\n operations and are usable as dictionary keys. (Note: the\n restriction that exceptions are not propagated by "__cmp__()" has\n been removed since Python 1.5.)\n\nobject.__rcmp__(self, other)\n\n Changed in version 2.1: No longer supported.\n\nobject.__hash__(self)\n\n Called by built-in function "hash()" and for operations on members\n of hashed collections including "set", "frozenset", and "dict".\n "__hash__()" should return an integer. The only required property\n is that objects which compare equal have the same hash value; it is\n advised to somehow mix together (e.g. using exclusive or) the hash\n values for the components of the object that also play a part in\n comparison of objects.\n\n If a class does not define a "__cmp__()" or "__eq__()" method it\n should not define a "__hash__()" operation either; if it defines\n "__cmp__()" or "__eq__()" but not "__hash__()", its instances will\n not be usable in hashed collections. If a class defines mutable\n objects and implements a "__cmp__()" or "__eq__()" method, it\n should not implement "__hash__()", since hashable collection\n implementations require that a object\'s hash value is immutable (if\n the object\'s hash value changes, it will be in the wrong hash\n bucket).\n\n User-defined classes have "__cmp__()" and "__hash__()" methods by\n default; with them, all objects compare unequal (except with\n themselves) and "x.__hash__()" returns a result derived from\n "id(x)".\n\n Classes which inherit a "__hash__()" method from a parent class but\n change the meaning of "__cmp__()" or "__eq__()" such that the hash\n value returned is no longer appropriate (e.g. by switching to a\n value-based concept of equality instead of the default identity\n based equality) can explicitly flag themselves as being unhashable\n by setting "__hash__ = None" in the class definition. Doing so\n means that not only will instances of the class raise an\n appropriate "TypeError" when a program attempts to retrieve their\n hash value, but they will also be correctly identified as\n unhashable when checking "isinstance(obj, collections.Hashable)"\n (unlike classes which define their own "__hash__()" to explicitly\n raise "TypeError").\n\n Changed in version 2.5: "__hash__()" may now also return a long\n integer object; the 32-bit integer is then derived from the hash of\n that object.\n\n Changed in version 2.6: "__hash__" may now be set to "None" to\n explicitly flag instances of a class as unhashable.\n\nobject.__nonzero__(self)\n\n Called to implement truth value testing and the built-in operation\n "bool()"; should return "False" or "True", or their integer\n equivalents "0" or "1". When this method is not defined,\n "__len__()" is called, if it is defined, and the object is\n considered true if its result is nonzero. If a class defines\n neither "__len__()" nor "__nonzero__()", all its instances are\n considered true.\n\nobject.__unicode__(self)\n\n Called to implement "unicode()" built-in; should return a Unicode\n object. When this method is not defined, string conversion is\n attempted, and the result of string conversion is converted to\n Unicode using the system default encoding.\n\n\nCustomizing attribute access\n============================\n\nThe following methods can be defined to customize the meaning of\nattribute access (use of, assignment to, or deletion of "x.name") for\nclass instances.\n\nobject.__getattr__(self, name)\n\n Called when an attribute lookup has not found the attribute in the\n usual places (i.e. it is not an instance attribute nor is it found\n in the class tree for "self"). "name" is the attribute name. This\n method should return the (computed) attribute value or raise an\n "AttributeError" exception.\n\n Note that if the attribute is found through the normal mechanism,\n "__getattr__()" is not called. (This is an intentional asymmetry\n between "__getattr__()" and "__setattr__()".) This is done both for\n efficiency reasons and because otherwise "__getattr__()" would have\n no way to access other attributes of the instance. Note that at\n least for instance variables, you can fake total control by not\n inserting any values in the instance attribute dictionary (but\n instead inserting them in another object). See the\n "__getattribute__()" method below for a way to actually get total\n control in new-style classes.\n\nobject.__setattr__(self, name, value)\n\n Called when an attribute assignment is attempted. This is called\n instead of the normal mechanism (i.e. store the value in the\n instance dictionary). *name* is the attribute name, *value* is the\n value to be assigned to it.\n\n If "__setattr__()" wants to assign to an instance attribute, it\n should not simply execute "self.name = value" --- this would cause\n a recursive call to itself. Instead, it should insert the value in\n the dictionary of instance attributes, e.g., "self.__dict__[name] =\n value". For new-style classes, rather than accessing the instance\n dictionary, it should call the base class method with the same\n name, for example, "object.__setattr__(self, name, value)".\n\nobject.__delattr__(self, name)\n\n Like "__setattr__()" but for attribute deletion instead of\n assignment. This should only be implemented if "del obj.name" is\n meaningful for the object.\n\n\nMore attribute access for new-style classes\n-------------------------------------------\n\nThe following methods only apply to new-style classes.\n\nobject.__getattribute__(self, name)\n\n Called unconditionally to implement attribute accesses for\n instances of the class. If the class also defines "__getattr__()",\n the latter will not be called unless "__getattribute__()" either\n calls it explicitly or raises an "AttributeError". This method\n should return the (computed) attribute value or raise an\n "AttributeError" exception. In order to avoid infinite recursion in\n this method, its implementation should always call the base class\n method with the same name to access any attributes it needs, for\n example, "object.__getattribute__(self, name)".\n\n Note: This method may still be bypassed when looking up special\n methods as the result of implicit invocation via language syntax\n or built-in functions. See Special method lookup for new-style\n classes.\n\n\nImplementing Descriptors\n------------------------\n\nThe following methods only apply when an instance of the class\ncontaining the method (a so-called *descriptor* class) appears in an\n*owner* class (the descriptor must be in either the owner\'s class\ndictionary or in the class dictionary for one of its parents). In the\nexamples below, "the attribute" refers to the attribute whose name is\nthe key of the property in the owner class\' "__dict__".\n\nobject.__get__(self, instance, owner)\n\n Called to get the attribute of the owner class (class attribute\n access) or of an instance of that class (instance attribute\n access). *owner* is always the owner class, while *instance* is the\n instance that the attribute was accessed through, or "None" when\n the attribute is accessed through the *owner*. This method should\n return the (computed) attribute value or raise an "AttributeError"\n exception.\n\nobject.__set__(self, instance, value)\n\n Called to set the attribute on an instance *instance* of the owner\n class to a new value, *value*.\n\nobject.__delete__(self, instance)\n\n Called to delete the attribute on an instance *instance* of the\n owner class.\n\n\nInvoking Descriptors\n--------------------\n\nIn general, a descriptor is an object attribute with "binding\nbehavior", one whose attribute access has been overridden by methods\nin the descriptor protocol: "__get__()", "__set__()", and\n"__delete__()". If any of those methods are defined for an object, it\nis said to be a descriptor.\n\nThe default behavior for attribute access is to get, set, or delete\nthe attribute from an object\'s dictionary. For instance, "a.x" has a\nlookup chain starting with "a.__dict__[\'x\']", then\n"type(a).__dict__[\'x\']", and continuing through the base classes of\n"type(a)" excluding metaclasses.\n\nHowever, if the looked-up value is an object defining one of the\ndescriptor methods, then Python may override the default behavior and\ninvoke the descriptor method instead. Where this occurs in the\nprecedence chain depends on which descriptor methods were defined and\nhow they were called. Note that descriptors are only invoked for new\nstyle objects or classes (ones that subclass "object()" or "type()").\n\nThe starting point for descriptor invocation is a binding, "a.x". How\nthe arguments are assembled depends on "a":\n\nDirect Call\n The simplest and least common call is when user code directly\n invokes a descriptor method: "x.__get__(a)".\n\nInstance Binding\n If binding to a new-style object instance, "a.x" is transformed\n into the call: "type(a).__dict__[\'x\'].__get__(a, type(a))".\n\nClass Binding\n If binding to a new-style class, "A.x" is transformed into the\n call: "A.__dict__[\'x\'].__get__(None, A)".\n\nSuper Binding\n If "a" is an instance of "super", then the binding "super(B,\n obj).m()" searches "obj.__class__.__mro__" for the base class "A"\n immediately preceding "B" and then invokes the descriptor with the\n call: "A.__dict__[\'m\'].__get__(obj, obj.__class__)".\n\nFor instance bindings, the precedence of descriptor invocation depends\non the which descriptor methods are defined. A descriptor can define\nany combination of "__get__()", "__set__()" and "__delete__()". If it\ndoes not define "__get__()", then accessing the attribute will return\nthe descriptor object itself unless there is a value in the object\'s\ninstance dictionary. If the descriptor defines "__set__()" and/or\n"__delete__()", it is a data descriptor; if it defines neither, it is\na non-data descriptor. Normally, data descriptors define both\n"__get__()" and "__set__()", while non-data descriptors have just the\n"__get__()" method. Data descriptors with "__set__()" and "__get__()"\ndefined always override a redefinition in an instance dictionary. In\ncontrast, non-data descriptors can be overridden by instances.\n\nPython methods (including "staticmethod()" and "classmethod()") are\nimplemented as non-data descriptors. Accordingly, instances can\nredefine and override methods. This allows individual instances to\nacquire behaviors that differ from other instances of the same class.\n\nThe "property()" function is implemented as a data descriptor.\nAccordingly, instances cannot override the behavior of a property.\n\n\n__slots__\n---------\n\nBy default, instances of both old and new-style classes have a\ndictionary for attribute storage. This wastes space for objects\nhaving very few instance variables. The space consumption can become\nacute when creating large numbers of instances.\n\nThe default can be overridden by defining *__slots__* in a new-style\nclass definition. The *__slots__* declaration takes a sequence of\ninstance variables and reserves just enough space in each instance to\nhold a value for each variable. Space is saved because *__dict__* is\nnot created for each instance.\n\n__slots__\n\n This class variable can be assigned a string, iterable, or sequence\n of strings with variable names used by instances. If defined in a\n new-style class, *__slots__* reserves space for the declared\n variables and prevents the automatic creation of *__dict__* and\n *__weakref__* for each instance.\n\n New in version 2.2.\n\nNotes on using *__slots__*\n\n* When inheriting from a class without *__slots__*, the *__dict__*\n attribute of that class will always be accessible, so a *__slots__*\n definition in the subclass is meaningless.\n\n* Without a *__dict__* variable, instances cannot be assigned new\n variables not listed in the *__slots__* definition. Attempts to\n assign to an unlisted variable name raises "AttributeError". If\n dynamic assignment of new variables is desired, then add\n "\'__dict__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n Changed in version 2.3: Previously, adding "\'__dict__\'" to the\n *__slots__* declaration would not enable the assignment of new\n attributes not specifically listed in the sequence of instance\n variable names.\n\n* Without a *__weakref__* variable for each instance, classes\n defining *__slots__* do not support weak references to its\n instances. If weak reference support is needed, then add\n "\'__weakref__\'" to the sequence of strings in the *__slots__*\n declaration.\n\n Changed in version 2.3: Previously, adding "\'__weakref__\'" to the\n *__slots__* declaration would not enable support for weak\n references.\n\n* *__slots__* are implemented at the class level by creating\n descriptors (Implementing Descriptors) for each variable name. As a\n result, class attributes cannot be used to set default values for\n instance variables defined by *__slots__*; otherwise, the class\n attribute would overwrite the descriptor assignment.\n\n* The action of a *__slots__* declaration is limited to the class\n where it is defined. As a result, subclasses will have a *__dict__*\n unless they also define *__slots__* (which must only contain names\n of any *additional* slots).\n\n* If a class defines a slot also defined in a base class, the\n instance variable defined by the base class slot is inaccessible\n (except by retrieving its descriptor directly from the base class).\n This renders the meaning of the program undefined. In the future, a\n check may be added to prevent this.\n\n* Nonempty *__slots__* does not work for classes derived from\n "variable-length" built-in types such as "long", "str" and "tuple".\n\n* Any non-string iterable may be assigned to *__slots__*. Mappings\n may also be used; however, in the future, special meaning may be\n assigned to the values corresponding to each key.\n\n* *__class__* assignment works only if both classes have the same\n *__slots__*.\n\n Changed in version 2.6: Previously, *__class__* assignment raised an\n error if either new or old class had *__slots__*.\n\n\nCustomizing class creation\n==========================\n\nBy default, new-style classes are constructed using "type()". A class\ndefinition is read into a separate namespace and the value of class\nname is bound to the result of "type(name, bases, dict)".\n\nWhen the class definition is read, if *__metaclass__* is defined then\nthe callable assigned to it will be called instead of "type()". This\nallows classes or functions to be written which monitor or alter the\nclass creation process:\n\n* Modifying the class dictionary prior to the class being created.\n\n* Returning an instance of another class -- essentially performing\n the role of a factory function.\n\nThese steps will have to be performed in the metaclass\'s "__new__()"\nmethod -- "type.__new__()" can then be called from this method to\ncreate a class with different properties. This example adds a new\nelement to the class dictionary before creating the class:\n\n class metacls(type):\n def __new__(mcs, name, bases, dict):\n dict[\'foo\'] = \'metacls was here\'\n return type.__new__(mcs, name, bases, dict)\n\nYou can of course also override other class methods (or add new\nmethods); for example defining a custom "__call__()" method in the\nmetaclass allows custom behavior when the class is called, e.g. not\nalways creating a new instance.\n\n__metaclass__\n\n This variable can be any callable accepting arguments for "name",\n "bases", and "dict". Upon class creation, the callable is used\n instead of the built-in "type()".\n\n New in version 2.2.\n\nThe appropriate metaclass is determined by the following precedence\nrules:\n\n* If "dict[\'__metaclass__\']" exists, it is used.\n\n* Otherwise, if there is at least one base class, its metaclass is\n used (this looks for a *__class__* attribute first and if not found,\n uses its type).\n\n* Otherwise, if a global variable named __metaclass__ exists, it is\n used.\n\n* Otherwise, the old-style, classic metaclass (types.ClassType) is\n used.\n\nThe potential uses for metaclasses are boundless. Some ideas that have\nbeen explored including logging, interface checking, automatic\ndelegation, automatic property creation, proxies, frameworks, and\nautomatic resource locking/synchronization.\n\n\nCustomizing instance and subclass checks\n========================================\n\nNew in version 2.6.\n\nThe following methods are used to override the default behavior of the\n"isinstance()" and "issubclass()" built-in functions.\n\nIn particular, the metaclass "abc.ABCMeta" implements these methods in\norder to allow the addition of Abstract Base Classes (ABCs) as\n"virtual base classes" to any class or type (including built-in\ntypes), including other ABCs.\n\nclass.__instancecheck__(self, instance)\n\n Return true if *instance* should be considered a (direct or\n indirect) instance of *class*. If defined, called to implement\n "isinstance(instance, class)".\n\nclass.__subclasscheck__(self, subclass)\n\n Return true if *subclass* should be considered a (direct or\n indirect) subclass of *class*. If defined, called to implement\n "issubclass(subclass, class)".\n\nNote that these methods are looked up on the type (metaclass) of a\nclass. They cannot be defined as class methods in the actual class.\nThis is consistent with the lookup of special methods that are called\non instances, only in this case the instance is itself a class.\n\nSee also:\n\n **PEP 3119** - Introducing Abstract Base Classes\n Includes the specification for customizing "isinstance()" and\n "issubclass()" behavior through "__instancecheck__()" and\n "__subclasscheck__()", with motivation for this functionality in\n the context of adding Abstract Base Classes (see the "abc"\n module) to the language.\n\n\nEmulating callable objects\n==========================\n\nobject.__call__(self[, args...])\n\n Called when the instance is "called" as a function; if this method\n is defined, "x(arg1, arg2, ...)" is a shorthand for\n "x.__call__(arg1, arg2, ...)".\n\n\nEmulating container types\n=========================\n\nThe following methods can be defined to implement container objects.\nContainers usually are sequences (such as lists or tuples) or mappings\n(like dictionaries), but can represent other containers as well. The\nfirst set of methods is used either to emulate a sequence or to\nemulate a mapping; the difference is that for a sequence, the\nallowable keys should be the integers *k* for which "0 <= k < N" where\n*N* is the length of the sequence, or slice objects, which define a\nrange of items. (For backwards compatibility, the method\n"__getslice__()" (see below) can also be defined to handle simple, but\nnot extended slices.) It is also recommended that mappings provide the\nmethods "keys()", "values()", "items()", "has_key()", "get()",\n"clear()", "setdefault()", "iterkeys()", "itervalues()",\n"iteritems()", "pop()", "popitem()", "copy()", and "update()" behaving\nsimilar to those for Python\'s standard dictionary objects. The\n"UserDict" module provides a "DictMixin" class to help create those\nmethods from a base set of "__getitem__()", "__setitem__()",\n"__delitem__()", and "keys()". Mutable sequences should provide\nmethods "append()", "count()", "index()", "extend()", "insert()",\n"pop()", "remove()", "reverse()" and "sort()", like Python standard\nlist objects. Finally, sequence types should implement addition\n(meaning concatenation) and multiplication (meaning repetition) by\ndefining the methods "__add__()", "__radd__()", "__iadd__()",\n"__mul__()", "__rmul__()" and "__imul__()" described below; they\nshould not define "__coerce__()" or other numerical operators. It is\nrecommended that both mappings and sequences implement the\n"__contains__()" method to allow efficient use of the "in" operator;\nfor mappings, "in" should be equivalent of "has_key()"; for sequences,\nit should search through the values. It is further recommended that\nboth mappings and sequences implement the "__iter__()" method to allow\nefficient iteration through the container; for mappings, "__iter__()"\nshould be the same as "iterkeys()"; for sequences, it should iterate\nthrough the values.\n\nobject.__len__(self)\n\n Called to implement the built-in function "len()". Should return\n the length of the object, an integer ">=" 0. Also, an object that\n doesn\'t define a "__nonzero__()" method and whose "__len__()"\n method returns zero is considered to be false in a Boolean context.\n\nobject.__getitem__(self, key)\n\n Called to implement evaluation of "self[key]". For sequence types,\n the accepted keys should be integers and slice objects. Note that\n the special interpretation of negative indexes (if the class wishes\n to emulate a sequence type) is up to the "__getitem__()" method. If\n *key* is of an inappropriate type, "TypeError" may be raised; if of\n a value outside the set of indexes for the sequence (after any\n special interpretation of negative values), "IndexError" should be\n raised. For mapping types, if *key* is missing (not in the\n container), "KeyError" should be raised.\n\n Note: "for" loops expect that an "IndexError" will be raised for\n illegal indexes to allow proper detection of the end of the\n sequence.\n\nobject.__missing__(self, key)\n\n Called by "dict"."__getitem__()" to implement "self[key]" for dict\n subclasses when key is not in the dictionary.\n\nobject.__setitem__(self, key, value)\n\n Called to implement assignment to "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support changes to the values for keys, or if new keys\n can be added, or for sequences if elements can be replaced. The\n same exceptions should be raised for improper *key* values as for\n the "__getitem__()" method.\n\nobject.__delitem__(self, key)\n\n Called to implement deletion of "self[key]". Same note as for\n "__getitem__()". This should only be implemented for mappings if\n the objects support removal of keys, or for sequences if elements\n can be removed from the sequence. The same exceptions should be\n raised for improper *key* values as for the "__getitem__()" method.\n\nobject.__iter__(self)\n\n This method is called when an iterator is required for a container.\n This method should return a new iterator object that can iterate\n over all the objects in the container. For mappings, it should\n iterate over the keys of the container, and should also be made\n available as the method "iterkeys()".\n\n Iterator objects also need to implement this method; they are\n required to return themselves. For more information on iterator\n objects, see Iterator Types.\n\nobject.__reversed__(self)\n\n Called (if present) by the "reversed()" built-in to implement\n reverse iteration. It should return a new iterator object that\n iterates over all the objects in the container in reverse order.\n\n If the "__reversed__()" method is not provided, the "reversed()"\n built-in will fall back to using the sequence protocol ("__len__()"\n and "__getitem__()"). Objects that support the sequence protocol\n should only provide "__reversed__()" if they can provide an\n implementation that is more efficient than the one provided by\n "reversed()".\n\n New in version 2.6.\n\nThe membership test operators ("in" and "not in") are normally\nimplemented as an iteration through a sequence. However, container\nobjects can supply the following special method with a more efficient\nimplementation, which also does not require the object be a sequence.\n\nobject.__contains__(self, item)\n\n Called to implement membership test operators. Should return true\n if *item* is in *self*, false otherwise. For mapping objects, this\n should consider the keys of the mapping rather than the values or\n the key-item pairs.\n\n For objects that don\'t define "__contains__()", the membership test\n first tries iteration via "__iter__()", then the old sequence\n iteration protocol via "__getitem__()", see this section in the\n language reference.\n\n\nAdditional methods for emulation of sequence types\n==================================================\n\nThe following optional methods can be defined to further emulate\nsequence objects. Immutable sequences methods should at most only\ndefine "__getslice__()"; mutable sequences might define all three\nmethods.\n\nobject.__getslice__(self, i, j)\n\n Deprecated since version 2.0: Support slice objects as parameters\n to the "__getitem__()" method. (However, built-in types in CPython\n currently still implement "__getslice__()". Therefore, you have to\n override it in derived classes when implementing slicing.)\n\n Called to implement evaluation of "self[i:j]". The returned object\n should be of the same type as *self*. Note that missing *i* or *j*\n in the slice expression are replaced by zero or "sys.maxsize",\n respectively. If negative indexes are used in the slice, the\n length of the sequence is added to that index. If the instance does\n not implement the "__len__()" method, an "AttributeError" is\n raised. No guarantee is made that indexes adjusted this way are not\n still negative. Indexes which are greater than the length of the\n sequence are not modified. If no "__getslice__()" is found, a slice\n object is created instead, and passed to "__getitem__()" instead.\n\nobject.__setslice__(self, i, j, sequence)\n\n Called to implement assignment to "self[i:j]". Same notes for *i*\n and *j* as for "__getslice__()".\n\n This method is deprecated. If no "__setslice__()" is found, or for\n extended slicing of the form "self[i:j:k]", a slice object is\n created, and passed to "__setitem__()", instead of "__setslice__()"\n being called.\n\nobject.__delslice__(self, i, j)\n\n Called to implement deletion of "self[i:j]". Same notes for *i* and\n *j* as for "__getslice__()". This method is deprecated. If no\n "__delslice__()" is found, or for extended slicing of the form\n "self[i:j:k]", a slice object is created, and passed to\n "__delitem__()", instead of "__delslice__()" being called.\n\nNotice that these methods are only invoked when a single slice with a\nsingle colon is used, and the slice method is available. For slice\noperations involving extended slice notation, or in absence of the\nslice methods, "__getitem__()", "__setitem__()" or "__delitem__()" is\ncalled with a slice object as argument.\n\nThe following example demonstrate how to make your program or module\ncompatible with earlier versions of Python (assuming that methods\n"__getitem__()", "__setitem__()" and "__delitem__()" support slice\nobjects as arguments):\n\n class MyClass:\n ...\n def __getitem__(self, index):\n ...\n def __setitem__(self, index, value):\n ...\n def __delitem__(self, index):\n ...\n\n if sys.version_info < (2, 0):\n # They won\'t be defined if version is at least 2.0 final\n\n def __getslice__(self, i, j):\n return self[max(0, i):max(0, j):]\n def __setslice__(self, i, j, seq):\n self[max(0, i):max(0, j):] = seq\n def __delslice__(self, i, j):\n del self[max(0, i):max(0, j):]\n ...\n\nNote the calls to "max()"; these are necessary because of the handling\nof negative indices before the "__*slice__()" methods are called.\nWhen negative indexes are used, the "__*item__()" methods receive them\nas provided, but the "__*slice__()" methods get a "cooked" form of the\nindex values. For each negative index value, the length of the\nsequence is added to the index before calling the method (which may\nstill result in a negative index); this is the customary handling of\nnegative indexes by the built-in sequence types, and the "__*item__()"\nmethods are expected to do this as well. However, since they should\nalready be doing that, negative indexes cannot be passed in; they must\nbe constrained to the bounds of the sequence before being passed to\nthe "__*item__()" methods. Calling "max(0, i)" conveniently returns\nthe proper value.\n\n\nEmulating numeric types\n=======================\n\nThe following methods can be defined to emulate numeric objects.\nMethods corresponding to operations that are not supported by the\nparticular kind of number implemented (e.g., bitwise operations for\nnon-integral numbers) should be left undefined.\n\nobject.__add__(self, other)\nobject.__sub__(self, other)\nobject.__mul__(self, other)\nobject.__floordiv__(self, other)\nobject.__mod__(self, other)\nobject.__divmod__(self, other)\nobject.__pow__(self, other[, modulo])\nobject.__lshift__(self, other)\nobject.__rshift__(self, other)\nobject.__and__(self, other)\nobject.__xor__(self, other)\nobject.__or__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "//", "%", "divmod()", "pow()", "**",\n "<<", ">>", "&", "^", "|"). For instance, to evaluate the\n expression "x + y", where *x* is an instance of a class that has an\n "__add__()" method, "x.__add__(y)" is called. The "__divmod__()"\n method should be the equivalent to using "__floordiv__()" and\n "__mod__()"; it should not be related to "__truediv__()" (described\n below). Note that "__pow__()" should be defined to accept an\n optional third argument if the ternary version of the built-in\n "pow()" function is to be supported.\n\n If one of those methods does not support the operation with the\n supplied arguments, it should return "NotImplemented".\n\nobject.__div__(self, other)\nobject.__truediv__(self, other)\n\n The division operator ("/") is implemented by these methods. The\n "__truediv__()" method is used when "__future__.division" is in\n effect, otherwise "__div__()" is used. If only one of these two\n methods is defined, the object will not support division in the\n alternate context; "TypeError" will be raised instead.\n\nobject.__radd__(self, other)\nobject.__rsub__(self, other)\nobject.__rmul__(self, other)\nobject.__rdiv__(self, other)\nobject.__rtruediv__(self, other)\nobject.__rfloordiv__(self, other)\nobject.__rmod__(self, other)\nobject.__rdivmod__(self, other)\nobject.__rpow__(self, other)\nobject.__rlshift__(self, other)\nobject.__rrshift__(self, other)\nobject.__rand__(self, other)\nobject.__rxor__(self, other)\nobject.__ror__(self, other)\n\n These methods are called to implement the binary arithmetic\n operations ("+", "-", "*", "/", "%", "divmod()", "pow()", "**",\n "<<", ">>", "&", "^", "|") with reflected (swapped) operands.\n These functions are only called if the left operand does not\n support the corresponding operation and the operands are of\n different types. [2] For instance, to evaluate the expression "x -\n y", where *y* is an instance of a class that has an "__rsub__()"\n method, "y.__rsub__(x)" is called if "x.__sub__(y)" returns\n *NotImplemented*.\n\n Note that ternary "pow()" will not try calling "__rpow__()" (the\n coercion rules would become too complicated).\n\n Note: If the right operand\'s type is a subclass of the left\n operand\'s type and that subclass provides the reflected method\n for the operation, this method will be called before the left\n operand\'s non-reflected method. This behavior allows subclasses\n to override their ancestors\' operations.\n\nobject.__iadd__(self, other)\nobject.__isub__(self, other)\nobject.__imul__(self, other)\nobject.__idiv__(self, other)\nobject.__itruediv__(self, other)\nobject.__ifloordiv__(self, other)\nobject.__imod__(self, other)\nobject.__ipow__(self, other[, modulo])\nobject.__ilshift__(self, other)\nobject.__irshift__(self, other)\nobject.__iand__(self, other)\nobject.__ixor__(self, other)\nobject.__ior__(self, other)\n\n These methods are called to implement the augmented arithmetic\n assignments ("+=", "-=", "*=", "/=", "//=", "%=", "**=", "<<=",\n ">>=", "&=", "^=", "|="). These methods should attempt to do the\n operation in-place (modifying *self*) and return the result (which\n could be, but does not have to be, *self*). If a specific method\n is not defined, the augmented assignment falls back to the normal\n methods. For instance, to execute the statement "x += y", where\n *x* is an instance of a class that has an "__iadd__()" method,\n "x.__iadd__(y)" is called. If *x* is an instance of a class that\n does not define a "__iadd__()" method, "x.__add__(y)" and\n "y.__radd__(x)" are considered, as with the evaluation of "x + y".\n\nobject.__neg__(self)\nobject.__pos__(self)\nobject.__abs__(self)\nobject.__invert__(self)\n\n Called to implement the unary arithmetic operations ("-", "+",\n "abs()" and "~").\n\nobject.__complex__(self)\nobject.__int__(self)\nobject.__long__(self)\nobject.__float__(self)\n\n Called to implement the built-in functions "complex()", "int()",\n "long()", and "float()". Should return a value of the appropriate\n type.\n\nobject.__oct__(self)\nobject.__hex__(self)\n\n Called to implement the built-in functions "oct()" and "hex()".\n Should return a string value.\n\nobject.__index__(self)\n\n Called to implement "operator.index()". Also called whenever\n Python needs an integer object (such as in slicing). Must return\n an integer (int or long).\n\n New in version 2.5.\n\nobject.__coerce__(self, other)\n\n Called to implement "mixed-mode" numeric arithmetic. Should either\n return a 2-tuple containing *self* and *other* converted to a\n common numeric type, or "None" if conversion is impossible. When\n the common type would be the type of "other", it is sufficient to\n return "None", since the interpreter will also ask the other object\n to attempt a coercion (but sometimes, if the implementation of the\n other type cannot be changed, it is useful to do the conversion to\n the other type here). A return value of "NotImplemented" is\n equivalent to returning "None".\n\n\nCoercion rules\n==============\n\nThis section used to document the rules for coercion. As the language\nhas evolved, the coercion rules have become hard to document\nprecisely; documenting what one version of one particular\nimplementation does is undesirable. Instead, here are some informal\nguidelines regarding coercion. In Python 3, coercion will not be\nsupported.\n\n* If the left operand of a % operator is a string or Unicode object,\n no coercion takes place and the string formatting operation is\n invoked instead.\n\n* It is no longer recommended to define a coercion operation. Mixed-\n mode operations on types that don\'t define coercion pass the\n original arguments to the operation.\n\n* New-style classes (those derived from "object") never invoke the\n "__coerce__()" method in response to a binary operator; the only\n time "__coerce__()" is invoked is when the built-in function\n "coerce()" is called.\n\n* For most intents and purposes, an operator that returns\n "NotImplemented" is treated the same as one that is not implemented\n at all.\n\n* Below, "__op__()" and "__rop__()" are used to signify the generic\n method names corresponding to an operator; "__iop__()" is used for\n the corresponding in-place operator. For example, for the operator\n \'"+"\', "__add__()" and "__radd__()" are used for the left and right\n variant of the binary operator, and "__iadd__()" for the in-place\n variant.\n\n* For objects *x* and *y*, first "x.__op__(y)" is tried. If this is\n not implemented or returns "NotImplemented", "y.__rop__(x)" is\n tried. If this is also not implemented or returns "NotImplemented",\n a "TypeError" exception is raised. But see the following exception:\n\n* Exception to the previous item: if the left operand is an instance\n of a built-in type or a new-style class, and the right operand is an\n instance of a proper subclass of that type or class and overrides\n the base\'s "__rop__()" method, the right operand\'s "__rop__()"\n method is tried *before* the left operand\'s "__op__()" method.\n\n This is done so that a subclass can completely override binary\n operators. Otherwise, the left operand\'s "__op__()" method would\n always accept the right operand: when an instance of a given class\n is expected, an instance of a subclass of that class is always\n acceptable.\n\n* When either operand type defines a coercion, this coercion is\n called before that type\'s "__op__()" or "__rop__()" method is\n called, but no sooner. If the coercion returns an object of a\n different type for the operand whose coercion is invoked, part of\n the process is redone using the new object.\n\n* When an in-place operator (like \'"+="\') is used, if the left\n operand implements "__iop__()", it is invoked without any coercion.\n When the operation falls back to "__op__()" and/or "__rop__()", the\n normal coercion rules apply.\n\n* In "x + y", if *x* is a sequence that implements sequence\n concatenation, sequence concatenation is invoked.\n\n* In "x * y", if one operand is a sequence that implements sequence\n repetition, and the other is an integer ("int" or "long"), sequence\n repetition is invoked.\n\n* Rich comparisons (implemented by methods "__eq__()" and so on)\n never use coercion. Three-way comparison (implemented by\n "__cmp__()") does use coercion under the same conditions as other\n binary operations use it.\n\n* In the current implementation, the built-in numeric types "int",\n "long", "float", and "complex" do not use coercion. All these types\n implement a "__coerce__()" method, for use by the built-in\n "coerce()" function.\n\n Changed in version 2.7: The complex type no longer makes implicit\n calls to the "__coerce__()" method for mixed-type binary arithmetic\n operations.\n\n\nWith Statement Context Managers\n===============================\n\nNew in version 2.5.\n\nA *context manager* is an object that defines the runtime context to\nbe established when executing a "with" statement. The context manager\nhandles the entry into, and the exit from, the desired runtime context\nfor the execution of the block of code. Context managers are normally\ninvoked using the "with" statement (described in section The with\nstatement), but can also be used by directly invoking their methods.\n\nTypical uses of context managers include saving and restoring various\nkinds of global state, locking and unlocking resources, closing opened\nfiles, etc.\n\nFor more information on context managers, see Context Manager Types.\n\nobject.__enter__(self)\n\n Enter the runtime context related to this object. The "with"\n statement will bind this method\'s return value to the target(s)\n specified in the "as" clause of the statement, if any.\n\nobject.__exit__(self, exc_type, exc_value, traceback)\n\n Exit the runtime context related to this object. The parameters\n describe the exception that caused the context to be exited. If the\n context was exited without an exception, all three arguments will\n be "None".\n\n If an exception is supplied, and the method wishes to suppress the\n exception (i.e., prevent it from being propagated), it should\n return a true value. Otherwise, the exception will be processed\n normally upon exit from this method.\n\n Note that "__exit__()" methods should not reraise the passed-in\n exception; this is the caller\'s responsibility.\n\nSee also:\n\n **PEP 0343** - The "with" statement\n The specification, background, and examples for the Python "with"\n statement.\n\n\nSpecial method lookup for old-style classes\n===========================================\n\nFor old-style classes, special methods are always looked up in exactly\nthe same way as any other method or attribute. This is the case\nregardless of whether the method is being looked up explicitly as in\n"x.__getitem__(i)" or implicitly as in "x[i]".\n\nThis behaviour means that special methods may exhibit different\nbehaviour for different instances of a single old-style class if the\nappropriate special attributes are set differently:\n\n >>> class C:\n ... pass\n ...\n >>> c1 = C()\n >>> c2 = C()\n >>> c1.__len__ = lambda: 5\n >>> c2.__len__ = lambda: 9\n >>> len(c1)\n 5\n >>> len(c2)\n 9\n\n\nSpecial method lookup for new-style classes\n===========================================\n\nFor new-style classes, implicit invocations of special methods are\nonly guaranteed to work correctly if defined on an object\'s type, not\nin the object\'s instance dictionary. That behaviour is the reason why\nthe following code raises an exception (unlike the equivalent example\nwith old-style classes):\n\n >>> class C(object):\n ... pass\n ...\n >>> c = C()\n >>> c.__len__ = lambda: 5\n >>> len(c)\n Traceback (most recent call last):\n File "", line 1, in \n TypeError: object of type \'C\' has no len()\n\nThe rationale behind this behaviour lies with a number of special\nmethods such as "__hash__()" and "__repr__()" that are implemented by\nall objects, including type objects. If the implicit lookup of these\nmethods used the conventional lookup process, they would fail when\ninvoked on the type object itself:\n\n >>> 1 .__hash__() == hash(1)\n True\n >>> int.__hash__() == hash(int)\n Traceback (most recent call last):\n File "", line 1, in \n TypeError: descriptor \'__hash__\' of \'int\' object needs an argument\n\nIncorrectly attempting to invoke an unbound method of a class in this\nway is sometimes referred to as \'metaclass confusion\', and is avoided\nby bypassing the instance when looking up special methods:\n\n >>> type(1).__hash__(1) == hash(1)\n True\n >>> type(int).__hash__(int) == hash(int)\n True\n\nIn addition to bypassing any instance attributes in the interest of\ncorrectness, implicit special method lookup generally also bypasses\nthe "__getattribute__()" method even of the object\'s metaclass:\n\n >>> class Meta(type):\n ... def __getattribute__(*args):\n ... print "Metaclass getattribute invoked"\n ... return type.__getattribute__(*args)\n ...\n >>> class C(object):\n ... __metaclass__ = Meta\n ... def __len__(self):\n ... return 10\n ... def __getattribute__(*args):\n ... print "Class getattribute invoked"\n ... return object.__getattribute__(*args)\n ...\n >>> c = C()\n >>> c.__len__() # Explicit lookup via instance\n Class getattribute invoked\n 10\n >>> type(c).__len__(c) # Explicit lookup via type\n Metaclass getattribute invoked\n 10\n >>> len(c) # Implicit lookup\n 10\n\nBypassing the "__getattribute__()" machinery in this fashion provides\nsignificant scope for speed optimisations within the interpreter, at\nthe cost of some flexibility in the handling of special methods (the\nspecial method *must* be set on the class object itself in order to be\nconsistently invoked by the interpreter).\n\n-[ Footnotes ]-\n\n[1] It *is* possible in some cases to change an object\'s type,\n under certain controlled conditions. It generally isn\'t a good\n idea though, since it can lead to some very strange behaviour if\n it is handled incorrectly.\n\n[2] For operands of the same type, it is assumed that if the non-\n reflected method (such as "__add__()") fails the operation is not\n supported, which is why the reflected method is not called.\n', + 'string-methods': u'\nString Methods\n**************\n\nBelow are listed the string methods which both 8-bit strings and\nUnicode objects support. Some of them are also available on\n"bytearray" objects.\n\nIn addition, Python\'s strings support the sequence type methods\ndescribed in the Sequence Types --- str, unicode, list, tuple,\nbytearray, buffer, xrange section. To output formatted strings use\ntemplate strings or the "%" operator described in the String\nFormatting Operations section. Also, see the "re" module for string\nfunctions based on regular expressions.\n\nstr.capitalize()\n\n Return a copy of the string with its first character capitalized\n and the rest lowercased.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.center(width[, fillchar])\n\n Return centered in a string of length *width*. Padding is done\n using the specified *fillchar* (default is a space).\n\n Changed in version 2.4: Support for the *fillchar* argument.\n\nstr.count(sub[, start[, end]])\n\n Return the number of non-overlapping occurrences of substring *sub*\n in the range [*start*, *end*]. Optional arguments *start* and\n *end* are interpreted as in slice notation.\n\nstr.decode([encoding[, errors]])\n\n Decodes the string using the codec registered for *encoding*.\n *encoding* defaults to the default string encoding. *errors* may\n be given to set a different error handling scheme. The default is\n "\'strict\'", meaning that encoding errors raise "UnicodeError".\n Other possible values are "\'ignore\'", "\'replace\'" and any other\n name registered via "codecs.register_error()", see section Codec\n Base Classes.\n\n New in version 2.2.\n\n Changed in version 2.3: Support for other error handling schemes\n added.\n\n Changed in version 2.7: Support for keyword arguments added.\n\nstr.encode([encoding[, errors]])\n\n Return an encoded version of the string. Default encoding is the\n current default string encoding. *errors* may be given to set a\n different error handling scheme. The default for *errors* is\n "\'strict\'", meaning that encoding errors raise a "UnicodeError".\n Other possible values are "\'ignore\'", "\'replace\'",\n "\'xmlcharrefreplace\'", "\'backslashreplace\'" and any other name\n registered via "codecs.register_error()", see section Codec Base\n Classes. For a list of possible encodings, see section Standard\n Encodings.\n\n New in version 2.0.\n\n Changed in version 2.3: Support for "\'xmlcharrefreplace\'" and\n "\'backslashreplace\'" and other error handling schemes added.\n\n Changed in version 2.7: Support for keyword arguments added.\n\nstr.endswith(suffix[, start[, end]])\n\n Return "True" if the string ends with the specified *suffix*,\n otherwise return "False". *suffix* can also be a tuple of suffixes\n to look for. With optional *start*, test beginning at that\n position. With optional *end*, stop comparing at that position.\n\n Changed in version 2.5: Accept tuples as *suffix*.\n\nstr.expandtabs([tabsize])\n\n Return a copy of the string where all tab characters are replaced\n by one or more spaces, depending on the current column and the\n given tab size. Tab positions occur every *tabsize* characters\n (default is 8, giving tab positions at columns 0, 8, 16 and so on).\n To expand the string, the current column is set to zero and the\n string is examined character by character. If the character is a\n tab ("\\t"), one or more space characters are inserted in the result\n until the current column is equal to the next tab position. (The\n tab character itself is not copied.) If the character is a newline\n ("\\n") or return ("\\r"), it is copied and the current column is\n reset to zero. Any other character is copied unchanged and the\n current column is incremented by one regardless of how the\n character is represented when printed.\n\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs()\n \'01 012 0123 01234\'\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs(4)\n \'01 012 0123 01234\'\n\nstr.find(sub[, start[, end]])\n\n Return the lowest index in the string where substring *sub* is\n found within the slice "s[start:end]". Optional arguments *start*\n and *end* are interpreted as in slice notation. Return "-1" if\n *sub* is not found.\n\n Note: The "find()" method should be used only if you need to know\n the position of *sub*. To check if *sub* is a substring or not,\n use the "in" operator:\n\n >>> \'Py\' in \'Python\'\n True\n\nstr.format(*args, **kwargs)\n\n Perform a string formatting operation. The string on which this\n method is called can contain literal text or replacement fields\n delimited by braces "{}". Each replacement field contains either\n the numeric index of a positional argument, or the name of a\n keyword argument. Returns a copy of the string where each\n replacement field is replaced with the string value of the\n corresponding argument.\n\n >>> "The sum of 1 + 2 is {0}".format(1+2)\n \'The sum of 1 + 2 is 3\'\n\n See Format String Syntax for a description of the various\n formatting options that can be specified in format strings.\n\n This method of string formatting is the new standard in Python 3,\n and should be preferred to the "%" formatting described in String\n Formatting Operations in new code.\n\n New in version 2.6.\n\nstr.index(sub[, start[, end]])\n\n Like "find()", but raise "ValueError" when the substring is not\n found.\n\nstr.isalnum()\n\n Return true if all characters in the string are alphanumeric and\n there is at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isalpha()\n\n Return true if all characters in the string are alphabetic and\n there is at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isdigit()\n\n Return true if all characters in the string are digits and there is\n at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.islower()\n\n Return true if all cased characters [4] in the string are lowercase\n and there is at least one cased character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isspace()\n\n Return true if there are only whitespace characters in the string\n and there is at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.istitle()\n\n Return true if the string is a titlecased string and there is at\n least one character, for example uppercase characters may only\n follow uncased characters and lowercase characters only cased ones.\n Return false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isupper()\n\n Return true if all cased characters [4] in the string are uppercase\n and there is at least one cased character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.join(iterable)\n\n Return a string which is the concatenation of the strings in the\n *iterable* *iterable*. The separator between elements is the\n string providing this method.\n\nstr.ljust(width[, fillchar])\n\n Return the string left justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is a\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\n Changed in version 2.4: Support for the *fillchar* argument.\n\nstr.lower()\n\n Return a copy of the string with all the cased characters [4]\n converted to lowercase.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.lstrip([chars])\n\n Return a copy of the string with leading characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a prefix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.lstrip()\n \'spacious \'\n >>> \'www.example.com\'.lstrip(\'cmowz.\')\n \'example.com\'\n\n Changed in version 2.2.2: Support for the *chars* argument.\n\nstr.partition(sep)\n\n Split the string at the first occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing the string itself, followed by\n two empty strings.\n\n New in version 2.5.\n\nstr.replace(old, new[, count])\n\n Return a copy of the string with all occurrences of substring *old*\n replaced by *new*. If the optional argument *count* is given, only\n the first *count* occurrences are replaced.\n\nstr.rfind(sub[, start[, end]])\n\n Return the highest index in the string where substring *sub* is\n found, such that *sub* is contained within "s[start:end]".\n Optional arguments *start* and *end* are interpreted as in slice\n notation. Return "-1" on failure.\n\nstr.rindex(sub[, start[, end]])\n\n Like "rfind()" but raises "ValueError" when the substring *sub* is\n not found.\n\nstr.rjust(width[, fillchar])\n\n Return the string right justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is a\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\n Changed in version 2.4: Support for the *fillchar* argument.\n\nstr.rpartition(sep)\n\n Split the string at the last occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing two empty strings, followed by\n the string itself.\n\n New in version 2.5.\n\nstr.rsplit([sep[, maxsplit]])\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit* splits\n are done, the *rightmost* ones. If *sep* is not specified or\n "None", any whitespace string is a separator. Except for splitting\n from the right, "rsplit()" behaves like "split()" which is\n described in detail below.\n\n New in version 2.4.\n\nstr.rstrip([chars])\n\n Return a copy of the string with trailing characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a suffix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.rstrip()\n \' spacious\'\n >>> \'mississippi\'.rstrip(\'ipz\')\n \'mississ\'\n\n Changed in version 2.2.2: Support for the *chars* argument.\n\nstr.split([sep[, maxsplit]])\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit*\n splits are done (thus, the list will have at most "maxsplit+1"\n elements). If *maxsplit* is not specified or "-1", then there is\n no limit on the number of splits (all possible splits are made).\n\n If *sep* is given, consecutive delimiters are not grouped together\n and are deemed to delimit empty strings (for example,\n "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', \'2\']"). The *sep* argument\n may consist of multiple characters (for example,\n "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', \'3\']"). Splitting an\n empty string with a specified separator returns "[\'\']".\n\n If *sep* is not specified or is "None", a different splitting\n algorithm is applied: runs of consecutive whitespace are regarded\n as a single separator, and the result will contain no empty strings\n at the start or end if the string has leading or trailing\n whitespace. Consequently, splitting an empty string or a string\n consisting of just whitespace with a "None" separator returns "[]".\n\n For example, "\' 1 2 3 \'.split()" returns "[\'1\', \'2\', \'3\']", and\n "\' 1 2 3 \'.split(None, 1)" returns "[\'1\', \'2 3 \']".\n\nstr.splitlines([keepends])\n\n Return a list of the lines in the string, breaking at line\n boundaries. This method uses the *universal newlines* approach to\n splitting lines. Line breaks are not included in the resulting list\n unless *keepends* is given and true.\n\n For example, "\'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines()" returns "[\'ab\n c\', \'\', \'de fg\', \'kl\']", while the same call with\n "splitlines(True)" returns "[\'ab c\\n\', \'\\n\', \'de fg\\r\', \'kl\\r\\n\']".\n\n Unlike "split()" when a delimiter string *sep* is given, this\n method returns an empty list for the empty string, and a terminal\n line break does not result in an extra line.\n\nstr.startswith(prefix[, start[, end]])\n\n Return "True" if string starts with the *prefix*, otherwise return\n "False". *prefix* can also be a tuple of prefixes to look for.\n With optional *start*, test string beginning at that position.\n With optional *end*, stop comparing string at that position.\n\n Changed in version 2.5: Accept tuples as *prefix*.\n\nstr.strip([chars])\n\n Return a copy of the string with the leading and trailing\n characters removed. The *chars* argument is a string specifying the\n set of characters to be removed. If omitted or "None", the *chars*\n argument defaults to removing whitespace. The *chars* argument is\n not a prefix or suffix; rather, all combinations of its values are\n stripped:\n\n >>> \' spacious \'.strip()\n \'spacious\'\n >>> \'www.example.com\'.strip(\'cmowz.\')\n \'example\'\n\n Changed in version 2.2.2: Support for the *chars* argument.\n\nstr.swapcase()\n\n Return a copy of the string with uppercase characters converted to\n lowercase and vice versa.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.title()\n\n Return a titlecased version of the string where words start with an\n uppercase character and the remaining characters are lowercase.\n\n The algorithm uses a simple language-independent definition of a\n word as groups of consecutive letters. The definition works in\n many contexts but it means that apostrophes in contractions and\n possessives form word boundaries, which may not be the desired\n result:\n\n >>> "they\'re bill\'s friends from the UK".title()\n "They\'Re Bill\'S Friends From The Uk"\n\n A workaround for apostrophes can be constructed using regular\n expressions:\n\n >>> import re\n >>> def titlecase(s):\n ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n ... lambda mo: mo.group(0)[0].upper() +\n ... mo.group(0)[1:].lower(),\n ... s)\n ...\n >>> titlecase("they\'re bill\'s friends.")\n "They\'re Bill\'s Friends."\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.translate(table[, deletechars])\n\n Return a copy of the string where all characters occurring in the\n optional argument *deletechars* are removed, and the remaining\n characters have been mapped through the given translation table,\n which must be a string of length 256.\n\n You can use the "maketrans()" helper function in the "string"\n module to create a translation table. For string objects, set the\n *table* argument to "None" for translations that only delete\n characters:\n\n >>> \'read this short text\'.translate(None, \'aeiou\')\n \'rd ths shrt txt\'\n\n New in version 2.6: Support for a "None" *table* argument.\n\n For Unicode objects, the "translate()" method does not accept the\n optional *deletechars* argument. Instead, it returns a copy of the\n *s* where all characters have been mapped through the given\n translation table which must be a mapping of Unicode ordinals to\n Unicode ordinals, Unicode strings or "None". Unmapped characters\n are left untouched. Characters mapped to "None" are deleted. Note,\n a more flexible approach is to create a custom character mapping\n codec using the "codecs" module (see "encodings.cp1251" for an\n example).\n\nstr.upper()\n\n Return a copy of the string with all the cased characters [4]\n converted to uppercase. Note that "str.upper().isupper()" might be\n "False" if "s" contains uncased characters or if the Unicode\n category of the resulting character(s) is not "Lu" (Letter,\n uppercase), but e.g. "Lt" (Letter, titlecase).\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.zfill(width)\n\n Return the numeric string left filled with zeros in a string of\n length *width*. A sign prefix is handled correctly. The original\n string is returned if *width* is less than or equal to "len(s)".\n\n New in version 2.2.2.\n\nThe following methods are present only on unicode objects:\n\nunicode.isnumeric()\n\n Return "True" if there are only numeric characters in S, "False"\n otherwise. Numeric characters include digit characters, and all\n characters that have the Unicode numeric value property, e.g.\n U+2155, VULGAR FRACTION ONE FIFTH.\n\nunicode.isdecimal()\n\n Return "True" if there are only decimal characters in S, "False"\n otherwise. Decimal characters include digit characters, and all\n characters that can be used to form decimal-radix numbers, e.g.\n U+0660, ARABIC-INDIC DIGIT ZERO.\n', + 'strings': u'\nString literals\n***************\n\nString literals are described by the following lexical definitions:\n\n stringliteral ::= [stringprefix](shortstring | longstring)\n stringprefix ::= "r" | "u" | "ur" | "R" | "U" | "UR" | "Ur" | "uR"\n | "b" | "B" | "br" | "Br" | "bR" | "BR"\n shortstring ::= "\'" shortstringitem* "\'" | \'"\' shortstringitem* \'"\'\n longstring ::= "\'\'\'" longstringitem* "\'\'\'"\n | \'"""\' longstringitem* \'"""\'\n shortstringitem ::= shortstringchar | escapeseq\n longstringitem ::= longstringchar | escapeseq\n shortstringchar ::= \n longstringchar ::= \n escapeseq ::= "\\" \n\nOne syntactic restriction not indicated by these productions is that\nwhitespace is not allowed between the "stringprefix" and the rest of\nthe string literal. The source character set is defined by the\nencoding declaration; it is ASCII if no encoding declaration is given\nin the source file; see section Encoding declarations.\n\nIn plain English: String literals can be enclosed in matching single\nquotes ("\'") or double quotes ("""). They can also be enclosed in\nmatching groups of three single or double quotes (these are generally\nreferred to as *triple-quoted strings*). The backslash ("\\")\ncharacter is used to escape characters that otherwise have a special\nmeaning, such as newline, backslash itself, or the quote character.\nString literals may optionally be prefixed with a letter "\'r\'" or\n"\'R\'"; such strings are called *raw strings* and use different rules\nfor interpreting backslash escape sequences. A prefix of "\'u\'" or\n"\'U\'" makes the string a Unicode string. Unicode strings use the\nUnicode character set as defined by the Unicode Consortium and ISO\n10646. Some additional escape sequences, described below, are\navailable in Unicode strings. A prefix of "\'b\'" or "\'B\'" is ignored in\nPython 2; it indicates that the literal should become a bytes literal\nin Python 3 (e.g. when code is automatically converted with 2to3). A\n"\'u\'" or "\'b\'" prefix may be followed by an "\'r\'" prefix.\n\nIn triple-quoted strings, unescaped newlines and quotes are allowed\n(and are retained), except that three unescaped quotes in a row\nterminate the string. (A "quote" is the character used to open the\nstring, i.e. either "\'" or """.)\n\nUnless an "\'r\'" or "\'R\'" prefix is present, escape sequences in\nstrings are interpreted according to rules similar to those used by\nStandard C. The recognized escape sequences are:\n\n+-------------------+-----------------------------------+---------+\n| Escape Sequence | Meaning | Notes |\n+===================+===================================+=========+\n| "\\newline" | Ignored | |\n+-------------------+-----------------------------------+---------+\n| "\\\\" | Backslash ("\\") | |\n+-------------------+-----------------------------------+---------+\n| "\\\'" | Single quote ("\'") | |\n+-------------------+-----------------------------------+---------+\n| "\\"" | Double quote (""") | |\n+-------------------+-----------------------------------+---------+\n| "\\a" | ASCII Bell (BEL) | |\n+-------------------+-----------------------------------+---------+\n| "\\b" | ASCII Backspace (BS) | |\n+-------------------+-----------------------------------+---------+\n| "\\f" | ASCII Formfeed (FF) | |\n+-------------------+-----------------------------------+---------+\n| "\\n" | ASCII Linefeed (LF) | |\n+-------------------+-----------------------------------+---------+\n| "\\N{name}" | Character named *name* in the | |\n| | Unicode database (Unicode only) | |\n+-------------------+-----------------------------------+---------+\n| "\\r" | ASCII Carriage Return (CR) | |\n+-------------------+-----------------------------------+---------+\n| "\\t" | ASCII Horizontal Tab (TAB) | |\n+-------------------+-----------------------------------+---------+\n| "\\uxxxx" | Character with 16-bit hex value | (1) |\n| | *xxxx* (Unicode only) | |\n+-------------------+-----------------------------------+---------+\n| "\\Uxxxxxxxx" | Character with 32-bit hex value | (2) |\n| | *xxxxxxxx* (Unicode only) | |\n+-------------------+-----------------------------------+---------+\n| "\\v" | ASCII Vertical Tab (VT) | |\n+-------------------+-----------------------------------+---------+\n| "\\ooo" | Character with octal value *ooo* | (3,5) |\n+-------------------+-----------------------------------+---------+\n| "\\xhh" | Character with hex value *hh* | (4,5) |\n+-------------------+-----------------------------------+---------+\n\nNotes:\n\n1. Individual code units which form parts of a surrogate pair can\n be encoded using this escape sequence.\n\n2. Any Unicode character can be encoded this way, but characters\n outside the Basic Multilingual Plane (BMP) will be encoded using a\n surrogate pair if Python is compiled to use 16-bit code units (the\n default).\n\n3. As in Standard C, up to three octal digits are accepted.\n\n4. Unlike in Standard C, exactly two hex digits are required.\n\n5. In a string literal, hexadecimal and octal escapes denote the\n byte with the given value; it is not necessary that the byte\n encodes a character in the source character set. In a Unicode\n literal, these escapes denote a Unicode character with the given\n value.\n\nUnlike Standard C, all unrecognized escape sequences are left in the\nstring unchanged, i.e., *the backslash is left in the string*. (This\nbehavior is useful when debugging: if an escape sequence is mistyped,\nthe resulting output is more easily recognized as broken.) It is also\nimportant to note that the escape sequences marked as "(Unicode only)"\nin the table above fall into the category of unrecognized escapes for\nnon-Unicode string literals.\n\nWhen an "\'r\'" or "\'R\'" prefix is present, a character following a\nbackslash is included in the string without change, and *all\nbackslashes are left in the string*. For example, the string literal\n"r"\\n"" consists of two characters: a backslash and a lowercase "\'n\'".\nString quotes can be escaped with a backslash, but the backslash\nremains in the string; for example, "r"\\""" is a valid string literal\nconsisting of two characters: a backslash and a double quote; "r"\\""\nis not a valid string literal (even a raw string cannot end in an odd\nnumber of backslashes). Specifically, *a raw string cannot end in a\nsingle backslash* (since the backslash would escape the following\nquote character). Note also that a single backslash followed by a\nnewline is interpreted as those two characters as part of the string,\n*not* as a line continuation.\n\nWhen an "\'r\'" or "\'R\'" prefix is used in conjunction with a "\'u\'" or\n"\'U\'" prefix, then the "\\uXXXX" and "\\UXXXXXXXX" escape sequences are\nprocessed while *all other backslashes are left in the string*. For\nexample, the string literal "ur"\\u0062\\n"" consists of three Unicode\ncharacters: \'LATIN SMALL LETTER B\', \'REVERSE SOLIDUS\', and \'LATIN\nSMALL LETTER N\'. Backslashes can be escaped with a preceding\nbackslash; however, both remain in the string. As a result, "\\uXXXX"\nescape sequences are only recognized when there are an odd number of\nbackslashes.\n', + 'subscriptions': u'\nSubscriptions\n*************\n\nA subscription selects an item of a sequence (string, tuple or list)\nor mapping (dictionary) object:\n\n subscription ::= primary "[" expression_list "]"\n\nThe primary must evaluate to an object of a sequence or mapping type.\n\nIf the primary is a mapping, the expression list must evaluate to an\nobject whose value is one of the keys of the mapping, and the\nsubscription selects the value in the mapping that corresponds to that\nkey. (The expression list is a tuple except if it has exactly one\nitem.)\n\nIf the primary is a sequence, the expression (list) must evaluate to a\nplain integer. If this value is negative, the length of the sequence\nis added to it (so that, e.g., "x[-1]" selects the last item of "x".)\nThe resulting value must be a nonnegative integer less than the number\nof items in the sequence, and the subscription selects the item whose\nindex is that value (counting from zero).\n\nA string\'s items are characters. A character is not a separate data\ntype but a string of exactly one character.\n', + 'truth': u'\nTruth Value Testing\n*******************\n\nAny object can be tested for truth value, for use in an "if" or\n"while" condition or as operand of the Boolean operations below. The\nfollowing values are considered false:\n\n* "None"\n\n* "False"\n\n* zero of any numeric type, for example, "0", "0L", "0.0", "0j".\n\n* any empty sequence, for example, "\'\'", "()", "[]".\n\n* any empty mapping, for example, "{}".\n\n* instances of user-defined classes, if the class defines a\n "__nonzero__()" or "__len__()" method, when that method returns the\n integer zero or "bool" value "False". [1]\n\nAll other values are considered true --- so objects of many types are\nalways true.\n\nOperations and built-in functions that have a Boolean result always\nreturn "0" or "False" for false and "1" or "True" for true, unless\notherwise stated. (Important exception: the Boolean operations "or"\nand "and" always return one of their operands.)\n', + 'try': u'\nThe "try" statement\n*******************\n\nThe "try" statement specifies exception handlers and/or cleanup code\nfor a group of statements:\n\n try_stmt ::= try1_stmt | try2_stmt\n try1_stmt ::= "try" ":" suite\n ("except" [expression [("as" | ",") identifier]] ":" suite)+\n ["else" ":" suite]\n ["finally" ":" suite]\n try2_stmt ::= "try" ":" suite\n "finally" ":" suite\n\nChanged in version 2.5: In previous versions of Python,\n"try"..."except"..."finally" did not work. "try"..."except" had to be\nnested in "try"..."finally".\n\nThe "except" clause(s) specify one or more exception handlers. When no\nexception occurs in the "try" clause, no exception handler is\nexecuted. When an exception occurs in the "try" suite, a search for an\nexception handler is started. This search inspects the except clauses\nin turn until one is found that matches the exception. An expression-\nless except clause, if present, must be last; it matches any\nexception. For an except clause with an expression, that expression\nis evaluated, and the clause matches the exception if the resulting\nobject is "compatible" with the exception. An object is compatible\nwith an exception if it is the class or a base class of the exception\nobject, or a tuple containing an item compatible with the exception.\n\nIf no except clause matches the exception, the search for an exception\nhandler continues in the surrounding code and on the invocation stack.\n[1]\n\nIf the evaluation of an expression in the header of an except clause\nraises an exception, the original search for a handler is canceled and\na search starts for the new exception in the surrounding code and on\nthe call stack (it is treated as if the entire "try" statement raised\nthe exception).\n\nWhen a matching except clause is found, the exception is assigned to\nthe target specified in that except clause, if present, and the except\nclause\'s suite is executed. All except clauses must have an\nexecutable block. When the end of this block is reached, execution\ncontinues normally after the entire try statement. (This means that\nif two nested handlers exist for the same exception, and the exception\noccurs in the try clause of the inner handler, the outer handler will\nnot handle the exception.)\n\nBefore an except clause\'s suite is executed, details about the\nexception are assigned to three variables in the "sys" module:\n"sys.exc_type" receives the object identifying the exception;\n"sys.exc_value" receives the exception\'s parameter;\n"sys.exc_traceback" receives a traceback object (see section The\nstandard type hierarchy) identifying the point in the program where\nthe exception occurred. These details are also available through the\n"sys.exc_info()" function, which returns a tuple "(exc_type,\nexc_value, exc_traceback)". Use of the corresponding variables is\ndeprecated in favor of this function, since their use is unsafe in a\nthreaded program. As of Python 1.5, the variables are restored to\ntheir previous values (before the call) when returning from a function\nthat handled an exception.\n\nThe optional "else" clause is executed if and when control flows off\nthe end of the "try" clause. [2] Exceptions in the "else" clause are\nnot handled by the preceding "except" clauses.\n\nIf "finally" is present, it specifies a \'cleanup\' handler. The "try"\nclause is executed, including any "except" and "else" clauses. If an\nexception occurs in any of the clauses and is not handled, the\nexception is temporarily saved. The "finally" clause is executed. If\nthere is a saved exception, it is re-raised at the end of the\n"finally" clause. If the "finally" clause raises another exception or\nexecutes a "return" or "break" statement, the saved exception is\ndiscarded:\n\n >>> def f():\n ... try:\n ... 1/0\n ... finally:\n ... return 42\n ...\n >>> f()\n 42\n\nThe exception information is not available to the program during\nexecution of the "finally" clause.\n\nWhen a "return", "break" or "continue" statement is executed in the\n"try" suite of a "try"..."finally" statement, the "finally" clause is\nalso executed \'on the way out.\' A "continue" statement is illegal in\nthe "finally" clause. (The reason is a problem with the current\nimplementation --- this restriction may be lifted in the future).\n\nThe return value of a function is determined by the last "return"\nstatement executed. Since the "finally" clause always executes, a\n"return" statement executed in the "finally" clause will always be the\nlast one executed:\n\n >>> def foo():\n ... try:\n ... return \'try\'\n ... finally:\n ... return \'finally\'\n ...\n >>> foo()\n \'finally\'\n\nAdditional information on exceptions can be found in section\nExceptions, and information on using the "raise" statement to generate\nexceptions may be found in section The raise statement.\n', + 'types': u'\nThe standard type hierarchy\n***************************\n\nBelow is a list of the types that are built into Python. Extension\nmodules (written in C, Java, or other languages, depending on the\nimplementation) can define additional types. Future versions of\nPython may add types to the type hierarchy (e.g., rational numbers,\nefficiently stored arrays of integers, etc.).\n\nSome of the type descriptions below contain a paragraph listing\n\'special attributes.\' These are attributes that provide access to the\nimplementation and are not intended for general use. Their definition\nmay change in the future.\n\nNone\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name "None". It\n is used to signify the absence of a value in many situations, e.g.,\n it is returned from functions that don\'t explicitly return\n anything. Its truth value is false.\n\nNotImplemented\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name\n "NotImplemented". Numeric methods and rich comparison methods may\n return this value if they do not implement the operation for the\n operands provided. (The interpreter will then try the reflected\n operation, or some other fallback, depending on the operator.) Its\n truth value is true.\n\nEllipsis\n This type has a single value. There is a single object with this\n value. This object is accessed through the built-in name\n "Ellipsis". It is used to indicate the presence of the "..." syntax\n in a slice. Its truth value is true.\n\n"numbers.Number"\n These are created by numeric literals and returned as results by\n arithmetic operators and arithmetic built-in functions. Numeric\n objects are immutable; once created their value never changes.\n Python numbers are of course strongly related to mathematical\n numbers, but subject to the limitations of numerical representation\n in computers.\n\n Python distinguishes between integers, floating point numbers, and\n complex numbers:\n\n "numbers.Integral"\n These represent elements from the mathematical set of integers\n (positive and negative).\n\n There are three types of integers:\n\n Plain integers\n These represent numbers in the range -2147483648 through\n 2147483647. (The range may be larger on machines with a\n larger natural word size, but not smaller.) When the result\n of an operation would fall outside this range, the result is\n normally returned as a long integer (in some cases, the\n exception "OverflowError" is raised instead). For the\n purpose of shift and mask operations, integers are assumed to\n have a binary, 2\'s complement notation using 32 or more bits,\n and hiding no bits from the user (i.e., all 4294967296\n different bit patterns correspond to different values).\n\n Long integers\n These represent numbers in an unlimited range, subject to\n available (virtual) memory only. For the purpose of shift\n and mask operations, a binary representation is assumed, and\n negative numbers are represented in a variant of 2\'s\n complement which gives the illusion of an infinite string of\n sign bits extending to the left.\n\n Booleans\n These represent the truth values False and True. The two\n objects representing the values "False" and "True" are the\n only Boolean objects. The Boolean type is a subtype of plain\n integers, and Boolean values behave like the values 0 and 1,\n respectively, in almost all contexts, the exception being\n that when converted to a string, the strings ""False"" or\n ""True"" are returned, respectively.\n\n The rules for integer representation are intended to give the\n most meaningful interpretation of shift and mask operations\n involving negative integers and the least surprises when\n switching between the plain and long integer domains. Any\n operation, if it yields a result in the plain integer domain,\n will yield the same result in the long integer domain or when\n using mixed operands. The switch between domains is transparent\n to the programmer.\n\n "numbers.Real" ("float")\n These represent machine-level double precision floating point\n numbers. You are at the mercy of the underlying machine\n architecture (and C or Java implementation) for the accepted\n range and handling of overflow. Python does not support single-\n precision floating point numbers; the savings in processor and\n memory usage that are usually the reason for using these are\n dwarfed by the overhead of using objects in Python, so there is\n no reason to complicate the language with two kinds of floating\n point numbers.\n\n "numbers.Complex"\n These represent complex numbers as a pair of machine-level\n double precision floating point numbers. The same caveats apply\n as for floating point numbers. The real and imaginary parts of a\n complex number "z" can be retrieved through the read-only\n attributes "z.real" and "z.imag".\n\nSequences\n These represent finite ordered sets indexed by non-negative\n numbers. The built-in function "len()" returns the number of items\n of a sequence. When the length of a sequence is *n*, the index set\n contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence *a* is\n selected by "a[i]".\n\n Sequences also support slicing: "a[i:j]" selects all items with\n index *k* such that *i* "<=" *k* "<" *j*. When used as an\n expression, a slice is a sequence of the same type. This implies\n that the index set is renumbered so that it starts at 0.\n\n Some sequences also support "extended slicing" with a third "step"\n parameter: "a[i:j:k]" selects all items of *a* with index *x* where\n "x = i + n*k", *n* ">=" "0" and *i* "<=" *x* "<" *j*.\n\n Sequences are distinguished according to their mutability:\n\n Immutable sequences\n An object of an immutable sequence type cannot change once it is\n created. (If the object contains references to other objects,\n these other objects may be mutable and may be changed; however,\n the collection of objects directly referenced by an immutable\n object cannot change.)\n\n The following types are immutable sequences:\n\n Strings\n The items of a string are characters. There is no separate\n character type; a character is represented by a string of one\n item. Characters represent (at least) 8-bit bytes. The\n built-in functions "chr()" and "ord()" convert between\n characters and nonnegative integers representing the byte\n values. Bytes with the values 0-127 usually represent the\n corresponding ASCII values, but the interpretation of values\n is up to the program. The string data type is also used to\n represent arrays of bytes, e.g., to hold data read from a\n file.\n\n (On systems whose native character set is not ASCII, strings\n may use EBCDIC in their internal representation, provided the\n functions "chr()" and "ord()" implement a mapping between\n ASCII and EBCDIC, and string comparison preserves the ASCII\n order. Or perhaps someone can propose a better rule?)\n\n Unicode\n The items of a Unicode object are Unicode code units. A\n Unicode code unit is represented by a Unicode object of one\n item and can hold either a 16-bit or 32-bit value\n representing a Unicode ordinal (the maximum value for the\n ordinal is given in "sys.maxunicode", and depends on how\n Python is configured at compile time). Surrogate pairs may\n be present in the Unicode object, and will be reported as two\n separate items. The built-in functions "unichr()" and\n "ord()" convert between code units and nonnegative integers\n representing the Unicode ordinals as defined in the Unicode\n Standard 3.0. Conversion from and to other encodings are\n possible through the Unicode method "encode()" and the built-\n in function "unicode()".\n\n Tuples\n The items of a tuple are arbitrary Python objects. Tuples of\n two or more items are formed by comma-separated lists of\n expressions. A tuple of one item (a \'singleton\') can be\n formed by affixing a comma to an expression (an expression by\n itself does not create a tuple, since parentheses must be\n usable for grouping of expressions). An empty tuple can be\n formed by an empty pair of parentheses.\n\n Mutable sequences\n Mutable sequences can be changed after they are created. The\n subscription and slicing notations can be used as the target of\n assignment and "del" (delete) statements.\n\n There are currently two intrinsic mutable sequence types:\n\n Lists\n The items of a list are arbitrary Python objects. Lists are\n formed by placing a comma-separated list of expressions in\n square brackets. (Note that there are no special cases needed\n to form lists of length 0 or 1.)\n\n Byte Arrays\n A bytearray object is a mutable array. They are created by\n the built-in "bytearray()" constructor. Aside from being\n mutable (and hence unhashable), byte arrays otherwise provide\n the same interface and functionality as immutable bytes\n objects.\n\n The extension module "array" provides an additional example of a\n mutable sequence type.\n\nSet types\n These represent unordered, finite sets of unique, immutable\n objects. As such, they cannot be indexed by any subscript. However,\n they can be iterated over, and the built-in function "len()"\n returns the number of items in a set. Common uses for sets are fast\n membership testing, removing duplicates from a sequence, and\n computing mathematical operations such as intersection, union,\n difference, and symmetric difference.\n\n For set elements, the same immutability rules apply as for\n dictionary keys. Note that numeric types obey the normal rules for\n numeric comparison: if two numbers compare equal (e.g., "1" and\n "1.0"), only one of them can be contained in a set.\n\n There are currently two intrinsic set types:\n\n Sets\n These represent a mutable set. They are created by the built-in\n "set()" constructor and can be modified afterwards by several\n methods, such as "add()".\n\n Frozen sets\n These represent an immutable set. They are created by the\n built-in "frozenset()" constructor. As a frozenset is immutable\n and *hashable*, it can be used again as an element of another\n set, or as a dictionary key.\n\nMappings\n These represent finite sets of objects indexed by arbitrary index\n sets. The subscript notation "a[k]" selects the item indexed by "k"\n from the mapping "a"; this can be used in expressions and as the\n target of assignments or "del" statements. The built-in function\n "len()" returns the number of items in a mapping.\n\n There is currently a single intrinsic mapping type:\n\n Dictionaries\n These represent finite sets of objects indexed by nearly\n arbitrary values. The only types of values not acceptable as\n keys are values containing lists or dictionaries or other\n mutable types that are compared by value rather than by object\n identity, the reason being that the efficient implementation of\n dictionaries requires a key\'s hash value to remain constant.\n Numeric types used for keys obey the normal rules for numeric\n comparison: if two numbers compare equal (e.g., "1" and "1.0")\n then they can be used interchangeably to index the same\n dictionary entry.\n\n Dictionaries are mutable; they can be created by the "{...}"\n notation (see section Dictionary displays).\n\n The extension modules "dbm", "gdbm", and "bsddb" provide\n additional examples of mapping types.\n\nCallable types\n These are the types to which the function call operation (see\n section Calls) can be applied:\n\n User-defined functions\n A user-defined function object is created by a function\n definition (see section Function definitions). It should be\n called with an argument list containing the same number of items\n as the function\'s formal parameter list.\n\n Special attributes:\n\n +-------------------------+---------------------------------+-------------+\n | Attribute | Meaning | |\n +=========================+=================================+=============+\n | "__doc__" "func_doc" | The function\'s documentation | Writable |\n | | string, or "None" if | |\n | | unavailable. | |\n +-------------------------+---------------------------------+-------------+\n | "__name__" "func_name" | The function\'s name. | Writable |\n +-------------------------+---------------------------------+-------------+\n | "__module__" | The name of the module the | Writable |\n | | function was defined in, or | |\n | | "None" if unavailable. | |\n +-------------------------+---------------------------------+-------------+\n | "__defaults__" | A tuple containing default | Writable |\n | "func_defaults" | argument values for those | |\n | | arguments that have defaults, | |\n | | or "None" if no arguments have | |\n | | a default value. | |\n +-------------------------+---------------------------------+-------------+\n | "__code__" "func_code" | The code object representing | Writable |\n | | the compiled function body. | |\n +-------------------------+---------------------------------+-------------+\n | "__globals__" | A reference to the dictionary | Read-only |\n | "func_globals" | that holds the function\'s | |\n | | global variables --- the global | |\n | | namespace of the module in | |\n | | which the function was defined. | |\n +-------------------------+---------------------------------+-------------+\n | "__dict__" "func_dict" | The namespace supporting | Writable |\n | | arbitrary function attributes. | |\n +-------------------------+---------------------------------+-------------+\n | "__closure__" | "None" or a tuple of cells that | Read-only |\n | "func_closure" | contain bindings for the | |\n | | function\'s free variables. | |\n +-------------------------+---------------------------------+-------------+\n\n Most of the attributes labelled "Writable" check the type of the\n assigned value.\n\n Changed in version 2.4: "func_name" is now writable.\n\n Changed in version 2.6: The double-underscore attributes\n "__closure__", "__code__", "__defaults__", and "__globals__"\n were introduced as aliases for the corresponding "func_*"\n attributes for forwards compatibility with Python 3.\n\n Function objects also support getting and setting arbitrary\n attributes, which can be used, for example, to attach metadata\n to functions. Regular attribute dot-notation is used to get and\n set such attributes. *Note that the current implementation only\n supports function attributes on user-defined functions. Function\n attributes on built-in functions may be supported in the\n future.*\n\n Additional information about a function\'s definition can be\n retrieved from its code object; see the description of internal\n types below.\n\n User-defined methods\n A user-defined method object combines a class, a class instance\n (or "None") and any callable object (normally a user-defined\n function).\n\n Special read-only attributes: "im_self" is the class instance\n object, "im_func" is the function object; "im_class" is the\n class of "im_self" for bound methods or the class that asked for\n the method for unbound methods; "__doc__" is the method\'s\n documentation (same as "im_func.__doc__"); "__name__" is the\n method name (same as "im_func.__name__"); "__module__" is the\n name of the module the method was defined in, or "None" if\n unavailable.\n\n Changed in version 2.2: "im_self" used to refer to the class\n that defined the method.\n\n Changed in version 2.6: For Python 3 forward-compatibility,\n "im_func" is also available as "__func__", and "im_self" as\n "__self__".\n\n Methods also support accessing (but not setting) the arbitrary\n function attributes on the underlying function object.\n\n User-defined method objects may be created when getting an\n attribute of a class (perhaps via an instance of that class), if\n that attribute is a user-defined function object, an unbound\n user-defined method object, or a class method object. When the\n attribute is a user-defined method object, a new method object\n is only created if the class from which it is being retrieved is\n the same as, or a derived class of, the class stored in the\n original method object; otherwise, the original method object is\n used as it is.\n\n When a user-defined method object is created by retrieving a\n user-defined function object from a class, its "im_self"\n attribute is "None" and the method object is said to be unbound.\n When one is created by retrieving a user-defined function object\n from a class via one of its instances, its "im_self" attribute\n is the instance, and the method object is said to be bound. In\n either case, the new method\'s "im_class" attribute is the class\n from which the retrieval takes place, and its "im_func"\n attribute is the original function object.\n\n When a user-defined method object is created by retrieving\n another method object from a class or instance, the behaviour is\n the same as for a function object, except that the "im_func"\n attribute of the new instance is not the original method object\n but its "im_func" attribute.\n\n When a user-defined method object is created by retrieving a\n class method object from a class or instance, its "im_self"\n attribute is the class itself, and its "im_func" attribute is\n the function object underlying the class method.\n\n When an unbound user-defined method object is called, the\n underlying function ("im_func") is called, with the restriction\n that the first argument must be an instance of the proper class\n ("im_class") or of a derived class thereof.\n\n When a bound user-defined method object is called, the\n underlying function ("im_func") is called, inserting the class\n instance ("im_self") in front of the argument list. For\n instance, when "C" is a class which contains a definition for a\n function "f()", and "x" is an instance of "C", calling "x.f(1)"\n is equivalent to calling "C.f(x, 1)".\n\n When a user-defined method object is derived from a class method\n object, the "class instance" stored in "im_self" will actually\n be the class itself, so that calling either "x.f(1)" or "C.f(1)"\n is equivalent to calling "f(C,1)" where "f" is the underlying\n function.\n\n Note that the transformation from function object to (unbound or\n bound) method object happens each time the attribute is\n retrieved from the class or instance. In some cases, a fruitful\n optimization is to assign the attribute to a local variable and\n call that local variable. Also notice that this transformation\n only happens for user-defined functions; other callable objects\n (and all non-callable objects) are retrieved without\n transformation. It is also important to note that user-defined\n functions which are attributes of a class instance are not\n converted to bound methods; this *only* happens when the\n function is an attribute of the class.\n\n Generator functions\n A function or method which uses the "yield" statement (see\n section The yield statement) is called a *generator function*.\n Such a function, when called, always returns an iterator object\n which can be used to execute the body of the function: calling\n the iterator\'s "next()" method will cause the function to\n execute until it provides a value using the "yield" statement.\n When the function executes a "return" statement or falls off the\n end, a "StopIteration" exception is raised and the iterator will\n have reached the end of the set of values to be returned.\n\n Built-in functions\n A built-in function object is a wrapper around a C function.\n Examples of built-in functions are "len()" and "math.sin()"\n ("math" is a standard built-in module). The number and type of\n the arguments are determined by the C function. Special read-\n only attributes: "__doc__" is the function\'s documentation\n string, or "None" if unavailable; "__name__" is the function\'s\n name; "__self__" is set to "None" (but see the next item);\n "__module__" is the name of the module the function was defined\n in or "None" if unavailable.\n\n Built-in methods\n This is really a different disguise of a built-in function, this\n time containing an object passed to the C function as an\n implicit extra argument. An example of a built-in method is\n "alist.append()", assuming *alist* is a list object. In this\n case, the special read-only attribute "__self__" is set to the\n object denoted by *alist*.\n\n Class Types\n Class types, or "new-style classes," are callable. These\n objects normally act as factories for new instances of\n themselves, but variations are possible for class types that\n override "__new__()". The arguments of the call are passed to\n "__new__()" and, in the typical case, to "__init__()" to\n initialize the new instance.\n\n Classic Classes\n Class objects are described below. When a class object is\n called, a new class instance (also described below) is created\n and returned. This implies a call to the class\'s "__init__()"\n method if it has one. Any arguments are passed on to the\n "__init__()" method. If there is no "__init__()" method, the\n class must be called without arguments.\n\n Class instances\n Class instances are described below. Class instances are\n callable only when the class has a "__call__()" method;\n "x(arguments)" is a shorthand for "x.__call__(arguments)".\n\nModules\n Modules are imported by the "import" statement (see section The\n import statement). A module object has a namespace implemented by a\n dictionary object (this is the dictionary referenced by the\n func_globals attribute of functions defined in the module).\n Attribute references are translated to lookups in this dictionary,\n e.g., "m.x" is equivalent to "m.__dict__["x"]". A module object\n does not contain the code object used to initialize the module\n (since it isn\'t needed once the initialization is done).\n\n Attribute assignment updates the module\'s namespace dictionary,\n e.g., "m.x = 1" is equivalent to "m.__dict__["x"] = 1".\n\n Special read-only attribute: "__dict__" is the module\'s namespace\n as a dictionary object.\n\n **CPython implementation detail:** Because of the way CPython\n clears module dictionaries, the module dictionary will be cleared\n when the module falls out of scope even if the dictionary still has\n live references. To avoid this, copy the dictionary or keep the\n module around while using its dictionary directly.\n\n Predefined (writable) attributes: "__name__" is the module\'s name;\n "__doc__" is the module\'s documentation string, or "None" if\n unavailable; "__file__" is the pathname of the file from which the\n module was loaded, if it was loaded from a file. The "__file__"\n attribute is not present for C modules that are statically linked\n into the interpreter; for extension modules loaded dynamically from\n a shared library, it is the pathname of the shared library file.\n\nClasses\n Both class types (new-style classes) and class objects (old-\n style/classic classes) are typically created by class definitions\n (see section Class definitions). A class has a namespace\n implemented by a dictionary object. Class attribute references are\n translated to lookups in this dictionary, e.g., "C.x" is translated\n to "C.__dict__["x"]" (although for new-style classes in particular\n there are a number of hooks which allow for other means of locating\n attributes). When the attribute name is not found there, the\n attribute search continues in the base classes. For old-style\n classes, the search is depth-first, left-to-right in the order of\n occurrence in the base class list. New-style classes use the more\n complex C3 method resolution order which behaves correctly even in\n the presence of \'diamond\' inheritance structures where there are\n multiple inheritance paths leading back to a common ancestor.\n Additional details on the C3 MRO used by new-style classes can be\n found in the documentation accompanying the 2.3 release at\n https://www.python.org/download/releases/2.3/mro/.\n\n When a class attribute reference (for class "C", say) would yield a\n user-defined function object or an unbound user-defined method\n object whose associated class is either "C" or one of its base\n classes, it is transformed into an unbound user-defined method\n object whose "im_class" attribute is "C". When it would yield a\n class method object, it is transformed into a bound user-defined\n method object whose "im_self" attribute is "C". When it would\n yield a static method object, it is transformed into the object\n wrapped by the static method object. See section Implementing\n Descriptors for another way in which attributes retrieved from a\n class may differ from those actually contained in its "__dict__"\n (note that only new-style classes support descriptors).\n\n Class attribute assignments update the class\'s dictionary, never\n the dictionary of a base class.\n\n A class object can be called (see above) to yield a class instance\n (see below).\n\n Special attributes: "__name__" is the class name; "__module__" is\n the module name in which the class was defined; "__dict__" is the\n dictionary containing the class\'s namespace; "__bases__" is a tuple\n (possibly empty or a singleton) containing the base classes, in the\n order of their occurrence in the base class list; "__doc__" is the\n class\'s documentation string, or None if undefined.\n\nClass instances\n A class instance is created by calling a class object (see above).\n A class instance has a namespace implemented as a dictionary which\n is the first place in which attribute references are searched.\n When an attribute is not found there, and the instance\'s class has\n an attribute by that name, the search continues with the class\n attributes. If a class attribute is found that is a user-defined\n function object or an unbound user-defined method object whose\n associated class is the class (call it "C") of the instance for\n which the attribute reference was initiated or one of its bases, it\n is transformed into a bound user-defined method object whose\n "im_class" attribute is "C" and whose "im_self" attribute is the\n instance. Static method and class method objects are also\n transformed, as if they had been retrieved from class "C"; see\n above under "Classes". See section Implementing Descriptors for\n another way in which attributes of a class retrieved via its\n instances may differ from the objects actually stored in the\n class\'s "__dict__". If no class attribute is found, and the\n object\'s class has a "__getattr__()" method, that is called to\n satisfy the lookup.\n\n Attribute assignments and deletions update the instance\'s\n dictionary, never a class\'s dictionary. If the class has a\n "__setattr__()" or "__delattr__()" method, this is called instead\n of updating the instance dictionary directly.\n\n Class instances can pretend to be numbers, sequences, or mappings\n if they have methods with certain special names. See section\n Special method names.\n\n Special attributes: "__dict__" is the attribute dictionary;\n "__class__" is the instance\'s class.\n\nFiles\n A file object represents an open file. File objects are created by\n the "open()" built-in function, and also by "os.popen()",\n "os.fdopen()", and the "makefile()" method of socket objects (and\n perhaps by other functions or methods provided by extension\n modules). The objects "sys.stdin", "sys.stdout" and "sys.stderr"\n are initialized to file objects corresponding to the interpreter\'s\n standard input, output and error streams. See File Objects for\n complete documentation of file objects.\n\nInternal types\n A few types used internally by the interpreter are exposed to the\n user. Their definitions may change with future versions of the\n interpreter, but they are mentioned here for completeness.\n\n Code objects\n Code objects represent *byte-compiled* executable Python code,\n or *bytecode*. The difference between a code object and a\n function object is that the function object contains an explicit\n reference to the function\'s globals (the module in which it was\n defined), while a code object contains no context; also the\n default argument values are stored in the function object, not\n in the code object (because they represent values calculated at\n run-time). Unlike function objects, code objects are immutable\n and contain no references (directly or indirectly) to mutable\n objects.\n\n Special read-only attributes: "co_name" gives the function name;\n "co_argcount" is the number of positional arguments (including\n arguments with default values); "co_nlocals" is the number of\n local variables used by the function (including arguments);\n "co_varnames" is a tuple containing the names of the local\n variables (starting with the argument names); "co_cellvars" is a\n tuple containing the names of local variables that are\n referenced by nested functions; "co_freevars" is a tuple\n containing the names of free variables; "co_code" is a string\n representing the sequence of bytecode instructions; "co_consts"\n is a tuple containing the literals used by the bytecode;\n "co_names" is a tuple containing the names used by the bytecode;\n "co_filename" is the filename from which the code was compiled;\n "co_firstlineno" is the first line number of the function;\n "co_lnotab" is a string encoding the mapping from bytecode\n offsets to line numbers (for details see the source code of the\n interpreter); "co_stacksize" is the required stack size\n (including local variables); "co_flags" is an integer encoding a\n number of flags for the interpreter.\n\n The following flag bits are defined for "co_flags": bit "0x04"\n is set if the function uses the "*arguments" syntax to accept an\n arbitrary number of positional arguments; bit "0x08" is set if\n the function uses the "**keywords" syntax to accept arbitrary\n keyword arguments; bit "0x20" is set if the function is a\n generator.\n\n Future feature declarations ("from __future__ import division")\n also use bits in "co_flags" to indicate whether a code object\n was compiled with a particular feature enabled: bit "0x2000" is\n set if the function was compiled with future division enabled;\n bits "0x10" and "0x1000" were used in earlier versions of\n Python.\n\n Other bits in "co_flags" are reserved for internal use.\n\n If a code object represents a function, the first item in\n "co_consts" is the documentation string of the function, or\n "None" if undefined.\n\n Frame objects\n Frame objects represent execution frames. They may occur in\n traceback objects (see below).\n\n Special read-only attributes: "f_back" is to the previous stack\n frame (towards the caller), or "None" if this is the bottom\n stack frame; "f_code" is the code object being executed in this\n frame; "f_locals" is the dictionary used to look up local\n variables; "f_globals" is used for global variables;\n "f_builtins" is used for built-in (intrinsic) names;\n "f_restricted" is a flag indicating whether the function is\n executing in restricted execution mode; "f_lasti" gives the\n precise instruction (this is an index into the bytecode string\n of the code object).\n\n Special writable attributes: "f_trace", if not "None", is a\n function called at the start of each source code line (this is\n used by the debugger); "f_exc_type", "f_exc_value",\n "f_exc_traceback" represent the last exception raised in the\n parent frame provided another exception was ever raised in the\n current frame (in all other cases they are None); "f_lineno" is\n the current line number of the frame --- writing to this from\n within a trace function jumps to the given line (only for the\n bottom-most frame). A debugger can implement a Jump command\n (aka Set Next Statement) by writing to f_lineno.\n\n Traceback objects\n Traceback objects represent a stack trace of an exception. A\n traceback object is created when an exception occurs. When the\n search for an exception handler unwinds the execution stack, at\n each unwound level a traceback object is inserted in front of\n the current traceback. When an exception handler is entered,\n the stack trace is made available to the program. (See section\n The try statement.) It is accessible as "sys.exc_traceback", and\n also as the third item of the tuple returned by\n "sys.exc_info()". The latter is the preferred interface, since\n it works correctly when the program is using multiple threads.\n When the program contains no suitable handler, the stack trace\n is written (nicely formatted) to the standard error stream; if\n the interpreter is interactive, it is also made available to the\n user as "sys.last_traceback".\n\n Special read-only attributes: "tb_next" is the next level in the\n stack trace (towards the frame where the exception occurred), or\n "None" if there is no next level; "tb_frame" points to the\n execution frame of the current level; "tb_lineno" gives the line\n number where the exception occurred; "tb_lasti" indicates the\n precise instruction. The line number and last instruction in\n the traceback may differ from the line number of its frame\n object if the exception occurred in a "try" statement with no\n matching except clause or with a finally clause.\n\n Slice objects\n Slice objects are used to represent slices when *extended slice\n syntax* is used. This is a slice using two colons, or multiple\n slices or ellipses separated by commas, e.g., "a[i:j:step]",\n "a[i:j, k:l]", or "a[..., i:j]". They are also created by the\n built-in "slice()" function.\n\n Special read-only attributes: "start" is the lower bound; "stop"\n is the upper bound; "step" is the step value; each is "None" if\n omitted. These attributes can have any type.\n\n Slice objects support one method:\n\n slice.indices(self, length)\n\n This method takes a single integer argument *length* and\n computes information about the extended slice that the slice\n object would describe if applied to a sequence of *length*\n items. It returns a tuple of three integers; respectively\n these are the *start* and *stop* indices and the *step* or\n stride length of the slice. Missing or out-of-bounds indices\n are handled in a manner consistent with regular slices.\n\n New in version 2.3.\n\n Static method objects\n Static method objects provide a way of defeating the\n transformation of function objects to method objects described\n above. A static method object is a wrapper around any other\n object, usually a user-defined method object. When a static\n method object is retrieved from a class or a class instance, the\n object actually returned is the wrapped object, which is not\n subject to any further transformation. Static method objects are\n not themselves callable, although the objects they wrap usually\n are. Static method objects are created by the built-in\n "staticmethod()" constructor.\n\n Class method objects\n A class method object, like a static method object, is a wrapper\n around another object that alters the way in which that object\n is retrieved from classes and class instances. The behaviour of\n class method objects upon such retrieval is described above,\n under "User-defined methods". Class method objects are created\n by the built-in "classmethod()" constructor.\n', + 'typesfunctions': u'\nFunctions\n*********\n\nFunction objects are created by function definitions. The only\noperation on a function object is to call it: "func(argument-list)".\n\nThere are really two flavors of function objects: built-in functions\nand user-defined functions. Both support the same operation (to call\nthe function), but the implementation is different, hence the\ndifferent object types.\n\nSee Function definitions for more information.\n', + 'typesmapping': u'\nMapping Types --- "dict"\n************************\n\nA *mapping* object maps *hashable* values to arbitrary objects.\nMappings are mutable objects. There is currently only one standard\nmapping type, the *dictionary*. (For other containers see the built\nin "list", "set", and "tuple" classes, and the "collections" module.)\n\nA dictionary\'s keys are *almost* arbitrary values. Values that are\nnot *hashable*, that is, values containing lists, dictionaries or\nother mutable types (that are compared by value rather than by object\nidentity) may not be used as keys. Numeric types used for keys obey\nthe normal rules for numeric comparison: if two numbers compare equal\n(such as "1" and "1.0") then they can be used interchangeably to index\nthe same dictionary entry. (Note however, that since computers store\nfloating-point numbers as approximations it is usually unwise to use\nthem as dictionary keys.)\n\nDictionaries can be created by placing a comma-separated list of "key:\nvalue" pairs within braces, for example: "{\'jack\': 4098, \'sjoerd\':\n4127}" or "{4098: \'jack\', 4127: \'sjoerd\'}", or by the "dict"\nconstructor.\n\nclass dict(**kwarg)\nclass dict(mapping, **kwarg)\nclass dict(iterable, **kwarg)\n\n Return a new dictionary initialized from an optional positional\n argument and a possibly empty set of keyword arguments.\n\n If no positional argument is given, an empty dictionary is created.\n If a positional argument is given and it is a mapping object, a\n dictionary is created with the same key-value pairs as the mapping\n object. Otherwise, the positional argument must be an *iterable*\n object. Each item in the iterable must itself be an iterable with\n exactly two objects. The first object of each item becomes a key\n in the new dictionary, and the second object the corresponding\n value. If a key occurs more than once, the last value for that key\n becomes the corresponding value in the new dictionary.\n\n If keyword arguments are given, the keyword arguments and their\n values are added to the dictionary created from the positional\n argument. If a key being added is already present, the value from\n the keyword argument replaces the value from the positional\n argument.\n\n To illustrate, the following examples all return a dictionary equal\n to "{"one": 1, "two": 2, "three": 3}":\n\n >>> a = dict(one=1, two=2, three=3)\n >>> b = {\'one\': 1, \'two\': 2, \'three\': 3}\n >>> c = dict(zip([\'one\', \'two\', \'three\'], [1, 2, 3]))\n >>> d = dict([(\'two\', 2), (\'one\', 1), (\'three\', 3)])\n >>> e = dict({\'three\': 3, \'one\': 1, \'two\': 2})\n >>> a == b == c == d == e\n True\n\n Providing keyword arguments as in the first example only works for\n keys that are valid Python identifiers. Otherwise, any valid keys\n can be used.\n\n New in version 2.2.\n\n Changed in version 2.3: Support for building a dictionary from\n keyword arguments added.\n\n These are the operations that dictionaries support (and therefore,\n custom mapping types should support too):\n\n len(d)\n\n Return the number of items in the dictionary *d*.\n\n d[key]\n\n Return the item of *d* with key *key*. Raises a "KeyError" if\n *key* is not in the map.\n\n If a subclass of dict defines a method "__missing__()" and *key*\n is not present, the "d[key]" operation calls that method with\n the key *key* as argument. The "d[key]" operation then returns\n or raises whatever is returned or raised by the\n "__missing__(key)" call. No other operations or methods invoke\n "__missing__()". If "__missing__()" is not defined, "KeyError"\n is raised. "__missing__()" must be a method; it cannot be an\n instance variable:\n\n >>> class Counter(dict):\n ... def __missing__(self, key):\n ... return 0\n >>> c = Counter()\n >>> c[\'red\']\n 0\n >>> c[\'red\'] += 1\n >>> c[\'red\']\n 1\n\n The example above shows part of the implementation of\n "collections.Counter". A different "__missing__" method is used\n by "collections.defaultdict".\n\n New in version 2.5: Recognition of __missing__ methods of dict\n subclasses.\n\n d[key] = value\n\n Set "d[key]" to *value*.\n\n del d[key]\n\n Remove "d[key]" from *d*. Raises a "KeyError" if *key* is not\n in the map.\n\n key in d\n\n Return "True" if *d* has a key *key*, else "False".\n\n New in version 2.2.\n\n key not in d\n\n Equivalent to "not key in d".\n\n New in version 2.2.\n\n iter(d)\n\n Return an iterator over the keys of the dictionary. This is a\n shortcut for "iterkeys()".\n\n clear()\n\n Remove all items from the dictionary.\n\n copy()\n\n Return a shallow copy of the dictionary.\n\n fromkeys(seq[, value])\n\n Create a new dictionary with keys from *seq* and values set to\n *value*.\n\n "fromkeys()" is a class method that returns a new dictionary.\n *value* defaults to "None".\n\n New in version 2.3.\n\n get(key[, default])\n\n Return the value for *key* if *key* is in the dictionary, else\n *default*. If *default* is not given, it defaults to "None", so\n that this method never raises a "KeyError".\n\n has_key(key)\n\n Test for the presence of *key* in the dictionary. "has_key()"\n is deprecated in favor of "key in d".\n\n items()\n\n Return a copy of the dictionary\'s list of "(key, value)" pairs.\n\n **CPython implementation detail:** Keys and values are listed in\n an arbitrary order which is non-random, varies across Python\n implementations, and depends on the dictionary\'s history of\n insertions and deletions.\n\n If "items()", "keys()", "values()", "iteritems()", "iterkeys()",\n and "itervalues()" are called with no intervening modifications\n to the dictionary, the lists will directly correspond. This\n allows the creation of "(value, key)" pairs using "zip()":\n "pairs = zip(d.values(), d.keys())". The same relationship\n holds for the "iterkeys()" and "itervalues()" methods: "pairs =\n zip(d.itervalues(), d.iterkeys())" provides the same value for\n "pairs". Another way to create the same list is "pairs = [(v, k)\n for (k, v) in d.iteritems()]".\n\n iteritems()\n\n Return an iterator over the dictionary\'s "(key, value)" pairs.\n See the note for "dict.items()".\n\n Using "iteritems()" while adding or deleting entries in the\n dictionary may raise a "RuntimeError" or fail to iterate over\n all entries.\n\n New in version 2.2.\n\n iterkeys()\n\n Return an iterator over the dictionary\'s keys. See the note for\n "dict.items()".\n\n Using "iterkeys()" while adding or deleting entries in the\n dictionary may raise a "RuntimeError" or fail to iterate over\n all entries.\n\n New in version 2.2.\n\n itervalues()\n\n Return an iterator over the dictionary\'s values. See the note\n for "dict.items()".\n\n Using "itervalues()" while adding or deleting entries in the\n dictionary may raise a "RuntimeError" or fail to iterate over\n all entries.\n\n New in version 2.2.\n\n keys()\n\n Return a copy of the dictionary\'s list of keys. See the note\n for "dict.items()".\n\n pop(key[, default])\n\n If *key* is in the dictionary, remove it and return its value,\n else return *default*. If *default* is not given and *key* is\n not in the dictionary, a "KeyError" is raised.\n\n New in version 2.3.\n\n popitem()\n\n Remove and return an arbitrary "(key, value)" pair from the\n dictionary.\n\n "popitem()" is useful to destructively iterate over a\n dictionary, as often used in set algorithms. If the dictionary\n is empty, calling "popitem()" raises a "KeyError".\n\n setdefault(key[, default])\n\n If *key* is in the dictionary, return its value. If not, insert\n *key* with a value of *default* and return *default*. *default*\n defaults to "None".\n\n update([other])\n\n Update the dictionary with the key/value pairs from *other*,\n overwriting existing keys. Return "None".\n\n "update()" accepts either another dictionary object or an\n iterable of key/value pairs (as tuples or other iterables of\n length two). If keyword arguments are specified, the dictionary\n is then updated with those key/value pairs: "d.update(red=1,\n blue=2)".\n\n Changed in version 2.4: Allowed the argument to be an iterable\n of key/value pairs and allowed keyword arguments.\n\n values()\n\n Return a copy of the dictionary\'s list of values. See the note\n for "dict.items()".\n\n viewitems()\n\n Return a new view of the dictionary\'s items ("(key, value)"\n pairs). See below for documentation of view objects.\n\n New in version 2.7.\n\n viewkeys()\n\n Return a new view of the dictionary\'s keys. See below for\n documentation of view objects.\n\n New in version 2.7.\n\n viewvalues()\n\n Return a new view of the dictionary\'s values. See below for\n documentation of view objects.\n\n New in version 2.7.\n\n Dictionaries compare equal if and only if they have the same "(key,\n value)" pairs.\n\n\nDictionary view objects\n=======================\n\nThe objects returned by "dict.viewkeys()", "dict.viewvalues()" and\n"dict.viewitems()" are *view objects*. They provide a dynamic view on\nthe dictionary\'s entries, which means that when the dictionary\nchanges, the view reflects these changes.\n\nDictionary views can be iterated over to yield their respective data,\nand support membership tests:\n\nlen(dictview)\n\n Return the number of entries in the dictionary.\n\niter(dictview)\n\n Return an iterator over the keys, values or items (represented as\n tuples of "(key, value)") in the dictionary.\n\n Keys and values are iterated over in an arbitrary order which is\n non-random, varies across Python implementations, and depends on\n the dictionary\'s history of insertions and deletions. If keys,\n values and items views are iterated over with no intervening\n modifications to the dictionary, the order of items will directly\n correspond. This allows the creation of "(value, key)" pairs using\n "zip()": "pairs = zip(d.values(), d.keys())". Another way to\n create the same list is "pairs = [(v, k) for (k, v) in d.items()]".\n\n Iterating views while adding or deleting entries in the dictionary\n may raise a "RuntimeError" or fail to iterate over all entries.\n\nx in dictview\n\n Return "True" if *x* is in the underlying dictionary\'s keys, values\n or items (in the latter case, *x* should be a "(key, value)"\n tuple).\n\nKeys views are set-like since their entries are unique and hashable.\nIf all values are hashable, so that (key, value) pairs are unique and\nhashable, then the items view is also set-like. (Values views are not\ntreated as set-like since the entries are generally not unique.) Then\nthese set operations are available ("other" refers either to another\nview or a set):\n\ndictview & other\n\n Return the intersection of the dictview and the other object as a\n new set.\n\ndictview | other\n\n Return the union of the dictview and the other object as a new set.\n\ndictview - other\n\n Return the difference between the dictview and the other object\n (all elements in *dictview* that aren\'t in *other*) as a new set.\n\ndictview ^ other\n\n Return the symmetric difference (all elements either in *dictview*\n or *other*, but not in both) of the dictview and the other object\n as a new set.\n\nAn example of dictionary view usage:\n\n >>> dishes = {\'eggs\': 2, \'sausage\': 1, \'bacon\': 1, \'spam\': 500}\n >>> keys = dishes.viewkeys()\n >>> values = dishes.viewvalues()\n\n >>> # iteration\n >>> n = 0\n >>> for val in values:\n ... n += val\n >>> print(n)\n 504\n\n >>> # keys and values are iterated over in the same order\n >>> list(keys)\n [\'eggs\', \'bacon\', \'sausage\', \'spam\']\n >>> list(values)\n [2, 1, 1, 500]\n\n >>> # view objects are dynamic and reflect dict changes\n >>> del dishes[\'eggs\']\n >>> del dishes[\'sausage\']\n >>> list(keys)\n [\'spam\', \'bacon\']\n\n >>> # set operations\n >>> keys & {\'eggs\', \'bacon\', \'salad\'}\n {\'bacon\'}\n', + 'typesmethods': u'\nMethods\n*******\n\nMethods are functions that are called using the attribute notation.\nThere are two flavors: built-in methods (such as "append()" on lists)\nand class instance methods. Built-in methods are described with the\ntypes that support them.\n\nThe implementation adds two special read-only attributes to class\ninstance methods: "m.im_self" is the object on which the method\noperates, and "m.im_func" is the function implementing the method.\nCalling "m(arg-1, arg-2, ..., arg-n)" is completely equivalent to\ncalling "m.im_func(m.im_self, arg-1, arg-2, ..., arg-n)".\n\nClass instance methods are either *bound* or *unbound*, referring to\nwhether the method was accessed through an instance or a class,\nrespectively. When a method is unbound, its "im_self" attribute will\nbe "None" and if called, an explicit "self" object must be passed as\nthe first argument. In this case, "self" must be an instance of the\nunbound method\'s class (or a subclass of that class), otherwise a\n"TypeError" is raised.\n\nLike function objects, methods objects support getting arbitrary\nattributes. However, since method attributes are actually stored on\nthe underlying function object ("meth.im_func"), setting method\nattributes on either bound or unbound methods is disallowed.\nAttempting to set an attribute on a method results in an\n"AttributeError" being raised. In order to set a method attribute,\nyou need to explicitly set it on the underlying function object:\n\n >>> class C:\n ... def method(self):\n ... pass\n ...\n >>> c = C()\n >>> c.method.whoami = \'my name is method\' # can\'t set on the method\n Traceback (most recent call last):\n File "", line 1, in \n AttributeError: \'instancemethod\' object has no attribute \'whoami\'\n >>> c.method.im_func.whoami = \'my name is method\'\n >>> c.method.whoami\n \'my name is method\'\n\nSee The standard type hierarchy for more information.\n', + 'typesmodules': u'\nModules\n*******\n\nThe only special operation on a module is attribute access: "m.name",\nwhere *m* is a module and *name* accesses a name defined in *m*\'s\nsymbol table. Module attributes can be assigned to. (Note that the\n"import" statement is not, strictly speaking, an operation on a module\nobject; "import foo" does not require a module object named *foo* to\nexist, rather it requires an (external) *definition* for a module\nnamed *foo* somewhere.)\n\nA special attribute of every module is "__dict__". This is the\ndictionary containing the module\'s symbol table. Modifying this\ndictionary will actually change the module\'s symbol table, but direct\nassignment to the "__dict__" attribute is not possible (you can write\n"m.__dict__[\'a\'] = 1", which defines "m.a" to be "1", but you can\'t\nwrite "m.__dict__ = {}"). Modifying "__dict__" directly is not\nrecommended.\n\nModules built into the interpreter are written like this: "". If loaded from a file, they are written as\n"".\n', + 'typesseq': u'\nSequence Types --- "str", "unicode", "list", "tuple", "bytearray", "buffer", "xrange"\n*************************************************************************************\n\nThere are seven sequence types: strings, Unicode strings, lists,\ntuples, bytearrays, buffers, and xrange objects.\n\nFor other containers see the built in "dict" and "set" classes, and\nthe "collections" module.\n\nString literals are written in single or double quotes: "\'xyzzy\'",\n""frobozz"". See String literals for more about string literals.\nUnicode strings are much like strings, but are specified in the syntax\nusing a preceding "\'u\'" character: "u\'abc\'", "u"def"". In addition to\nthe functionality described here, there are also string-specific\nmethods described in the String Methods section. Lists are constructed\nwith square brackets, separating items with commas: "[a, b, c]".\nTuples are constructed by the comma operator (not within square\nbrackets), with or without enclosing parentheses, but an empty tuple\nmust have the enclosing parentheses, such as "a, b, c" or "()". A\nsingle item tuple must have a trailing comma, such as "(d,)".\n\nBytearray objects are created with the built-in function\n"bytearray()".\n\nBuffer objects are not directly supported by Python syntax, but can be\ncreated by calling the built-in function "buffer()". They don\'t\nsupport concatenation or repetition.\n\nObjects of type xrange are similar to buffers in that there is no\nspecific syntax to create them, but they are created using the\n"xrange()" function. They don\'t support slicing, concatenation or\nrepetition, and using "in", "not in", "min()" or "max()" on them is\ninefficient.\n\nMost sequence types support the following operations. The "in" and\n"not in" operations have the same priorities as the comparison\noperations. The "+" and "*" operations have the same priority as the\ncorresponding numeric operations. [3] Additional methods are provided\nfor Mutable Sequence Types.\n\nThis table lists the sequence operations sorted in ascending priority.\nIn the table, *s* and *t* are sequences of the same type; *n*, *i* and\n*j* are integers:\n\n+--------------------+----------------------------------+------------+\n| Operation | Result | Notes |\n+====================+==================================+============+\n| "x in s" | "True" if an item of *s* is | (1) |\n| | equal to *x*, else "False" | |\n+--------------------+----------------------------------+------------+\n| "x not in s" | "False" if an item of *s* is | (1) |\n| | equal to *x*, else "True" | |\n+--------------------+----------------------------------+------------+\n| "s + t" | the concatenation of *s* and *t* | (6) |\n+--------------------+----------------------------------+------------+\n| "s * n, n * s" | equivalent to adding *s* to | (2) |\n| | itself *n* times | |\n+--------------------+----------------------------------+------------+\n| "s[i]" | *i*th item of *s*, origin 0 | (3) |\n+--------------------+----------------------------------+------------+\n| "s[i:j]" | slice of *s* from *i* to *j* | (3)(4) |\n+--------------------+----------------------------------+------------+\n| "s[i:j:k]" | slice of *s* from *i* to *j* | (3)(5) |\n| | with step *k* | |\n+--------------------+----------------------------------+------------+\n| "len(s)" | length of *s* | |\n+--------------------+----------------------------------+------------+\n| "min(s)" | smallest item of *s* | |\n+--------------------+----------------------------------+------------+\n| "max(s)" | largest item of *s* | |\n+--------------------+----------------------------------+------------+\n| "s.index(x)" | index of the first occurrence of | |\n| | *x* in *s* | |\n+--------------------+----------------------------------+------------+\n| "s.count(x)" | total number of occurrences of | |\n| | *x* in *s* | |\n+--------------------+----------------------------------+------------+\n\nSequence types also support comparisons. In particular, tuples and\nlists are compared lexicographically by comparing corresponding\nelements. This means that to compare equal, every element must compare\nequal and the two sequences must be of the same type and have the same\nlength. (For full details see Comparisons in the language reference.)\n\nNotes:\n\n1. When *s* is a string or Unicode string object the "in" and "not\n in" operations act like a substring test. In Python versions\n before 2.3, *x* had to be a string of length 1. In Python 2.3 and\n beyond, *x* may be a string of any length.\n\n2. Values of *n* less than "0" are treated as "0" (which yields an\n empty sequence of the same type as *s*). Note that items in the\n sequence *s* are not copied; they are referenced multiple times.\n This often haunts new Python programmers; consider:\n\n >>> lists = [[]] * 3\n >>> lists\n [[], [], []]\n >>> lists[0].append(3)\n >>> lists\n [[3], [3], [3]]\n\n What has happened is that "[[]]" is a one-element list containing\n an empty list, so all three elements of "[[]] * 3" are references\n to this single empty list. Modifying any of the elements of\n "lists" modifies this single list. You can create a list of\n different lists this way:\n\n >>> lists = [[] for i in range(3)]\n >>> lists[0].append(3)\n >>> lists[1].append(5)\n >>> lists[2].append(7)\n >>> lists\n [[3], [5], [7]]\n\n Further explanation is available in the FAQ entry How do I create a\n multidimensional list?.\n\n3. If *i* or *j* is negative, the index is relative to the end of\n the string: "len(s) + i" or "len(s) + j" is substituted. But note\n that "-0" is still "0".\n\n4. The slice of *s* from *i* to *j* is defined as the sequence of\n items with index *k* such that "i <= k < j". If *i* or *j* is\n greater than "len(s)", use "len(s)". If *i* is omitted or "None",\n use "0". If *j* is omitted or "None", use "len(s)". If *i* is\n greater than or equal to *j*, the slice is empty.\n\n5. The slice of *s* from *i* to *j* with step *k* is defined as the\n sequence of items with index "x = i + n*k" such that "0 <= n <\n (j-i)/k". In other words, the indices are "i", "i+k", "i+2*k",\n "i+3*k" and so on, stopping when *j* is reached (but never\n including *j*). If *i* or *j* is greater than "len(s)", use\n "len(s)". If *i* or *j* are omitted or "None", they become "end"\n values (which end depends on the sign of *k*). Note, *k* cannot be\n zero. If *k* is "None", it is treated like "1".\n\n6. **CPython implementation detail:** If *s* and *t* are both\n strings, some Python implementations such as CPython can usually\n perform an in-place optimization for assignments of the form "s = s\n + t" or "s += t". When applicable, this optimization makes\n quadratic run-time much less likely. This optimization is both\n version and implementation dependent. For performance sensitive\n code, it is preferable to use the "str.join()" method which assures\n consistent linear concatenation performance across versions and\n implementations.\n\n Changed in version 2.4: Formerly, string concatenation never\n occurred in-place.\n\n\nString Methods\n==============\n\nBelow are listed the string methods which both 8-bit strings and\nUnicode objects support. Some of them are also available on\n"bytearray" objects.\n\nIn addition, Python\'s strings support the sequence type methods\ndescribed in the Sequence Types --- str, unicode, list, tuple,\nbytearray, buffer, xrange section. To output formatted strings use\ntemplate strings or the "%" operator described in the String\nFormatting Operations section. Also, see the "re" module for string\nfunctions based on regular expressions.\n\nstr.capitalize()\n\n Return a copy of the string with its first character capitalized\n and the rest lowercased.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.center(width[, fillchar])\n\n Return centered in a string of length *width*. Padding is done\n using the specified *fillchar* (default is a space).\n\n Changed in version 2.4: Support for the *fillchar* argument.\n\nstr.count(sub[, start[, end]])\n\n Return the number of non-overlapping occurrences of substring *sub*\n in the range [*start*, *end*]. Optional arguments *start* and\n *end* are interpreted as in slice notation.\n\nstr.decode([encoding[, errors]])\n\n Decodes the string using the codec registered for *encoding*.\n *encoding* defaults to the default string encoding. *errors* may\n be given to set a different error handling scheme. The default is\n "\'strict\'", meaning that encoding errors raise "UnicodeError".\n Other possible values are "\'ignore\'", "\'replace\'" and any other\n name registered via "codecs.register_error()", see section Codec\n Base Classes.\n\n New in version 2.2.\n\n Changed in version 2.3: Support for other error handling schemes\n added.\n\n Changed in version 2.7: Support for keyword arguments added.\n\nstr.encode([encoding[, errors]])\n\n Return an encoded version of the string. Default encoding is the\n current default string encoding. *errors* may be given to set a\n different error handling scheme. The default for *errors* is\n "\'strict\'", meaning that encoding errors raise a "UnicodeError".\n Other possible values are "\'ignore\'", "\'replace\'",\n "\'xmlcharrefreplace\'", "\'backslashreplace\'" and any other name\n registered via "codecs.register_error()", see section Codec Base\n Classes. For a list of possible encodings, see section Standard\n Encodings.\n\n New in version 2.0.\n\n Changed in version 2.3: Support for "\'xmlcharrefreplace\'" and\n "\'backslashreplace\'" and other error handling schemes added.\n\n Changed in version 2.7: Support for keyword arguments added.\n\nstr.endswith(suffix[, start[, end]])\n\n Return "True" if the string ends with the specified *suffix*,\n otherwise return "False". *suffix* can also be a tuple of suffixes\n to look for. With optional *start*, test beginning at that\n position. With optional *end*, stop comparing at that position.\n\n Changed in version 2.5: Accept tuples as *suffix*.\n\nstr.expandtabs([tabsize])\n\n Return a copy of the string where all tab characters are replaced\n by one or more spaces, depending on the current column and the\n given tab size. Tab positions occur every *tabsize* characters\n (default is 8, giving tab positions at columns 0, 8, 16 and so on).\n To expand the string, the current column is set to zero and the\n string is examined character by character. If the character is a\n tab ("\\t"), one or more space characters are inserted in the result\n until the current column is equal to the next tab position. (The\n tab character itself is not copied.) If the character is a newline\n ("\\n") or return ("\\r"), it is copied and the current column is\n reset to zero. Any other character is copied unchanged and the\n current column is incremented by one regardless of how the\n character is represented when printed.\n\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs()\n \'01 012 0123 01234\'\n >>> \'01\\t012\\t0123\\t01234\'.expandtabs(4)\n \'01 012 0123 01234\'\n\nstr.find(sub[, start[, end]])\n\n Return the lowest index in the string where substring *sub* is\n found within the slice "s[start:end]". Optional arguments *start*\n and *end* are interpreted as in slice notation. Return "-1" if\n *sub* is not found.\n\n Note: The "find()" method should be used only if you need to know\n the position of *sub*. To check if *sub* is a substring or not,\n use the "in" operator:\n\n >>> \'Py\' in \'Python\'\n True\n\nstr.format(*args, **kwargs)\n\n Perform a string formatting operation. The string on which this\n method is called can contain literal text or replacement fields\n delimited by braces "{}". Each replacement field contains either\n the numeric index of a positional argument, or the name of a\n keyword argument. Returns a copy of the string where each\n replacement field is replaced with the string value of the\n corresponding argument.\n\n >>> "The sum of 1 + 2 is {0}".format(1+2)\n \'The sum of 1 + 2 is 3\'\n\n See Format String Syntax for a description of the various\n formatting options that can be specified in format strings.\n\n This method of string formatting is the new standard in Python 3,\n and should be preferred to the "%" formatting described in String\n Formatting Operations in new code.\n\n New in version 2.6.\n\nstr.index(sub[, start[, end]])\n\n Like "find()", but raise "ValueError" when the substring is not\n found.\n\nstr.isalnum()\n\n Return true if all characters in the string are alphanumeric and\n there is at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isalpha()\n\n Return true if all characters in the string are alphabetic and\n there is at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isdigit()\n\n Return true if all characters in the string are digits and there is\n at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.islower()\n\n Return true if all cased characters [4] in the string are lowercase\n and there is at least one cased character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isspace()\n\n Return true if there are only whitespace characters in the string\n and there is at least one character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.istitle()\n\n Return true if the string is a titlecased string and there is at\n least one character, for example uppercase characters may only\n follow uncased characters and lowercase characters only cased ones.\n Return false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.isupper()\n\n Return true if all cased characters [4] in the string are uppercase\n and there is at least one cased character, false otherwise.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.join(iterable)\n\n Return a string which is the concatenation of the strings in the\n *iterable* *iterable*. The separator between elements is the\n string providing this method.\n\nstr.ljust(width[, fillchar])\n\n Return the string left justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is a\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\n Changed in version 2.4: Support for the *fillchar* argument.\n\nstr.lower()\n\n Return a copy of the string with all the cased characters [4]\n converted to lowercase.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.lstrip([chars])\n\n Return a copy of the string with leading characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a prefix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.lstrip()\n \'spacious \'\n >>> \'www.example.com\'.lstrip(\'cmowz.\')\n \'example.com\'\n\n Changed in version 2.2.2: Support for the *chars* argument.\n\nstr.partition(sep)\n\n Split the string at the first occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing the string itself, followed by\n two empty strings.\n\n New in version 2.5.\n\nstr.replace(old, new[, count])\n\n Return a copy of the string with all occurrences of substring *old*\n replaced by *new*. If the optional argument *count* is given, only\n the first *count* occurrences are replaced.\n\nstr.rfind(sub[, start[, end]])\n\n Return the highest index in the string where substring *sub* is\n found, such that *sub* is contained within "s[start:end]".\n Optional arguments *start* and *end* are interpreted as in slice\n notation. Return "-1" on failure.\n\nstr.rindex(sub[, start[, end]])\n\n Like "rfind()" but raises "ValueError" when the substring *sub* is\n not found.\n\nstr.rjust(width[, fillchar])\n\n Return the string right justified in a string of length *width*.\n Padding is done using the specified *fillchar* (default is a\n space). The original string is returned if *width* is less than or\n equal to "len(s)".\n\n Changed in version 2.4: Support for the *fillchar* argument.\n\nstr.rpartition(sep)\n\n Split the string at the last occurrence of *sep*, and return a\n 3-tuple containing the part before the separator, the separator\n itself, and the part after the separator. If the separator is not\n found, return a 3-tuple containing two empty strings, followed by\n the string itself.\n\n New in version 2.5.\n\nstr.rsplit([sep[, maxsplit]])\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit* splits\n are done, the *rightmost* ones. If *sep* is not specified or\n "None", any whitespace string is a separator. Except for splitting\n from the right, "rsplit()" behaves like "split()" which is\n described in detail below.\n\n New in version 2.4.\n\nstr.rstrip([chars])\n\n Return a copy of the string with trailing characters removed. The\n *chars* argument is a string specifying the set of characters to be\n removed. If omitted or "None", the *chars* argument defaults to\n removing whitespace. The *chars* argument is not a suffix; rather,\n all combinations of its values are stripped:\n\n >>> \' spacious \'.rstrip()\n \' spacious\'\n >>> \'mississippi\'.rstrip(\'ipz\')\n \'mississ\'\n\n Changed in version 2.2.2: Support for the *chars* argument.\n\nstr.split([sep[, maxsplit]])\n\n Return a list of the words in the string, using *sep* as the\n delimiter string. If *maxsplit* is given, at most *maxsplit*\n splits are done (thus, the list will have at most "maxsplit+1"\n elements). If *maxsplit* is not specified or "-1", then there is\n no limit on the number of splits (all possible splits are made).\n\n If *sep* is given, consecutive delimiters are not grouped together\n and are deemed to delimit empty strings (for example,\n "\'1,,2\'.split(\',\')" returns "[\'1\', \'\', \'2\']"). The *sep* argument\n may consist of multiple characters (for example,\n "\'1<>2<>3\'.split(\'<>\')" returns "[\'1\', \'2\', \'3\']"). Splitting an\n empty string with a specified separator returns "[\'\']".\n\n If *sep* is not specified or is "None", a different splitting\n algorithm is applied: runs of consecutive whitespace are regarded\n as a single separator, and the result will contain no empty strings\n at the start or end if the string has leading or trailing\n whitespace. Consequently, splitting an empty string or a string\n consisting of just whitespace with a "None" separator returns "[]".\n\n For example, "\' 1 2 3 \'.split()" returns "[\'1\', \'2\', \'3\']", and\n "\' 1 2 3 \'.split(None, 1)" returns "[\'1\', \'2 3 \']".\n\nstr.splitlines([keepends])\n\n Return a list of the lines in the string, breaking at line\n boundaries. This method uses the *universal newlines* approach to\n splitting lines. Line breaks are not included in the resulting list\n unless *keepends* is given and true.\n\n For example, "\'ab c\\n\\nde fg\\rkl\\r\\n\'.splitlines()" returns "[\'ab\n c\', \'\', \'de fg\', \'kl\']", while the same call with\n "splitlines(True)" returns "[\'ab c\\n\', \'\\n\', \'de fg\\r\', \'kl\\r\\n\']".\n\n Unlike "split()" when a delimiter string *sep* is given, this\n method returns an empty list for the empty string, and a terminal\n line break does not result in an extra line.\n\nstr.startswith(prefix[, start[, end]])\n\n Return "True" if string starts with the *prefix*, otherwise return\n "False". *prefix* can also be a tuple of prefixes to look for.\n With optional *start*, test string beginning at that position.\n With optional *end*, stop comparing string at that position.\n\n Changed in version 2.5: Accept tuples as *prefix*.\n\nstr.strip([chars])\n\n Return a copy of the string with the leading and trailing\n characters removed. The *chars* argument is a string specifying the\n set of characters to be removed. If omitted or "None", the *chars*\n argument defaults to removing whitespace. The *chars* argument is\n not a prefix or suffix; rather, all combinations of its values are\n stripped:\n\n >>> \' spacious \'.strip()\n \'spacious\'\n >>> \'www.example.com\'.strip(\'cmowz.\')\n \'example\'\n\n Changed in version 2.2.2: Support for the *chars* argument.\n\nstr.swapcase()\n\n Return a copy of the string with uppercase characters converted to\n lowercase and vice versa.\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.title()\n\n Return a titlecased version of the string where words start with an\n uppercase character and the remaining characters are lowercase.\n\n The algorithm uses a simple language-independent definition of a\n word as groups of consecutive letters. The definition works in\n many contexts but it means that apostrophes in contractions and\n possessives form word boundaries, which may not be the desired\n result:\n\n >>> "they\'re bill\'s friends from the UK".title()\n "They\'Re Bill\'S Friends From The Uk"\n\n A workaround for apostrophes can be constructed using regular\n expressions:\n\n >>> import re\n >>> def titlecase(s):\n ... return re.sub(r"[A-Za-z]+(\'[A-Za-z]+)?",\n ... lambda mo: mo.group(0)[0].upper() +\n ... mo.group(0)[1:].lower(),\n ... s)\n ...\n >>> titlecase("they\'re bill\'s friends.")\n "They\'re Bill\'s Friends."\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.translate(table[, deletechars])\n\n Return a copy of the string where all characters occurring in the\n optional argument *deletechars* are removed, and the remaining\n characters have been mapped through the given translation table,\n which must be a string of length 256.\n\n You can use the "maketrans()" helper function in the "string"\n module to create a translation table. For string objects, set the\n *table* argument to "None" for translations that only delete\n characters:\n\n >>> \'read this short text\'.translate(None, \'aeiou\')\n \'rd ths shrt txt\'\n\n New in version 2.6: Support for a "None" *table* argument.\n\n For Unicode objects, the "translate()" method does not accept the\n optional *deletechars* argument. Instead, it returns a copy of the\n *s* where all characters have been mapped through the given\n translation table which must be a mapping of Unicode ordinals to\n Unicode ordinals, Unicode strings or "None". Unmapped characters\n are left untouched. Characters mapped to "None" are deleted. Note,\n a more flexible approach is to create a custom character mapping\n codec using the "codecs" module (see "encodings.cp1251" for an\n example).\n\nstr.upper()\n\n Return a copy of the string with all the cased characters [4]\n converted to uppercase. Note that "str.upper().isupper()" might be\n "False" if "s" contains uncased characters or if the Unicode\n category of the resulting character(s) is not "Lu" (Letter,\n uppercase), but e.g. "Lt" (Letter, titlecase).\n\n For 8-bit strings, this method is locale-dependent.\n\nstr.zfill(width)\n\n Return the numeric string left filled with zeros in a string of\n length *width*. A sign prefix is handled correctly. The original\n string is returned if *width* is less than or equal to "len(s)".\n\n New in version 2.2.2.\n\nThe following methods are present only on unicode objects:\n\nunicode.isnumeric()\n\n Return "True" if there are only numeric characters in S, "False"\n otherwise. Numeric characters include digit characters, and all\n characters that have the Unicode numeric value property, e.g.\n U+2155, VULGAR FRACTION ONE FIFTH.\n\nunicode.isdecimal()\n\n Return "True" if there are only decimal characters in S, "False"\n otherwise. Decimal characters include digit characters, and all\n characters that can be used to form decimal-radix numbers, e.g.\n U+0660, ARABIC-INDIC DIGIT ZERO.\n\n\nString Formatting Operations\n============================\n\nString and Unicode objects have one unique built-in operation: the "%"\noperator (modulo). This is also known as the string *formatting* or\n*interpolation* operator. Given "format % values" (where *format* is\na string or Unicode object), "%" conversion specifications in *format*\nare replaced with zero or more elements of *values*. The effect is\nsimilar to the using "sprintf()" in the C language. If *format* is a\nUnicode object, or if any of the objects being converted using the\n"%s" conversion are Unicode objects, the result will also be a Unicode\nobject.\n\nIf *format* requires a single argument, *values* may be a single non-\ntuple object. [5] Otherwise, *values* must be a tuple with exactly\nthe number of items specified by the format string, or a single\nmapping object (for example, a dictionary).\n\nA conversion specifier contains two or more characters and has the\nfollowing components, which must occur in this order:\n\n1. The "\'%\'" character, which marks the start of the specifier.\n\n2. Mapping key (optional), consisting of a parenthesised sequence\n of characters (for example, "(somename)").\n\n3. Conversion flags (optional), which affect the result of some\n conversion types.\n\n4. Minimum field width (optional). If specified as an "\'*\'"\n (asterisk), the actual width is read from the next element of the\n tuple in *values*, and the object to convert comes after the\n minimum field width and optional precision.\n\n5. Precision (optional), given as a "\'.\'" (dot) followed by the\n precision. If specified as "\'*\'" (an asterisk), the actual width\n is read from the next element of the tuple in *values*, and the\n value to convert comes after the precision.\n\n6. Length modifier (optional).\n\n7. Conversion type.\n\nWhen the right argument is a dictionary (or other mapping type), then\nthe formats in the string *must* include a parenthesised mapping key\ninto that dictionary inserted immediately after the "\'%\'" character.\nThe mapping key selects the value to be formatted from the mapping.\nFor example:\n\n>>> print \'%(language)s has %(number)03d quote types.\' % \\\n... {"language": "Python", "number": 2}\nPython has 002 quote types.\n\nIn this case no "*" specifiers may occur in a format (since they\nrequire a sequential parameter list).\n\nThe conversion flag characters are:\n\n+-----------+-----------------------------------------------------------------------+\n| Flag | Meaning |\n+===========+=======================================================================+\n| "\'#\'" | The value conversion will use the "alternate form" (where defined |\n| | below). |\n+-----------+-----------------------------------------------------------------------+\n| "\'0\'" | The conversion will be zero padded for numeric values. |\n+-----------+-----------------------------------------------------------------------+\n| "\'-\'" | The converted value is left adjusted (overrides the "\'0\'" conversion |\n| | if both are given). |\n+-----------+-----------------------------------------------------------------------+\n| "\' \'" | (a space) A blank should be left before a positive number (or empty |\n| | string) produced by a signed conversion. |\n+-----------+-----------------------------------------------------------------------+\n| "\'+\'" | A sign character ("\'+\'" or "\'-\'") will precede the conversion |\n| | (overrides a "space" flag). |\n+-----------+-----------------------------------------------------------------------+\n\nA length modifier ("h", "l", or "L") may be present, but is ignored as\nit is not necessary for Python -- so e.g. "%ld" is identical to "%d".\n\nThe conversion types are:\n\n+--------------+-------------------------------------------------------+---------+\n| Conversion | Meaning | Notes |\n+==============+=======================================================+=========+\n| "\'d\'" | Signed integer decimal. | |\n+--------------+-------------------------------------------------------+---------+\n| "\'i\'" | Signed integer decimal. | |\n+--------------+-------------------------------------------------------+---------+\n| "\'o\'" | Signed octal value. | (1) |\n+--------------+-------------------------------------------------------+---------+\n| "\'u\'" | Obsolete type -- it is identical to "\'d\'". | (7) |\n+--------------+-------------------------------------------------------+---------+\n| "\'x\'" | Signed hexadecimal (lowercase). | (2) |\n+--------------+-------------------------------------------------------+---------+\n| "\'X\'" | Signed hexadecimal (uppercase). | (2) |\n+--------------+-------------------------------------------------------+---------+\n| "\'e\'" | Floating point exponential format (lowercase). | (3) |\n+--------------+-------------------------------------------------------+---------+\n| "\'E\'" | Floating point exponential format (uppercase). | (3) |\n+--------------+-------------------------------------------------------+---------+\n| "\'f\'" | Floating point decimal format. | (3) |\n+--------------+-------------------------------------------------------+---------+\n| "\'F\'" | Floating point decimal format. | (3) |\n+--------------+-------------------------------------------------------+---------+\n| "\'g\'" | Floating point format. Uses lowercase exponential | (4) |\n| | format if exponent is less than -4 or not less than | |\n| | precision, decimal format otherwise. | |\n+--------------+-------------------------------------------------------+---------+\n| "\'G\'" | Floating point format. Uses uppercase exponential | (4) |\n| | format if exponent is less than -4 or not less than | |\n| | precision, decimal format otherwise. | |\n+--------------+-------------------------------------------------------+---------+\n| "\'c\'" | Single character (accepts integer or single character | |\n| | string). | |\n+--------------+-------------------------------------------------------+---------+\n| "\'r\'" | String (converts any Python object using repr()). | (5) |\n+--------------+-------------------------------------------------------+---------+\n| "\'s\'" | String (converts any Python object using "str()"). | (6) |\n+--------------+-------------------------------------------------------+---------+\n| "\'%\'" | No argument is converted, results in a "\'%\'" | |\n| | character in the result. | |\n+--------------+-------------------------------------------------------+---------+\n\nNotes:\n\n1. The alternate form causes a leading zero ("\'0\'") to be inserted\n between left-hand padding and the formatting of the number if the\n leading character of the result is not already a zero.\n\n2. The alternate form causes a leading "\'0x\'" or "\'0X\'" (depending\n on whether the "\'x\'" or "\'X\'" format was used) to be inserted\n between left-hand padding and the formatting of the number if the\n leading character of the result is not already a zero.\n\n3. The alternate form causes the result to always contain a decimal\n point, even if no digits follow it.\n\n The precision determines the number of digits after the decimal\n point and defaults to 6.\n\n4. The alternate form causes the result to always contain a decimal\n point, and trailing zeroes are not removed as they would otherwise\n be.\n\n The precision determines the number of significant digits before\n and after the decimal point and defaults to 6.\n\n5. The "%r" conversion was added in Python 2.0.\n\n The precision determines the maximal number of characters used.\n\n6. If the object or format provided is a "unicode" string, the\n resulting string will also be "unicode".\n\n The precision determines the maximal number of characters used.\n\n7. See **PEP 237**.\n\nSince Python strings have an explicit length, "%s" conversions do not\nassume that "\'\\0\'" is the end of the string.\n\nChanged in version 2.7: "%f" conversions for numbers whose absolute\nvalue is over 1e50 are no longer replaced by "%g" conversions.\n\nAdditional string operations are defined in standard modules "string"\nand "re".\n\n\nXRange Type\n===========\n\nThe "xrange" type is an immutable sequence which is commonly used for\nlooping. The advantage of the "xrange" type is that an "xrange"\nobject will always take the same amount of memory, no matter the size\nof the range it represents. There are no consistent performance\nadvantages.\n\nXRange objects have very little behavior: they only support indexing,\niteration, and the "len()" function.\n\n\nMutable Sequence Types\n======================\n\nList and "bytearray" objects support additional operations that allow\nin-place modification of the object. Other mutable sequence types\n(when added to the language) should also support these operations.\nStrings and tuples are immutable sequence types: such objects cannot\nbe modified once created. The following operations are defined on\nmutable sequence types (where *x* is an arbitrary object):\n\n+--------------------------------+----------------------------------+-----------------------+\n| Operation | Result | Notes |\n+================================+==================================+=======================+\n| "s[i] = x" | item *i* of *s* is replaced by | |\n| | *x* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j] = t" | slice of *s* from *i* to *j* is | |\n| | replaced by the contents of the | |\n| | iterable *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j]" | same as "s[i:j] = []" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j:k] = t" | the elements of "s[i:j:k]" are | (1) |\n| | replaced by those of *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j:k]" | removes the elements of | |\n| | "s[i:j:k]" from the list | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.append(x)" | same as "s[len(s):len(s)] = [x]" | (2) |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.extend(x)" or "s += t" | for the most part the same as | (3) |\n| | "s[len(s):len(s)] = x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s *= n" | updates *s* with its contents | (11) |\n| | repeated *n* times | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.count(x)" | return number of *i*\'s for which | |\n| | "s[i] == x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.index(x[, i[, j]])" | return smallest *k* such that | (4) |\n| | "s[k] == x" and "i <= k < j" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.insert(i, x)" | same as "s[i:i] = [x]" | (5) |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.pop([i])" | same as "x = s[i]; del s[i]; | (6) |\n| | return x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.remove(x)" | same as "del s[s.index(x)]" | (4) |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.reverse()" | reverses the items of *s* in | (7) |\n| | place | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.sort([cmp[, key[, | sort the items of *s* in place | (7)(8)(9)(10) |\n| reverse]]])" | | |\n+--------------------------------+----------------------------------+-----------------------+\n\nNotes:\n\n1. *t* must have the same length as the slice it is replacing.\n\n2. The C implementation of Python has historically accepted\n multiple parameters and implicitly joined them into a tuple; this\n no longer works in Python 2.0. Use of this misfeature has been\n deprecated since Python 1.4.\n\n3. *x* can be any iterable object.\n\n4. Raises "ValueError" when *x* is not found in *s*. When a\n negative index is passed as the second or third parameter to the\n "index()" method, the list length is added, as for slice indices.\n If it is still negative, it is truncated to zero, as for slice\n indices.\n\n Changed in version 2.3: Previously, "index()" didn\'t have arguments\n for specifying start and stop positions.\n\n5. When a negative index is passed as the first parameter to the\n "insert()" method, the list length is added, as for slice indices.\n If it is still negative, it is truncated to zero, as for slice\n indices.\n\n Changed in version 2.3: Previously, all negative indices were\n truncated to zero.\n\n6. The "pop()" method\'s optional argument *i* defaults to "-1", so\n that by default the last item is removed and returned.\n\n7. The "sort()" and "reverse()" methods modify the list in place\n for economy of space when sorting or reversing a large list. To\n remind you that they operate by side effect, they don\'t return the\n sorted or reversed list.\n\n8. The "sort()" method takes optional arguments for controlling the\n comparisons.\n\n *cmp* specifies a custom comparison function of two arguments (list\n items) which should return a negative, zero or positive number\n depending on whether the first argument is considered smaller than,\n equal to, or larger than the second argument: "cmp=lambda x,y:\n cmp(x.lower(), y.lower())". The default value is "None".\n\n *key* specifies a function of one argument that is used to extract\n a comparison key from each list element: "key=str.lower". The\n default value is "None".\n\n *reverse* is a boolean value. If set to "True", then the list\n elements are sorted as if each comparison were reversed.\n\n In general, the *key* and *reverse* conversion processes are much\n faster than specifying an equivalent *cmp* function. This is\n because *cmp* is called multiple times for each list element while\n *key* and *reverse* touch each element only once. Use\n "functools.cmp_to_key()" to convert an old-style *cmp* function to\n a *key* function.\n\n Changed in version 2.3: Support for "None" as an equivalent to\n omitting *cmp* was added.\n\n Changed in version 2.4: Support for *key* and *reverse* was added.\n\n9. Starting with Python 2.3, the "sort()" method is guaranteed to\n be stable. A sort is stable if it guarantees not to change the\n relative order of elements that compare equal --- this is helpful\n for sorting in multiple passes (for example, sort by department,\n then by salary grade).\n\n10. **CPython implementation detail:** While a list is being\n sorted, the effect of attempting to mutate, or even inspect, the\n list is undefined. The C implementation of Python 2.3 and newer\n makes the list appear empty for the duration, and raises\n "ValueError" if it can detect that the list has been mutated\n during a sort.\n\n11. The value *n* is an integer, or an object implementing\n "__index__()". Zero and negative values of *n* clear the\n sequence. Items in the sequence are not copied; they are\n referenced multiple times, as explained for "s * n" under Sequence\n Types --- str, unicode, list, tuple, bytearray, buffer, xrange.\n', + 'typesseq-mutable': u'\nMutable Sequence Types\n**********************\n\nList and "bytearray" objects support additional operations that allow\nin-place modification of the object. Other mutable sequence types\n(when added to the language) should also support these operations.\nStrings and tuples are immutable sequence types: such objects cannot\nbe modified once created. The following operations are defined on\nmutable sequence types (where *x* is an arbitrary object):\n\n+--------------------------------+----------------------------------+-----------------------+\n| Operation | Result | Notes |\n+================================+==================================+=======================+\n| "s[i] = x" | item *i* of *s* is replaced by | |\n| | *x* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j] = t" | slice of *s* from *i* to *j* is | |\n| | replaced by the contents of the | |\n| | iterable *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j]" | same as "s[i:j] = []" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s[i:j:k] = t" | the elements of "s[i:j:k]" are | (1) |\n| | replaced by those of *t* | |\n+--------------------------------+----------------------------------+-----------------------+\n| "del s[i:j:k]" | removes the elements of | |\n| | "s[i:j:k]" from the list | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.append(x)" | same as "s[len(s):len(s)] = [x]" | (2) |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.extend(x)" or "s += t" | for the most part the same as | (3) |\n| | "s[len(s):len(s)] = x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s *= n" | updates *s* with its contents | (11) |\n| | repeated *n* times | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.count(x)" | return number of *i*\'s for which | |\n| | "s[i] == x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.index(x[, i[, j]])" | return smallest *k* such that | (4) |\n| | "s[k] == x" and "i <= k < j" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.insert(i, x)" | same as "s[i:i] = [x]" | (5) |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.pop([i])" | same as "x = s[i]; del s[i]; | (6) |\n| | return x" | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.remove(x)" | same as "del s[s.index(x)]" | (4) |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.reverse()" | reverses the items of *s* in | (7) |\n| | place | |\n+--------------------------------+----------------------------------+-----------------------+\n| "s.sort([cmp[, key[, | sort the items of *s* in place | (7)(8)(9)(10) |\n| reverse]]])" | | |\n+--------------------------------+----------------------------------+-----------------------+\n\nNotes:\n\n1. *t* must have the same length as the slice it is replacing.\n\n2. The C implementation of Python has historically accepted\n multiple parameters and implicitly joined them into a tuple; this\n no longer works in Python 2.0. Use of this misfeature has been\n deprecated since Python 1.4.\n\n3. *x* can be any iterable object.\n\n4. Raises "ValueError" when *x* is not found in *s*. When a\n negative index is passed as the second or third parameter to the\n "index()" method, the list length is added, as for slice indices.\n If it is still negative, it is truncated to zero, as for slice\n indices.\n\n Changed in version 2.3: Previously, "index()" didn\'t have arguments\n for specifying start and stop positions.\n\n5. When a negative index is passed as the first parameter to the\n "insert()" method, the list length is added, as for slice indices.\n If it is still negative, it is truncated to zero, as for slice\n indices.\n\n Changed in version 2.3: Previously, all negative indices were\n truncated to zero.\n\n6. The "pop()" method\'s optional argument *i* defaults to "-1", so\n that by default the last item is removed and returned.\n\n7. The "sort()" and "reverse()" methods modify the list in place\n for economy of space when sorting or reversing a large list. To\n remind you that they operate by side effect, they don\'t return the\n sorted or reversed list.\n\n8. The "sort()" method takes optional arguments for controlling the\n comparisons.\n\n *cmp* specifies a custom comparison function of two arguments (list\n items) which should return a negative, zero or positive number\n depending on whether the first argument is considered smaller than,\n equal to, or larger than the second argument: "cmp=lambda x,y:\n cmp(x.lower(), y.lower())". The default value is "None".\n\n *key* specifies a function of one argument that is used to extract\n a comparison key from each list element: "key=str.lower". The\n default value is "None".\n\n *reverse* is a boolean value. If set to "True", then the list\n elements are sorted as if each comparison were reversed.\n\n In general, the *key* and *reverse* conversion processes are much\n faster than specifying an equivalent *cmp* function. This is\n because *cmp* is called multiple times for each list element while\n *key* and *reverse* touch each element only once. Use\n "functools.cmp_to_key()" to convert an old-style *cmp* function to\n a *key* function.\n\n Changed in version 2.3: Support for "None" as an equivalent to\n omitting *cmp* was added.\n\n Changed in version 2.4: Support for *key* and *reverse* was added.\n\n9. Starting with Python 2.3, the "sort()" method is guaranteed to\n be stable. A sort is stable if it guarantees not to change the\n relative order of elements that compare equal --- this is helpful\n for sorting in multiple passes (for example, sort by department,\n then by salary grade).\n\n10. **CPython implementation detail:** While a list is being\n sorted, the effect of attempting to mutate, or even inspect, the\n list is undefined. The C implementation of Python 2.3 and newer\n makes the list appear empty for the duration, and raises\n "ValueError" if it can detect that the list has been mutated\n during a sort.\n\n11. The value *n* is an integer, or an object implementing\n "__index__()". Zero and negative values of *n* clear the\n sequence. Items in the sequence are not copied; they are\n referenced multiple times, as explained for "s * n" under Sequence\n Types --- str, unicode, list, tuple, bytearray, buffer, xrange.\n', + 'unary': u'\nUnary arithmetic and bitwise operations\n***************************************\n\nAll unary arithmetic and bitwise operations have the same priority:\n\n u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr\n\nThe unary "-" (minus) operator yields the negation of its numeric\nargument.\n\nThe unary "+" (plus) operator yields its numeric argument unchanged.\n\nThe unary "~" (invert) operator yields the bitwise inversion of its\nplain or long integer argument. The bitwise inversion of "x" is\ndefined as "-(x+1)". It only applies to integral numbers.\n\nIn all three cases, if the argument does not have the proper type, a\n"TypeError" exception is raised.\n', + 'while': u'\nThe "while" statement\n*********************\n\nThe "while" statement is used for repeated execution as long as an\nexpression is true:\n\n while_stmt ::= "while" expression ":" suite\n ["else" ":" suite]\n\nThis repeatedly tests the expression and, if it is true, executes the\nfirst suite; if the expression is false (which may be the first time\nit is tested) the suite of the "else" clause, if present, is executed\nand the loop terminates.\n\nA "break" statement executed in the first suite terminates the loop\nwithout executing the "else" clause\'s suite. A "continue" statement\nexecuted in the first suite skips the rest of the suite and goes back\nto testing the expression.\n', + 'with': u'\nThe "with" statement\n********************\n\nNew in version 2.5.\n\nThe "with" statement is used to wrap the execution of a block with\nmethods defined by a context manager (see section With Statement\nContext Managers). This allows common "try"..."except"..."finally"\nusage patterns to be encapsulated for convenient reuse.\n\n with_stmt ::= "with" with_item ("," with_item)* ":" suite\n with_item ::= expression ["as" target]\n\nThe execution of the "with" statement with one "item" proceeds as\nfollows:\n\n1. The context expression (the expression given in the "with_item")\n is evaluated to obtain a context manager.\n\n2. The context manager\'s "__exit__()" is loaded for later use.\n\n3. The context manager\'s "__enter__()" method is invoked.\n\n4. If a target was included in the "with" statement, the return\n value from "__enter__()" is assigned to it.\n\n Note: The "with" statement guarantees that if the "__enter__()"\n method returns without an error, then "__exit__()" will always be\n called. Thus, if an error occurs during the assignment to the\n target list, it will be treated the same as an error occurring\n within the suite would be. See step 6 below.\n\n5. The suite is executed.\n\n6. The context manager\'s "__exit__()" method is invoked. If an\n exception caused the suite to be exited, its type, value, and\n traceback are passed as arguments to "__exit__()". Otherwise, three\n "None" arguments are supplied.\n\n If the suite was exited due to an exception, and the return value\n from the "__exit__()" method was false, the exception is reraised.\n If the return value was true, the exception is suppressed, and\n execution continues with the statement following the "with"\n statement.\n\n If the suite was exited for any reason other than an exception, the\n return value from "__exit__()" is ignored, and execution proceeds\n at the normal location for the kind of exit that was taken.\n\nWith more than one item, the context managers are processed as if\nmultiple "with" statements were nested:\n\n with A() as a, B() as b:\n suite\n\nis equivalent to\n\n with A() as a:\n with B() as b:\n suite\n\nNote: In Python 2.5, the "with" statement is only allowed when the\n "with_statement" feature has been enabled. It is always enabled in\n Python 2.6.\n\nChanged in version 2.7: Support for multiple context expressions.\n\nSee also:\n\n **PEP 0343** - The "with" statement\n The specification, background, and examples for the Python "with"\n statement.\n', + 'yield': u'\nThe "yield" statement\n*********************\n\n yield_stmt ::= yield_expression\n\nThe "yield" statement is only used when defining a generator function,\nand is only used in the body of the generator function. Using a\n"yield" statement in a function definition is sufficient to cause that\ndefinition to create a generator function instead of a normal\nfunction.\n\nWhen a generator function is called, it returns an iterator known as a\ngenerator iterator, or more commonly, a generator. The body of the\ngenerator function is executed by calling the generator\'s "next()"\nmethod repeatedly until it raises an exception.\n\nWhen a "yield" statement is executed, the state of the generator is\nfrozen and the value of "expression_list" is returned to "next()"\'s\ncaller. By "frozen" we mean that all local state is retained,\nincluding the current bindings of local variables, the instruction\npointer, and the internal evaluation stack: enough information is\nsaved so that the next time "next()" is invoked, the function can\nproceed exactly as if the "yield" statement were just another external\ncall.\n\nAs of Python version 2.5, the "yield" statement is now allowed in the\n"try" clause of a "try" ... "finally" construct. If the generator is\nnot resumed before it is finalized (by reaching a zero reference count\nor by being garbage collected), the generator-iterator\'s "close()"\nmethod will be called, allowing any pending "finally" clauses to\nexecute.\n\nFor full details of "yield" semantics, refer to the Yield expressions\nsection.\n\nNote: In Python 2.2, the "yield" statement was only allowed when the\n "generators" feature has been enabled. This "__future__" import\n statement was used to enable the feature:\n\n from __future__ import generators\n\nSee also:\n\n **PEP 0255** - Simple Generators\n The proposal for adding generators and the "yield" statement to\n Python.\n\n **PEP 0342** - Coroutines via Enhanced Generators\n The proposal that, among other generator enhancements, proposed\n allowing "yield" to appear inside a "try" ... "finally" block.\n'} -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Jan 6 12:10:13 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 6 Jan 2016 17:10:13 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-06 Message-ID: <12b5371b-d980-49c9-999c-a9564694d2a0@irsmsx102.ger.corp.intel.com> Results for project Python default, build date 2016-01-06 03:09:09 +0000 commit: 1118dfcbcc3530a1a8a70b3e3a79df7c5b24275a previous commit: 136adbcefa5f1c803a5584e74152eace9cb43852 revision date: 2016-01-05 19:27:54 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.43% -0.24% 10.63% 13.78% :-| pybench 0.12% -0.16% -1.63% 5.85% :-( regex_v8 2.64% -0.12% -5.04% 5.26% :-| nbody 0.12% 0.52% 1.56% 7.97% :-( json_dump_v2 0.21% -1.09% -2.09% 11.84% :-( normal_startup 0.78% -0.12% -2.02% 5.18% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Jan 6 12:11:24 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 6 Jan 2016 17:11:24 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-06 Message-ID: Results for project Python 2.7, build date 2016-01-06 04:00:47 +0000 commit: e2faa18802bb6b8b5d0e6eac5ba34072c940fa2f previous commit: da934a19855bbdc46dad918874216b9a64ac6c08 revision date: 2016-01-06 01:08:12 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.19% -0.42% 3.82% 3.75% :-) pybench 0.13% 0.03% 6.03% 4.75% :-( regex_v8 1.08% 0.18% -2.36% 10.85% :-) nbody 0.11% -0.01% 7.88% 3.28% :-) json_dump_v2 0.26% -0.52% 5.47% 8.43% :-( normal_startup 2.17% -0.75% -6.14% 2.70% :-| ssbench 0.20% 0.27% 1.36% 0.93% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Jan 6 12:54:08 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 17:54:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI0MTIw?= =?utf-8?q?=3A_Ignore_PermissionError_in_pathlib=2EPath=2E=5Br=5Dglob=28?= =?utf-8?q?=29=2E_Ulrich_Petri=2E?= Message-ID: <20160106175408.8281.53373@psf.io> https://hg.python.org/cpython/rev/bac18cb7b011 changeset: 99775:bac18cb7b011 branch: 3.4 parent: 99766:7150e8a38c63 user: Guido van Rossum date: Wed Jan 06 09:42:07 2016 -0800 summary: Issue #24120: Ignore PermissionError in pathlib.Path.[r]glob(). Ulrich Petri. files: Lib/pathlib.py | 60 ++++++++++++++++----------- Lib/test/test_pathlib.py | 32 +++++++++----- Misc/ACKS | 1 + Misc/NEWS | 3 + 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -454,12 +454,15 @@ _Selector.__init__(self, child_parts) def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + path = parent_path._make_child_relpath(self.name) + if exists(path): + for p in self.successor._select_from(path, is_dir, exists, listdir): + yield p + except PermissionError: return - path = parent_path._make_child_relpath(self.name) - if exists(path): - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p class _WildcardSelector(_Selector): @@ -469,15 +472,19 @@ _Selector.__init__(self, child_parts) def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + cf = parent_path._flavour.casefold + for name in listdir(parent_path): + casefolded = cf(name) + if self.pat.match(casefolded): + path = parent_path._make_child_relpath(name) + for p in self.successor._select_from(path, is_dir, exists, listdir): + yield p + except PermissionError: return - cf = parent_path._flavour.casefold - for name in listdir(parent_path): - casefolded = cf(name) - if self.pat.match(casefolded): - path = parent_path._make_child_relpath(name) - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p + class _RecursiveWildcardSelector(_Selector): @@ -494,19 +501,22 @@ yield p def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + with _cached(listdir) as listdir: + yielded = set() + try: + successor_select = self.successor._select_from + for starting_point in self._iterate_directories(parent_path, is_dir, listdir): + for p in successor_select(starting_point, is_dir, exists, listdir): + if p not in yielded: + yield p + yielded.add(p) + finally: + yielded.clear() + except PermissionError: return - with _cached(listdir) as listdir: - yielded = set() - try: - successor_select = self.successor._select_from - for starting_point in self._iterate_directories(parent_path, is_dir, listdir): - for p in successor_select(starting_point, is_dir, exists, listdir): - if p not in yielded: - yield p - yielded.add(p) - finally: - yielded.clear() # diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1203,26 +1203,33 @@ # (BASE) # | - # |-- dirA/ - # |-- linkC -> "../dirB" - # |-- dirB/ - # | |-- fileB - # |-- linkD -> "../dirB" - # |-- dirC/ - # | |-- fileC - # | |-- fileD + # |-- brokenLink -> non-existing + # |-- dirA + # | `-- linkC -> ../dirB + # |-- dirB + # | |-- fileB + # | `-- linkD -> ../dirB + # |-- dirC + # | |-- dirD + # | | `-- fileD + # | `-- fileC + # |-- dirE # |-- fileA - # |-- linkA -> "fileA" - # |-- linkB -> "dirB" + # |-- linkA -> fileA + # `-- linkB -> dirB # def setUp(self): + def cleanup(): + os.chmod(join('dirE'), 0o777) + support.rmtree(BASE) + self.addCleanup(cleanup) os.mkdir(BASE) - self.addCleanup(support.rmtree, BASE) os.mkdir(join('dirA')) os.mkdir(join('dirB')) os.mkdir(join('dirC')) os.mkdir(join('dirC', 'dirD')) + os.mkdir(join('dirE')) with open(join('fileA'), 'wb') as f: f.write(b"this is file A\n") with open(join('dirB', 'fileB'), 'wb') as f: @@ -1231,6 +1238,7 @@ f.write(b"this is file C\n") with open(join('dirC', 'dirD', 'fileD'), 'wb') as f: f.write(b"this is file D\n") + os.chmod(join('dirE'), 0) if not symlink_skip_reason: # Relative symlinks os.symlink('fileA', join('linkA')) @@ -1306,7 +1314,7 @@ p = P(BASE) it = p.iterdir() paths = set(it) - expected = ['dirA', 'dirB', 'dirC', 'fileA'] + expected = ['dirA', 'dirB', 'dirC', 'dirE', 'fileA'] if not symlink_skip_reason: expected += ['linkA', 'linkB', 'brokenLink'] self.assertEqual(paths, { P(BASE, q) for q in expected }) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1075,6 +1075,7 @@ Tim Peters Benjamin Peterson Joe Peterson +Ulrich Petri Chris Petrilli Roumen Petrov Bjorn Pettersen diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- Issue #24120: Ignore PermissionError when traversing a tree with + pathlib.Path.[r]glob(). Patch by Ulrich Petri. + What's New in Python 3.4.4? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 12:54:13 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 17:54:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2324120=3A_Ignore_PermissionError_in_pathlib=2EPa?= =?utf-8?b?dGguW3JdZ2xvYigpLiBVbHJpY2ggUGV0cmku?= Message-ID: <20160106175409.17319.89977@psf.io> https://hg.python.org/cpython/rev/f6ae90450a4d changeset: 99777:f6ae90450a4d parent: 99771:1118dfcbcc35 parent: 99776:224a026b4ca1 user: Guido van Rossum date: Wed Jan 06 09:53:51 2016 -0800 summary: Issue #24120: Ignore PermissionError in pathlib.Path.[r]glob(). Ulrich Petri. (Merge 3.5->3.6) files: Lib/pathlib.py | 60 ++++++++++++++++----------- Lib/test/test_pathlib.py | 32 +++++++++----- Misc/ACKS | 1 + Misc/NEWS | 3 + 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -499,12 +499,15 @@ _Selector.__init__(self, child_parts) def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + path = parent_path._make_child_relpath(self.name) + if exists(path): + for p in self.successor._select_from(path, is_dir, exists, listdir): + yield p + except PermissionError: return - path = parent_path._make_child_relpath(self.name) - if exists(path): - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p class _WildcardSelector(_Selector): @@ -514,15 +517,19 @@ _Selector.__init__(self, child_parts) def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + cf = parent_path._flavour.casefold + for name in listdir(parent_path): + casefolded = cf(name) + if self.pat.match(casefolded): + path = parent_path._make_child_relpath(name) + for p in self.successor._select_from(path, is_dir, exists, listdir): + yield p + except PermissionError: return - cf = parent_path._flavour.casefold - for name in listdir(parent_path): - casefolded = cf(name) - if self.pat.match(casefolded): - path = parent_path._make_child_relpath(name) - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p + class _RecursiveWildcardSelector(_Selector): @@ -539,19 +546,22 @@ yield p def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + with _cached(listdir) as listdir: + yielded = set() + try: + successor_select = self.successor._select_from + for starting_point in self._iterate_directories(parent_path, is_dir, listdir): + for p in successor_select(starting_point, is_dir, exists, listdir): + if p not in yielded: + yield p + yielded.add(p) + finally: + yielded.clear() + except PermissionError: return - with _cached(listdir) as listdir: - yielded = set() - try: - successor_select = self.successor._select_from - for starting_point in self._iterate_directories(parent_path, is_dir, listdir): - for p in successor_select(starting_point, is_dir, exists, listdir): - if p not in yielded: - yield p - yielded.add(p) - finally: - yielded.clear() # diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1199,26 +1199,33 @@ # (BASE) # | - # |-- dirA/ - # |-- linkC -> "../dirB" - # |-- dirB/ - # | |-- fileB - # |-- linkD -> "../dirB" - # |-- dirC/ - # | |-- fileC - # | |-- fileD + # |-- brokenLink -> non-existing + # |-- dirA + # | `-- linkC -> ../dirB + # |-- dirB + # | |-- fileB + # | `-- linkD -> ../dirB + # |-- dirC + # | |-- dirD + # | | `-- fileD + # | `-- fileC + # |-- dirE # |-- fileA - # |-- linkA -> "fileA" - # |-- linkB -> "dirB" + # |-- linkA -> fileA + # `-- linkB -> dirB # def setUp(self): + def cleanup(): + os.chmod(join('dirE'), 0o777) + support.rmtree(BASE) + self.addCleanup(cleanup) os.mkdir(BASE) - self.addCleanup(support.rmtree, BASE) os.mkdir(join('dirA')) os.mkdir(join('dirB')) os.mkdir(join('dirC')) os.mkdir(join('dirC', 'dirD')) + os.mkdir(join('dirE')) with open(join('fileA'), 'wb') as f: f.write(b"this is file A\n") with open(join('dirB', 'fileB'), 'wb') as f: @@ -1227,6 +1234,7 @@ f.write(b"this is file C\n") with open(join('dirC', 'dirD', 'fileD'), 'wb') as f: f.write(b"this is file D\n") + os.chmod(join('dirE'), 0) if not symlink_skip_reason: # Relative symlinks os.symlink('fileA', join('linkA')) @@ -1363,7 +1371,7 @@ p = P(BASE) it = p.iterdir() paths = set(it) - expected = ['dirA', 'dirB', 'dirC', 'fileA'] + expected = ['dirA', 'dirB', 'dirC', 'dirE', 'fileA'] if not symlink_skip_reason: expected += ['linkA', 'linkB', 'brokenLink'] self.assertEqual(paths, { P(BASE, q) for q in expected }) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1110,6 +1110,7 @@ Tim Peters Benjamin Peterson Joe Peterson +Ulrich Petri Chris Petrilli Roumen Petrov Bjorn Pettersen diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #24120: Ignore PermissionError when traversing a tree with + pathlib.Path.[r]glob(). Patch by Ulrich Petri. + - Issue #21815: Accept ] characters in the data portion of imap responses, in order to handle the flags with square brackets accepted and produced by servers such as gmail. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 12:54:13 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 17:54:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2324120=3A_Ignore_PermissionError_in_pathlib=2EPath=2E?= =?utf-8?b?W3JdZ2xvYigpLiBVbHJpY2ggUGV0cmku?= Message-ID: <20160106175408.7766.90463@psf.io> https://hg.python.org/cpython/rev/224a026b4ca1 changeset: 99776:224a026b4ca1 branch: 3.5 parent: 99767:164b564e3c1a parent: 99775:bac18cb7b011 user: Guido van Rossum date: Wed Jan 06 09:51:42 2016 -0800 summary: Issue #24120: Ignore PermissionError in pathlib.Path.[r]glob(). Ulrich Petri. (Merge 3.4->3.5) files: Lib/pathlib.py | 60 ++++++++++++++++----------- Lib/test/test_pathlib.py | 32 +++++++++----- Misc/ACKS | 1 + Misc/NEWS | 3 + 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -499,12 +499,15 @@ _Selector.__init__(self, child_parts) def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + path = parent_path._make_child_relpath(self.name) + if exists(path): + for p in self.successor._select_from(path, is_dir, exists, listdir): + yield p + except PermissionError: return - path = parent_path._make_child_relpath(self.name) - if exists(path): - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p class _WildcardSelector(_Selector): @@ -514,15 +517,19 @@ _Selector.__init__(self, child_parts) def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + cf = parent_path._flavour.casefold + for name in listdir(parent_path): + casefolded = cf(name) + if self.pat.match(casefolded): + path = parent_path._make_child_relpath(name) + for p in self.successor._select_from(path, is_dir, exists, listdir): + yield p + except PermissionError: return - cf = parent_path._flavour.casefold - for name in listdir(parent_path): - casefolded = cf(name) - if self.pat.match(casefolded): - path = parent_path._make_child_relpath(name) - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p + class _RecursiveWildcardSelector(_Selector): @@ -539,19 +546,22 @@ yield p def _select_from(self, parent_path, is_dir, exists, listdir): - if not is_dir(parent_path): + try: + if not is_dir(parent_path): + return + with _cached(listdir) as listdir: + yielded = set() + try: + successor_select = self.successor._select_from + for starting_point in self._iterate_directories(parent_path, is_dir, listdir): + for p in successor_select(starting_point, is_dir, exists, listdir): + if p not in yielded: + yield p + yielded.add(p) + finally: + yielded.clear() + except PermissionError: return - with _cached(listdir) as listdir: - yielded = set() - try: - successor_select = self.successor._select_from - for starting_point in self._iterate_directories(parent_path, is_dir, listdir): - for p in successor_select(starting_point, is_dir, exists, listdir): - if p not in yielded: - yield p - yielded.add(p) - finally: - yielded.clear() # diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1199,26 +1199,33 @@ # (BASE) # | - # |-- dirA/ - # |-- linkC -> "../dirB" - # |-- dirB/ - # | |-- fileB - # |-- linkD -> "../dirB" - # |-- dirC/ - # | |-- fileC - # | |-- fileD + # |-- brokenLink -> non-existing + # |-- dirA + # | `-- linkC -> ../dirB + # |-- dirB + # | |-- fileB + # | `-- linkD -> ../dirB + # |-- dirC + # | |-- dirD + # | | `-- fileD + # | `-- fileC + # |-- dirE # |-- fileA - # |-- linkA -> "fileA" - # |-- linkB -> "dirB" + # |-- linkA -> fileA + # `-- linkB -> dirB # def setUp(self): + def cleanup(): + os.chmod(join('dirE'), 0o777) + support.rmtree(BASE) + self.addCleanup(cleanup) os.mkdir(BASE) - self.addCleanup(support.rmtree, BASE) os.mkdir(join('dirA')) os.mkdir(join('dirB')) os.mkdir(join('dirC')) os.mkdir(join('dirC', 'dirD')) + os.mkdir(join('dirE')) with open(join('fileA'), 'wb') as f: f.write(b"this is file A\n") with open(join('dirB', 'fileB'), 'wb') as f: @@ -1227,6 +1234,7 @@ f.write(b"this is file C\n") with open(join('dirC', 'dirD', 'fileD'), 'wb') as f: f.write(b"this is file D\n") + os.chmod(join('dirE'), 0) if not symlink_skip_reason: # Relative symlinks os.symlink('fileA', join('linkA')) @@ -1363,7 +1371,7 @@ p = P(BASE) it = p.iterdir() paths = set(it) - expected = ['dirA', 'dirB', 'dirC', 'fileA'] + expected = ['dirA', 'dirB', 'dirC', 'dirE', 'fileA'] if not symlink_skip_reason: expected += ['linkA', 'linkB', 'brokenLink'] self.assertEqual(paths, { P(BASE, q) for q in expected }) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1106,6 +1106,7 @@ Tim Peters Benjamin Peterson Joe Peterson +Ulrich Petri Chris Petrilli Roumen Petrov Bjorn Pettersen diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,9 @@ Library ------- +- Issue #24120: Ignore PermissionError when traversing a tree with + pathlib.Path.[r]glob(). Patch by Ulrich Petri. + - Issue #25447: fileinput now uses sys.stdin as-is if it does not have a buffer attribute (restores backward compatibility). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 13:36:38 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 18:36:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI2MDEy?= =?utf-8?q?=3A_Don=27t_traverse_into_symlinks_for_**_pattern_in?= Message-ID: <20160106183637.21231.2143@psf.io> https://hg.python.org/cpython/rev/18f5b125a863 changeset: 99778:18f5b125a863 branch: 3.4 parent: 99775:bac18cb7b011 user: Guido van Rossum date: Wed Jan 06 10:31:33 2016 -0800 summary: Issue #26012: Don't traverse into symlinks for ** pattern in pathlib.Path.[r]glob(). files: Lib/pathlib.py | 2 +- Lib/test/test_pathlib.py | 19 ++++++++++++++++++- Misc/NEWS | 3 +++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -496,7 +496,7 @@ yield parent_path for name in listdir(parent_path): path = parent_path._make_child_relpath(name) - if is_dir(path): + if is_dir(path) and not path.is_symlink(): for p in self._iterate_directories(path, is_dir, listdir): yield p diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1245,7 +1245,7 @@ os.symlink('non-existing', join('brokenLink')) self.dirlink('dirB', join('linkB')) self.dirlink(os.path.join('..', 'dirB'), join('dirA', 'linkC')) - # This one goes upwards but doesn't create a loop + # This one goes upwards, creating a loop self.dirlink(os.path.join('..', 'dirB'), join('dirB', 'linkD')) if os.name == 'nt': @@ -1380,6 +1380,23 @@ _check(p.rglob("file*"), ["dirC/fileC", "dirC/dirD/fileD"]) _check(p.rglob("*/*"), ["dirC/dirD/fileD"]) + @with_symlinks + def test_rglob_symlink_loop(self): + # Don't get fooled by symlink loops (Issue #26012) + P = self.cls + p = P(BASE) + given = set(p.rglob('*')) + expect = {'brokenLink', + 'dirA', 'dirA/linkC', + 'dirB', 'dirB/fileB', 'dirB/linkD', + 'dirC', 'dirC/dirD', 'dirC/dirD/fileD', 'dirC/fileC', + 'dirE', + 'fileA', + 'linkA', + 'linkB', + } + self.assertEqual(given, {p / x for x in expect}) + def test_glob_dotdot(self): # ".." is not special in globs P = self.cls diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- Issue #26012: Don't traverse into symlinks for ** pattern in + pathlib.Path.[r]glob(). + - Issue #24120: Ignore PermissionError when traversing a tree with pathlib.Path.[r]glob(). Patch by Ulrich Petri. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 13:36:38 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 18:36:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326012=3A_Don=27t_traverse_into_symlinks_for_**_?= =?utf-8?q?pattern_in?= Message-ID: <20160106183637.21231.9396@psf.io> https://hg.python.org/cpython/rev/36864abbfe02 changeset: 99780:36864abbfe02 parent: 99777:f6ae90450a4d parent: 99779:9826dbad1252 user: Guido van Rossum date: Wed Jan 06 10:36:19 2016 -0800 summary: Issue #26012: Don't traverse into symlinks for ** pattern in pathlib.Path.[r]glob(). (Merge 3.5->3.6) files: Lib/pathlib.py | 2 +- Lib/test/test_pathlib.py | 19 ++++++++++++++++++- Misc/NEWS | 3 +++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -541,7 +541,7 @@ yield parent_path for name in listdir(parent_path): path = parent_path._make_child_relpath(name) - if is_dir(path): + if is_dir(path) and not path.is_symlink(): for p in self._iterate_directories(path, is_dir, listdir): yield p diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1241,7 +1241,7 @@ os.symlink('non-existing', join('brokenLink')) self.dirlink('dirB', join('linkB')) self.dirlink(os.path.join('..', 'dirB'), join('dirA', 'linkC')) - # This one goes upwards but doesn't create a loop + # This one goes upwards, creating a loop self.dirlink(os.path.join('..', 'dirB'), join('dirB', 'linkD')) if os.name == 'nt': @@ -1437,6 +1437,23 @@ _check(p.rglob("file*"), ["dirC/fileC", "dirC/dirD/fileD"]) _check(p.rglob("*/*"), ["dirC/dirD/fileD"]) + @with_symlinks + def test_rglob_symlink_loop(self): + # Don't get fooled by symlink loops (Issue #26012) + P = self.cls + p = P(BASE) + given = set(p.rglob('*')) + expect = {'brokenLink', + 'dirA', 'dirA/linkC', + 'dirB', 'dirB/fileB', 'dirB/linkD', + 'dirC', 'dirC/dirD', 'dirC/dirD/fileD', 'dirC/fileC', + 'dirE', + 'fileA', + 'linkA', + 'linkB', + } + self.assertEqual(given, {p / x for x in expect}) + def test_glob_dotdot(self): # ".." is not special in globs P = self.cls diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #26012: Don't traverse into symlinks for ** pattern in + pathlib.Path.[r]glob(). + - Issue #24120: Ignore PermissionError when traversing a tree with pathlib.Path.[r]glob(). Patch by Ulrich Petri. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 13:36:38 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 18:36:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2326012=3A_Don=27t_traverse_into_symlinks_for_**_patter?= =?utf-8?q?n_in?= Message-ID: <20160106183637.3393.4407@psf.io> https://hg.python.org/cpython/rev/9826dbad1252 changeset: 99779:9826dbad1252 branch: 3.5 parent: 99776:224a026b4ca1 parent: 99778:18f5b125a863 user: Guido van Rossum date: Wed Jan 06 10:35:30 2016 -0800 summary: Issue #26012: Don't traverse into symlinks for ** pattern in pathlib.Path.[r]glob(). (Merge 3.4->3.5) files: Lib/pathlib.py | 2 +- Lib/test/test_pathlib.py | 19 ++++++++++++++++++- Misc/NEWS | 3 +++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -541,7 +541,7 @@ yield parent_path for name in listdir(parent_path): path = parent_path._make_child_relpath(name) - if is_dir(path): + if is_dir(path) and not path.is_symlink(): for p in self._iterate_directories(path, is_dir, listdir): yield p diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1241,7 +1241,7 @@ os.symlink('non-existing', join('brokenLink')) self.dirlink('dirB', join('linkB')) self.dirlink(os.path.join('..', 'dirB'), join('dirA', 'linkC')) - # This one goes upwards but doesn't create a loop + # This one goes upwards, creating a loop self.dirlink(os.path.join('..', 'dirB'), join('dirB', 'linkD')) if os.name == 'nt': @@ -1437,6 +1437,23 @@ _check(p.rglob("file*"), ["dirC/fileC", "dirC/dirD/fileD"]) _check(p.rglob("*/*"), ["dirC/dirD/fileD"]) + @with_symlinks + def test_rglob_symlink_loop(self): + # Don't get fooled by symlink loops (Issue #26012) + P = self.cls + p = P(BASE) + given = set(p.rglob('*')) + expect = {'brokenLink', + 'dirA', 'dirA/linkC', + 'dirB', 'dirB/fileB', 'dirB/linkD', + 'dirC', 'dirC/dirD', 'dirC/dirD/fileD', 'dirC/fileC', + 'dirE', + 'fileA', + 'linkA', + 'linkB', + } + self.assertEqual(given, {p / x for x in expect}) + def test_glob_dotdot(self): # ".." is not special in globs P = self.cls diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,9 @@ Library ------- +- Issue #26012: Don't traverse into symlinks for ** pattern in + pathlib.Path.[r]glob(). + - Issue #24120: Ignore PermissionError when traversing a tree with pathlib.Path.[r]glob(). Patch by Ulrich Petri. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:04:20 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:04:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIyNTcw?= =?utf-8?q?=3A_Add_=27path=27_attribute_to_pathlib=2EPath_objects=2E?= Message-ID: <20160106190419.8269.73649@psf.io> https://hg.python.org/cpython/rev/7e9605697dfc changeset: 99781:7e9605697dfc branch: 3.4 parent: 99778:18f5b125a863 user: Guido van Rossum date: Wed Jan 06 11:01:42 2016 -0800 summary: Issue #22570: Add 'path' attribute to pathlib.Path objects. files: Lib/pathlib.py | 7 +++++++ Lib/test/test_pathlib.py | 27 +++++++++++++++++++++++++++ Misc/NEWS | 6 ++++++ 3 files changed, 40 insertions(+), 0 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -645,6 +645,13 @@ self._parts) or '.' return self._str + @property + def path(self): + try: + return self._str + except AttributeError: + return str(self) + def as_posix(self): """Return the string representation of the path with forward (/) slashes.""" diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -480,6 +480,22 @@ self.assertEqual(P('a/b.py').name, 'b.py') self.assertEqual(P('/a/b.py').name, 'b.py') + def test_path_common(self): + P = self.cls + def check(arg, expected=None): + if expected is None: + expected = arg + self.assertEqual(P(arg).path, expected.replace('/', self.sep)) + check('', '.') + check('.') + check('/') + check('a/b') + check('/a/b') + check('/a/b/', '/a/b') + check('/a/b/.', '/a/b') + check('a/b.py') + check('/a/b.py') + def test_suffix_common(self): P = self.cls self.assertEqual(P('').suffix, '') @@ -903,6 +919,17 @@ self.assertEqual(P('//My.py/Share.php').name, '') self.assertEqual(P('//My.py/Share.php/a/b').name, 'b') + def test_path(self): + P = self.cls + self.assertEqual(P('c:').path, 'c:') + self.assertEqual(P('c:/').path, 'c:\\') + self.assertEqual(P('c:a/b').path, 'c:a\\b') + self.assertEqual(P('c:/a/b').path, 'c:\\a\\b') + self.assertEqual(P('c:a/b.py').path, 'c:a\\b.py') + self.assertEqual(P('c:/a/b.py').path, 'c:\\a\\b.py') + self.assertEqual(P('//My.py/Share.php').path, '\\\\My.py\\Share.php\\') + self.assertEqual(P('//My.py/Share.php/a/b').path, '\\\\My.py\\Share.php\\a\\b') + def test_suffix(self): P = self.cls self.assertEqual(P('c:').suffix, '') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,12 @@ Library ------- +- Issue #22570: Add 'path' attribute to pathlib.Path objects, + returning the same as str(), to make it more similar to DirEntry. + Library code can now write getattr(p, 'path', p) to get the path as + a string from a Path, a DirEntry, or a plain string. This is + essentially a small one-off protocol. + - Issue #26012: Don't traverse into symlinks for ** pattern in pathlib.Path.[r]glob(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:04:28 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:04:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2322570=3A_Add_=27path=27_attribute_to_pathlib=2E?= =?utf-8?b?UGF0aCBvYmplY3RzLiAoTWVyZ2UgMy41LT4zLjYp?= Message-ID: <20160106190419.17335.42216@psf.io> https://hg.python.org/cpython/rev/d5f96a5da219 changeset: 99783:d5f96a5da219 parent: 99780:36864abbfe02 parent: 99782:9c49c417a68a user: Guido van Rossum date: Wed Jan 06 11:03:47 2016 -0800 summary: Issue #22570: Add 'path' attribute to pathlib.Path objects. (Merge 3.5->3.6) files: Lib/pathlib.py | 7 +++++++ Lib/test/test_pathlib.py | 27 +++++++++++++++++++++++++++ Misc/NEWS | 6 ++++++ 3 files changed, 40 insertions(+), 0 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -690,6 +690,13 @@ self._parts) or '.' return self._str + @property + def path(self): + try: + return self._str + except AttributeError: + return str(self) + def as_posix(self): """Return the string representation of the path with forward (/) slashes.""" diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -477,6 +477,22 @@ self.assertEqual(P('a/b.py').name, 'b.py') self.assertEqual(P('/a/b.py').name, 'b.py') + def test_path_common(self): + P = self.cls + def check(arg, expected=None): + if expected is None: + expected = arg + self.assertEqual(P(arg).path, expected.replace('/', self.sep)) + check('', '.') + check('.') + check('/') + check('a/b') + check('/a/b') + check('/a/b/', '/a/b') + check('/a/b/.', '/a/b') + check('a/b.py') + check('/a/b.py') + def test_suffix_common(self): P = self.cls self.assertEqual(P('').suffix, '') @@ -899,6 +915,17 @@ self.assertEqual(P('//My.py/Share.php').name, '') self.assertEqual(P('//My.py/Share.php/a/b').name, 'b') + def test_path(self): + P = self.cls + self.assertEqual(P('c:').path, 'c:') + self.assertEqual(P('c:/').path, 'c:\\') + self.assertEqual(P('c:a/b').path, 'c:a\\b') + self.assertEqual(P('c:/a/b').path, 'c:\\a\\b') + self.assertEqual(P('c:a/b.py').path, 'c:a\\b.py') + self.assertEqual(P('c:/a/b.py').path, 'c:\\a\\b.py') + self.assertEqual(P('//My.py/Share.php').path, '\\\\My.py\\Share.php\\') + self.assertEqual(P('//My.py/Share.php/a/b').path, '\\\\My.py\\Share.php\\a\\b') + def test_suffix(self): P = self.cls self.assertEqual(P('c:').suffix, '') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,12 @@ Library ------- +- Issue #22570: Add 'path' attribute to pathlib.Path objects, + returning the same as str(), to make it more similar to DirEntry. + Library code can now write getattr(p, 'path', p) to get the path as + a string from a Path, a DirEntry, or a plain string. This is + essentially a small one-off protocol. + - Issue #26012: Don't traverse into symlinks for ** pattern in pathlib.Path.[r]glob(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:04:31 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:04:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2322570=3A_Add_=27path=27_attribute_to_pathlib=2EPath_o?= =?utf-8?b?YmplY3RzLiAoTWVyZ2UgMy40LT4zLjUp?= Message-ID: <20160106190419.17325.8113@psf.io> https://hg.python.org/cpython/rev/9c49c417a68a changeset: 99782:9c49c417a68a branch: 3.5 parent: 99779:9826dbad1252 parent: 99781:7e9605697dfc user: Guido van Rossum date: Wed Jan 06 11:03:15 2016 -0800 summary: Issue #22570: Add 'path' attribute to pathlib.Path objects. (Merge 3.4->3.5) files: Lib/pathlib.py | 7 +++++++ Lib/test/test_pathlib.py | 27 +++++++++++++++++++++++++++ Misc/NEWS | 6 ++++++ 3 files changed, 40 insertions(+), 0 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -690,6 +690,13 @@ self._parts) or '.' return self._str + @property + def path(self): + try: + return self._str + except AttributeError: + return str(self) + def as_posix(self): """Return the string representation of the path with forward (/) slashes.""" diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -477,6 +477,22 @@ self.assertEqual(P('a/b.py').name, 'b.py') self.assertEqual(P('/a/b.py').name, 'b.py') + def test_path_common(self): + P = self.cls + def check(arg, expected=None): + if expected is None: + expected = arg + self.assertEqual(P(arg).path, expected.replace('/', self.sep)) + check('', '.') + check('.') + check('/') + check('a/b') + check('/a/b') + check('/a/b/', '/a/b') + check('/a/b/.', '/a/b') + check('a/b.py') + check('/a/b.py') + def test_suffix_common(self): P = self.cls self.assertEqual(P('').suffix, '') @@ -899,6 +915,17 @@ self.assertEqual(P('//My.py/Share.php').name, '') self.assertEqual(P('//My.py/Share.php/a/b').name, 'b') + def test_path(self): + P = self.cls + self.assertEqual(P('c:').path, 'c:') + self.assertEqual(P('c:/').path, 'c:\\') + self.assertEqual(P('c:a/b').path, 'c:a\\b') + self.assertEqual(P('c:/a/b').path, 'c:\\a\\b') + self.assertEqual(P('c:a/b.py').path, 'c:a\\b.py') + self.assertEqual(P('c:/a/b.py').path, 'c:\\a\\b.py') + self.assertEqual(P('//My.py/Share.php').path, '\\\\My.py\\Share.php\\') + self.assertEqual(P('//My.py/Share.php/a/b').path, '\\\\My.py\\Share.php\\a\\b') + def test_suffix(self): P = self.cls self.assertEqual(P('c:').suffix, '') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,12 @@ Library ------- +- Issue #22570: Add 'path' attribute to pathlib.Path objects, + returning the same as str(), to make it more similar to DirEntry. + Library code can now write getattr(p, 'path', p) to get the path as + a string from a Path, a DirEntry, or a plain string. This is + essentially a small one-off protocol. + - Issue #26012: Don't traverse into symlinks for ** pattern in pathlib.Path.[r]glob(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:38:35 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:38:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Docs_for_issue?= =?utf-8?q?_=2322570=2E?= Message-ID: <20160106193834.7758.12365@psf.io> https://hg.python.org/cpython/rev/2e3c31ab586a changeset: 99784:2e3c31ab586a branch: 3.4 parent: 99781:7e9605697dfc user: Guido van Rossum date: Wed Jan 06 11:15:52 2016 -0800 summary: Docs for issue #22570. files: Doc/library/pathlib.rst | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -365,6 +365,23 @@ '' +.. data:: PurePath.path + + A string representing the full path:: + + >>> PurePosixPath('my/library/setup.py').path + 'my/library/setup.py' + + This always returns the same value as ``str(p)``; it is included to + serve as a one-off protocol. Code that wants to support both + strings and ``pathlib.Path`` objects as filenames can write + ``arg = getattr(arg, 'path', arg)`` to get the path as a string. + This can then be passed to various system calls or library + functions that expect a string. Unlike the alternative + ``arg = str(arg)``, this will still raise an exception if an object + of some other type is given by accident. + + .. data:: PurePath.suffix The file extension of the final component, if any:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:38:35 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:38:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Docs_for_issue_=2322570=2E_=28Merge_3=2E4-=3E3=2E5=29?= Message-ID: <20160106193835.21227.95549@psf.io> https://hg.python.org/cpython/rev/408f8b255b56 changeset: 99785:408f8b255b56 branch: 3.5 parent: 99782:9c49c417a68a parent: 99784:2e3c31ab586a user: Guido van Rossum date: Wed Jan 06 11:16:28 2016 -0800 summary: Docs for issue #22570. (Merge 3.4->3.5) files: Doc/library/pathlib.rst | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -365,6 +365,23 @@ '' +.. data:: PurePath.path + + A string representing the full path:: + + >>> PurePosixPath('my/library/setup.py').path + 'my/library/setup.py' + + This always returns the same value as ``str(p)``; it is included to + serve as a one-off protocol. Code that wants to support both + strings and ``pathlib.Path`` objects as filenames can write + ``arg = getattr(arg, 'path', arg)`` to get the path as a string. + This can then be passed to various system calls or library + functions that expect a string. Unlike the alternative + ``arg = str(arg)``, this will still raise an exception if an object + of some other type is given by accident. + + .. data:: PurePath.suffix The file extension of the final component, if any:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:38:35 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:38:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_versionadd?= =?utf-8?q?ed_=283=2E4=2E5=29_to_docs_for_issue_=2322570=2E?= Message-ID: <20160106193835.17337.74445@psf.io> https://hg.python.org/cpython/rev/759b2cecc289 changeset: 99786:759b2cecc289 branch: 3.4 parent: 99784:2e3c31ab586a user: Guido van Rossum date: Wed Jan 06 11:23:31 2016 -0800 summary: Add versionadded (3.4.5) to docs for issue #22570. files: Doc/library/pathlib.rst | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -381,6 +381,7 @@ ``arg = str(arg)``, this will still raise an exception if an object of some other type is given by accident. + .. versionadded:: 3.4.5 .. data:: PurePath.suffix -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:38:35 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:38:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogRG9jcyBmb3IgaXNzdWUgIzIyNTcwLiAoTWVyZ2UgMy41LT4zLjYp?= Message-ID: <20160106193835.3387.68842@psf.io> https://hg.python.org/cpython/rev/97ab0ccac893 changeset: 99789:97ab0ccac893 parent: 99783:d5f96a5da219 parent: 99788:eab349b5c6d7 user: Guido van Rossum date: Wed Jan 06 11:37:52 2016 -0800 summary: Docs for issue #22570. (Merge 3.5->3.6) files: Doc/library/os.rst | 6 ++++++ Doc/library/pathlib.rst | 22 ++++++++++++++++++++++ 2 files changed, 28 insertions(+), 0 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2035,6 +2035,12 @@ The result is cached on the ``DirEntry`` object. Call :func:`os.stat` to fetch up-to-date information. + Note that there is a nice correspondence between several attributes + and methods of ``DirEntry`` and of :class:`pathlib.Path`. In + particular, the ``name`` and ``path`` attributes have the same + meaning, as do the ``is_dir()``, ``is_file()``, ``is_symlink()`` + and ``stat()`` methods. + .. versionadded:: 3.5 diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -365,6 +365,28 @@ '' +.. data:: PurePath.path + + A string representing the full path:: + + >>> PurePosixPath('my/library/setup.py').path + 'my/library/setup.py' + + This always returns the same value as ``str(p)``; it is included to + serve as a one-off protocol. Code that wants to support both + strings and ``pathlib.Path`` objects as filenames can write + ``arg = getattr(arg, 'path', arg)`` to get the path as a string. + This can then be passed to various system calls or library + functions that expect a string. Unlike the alternative + ``arg = str(arg)``, this will still raise an exception if an object + of some other type is given by accident. + + A nice advantage is that this protocol is also supported by + :class:`os.DirEntry` objects returned by :func:`os.scandir`. + + .. versionadded:: 3.4.5 + .. versionadded:: 3.5.2 + .. data:: PurePath.suffix The file extension of the final component, if any:: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:38:35 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:38:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Cross-referenc?= =?utf-8?q?e_os=2EDirEntry_and_pathlib=2EPath_for_issue_=2322570=2E?= Message-ID: <20160106193835.3365.36605@psf.io> https://hg.python.org/cpython/rev/eab349b5c6d7 changeset: 99788:eab349b5c6d7 branch: 3.5 user: Guido van Rossum date: Wed Jan 06 11:36:03 2016 -0800 summary: Cross-reference os.DirEntry and pathlib.Path for issue #22570. files: Doc/library/os.rst | 6 ++++++ Doc/library/pathlib.rst | 4 ++++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2035,6 +2035,12 @@ The result is cached on the ``DirEntry`` object. Call :func:`os.stat` to fetch up-to-date information. + Note that there is a nice correspondence between several attributes + and methods of ``DirEntry`` and of :class:`pathlib.Path`. In + particular, the ``name`` and ``path`` attributes have the same + meaning, as do the ``is_dir()``, ``is_file()``, ``is_symlink()`` + and ``stat()`` methods. + .. versionadded:: 3.5 diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -381,7 +381,11 @@ ``arg = str(arg)``, this will still raise an exception if an object of some other type is given by accident. + A nice advantage is that this protocol is also supported by + :class:`os.DirEntry` objects returned by :func:`os.scandir`. + .. versionadded:: 3.4.5 + .. versionadded:: 3.5.2 .. data:: PurePath.suffix -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 6 14:38:36 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 06 Jan 2016 19:38:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Add_versionadded_=283=2E4=2E5=29_to_docs_for_issue_=2322570=2E?= =?utf-8?b?IChNZXJnZSAzLjQtPjMuNSk=?= Message-ID: <20160106193835.53708.2683@psf.io> https://hg.python.org/cpython/rev/1a6b485e717f changeset: 99787:1a6b485e717f branch: 3.5 parent: 99785:408f8b255b56 parent: 99786:759b2cecc289 user: Guido van Rossum date: Wed Jan 06 11:26:36 2016 -0800 summary: Add versionadded (3.4.5) to docs for issue #22570. (Merge 3.4->3.5) files: Doc/library/pathlib.rst | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -381,6 +381,7 @@ ``arg = str(arg)``, this will still raise an exception if an object of some other type is given by accident. + .. versionadded:: 3.4.5 .. data:: PurePath.suffix -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 00:31:17 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 07 Jan 2016 05:31:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_from_3=2E4?= Message-ID: <20160107053117.21250.40548@psf.io> https://hg.python.org/cpython/rev/ebf164b07be1 changeset: 99791:ebf164b07be1 branch: 3.5 parent: 99788:eab349b5c6d7 parent: 99790:94b0e5e9b6e8 user: Senthil Kumaran date: Wed Jan 06 21:30:30 2016 -0800 summary: merge from 3.4 Issue16544 - Add a link to an external documentation resource in ast module docs. files: Doc/library/ast.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -249,3 +249,8 @@ wanted *annotate_fields* must be set to ``False``. Attributes such as line numbers and column offsets are not dumped by default. If this is wanted, *include_attributes* can be set to ``True``. + +.. seealso:: + + `Green Tree Snakes `_, an external documentation resource, has good + details on working with Python ASTs. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 00:31:17 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 07 Jan 2016 05:31:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Issue16544_-_A?= =?utf-8?q?dd_a_link_to_an_external_documentation_resource_in_ast_module?= Message-ID: <20160107053117.17325.21453@psf.io> https://hg.python.org/cpython/rev/94b0e5e9b6e8 changeset: 99790:94b0e5e9b6e8 branch: 3.4 parent: 99786:759b2cecc289 user: Senthil Kumaran date: Wed Jan 06 21:26:53 2016 -0800 summary: Issue16544 - Add a link to an external documentation resource in ast module docs. files: Doc/library/ast.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -249,3 +249,8 @@ wanted *annotate_fields* must be set to ``False``. Attributes such as line numbers and column offsets are not dumped by default. If this is wanted, *include_attributes* can be set to ``True``. + +.. seealso:: + + `Green Tree Snakes `_, an external documentation resource, has good + details on working with Python ASTs. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 00:31:18 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 07 Jan 2016 05:31:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160107053117.53704.5040@psf.io> https://hg.python.org/cpython/rev/62e88e5a61c0 changeset: 99792:62e88e5a61c0 parent: 99789:97ab0ccac893 parent: 99791:ebf164b07be1 user: Senthil Kumaran date: Wed Jan 06 21:31:09 2016 -0800 summary: merge from 3.5 Issue16544 - Add a link to an external documentation resource in ast module docs. files: Doc/library/ast.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -249,3 +249,8 @@ wanted *annotate_fields* must be set to ``False``. Attributes such as line numbers and column offsets are not dumped by default. If this is wanted, *include_attributes* can be set to ``True``. + +.. seealso:: + + `Green Tree Snakes `_, an external documentation resource, has good + details on working with Python ASTs. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 01:38:51 2016 From: python-checkins at python.org (ned.deily) Date: Thu, 07 Jan 2016 06:38:51 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Include_Benjamin=27s_remi?= =?utf-8?q?nder_to_fix_hard_failing_tests_in_security_branches=2E?= Message-ID: <20160107063850.69621.79335@psf.io> https://hg.python.org/devguide/rev/20bde416f0d2 changeset: 779:20bde416f0d2 user: Ned Deily date: Thu Jan 07 01:38:28 2016 -0500 summary: Include Benjamin's reminder to fix hard failing tests in security branches. files: devcycle.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/devcycle.rst b/devcycle.rst --- a/devcycle.rst +++ b/devcycle.rst @@ -103,6 +103,8 @@ by attackers such as crashes, privilege escalation and, optionally, other issues such as denial of service attacks. Any other changes are **not** considered a security risk and thus not backported to a security branch. +You should also consider fixing hard-failing tests in open security branches +since it is important to be able to run the tests successfully before releasing. Commits to security branches are to be coordinated with the release manager for the corresponding feature version, as listed below in the Summary_. -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Thu Jan 7 01:38:51 2016 From: python-checkins at python.org (ned.deily) Date: Thu, 07 Jan 2016 06:38:51 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Update_branch_and_release?= =?utf-8?q?_versions_to_reflect_release_of_3=2E5_and_the_transition?= Message-ID: <20160107063850.7754.43732@psf.io> https://hg.python.org/devguide/rev/f53eac8da0b8 changeset: 778:f53eac8da0b8 user: Ned Deily date: Thu Jan 07 01:18:29 2016 -0500 summary: Update branch and release versions to reflect release of 3.5 and the transition of 3.4.x to security-fix-only mode. files: committing.rst | 36 ++++++++++++++++++------------------ devcycle.rst | 30 ++++++++++++++++-------------- 2 files changed, 34 insertions(+), 32 deletions(-) diff --git a/committing.rst b/committing.rst --- a/committing.rst +++ b/committing.rst @@ -409,15 +409,15 @@ $ hg import --no-c http://bugs.python.org/url/to/the/patch.diff $ # review, run tests, run `make patchcheck` $ hg ci -m '#12345: fix some issue.' - $ # switch to 3.4 and port the changeset using `hg graft` - $ cd ../3.4 + $ # switch to 3.5 and port the changeset using `hg graft` + $ cd ../3.5 $ hg up $ hg graft 2.7 $ # switch to 3.x, merge, commit, and push everything $ cd ../3.x $ hg up - $ hg merge 3.4 - $ hg ci -m '#12345: merge with 3.4.' + $ hg merge 3.5 + $ hg ci -m '#12345: merge with 3.5.' $ hg push Unless noted otherwise, the rest of the page will assume you are using the @@ -458,25 +458,25 @@ Merging between different branches (within the same major version) ------------------------------------------------------------------ -Assume that Python 3.5 is the current in-development version of Python and that -you have a patch that should also be applied to Python 3.4. To properly port +Assume that Python 3.6 is the current in-development version of Python and that +you have a patch that should also be applied to Python 3.5. To properly port the patch to both versions of Python, you should first apply the patch to -Python 3.4:: +Python 3.5:: - cd 3.4 + cd 3.5 hg import --no-commit patch.diff # Compile; run the test suite hg ci -m '#12345: fix some issue.' Then you can switch to the ``3.5`` clone, merge, run the tests and commit:: - cd ../3.5 - hg merge 3.4 + cd ../3.6 + hg merge 3.5 # Fix any conflicts (e.g. ``hg revert -r default Misc/NEWS``); compile; run the test suite - hg ci -m '#12345: merge with 3.4.' + hg ci -m '#12345: merge with 3.5.' If you are not using the share extension, you will need to use -``hg pull ../3.4`` before being able to merge. +``hg pull ../3.5`` before being able to merge. .. note:: Even when porting an already committed patch, you should *still* check the @@ -488,13 +488,13 @@ Porting changesets between the two major Python versions (2.x and 3.x) ---------------------------------------------------------------------- -Assume you just committed something on ``2.7``, and want to port it to ``3.4``. +Assume you just committed something on ``2.7``, and want to port it to ``3.5``. You can use ``hg graft`` as follow:: - cd ../3.4 + cd ../3.5 hg graft 2.7 -This will port the latest changeset committed in the 2.7 clone to the 3.4 clone. +This will port the latest changeset committed in the 2.7 clone to the 3.5 clone. ``hg graft`` always commits automatically, except in case of conflicts, when you have to resolve them and run ``hg graft --continue`` afterwards. Instead of the branch name you can also specify a changeset id, and you can @@ -502,15 +502,15 @@ On older version of Mercurial where ``hg graft`` is not available, you can use:: - cd ../3.4 + cd ../3.5 hg export 2.7 | hg import - The result will be the same, but in case of conflict this will create ``.rej`` files rather than using Mercurial merge capabilities. -A third option is to apply manually the patch on ``3.4``. This is convenient +A third option is to apply manually the patch on ``3.5``. This is convenient when there are too many differences with ``2.7`` or when there is already a -specific patch for ``3.4``. +specific patch for ``3.5``. .. warning:: Never use ``hg merge`` to port changes between 2.x and 3.x (or vice versa). diff --git a/devcycle.rst b/devcycle.rst --- a/devcycle.rst +++ b/devcycle.rst @@ -35,13 +35,13 @@ no merging happens between 2.x and 3.x branches. In each of the 2.x and 3.x realms, the branch for a feature version is always a -descendant of the previous feature version: for example, the ``3.3`` branch is a -descendant of the ``3.2`` branch. +descendant of the previous feature version: for example, the ``3.5`` branch is a +descendant of the ``3.4`` branch. Therefore, each change should be made **first** in the oldest branch to which it applies and forward-ported as appropriate: if a bug must be fixed in both Python -3.4 and 3.5, first fix it in ``3.4`` and then merge ``3.4`` into ``default`` -(which holds the future 3.5). +3.5 and 3.6, first fix it in ``3.5`` and then merge ``3.5`` into ``default`` +(which holds the future 3.6). .. _indevbranch: @@ -80,7 +80,7 @@ The only changes allowed to occur in a maintenance branch without debate are bug fixes. Also, a general rule for maintenance branches is that compatibility -must not be broken at any point between sibling minor releases (3.4.1, 3.4.2, +must not be broken at any point between sibling minor releases (3.5.1, 3.5.2, etc.). For both rules, only rare exceptions are accepted and **must** be discussed first. @@ -88,9 +88,9 @@ is released), the old maintenance branch on that major version will go into :ref:`security mode `, usually after one last maintenance release at the discretion of the -release manager. For example, the 3.2 maintenance branch was put into -:ref:`security mode ` after the 3.2.4 final maintenance release -following the release of 3.3.0. +release manager. For example, the 3.4 maintenance branch was put into +:ref:`security mode ` after the 3.4.4 final maintenance release +following the release of 3.5.1. .. _secbranch: @@ -115,16 +115,18 @@ Summary ------- -There are 5 open branches right now in the Mercurial repository: +There are 6 open branches right now in the Mercurial repository: -- the ``default`` branch holds the future 3.5 version and descends from ``3.4`` - (RM: Larry Hastings) -- the ``3.4`` branch holds bug fixes for future 3.4.x maintenance releases +- the ``default`` branch holds the future 3.6 version and descends from ``3.5`` + (RM: Ned Deily) +- the ``3.5`` branch holds bug fixes for future 3.5.x maintenance releases + and descends from ``3.4`` (RM: Larry Hastings) +- the ``3.4`` branch holds security fixes for future 3.4.x security releases and descends from ``3.3`` (RM: Larry Hastings) - the ``3.3`` branch holds security fixes for future 3.3.x security releases - and descends from ``3.2`` (RM: Georg Brandl) + until September 2017 and descends from ``3.2`` (RM: Georg Brandl) - the ``3.2`` branch holds security fixes for future 3.2.x security releases - (RM: Georg Brandl) + until February 2016 (RM: Georg Brandl) - the ``2.7`` branch holds bug fixes for future 2.7.x maintenance releases and descends from ``2.6`` (RM: Benjamin Peterson) -- Repository URL: https://hg.python.org/devguide From solipsis at pitrou.net Thu Jan 7 03:45:00 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 07 Jan 2016 08:45:00 +0000 Subject: [Python-checkins] Daily reference leaks (62e88e5a61c0): sum=9 Message-ID: <20160107084500.21378.63380@psf.io> results for 62e88e5a61c0 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_collections leaked [2, 0, 0] references, sum=2 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogfFsolj', '--timeout', '7200'] From python-checkins at python.org Thu Jan 7 11:45:53 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 07 Jan 2016 16:45:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzU1MDE6?= =?utf-8?q?_Clarify_that_invoking_freeze=5Fsupport=28=29_on_non-Windows_pl?= =?utf-8?q?atforms?= Message-ID: <20160107164553.21390.37302@psf.io> https://hg.python.org/cpython/rev/f08d4712a055 changeset: 99793:f08d4712a055 branch: 3.5 parent: 99791:ebf164b07be1 user: Berker Peksag date: Thu Jan 07 18:45:22 2016 +0200 summary: Issue #5501: Clarify that invoking freeze_support() on non-Windows platforms has no effect Patch by Davin Potts and Camilla Montonen. files: Doc/library/multiprocessing.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -901,8 +901,10 @@ If the ``freeze_support()`` line is omitted then trying to run the frozen executable will raise :exc:`RuntimeError`. - If the module is being run normally by the Python interpreter then - :func:`freeze_support` has no effect. + Calling ``freeze_support()`` has no effect when invoked on any operating + system other than Windows. In addition, if the module is being run + normally by the Python interpreter on Windows (the program has not been + frozen), then ``freeze_support()`` has no effect. .. function:: get_all_start_methods() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 11:45:53 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 07 Jan 2016 16:45:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=235501=3A_Clarify_that_invoking_freeze=5Fsupport?= =?utf-8?q?=28=29_on_non-Windows_platforms?= Message-ID: <20160107164553.21244.40848@psf.io> https://hg.python.org/cpython/rev/2902f32d1ae9 changeset: 99794:2902f32d1ae9 parent: 99792:62e88e5a61c0 parent: 99793:f08d4712a055 user: Berker Peksag date: Thu Jan 07 18:45:47 2016 +0200 summary: Issue #5501: Clarify that invoking freeze_support() on non-Windows platforms has no effect Patch by Davin Potts and Camilla Montonen. files: Doc/library/multiprocessing.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -906,8 +906,10 @@ If the ``freeze_support()`` line is omitted then trying to run the frozen executable will raise :exc:`RuntimeError`. - If the module is being run normally by the Python interpreter then - :func:`freeze_support` has no effect. + Calling ``freeze_support()`` has no effect when invoked on any operating + system other than Windows. In addition, if the module is being run + normally by the Python interpreter on Windows (the program has not been + frozen), then ``freeze_support()`` has no effect. .. function:: get_all_start_methods() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 11:49:57 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 07 Jan 2016 16:49:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzU1MDE6?= =?utf-8?q?_Clarify_that_invoking_freeze=5Fsupport=28=29_on_non-Windows_pl?= =?utf-8?q?atforms?= Message-ID: <20160107164957.21378.18901@psf.io> https://hg.python.org/cpython/rev/8b20a606ecde changeset: 99795:8b20a606ecde branch: 2.7 parent: 99774:0792a0ad7f22 user: Berker Peksag date: Thu Jan 07 18:49:53 2016 +0200 summary: Issue #5501: Clarify that invoking freeze_support() on non-Windows platforms has no effect Patch by Davin Potts and Camilla Montonen. files: Doc/library/multiprocessing.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -751,8 +751,10 @@ If the ``freeze_support()`` line is omitted then trying to run the frozen executable will raise :exc:`RuntimeError`. - If the module is being run normally by the Python interpreter then - :func:`freeze_support` has no effect. + Calling ``freeze_support()`` has no effect when invoked on any operating + system other than Windows. In addition, if the module is being run + normally by the Python interpreter on Windows (the program has not been + frozen), then ``freeze_support()`` has no effect. .. function:: set_executable() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 12:03:21 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 07 Jan 2016 17:03:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fix_typo_in_docstring_of_multiprocessing=2Espawn=2Espawn?= =?utf-8?b?X21haW4oKQ==?= Message-ID: <20160107170320.21235.42094@psf.io> https://hg.python.org/cpython/rev/58e8125d7f4e changeset: 99797:58e8125d7f4e parent: 99794:2902f32d1ae9 parent: 99796:97ce60e8958c user: Berker Peksag date: Thu Jan 07 19:03:15 2016 +0200 summary: Fix typo in docstring of multiprocessing.spawn.spawn_main() files: Lib/multiprocessing/spawn.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/spawn.py b/Lib/multiprocessing/spawn.py --- a/Lib/multiprocessing/spawn.py +++ b/Lib/multiprocessing/spawn.py @@ -91,7 +91,7 @@ def spawn_main(pipe_handle, parent_pid=None, tracker_fd=None): ''' - Run code specifed by data received over pipe + Run code specified by data received over pipe ''' assert is_forking(sys.argv) if sys.platform == 'win32': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 12:03:21 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 07 Jan 2016 17:03:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_typo_in_do?= =?utf-8?q?cstring_of_multiprocessing=2Espawn=2Espawn=5Fmain=28=29?= Message-ID: <20160107170320.21239.27945@psf.io> https://hg.python.org/cpython/rev/97ce60e8958c changeset: 99796:97ce60e8958c branch: 3.5 parent: 99793:f08d4712a055 user: Berker Peksag date: Thu Jan 07 19:02:55 2016 +0200 summary: Fix typo in docstring of multiprocessing.spawn.spawn_main() files: Lib/multiprocessing/spawn.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/spawn.py b/Lib/multiprocessing/spawn.py --- a/Lib/multiprocessing/spawn.py +++ b/Lib/multiprocessing/spawn.py @@ -91,7 +91,7 @@ def spawn_main(pipe_handle, parent_pid=None, tracker_fd=None): ''' - Run code specifed by data received over pipe + Run code specified by data received over pipe ''' assert is_forking(sys.argv) if sys.platform == 'win32': -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Jan 7 12:48:40 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 7 Jan 2016 17:48:40 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-07 Message-ID: Results for project Python default, build date 2016-01-07 03:13:02 +0000 commit: 97ab0ccac893dcc5879f1ae3989e7723c2197e21 previous commit: 1118dfcbcc3530a1a8a70b3e3a79df7c5b24275a revision date: 2016-01-06 19:37:52 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% -0.52% 10.17% 15.18% :-| pybench 0.07% 0.01% -1.62% 5.15% :-( regex_v8 2.65% -0.12% -5.17% 5.40% :-| nbody 0.07% -0.05% 1.51% 4.89% :-| json_dump_v2 0.22% 0.42% -1.67% 10.62% :-| normal_startup 1.00% -0.13% -1.80% 5.39% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Jan 7 12:49:20 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 7 Jan 2016 17:49:20 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-07 Message-ID: <1d08384b-5b6a-4fdf-be7f-240d892e146c@irsmsx102.ger.corp.intel.com> Results for project Python 2.7, build date 2016-01-07 04:04:53 +0000 commit: 0792a0ad7f2209e1815a8d89f61271bb93139f81 previous commit: e2faa18802bb6b8b5d0e6eac5ba34072c940fa2f revision date: 2016-01-06 11:54:18 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% 1.09% 4.86% 2.28% :-) pybench 0.17% 0.00% 6.04% 4.56% :-( regex_v8 1.07% 0.04% -2.31% 10.47% :-) nbody 0.10% 0.01% 7.89% 4.49% :-) json_dump_v2 0.18% -0.08% 5.39% 9.97% :-( normal_startup 1.68% 1.19% -4.88% 1.19% :-| ssbench 0.24% -0.17% 1.18% 1.45% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Jan 7 13:59:01 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 07 Jan 2016 18:59:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Add_another_try/except_PermissionError_to_avoid_dependin?= =?utf-8?q?g_on_listdir_order=2E_Fix?= Message-ID: <20160107185858.53718.54443@psf.io> https://hg.python.org/cpython/rev/398cb8c183da changeset: 99800:398cb8c183da parent: 99797:58e8125d7f4e parent: 99799:8a3b0c1fb3d3 user: Guido van Rossum date: Thu Jan 07 10:58:20 2016 -0800 summary: Add another try/except PermissionError to avoid depending on listdir order. Fix issues #24120 and #26012. (Merge 3.5->3.6) files: Lib/pathlib.py | 13 ++++++++----- Lib/test/test_pathlib.py | 16 ++++++++-------- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -539,11 +539,14 @@ def _iterate_directories(self, parent_path, is_dir, listdir): yield parent_path - for name in listdir(parent_path): - path = parent_path._make_child_relpath(name) - if is_dir(path) and not path.is_symlink(): - for p in self._iterate_directories(path, is_dir, listdir): - yield p + try: + for name in listdir(parent_path): + path = parent_path._make_child_relpath(name) + if is_dir(path) and not path.is_symlink(): + for p in self._iterate_directories(path, is_dir, listdir): + yield p + except PermissionError: + return def _select_from(self, parent_path, is_dir, exists, listdir): try: diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1236,7 +1236,7 @@ # | |-- dirD # | | `-- fileD # | `-- fileC - # |-- dirE + # |-- dirE # No permissions # |-- fileA # |-- linkA -> fileA # `-- linkB -> dirB @@ -1453,13 +1453,13 @@ p = P(BASE) it = p.rglob("fileA") self.assertIsInstance(it, collections.Iterator) - # XXX cannot test because of symlink loops in the test setup - #_check(it, ["fileA"]) - #_check(p.rglob("fileB"), ["dirB/fileB"]) - #_check(p.rglob("*/fileA"), [""]) - #_check(p.rglob("*/fileB"), ["dirB/fileB"]) - #_check(p.rglob("file*"), ["fileA", "dirB/fileB"]) - # No symlink loops here + _check(it, ["fileA"]) + _check(p.rglob("fileB"), ["dirB/fileB"]) + _check(p.rglob("*/fileA"), []) + _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", + "linkB/fileB", "dirA/linkC/fileB"]) + _check(p.rglob("file*"), ["fileA", "dirB/fileB", + "dirC/fileC", "dirC/dirD/fileD"]) p = P(BASE, "dirC") _check(p.rglob("file*"), ["dirC/fileC", "dirC/dirD/fileD"]) _check(p.rglob("*/*"), ["dirC/dirD/fileD"]) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 13:59:01 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 07 Jan 2016 18:59:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_another_tr?= =?utf-8?q?y/except_PermissionError_to_avoid_depending_on_listdir_order=2E?= =?utf-8?q?_Fix?= Message-ID: <20160107185857.30374.85723@psf.io> https://hg.python.org/cpython/rev/4043e08e6e52 changeset: 99798:4043e08e6e52 branch: 3.4 parent: 99790:94b0e5e9b6e8 user: Guido van Rossum date: Thu Jan 07 10:56:36 2016 -0800 summary: Add another try/except PermissionError to avoid depending on listdir order. Fix issues #24120 and #26012. files: Lib/pathlib.py | 13 ++++++++----- Lib/test/test_pathlib.py | 16 ++++++++-------- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -494,11 +494,14 @@ def _iterate_directories(self, parent_path, is_dir, listdir): yield parent_path - for name in listdir(parent_path): - path = parent_path._make_child_relpath(name) - if is_dir(path) and not path.is_symlink(): - for p in self._iterate_directories(path, is_dir, listdir): - yield p + try: + for name in listdir(parent_path): + path = parent_path._make_child_relpath(name) + if is_dir(path) and not path.is_symlink(): + for p in self._iterate_directories(path, is_dir, listdir): + yield p + except PermissionError: + return def _select_from(self, parent_path, is_dir, exists, listdir): try: diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1240,7 +1240,7 @@ # | |-- dirD # | | `-- fileD # | `-- fileC - # |-- dirE + # |-- dirE # No permissions # |-- fileA # |-- linkA -> fileA # `-- linkB -> dirB @@ -1396,13 +1396,13 @@ p = P(BASE) it = p.rglob("fileA") self.assertIsInstance(it, collections.Iterator) - # XXX cannot test because of symlink loops in the test setup - #_check(it, ["fileA"]) - #_check(p.rglob("fileB"), ["dirB/fileB"]) - #_check(p.rglob("*/fileA"), [""]) - #_check(p.rglob("*/fileB"), ["dirB/fileB"]) - #_check(p.rglob("file*"), ["fileA", "dirB/fileB"]) - # No symlink loops here + _check(it, ["fileA"]) + _check(p.rglob("fileB"), ["dirB/fileB"]) + _check(p.rglob("*/fileA"), []) + _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", + "linkB/fileB", "dirA/linkC/fileB"]) + _check(p.rglob("file*"), ["fileA", "dirB/fileB", + "dirC/fileC", "dirC/dirD/fileD"]) p = P(BASE, "dirC") _check(p.rglob("file*"), ["dirC/fileC", "dirC/dirD/fileD"]) _check(p.rglob("*/*"), ["dirC/dirD/fileD"]) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 13:59:01 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 07 Jan 2016 18:59:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Add_another_try/except_PermissionError_to_avoid_depending_on_l?= =?utf-8?q?istdir_order=2E_Fix?= Message-ID: <20160107185857.7780.43870@psf.io> https://hg.python.org/cpython/rev/8a3b0c1fb3d3 changeset: 99799:8a3b0c1fb3d3 branch: 3.5 parent: 99796:97ce60e8958c parent: 99798:4043e08e6e52 user: Guido van Rossum date: Thu Jan 07 10:57:37 2016 -0800 summary: Add another try/except PermissionError to avoid depending on listdir order. Fix issues #24120 and #26012. (Merge 3.4->3.5) files: Lib/pathlib.py | 13 ++++++++----- Lib/test/test_pathlib.py | 16 ++++++++-------- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -539,11 +539,14 @@ def _iterate_directories(self, parent_path, is_dir, listdir): yield parent_path - for name in listdir(parent_path): - path = parent_path._make_child_relpath(name) - if is_dir(path) and not path.is_symlink(): - for p in self._iterate_directories(path, is_dir, listdir): - yield p + try: + for name in listdir(parent_path): + path = parent_path._make_child_relpath(name) + if is_dir(path) and not path.is_symlink(): + for p in self._iterate_directories(path, is_dir, listdir): + yield p + except PermissionError: + return def _select_from(self, parent_path, is_dir, exists, listdir): try: diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1236,7 +1236,7 @@ # | |-- dirD # | | `-- fileD # | `-- fileC - # |-- dirE + # |-- dirE # No permissions # |-- fileA # |-- linkA -> fileA # `-- linkB -> dirB @@ -1453,13 +1453,13 @@ p = P(BASE) it = p.rglob("fileA") self.assertIsInstance(it, collections.Iterator) - # XXX cannot test because of symlink loops in the test setup - #_check(it, ["fileA"]) - #_check(p.rglob("fileB"), ["dirB/fileB"]) - #_check(p.rglob("*/fileA"), [""]) - #_check(p.rglob("*/fileB"), ["dirB/fileB"]) - #_check(p.rglob("file*"), ["fileA", "dirB/fileB"]) - # No symlink loops here + _check(it, ["fileA"]) + _check(p.rglob("fileB"), ["dirB/fileB"]) + _check(p.rglob("*/fileA"), []) + _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", + "linkB/fileB", "dirA/linkC/fileB"]) + _check(p.rglob("file*"), ["fileA", "dirB/fileB", + "dirC/fileC", "dirC/dirD/fileD"]) p = P(BASE, "dirC") _check(p.rglob("file*"), ["dirC/fileC", "dirC/dirD/fileD"]) _check(p.rglob("*/*"), ["dirC/dirD/fileD"]) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 16:13:50 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 07 Jan 2016 21:13:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Hopeful_fix_for_test=5Frglob=5Fcommon_on_Windows_without?= =?utf-8?b?IHN5bWxpbmtzLiAoTWVyZ2UgMy41LT4zLjYp?= Message-ID: <20160107211349.9618.51522@psf.io> https://hg.python.org/cpython/rev/74adca5e885c changeset: 99803:74adca5e885c parent: 99800:398cb8c183da parent: 99802:9ead69bd94f1 user: Guido van Rossum date: Thu Jan 07 13:13:31 2016 -0800 summary: Hopeful fix for test_rglob_common on Windows without symlinks. (Merge 3.5->3.6) files: Lib/test/test_pathlib.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1456,8 +1456,11 @@ _check(it, ["fileA"]) _check(p.rglob("fileB"), ["dirB/fileB"]) _check(p.rglob("*/fileA"), []) - _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", - "linkB/fileB", "dirA/linkC/fileB"]) + if symlink_skip_reason: + _check(p.rglob("*/fileB"), ["dirB/fileB"]) + else: + _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", + "linkB/fileB", "dirA/linkC/fileB"]) _check(p.rglob("file*"), ["fileA", "dirB/fileB", "dirC/fileC", "dirC/dirD/fileD"]) p = P(BASE, "dirC") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 16:13:52 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 07 Jan 2016 21:13:52 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Hopeful_fix_for_test=5Frglob=5Fcommon_on_Windows_without_symli?= =?utf-8?b?bmtzLiAoTWVyZ2UgMy40LT4zLjUp?= Message-ID: <20160107211349.53698.80343@psf.io> https://hg.python.org/cpython/rev/9ead69bd94f1 changeset: 99802:9ead69bd94f1 branch: 3.5 parent: 99799:8a3b0c1fb3d3 parent: 99801:8e1788329e77 user: Guido van Rossum date: Thu Jan 07 13:13:04 2016 -0800 summary: Hopeful fix for test_rglob_common on Windows without symlinks. (Merge 3.4->3.5) files: Lib/test/test_pathlib.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1456,8 +1456,11 @@ _check(it, ["fileA"]) _check(p.rglob("fileB"), ["dirB/fileB"]) _check(p.rglob("*/fileA"), []) - _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", - "linkB/fileB", "dirA/linkC/fileB"]) + if symlink_skip_reason: + _check(p.rglob("*/fileB"), ["dirB/fileB"]) + else: + _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", + "linkB/fileB", "dirA/linkC/fileB"]) _check(p.rglob("file*"), ["fileA", "dirB/fileB", "dirC/fileC", "dirC/dirD/fileD"]) p = P(BASE, "dirC") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 7 16:13:53 2016 From: python-checkins at python.org (guido.van.rossum) Date: Thu, 07 Jan 2016 21:13:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Hopeful_fix_fo?= =?utf-8?q?r_test=5Frglob=5Fcommon_on_Windows_without_symlinks=2E?= Message-ID: <20160107211348.17327.86758@psf.io> https://hg.python.org/cpython/rev/8e1788329e77 changeset: 99801:8e1788329e77 branch: 3.4 parent: 99798:4043e08e6e52 user: Guido van Rossum date: Thu Jan 07 13:12:34 2016 -0800 summary: Hopeful fix for test_rglob_common on Windows without symlinks. files: Lib/test/test_pathlib.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1399,8 +1399,11 @@ _check(it, ["fileA"]) _check(p.rglob("fileB"), ["dirB/fileB"]) _check(p.rglob("*/fileA"), []) - _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", - "linkB/fileB", "dirA/linkC/fileB"]) + if symlink_skip_reason: + _check(p.rglob("*/fileB"), ["dirB/fileB"]) + else: + _check(p.rglob("*/fileB"), ["dirB/fileB", "dirB/linkD/fileB", + "linkB/fileB", "dirA/linkC/fileB"]) _check(p.rglob("file*"), ["fileA", "dirB/fileB", "dirC/fileC", "dirC/dirD/fileD"]) p = P(BASE, "dirC") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 00:39:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 08 Jan 2016 05:39:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_3=2E5_=28closes_=2325672=29?= Message-ID: <20160108053857.21402.81223@psf.io> https://hg.python.org/cpython/rev/b5b0394ed20b changeset: 99805:b5b0394ed20b parent: 99803:74adca5e885c parent: 99804:efc8627dcf28 user: Benjamin Peterson date: Thu Jan 07 21:38:51 2016 -0800 summary: merge 3.5 (closes #25672) files: Misc/NEWS | 3 +++ Modules/_ssl.c | 17 +++++++++++++++++ 2 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #25672: In the ssl module, enable the SSL_MODE_RELEASE_BUFFERS mode + option if it is safe to do so. + - Issue #22570: Add 'path' attribute to pathlib.Path objects, returning the same as str(), to make it more similar to DirEntry. Library code can now write getattr(p, 'path', p) to get the path as diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2219,6 +2219,7 @@ PySSLContext *self; long options; SSL_CTX *ctx = NULL; + unsigned long libver; PySSL_BEGIN_ALLOW_THREADS if (proto_version == PY_SSL_VERSION_TLS1) @@ -2281,6 +2282,22 @@ options |= SSL_OP_NO_SSLv3; SSL_CTX_set_options(self->ctx, options); +#if defined(SSL_MODE_RELEASE_BUFFERS) + /* Set SSL_MODE_RELEASE_BUFFERS. This potentially greatly reduces memory + usage for no cost at all. However, don't do this for OpenSSL versions + between 1.0.1 and 1.0.1h or 1.0.0 and 1.0.0m, which are affected by CVE + 2014-0198. I can't find exactly which beta fixed this CVE, so be + conservative and assume it wasn't fixed until release. We do this check + at runtime to avoid problems from the dynamic linker. + See #25672 for more on this. */ + libver = SSLeay(); + if (!(libver >= 0x10001000UL && libver < 0x1000108fUL) && + !(libver >= 0x10000000UL && libver < 0x100000dfUL)) { + SSL_CTX_set_mode(self->ctx, SSL_MODE_RELEASE_BUFFERS); + } +#endif + + #ifndef OPENSSL_NO_ECDH /* Allow automatic ECDH curve selection (on OpenSSL 1.0.2+), or use prime256v1 by default. This is Apache mod_ssl's initialization -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 01:02:24 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 08 Jan 2016 06:02:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_typo_in_un?= =?utf-8?q?ittest_docs_=28closes_=2326046=29?= Message-ID: <20160108060224.21386.78741@psf.io> https://hg.python.org/cpython/rev/ff3655d6c257 changeset: 99806:ff3655d6c257 branch: 2.7 parent: 99795:8b20a606ecde user: Benjamin Peterson date: Thu Jan 07 22:01:26 2016 -0800 summary: fix typo in unittest docs (closes #26046) Patch from Upendra Kumar. files: Doc/library/unittest.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -147,7 +147,7 @@ The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you to define instructions that will be executed before and after each test method. -They are covered in more details in the section :ref:`organizing-tests`. +They are covered in more detail in the section :ref:`organizing-tests`. The final block shows a simple way to run the tests. :func:`unittest.main` provides a command-line interface to the test script. When run from the command -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 01:02:24 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 08 Jan 2016 06:02:24 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2326046=29?= Message-ID: <20160108060224.3377.18145@psf.io> https://hg.python.org/cpython/rev/21567b4493c6 changeset: 99808:21567b4493c6 branch: 3.5 parent: 99804:efc8627dcf28 parent: 99807:f24d22b44099 user: Benjamin Peterson date: Thu Jan 07 22:02:01 2016 -0800 summary: merge 3.4 (#26046) files: Doc/library/unittest.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -118,7 +118,7 @@ The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you to define instructions that will be executed before and after each test method. -They are covered in more details in the section :ref:`organizing-tests`. +They are covered in more detail in the section :ref:`organizing-tests`. The final block shows a simple way to run the tests. :func:`unittest.main` provides a command-line interface to the test script. When run from the command -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 01:02:24 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 08 Jan 2016 06:02:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_typo_in_un?= =?utf-8?q?ittest_docs_=28closes_=2326046=29?= Message-ID: <20160108060224.30376.62810@psf.io> https://hg.python.org/cpython/rev/f24d22b44099 changeset: 99807:f24d22b44099 branch: 3.4 parent: 99801:8e1788329e77 user: Benjamin Peterson date: Thu Jan 07 22:01:26 2016 -0800 summary: fix typo in unittest docs (closes #26046) Patch from Upendra Kumar. files: Doc/library/unittest.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -118,7 +118,7 @@ The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you to define instructions that will be executed before and after each test method. -They are covered in more details in the section :ref:`organizing-tests`. +They are covered in more detail in the section :ref:`organizing-tests`. The final block shows a simple way to run the tests. :func:`unittest.main` provides a command-line interface to the test script. When run from the command -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 01:02:25 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 08 Jan 2016 06:02:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjYwNDYp?= Message-ID: <20160108060224.69609.29594@psf.io> https://hg.python.org/cpython/rev/eeec6c5b5b79 changeset: 99809:eeec6c5b5b79 parent: 99805:b5b0394ed20b parent: 99808:21567b4493c6 user: Benjamin Peterson date: Thu Jan 07 22:02:13 2016 -0800 summary: merge 3.5 (#26046) files: Doc/library/unittest.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -118,7 +118,7 @@ The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you to define instructions that will be executed before and after each test method. -They are covered in more details in the section :ref:`organizing-tests`. +They are covered in more detail in the section :ref:`organizing-tests`. The final block shows a simple way to run the tests. :func:`unittest.main` provides a command-line interface to the test script. When run from the command -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 01:11:04 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 08 Jan 2016 06:11:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogZW5hYmxlIFNTTF9N?= =?utf-8?q?ODE=5FRELEASE=5FBUFFERS?= Message-ID: <20160108053857.7770.40548@psf.io> https://hg.python.org/cpython/rev/efc8627dcf28 changeset: 99804:efc8627dcf28 branch: 3.5 parent: 99802:9ead69bd94f1 user: Benjamin Peterson date: Thu Jan 07 21:37:37 2016 -0800 summary: enable SSL_MODE_RELEASE_BUFFERS Patch by Cory Benfield. files: Misc/NEWS | 3 +++ Modules/_ssl.c | 17 +++++++++++++++++ 2 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,9 @@ Library ------- +- Issue #25672: In the ssl module, enable the SSL_MODE_RELEASE_BUFFERS mode + option if it is safe to do so. + - Issue #22570: Add 'path' attribute to pathlib.Path objects, returning the same as str(), to make it more similar to DirEntry. Library code can now write getattr(p, 'path', p) to get the path as diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2219,6 +2219,7 @@ PySSLContext *self; long options; SSL_CTX *ctx = NULL; + unsigned long libver; PySSL_BEGIN_ALLOW_THREADS if (proto_version == PY_SSL_VERSION_TLS1) @@ -2281,6 +2282,22 @@ options |= SSL_OP_NO_SSLv3; SSL_CTX_set_options(self->ctx, options); +#if defined(SSL_MODE_RELEASE_BUFFERS) + /* Set SSL_MODE_RELEASE_BUFFERS. This potentially greatly reduces memory + usage for no cost at all. However, don't do this for OpenSSL versions + between 1.0.1 and 1.0.1h or 1.0.0 and 1.0.0m, which are affected by CVE + 2014-0198. I can't find exactly which beta fixed this CVE, so be + conservative and assume it wasn't fixed until release. We do this check + at runtime to avoid problems from the dynamic linker. + See #25672 for more on this. */ + libver = SSLeay(); + if (!(libver >= 0x10001000UL && libver < 0x1000108fUL) && + !(libver >= 0x10000000UL && libver < 0x100000dfUL)) { + SSL_CTX_set_mode(self->ctx, SSL_MODE_RELEASE_BUFFERS); + } +#endif + + #ifndef OPENSSL_NO_ECDH /* Allow automatic ECDH curve selection (on OpenSSL 1.0.2+), or use prime256v1 by default. This is Apache mod_ssl's initialization -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Fri Jan 8 03:42:10 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 08 Jan 2016 08:42:10 +0000 Subject: [Python-checkins] Daily reference leaks (b5b0394ed20b): sum=4 Message-ID: <20160108084207.30370.58834@psf.io> results for b5b0394ed20b on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflognOTOWK', '--timeout', '7200'] From python-checkins at python.org Fri Jan 8 04:03:22 2016 From: python-checkins at python.org (senthil.kumaran) Date: Fri, 08 Jan 2016 09:03:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160108090320.7782.13223@psf.io> https://hg.python.org/cpython/rev/4daffae06f64 changeset: 99811:4daffae06f64 parent: 99809:eeec6c5b5b79 parent: 99810:7fe960f1a101 user: Senthil Kumaran date: Fri Jan 08 01:03:03 2016 -0800 summary: merge from 3.5 Issue20969 - Set the Epub Author and Epub Publisher in Python docs. files: Doc/conf.py | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/conf.py b/Doc/conf.py --- a/Doc/conf.py +++ b/Doc/conf.py @@ -138,6 +138,11 @@ # Get LaTeX to handle Unicode correctly latex_elements = {'inputenc': r'\usepackage[utf8x]{inputenc}', 'utf8extra': ''} +# Options for Epub output +# ----------------------- + +epub_author = 'Python Documentation Authors' +epub_publisher = 'Python Software Foundation' # Options for the coverage checker # -------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 04:03:22 2016 From: python-checkins at python.org (senthil.kumaran) Date: Fri, 08 Jan 2016 09:03:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Issue20969_-_S?= =?utf-8?q?et_the_Epub_Author_and_Epub_Publisher_in_Python_docs=2E?= Message-ID: <20160108090319.30368.25451@psf.io> https://hg.python.org/cpython/rev/7fe960f1a101 changeset: 99810:7fe960f1a101 branch: 3.5 parent: 99808:21567b4493c6 user: Senthil Kumaran date: Fri Jan 08 01:01:56 2016 -0800 summary: Issue20969 - Set the Epub Author and Epub Publisher in Python docs. files: Doc/conf.py | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/conf.py b/Doc/conf.py --- a/Doc/conf.py +++ b/Doc/conf.py @@ -138,6 +138,11 @@ # Get LaTeX to handle Unicode correctly latex_elements = {'inputenc': r'\usepackage[utf8x]{inputenc}', 'utf8extra': ''} +# Options for Epub output +# ----------------------- + +epub_author = 'Python Documentation Authors' +epub_publisher = 'Python Software Foundation' # Options for the coverage checker # -------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 04:06:03 2016 From: python-checkins at python.org (senthil.kumaran) Date: Fri, 08 Jan 2016 09:06:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue20969_-_S?= =?utf-8?q?et_the_Epub_Author_and_Epub_Publisher_in_Python_docs=2E?= Message-ID: <20160108090602.3387.74823@psf.io> https://hg.python.org/cpython/rev/6b72b4b79aef changeset: 99812:6b72b4b79aef branch: 2.7 parent: 99806:ff3655d6c257 user: Senthil Kumaran date: Fri Jan 08 01:05:57 2016 -0800 summary: Issue20969 - Set the Epub Author and Epub Publisher in Python docs. files: Doc/conf.py | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/conf.py b/Doc/conf.py --- a/Doc/conf.py +++ b/Doc/conf.py @@ -135,6 +135,14 @@ # Get LaTeX to handle Unicode correctly latex_elements = {'inputenc': r'\usepackage[utf8x]{inputenc}', 'utf8extra': ''} + +# Options for Epub output +# ----------------------- + +epub_author = 'Python Documentation Authors' +epub_publisher = 'Python Software Foundation' + + # Options for the coverage checker # -------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 8 07:29:08 2016 From: python-checkins at python.org (ezio.melotti) Date: Fri, 08 Jan 2016 12:29:08 +0000 Subject: [Python-checkins] =?utf-8?q?test=3A_Fix_incomplete_word=2E?= Message-ID: <20160108122908.8287.63470@psf.io> https://hg.python.org/test/rev/e78f7159606d changeset: 229:e78f7159606d user: Ezio Melotti date: Fri Jan 08 14:29:00 2016 +0200 summary: Fix incomplete word. files: a | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/a b/a --- a/a +++ b/a @@ -5,7 +5,7 @@ a3 zou sdf -qwert +qwerty eidfjzeijd snthdiueoa (obviously it should have gone this direction the first time) spam eggs -- Repository URL: https://hg.python.org/test From lp_benchmark_robot at intel.com Fri Jan 8 08:58:58 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 8 Jan 2016 13:58:58 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-08 Message-ID: Results for project Python default, build date 2016-01-08 03:08:23 +0000 commit: 74adca5e885c32b6c61d44c4a1108ab39ee1cf96 previous commit: 97ab0ccac893dcc5879f1ae3989e7723c2197e21 revision date: 2016-01-07 21:13:31 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.27% 1.02% 11.09% 15.73% :-| pybench 0.05% 0.01% -1.60% 6.19% :-( regex_v8 2.65% 0.20% -4.97% 5.81% :-| nbody 0.07% 0.02% 1.53% 7.56% :-( json_dump_v2 0.22% -0.44% -2.11% 10.49% :-| normal_startup 0.68% 0.28% -1.56% 5.10% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Jan 8 08:59:27 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 8 Jan 2016 13:59:27 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-08 Message-ID: Results for project Python 2.7, build date 2016-01-08 04:00:07 +0000 commit: 8b20a606ecde95d334ea3658ce2ec3169a0d27f5 previous commit: 0792a0ad7f2209e1815a8d89f61271bb93139f81 revision date: 2016-01-07 16:49:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.37% -0.64% 4.25% 2.33% :-) pybench 0.11% -0.08% 5.96% 5.39% :-( regex_v8 1.09% 0.06% -2.25% 10.44% :-) nbody 0.56% -0.31% 7.61% 2.42% :-) json_dump_v2 0.20% 0.26% 5.64% 8.19% :-( normal_startup 1.84% -1.18% -6.12% 2.64% :-| ssbench 0.30% -0.40% 0.78% 1.23% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Jan 8 17:33:14 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 08 Jan 2016 22:33:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325802=3A_Add_an_e?= =?utf-8?q?xamples_section_to_importlib=2E?= Message-ID: <20160108223314.21382.47931@psf.io> https://hg.python.org/cpython/rev/d733b6275e17 changeset: 99813:d733b6275e17 parent: 99811:4daffae06f64 user: Brett Cannon date: Fri Jan 08 14:33:09 2016 -0800 summary: Issue #25802: Add an examples section to importlib. Thanks to Berker Peksag for the patch review. files: Doc/library/imp.rst | 15 ++- Doc/library/importlib.rst | 91 ++++++++++++++++++++++++++- 2 files changed, 99 insertions(+), 7 deletions(-) diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst --- a/Doc/library/imp.rst +++ b/Doc/library/imp.rst @@ -81,7 +81,9 @@ .. deprecated:: 3.3 Use :func:`importlib.util.find_spec` instead unless Python 3.3 compatibility is required, in which case use - :func:`importlib.find_loader`. + :func:`importlib.find_loader`. For example usage of the former case, + see the :ref:`importlib-examples` section of the :mod:`importlib` + documentation. .. function:: load_module(name, file, pathname, description) @@ -108,9 +110,12 @@ If previously used in conjunction with :func:`imp.find_module` then consider using :func:`importlib.import_module`, otherwise use the loader returned by the replacement you chose for :func:`imp.find_module`. If you - called :func:`imp.load_module` and related functions directly then use the - classes in :mod:`importlib.machinery`, e.g. - ``importlib.machinery.SourceFileLoader(name, path).load_module()``. + called :func:`imp.load_module` and related functions directly with file + path arguments then use a combination of + :func:`importlib.util.spec_from_file_location` and + :func:`importlib.util.module_from_spec`. See the :ref:`importlib-examples` + section of the :mod:`importlib` documentation for details of the various + approaches. .. function:: new_module(name) @@ -119,7 +124,7 @@ in ``sys.modules``. .. deprecated:: 3.4 - Use :class:`types.ModuleType` instead. + Use :func:`importlib.util.module_from_spec` instead. .. function:: reload(module) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -256,7 +256,7 @@ module and *path* will be the value of :attr:`__path__` from the parent package. If a spec cannot be found, ``None`` is returned. When passed in, ``target`` is a module object that the finder may - use to make a more educated about what spec to return. + use to make a more educated guess about what spec to return. .. versionadded:: 3.4 @@ -306,7 +306,7 @@ within the :term:`path entry` to which it is assigned. If a spec cannot be found, ``None`` is returned. When passed in, ``target`` is a module object that the finder may use to make a more educated - about what spec to return. + guess about what spec to return. .. versionadded:: 3.4 @@ -1307,3 +1307,90 @@ loader = importlib.machinery.SourceFileLoader lazy_loader = importlib.util.LazyLoader.factory(loader) finder = importlib.machinery.FileFinder(path, [(lazy_loader, suffixes)]) + +.. _importlib-examples: + +Examples +-------- + +To programmatically import a module, use :func:`importlib.import_module`. +:: + + import importlib + + itertools = importlib.import_module('itertools') + +If you need to find out if a module can be imported without actually doing the +import, then you should use :func:`importlib.util.find_spec`. +:: + + import importlib.util + import sys + + # For illustrative purposes. + name = 'itertools' + + spec = importlib.util.find_spec(name) + if spec is None: + print("can't find the itertools module") + else: + # If you chose to perform the actual import. + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + # Adding the module to sys.modules is optional. + sys.modules[name] = module + +To import a Python source file directly, use the following recipe +(Python 3.4 and newer only):: + + import importlib.util + import sys + + # For illustrative purposes. + import tokenize + file_path = tokenize.__file__ + module_name = tokenize.__name__ + + spec = importlib.util.spec_from_file_location(module_name, file_path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + # Optional; only necessary if you want to be able to import the module + # by name later. + sys.modules[module_name] = module + +Import itself is implemented in Python code, making it possible to +expose most of the import machinery through importlib. The following +helps illustrate the various APIs that importlib exposes by providing an +approximate implementation of +:func:`importlib.import_module` (Python 3.4 and newer for importlib usage, +Python 3.6 and newer for other parts of the code). +:: + + import importlib.util + import sys + + def import_module(name, package=None): + """An approximate implementation of import.""" + absolute_name = importlib.util.resolve_name(name, package) + try: + return sys.modules[absolute_name] + except KeyError: + pass + + path = None + if '.' in absolute_name: + parent_name, _, child_name = absolute_name.rpartition('.') + parent_module = import_module(parent_name) + path = parent_module.spec.submodule_search_locations + for finder in sys.meta_path: + spec = finder.find_spec(absolute_name, path) + if spec is not None: + break + else: + raise ImportError(f'No module named {absolute_name!r}') + module = spec.loader.create_module(spec) + spec.loader.exec_module(module) + sys.modules[absolute_name] = module + if path is not None: + setattr(parent_module, child_name, module) + return module -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 02:44:18 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 09 Jan 2016 07:44:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160109074418.8283.58746@psf.io> https://hg.python.org/cpython/rev/9b21dfd71561 changeset: 99815:9b21dfd71561 parent: 99813:d733b6275e17 parent: 99814:b67ed559a7d3 user: Senthil Kumaran date: Fri Jan 08 23:44:10 2016 -0800 summary: merge from 3.5 Issue #22138: Fix mock.patch behavior when patching descriptors. Restore original values after patching. Patch contributed by Sean McCully. files: Lib/unittest/mock.py | 5 ++- Lib/unittest/test/testmock/testpatch.py | 26 +++++++++++++ Misc/NEWS | 3 + 3 files changed, 33 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1332,7 +1332,10 @@ setattr(self.target, self.attribute, self.temp_original) else: delattr(self.target, self.attribute) - if not self.create and not hasattr(self.target, self.attribute): + if not self.create and (not hasattr(self.target, self.attribute) or + self.attribute in ('__doc__', '__module__', + '__defaults__', '__annotations__', + '__kwdefaults__')): # needed for proxy objects like django settings setattr(self.target, self.attribute, self.temp_original) diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -1817,5 +1817,31 @@ self.assertEqual(stopped, ["three", "two", "one"]) + def test_special_attrs(self): + def foo(x=0): + """TEST""" + return x + with patch.object(foo, '__defaults__', (1, )): + self.assertEqual(foo(), 1) + self.assertEqual(foo(), 0) + + with patch.object(foo, '__doc__', "FUN"): + self.assertEqual(foo.__doc__, "FUN") + self.assertEqual(foo.__doc__, "TEST") + + with patch.object(foo, '__module__', "testpatch2"): + self.assertEqual(foo.__module__, "testpatch2") + self.assertEqual(foo.__module__, 'unittest.test.testmock.testpatch') + + with patch.object(foo, '__annotations__', dict([('s', 1, )])): + self.assertEqual(foo.__annotations__, dict([('s', 1, )])) + self.assertEqual(foo.__annotations__, dict()) + + def foo(*a, x=0): + return x + with patch.object(foo, '__kwdefaults__', dict([('x', 1, )])): + self.assertEqual(foo(), 1) + self.assertEqual(foo(), 0) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #22138: Fix mock.patch behavior when patching descriptors. Restore + original values after patching. Patch contributed by Sean McCully. + - Issue #25672: In the ssl module, enable the SSL_MODE_RELEASE_BUFFERS mode option if it is safe to do so. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 02:44:18 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 09 Jan 2016 07:44:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIyMTM4?= =?utf-8?q?=3A_Fix_mock=2Epatch_behavior_when_patching_descriptors=2E_Rest?= =?utf-8?q?ore?= Message-ID: <20160109074417.70875.59868@psf.io> https://hg.python.org/cpython/rev/b67ed559a7d3 changeset: 99814:b67ed559a7d3 branch: 3.5 parent: 99810:7fe960f1a101 user: Senthil Kumaran date: Fri Jan 08 23:43:29 2016 -0800 summary: Issue #22138: Fix mock.patch behavior when patching descriptors. Restore original values after patching. Patch contributed by Sean McCully. files: Lib/unittest/mock.py | 5 ++- Lib/unittest/test/testmock/testpatch.py | 26 +++++++++++++ Misc/NEWS | 3 + 3 files changed, 33 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1332,7 +1332,10 @@ setattr(self.target, self.attribute, self.temp_original) else: delattr(self.target, self.attribute) - if not self.create and not hasattr(self.target, self.attribute): + if not self.create and (not hasattr(self.target, self.attribute) or + self.attribute in ('__doc__', '__module__', + '__defaults__', '__annotations__', + '__kwdefaults__')): # needed for proxy objects like django settings setattr(self.target, self.attribute, self.temp_original) diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -1817,5 +1817,31 @@ self.assertEqual(stopped, ["three", "two", "one"]) + def test_special_attrs(self): + def foo(x=0): + """TEST""" + return x + with patch.object(foo, '__defaults__', (1, )): + self.assertEqual(foo(), 1) + self.assertEqual(foo(), 0) + + with patch.object(foo, '__doc__', "FUN"): + self.assertEqual(foo.__doc__, "FUN") + self.assertEqual(foo.__doc__, "TEST") + + with patch.object(foo, '__module__', "testpatch2"): + self.assertEqual(foo.__module__, "testpatch2") + self.assertEqual(foo.__module__, 'unittest.test.testmock.testpatch') + + with patch.object(foo, '__annotations__', dict([('s', 1, )])): + self.assertEqual(foo.__annotations__, dict([('s', 1, )])) + self.assertEqual(foo.__annotations__, dict()) + + def foo(*a, x=0): + return x + with patch.object(foo, '__kwdefaults__', dict([('x', 1, )])): + self.assertEqual(foo(), 1) + self.assertEqual(foo(), 0) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,9 @@ Library ------- +- Issue #22138: Fix mock.patch behavior when patching descriptors. Restore + original values after patching. Patch contributed by Sean McCully. + - Issue #25672: In the ssl module, enable the SSL_MODE_RELEASE_BUFFERS mode option if it is safe to do so. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 03:28:16 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 09 Jan 2016 08:28:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDI5?= =?utf-8?q?=3A_Remove_extraneous_word=2E__Patch_by_Upendra_Kumar=2E?= Message-ID: <20160109082816.3371.92666@psf.io> https://hg.python.org/cpython/rev/3b860ce5bb51 changeset: 99816:3b860ce5bb51 branch: 3.5 parent: 99814:b67ed559a7d3 user: Terry Jan Reedy date: Sat Jan 09 03:27:37 2016 -0500 summary: Issue #26029: Remove extraneous word. Patch by Upendra Kumar. files: Doc/extending/building.rst | 2 +- Misc/ACKS | 1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/extending/building.rst b/Doc/extending/building.rst --- a/Doc/extending/building.rst +++ b/Doc/extending/building.rst @@ -81,7 +81,7 @@ function. This takes a variable number of keyword arguments, of which the example above uses only a subset. Specifically, the example specifies meta-information to build packages, and it specifies the contents of the -package. Normally, a package will contain of addition modules, like Python +package. Normally, a package will contain addition modules, like Python source modules, documentation, subpackages, etc. Please refer to the distutils documentation in :ref:`distutils-index` to learn more about the features of distutils; this section explains building extension modules only. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -790,6 +790,7 @@ Andrew Kuchling Dave Kuhlman Jon Kuhn +Upendra Kumar Toshio Kuratomi Ilia Kurenkov Vladimir Kushnir -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 03:28:17 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 09 Jan 2016 08:28:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160109082817.69139.10521@psf.io> https://hg.python.org/cpython/rev/60b266d7dcbd changeset: 99817:60b266d7dcbd parent: 99815:9b21dfd71561 parent: 99816:3b860ce5bb51 user: Terry Jan Reedy date: Sat Jan 09 03:28:01 2016 -0500 summary: Merge with 3.5 files: Doc/extending/building.rst | 2 +- Misc/ACKS | 1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/extending/building.rst b/Doc/extending/building.rst --- a/Doc/extending/building.rst +++ b/Doc/extending/building.rst @@ -81,7 +81,7 @@ function. This takes a variable number of keyword arguments, of which the example above uses only a subset. Specifically, the example specifies meta-information to build packages, and it specifies the contents of the -package. Normally, a package will contain of addition modules, like Python +package. Normally, a package will contain addition modules, like Python source modules, documentation, subpackages, etc. Please refer to the distutils documentation in :ref:`distutils-index` to learn more about the features of distutils; this section explains building extension modules only. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -793,6 +793,7 @@ Andrew Kuchling Dave Kuhlman Jon Kuhn +Upendra Kumar Toshio Kuratomi Ilia Kurenkov Vladimir Kushnir -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sat Jan 9 03:41:43 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 09 Jan 2016 08:41:43 +0000 Subject: [Python-checkins] Daily reference leaks (d733b6275e17): sum=6 Message-ID: <20160109084143.69139.92757@psf.io> results for d733b6275e17 on branch "default" -------------------------------------------- test_asyncio leaked [0, -34, 34] references, sum=0 test_asyncio leaked [0, -11, 13] memory blocks, sum=2 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloghaPzNX', '--timeout', '7200'] From python-checkins at python.org Sat Jan 9 09:09:36 2016 From: python-checkins at python.org (ezio.melotti) Date: Sat, 09 Jan 2016 14:09:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI0Nzg5OiBmaXgg?= =?utf-8?q?docstring_of_ctypes=2Ecreate=5Fstring=5Fbuffer=2E__Patch_by_Mat?= =?utf-8?q?heus_Vieira?= Message-ID: <20160109140935.70885.5113@psf.io> https://hg.python.org/cpython/rev/1753a28acad4 changeset: 99818:1753a28acad4 branch: 3.5 parent: 99816:3b860ce5bb51 user: Ezio Melotti date: Sat Jan 09 16:08:24 2016 +0200 summary: #24789: fix docstring of ctypes.create_string_buffer. Patch by Matheus Vieira Portela. files: Lib/ctypes/__init__.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -47,7 +47,7 @@ def create_string_buffer(init, size=None): """create_string_buffer(aBytes) -> character array create_string_buffer(anInteger) -> character array - create_string_buffer(aString, anInteger) -> character array + create_string_buffer(aBytes, anInteger) -> character array """ if isinstance(init, bytes): if size is None: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 09:09:36 2016 From: python-checkins at python.org (ezio.melotti) Date: Sat, 09 Jan 2016 14:09:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIzI0Nzg5OiBtZXJnZSB3aXRoIDMuNS4=?= Message-ID: <20160109140935.47393.82102@psf.io> https://hg.python.org/cpython/rev/57964ca3ce0f changeset: 99819:57964ca3ce0f parent: 99817:60b266d7dcbd parent: 99818:1753a28acad4 user: Ezio Melotti date: Sat Jan 09 16:09:19 2016 +0200 summary: #24789: merge with 3.5. files: Lib/ctypes/__init__.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -47,7 +47,7 @@ def create_string_buffer(init, size=None): """create_string_buffer(aBytes) -> character array create_string_buffer(anInteger) -> character array - create_string_buffer(aString, anInteger) -> character array + create_string_buffer(aBytes, anInteger) -> character array """ if isinstance(init, bytes): if size is None: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 09:35:20 2016 From: python-checkins at python.org (ezio.melotti) Date: Sat, 09 Jan 2016 14:35:20 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_=2326063=3A_update_copyri?= =?utf-8?q?ght=2E__Patch_by_Chris_Gonzalez=2E?= Message-ID: <20160109143520.9636.40213@psf.io> https://hg.python.org/devguide/rev/ac2b3bafd41c changeset: 780:ac2b3bafd41c user: Ezio Melotti date: Sat Jan 09 16:35:11 2016 +0200 summary: #26063: update copyright. Patch by Chris Gonzalez. files: conf.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/conf.py b/conf.py --- a/conf.py +++ b/conf.py @@ -43,7 +43,7 @@ # General information about the project. project = u'Python Developer\'s Guide' -copyright = u'2011-2015, Python Software Foundation' +copyright = u'2011-2016, Python Software Foundation' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Sat Jan 9 12:22:28 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 09 Jan 2016 17:22:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDI5?= =?utf-8?q?=3A_Finish_correction=2E?= Message-ID: <20160109172228.9622.31375@psf.io> https://hg.python.org/cpython/rev/3f9b9244488d changeset: 99820:3f9b9244488d branch: 3.5 parent: 99818:1753a28acad4 user: Terry Jan Reedy date: Sat Jan 09 12:22:00 2016 -0500 summary: Issue #26029: Finish correction. files: Doc/extending/building.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/extending/building.rst b/Doc/extending/building.rst --- a/Doc/extending/building.rst +++ b/Doc/extending/building.rst @@ -81,7 +81,7 @@ function. This takes a variable number of keyword arguments, of which the example above uses only a subset. Specifically, the example specifies meta-information to build packages, and it specifies the contents of the -package. Normally, a package will contain addition modules, like Python +package. Normally, a package will contain additional modules, like Python source modules, documentation, subpackages, etc. Please refer to the distutils documentation in :ref:`distutils-index` to learn more about the features of distutils; this section explains building extension modules only. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 12:22:28 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 09 Jan 2016 17:22:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160109172228.59568.71027@psf.io> https://hg.python.org/cpython/rev/2c8062da7250 changeset: 99821:2c8062da7250 parent: 99819:57964ca3ce0f parent: 99820:3f9b9244488d user: Terry Jan Reedy date: Sat Jan 09 12:22:13 2016 -0500 summary: Merge with 3.5 files: Doc/extending/building.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/extending/building.rst b/Doc/extending/building.rst --- a/Doc/extending/building.rst +++ b/Doc/extending/building.rst @@ -81,7 +81,7 @@ function. This takes a variable number of keyword arguments, of which the example above uses only a subset. Specifically, the example specifies meta-information to build packages, and it specifies the contents of the -package. Normally, a package will contain addition modules, like Python +package. Normally, a package will contain additional modules, like Python source modules, documentation, subpackages, etc. Please refer to the distutils documentation in :ref:`distutils-index` to learn more about the features of distutils; this section explains building extension modules only. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 9 17:28:50 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 09 Jan 2016 22:28:50 +0000 Subject: [Python-checkins] =?utf-8?b?cGVwczogQWRkIFBFUCA1MDkgKGRpY3QuX192?= =?utf-8?q?ersion=5F=5F=29_and_510_=28func=2Especialize=29?= Message-ID: <20160109222850.59582.2609@psf.io> https://hg.python.org/peps/rev/5ad62448c882 changeset: 6153:5ad62448c882 user: Victor Stinner date: Sat Jan 09 23:28:43 2016 +0100 summary: Add PEP 509 (dict.__version__) and 510 (func.specialize) files: pep-0509.txt | 387 +++++++++++++++++++++++++++++++++++++++ pep-0510.txt | 213 +++++++++++++++++++++ 2 files changed, 600 insertions(+), 0 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt new file mode 100644 --- /dev/null +++ b/pep-0509.txt @@ -0,0 +1,387 @@ +PEP: 509 +Title: Add dict.__version__ +Version: $Revision$ +Last-Modified: $Date$ +Author: Victor Stinner +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 4-January-2016 +Python-Version: 3.6 + + +Abstract +======== + +Add a new read-only ``__version__`` property to ``dict`` and +``collections.UserDict`` types, incremented at each change. + + +Rationale +========= + +In Python, the builtin ``dict`` type is used by many instructions. For +example, the ``LOAD_GLOBAL`` instruction searchs for a variable in the +global namespace, or in the builtins namespace (two dict lookups). +Python uses ``dict`` for the builtins namespace, globals namespace, type +namespaces, instance namespaces, etc. The local namespace (namespace of +a function) is usually optimized to an array, but it can be a dict too. + +Python is hard to optimize because almost everything is mutable: builtin +functions, function code, global variables, local variables, ... can be +modified at runtime. Implementing optimizations respecting the Python +semantic requires to detect when "something changes": we will call these +checks "guards". + +The speedup of optimizations depends on the speed of guard checks. This +PEP proposes to add a version to dictionaries to implement efficient +guards on namespaces. + +Example of optimization: replace loading a global variable with a +constant. This optimization requires a guard on the global variable to +check if it was modified. If the variable is modified, the variable must +be loaded at runtime, instead of using the constant. + + +Guard example +============= + +Pseudo-code of an efficient guard to check if a dictionary key was +modified (created, updated or deleted):: + + UNSET = object() + + class Guard: + def __init__(self, dict, key): + self.dict = dict + self.key = key + self.value = dict.get(key, UNSET) + self.version = dict.__version__ + + def check(self): + """Return True if the dictionary value did not changed.""" + version = self.dict.__version__ + if version == self.version: + # Fast-path: avoid the dictionary lookup + return True + + value = self.dict.get(self.key, UNSET) + if value == self.value: + # another key was modified: + # cache the new dictionary version + self.version = version + return True + + return False + + +Changes +======= + +Add a read-only ``__version__`` property to builtin ``dict`` type and to +the ``collections.UserDict`` type. New empty dictionaries are initilized +to version ``0``. The version is incremented at each change: + +* ``clear()`` if the dict was non-empty +* ``pop(key)`` if the key exists +* ``popitem()`` if the dict is non-empty +* ``setdefault(key, value)`` if the `key` does not exist +* ``__detitem__(key)`` if the key exists +* ``__setitem__(key, value)`` if the `key` doesn't exist or if the value + is different +* ``update(...)`` if new values are different than existing values (the + version can be incremented multiple times) + +Example:: + + >>> d = {} + >>> d.__version__ + 0 + >>> d['key'] = 'value' + >>> d.__version__ + 1 + >>> d['key'] = 'new value' + >>> d.__version__ + 2 + >>> del d['key'] + >>> d.__version__ + 3 + +If a dictionary is created with items, the version is also incremented +at each dictionary insertion. Example:: + + >>> d=dict(x=7, y=33) + >>> d.__version__ + 2 + +The version is not incremented is an existing key is modified to the +same value, but only the identifier of the value is tested, not the +content of the value. Example:: + + >>> d={} + >>> value = object() + >>> d['key'] = value + >>> d.__version__ + 2 + >>> d['key'] = value + >>> d.__version__ + 2 + +.. note:: + CPython uses some singleton like integers in the range [-5; 257], + empty tuple, empty strings, Unicode strings of a single character in + the range [U+0000; U+00FF], etc. When a key is set twice to the same + singleton, the version is not modified. + +The PEP is designed to implement guards on namespaces, only the ``dict`` +type can be used for namespaces in practice. ``collections.UserDict`` +is modified because it must mimicks ``dict``. ``collections.Mapping`` is +unchanged. + + +Integer overflow +================ + +The implementation uses the C unsigned integer type ``size_t`` to store +the version. On 32-bit systems, the maximum version is ``2**32-1`` +(more than ``4.2 * 10 ** 9``, 4 billions). On 64-bit systems, the maximum +version is ``2**64-1`` (more than ``1.8 * 10**19``). + +The C code uses ``version++``. On integer overflow, the version is +wrapped to ``0`` (and then continue to be incremented). + +The check ``dict.__version__ == old_version`` can be true after an +integer overflow, so a guard can return false even if the value changed, +which is wrong. The bug occurs if the dict is modified at least ``2**64`` +times (on 64-bit system) between two checks of the guard. + +Using a more complex type (ex: ``PyLongObject``) to avoid the overflow +would slow down operations on the ``dict`` type. Even if there is a +theorical risk of missing a value change, the risk is considered too low +compared to the slow down of using a more complex type. + + +Usage of dict.__version__ +========================= + +Detect dictionary mutation during iteration +------------------------------------------- + +Currently, iterating on a dictionary only detects when the dictionary +size changes, but not when keys or values are modified. Using the +dictionary version, it would be possible to detect when keys and values +are modified. + +See the `issue #19332: Guard against changing dict during iteration +`_. + + +astoptimizer of FAT Python +-------------------------- + +The astoptimizer of the FAT Python project implements many optimizations +which require guards on namespaces. Examples: + +* Call pure builtins: to replace ``len("abc")`` with ``3``, guards on + ``builtins.__dict__['len']`` and ``globals()['len']`` are required +* Loop unrolling: to unroll the loop ``for i in range(...): ...``, + guards on ``builtins.__dict__['range']`` and ``globals()['range']`` + are required + +The `FAT Python +`_ project is a +static optimizer for Python 3.6. + + +Pyjion +------ + +According of Brett Cannon, one of the two main developers of Pyjion, Pyjion can +also benefit from dictionary version to implement optimizations. + +Pyjion is a JIT compiler for Python based upon CoreCLR (Microsoft .NET Core +runtime). + + +Unladen Swallow +--------------- + +Even if dictionary version was not explicitly mentionned, optimization globals +and builtins lookup was part of the Unladen Swallow plan: "Implement one of the +several proposed schemes for speeding lookups of globals and builtins." +Source: `Unladen Swallow ProjectPlan +`_. + +Unladen Swallow is a fork of CPython 2.6.1 adding a JIT compiler implemented +with LLVM. The project stopped in 2011: `Unladen Swallow Retrospective +`_. + + +Implementation +============== + +See the `issue #26058: Add dict.__version__ read-only property +`_. + +On pybench and timeit microbenchmarks, the patch does not seem to add +any overhead on dictionary operations. + + +Alternatives +============ + +Add a version to each dict entry +-------------------------------- + +A single version per dictionary requires to keep a strong reference to +the value which can keep the value alive longer than expected. If we add +also a version per dictionary entry, the guard can rely on the entry +version and so avoid the strong reference to the value (only strong +references to a dictionary and key are needed). + +Changes: add a ``getversion(key)`` method to dictionary which returns +``None`` if the key doesn't exist. When a key is created or modified, +the entry version is set to the dictionary version which is incremented +at each change (create, modify, delete). + +Pseudo-code of an efficient guard to check if a dict key was modified +using ``getversion()``:: + + UNSET = object() + + class Guard: + def __init__(self, dict, key): + self.dict = dict + self.key = key + self.dict_version = dict.__version__ + self.entry_version = dict.getversion(key) + + def check(self): + """Return True if the dictionary value did not changed.""" + dict_version = self.dict.__version__ + if dict_version == self.version: + # Fast-path: avoid the dictionary lookup + return True + + # lookup in the dictionary, but get the entry version, + #not the value + entry_version = self.dict.getversion(self.key) + if entry_version == self.entry_version: + # another key was modified: + # cache the new dictionary version + self.dict_version = dict_version + return True + + return False + +This main drawback of this option is the impact on the memory footprint. +It increases the size of each dictionary entry, so the overhead depends +on the number of buckets (dictionary entries, used or unused yet). For +example, it increases the size of each dictionary entry by 8 bytes on +64-bit system if we use ``size_t``. + +In Python, the memory footprint matters and the trend is more to reduce +it. Examples: + +* `PEP 393 -- Flexible String Representation + `_ +* `PEP 412 -- Key-Sharing Dictionary + `_ + + +Add a new dict subtype +---------------------- + +Add a new ``verdict`` type, subtype of ``dict``. When guards are needed, +use the ``verdict`` for namespaces (module namespace, type namespace, +instance namespace, etc.) instead of ``dict``. + +Leave the ``dict`` type unchanged to not add any overhead (memory +footprint) when guards are not needed. + +Technical issue: a lot of C code in the wild, including CPython core, +expect the exact ``dict`` type. Issues: + +* ``exec()`` requires a ``dict`` for globals and locals. A lot of code + use ``globals={}``. It is not possible to cast the ``dict`` to a + ``dict`` subtype because the caller expects the ``globals`` parameter + to be modified (``dict`` is mutable). +* Functions call directly ``PyDict_xxx()`` functions, instead of calling + ``PyObject_xxx()`` if the object is a ``dict`` subtype +* ``PyDict_CheckExact()`` check fails on ``dict`` subtype, whereas some + functions require the exact ``dict`` type. +* ``Python/ceval.c`` does not completly supports dict subtypes for + namespaces + + +The ``exec()`` issue is a blocker issue. + +Other issues: + +* The garbage collector has a special code to "untrack" ``dict`` + instances. If a ``dict`` subtype is used for namespaces, the garbage + collector may be unable to break some reference cycles. +* Some functions have a fast-path for ``dict`` which would not be taken + for ``dict`` subtypes, and so it would make Python a little bit + slower. + + +Prior Art +========= + +Guard against changing dict during iteration +-------------------------------------------- + +In 2013, Serhiy Storchaka proposed a patch for the `issue #19332: Guard +against changing dict during iteration +`_ (mentioned above) which adds a +``size_t ma_count`` field to the ``PyDictObject`` structure. This field +is incremented when the dictionary is modified, and so is very similar +to the proposed ``dict.__version__``. + + +Cached globals+builtins lookup +------------------------------ + +In 2006, Andrea Griffini proposes a patch implementing a `Cached +globals+builtins lookup optimization `_. +The patch adds a private ``timestamp`` field to dict. + +See the thread on python-dev: `About dictionary lookup caching +`_. + + +Globals / builtins cache +------------------------ + +In 2010, Antoine Pitrou proposed a `Globals / builtins cache +`_ which adds a private +``ma_version`` field to the ``dict`` type. The patch adds a "global and +builtin cache" to functions and frames, and changes ``LOAD_GLOBAL`` and +``STORE_GLOBAL`` instructions to use the cache. + + +PySizer +------- + +`PySizer `_: a memory profiler for Python, +Google Summer of Code 2005 project by Nick Smallbone. + +This project has a patch for CPython 2.4 which adds ``key_time`` and +``value_time`` fields to dictionary entries. It uses a global +process-wide counter for dictionaries, incremented each time that a +dictionary is modified. The times are used to decide when child objects +first appeared in their parent objects. + + +Discussion +========== + +Thread on the python-ideas mailing list: `RFC: PEP: Add dict.__version__ +`_. + + +Copyright +========= + +This document has been placed in the public domain. diff --git a/pep-0510.txt b/pep-0510.txt new file mode 100644 --- /dev/null +++ b/pep-0510.txt @@ -0,0 +1,213 @@ +PEP: 510 +Title: Specialized functions with guards +Version: $Revision$ +Last-Modified: $Date$ +Author: Victor Stinner +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 4-January-2016 +Python-Version: 3.6 + + +Abstract +======== + +Add an API to add specialized functions with guards to functions, to +support static optimizers respecting the Python semantic. + + +Rationale +========= + +Python is hard to optimize because almost everything is mutable: builtin +functions, function code, global variables, local variables, ... can be +modified at runtime. Implement optimizations respecting the Python +semantic requires to detect when "something changes", we will call these +checks "guards". + +This PEP proposes to add a ``specialize()`` method to functions to add a +specialized functions with guards. When the function is called, the +specialized function is used if nothing changed, otherwise use the +original bytecode. + +Writing an optimizer is out of the scope of this PEP. + + +Example +======= + +Using bytecode +-------------- + +Replace ``chr(65)`` with ``"A"``:: + + import myoptimizer + + def func(): + return chr(65) + + def fast_func(): + return "A" + + func.specialize(fast_func.__code__, [myoptimizer.GuardBuiltins("chr")]) + del fast_func + + print("func(): %s" % func()) + print("#specialized: %s" % len(func.get_specialized())) + print() + + import builtins + builtins.chr = lambda obj: "mock" + + print("func(): %s" % func()) + print("#specialized: %s" % len(func.get_specialized())) + +Output:: + + func(): A + #specialized: 1 + + func(): mock + #specialized: 0 + +The hypothetical ``myoptimizer.GuardBuiltins("len")`` is a guard on the +builtin ``len()`` function and the ``len`` name in the global namespace. +The guard fails if the builtin function is replaced or if a ``len`` name +is defined in the global namespace. + +The first call returns directly the string ``"A"``. The second call +removes the specialized function because the builtin ``chr()`` function +was replaced, and executes the original bytecode + +On a microbenchmark, calling the specialized function takes 88 ns, +whereas the original bytecode takes 145 ns (+57 ns): 1.6 times as fast. + + +Using builtin function +---------------------- + +Replace a slow Python function calling ``chr(obj)`` with a direct call +to the builtin ``chr()`` function:: + + import myoptimizer + + def func(arg): + return chr(arg) + + func.specialize(chr, [myoptimizer.GuardBuiltins("chr")]) + + print("func(65): %s" % func(65)) + print("#specialized: %s" % len(func.get_specialized())) + print() + + import builtins + builtins.chr = lambda obj: "mock" + + print("func(65): %s" % func(65)) + print("#specialized: %s" % len(func.get_specialized())) + +Output:: + + func(): A + #specialized: 1 + + func(): mock + #specialized: 0 + +The first call returns directly the builtin ``chr()`` function (without +creating a Python frame). The second call removes the specialized +function because the builtin ``chr()`` function was replaced, and +executes the original bytecode. + +On a microbenchmark, calling the specialized function takes 95 ns, +whereas the original bytecode takes 155 ns (+60 ns): 1.6 times as fast. +Calling directly ``chr(65)`` takes 76 ns. + + +Python Function Call +==================== + +Pseudo-code to call a Python function having specialized functions with +guards:: + + def call_func(func, *args, **kwargs): + # by default, call the regular bytecode + code = func.__code__.co_code + specialized = func.get_specialized() + nspecialized = len(specialized) + + index = 0 + while index < nspecialized: + guard = specialized[index].guard + # pass arguments, some guards need them + check = guard(args, kwargs) + if check == 1: + # guard succeeded: we can use the specialized function + code = specialized[index].code + break + elif check == -1: + # guard will always fail: remove the specialized function + del specialized[index] + elif check == 0: + # guard failed temporarely + index += 1 + + # code can be a code object or any callable object + execute_code(code, args, kwargs) + + +Changes +======= + +* Add two new methods to functions: + + - ``specialize(code, guards: list)``: add specialized + function with guard. `code` is a code object (ex: + ``func2.__code__``) or any callable object (ex: ``len``). + The specialization can be ignored if a guard already fails. + - ``get_specialized()``: get the list of specialized functions with + guards + +* Base ``Guard`` type which can be used as parent type to implement + guards. It requires to implement a ``check()`` function, with an + optional ``first_check()`` function. API: + + * ``int check(PyObject *guard, PyObject **stack)``: return 1 on + success, 0 if the guard failed temporarely, -1 if the guard will + always fail + * ``int first_check(PyObject *guard, PyObject *func)``: return 0 on + success, -1 if the guard will always fail + +Microbenchmark on ``python3.6 -m timeit -s 'def f(): pass' 'f()'`` (best +of 3 runs): + +* Original Python: 79 ns +* Patched Python: 79 ns + +According to this microbenchmark, the changes has no overhead on calling +a Python function without specialization. + + +Behaviour +========= + +When a function code is replaced (``func.__code__ = new_code``), all +specialized functions are removed. + +When a function is serialized (by ``marshal`` or ``pickle`` for +example), specialized functions and guards are ignored (not serialized). + + +Discussion +========== + +Thread on the python-ideas mailing list: `RFC: PEP: Specialized +functions with guards +`_. + + +Copyright +========= + +This document has been placed in the public domain. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sat Jan 9 18:30:37 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 09 Jan 2016 23:30:37 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_add_alternatives?= Message-ID: <20160109233037.27368.23859@psf.io> https://hg.python.org/peps/rev/99fe19f94ee6 changeset: 6154:99fe19f94ee6 user: Victor Stinner date: Sun Jan 10 00:30:32 2016 +0100 summary: PEP 509: add alternatives files: pep-0509.txt | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -230,6 +230,23 @@ Alternatives ============ +Bikeshedding on the property name +--------------------------------- + +* ``__cache_token__``: name proposed by Nick Coghlan, name coming from + `abc.get_cache_token() + `_. +* ``__version__`` +* ``__timestamp__`` + + +Add a version but don't expose it at the Python level +----------------------------------------------------- + +Guards can be implemented in C to access directly the ``ma_version`` +field of the ``PyDictObject`` structure. + + Add a version to each dict entry -------------------------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 10 01:34:51 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 10 Jan 2016 06:34:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160110063451.27378.15394@psf.io> https://hg.python.org/cpython/rev/60d4c3a29201 changeset: 99823:60d4c3a29201 parent: 99821:2c8062da7250 parent: 99822:9ba001cb107b user: Senthil Kumaran date: Sat Jan 09 22:34:40 2016 -0800 summary: merge from 3.5 Issue23675 - A tiny clarification in the MRO glossary term. files: Doc/glossary.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -626,7 +626,8 @@ method resolution order Method Resolution Order is the order in which base classes are searched for a member during lookup. See `The Python 2.3 Method Resolution Order - `_. + `_ for details of the + algorithm used by the Python interpreter since the 2.3 release. module An object that serves as an organizational unit of Python code. Modules -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 01:34:51 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 10 Jan 2016 06:34:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Issue23675_-_A?= =?utf-8?q?_tiny_clarification_in_the_MRO_glossary_term=2E?= Message-ID: <20160110063451.11405.10029@psf.io> https://hg.python.org/cpython/rev/9ba001cb107b changeset: 99822:9ba001cb107b branch: 3.5 parent: 99820:3f9b9244488d user: Senthil Kumaran date: Sat Jan 09 22:33:54 2016 -0800 summary: Issue23675 - A tiny clarification in the MRO glossary term. files: Doc/glossary.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -626,7 +626,8 @@ method resolution order Method Resolution Order is the order in which base classes are searched for a member during lookup. See `The Python 2.3 Method Resolution Order - `_. + `_ for details of the + algorithm used by the Python interpreter since the 2.3 release. module An object that serves as an organizational unit of Python code. Modules -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 01:36:18 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 10 Jan 2016 06:36:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue23675_-_A?= =?utf-8?q?_tiny_clarification_in_the_MRO_glossary_term=2E?= Message-ID: <20160110063618.11411.2542@psf.io> https://hg.python.org/cpython/rev/99f5276e4197 changeset: 99824:99f5276e4197 branch: 2.7 parent: 99812:6b72b4b79aef user: Senthil Kumaran date: Sat Jan 09 22:36:13 2016 -0800 summary: Issue23675 - A tiny clarification in the MRO glossary term. files: Doc/glossary.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -519,7 +519,8 @@ method resolution order Method Resolution Order is the order in which base classes are searched for a member during lookup. See `The Python 2.3 Method Resolution Order - `_. + `_ for details of the + algorithm used by the Python interpreter since the 2.3 release. module An object that serves as an organizational unit of Python code. Modules -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 02:56:50 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 10 Jan 2016 07:56:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_delete_old_cry?= =?utf-8?q?pto_prose_=28closes_=2326066=29?= Message-ID: <20160110075649.70885.60037@psf.io> https://hg.python.org/cpython/rev/d8944e059c7f changeset: 99826:d8944e059c7f branch: 3.4 parent: 99807:f24d22b44099 user: Benjamin Peterson date: Sat Jan 09 23:55:47 2016 -0800 summary: delete old crypto prose (closes #26066) files: Doc/library/crypto.rst | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) diff --git a/Doc/library/crypto.rst b/Doc/library/crypto.rst --- a/Doc/library/crypto.rst +++ b/Doc/library/crypto.rst @@ -16,14 +16,3 @@ hashlib.rst hmac.rst - -.. index:: - pair: AES; algorithm - single: cryptography - single: Kuchling, Andrew - -Hardcore cypherpunks will probably find the cryptographic modules written by -A.M. Kuchling of further interest; the package contains modules for various -encryption algorithms, most notably AES. These modules are not distributed with -Python but available separately. See the URL http://www.pycrypto.org/ for more -information. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 02:56:49 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 10 Jan 2016 07:56:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_delete_old_cry?= =?utf-8?q?pto_prose_=28closes_=2326066=29?= Message-ID: <20160110075649.69151.45481@psf.io> https://hg.python.org/cpython/rev/84fdbdd483a3 changeset: 99825:84fdbdd483a3 branch: 2.7 user: Benjamin Peterson date: Sat Jan 09 23:55:47 2016 -0800 summary: delete old crypto prose (closes #26066) files: Doc/library/crypto.rst | 10 ---------- 1 files changed, 0 insertions(+), 10 deletions(-) diff --git a/Doc/library/crypto.rst b/Doc/library/crypto.rst --- a/Doc/library/crypto.rst +++ b/Doc/library/crypto.rst @@ -18,13 +18,3 @@ hmac.rst md5.rst sha.rst -.. index:: - pair: AES; algorithm - single: cryptography - single: Kuchling, Andrew - -Hardcore cypherpunks will probably find the cryptographic modules written by -A.M. Kuchling of further interest; the package contains modules for various -encryption algorithms, most notably AES. These modules are not distributed with -Python but available separately. See the URL http://www.pycrypto.org/ for more -information. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 02:56:50 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 10 Jan 2016 07:56:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2326066=29?= Message-ID: <20160110075650.59570.49901@psf.io> https://hg.python.org/cpython/rev/cc9ffb4829e7 changeset: 99827:cc9ffb4829e7 branch: 3.5 parent: 99822:9ba001cb107b parent: 99826:d8944e059c7f user: Benjamin Peterson date: Sat Jan 09 23:56:31 2016 -0800 summary: merge 3.4 (#26066) files: Doc/library/crypto.rst | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) diff --git a/Doc/library/crypto.rst b/Doc/library/crypto.rst --- a/Doc/library/crypto.rst +++ b/Doc/library/crypto.rst @@ -16,14 +16,3 @@ hashlib.rst hmac.rst - -.. index:: - pair: AES; algorithm - single: cryptography - single: Kuchling, Andrew - -Hardcore cypherpunks will probably find the cryptographic modules written by -A.M. Kuchling of further interest; the package contains modules for various -encryption algorithms, most notably AES. These modules are not distributed with -Python but available separately. See the URL http://www.pycrypto.org/ for more -information. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 02:56:50 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 10 Jan 2016 07:56:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjYwNjYp?= Message-ID: <20160110075650.997.98233@psf.io> https://hg.python.org/cpython/rev/8ee02ec9b634 changeset: 99828:8ee02ec9b634 parent: 99823:60d4c3a29201 parent: 99827:cc9ffb4829e7 user: Benjamin Peterson date: Sat Jan 09 23:56:40 2016 -0800 summary: merge 3.5 (#26066) files: Doc/library/crypto.rst | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) diff --git a/Doc/library/crypto.rst b/Doc/library/crypto.rst --- a/Doc/library/crypto.rst +++ b/Doc/library/crypto.rst @@ -16,14 +16,3 @@ hashlib.rst hmac.rst - -.. index:: - pair: AES; algorithm - single: cryptography - single: Kuchling, Andrew - -Hardcore cypherpunks will probably find the cryptographic modules written by -A.M. Kuchling of further interest; the package contains modules for various -encryption algorithms, most notably AES. These modules are not distributed with -Python but available separately. See the URL http://www.pycrypto.org/ for more -information. -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sun Jan 10 03:42:51 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 10 Jan 2016 08:42:51 +0000 Subject: [Python-checkins] Daily reference leaks (2c8062da7250): sum=4 Message-ID: <20160110084251.11401.29570@psf.io> results for 2c8062da7250 on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog7tSSgI', '--timeout', '7200'] From python-checkins at python.org Sun Jan 10 09:09:49 2016 From: python-checkins at python.org (victor.stinner) Date: Sun, 10 Jan 2016 14:09:49 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_rationale_for_stat?= =?utf-8?q?ic_optimize_vs_JIT_compiler?= Message-ID: <20160110140949.31069.95004@psf.io> https://hg.python.org/peps/rev/b463c740990c changeset: 6155:b463c740990c user: Victor Stinner date: Sun Jan 10 14:58:17 2016 +0100 summary: PEP 510: rationale for static optimize vs JIT compiler files: pep-0510.txt | 62 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 62 insertions(+), 0 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -20,6 +20,9 @@ Rationale ========= +Python semantic +--------------- + Python is hard to optimize because almost everything is mutable: builtin functions, function code, global variables, local variables, ... can be modified at runtime. Implement optimizations respecting the Python @@ -34,6 +37,65 @@ Writing an optimizer is out of the scope of this PEP. +Why not a JIT compiler? +----------------------- + +There are multiple JIT compilers for Python actively developed: + +* `PyPy `_ +* `Pyston `_ +* `Numba `_ +* `Pyjion `_ + +Numba is specific to numerical computation. Pyston and Pyjion are still +young. PyPy is the most complete Python interpreter, it is much faster +than CPython and has a very good compatibility with CPython (it respects +the Python semantic). There are still issues with Python JIT compilers +which avoid them to be widely used instead of CPython. + +Many popular libraries like numpy, PyGTK, PyQt, PySide and wxPython are +implemented in C or C++ and use the Python C API. To have a small memory +footprint and better performances, Python JIT compilers do not use +reference counting to use a faster garbage collector, do not use C +structures of CPython objects and manage memory allocations differently. +PyPy has a ``cpyext`` module which emulates the Python C API but it has +worse performances than CPython and does not support the full Python C +API. + +New features are first developped in CPython. In january 2016, the +latest CPython stable version is 3.5, whereas PyPy only supports Python +2.7 and 3.2, and Pyston only supports Python 2.7. + +Even if PyPy has a very good compatibility with Python, some modules are +still not compatible with PyPy: see `PyPy Compatibility Wiki +`_. The incomplete +support of the the Python C API is part of this problem. There are also +subtle differences between PyPy and CPython like reference counting: +object destructors are always called in PyPy, but can be called "later" +than in CPython. Using context managers helps to control when resources +are released. + +Even if PyPy is much faster than CPython in a wide range of benchmarks, +some users still report worse performances than CPython on some specific +use cases or unstable performances. + +When Python is used as a scripting program for programs running less +than 1 minute, JIT compilers can be slower because their startup time is +higher and the JIT compiler takes time to optimize the code. For +example, most Mercurial commands take a few seconds. + +Numba now supports ahead of time compilation, but it requires decorator +to specify arguments types and it only supports numerical types. + +CPython 3.5 has almost no optimization: the peephole optimizer only +implements basic optimizations. A static compiler is a compromise +between CPython 3.5 and PyPy. + +.. note:: + There was also the Unladen Swallow project, but it was abandonned in + 2011. + + Example ======= -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 10 18:16:25 2016 From: python-checkins at python.org (victor.stinner) Date: Sun, 10 Jan 2016 23:16:25 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_document_the_type_?= =?utf-8?q?version_tag?= Message-ID: <20160110231624.47397.42764@psf.io> https://hg.python.org/peps/rev/f009d45e8a6a changeset: 6156:f009d45e8a6a user: Victor Stinner date: Mon Jan 11 00:15:41 2016 +0100 summary: PEP 509: document the type version tag files: pep-0509.txt | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -346,6 +346,39 @@ Prior Art ========= +Method cache and type version tag +--------------------------------- + +In 2007, Armin Rigo wrote a patch to to implement a cache of methods. It +was merged into Python 2.6. The patch adds a "type attribute cache +version tag" (``tp_version_tag``) and a "valid version tag" flag to +types (the ``PyTypeObject`` structure). + +The type version tag is not available at the Python level. + +The version tag has the C type ``unsigned int``. The cache is a global +hash table of 4096 entries, shared by all types. The cache is global to +"make it fast, have a deterministic and low memory footprint, and be +easy to invalidate". Each cache entry has a version tag. A global +version tag is used to create the next version tag, it also has the C +type ``unsigned int``. + +By default, a type has its "valid version tag" flag cleared to indicate +that the version tag is invalid. When the first method of the type is +cached, the version tag and the "valid version tag" flag are set. When a +type is modified, the "valid version tag" flag of the type and its +subclasses is cleared. Later, when a cache entry of these types is used, +the entry is removed because its version tag is outdated. + +On integer overflow, the whole cache is cleared and the global version +tag is reset to ``0``. + +See also `issue #1685986: Method cache +`_ and `issue #1700288: Armin's +method cache optimization updated for Python 2.6 +`_. + + Guard against changing dict during iteration -------------------------------------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 10 18:43:51 2016 From: python-checkins at python.org (victor.stinner) Date: Sun, 10 Jan 2016 23:43:51 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509?= Message-ID: <20160110234351.70905.21523@psf.io> https://hg.python.org/peps/rev/a576199a5350 changeset: 6157:a576199a5350 user: Victor Stinner date: Mon Jan 11 00:28:46 2016 +0100 summary: PEP 509 * point to PEP 510 for static optimizers * remove link to issue 19332, dict.__version__ doesn't help files: pep-0509.txt | 17 +++++------------ 1 files changed, 5 insertions(+), 12 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -42,6 +42,11 @@ check if it was modified. If the variable is modified, the variable must be loaded at runtime, instead of using the constant. +See the `PEP 510 -- Specialized functions with guards +`_ for the concrete usage of +guards to specialize functions and for the rationale on Python static +optimizers. + Guard example ============= @@ -164,18 +169,6 @@ Usage of dict.__version__ ========================= -Detect dictionary mutation during iteration -------------------------------------------- - -Currently, iterating on a dictionary only detects when the dictionary -size changes, but not when keys or values are modified. Using the -dictionary version, it would be possible to detect when keys and values -are modified. - -See the `issue #19332: Guard against changing dict during iteration -`_. - - astoptimizer of FAT Python -------------------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 10 21:02:43 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 02:02:43 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_add_missing_parame?= =?utf-8?q?ters_to_check=28=29_to_support_keywords?= Message-ID: <20160111020243.51244.62023@psf.io> https://hg.python.org/peps/rev/3b9eb6edb7c3 changeset: 6158:3b9eb6edb7c3 user: Victor Stinner date: Mon Jan 11 03:02:32 2016 +0100 summary: PEP 510: add missing parameters to check() to support keywords files: pep-0510.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -235,11 +235,11 @@ guards. It requires to implement a ``check()`` function, with an optional ``first_check()`` function. API: - * ``int check(PyObject *guard, PyObject **stack)``: return 1 on - success, 0 if the guard failed temporarely, -1 if the guard will - always fail * ``int first_check(PyObject *guard, PyObject *func)``: return 0 on success, -1 if the guard will always fail + * ``int check(PyObject *guard, PyObject **stack, int na, int nk)``: + return 1 on success, 0 if the guard failed temporarely, -1 if the + guard will always fail Microbenchmark on ``python3.6 -m timeit -s 'def f(): pass' 'f()'`` (best of 3 runs): -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 10 22:54:10 2016 From: python-checkins at python.org (mark.hammond) Date: Mon, 11 Jan 2016 03:54:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326070=3A_py=2Eexe?= =?utf-8?q?_launcher_fails_to_find_in-place_built_binaries_from?= Message-ID: <20160111035409.70893.90574@psf.io> https://hg.python.org/cpython/rev/bfb4a09149b1 changeset: 99830:bfb4a09149b1 parent: 99828:8ee02ec9b634 user: Mark Hammond date: Mon Jan 11 14:53:01 2016 +1100 summary: Issue #26070: py.exe launcher fails to find in-place built binaries from earlier Python versions. files: PC/launcher.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -171,6 +171,9 @@ L"\\", L"\\PCBuild\\win32\\", L"\\PCBuild\\amd64\\", + // To support early 32bit versions of Python that stuck the build binaries + // directly in PCBuild... + L"\\PCBuild\\", NULL }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 10 22:54:10 2016 From: python-checkins at python.org (mark.hammond) Date: Mon, 11 Jan 2016 03:54:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDcw?= =?utf-8?q?=3A_py=2Eexe_launcher_fails_to_find_in-place_built_binaries_fro?= =?utf-8?q?m?= Message-ID: <20160111035409.70895.68461@psf.io> https://hg.python.org/cpython/rev/51cc7adac78d changeset: 99829:51cc7adac78d branch: 3.5 parent: 99827:cc9ffb4829e7 user: Mark Hammond date: Mon Jan 11 14:50:22 2016 +1100 summary: Issue #26070: py.exe launcher fails to find in-place built binaries from earlier Python versions. files: PC/launcher.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -171,6 +171,9 @@ L"\\", L"\\PCBuild\\win32\\", L"\\PCBuild\\amd64\\", + // To support early 32bit versions of Python that stuck the build binaries + // directly in PCBuild... + L"\\PCBuild\\", NULL }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 02:13:20 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 07:13:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Sync_with_asyn?= =?utf-8?q?cio_repo?= Message-ID: <20160111071319.4250.21162@psf.io> https://hg.python.org/cpython/rev/f4fe55dd5659 changeset: 99831:f4fe55dd5659 branch: 3.4 parent: 99826:d8944e059c7f user: Andrew Svetlov date: Mon Jan 11 08:42:49 2016 +0200 summary: Sync with asyncio repo files: Lib/asyncio/coroutines.py | 17 +- Lib/asyncio/tasks.py | 51 ++++ Lib/test/test_asyncio/test_tasks.py | 169 ++++++++++++++++ Misc/NEWS | 5 + 4 files changed, 235 insertions(+), 7 deletions(-) diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py --- a/Lib/asyncio/coroutines.py +++ b/Lib/asyncio/coroutines.py @@ -27,8 +27,8 @@ # before you define your coroutines. A downside of using this feature # is that tracebacks show entries for the CoroWrapper.__next__ method # when _DEBUG is true. -_DEBUG = (not sys.flags.ignore_environment - and bool(os.environ.get('PYTHONASYNCIODEBUG'))) +_DEBUG = (not sys.flags.ignore_environment and + bool(os.environ.get('PYTHONASYNCIODEBUG'))) try: @@ -86,7 +86,7 @@ def __init__(self, gen, func=None): assert inspect.isgenerator(gen) or inspect.iscoroutine(gen), gen self.gen = gen - self.func = func # Used to unwrap @coroutine decorator + self.func = func # Used to unwrap @coroutine decorator self._source_traceback = traceback.extract_stack(sys._getframe(1)) self.__name__ = getattr(gen, '__name__', None) self.__qualname__ = getattr(gen, '__qualname__', None) @@ -283,10 +283,13 @@ coro_frame = coro.cr_frame filename = coro_code.co_filename - if (isinstance(coro, CoroWrapper) - and not inspect.isgeneratorfunction(coro.func) - and coro.func is not None): - filename, lineno = events._get_function_source(coro.func) + lineno = 0 + if (isinstance(coro, CoroWrapper) and + not inspect.isgeneratorfunction(coro.func) and + coro.func is not None): + source = events._get_function_source(coro.func) + if source is not None: + filename, lineno = source if coro_frame is None: coro_repr = ('%s done, defined at %s:%s' % (coro_name, filename, lineno)) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -4,6 +4,7 @@ 'FIRST_COMPLETED', 'FIRST_EXCEPTION', 'ALL_COMPLETED', 'wait', 'wait_for', 'as_completed', 'sleep', 'async', 'gather', 'shield', 'ensure_future', 'run_coroutine_threadsafe', + 'timeout', ] import concurrent.futures @@ -732,3 +733,53 @@ loop.call_soon_threadsafe(callback) return future + + +def timeout(timeout, *, loop=None): + """A factory which produce a context manager with timeout. + + Useful in cases when you want to apply timeout logic around block + of code or in cases when asyncio.wait_for is not suitable. + + For example: + + >>> with asyncio.timeout(0.001): + >>> yield from coro() + + + timeout: timeout value in seconds + loop: asyncio compatible event loop + """ + if loop is None: + loop = events.get_event_loop() + return _Timeout(timeout, loop=loop) + + +class _Timeout: + def __init__(self, timeout, *, loop): + self._timeout = timeout + self._loop = loop + self._task = None + self._cancelled = False + self._cancel_handler = None + + def __enter__(self): + self._task = Task.current_task(loop=self._loop) + if self._task is None: + raise RuntimeError('Timeout context manager should be used ' + 'inside a task') + self._cancel_handler = self._loop.call_later( + self._timeout, self._cancel_task) + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + if exc_type is futures.CancelledError and self._cancelled: + self._cancel_handler = None + self._task = None + raise futures.TimeoutError + self._cancel_handler.cancel() + self._cancel_handler = None + self._task = None + + def _cancel_task(self): + self._cancelled = self._task.cancel() diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -6,6 +6,7 @@ import os import re import sys +import time import types import unittest import weakref @@ -2235,5 +2236,173 @@ self.assertEqual(result, 11) +class TimeoutTests(test_utils.TestCase): + def setUp(self): + self.loop = asyncio.new_event_loop() + asyncio.set_event_loop(None) + + def tearDown(self): + self.loop.close() + self.loop = None + + def test_timeout(self): + canceled_raised = [False] + + @asyncio.coroutine + def long_running_task(): + try: + yield from asyncio.sleep(10, loop=self.loop) + except asyncio.CancelledError: + canceled_raised[0] = True + raise + + @asyncio.coroutine + def go(): + with self.assertRaises(asyncio.TimeoutError): + with asyncio.timeout(0.01, loop=self.loop) as t: + yield from long_running_task() + self.assertIs(t._loop, self.loop) + + self.loop.run_until_complete(go()) + self.assertTrue(canceled_raised[0], 'CancelledError was not raised') + + def test_timeout_finish_in_time(self): + @asyncio.coroutine + def long_running_task(): + yield from asyncio.sleep(0.01, loop=self.loop) + return 'done' + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.1, loop=self.loop): + resp = yield from long_running_task() + self.assertEqual(resp, 'done') + + self.loop.run_until_complete(go()) + + def test_timeout_gloabal_loop(self): + asyncio.set_event_loop(self.loop) + + @asyncio.coroutine + def run(): + with asyncio.timeout(0.1) as t: + yield from asyncio.sleep(0.01) + self.assertIs(t._loop, self.loop) + + self.loop.run_until_complete(run()) + + def test_timeout_not_relevant_exception(self): + @asyncio.coroutine + def go(): + yield from asyncio.sleep(0, loop=self.loop) + with self.assertRaises(KeyError): + with asyncio.timeout(0.1, loop=self.loop): + raise KeyError + + self.loop.run_until_complete(go()) + + def test_timeout_canceled_error_is_converted_to_timeout(self): + @asyncio.coroutine + def go(): + yield from asyncio.sleep(0, loop=self.loop) + with self.assertRaises(asyncio.CancelledError): + with asyncio.timeout(0.001, loop=self.loop): + raise asyncio.CancelledError + + self.loop.run_until_complete(go()) + + def test_timeout_blocking_loop(self): + @asyncio.coroutine + def long_running_task(): + time.sleep(0.05) + return 'done' + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.01, loop=self.loop): + result = yield from long_running_task() + self.assertEqual(result, 'done') + + self.loop.run_until_complete(go()) + + def test_for_race_conditions(self): + fut = asyncio.Future(loop=self.loop) + self.loop.call_later(0.1, fut.set_result('done')) + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.2, loop=self.loop): + resp = yield from fut + self.assertEqual(resp, 'done') + + self.loop.run_until_complete(go()) + + def test_timeout_time(self): + @asyncio.coroutine + def go(): + foo_running = None + + start = self.loop.time() + with self.assertRaises(asyncio.TimeoutError): + with asyncio.timeout(0.1, loop=self.loop): + foo_running = True + try: + yield from asyncio.sleep(0.2, loop=self.loop) + finally: + foo_running = False + + dt = self.loop.time() - start + self.assertTrue(0.09 < dt < 0.11, dt) + self.assertFalse(foo_running) + + self.loop.run_until_complete(go()) + + def test_raise_runtimeerror_if_no_task(self): + with self.assertRaises(RuntimeError): + with asyncio.timeout(0.1, loop=self.loop): + pass + + def test_outer_coro_is_not_cancelled(self): + + has_timeout = [False] + + @asyncio.coroutine + def outer(): + try: + with asyncio.timeout(0.001, loop=self.loop): + yield from asyncio.sleep(1, loop=self.loop) + except asyncio.TimeoutError: + has_timeout[0] = True + + @asyncio.coroutine + def go(): + task = asyncio.ensure_future(outer(), loop=self.loop) + yield from task + self.assertTrue(has_timeout[0]) + self.assertFalse(task.cancelled()) + self.assertTrue(task.done()) + + self.loop.run_until_complete(go()) + + def test_cancel_outer_coro(self): + fut = asyncio.Future(loop=self.loop) + + @asyncio.coroutine + def outer(): + fut.set_result(None) + yield from asyncio.sleep(1, loop=self.loop) + + @asyncio.coroutine + def go(): + task = asyncio.ensure_future(outer(), loop=self.loop) + yield from fut + task.cancel() + with self.assertRaises(asyncio.CancelledError): + yield from task + self.assertTrue(task.cancelled()) + self.assertTrue(task.done()) + + self.loop.run_until_complete(go()) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -25,6 +25,11 @@ - Issue #24120: Ignore PermissionError when traversing a tree with pathlib.Path.[r]glob(). Patch by Ulrich Petri. +- Skip getaddrinfo if host is already resolved. + Patch by A. Jesse Jiryu Davis. + +- Add asyncio.timeout() context manager. + What's New in Python 3.4.4? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 02:13:20 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 07:13:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160111071319.31085.16569@psf.io> https://hg.python.org/cpython/rev/b2fac2ff7b01 changeset: 99832:b2fac2ff7b01 branch: 3.5 parent: 99829:51cc7adac78d parent: 99831:f4fe55dd5659 user: Andrew Svetlov date: Mon Jan 11 09:09:10 2016 +0200 summary: merge 3.4 files: Lib/asyncio/coroutines.py | 17 +- Lib/asyncio/tasks.py | 51 ++++ Lib/test/test_asyncio/test_tasks.py | 169 ++++++++++++++++ 3 files changed, 230 insertions(+), 7 deletions(-) diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py --- a/Lib/asyncio/coroutines.py +++ b/Lib/asyncio/coroutines.py @@ -27,8 +27,8 @@ # before you define your coroutines. A downside of using this feature # is that tracebacks show entries for the CoroWrapper.__next__ method # when _DEBUG is true. -_DEBUG = (not sys.flags.ignore_environment - and bool(os.environ.get('PYTHONASYNCIODEBUG'))) +_DEBUG = (not sys.flags.ignore_environment and + bool(os.environ.get('PYTHONASYNCIODEBUG'))) try: @@ -86,7 +86,7 @@ def __init__(self, gen, func=None): assert inspect.isgenerator(gen) or inspect.iscoroutine(gen), gen self.gen = gen - self.func = func # Used to unwrap @coroutine decorator + self.func = func # Used to unwrap @coroutine decorator self._source_traceback = traceback.extract_stack(sys._getframe(1)) self.__name__ = getattr(gen, '__name__', None) self.__qualname__ = getattr(gen, '__qualname__', None) @@ -283,10 +283,13 @@ coro_frame = coro.cr_frame filename = coro_code.co_filename - if (isinstance(coro, CoroWrapper) - and not inspect.isgeneratorfunction(coro.func) - and coro.func is not None): - filename, lineno = events._get_function_source(coro.func) + lineno = 0 + if (isinstance(coro, CoroWrapper) and + not inspect.isgeneratorfunction(coro.func) and + coro.func is not None): + source = events._get_function_source(coro.func) + if source is not None: + filename, lineno = source if coro_frame is None: coro_repr = ('%s done, defined at %s:%s' % (coro_name, filename, lineno)) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -4,6 +4,7 @@ 'FIRST_COMPLETED', 'FIRST_EXCEPTION', 'ALL_COMPLETED', 'wait', 'wait_for', 'as_completed', 'sleep', 'async', 'gather', 'shield', 'ensure_future', 'run_coroutine_threadsafe', + 'timeout', ] import concurrent.futures @@ -732,3 +733,53 @@ loop.call_soon_threadsafe(callback) return future + + +def timeout(timeout, *, loop=None): + """A factory which produce a context manager with timeout. + + Useful in cases when you want to apply timeout logic around block + of code or in cases when asyncio.wait_for is not suitable. + + For example: + + >>> with asyncio.timeout(0.001): + >>> yield from coro() + + + timeout: timeout value in seconds + loop: asyncio compatible event loop + """ + if loop is None: + loop = events.get_event_loop() + return _Timeout(timeout, loop=loop) + + +class _Timeout: + def __init__(self, timeout, *, loop): + self._timeout = timeout + self._loop = loop + self._task = None + self._cancelled = False + self._cancel_handler = None + + def __enter__(self): + self._task = Task.current_task(loop=self._loop) + if self._task is None: + raise RuntimeError('Timeout context manager should be used ' + 'inside a task') + self._cancel_handler = self._loop.call_later( + self._timeout, self._cancel_task) + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + if exc_type is futures.CancelledError and self._cancelled: + self._cancel_handler = None + self._task = None + raise futures.TimeoutError + self._cancel_handler.cancel() + self._cancel_handler = None + self._task = None + + def _cancel_task(self): + self._cancelled = self._task.cancel() diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -6,6 +6,7 @@ import os import re import sys +import time import types import unittest import weakref @@ -2235,5 +2236,173 @@ self.assertEqual(result, 11) +class TimeoutTests(test_utils.TestCase): + def setUp(self): + self.loop = asyncio.new_event_loop() + asyncio.set_event_loop(None) + + def tearDown(self): + self.loop.close() + self.loop = None + + def test_timeout(self): + canceled_raised = [False] + + @asyncio.coroutine + def long_running_task(): + try: + yield from asyncio.sleep(10, loop=self.loop) + except asyncio.CancelledError: + canceled_raised[0] = True + raise + + @asyncio.coroutine + def go(): + with self.assertRaises(asyncio.TimeoutError): + with asyncio.timeout(0.01, loop=self.loop) as t: + yield from long_running_task() + self.assertIs(t._loop, self.loop) + + self.loop.run_until_complete(go()) + self.assertTrue(canceled_raised[0], 'CancelledError was not raised') + + def test_timeout_finish_in_time(self): + @asyncio.coroutine + def long_running_task(): + yield from asyncio.sleep(0.01, loop=self.loop) + return 'done' + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.1, loop=self.loop): + resp = yield from long_running_task() + self.assertEqual(resp, 'done') + + self.loop.run_until_complete(go()) + + def test_timeout_gloabal_loop(self): + asyncio.set_event_loop(self.loop) + + @asyncio.coroutine + def run(): + with asyncio.timeout(0.1) as t: + yield from asyncio.sleep(0.01) + self.assertIs(t._loop, self.loop) + + self.loop.run_until_complete(run()) + + def test_timeout_not_relevant_exception(self): + @asyncio.coroutine + def go(): + yield from asyncio.sleep(0, loop=self.loop) + with self.assertRaises(KeyError): + with asyncio.timeout(0.1, loop=self.loop): + raise KeyError + + self.loop.run_until_complete(go()) + + def test_timeout_canceled_error_is_converted_to_timeout(self): + @asyncio.coroutine + def go(): + yield from asyncio.sleep(0, loop=self.loop) + with self.assertRaises(asyncio.CancelledError): + with asyncio.timeout(0.001, loop=self.loop): + raise asyncio.CancelledError + + self.loop.run_until_complete(go()) + + def test_timeout_blocking_loop(self): + @asyncio.coroutine + def long_running_task(): + time.sleep(0.05) + return 'done' + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.01, loop=self.loop): + result = yield from long_running_task() + self.assertEqual(result, 'done') + + self.loop.run_until_complete(go()) + + def test_for_race_conditions(self): + fut = asyncio.Future(loop=self.loop) + self.loop.call_later(0.1, fut.set_result('done')) + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.2, loop=self.loop): + resp = yield from fut + self.assertEqual(resp, 'done') + + self.loop.run_until_complete(go()) + + def test_timeout_time(self): + @asyncio.coroutine + def go(): + foo_running = None + + start = self.loop.time() + with self.assertRaises(asyncio.TimeoutError): + with asyncio.timeout(0.1, loop=self.loop): + foo_running = True + try: + yield from asyncio.sleep(0.2, loop=self.loop) + finally: + foo_running = False + + dt = self.loop.time() - start + self.assertTrue(0.09 < dt < 0.11, dt) + self.assertFalse(foo_running) + + self.loop.run_until_complete(go()) + + def test_raise_runtimeerror_if_no_task(self): + with self.assertRaises(RuntimeError): + with asyncio.timeout(0.1, loop=self.loop): + pass + + def test_outer_coro_is_not_cancelled(self): + + has_timeout = [False] + + @asyncio.coroutine + def outer(): + try: + with asyncio.timeout(0.001, loop=self.loop): + yield from asyncio.sleep(1, loop=self.loop) + except asyncio.TimeoutError: + has_timeout[0] = True + + @asyncio.coroutine + def go(): + task = asyncio.ensure_future(outer(), loop=self.loop) + yield from task + self.assertTrue(has_timeout[0]) + self.assertFalse(task.cancelled()) + self.assertTrue(task.done()) + + self.loop.run_until_complete(go()) + + def test_cancel_outer_coro(self): + fut = asyncio.Future(loop=self.loop) + + @asyncio.coroutine + def outer(): + fut.set_result(None) + yield from asyncio.sleep(1, loop=self.loop) + + @asyncio.coroutine + def go(): + task = asyncio.ensure_future(outer(), loop=self.loop) + yield from fut + task.cancel() + with self.assertRaises(asyncio.CancelledError): + yield from task + self.assertTrue(task.cancelled()) + self.assertTrue(task.done()) + + self.loop.run_until_complete(go()) + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 02:13:20 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 07:13:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160111071320.70877.14290@psf.io> https://hg.python.org/cpython/rev/abbf2f253720 changeset: 99833:abbf2f253720 parent: 99830:bfb4a09149b1 parent: 99832:b2fac2ff7b01 user: Andrew Svetlov date: Mon Jan 11 09:13:10 2016 +0200 summary: merge 3.5 files: Lib/asyncio/coroutines.py | 17 +- Lib/asyncio/tasks.py | 51 ++++ Lib/test/test_asyncio/test_tasks.py | 169 ++++++++++++++++ Misc/NEWS | 5 + 4 files changed, 235 insertions(+), 7 deletions(-) diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py --- a/Lib/asyncio/coroutines.py +++ b/Lib/asyncio/coroutines.py @@ -27,8 +27,8 @@ # before you define your coroutines. A downside of using this feature # is that tracebacks show entries for the CoroWrapper.__next__ method # when _DEBUG is true. -_DEBUG = (not sys.flags.ignore_environment - and bool(os.environ.get('PYTHONASYNCIODEBUG'))) +_DEBUG = (not sys.flags.ignore_environment and + bool(os.environ.get('PYTHONASYNCIODEBUG'))) try: @@ -86,7 +86,7 @@ def __init__(self, gen, func=None): assert inspect.isgenerator(gen) or inspect.iscoroutine(gen), gen self.gen = gen - self.func = func # Used to unwrap @coroutine decorator + self.func = func # Used to unwrap @coroutine decorator self._source_traceback = traceback.extract_stack(sys._getframe(1)) self.__name__ = getattr(gen, '__name__', None) self.__qualname__ = getattr(gen, '__qualname__', None) @@ -283,10 +283,13 @@ coro_frame = coro.cr_frame filename = coro_code.co_filename - if (isinstance(coro, CoroWrapper) - and not inspect.isgeneratorfunction(coro.func) - and coro.func is not None): - filename, lineno = events._get_function_source(coro.func) + lineno = 0 + if (isinstance(coro, CoroWrapper) and + not inspect.isgeneratorfunction(coro.func) and + coro.func is not None): + source = events._get_function_source(coro.func) + if source is not None: + filename, lineno = source if coro_frame is None: coro_repr = ('%s done, defined at %s:%s' % (coro_name, filename, lineno)) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -4,6 +4,7 @@ 'FIRST_COMPLETED', 'FIRST_EXCEPTION', 'ALL_COMPLETED', 'wait', 'wait_for', 'as_completed', 'sleep', 'async', 'gather', 'shield', 'ensure_future', 'run_coroutine_threadsafe', + 'timeout', ] import concurrent.futures @@ -732,3 +733,53 @@ loop.call_soon_threadsafe(callback) return future + + +def timeout(timeout, *, loop=None): + """A factory which produce a context manager with timeout. + + Useful in cases when you want to apply timeout logic around block + of code or in cases when asyncio.wait_for is not suitable. + + For example: + + >>> with asyncio.timeout(0.001): + >>> yield from coro() + + + timeout: timeout value in seconds + loop: asyncio compatible event loop + """ + if loop is None: + loop = events.get_event_loop() + return _Timeout(timeout, loop=loop) + + +class _Timeout: + def __init__(self, timeout, *, loop): + self._timeout = timeout + self._loop = loop + self._task = None + self._cancelled = False + self._cancel_handler = None + + def __enter__(self): + self._task = Task.current_task(loop=self._loop) + if self._task is None: + raise RuntimeError('Timeout context manager should be used ' + 'inside a task') + self._cancel_handler = self._loop.call_later( + self._timeout, self._cancel_task) + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + if exc_type is futures.CancelledError and self._cancelled: + self._cancel_handler = None + self._task = None + raise futures.TimeoutError + self._cancel_handler.cancel() + self._cancel_handler = None + self._task = None + + def _cancel_task(self): + self._cancelled = self._task.cancel() diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -6,6 +6,7 @@ import os import re import sys +import time import types import unittest import weakref @@ -2235,5 +2236,173 @@ self.assertEqual(result, 11) +class TimeoutTests(test_utils.TestCase): + def setUp(self): + self.loop = asyncio.new_event_loop() + asyncio.set_event_loop(None) + + def tearDown(self): + self.loop.close() + self.loop = None + + def test_timeout(self): + canceled_raised = [False] + + @asyncio.coroutine + def long_running_task(): + try: + yield from asyncio.sleep(10, loop=self.loop) + except asyncio.CancelledError: + canceled_raised[0] = True + raise + + @asyncio.coroutine + def go(): + with self.assertRaises(asyncio.TimeoutError): + with asyncio.timeout(0.01, loop=self.loop) as t: + yield from long_running_task() + self.assertIs(t._loop, self.loop) + + self.loop.run_until_complete(go()) + self.assertTrue(canceled_raised[0], 'CancelledError was not raised') + + def test_timeout_finish_in_time(self): + @asyncio.coroutine + def long_running_task(): + yield from asyncio.sleep(0.01, loop=self.loop) + return 'done' + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.1, loop=self.loop): + resp = yield from long_running_task() + self.assertEqual(resp, 'done') + + self.loop.run_until_complete(go()) + + def test_timeout_gloabal_loop(self): + asyncio.set_event_loop(self.loop) + + @asyncio.coroutine + def run(): + with asyncio.timeout(0.1) as t: + yield from asyncio.sleep(0.01) + self.assertIs(t._loop, self.loop) + + self.loop.run_until_complete(run()) + + def test_timeout_not_relevant_exception(self): + @asyncio.coroutine + def go(): + yield from asyncio.sleep(0, loop=self.loop) + with self.assertRaises(KeyError): + with asyncio.timeout(0.1, loop=self.loop): + raise KeyError + + self.loop.run_until_complete(go()) + + def test_timeout_canceled_error_is_converted_to_timeout(self): + @asyncio.coroutine + def go(): + yield from asyncio.sleep(0, loop=self.loop) + with self.assertRaises(asyncio.CancelledError): + with asyncio.timeout(0.001, loop=self.loop): + raise asyncio.CancelledError + + self.loop.run_until_complete(go()) + + def test_timeout_blocking_loop(self): + @asyncio.coroutine + def long_running_task(): + time.sleep(0.05) + return 'done' + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.01, loop=self.loop): + result = yield from long_running_task() + self.assertEqual(result, 'done') + + self.loop.run_until_complete(go()) + + def test_for_race_conditions(self): + fut = asyncio.Future(loop=self.loop) + self.loop.call_later(0.1, fut.set_result('done')) + + @asyncio.coroutine + def go(): + with asyncio.timeout(0.2, loop=self.loop): + resp = yield from fut + self.assertEqual(resp, 'done') + + self.loop.run_until_complete(go()) + + def test_timeout_time(self): + @asyncio.coroutine + def go(): + foo_running = None + + start = self.loop.time() + with self.assertRaises(asyncio.TimeoutError): + with asyncio.timeout(0.1, loop=self.loop): + foo_running = True + try: + yield from asyncio.sleep(0.2, loop=self.loop) + finally: + foo_running = False + + dt = self.loop.time() - start + self.assertTrue(0.09 < dt < 0.11, dt) + self.assertFalse(foo_running) + + self.loop.run_until_complete(go()) + + def test_raise_runtimeerror_if_no_task(self): + with self.assertRaises(RuntimeError): + with asyncio.timeout(0.1, loop=self.loop): + pass + + def test_outer_coro_is_not_cancelled(self): + + has_timeout = [False] + + @asyncio.coroutine + def outer(): + try: + with asyncio.timeout(0.001, loop=self.loop): + yield from asyncio.sleep(1, loop=self.loop) + except asyncio.TimeoutError: + has_timeout[0] = True + + @asyncio.coroutine + def go(): + task = asyncio.ensure_future(outer(), loop=self.loop) + yield from task + self.assertTrue(has_timeout[0]) + self.assertFalse(task.cancelled()) + self.assertTrue(task.done()) + + self.loop.run_until_complete(go()) + + def test_cancel_outer_coro(self): + fut = asyncio.Future(loop=self.loop) + + @asyncio.coroutine + def outer(): + fut.set_result(None) + yield from asyncio.sleep(1, loop=self.loop) + + @asyncio.coroutine + def go(): + task = asyncio.ensure_future(outer(), loop=self.loop) + yield from fut + task.cancel() + with self.assertRaises(asyncio.CancelledError): + yield from task + self.assertTrue(task.cancelled()) + self.assertTrue(task.done()) + + self.loop.run_until_complete(go()) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -440,6 +440,11 @@ - Issue #25319: When threading.Event is reinitialized, the underlying condition should use a regular lock rather than a recursive lock. +- Skip getaddrinfo if host is already resolved. + Patch by A. Jesse Jiryu Davis. + +- Add asyncio.timeout() context manager. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 02:21:09 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 07:21:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160111072109.47399.58278@psf.io> https://hg.python.org/cpython/rev/69f90308e327 changeset: 99835:69f90308e327 parent: 99833:abbf2f253720 parent: 99834:2266e349692c user: Andrew Svetlov date: Mon Jan 11 09:21:02 2016 +0200 summary: merge 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 02:21:09 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 07:21:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Add_missing_Mi?= =?utf-8?q?sc/NEWS_update?= Message-ID: <20160111072109.70905.75244@psf.io> https://hg.python.org/cpython/rev/2266e349692c changeset: 99834:2266e349692c branch: 3.5 parent: 99832:b2fac2ff7b01 user: Andrew Svetlov date: Mon Jan 11 09:16:58 2016 +0200 summary: Add missing Misc/NEWS update files: Misc/NEWS | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -122,6 +122,11 @@ - Issue #25624: ZipFile now always writes a ZIP_STORED header for directory entries. Patch by Dingyuan Wang. +- Skip getaddrinfo if host is already resolved. + Patch by A. Jesse Jiryu Davis. + +- Add asyncio.timeout() context manager. + Documentation ------------- -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Mon Jan 11 03:41:33 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 11 Jan 2016 08:41:33 +0000 Subject: [Python-checkins] Daily reference leaks (bfb4a09149b1): sum=4 Message-ID: <20160111084133.27356.5481@psf.io> results for bfb4a09149b1 on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogd1cVv0', '--timeout', '7200'] From python-checkins at python.org Mon Jan 11 04:18:59 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 09:18:59 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_typo=3A_semantic_=3D=3E_seman?= =?utf-8?q?tics?= Message-ID: <20160111091859.4252.57701@psf.io> https://hg.python.org/peps/rev/23563c9796a6 changeset: 6159:23563c9796a6 user: Victor Stinner date: Mon Jan 11 10:18:52 2016 +0100 summary: typo: semantic => semantics files: pep-0509.txt | 4 ++-- pep-0510.txt | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -30,8 +30,8 @@ Python is hard to optimize because almost everything is mutable: builtin functions, function code, global variables, local variables, ... can be modified at runtime. Implementing optimizations respecting the Python -semantic requires to detect when "something changes": we will call these -checks "guards". +semantics requires to detect when "something changes": we will call +these checks "guards". The speedup of optimizations depends on the speed of guard checks. This PEP proposes to add a version to dictionaries to implement efficient diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -14,19 +14,19 @@ ======== Add an API to add specialized functions with guards to functions, to -support static optimizers respecting the Python semantic. +support static optimizers respecting the Python semantics. Rationale ========= -Python semantic ---------------- +Python semantics +---------------- Python is hard to optimize because almost everything is mutable: builtin functions, function code, global variables, local variables, ... can be modified at runtime. Implement optimizations respecting the Python -semantic requires to detect when "something changes", we will call these +semantics requires to detect when "something changes", we will call these checks "guards". This PEP proposes to add a ``specialize()`` method to functions to add a @@ -50,7 +50,7 @@ Numba is specific to numerical computation. Pyston and Pyjion are still young. PyPy is the most complete Python interpreter, it is much faster than CPython and has a very good compatibility with CPython (it respects -the Python semantic). There are still issues with Python JIT compilers +the Python semantics). There are still issues with Python JIT compilers which avoid them to be widely used instead of CPython. Many popular libraries like numpy, PyGTK, PyQt, PySide and wxPython are -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 05:26:23 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 10:26:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160111102623.4226.57911@psf.io> https://hg.python.org/cpython/rev/c8f7c1cf380a changeset: 99837:c8f7c1cf380a branch: 3.5 parent: 99834:2266e349692c parent: 99836:c6a608f0bb67 user: Andrew Svetlov date: Mon Jan 11 12:25:40 2016 +0200 summary: merge 3.4 files: Lib/asyncio/tasks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -744,7 +744,7 @@ For example: >>> with asyncio.timeout(0.001): - >>> yield from coro() + ... yield from coro() timeout: timeout value in seconds -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 05:26:25 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 10:26:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160111102623.69141.7750@psf.io> https://hg.python.org/cpython/rev/2a88b422c981 changeset: 99838:2a88b422c981 parent: 99835:69f90308e327 parent: 99837:c8f7c1cf380a user: Andrew Svetlov date: Mon Jan 11 12:26:02 2016 +0200 summary: merge 3.5 files: Lib/asyncio/tasks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -744,7 +744,7 @@ For example: >>> with asyncio.timeout(0.001): - >>> yield from coro() + ... yield from coro() timeout: timeout value in seconds -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 05:26:25 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 10:26:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Sync_with_asyn?= =?utf-8?q?cio?= Message-ID: <20160111102623.51240.97433@psf.io> https://hg.python.org/cpython/rev/c6a608f0bb67 changeset: 99836:c6a608f0bb67 branch: 3.4 parent: 99831:f4fe55dd5659 user: Andrew Svetlov date: Mon Jan 11 12:25:23 2016 +0200 summary: Sync with asyncio files: Lib/asyncio/tasks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -744,7 +744,7 @@ For example: >>> with asyncio.timeout(0.001): - >>> yield from coro() + ... yield from coro() timeout: timeout value in seconds -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 07:45:59 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 12:45:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Document_async?= =?utf-8?b?aW8udGltZW91dCgp?= Message-ID: <20160111124558.11383.66137@psf.io> https://hg.python.org/cpython/rev/5c13957152d6 changeset: 99839:5c13957152d6 branch: 3.4 parent: 99836:c6a608f0bb67 user: Andrew Svetlov date: Mon Jan 11 14:40:35 2016 +0200 summary: Document asyncio.timeout() files: Doc/library/asyncio-task.rst | 109 +++++++++++++--------- 1 files changed, 63 insertions(+), 46 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -363,9 +363,10 @@ running in different threads. While a task waits for the completion of a future, the event loop executes a new task. - The cancellation of a task is different from the cancelation of a future. Calling - :meth:`cancel` will throw a :exc:`~concurrent.futures.CancelledError` to the - wrapped coroutine. :meth:`~Future.cancelled` only returns ``True`` if the + The cancellation of a task is different from the cancelation of a + future. Calling :meth:`cancel` will throw a + :exc:`~concurrent.futures.CancelledError` to the wrapped + coroutine. :meth:`~Future.cancelled` only returns ``True`` if the wrapped coroutine did not catch the :exc:`~concurrent.futures.CancelledError` exception, or raised a :exc:`~concurrent.futures.CancelledError` exception. @@ -417,10 +418,11 @@ Return the list of stack frames for this task's coroutine. - If the coroutine is not done, this returns the stack where it is suspended. - If the coroutine has completed successfully or was cancelled, this - returns an empty list. If the coroutine was terminated by an exception, - this returns the list of traceback frames. + If the coroutine is not done, this returns the stack where it is + suspended. If the coroutine has completed successfully or was + cancelled, this returns an empty list. If the coroutine was + terminated by an exception, this returns the list of traceback + frames. The frames are always ordered from oldest to newest. @@ -557,6 +559,45 @@ Return ``True`` if *func* is a decorated :ref:`coroutine function `. +.. function:: run_coroutine_threadsafe(coro, loop) + + Submit a :ref:`coroutine object ` to a given event loop. + + Return a :class:`concurrent.futures.Future` to access the result. + + This function is meant to be called from a different thread than the one + where the event loop is running. Usage:: + + # Create a coroutine + coro = asyncio.sleep(1, result=3) + # Submit the coroutine to a given loop + future = asyncio.run_coroutine_threadsafe(coro, loop) + # Wait for the result with an optional timeout argument + assert future.result(timeout) == 3 + + If an exception is raised in the coroutine, the returned future will be + notified. It can also be used to cancel the task in the event loop:: + + try: + result = future.result(timeout) + except asyncio.TimeoutError: + print('The coroutine took too long, cancelling the task...') + future.cancel() + except Exception as exc: + print('The coroutine raised an exception: {!r}'.format(exc)) + else: + print('The coroutine returned: {!r}'.format(result)) + + See the :ref:`concurrency and multithreading ` + section of the documentation. + + .. note:: + + Unlike the functions above, :func:`run_coroutine_threadsafe` requires the + *loop* argument to be passed explicitely. + + .. versionadded:: 3.4.4, 3.5.1 + .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) Create a :ref:`coroutine ` that completes after a given @@ -595,7 +636,21 @@ except CancelledError: res = None -.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None, return_when=ALL_COMPLETED) +.. function:: timeout(timeout, \*, loop=None) + + Return a context manager that cancels a block on *timeout* expiring:: + + with timeout(1.5): + yield from inner() + + 1. If ``inner()`` is executed faster than in ``1.5`` seconds + nothing happens. + 2. Otherwise ``inner()`` is cancelled internally but + :exc:`asyncio.TimeoutError` is raised outside of + context manager scope. + +.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None,\ + return_when=ALL_COMPLETED) Wait for the Futures and coroutine objects given by the sequence *futures* to complete. Coroutines will be wrapped in Tasks. Returns two sets of @@ -662,41 +717,3 @@ If the wait is cancelled, the future *fut* is now also cancelled. -.. function:: run_coroutine_threadsafe(coro, loop) - - Submit a :ref:`coroutine object ` to a given event loop. - - Return a :class:`concurrent.futures.Future` to access the result. - - This function is meant to be called from a different thread than the one - where the event loop is running. Usage:: - - # Create a coroutine - coro = asyncio.sleep(1, result=3) - # Submit the coroutine to a given loop - future = asyncio.run_coroutine_threadsafe(coro, loop) - # Wait for the result with an optional timeout argument - assert future.result(timeout) == 3 - - If an exception is raised in the coroutine, the returned future will be - notified. It can also be used to cancel the task in the event loop:: - - try: - result = future.result(timeout) - except asyncio.TimeoutError: - print('The coroutine took too long, cancelling the task...') - future.cancel() - except Exception as exc: - print('The coroutine raised an exception: {!r}'.format(exc)) - else: - print('The coroutine returned: {!r}'.format(result)) - - See the :ref:`concurrency and multithreading ` - section of the documentation. - - .. note:: - - Unlike the functions above, :func:`run_coroutine_threadsafe` requires the - *loop* argument to be passed explicitely. - - .. versionadded:: 3.4.4, 3.5.1 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 07:45:59 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 12:45:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160111124558.11405.66035@psf.io> https://hg.python.org/cpython/rev/6cc1ead02ec9 changeset: 99841:6cc1ead02ec9 parent: 99838:2a88b422c981 parent: 99840:858b172b77f6 user: Andrew Svetlov date: Mon Jan 11 14:45:49 2016 +0200 summary: merge 3.5 files: Doc/library/asyncio-task.rst | 112 +++++++++++++--------- 1 files changed, 64 insertions(+), 48 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -383,9 +383,10 @@ running in different threads. While a task waits for the completion of a future, the event loop executes a new task. - The cancellation of a task is different from the cancelation of a future. Calling - :meth:`cancel` will throw a :exc:`~concurrent.futures.CancelledError` to the - wrapped coroutine. :meth:`~Future.cancelled` only returns ``True`` if the + The cancellation of a task is different from the cancelation of a + future. Calling :meth:`cancel` will throw a + :exc:`~concurrent.futures.CancelledError` to the wrapped + coroutine. :meth:`~Future.cancelled` only returns ``True`` if the wrapped coroutine did not catch the :exc:`~concurrent.futures.CancelledError` exception, or raised a :exc:`~concurrent.futures.CancelledError` exception. @@ -437,10 +438,11 @@ Return the list of stack frames for this task's coroutine. - If the coroutine is not done, this returns the stack where it is suspended. - If the coroutine has completed successfully or was cancelled, this - returns an empty list. If the coroutine was terminated by an exception, - this returns the list of traceback frames. + If the coroutine is not done, this returns the stack where it is + suspended. If the coroutine has completed successfully or was + cancelled, this returns an empty list. If the coroutine was + terminated by an exception, this returns the list of traceback + frames. The frames are always ordered from oldest to newest. @@ -582,6 +584,46 @@ `, which may be a decorated generator function or an :keyword:`async def` function. +.. function:: run_coroutine_threadsafe(coro, loop) + + Submit a :ref:`coroutine object ` to a given event loop. + + Return a :class:`concurrent.futures.Future` to access the result. + + This function is meant to be called from a different thread than the one + where the event loop is running. Usage:: + + # Create a coroutine + coro = asyncio.sleep(1, result=3) + # Submit the coroutine to a given loop + future = asyncio.run_coroutine_threadsafe(coro, loop) + # Wait for the result with an optional timeout argument + assert future.result(timeout) == 3 + + If an exception is raised in the coroutine, the returned future will be + notified. It can also be used to cancel the task in the event loop:: + + try: + result = future.result(timeout) + except asyncio.TimeoutError: + print('The coroutine took too long, cancelling the task...') + future.cancel() + except Exception as exc: + print('The coroutine raised an exception: {!r}'.format(exc)) + else: + print('The coroutine returned: {!r}'.format(result)) + + See the :ref:`concurrency and multithreading ` + section of the documentation. + + .. note:: + + Unlike other functions from the module, + :func:`run_coroutine_threadsafe` requires the *loop* argument to + be passed explicitely. + + .. versionadded:: 3.4.4, 3.5.1 + .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) Create a :ref:`coroutine ` that completes after a given @@ -620,7 +662,21 @@ except CancelledError: res = None -.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None, return_when=ALL_COMPLETED) +.. function:: timeout(timeout, \*, loop=None) + + Return a context manager that cancels a block on *timeout* expiring:: + + with timeout(1.5): + yield from inner() + + 1. If ``inner()`` is executed faster than in ``1.5`` seconds + nothing happens. + 2. Otherwise ``inner()`` is cancelled internally but + :exc:`asyncio.TimeoutError` is raised outside of + context manager scope. + +.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None,\ + return_when=ALL_COMPLETED) Wait for the Futures and coroutine objects given by the sequence *futures* to complete. Coroutines will be wrapped in Tasks. Returns two sets of @@ -685,43 +741,3 @@ .. versionchanged:: 3.4.3 If the wait is cancelled, the future *fut* is now also cancelled. - - -.. function:: run_coroutine_threadsafe(coro, loop) - - Submit a :ref:`coroutine object ` to a given event loop. - - Return a :class:`concurrent.futures.Future` to access the result. - - This function is meant to be called from a different thread than the one - where the event loop is running. Usage:: - - # Create a coroutine - coro = asyncio.sleep(1, result=3) - # Submit the coroutine to a given loop - future = asyncio.run_coroutine_threadsafe(coro, loop) - # Wait for the result with an optional timeout argument - assert future.result(timeout) == 3 - - If an exception is raised in the coroutine, the returned future will be - notified. It can also be used to cancel the task in the event loop:: - - try: - result = future.result(timeout) - except asyncio.TimeoutError: - print('The coroutine took too long, cancelling the task...') - future.cancel() - except Exception as exc: - print('The coroutine raised an exception: {!r}'.format(exc)) - else: - print('The coroutine returned: {!r}'.format(result)) - - See the :ref:`concurrency and multithreading ` - section of the documentation. - - .. note:: - - Unlike the functions above, :func:`run_coroutine_threadsafe` requires the - *loop* argument to be passed explicitely. - - .. versionadded:: 3.5.1 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 07:45:59 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 12:45:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4?= Message-ID: <20160111124558.4224.89995@psf.io> https://hg.python.org/cpython/rev/858b172b77f6 changeset: 99840:858b172b77f6 branch: 3.5 parent: 99837:c8f7c1cf380a parent: 99839:5c13957152d6 user: Andrew Svetlov date: Mon Jan 11 14:45:25 2016 +0200 summary: merge 3.4 files: Doc/library/asyncio-task.rst | 112 +++++++++++++--------- 1 files changed, 64 insertions(+), 48 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -383,9 +383,10 @@ running in different threads. While a task waits for the completion of a future, the event loop executes a new task. - The cancellation of a task is different from the cancelation of a future. Calling - :meth:`cancel` will throw a :exc:`~concurrent.futures.CancelledError` to the - wrapped coroutine. :meth:`~Future.cancelled` only returns ``True`` if the + The cancellation of a task is different from the cancelation of a + future. Calling :meth:`cancel` will throw a + :exc:`~concurrent.futures.CancelledError` to the wrapped + coroutine. :meth:`~Future.cancelled` only returns ``True`` if the wrapped coroutine did not catch the :exc:`~concurrent.futures.CancelledError` exception, or raised a :exc:`~concurrent.futures.CancelledError` exception. @@ -437,10 +438,11 @@ Return the list of stack frames for this task's coroutine. - If the coroutine is not done, this returns the stack where it is suspended. - If the coroutine has completed successfully or was cancelled, this - returns an empty list. If the coroutine was terminated by an exception, - this returns the list of traceback frames. + If the coroutine is not done, this returns the stack where it is + suspended. If the coroutine has completed successfully or was + cancelled, this returns an empty list. If the coroutine was + terminated by an exception, this returns the list of traceback + frames. The frames are always ordered from oldest to newest. @@ -582,6 +584,46 @@ `, which may be a decorated generator function or an :keyword:`async def` function. +.. function:: run_coroutine_threadsafe(coro, loop) + + Submit a :ref:`coroutine object ` to a given event loop. + + Return a :class:`concurrent.futures.Future` to access the result. + + This function is meant to be called from a different thread than the one + where the event loop is running. Usage:: + + # Create a coroutine + coro = asyncio.sleep(1, result=3) + # Submit the coroutine to a given loop + future = asyncio.run_coroutine_threadsafe(coro, loop) + # Wait for the result with an optional timeout argument + assert future.result(timeout) == 3 + + If an exception is raised in the coroutine, the returned future will be + notified. It can also be used to cancel the task in the event loop:: + + try: + result = future.result(timeout) + except asyncio.TimeoutError: + print('The coroutine took too long, cancelling the task...') + future.cancel() + except Exception as exc: + print('The coroutine raised an exception: {!r}'.format(exc)) + else: + print('The coroutine returned: {!r}'.format(result)) + + See the :ref:`concurrency and multithreading ` + section of the documentation. + + .. note:: + + Unlike other functions from the module, + :func:`run_coroutine_threadsafe` requires the *loop* argument to + be passed explicitely. + + .. versionadded:: 3.4.4, 3.5.1 + .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) Create a :ref:`coroutine ` that completes after a given @@ -620,7 +662,21 @@ except CancelledError: res = None -.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None, return_when=ALL_COMPLETED) +.. function:: timeout(timeout, \*, loop=None) + + Return a context manager that cancels a block on *timeout* expiring:: + + with timeout(1.5): + yield from inner() + + 1. If ``inner()`` is executed faster than in ``1.5`` seconds + nothing happens. + 2. Otherwise ``inner()`` is cancelled internally but + :exc:`asyncio.TimeoutError` is raised outside of + context manager scope. + +.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None,\ + return_when=ALL_COMPLETED) Wait for the Futures and coroutine objects given by the sequence *futures* to complete. Coroutines will be wrapped in Tasks. Returns two sets of @@ -685,43 +741,3 @@ .. versionchanged:: 3.4.3 If the wait is cancelled, the future *fut* is now also cancelled. - - -.. function:: run_coroutine_threadsafe(coro, loop) - - Submit a :ref:`coroutine object ` to a given event loop. - - Return a :class:`concurrent.futures.Future` to access the result. - - This function is meant to be called from a different thread than the one - where the event loop is running. Usage:: - - # Create a coroutine - coro = asyncio.sleep(1, result=3) - # Submit the coroutine to a given loop - future = asyncio.run_coroutine_threadsafe(coro, loop) - # Wait for the result with an optional timeout argument - assert future.result(timeout) == 3 - - If an exception is raised in the coroutine, the returned future will be - notified. It can also be used to cancel the task in the event loop:: - - try: - result = future.result(timeout) - except asyncio.TimeoutError: - print('The coroutine took too long, cancelling the task...') - future.cancel() - except Exception as exc: - print('The coroutine raised an exception: {!r}'.format(exc)) - else: - print('The coroutine returned: {!r}'.format(result)) - - See the :ref:`concurrency and multithreading ` - section of the documentation. - - .. note:: - - Unlike the functions above, :func:`run_coroutine_threadsafe` requires the - *loop* argument to be passed explicitely. - - .. versionadded:: 3.5.1 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 08:42:12 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 13:42:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160111134212.27360.62052@psf.io> https://hg.python.org/cpython/rev/aa6e6574a2b7 changeset: 99843:aa6e6574a2b7 parent: 99841:6cc1ead02ec9 parent: 99842:8e0c7c78f532 user: Andrew Svetlov date: Mon Jan 11 15:42:05 2016 +0200 summary: merge 3.5 files: Doc/library/asyncio-task.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -622,7 +622,7 @@ :func:`run_coroutine_threadsafe` requires the *loop* argument to be passed explicitely. - .. versionadded:: 3.4.4, 3.5.1 + .. versionadded:: 3.5.1 .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 08:42:15 2016 From: python-checkins at python.org (andrew.svetlov) Date: Mon, 11 Jan 2016 13:42:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_versionadd?= =?utf-8?q?ed_to_use_3=2E5=2E1_only_for_3=2E5_branch?= Message-ID: <20160111134212.70887.96837@psf.io> https://hg.python.org/cpython/rev/8e0c7c78f532 changeset: 99842:8e0c7c78f532 branch: 3.5 parent: 99840:858b172b77f6 user: Andrew Svetlov date: Mon Jan 11 15:41:43 2016 +0200 summary: Fix versionadded to use 3.5.1 only for 3.5 branch files: Doc/library/asyncio-task.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -622,7 +622,7 @@ :func:`run_coroutine_threadsafe` requires the *loop* argument to be passed explicitely. - .. versionadded:: 3.4.4, 3.5.1 + .. versionadded:: 3.5.1 .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 10:09:47 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 11 Jan 2016 15:09:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326069=3A_Remove_t?= =?utf-8?q?he_deprecated_apis_in_the_trace_module=2E?= Message-ID: <20160111150947.11385.10454@psf.io> https://hg.python.org/cpython/rev/6f05bdc18661 changeset: 99844:6f05bdc18661 user: Senthil Kumaran date: Mon Jan 11 07:09:42 2016 -0800 summary: Issue #26069: Remove the deprecated apis in the trace module. files: Lib/test/test_trace.py | 48 +----------------------------- Lib/trace.py | 43 -------------------------- Misc/NEWS | 2 + 3 files changed, 3 insertions(+), 90 deletions(-) diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py --- a/Lib/test/test_trace.py +++ b/Lib/test/test_trace.py @@ -1,11 +1,10 @@ import os -import io import sys from test.support import TESTFN, rmtree, unlink, captured_stdout import unittest import trace -from trace import CoverageResults, Trace +from trace import Trace from test.tracedmodules import testmod @@ -366,50 +365,5 @@ self.assertTrue(ignore.names(jn('bar', 'baz.py'), 'baz')) -class TestDeprecatedMethods(unittest.TestCase): - - def test_deprecated_usage(self): - sio = io.StringIO() - with self.assertWarns(DeprecationWarning): - trace.usage(sio) - self.assertIn('Usage:', sio.getvalue()) - - def test_deprecated_Ignore(self): - with self.assertWarns(DeprecationWarning): - trace.Ignore() - - def test_deprecated_modname(self): - with self.assertWarns(DeprecationWarning): - self.assertEqual("spam", trace.modname("spam")) - - def test_deprecated_fullmodname(self): - with self.assertWarns(DeprecationWarning): - self.assertEqual("spam", trace.fullmodname("spam")) - - def test_deprecated_find_lines_from_code(self): - with self.assertWarns(DeprecationWarning): - def foo(): - pass - trace.find_lines_from_code(foo.__code__, ["eggs"]) - - def test_deprecated_find_lines(self): - with self.assertWarns(DeprecationWarning): - def foo(): - pass - trace.find_lines(foo.__code__, ["eggs"]) - - def test_deprecated_find_strings(self): - with open(TESTFN, 'w') as fd: - self.addCleanup(unlink, TESTFN) - with self.assertWarns(DeprecationWarning): - trace.find_strings(fd.name) - - def test_deprecated_find_executable_linenos(self): - with open(TESTFN, 'w') as fd: - self.addCleanup(unlink, TESTFN) - with self.assertWarns(DeprecationWarning): - trace.find_executable_linenos(fd.name) - - if __name__ == '__main__': unittest.main() diff --git a/Lib/trace.py b/Lib/trace.py --- a/Lib/trace.py +++ b/Lib/trace.py @@ -58,7 +58,6 @@ import gc import dis import pickle -from warnings import warn as _warn from time import monotonic as _time try: @@ -810,47 +809,5 @@ if not no_report: results.write_results(missing, summary=summary, coverdir=coverdir) -# Deprecated API -def usage(outfile): - _warn("The trace.usage() function is deprecated", - DeprecationWarning, 2) - _usage(outfile) - -class Ignore(_Ignore): - def __init__(self, modules=None, dirs=None): - _warn("The class trace.Ignore is deprecated", - DeprecationWarning, 2) - _Ignore.__init__(self, modules, dirs) - -def modname(path): - _warn("The trace.modname() function is deprecated", - DeprecationWarning, 2) - return _modname(path) - -def fullmodname(path): - _warn("The trace.fullmodname() function is deprecated", - DeprecationWarning, 2) - return _fullmodname(path) - -def find_lines_from_code(code, strs): - _warn("The trace.find_lines_from_code() function is deprecated", - DeprecationWarning, 2) - return _find_lines_from_code(code, strs) - -def find_lines(code, strs): - _warn("The trace.find_lines() function is deprecated", - DeprecationWarning, 2) - return _find_lines(code, strs) - -def find_strings(filename, encoding=None): - _warn("The trace.find_strings() function is deprecated", - DeprecationWarning, 2) - return _find_strings(filename, encoding=None) - -def find_executable_linenos(filename): - _warn("The trace.find_executable_linenos() function is deprecated", - DeprecationWarning, 2) - return _find_executable_linenos(filename) - if __name__=='__main__': main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,8 @@ Library ------- +- Issue #26069: Remove the deprecated apis in the trace module. + - Issue #22138: Fix mock.patch behavior when patching descriptors. Restore original values after patching. Patch contributed by Sean McCully. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 10:16:57 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 15:16:57 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_make_the_version_p?= =?utf-8?q?rivate?= Message-ID: <20160111151656.51244.32178@psf.io> https://hg.python.org/peps/rev/42b276dc6a9b changeset: 6160:42b276dc6a9b user: Victor Stinner date: Mon Jan 11 16:16:46 2016 +0100 summary: PEP 509: make the version private * reorganize sections * add microbenchmarks numbers * rationale to not expose the version at Python level files: pep-0509.txt | 356 ++++++++++++++++++++++---------------- 1 files changed, 204 insertions(+), 152 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -1,5 +1,5 @@ PEP: 509 -Title: Add dict.__version__ +Title: Add a private version to dict Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner @@ -13,8 +13,8 @@ Abstract ======== -Add a new read-only ``__version__`` property to ``dict`` and -``collections.UserDict`` types, incremented at each change. +Add a new private version to builtin ``dict`` type, incremented at each +change, to implement fast guards on namespaces. Rationale @@ -34,13 +34,19 @@ these checks "guards". The speedup of optimizations depends on the speed of guard checks. This -PEP proposes to add a version to dictionaries to implement efficient -guards on namespaces. +PEP proposes to add a version to dictionaries to implement fast guards +on namespaces. -Example of optimization: replace loading a global variable with a -constant. This optimization requires a guard on the global variable to +Dictionary lookups can be skipped if the version does not change which +is the common case for most namespaces. The performance of a guard does +not depend on the number of watched dictionary entries, complexity of +O(1), if the dictionary version does not change. + +Example of optimization: copy the value of a global variable to function +constants. This optimization requires a guard on the global variable to check if it was modified. If the variable is modified, the variable must -be loaded at runtime, instead of using the constant. +be loaded at runtime when the function is called, instead of using the +constant. See the `PEP 510 -- Specialized functions with guards `_ for the concrete usage of @@ -51,21 +57,22 @@ Guard example ============= -Pseudo-code of an efficient guard to check if a dictionary key was -modified (created, updated or deleted):: +Pseudo-code of an fast guard to check if a dictionary key was modified +(created, updated or deleted) using an hypothetical +``get_dict_version(dict)`` function:: UNSET = object() - class Guard: + class GuardDictKey: def __init__(self, dict, key): self.dict = dict self.key = key self.value = dict.get(key, UNSET) - self.version = dict.__version__ + self.version = get_dict_version(dict) def check(self): """Return True if the dictionary value did not changed.""" - version = self.dict.__version__ + version = get_dict_version(self.dict) if version == self.version: # Fast-path: avoid the dictionary lookup return True @@ -80,100 +87,21 @@ return False -Changes -======= - -Add a read-only ``__version__`` property to builtin ``dict`` type and to -the ``collections.UserDict`` type. New empty dictionaries are initilized -to version ``0``. The version is incremented at each change: - -* ``clear()`` if the dict was non-empty -* ``pop(key)`` if the key exists -* ``popitem()`` if the dict is non-empty -* ``setdefault(key, value)`` if the `key` does not exist -* ``__detitem__(key)`` if the key exists -* ``__setitem__(key, value)`` if the `key` doesn't exist or if the value - is different -* ``update(...)`` if new values are different than existing values (the - version can be incremented multiple times) - -Example:: - - >>> d = {} - >>> d.__version__ - 0 - >>> d['key'] = 'value' - >>> d.__version__ - 1 - >>> d['key'] = 'new value' - >>> d.__version__ - 2 - >>> del d['key'] - >>> d.__version__ - 3 - -If a dictionary is created with items, the version is also incremented -at each dictionary insertion. Example:: - - >>> d=dict(x=7, y=33) - >>> d.__version__ - 2 - -The version is not incremented is an existing key is modified to the -same value, but only the identifier of the value is tested, not the -content of the value. Example:: - - >>> d={} - >>> value = object() - >>> d['key'] = value - >>> d.__version__ - 2 - >>> d['key'] = value - >>> d.__version__ - 2 - -.. note:: - CPython uses some singleton like integers in the range [-5; 257], - empty tuple, empty strings, Unicode strings of a single character in - the range [U+0000; U+00FF], etc. When a key is set twice to the same - singleton, the version is not modified. - -The PEP is designed to implement guards on namespaces, only the ``dict`` -type can be used for namespaces in practice. ``collections.UserDict`` -is modified because it must mimicks ``dict``. ``collections.Mapping`` is -unchanged. - - -Integer overflow -================ - -The implementation uses the C unsigned integer type ``size_t`` to store -the version. On 32-bit systems, the maximum version is ``2**32-1`` -(more than ``4.2 * 10 ** 9``, 4 billions). On 64-bit systems, the maximum -version is ``2**64-1`` (more than ``1.8 * 10**19``). - -The C code uses ``version++``. On integer overflow, the version is -wrapped to ``0`` (and then continue to be incremented). - -The check ``dict.__version__ == old_version`` can be true after an -integer overflow, so a guard can return false even if the value changed, -which is wrong. The bug occurs if the dict is modified at least ``2**64`` -times (on 64-bit system) between two checks of the guard. - -Using a more complex type (ex: ``PyLongObject``) to avoid the overflow -would slow down operations on the ``dict`` type. Even if there is a -theorical risk of missing a value change, the risk is considered too low -compared to the slow down of using a more complex type. - - -Usage of dict.__version__ +Usage of the dict version ========================= -astoptimizer of FAT Python --------------------------- +Specialized functions using guards +---------------------------------- -The astoptimizer of the FAT Python project implements many optimizations -which require guards on namespaces. Examples: +The `PEP 510 -- Specialized functions with guards +`_ proposes an API to support +specialized functions with guards. It allows to implement static +optimizers for Python without breaking the Python semantics. + +Example of a static Python optimizer: the astoptimizer of the `FAT +Python `_ project +implements many optimizations which require guards on namespaces. +Examples: * Call pure builtins: to replace ``len("abc")`` with ``3``, guards on ``builtins.__dict__['len']`` and ``globals()['len']`` are required @@ -181,10 +109,6 @@ guards on ``builtins.__dict__['range']`` and ``globals()['range']`` are required -The `FAT Python -`_ project is a -static optimizer for Python 3.6. - Pyjion ------ @@ -210,21 +134,143 @@ `_. +Changes +======= + +Add a ``PY_INT64_T ma_version`` field to the ``PyDictObject`` structure: +64-bit unsigned integer. New empty dictionaries are initilized to +version ``0``. The version is incremented at each change: + +* ``clear()`` if the dict was non-empty +* ``pop(key)`` if the key exists +* ``popitem()`` if the dict is non-empty +* ``setdefault(key, value)`` if the `key` does not exist +* ``__detitem__(key)`` if the key exists +* ``__setitem__(key, value)`` if the `key` doesn't exist or if the value + is different +* ``update(...)`` if new values are different than existing values (the + version can be incremented multiple times) + +Example using an hypothetical ``get_dict_version(dict)`` function:: + + >>> d = {} + >>> get_dict_version(d) + 0 + >>> d['key'] = 'value' + >>> get_dict_version(d) + 1 + >>> d['key'] = 'new value' + >>> get_dict_version(d) + 2 + >>> del d['key'] + >>> get_dict_version(d) + 3 + +If a dictionary is created with items, the version is also incremented +at each dictionary insertion. Example:: + + >>> d=dict(x=7, y=33) + >>> get_dict_version(d) + 2 + +The version is not incremented is an existing key is modified to the +same value, but only the identifier of the value is tested, not the +content of the value. Example:: + + >>> d={} + >>> value = object() + >>> d['key'] = value + >>> get_dict_version(d) + 2 + >>> d['key'] = value + >>> get_dict_version(d) + 2 + +.. note:: + CPython uses some singleton like integers in the range [-5; 257], + empty tuple, empty strings, Unicode strings of a single character in + the range [U+0000; U+00FF], etc. When a key is set twice to the same + singleton, the version is not modified. + + Implementation ============== -See the `issue #26058: Add dict.__version__ read-only property -`_. +The `issue #26058: PEP 509: Add ma_version to PyDictObject +`_ contains a patch implementing +this PEP. On pybench and timeit microbenchmarks, the patch does not seem to add any overhead on dictionary operations. +When the version does not change, ``PyDict_GetItem()`` takes 14.8 ns for +a dictioanry lookup, whereas a guard check only takes 3.8 ns. Moreover, +a guard can watch multiple keys. For example, for an optimization using +10 global variables in a function, the check costs 148 ns for 10 dict +lookups, whereas the guard still only cost 3.8 ns when the version does +not change (39x as fast). + + +Integer overflow +================ + +The implementation uses the C unsigned integer type ``PY_INT64_T`` to +store the version, a 64 bits unsigned integer. The C code uses +``version++``. On integer overflow, the version is wrapped to ``0`` (and +then continue to be incremented) according to the C standard. + +After an integer overflow, a guard can succeed whereas the watched +dictionary key was modified. The bug occurs if the dictionary is +modified at least ``2 ** 64`` times between two checks of the guard and +if the new version (theorical value with no integer overflow) is equal +to the old version modulo ``2 ** 64``. + +If a dictionary is modified each nanosecond, an overflow takes longer +than 584 years. Using a 32-bit version, the overflow occurs only after 4 +seconds. That's why a 64-bit unsigned type is also used on 32-bit +systems. + +A risk of a bug every 584 years is acceptable. + Alternatives ============ -Bikeshedding on the property name ---------------------------------- +Expose the version at Python level as a read-only __version__ property +---------------------------------------------------------------------- + +The first version of the PEP proposed to expose the dictionary version +as a read-only ``__version__`` property at Python level, and also to add +the property to ``collections.UserDict`` (since this type must mimick +the ``dict`` API). + +There are multiple issues: + +* To be consistent and avoid bad surprises, the version must be added to + all mapping type. Implementing a new mapping type would require extra + work for no benefit, since the version is only required on the + ``dict`` type in practice. +* All Python implementations must implement this new property, it gives + more work to other implementations, whereas they may not use the + dictionary version at all. +* The ``__version__`` can be wrapped on integer overflow. It is error + prone: using ``dict.__version__ <= guard_version`` is wrong, + ``dict.__version__ == guard_version`` must be used instead to reduce + the risk of bug on integer overflow (even if the integer overflow is + unlikely in practice). +* Exposing the dictioanry version can lead the + false assumption on performances. Checking ``dict.__version__`` at + the Python level is not faster than a dictionary lookup. The lookup + has a cost of 48.7 ns and checking a guard has a cost of 47.5 ns, the + difference is only 1.2 ns (3%):: + + + $ ./python -m timeit -s 'd = {str(i):i for i in range(100)}' 'd["33"] == 33' + 10000000 loops, best of 3: 0.0487 usec per loop + $ ./python -m timeit -s 'd = {str(i):i for i in range(100)}' 'd.__version__ == 100' + 10000000 loops, best of 3: 0.0475 usec per loop + +Bikeshedding on the property name: * ``__cache_token__``: name proposed by Nick Coghlan, name coming from `abc.get_cache_token() @@ -233,13 +279,6 @@ * ``__timestamp__`` -Add a version but don't expose it at the Python level ------------------------------------------------------ - -Guards can be implemented in C to access directly the ``ma_version`` -field of the ``PyDictObject`` structure. - - Add a version to each dict entry -------------------------------- @@ -254,8 +293,8 @@ the entry version is set to the dictionary version which is incremented at each change (create, modify, delete). -Pseudo-code of an efficient guard to check if a dict key was modified -using ``getversion()``:: +Pseudo-code of an fast guard to check if a dict key was modified using +``getversion()``:: UNSET = object() @@ -263,12 +302,12 @@ def __init__(self, dict, key): self.dict = dict self.key = key - self.dict_version = dict.__version__ + self.dict_version = get_dict_version(dict) self.entry_version = dict.getversion(key) def check(self): """Return True if the dictionary value did not changed.""" - dict_version = self.dict.__version__ + dict_version = get_dict_version(self.dict) if dict_version == self.version: # Fast-path: avoid the dictionary lookup return True @@ -366,42 +405,55 @@ On integer overflow, the whole cache is cleared and the global version tag is reset to ``0``. -See also `issue #1685986: Method cache -`_ and `issue #1700288: Armin's -method cache optimization updated for Python 2.6 +See `Method cache (issue #1685986) +`_ and `Armin's method cache +optimization updated for Python 2.6 (issue #1700288) `_. +Globals / builtins cache +------------------------ + +In 2010, Antoine Pitrou proposed a `Globals / builtins cache (issue +#10401) `_ which adds a private +``ma_version`` field to the ``PyDictObject`` structure (``dict`` type), +the field has the C type ``Py_ssize_t``. + +The patch adds a "global and builtin cache" to functions and frames, and +changes ``LOAD_GLOBAL`` and ``STORE_GLOBAL`` instructions to use the +cache. + +The change on the ``PyDictObject`` structure is very similar to this +PEP. + + +Cached globals+builtins lookup +------------------------------ + +In 2006, Andrea Griffini proposed a patch implementing a `Cached +globals+builtins lookup optimization +`_. The patch adds a private +``timestamp`` field to the ``PyDictObject`` structure (``dict`` type), +the field has the C type ``size_t``. + +Thread on python-dev: `About dictionary lookup caching +`_. + + Guard against changing dict during iteration -------------------------------------------- -In 2013, Serhiy Storchaka proposed a patch for the `issue #19332: Guard -against changing dict during iteration -`_ (mentioned above) which adds a -``size_t ma_count`` field to the ``PyDictObject`` structure. This field -is incremented when the dictionary is modified, and so is very similar -to the proposed ``dict.__version__``. +In 2013, Serhiy Storchaka proposed `Guard against changing dict during +iteration (issue #19332) `_ which +adds a ``ma_count`` field to the ``PyDictObject`` structure (``dict`` +type), the field has the C type ``size_t``. This field is incremented +when the dictionary is modified, and so is very similar to the proposed +dictionary version. - -Cached globals+builtins lookup ------------------------------- - -In 2006, Andrea Griffini proposes a patch implementing a `Cached -globals+builtins lookup optimization `_. -The patch adds a private ``timestamp`` field to dict. - -See the thread on python-dev: `About dictionary lookup caching -`_. - - -Globals / builtins cache ------------------------- - -In 2010, Antoine Pitrou proposed a `Globals / builtins cache -`_ which adds a private -``ma_version`` field to the ``dict`` type. The patch adds a "global and -builtin cache" to functions and frames, and changes ``LOAD_GLOBAL`` and -``STORE_GLOBAL`` instructions to use the cache. +Sadly, the dictionary version proposed in this PEP doesn't help to +detect dictionary mutation. The dictionary version changes when values +are replaced, whereas modifying dictionary values while iterating on +dictionary keys is legit in Python. PySizer -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 10:27:52 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 15:27:52 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_reformat_to_72_col?= =?utf-8?q?umns?= Message-ID: <20160111152751.70877.88420@psf.io> https://hg.python.org/peps/rev/37799262bc35 changeset: 6161:37799262bc35 user: Victor Stinner date: Mon Jan 11 16:27:47 2016 +0100 summary: PEP 509: reformat to 72 columns files: pep-0509.txt | 22 ++++++++++++---------- 1 files changed, 12 insertions(+), 10 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -113,24 +113,26 @@ Pyjion ------ -According of Brett Cannon, one of the two main developers of Pyjion, Pyjion can -also benefit from dictionary version to implement optimizations. +According of Brett Cannon, one of the two main developers of Pyjion, +Pyjion can also benefit from dictionary version to implement +optimizations. -Pyjion is a JIT compiler for Python based upon CoreCLR (Microsoft .NET Core -runtime). +Pyjion is a JIT compiler for Python based upon CoreCLR (Microsoft .NET +Core runtime). Unladen Swallow --------------- -Even if dictionary version was not explicitly mentionned, optimization globals -and builtins lookup was part of the Unladen Swallow plan: "Implement one of the -several proposed schemes for speeding lookups of globals and builtins." -Source: `Unladen Swallow ProjectPlan +Even if dictionary version was not explicitly mentionned, optimization +globals and builtins lookup was part of the Unladen Swallow plan: +"Implement one of the several proposed schemes for speeding lookups of +globals and builtins." Source: `Unladen Swallow ProjectPlan `_. -Unladen Swallow is a fork of CPython 2.6.1 adding a JIT compiler implemented -with LLVM. The project stopped in 2011: `Unladen Swallow Retrospective +Unladen Swallow is a fork of CPython 2.6.1 adding a JIT compiler +implemented with LLVM. The project stopped in 2011: `Unladen Swallow +Retrospective `_. -- Repository URL: https://hg.python.org/peps From lp_benchmark_robot at intel.com Mon Jan 11 10:49:22 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 11 Jan 2016 15:49:22 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-11 Message-ID: <6ccb6a1e-7d13-4876-a6d6-0a7f2a636b88@irsmsx103.ger.corp.intel.com> Results for project Python default, build date 2016-01-11 03:09:57 +0000 commit: 8ee02ec9b6340d4da9a6ee0a8a2e9fbcfa296372 previous commit: 74adca5e885c32b6c61d44c4a1108ab39ee1cf96 revision date: 2016-01-10 07:56:40 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.19% 0.71% 11.72% 12.23% :-| pybench 0.06% -0.04% -1.65% 5.46% :-( regex_v8 2.66% -0.19% -5.16% 4.98% :-| nbody 0.07% -0.01% 1.53% 5.46% :-| json_dump_v2 0.23% 0.17% -1.93% 12.74% :-( normal_startup 0.95% -0.45% -2.22% 5.48% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-default-2016-01-11/ Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Jan 11 10:50:49 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 11 Jan 2016 15:50:49 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-11 Message-ID: Results for project Python 2.7, build date 2016-01-11 04:01:47 +0000 commit: 84fdbdd483a3551c3342c2a9c5a8aee83731a096 previous commit: 8b20a606ecde95d334ea3658ce2ec3169a0d27f5 revision date: 2016-01-10 07:55:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.39% 0.43% 4.67% 1.46% :-) pybench 0.14% -0.02% 5.95% 4.35% :-( regex_v8 1.10% -0.19% -2.44% 10.72% :-) nbody 0.10% 0.22% 7.81% -0.10% :-) json_dump_v2 0.27% -0.77% 4.91% 8.60% :-( normal_startup 1.94% 0.68% -5.39% 1.79% :-| ssbench 0.22% -0.17% 0.61% 1.42% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-2-7-2016-01-11/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Jan 11 11:32:16 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 16:32:16 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_minor_edits?= Message-ID: <20160111163216.11403.23285@psf.io> https://hg.python.org/peps/rev/055e3682090e changeset: 6162:055e3682090e user: Victor Stinner date: Mon Jan 11 17:18:06 2016 +0100 summary: PEP 509: minor edits files: pep-0509.txt | 118 ++++++++++++++++++++------------------ 1 files changed, 63 insertions(+), 55 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -57,9 +57,9 @@ Guard example ============= -Pseudo-code of an fast guard to check if a dictionary key was modified +Pseudo-code of an fast guard to check if a dictionary entry was modified (created, updated or deleted) using an hypothetical -``get_dict_version(dict)`` function:: +``dict_get_version(dict)`` function:: UNSET = object() @@ -68,22 +68,26 @@ self.dict = dict self.key = key self.value = dict.get(key, UNSET) - self.version = get_dict_version(dict) + self.version = dict_get_version(dict) def check(self): - """Return True if the dictionary value did not changed.""" - version = get_dict_version(self.dict) + """Return True if the dictionary entry did not changed.""" + + # read the version field of the dict structure + version = dict_get_version(self.dict) if version == self.version: - # Fast-path: avoid the dictionary lookup + # Fast-path: dictionary lookup avoided return True + # lookup in the dictionary value = self.dict.get(self.key, UNSET) - if value == self.value: + if value is self.value: # another key was modified: # cache the new dictionary version self.version = version return True + # the key was modified return False @@ -139,9 +143,9 @@ Changes ======= -Add a ``PY_INT64_T ma_version`` field to the ``PyDictObject`` structure: -64-bit unsigned integer. New empty dictionaries are initilized to -version ``0``. The version is incremented at each change: +Add a ``ma_version`` field to the ``PyDictObject`` structure with the C +type ``PY_INT64_T``, 64-bit unsigned integer. New empty dictionaries are +initilized to version ``0``. The version is incremented at each change: * ``clear()`` if the dict was non-empty * ``pop(key)`` if the key exists @@ -153,39 +157,40 @@ * ``update(...)`` if new values are different than existing values (the version can be incremented multiple times) -Example using an hypothetical ``get_dict_version(dict)`` function:: +Example using an hypothetical ``dict_get_version(dict)`` function:: >>> d = {} - >>> get_dict_version(d) + >>> dict_get_version(d) 0 >>> d['key'] = 'value' - >>> get_dict_version(d) + >>> dict_get_version(d) 1 >>> d['key'] = 'new value' - >>> get_dict_version(d) + >>> dict_get_version(d) 2 >>> del d['key'] - >>> get_dict_version(d) + >>> dict_get_version(d) 3 If a dictionary is created with items, the version is also incremented at each dictionary insertion. Example:: >>> d=dict(x=7, y=33) - >>> get_dict_version(d) + >>> dict_get_version(d) 2 -The version is not incremented is an existing key is modified to the -same value, but only the identifier of the value is tested, not the -content of the value. Example:: +The version is not incremented if an existing key is set to the same +value. For efficiency, values are compared by their identity: +``new_value is old_value``, not by their content: +``new_value == old_value``. Example:: >>> d={} >>> value = object() >>> d['key'] = value - >>> get_dict_version(d) + >>> dict_get_version(d) 2 >>> d['key'] = value - >>> get_dict_version(d) + >>> dict_get_version(d) 2 .. note:: @@ -207,10 +212,10 @@ When the version does not change, ``PyDict_GetItem()`` takes 14.8 ns for a dictioanry lookup, whereas a guard check only takes 3.8 ns. Moreover, -a guard can watch multiple keys. For example, for an optimization using -10 global variables in a function, the check costs 148 ns for 10 dict -lookups, whereas the guard still only cost 3.8 ns when the version does -not change (39x as fast). +a guard can watch for multiple keys. For example, for an optimization +using 10 global variables in a function, 10 dictionary lookups costs 148 +ns, whereas the guard still only costs 3.8 ns when the version does not +change (39x as fast). Integer overflow @@ -230,7 +235,7 @@ If a dictionary is modified each nanosecond, an overflow takes longer than 584 years. Using a 32-bit version, the overflow occurs only after 4 seconds. That's why a 64-bit unsigned type is also used on 32-bit -systems. +systems. A dictionary lookup at the C level takes 14.8 ns. A risk of a bug every 584 years is acceptable. @@ -249,7 +254,7 @@ There are multiple issues: * To be consistent and avoid bad surprises, the version must be added to - all mapping type. Implementing a new mapping type would require extra + all mapping types. Implementing a new mapping type would require extra work for no benefit, since the version is only required on the ``dict`` type in practice. * All Python implementations must implement this new property, it gives @@ -260,11 +265,11 @@ ``dict.__version__ == guard_version`` must be used instead to reduce the risk of bug on integer overflow (even if the integer overflow is unlikely in practice). -* Exposing the dictioanry version can lead the +* Exposing the dictionary version at Python level can lead the false assumption on performances. Checking ``dict.__version__`` at - the Python level is not faster than a dictionary lookup. The lookup - has a cost of 48.7 ns and checking a guard has a cost of 47.5 ns, the - difference is only 1.2 ns (3%):: + the Python level is not faster than a dictionary lookup. A dictionary + lookup has a cost of 48.7 ns and checking a guard has a cost of 47.5 + ns, the difference is only 1.2 ns (3%):: $ ./python -m timeit -s 'd = {str(i):i for i in range(100)}' 'd["33"] == 33' @@ -286,53 +291,56 @@ A single version per dictionary requires to keep a strong reference to the value which can keep the value alive longer than expected. If we add -also a version per dictionary entry, the guard can rely on the entry -version and so avoid the strong reference to the value (only strong -references to a dictionary and key are needed). +also a version per dictionary entry, the guard can only store the entry +version to avoid the strong reference to the value (only strong +references to the dictionary and to the key are needed). -Changes: add a ``getversion(key)`` method to dictionary which returns -``None`` if the key doesn't exist. When a key is created or modified, -the entry version is set to the dictionary version which is incremented -at each change (create, modify, delete). +Changes: add a ``me_version`` field to the ``PyDictKeyEntry`` structure, +the field has the C type ``PY_INT64_T``. When a key is created or +modified, the entry version is set to the dictionary version which is +incremented at any change (create, modify, delete). -Pseudo-code of an fast guard to check if a dict key was modified using -``getversion()``:: +Pseudo-code of an fast guard to check if a dictionary key was modified +using hypothetical ``dict_get_version(dict)`` +``dict_get_entry_version(dict)`` functions:: UNSET = object() - class Guard: + class GuardDictKey: def __init__(self, dict, key): self.dict = dict self.key = key - self.dict_version = get_dict_version(dict) - self.entry_version = dict.getversion(key) + self.dict_version = dict_get_version(dict) + self.entry_version = dict_get_entry_version(dict, key) def check(self): - """Return True if the dictionary value did not changed.""" - dict_version = get_dict_version(self.dict) + """Return True if the dictionary entry did not changed.""" + + # read the version field of the dict structure + dict_version = dict_get_version(self.dict) if dict_version == self.version: - # Fast-path: avoid the dictionary lookup + # Fast-path: dictionary lookup avoided return True - # lookup in the dictionary, but get the entry version, - #not the value - entry_version = self.dict.getversion(self.key) + # lookup in the dictionary + entry_version = get_dict_key_version(dict, key) if entry_version == self.entry_version: # another key was modified: # cache the new dictionary version self.dict_version = dict_version return True + # the key was modified return False -This main drawback of this option is the impact on the memory footprint. +The main drawback of this option is the impact on the memory footprint. It increases the size of each dictionary entry, so the overhead depends on the number of buckets (dictionary entries, used or unused yet). For example, it increases the size of each dictionary entry by 8 bytes on -64-bit system if we use ``size_t``. +64-bit system. -In Python, the memory footprint matters and the trend is more to reduce -it. Examples: +In Python, the memory footprint matters and the trend is to reduce it. +Examples: * `PEP 393 -- Flexible String Representation `_ @@ -351,7 +359,7 @@ footprint) when guards are not needed. Technical issue: a lot of C code in the wild, including CPython core, -expect the exact ``dict`` type. Issues: +expecting the exact ``dict`` type. Issues: * ``exec()`` requires a ``dict`` for globals and locals. A lot of code use ``globals={}``. It is not possible to cast the ``dict`` to a @@ -371,7 +379,7 @@ * The garbage collector has a special code to "untrack" ``dict`` instances. If a ``dict`` subtype is used for namespaces, the garbage - collector may be unable to break some reference cycles. + collector can be unable to break some reference cycles. * Some functions have a fast-path for ``dict`` which would not be taken for ``dict`` subtypes, and so it would make Python a little bit slower. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 11:49:11 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 16:49:11 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_rephrase_the_sente?= =?utf-8?q?nce_about_serialization?= Message-ID: <20160111164911.31091.37693@psf.io> https://hg.python.org/peps/rev/6e7b7e0bf5f4 changeset: 6163:6e7b7e0bf5f4 user: Victor Stinner date: Mon Jan 11 17:49:07 2016 +0100 summary: PEP 510: rephrase the sentence about serialization files: pep-0510.txt | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -257,8 +257,10 @@ When a function code is replaced (``func.__code__ = new_code``), all specialized functions are removed. -When a function is serialized (by ``marshal`` or ``pickle`` for -example), specialized functions and guards are ignored (not serialized). +When a function is serialized ``pickle``, specialized functions and +guards are ignored (not serialized). Specialized functions and guards +are not stored in ``.pyc`` files but created and registered at runtime, +when a module is loaded. Discussion -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 12:10:59 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 17:10:59 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_abandonned_=3D=3E_?= =?utf-8?q?abandoned?= Message-ID: <20160111171058.51230.76383@psf.io> https://hg.python.org/peps/rev/63151533cebf changeset: 6164:63151533cebf user: Victor Stinner date: Mon Jan 11 18:10:49 2016 +0100 summary: PEP 510: abandonned => abandoned Typo reported in issue #26080. files: pep-0510.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -92,7 +92,7 @@ between CPython 3.5 and PyPy. .. note:: - There was also the Unladen Swallow project, but it was abandonned in + There was also the Unladen Swallow project, but it was abandoned in 2011. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 12:32:55 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 17:32:55 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_typo=2C_PY=5FUINT6?= =?utf-8?b?NF9UICh1bnNpZ25lZCksIG5vdCBQWV9JTlQ2NF9UIChzaWduZWQp?= Message-ID: <20160111173241.59582.68334@psf.io> https://hg.python.org/peps/rev/63539f1be6c8 changeset: 6165:63539f1be6c8 user: Victor Stinner date: Mon Jan 11 18:32:22 2016 +0100 summary: PEP 509: typo, PY_UINT64_T (unsigned), not PY_INT64_T (signed) files: pep-0509.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -221,7 +221,7 @@ Integer overflow ================ -The implementation uses the C unsigned integer type ``PY_INT64_T`` to +The implementation uses the C unsigned integer type ``PY_UINT64_T`` to store the version, a 64 bits unsigned integer. The C code uses ``version++``. On integer overflow, the version is wrapped to ``0`` (and then continue to be incremented) according to the C standard. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 12:33:01 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 11 Jan 2016 17:33:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpc3N1ZSAjMjYwNTAp?= Message-ID: <20160111173300.31081.82389@psf.io> https://hg.python.org/cpython/rev/c2cd3b435afe changeset: 99847:c2cd3b435afe parent: 99844:6f05bdc18661 parent: 99846:a6288fe5420f user: Yury Selivanov date: Mon Jan 11 12:32:53 2016 -0500 summary: Merge 3.5 (issue #26050) files: Lib/asyncio/streams.py | 221 +++++++++++-- Lib/test/test_asyncio/test_streams.py | 128 ++++++++- Misc/NEWS | 3 + 3 files changed, 315 insertions(+), 37 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -3,6 +3,7 @@ __all__ = ['StreamReader', 'StreamWriter', 'StreamReaderProtocol', 'open_connection', 'start_server', 'IncompleteReadError', + 'LimitOverrunError', ] import socket @@ -27,15 +28,28 @@ Incomplete read error. Attributes: - partial: read bytes string before the end of stream was reached - - expected: total number of expected bytes + - expected: total number of expected bytes (or None if unknown) """ def __init__(self, partial, expected): - EOFError.__init__(self, "%s bytes read on a total of %s expected bytes" - % (len(partial), expected)) + super().__init__("%d bytes read on a total of %r expected bytes" + % (len(partial), expected)) self.partial = partial self.expected = expected +class LimitOverrunError(Exception): + """Reached buffer limit while looking for the separator. + + Attributes: + - message: error message + - consumed: total number of bytes that should be consumed + """ + def __init__(self, message, consumed): + super().__init__(message) + self.message = message + self.consumed = consumed + + @coroutine def open_connection(host=None, port=None, *, loop=None, limit=_DEFAULT_LIMIT, **kwds): @@ -318,6 +332,10 @@ def __init__(self, limit=_DEFAULT_LIMIT, loop=None): # The line length limit is a security feature; # it also doubles as half the buffer limit. + + if limit <= 0: + raise ValueError('Limit cannot be <= 0') + self._limit = limit if loop is None: self._loop = events.get_event_loop() @@ -361,7 +379,7 @@ waiter.set_exception(exc) def _wakeup_waiter(self): - """Wakeup read() or readline() function waiting for data or EOF.""" + """Wakeup read*() functions waiting for data or EOF.""" waiter = self._waiter if waiter is not None: self._waiter = None @@ -409,7 +427,10 @@ @coroutine def _wait_for_data(self, func_name): - """Wait until feed_data() or feed_eof() is called.""" + """Wait until feed_data() or feed_eof() is called. + + If stream was paused, automatically resume it. + """ # StreamReader uses a future to link the protocol feed_data() method # to a read coroutine. Running two read coroutines at the same time # would have an unexpected behaviour. It would not possible to know @@ -418,6 +439,13 @@ raise RuntimeError('%s() called while another coroutine is ' 'already waiting for incoming data' % func_name) + assert not self._eof, '_wait_for_data after EOF' + + # Waiting for data while paused will make deadlock, so prevent it. + if self._paused: + self._paused = False + self._transport.resume_reading() + self._waiter = futures.Future(loop=self._loop) try: yield from self._waiter @@ -426,43 +454,150 @@ @coroutine def readline(self): + """Read chunk of data from the stream until newline (b'\n') is found. + + On success, return chunk that ends with newline. If only partial + line can be read due to EOF, return incomplete line without + terminating newline. When EOF was reached while no bytes read, empty + bytes object is returned. + + If limit is reached, ValueError will be raised. In that case, if + newline was found, complete line including newline will be removed + from internal buffer. Else, internal buffer will be cleared. Limit is + compared against part of the line without newline. + + If stream was paused, this function will automatically resume it if + needed. + """ + sep = b'\n' + seplen = len(sep) + try: + line = yield from self.readuntil(sep) + except IncompleteReadError as e: + return e.partial + except LimitOverrunError as e: + if self._buffer.startswith(sep, e.consumed): + del self._buffer[:e.consumed + seplen] + else: + self._buffer.clear() + self._maybe_resume_transport() + raise ValueError(e.args[0]) + return line + + @coroutine + def readuntil(self, separator=b'\n'): + """Read chunk of data from the stream until `separator` is found. + + On success, chunk and its separator will be removed from internal buffer + (i.e. consumed). Returned chunk will include separator at the end. + + Configured stream limit is used to check result. Limit means maximal + length of chunk that can be returned, not counting the separator. + + If EOF occurs and complete separator still not found, + IncompleteReadError(, None) will be raised and internal + buffer becomes empty. This partial data may contain a partial separator. + + If chunk cannot be read due to overlimit, LimitOverrunError will be raised + and data will be left in internal buffer, so it can be read again, in + some different way. + + If stream was paused, this function will automatically resume it if + needed. + """ + seplen = len(separator) + if seplen == 0: + raise ValueError('Separator should be at least one-byte string') + if self._exception is not None: raise self._exception - line = bytearray() - not_enough = True + # Consume whole buffer except last bytes, which length is + # one less than seplen. Let's check corner cases with + # separator='SEPARATOR': + # * we have received almost complete separator (without last + # byte). i.e buffer='some textSEPARATO'. In this case we + # can safely consume len(separator) - 1 bytes. + # * last byte of buffer is first byte of separator, i.e. + # buffer='abcdefghijklmnopqrS'. We may safely consume + # everything except that last byte, but this require to + # analyze bytes of buffer that match partial separator. + # This is slow and/or require FSM. For this case our + # implementation is not optimal, since require rescanning + # of data that is known to not belong to separator. In + # real world, separator will not be so long to notice + # performance problems. Even when reading MIME-encoded + # messages :) - while not_enough: - while self._buffer and not_enough: - ichar = self._buffer.find(b'\n') - if ichar < 0: - line.extend(self._buffer) - self._buffer.clear() - else: - ichar += 1 - line.extend(self._buffer[:ichar]) - del self._buffer[:ichar] - not_enough = False + # `offset` is the number of bytes from the beginning of the buffer where + # is no occurrence of `separator`. + offset = 0 - if len(line) > self._limit: - self._maybe_resume_transport() - raise ValueError('Line is too long') + # Loop until we find `separator` in the buffer, exceed the buffer size, + # or an EOF has happened. + while True: + buflen = len(self._buffer) + # Check if we now have enough data in the buffer for `separator` to + # fit. + if buflen - offset >= seplen: + isep = self._buffer.find(separator, offset) + + if isep != -1: + # `separator` is in the buffer. `isep` will be used later to + # retrieve the data. + break + + # see upper comment for explanation. + offset = buflen + 1 - seplen + if offset > self._limit: + raise LimitOverrunError('Separator is not found, and chunk exceed the limit', offset) + + # Complete message (with full separator) may be present in buffer + # even when EOF flag is set. This may happen when the last chunk + # adds data which makes separator be found. That's why we check for + # EOF *ater* inspecting the buffer. if self._eof: - break + chunk = bytes(self._buffer) + self._buffer.clear() + raise IncompleteReadError(chunk, None) - if not_enough: - yield from self._wait_for_data('readline') + # _wait_for_data() will resume reading if stream was paused. + yield from self._wait_for_data('readuntil') + if isep > self._limit: + raise LimitOverrunError('Separator is found, but chunk is longer than limit', isep) + + chunk = self._buffer[:isep + seplen] + del self._buffer[:isep + seplen] self._maybe_resume_transport() - return bytes(line) + return bytes(chunk) @coroutine def read(self, n=-1): + """Read up to `n` bytes from the stream. + + If n is not provided, or set to -1, read until EOF and return all read + bytes. If the EOF was received and the internal buffer is empty, return + an empty bytes object. + + If n is zero, return empty bytes object immediatelly. + + If n is positive, this function try to read `n` bytes, and may return + less or equal bytes than requested, but at least one byte. If EOF was + received before any byte is read, this function returns empty byte + object. + + Returned value is not limited with limit, configured at stream creation. + + If stream was paused, this function will automatically resume it if + needed. + """ + if self._exception is not None: raise self._exception - if not n: + if n == 0: return b'' if n < 0: @@ -477,29 +612,41 @@ break blocks.append(block) return b''.join(blocks) - else: - if not self._buffer and not self._eof: - yield from self._wait_for_data('read') - if n < 0 or len(self._buffer) <= n: - data = bytes(self._buffer) - self._buffer.clear() - else: - # n > 0 and len(self._buffer) > n - data = bytes(self._buffer[:n]) - del self._buffer[:n] + if not self._buffer and not self._eof: + yield from self._wait_for_data('read') + + # This will work right even if buffer is less than n bytes + data = bytes(self._buffer[:n]) + del self._buffer[:n] self._maybe_resume_transport() return data @coroutine def readexactly(self, n): + """Read exactly `n` bytes. + + Raise an `IncompleteReadError` if EOF is reached before `n` bytes can be + read. The `IncompleteReadError.partial` attribute of the exception will + contain the partial read bytes. + + if n is zero, return empty bytes object. + + Returned value is not limited with limit, configured at stream creation. + + If stream was paused, this function will automatically resume it if + needed. + """ if n < 0: raise ValueError('readexactly size can not be less than zero') if self._exception is not None: raise self._exception + if n == 0: + return b'' + # There used to be "optimized" code here. It created its own # Future and waited until self._buffer had at least the n # bytes, then called read(n). Unfortunately, this could pause @@ -516,6 +663,8 @@ blocks.append(block) n -= len(block) + assert n == 0 + return b''.join(blocks) if compat.PY35: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -203,6 +203,20 @@ self.assertRaises( ValueError, self.loop.run_until_complete, stream.read(2)) + def test_invalid_limit(self): + with self.assertRaisesRegex(ValueError, 'imit'): + asyncio.StreamReader(limit=0, loop=self.loop) + + with self.assertRaisesRegex(ValueError, 'imit'): + asyncio.StreamReader(limit=-1, loop=self.loop) + + def test_read_limit(self): + stream = asyncio.StreamReader(limit=3, loop=self.loop) + stream.feed_data(b'chunk') + data = self.loop.run_until_complete(stream.read(5)) + self.assertEqual(b'chunk', data) + self.assertEqual(b'', stream._buffer) + def test_readline(self): # Read one line. 'readline' will need to wait for the data # to come from 'cb' @@ -292,6 +306,23 @@ ValueError, self.loop.run_until_complete, stream.readline()) self.assertEqual(b'chunk3\n', stream._buffer) + # check strictness of the limit + stream = asyncio.StreamReader(limit=7, loop=self.loop) + stream.feed_data(b'1234567\n') + line = self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'1234567\n', line) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'12345678\n') + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'12345678') + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'', stream._buffer) + def test_readline_nolimit_nowait(self): # All needed data for the first 'readline' call will be # in the buffer. @@ -342,6 +373,92 @@ ValueError, self.loop.run_until_complete, stream.readline()) self.assertEqual(b'', stream._buffer) + def test_readuntil_separator(self): + stream = asyncio.StreamReader(loop=self.loop) + with self.assertRaisesRegex(ValueError, 'Separator should be'): + self.loop.run_until_complete(stream.readuntil(separator=b'')) + + def test_readuntil_multi_chunks(self): + stream = asyncio.StreamReader(loop=self.loop) + + stream.feed_data(b'lineAAA') + data = self.loop.run_until_complete(stream.readuntil(separator=b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'lineAAA') + data = self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'lineAAAxxx') + data = self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'xxx', stream._buffer) + + def test_readuntil_multi_chunks_1(self): + stream = asyncio.StreamReader(loop=self.loop) + + stream.feed_data(b'QWEaa') + stream.feed_data(b'XYaa') + stream.feed_data(b'a') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'QWEaaXYaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'QWEaa') + stream.feed_data(b'XYa') + stream.feed_data(b'aa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'QWEaaXYaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'aaa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'aaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'Xaaa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'Xaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'XXX') + stream.feed_data(b'a') + stream.feed_data(b'a') + stream.feed_data(b'a') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'XXXaaa', data) + self.assertEqual(b'', stream._buffer) + + def test_readuntil_eof(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_data(b'some dataAA') + stream.feed_eof() + + with self.assertRaises(asyncio.IncompleteReadError) as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(cm.exception.partial, b'some dataAA') + self.assertIsNone(cm.exception.expected) + self.assertEqual(b'', stream._buffer) + + def test_readuntil_limit_found_sep(self): + stream = asyncio.StreamReader(loop=self.loop, limit=3) + stream.feed_data(b'some dataAA') + + with self.assertRaisesRegex(asyncio.LimitOverrunError, + 'not found') as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + + self.assertEqual(b'some dataAA', stream._buffer) + + stream.feed_data(b'A') + with self.assertRaisesRegex(asyncio.LimitOverrunError, + 'is found') as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + + self.assertEqual(b'some dataAAA', stream._buffer) + def test_readexactly_zero_or_less(self): # Read exact number of bytes (zero or less). stream = asyncio.StreamReader(loop=self.loop) @@ -372,6 +489,13 @@ self.assertEqual(self.DATA + self.DATA, data) self.assertEqual(self.DATA, stream._buffer) + def test_readexactly_limit(self): + stream = asyncio.StreamReader(limit=3, loop=self.loop) + stream.feed_data(b'chunk') + data = self.loop.run_until_complete(stream.readexactly(5)) + self.assertEqual(b'chunk', data) + self.assertEqual(b'', stream._buffer) + def test_readexactly_eof(self): # Read exact number of bytes (eof). stream = asyncio.StreamReader(loop=self.loop) @@ -657,7 +781,9 @@ @asyncio.coroutine def client(host, port): - reader, writer = yield from asyncio.open_connection(host, port, loop=self.loop) + reader, writer = yield from asyncio.open_connection( + host, port, loop=self.loop) + while True: writer.write(b"foo\n") yield from writer.drain() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -447,6 +447,9 @@ - Add asyncio.timeout() context manager. +- Issue #26050: Add asyncio.StreamReader.readuntil() method. + Patch by ???? ?????????. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 12:33:01 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 11 Jan 2016 17:33:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4_=28Issue_=2326050=29?= Message-ID: <20160111173300.51242.44934@psf.io> https://hg.python.org/cpython/rev/a6288fe5420f changeset: 99846:a6288fe5420f branch: 3.5 parent: 99842:8e0c7c78f532 parent: 99845:db11d0db1598 user: Yury Selivanov date: Mon Jan 11 12:30:56 2016 -0500 summary: Merge 3.4 (Issue #26050) files: Lib/asyncio/streams.py | 221 +++++++++++-- Lib/test/test_asyncio/test_streams.py | 128 ++++++++- Misc/NEWS | 3 + 3 files changed, 315 insertions(+), 37 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -3,6 +3,7 @@ __all__ = ['StreamReader', 'StreamWriter', 'StreamReaderProtocol', 'open_connection', 'start_server', 'IncompleteReadError', + 'LimitOverrunError', ] import socket @@ -27,15 +28,28 @@ Incomplete read error. Attributes: - partial: read bytes string before the end of stream was reached - - expected: total number of expected bytes + - expected: total number of expected bytes (or None if unknown) """ def __init__(self, partial, expected): - EOFError.__init__(self, "%s bytes read on a total of %s expected bytes" - % (len(partial), expected)) + super().__init__("%d bytes read on a total of %r expected bytes" + % (len(partial), expected)) self.partial = partial self.expected = expected +class LimitOverrunError(Exception): + """Reached buffer limit while looking for the separator. + + Attributes: + - message: error message + - consumed: total number of bytes that should be consumed + """ + def __init__(self, message, consumed): + super().__init__(message) + self.message = message + self.consumed = consumed + + @coroutine def open_connection(host=None, port=None, *, loop=None, limit=_DEFAULT_LIMIT, **kwds): @@ -318,6 +332,10 @@ def __init__(self, limit=_DEFAULT_LIMIT, loop=None): # The line length limit is a security feature; # it also doubles as half the buffer limit. + + if limit <= 0: + raise ValueError('Limit cannot be <= 0') + self._limit = limit if loop is None: self._loop = events.get_event_loop() @@ -361,7 +379,7 @@ waiter.set_exception(exc) def _wakeup_waiter(self): - """Wakeup read() or readline() function waiting for data or EOF.""" + """Wakeup read*() functions waiting for data or EOF.""" waiter = self._waiter if waiter is not None: self._waiter = None @@ -409,7 +427,10 @@ @coroutine def _wait_for_data(self, func_name): - """Wait until feed_data() or feed_eof() is called.""" + """Wait until feed_data() or feed_eof() is called. + + If stream was paused, automatically resume it. + """ # StreamReader uses a future to link the protocol feed_data() method # to a read coroutine. Running two read coroutines at the same time # would have an unexpected behaviour. It would not possible to know @@ -418,6 +439,13 @@ raise RuntimeError('%s() called while another coroutine is ' 'already waiting for incoming data' % func_name) + assert not self._eof, '_wait_for_data after EOF' + + # Waiting for data while paused will make deadlock, so prevent it. + if self._paused: + self._paused = False + self._transport.resume_reading() + self._waiter = futures.Future(loop=self._loop) try: yield from self._waiter @@ -426,43 +454,150 @@ @coroutine def readline(self): + """Read chunk of data from the stream until newline (b'\n') is found. + + On success, return chunk that ends with newline. If only partial + line can be read due to EOF, return incomplete line without + terminating newline. When EOF was reached while no bytes read, empty + bytes object is returned. + + If limit is reached, ValueError will be raised. In that case, if + newline was found, complete line including newline will be removed + from internal buffer. Else, internal buffer will be cleared. Limit is + compared against part of the line without newline. + + If stream was paused, this function will automatically resume it if + needed. + """ + sep = b'\n' + seplen = len(sep) + try: + line = yield from self.readuntil(sep) + except IncompleteReadError as e: + return e.partial + except LimitOverrunError as e: + if self._buffer.startswith(sep, e.consumed): + del self._buffer[:e.consumed + seplen] + else: + self._buffer.clear() + self._maybe_resume_transport() + raise ValueError(e.args[0]) + return line + + @coroutine + def readuntil(self, separator=b'\n'): + """Read chunk of data from the stream until `separator` is found. + + On success, chunk and its separator will be removed from internal buffer + (i.e. consumed). Returned chunk will include separator at the end. + + Configured stream limit is used to check result. Limit means maximal + length of chunk that can be returned, not counting the separator. + + If EOF occurs and complete separator still not found, + IncompleteReadError(, None) will be raised and internal + buffer becomes empty. This partial data may contain a partial separator. + + If chunk cannot be read due to overlimit, LimitOverrunError will be raised + and data will be left in internal buffer, so it can be read again, in + some different way. + + If stream was paused, this function will automatically resume it if + needed. + """ + seplen = len(separator) + if seplen == 0: + raise ValueError('Separator should be at least one-byte string') + if self._exception is not None: raise self._exception - line = bytearray() - not_enough = True + # Consume whole buffer except last bytes, which length is + # one less than seplen. Let's check corner cases with + # separator='SEPARATOR': + # * we have received almost complete separator (without last + # byte). i.e buffer='some textSEPARATO'. In this case we + # can safely consume len(separator) - 1 bytes. + # * last byte of buffer is first byte of separator, i.e. + # buffer='abcdefghijklmnopqrS'. We may safely consume + # everything except that last byte, but this require to + # analyze bytes of buffer that match partial separator. + # This is slow and/or require FSM. For this case our + # implementation is not optimal, since require rescanning + # of data that is known to not belong to separator. In + # real world, separator will not be so long to notice + # performance problems. Even when reading MIME-encoded + # messages :) - while not_enough: - while self._buffer and not_enough: - ichar = self._buffer.find(b'\n') - if ichar < 0: - line.extend(self._buffer) - self._buffer.clear() - else: - ichar += 1 - line.extend(self._buffer[:ichar]) - del self._buffer[:ichar] - not_enough = False + # `offset` is the number of bytes from the beginning of the buffer where + # is no occurrence of `separator`. + offset = 0 - if len(line) > self._limit: - self._maybe_resume_transport() - raise ValueError('Line is too long') + # Loop until we find `separator` in the buffer, exceed the buffer size, + # or an EOF has happened. + while True: + buflen = len(self._buffer) + # Check if we now have enough data in the buffer for `separator` to + # fit. + if buflen - offset >= seplen: + isep = self._buffer.find(separator, offset) + + if isep != -1: + # `separator` is in the buffer. `isep` will be used later to + # retrieve the data. + break + + # see upper comment for explanation. + offset = buflen + 1 - seplen + if offset > self._limit: + raise LimitOverrunError('Separator is not found, and chunk exceed the limit', offset) + + # Complete message (with full separator) may be present in buffer + # even when EOF flag is set. This may happen when the last chunk + # adds data which makes separator be found. That's why we check for + # EOF *ater* inspecting the buffer. if self._eof: - break + chunk = bytes(self._buffer) + self._buffer.clear() + raise IncompleteReadError(chunk, None) - if not_enough: - yield from self._wait_for_data('readline') + # _wait_for_data() will resume reading if stream was paused. + yield from self._wait_for_data('readuntil') + if isep > self._limit: + raise LimitOverrunError('Separator is found, but chunk is longer than limit', isep) + + chunk = self._buffer[:isep + seplen] + del self._buffer[:isep + seplen] self._maybe_resume_transport() - return bytes(line) + return bytes(chunk) @coroutine def read(self, n=-1): + """Read up to `n` bytes from the stream. + + If n is not provided, or set to -1, read until EOF and return all read + bytes. If the EOF was received and the internal buffer is empty, return + an empty bytes object. + + If n is zero, return empty bytes object immediatelly. + + If n is positive, this function try to read `n` bytes, and may return + less or equal bytes than requested, but at least one byte. If EOF was + received before any byte is read, this function returns empty byte + object. + + Returned value is not limited with limit, configured at stream creation. + + If stream was paused, this function will automatically resume it if + needed. + """ + if self._exception is not None: raise self._exception - if not n: + if n == 0: return b'' if n < 0: @@ -477,29 +612,41 @@ break blocks.append(block) return b''.join(blocks) - else: - if not self._buffer and not self._eof: - yield from self._wait_for_data('read') - if n < 0 or len(self._buffer) <= n: - data = bytes(self._buffer) - self._buffer.clear() - else: - # n > 0 and len(self._buffer) > n - data = bytes(self._buffer[:n]) - del self._buffer[:n] + if not self._buffer and not self._eof: + yield from self._wait_for_data('read') + + # This will work right even if buffer is less than n bytes + data = bytes(self._buffer[:n]) + del self._buffer[:n] self._maybe_resume_transport() return data @coroutine def readexactly(self, n): + """Read exactly `n` bytes. + + Raise an `IncompleteReadError` if EOF is reached before `n` bytes can be + read. The `IncompleteReadError.partial` attribute of the exception will + contain the partial read bytes. + + if n is zero, return empty bytes object. + + Returned value is not limited with limit, configured at stream creation. + + If stream was paused, this function will automatically resume it if + needed. + """ if n < 0: raise ValueError('readexactly size can not be less than zero') if self._exception is not None: raise self._exception + if n == 0: + return b'' + # There used to be "optimized" code here. It created its own # Future and waited until self._buffer had at least the n # bytes, then called read(n). Unfortunately, this could pause @@ -516,6 +663,8 @@ blocks.append(block) n -= len(block) + assert n == 0 + return b''.join(blocks) if compat.PY35: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -203,6 +203,20 @@ self.assertRaises( ValueError, self.loop.run_until_complete, stream.read(2)) + def test_invalid_limit(self): + with self.assertRaisesRegex(ValueError, 'imit'): + asyncio.StreamReader(limit=0, loop=self.loop) + + with self.assertRaisesRegex(ValueError, 'imit'): + asyncio.StreamReader(limit=-1, loop=self.loop) + + def test_read_limit(self): + stream = asyncio.StreamReader(limit=3, loop=self.loop) + stream.feed_data(b'chunk') + data = self.loop.run_until_complete(stream.read(5)) + self.assertEqual(b'chunk', data) + self.assertEqual(b'', stream._buffer) + def test_readline(self): # Read one line. 'readline' will need to wait for the data # to come from 'cb' @@ -292,6 +306,23 @@ ValueError, self.loop.run_until_complete, stream.readline()) self.assertEqual(b'chunk3\n', stream._buffer) + # check strictness of the limit + stream = asyncio.StreamReader(limit=7, loop=self.loop) + stream.feed_data(b'1234567\n') + line = self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'1234567\n', line) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'12345678\n') + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'12345678') + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'', stream._buffer) + def test_readline_nolimit_nowait(self): # All needed data for the first 'readline' call will be # in the buffer. @@ -342,6 +373,92 @@ ValueError, self.loop.run_until_complete, stream.readline()) self.assertEqual(b'', stream._buffer) + def test_readuntil_separator(self): + stream = asyncio.StreamReader(loop=self.loop) + with self.assertRaisesRegex(ValueError, 'Separator should be'): + self.loop.run_until_complete(stream.readuntil(separator=b'')) + + def test_readuntil_multi_chunks(self): + stream = asyncio.StreamReader(loop=self.loop) + + stream.feed_data(b'lineAAA') + data = self.loop.run_until_complete(stream.readuntil(separator=b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'lineAAA') + data = self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'lineAAAxxx') + data = self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'xxx', stream._buffer) + + def test_readuntil_multi_chunks_1(self): + stream = asyncio.StreamReader(loop=self.loop) + + stream.feed_data(b'QWEaa') + stream.feed_data(b'XYaa') + stream.feed_data(b'a') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'QWEaaXYaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'QWEaa') + stream.feed_data(b'XYa') + stream.feed_data(b'aa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'QWEaaXYaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'aaa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'aaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'Xaaa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'Xaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'XXX') + stream.feed_data(b'a') + stream.feed_data(b'a') + stream.feed_data(b'a') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'XXXaaa', data) + self.assertEqual(b'', stream._buffer) + + def test_readuntil_eof(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_data(b'some dataAA') + stream.feed_eof() + + with self.assertRaises(asyncio.IncompleteReadError) as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(cm.exception.partial, b'some dataAA') + self.assertIsNone(cm.exception.expected) + self.assertEqual(b'', stream._buffer) + + def test_readuntil_limit_found_sep(self): + stream = asyncio.StreamReader(loop=self.loop, limit=3) + stream.feed_data(b'some dataAA') + + with self.assertRaisesRegex(asyncio.LimitOverrunError, + 'not found') as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + + self.assertEqual(b'some dataAA', stream._buffer) + + stream.feed_data(b'A') + with self.assertRaisesRegex(asyncio.LimitOverrunError, + 'is found') as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + + self.assertEqual(b'some dataAAA', stream._buffer) + def test_readexactly_zero_or_less(self): # Read exact number of bytes (zero or less). stream = asyncio.StreamReader(loop=self.loop) @@ -372,6 +489,13 @@ self.assertEqual(self.DATA + self.DATA, data) self.assertEqual(self.DATA, stream._buffer) + def test_readexactly_limit(self): + stream = asyncio.StreamReader(limit=3, loop=self.loop) + stream.feed_data(b'chunk') + data = self.loop.run_until_complete(stream.readexactly(5)) + self.assertEqual(b'chunk', data) + self.assertEqual(b'', stream._buffer) + def test_readexactly_eof(self): # Read exact number of bytes (eof). stream = asyncio.StreamReader(loop=self.loop) @@ -657,7 +781,9 @@ @asyncio.coroutine def client(host, port): - reader, writer = yield from asyncio.open_connection(host, port, loop=self.loop) + reader, writer = yield from asyncio.open_connection( + host, port, loop=self.loop) + while True: writer.write(b"foo\n") yield from writer.drain() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -127,6 +127,9 @@ - Add asyncio.timeout() context manager. +- Issue #26050: Add asyncio.StreamReader.readuntil() method. + Patch by ???? ?????????. + Documentation ------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 12:33:01 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 11 Jan 2016 17:33:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI2MDUw?= =?utf-8?q?=3A_Add_asyncio=2EStreamReader=2Ereaduntil=28=29_method=2E?= Message-ID: <20160111173300.47397.22100@psf.io> https://hg.python.org/cpython/rev/db11d0db1598 changeset: 99845:db11d0db1598 branch: 3.4 parent: 99839:5c13957152d6 user: Yury Selivanov date: Mon Jan 11 12:28:19 2016 -0500 summary: Issue #26050: Add asyncio.StreamReader.readuntil() method. Patch by ???? ?????????. files: Lib/asyncio/streams.py | 221 +++++++++++-- Lib/test/test_asyncio/test_streams.py | 128 ++++++++- Misc/NEWS | 3 + 3 files changed, 315 insertions(+), 37 deletions(-) diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -3,6 +3,7 @@ __all__ = ['StreamReader', 'StreamWriter', 'StreamReaderProtocol', 'open_connection', 'start_server', 'IncompleteReadError', + 'LimitOverrunError', ] import socket @@ -27,15 +28,28 @@ Incomplete read error. Attributes: - partial: read bytes string before the end of stream was reached - - expected: total number of expected bytes + - expected: total number of expected bytes (or None if unknown) """ def __init__(self, partial, expected): - EOFError.__init__(self, "%s bytes read on a total of %s expected bytes" - % (len(partial), expected)) + super().__init__("%d bytes read on a total of %r expected bytes" + % (len(partial), expected)) self.partial = partial self.expected = expected +class LimitOverrunError(Exception): + """Reached buffer limit while looking for the separator. + + Attributes: + - message: error message + - consumed: total number of bytes that should be consumed + """ + def __init__(self, message, consumed): + super().__init__(message) + self.message = message + self.consumed = consumed + + @coroutine def open_connection(host=None, port=None, *, loop=None, limit=_DEFAULT_LIMIT, **kwds): @@ -318,6 +332,10 @@ def __init__(self, limit=_DEFAULT_LIMIT, loop=None): # The line length limit is a security feature; # it also doubles as half the buffer limit. + + if limit <= 0: + raise ValueError('Limit cannot be <= 0') + self._limit = limit if loop is None: self._loop = events.get_event_loop() @@ -361,7 +379,7 @@ waiter.set_exception(exc) def _wakeup_waiter(self): - """Wakeup read() or readline() function waiting for data or EOF.""" + """Wakeup read*() functions waiting for data or EOF.""" waiter = self._waiter if waiter is not None: self._waiter = None @@ -409,7 +427,10 @@ @coroutine def _wait_for_data(self, func_name): - """Wait until feed_data() or feed_eof() is called.""" + """Wait until feed_data() or feed_eof() is called. + + If stream was paused, automatically resume it. + """ # StreamReader uses a future to link the protocol feed_data() method # to a read coroutine. Running two read coroutines at the same time # would have an unexpected behaviour. It would not possible to know @@ -418,6 +439,13 @@ raise RuntimeError('%s() called while another coroutine is ' 'already waiting for incoming data' % func_name) + assert not self._eof, '_wait_for_data after EOF' + + # Waiting for data while paused will make deadlock, so prevent it. + if self._paused: + self._paused = False + self._transport.resume_reading() + self._waiter = futures.Future(loop=self._loop) try: yield from self._waiter @@ -426,43 +454,150 @@ @coroutine def readline(self): + """Read chunk of data from the stream until newline (b'\n') is found. + + On success, return chunk that ends with newline. If only partial + line can be read due to EOF, return incomplete line without + terminating newline. When EOF was reached while no bytes read, empty + bytes object is returned. + + If limit is reached, ValueError will be raised. In that case, if + newline was found, complete line including newline will be removed + from internal buffer. Else, internal buffer will be cleared. Limit is + compared against part of the line without newline. + + If stream was paused, this function will automatically resume it if + needed. + """ + sep = b'\n' + seplen = len(sep) + try: + line = yield from self.readuntil(sep) + except IncompleteReadError as e: + return e.partial + except LimitOverrunError as e: + if self._buffer.startswith(sep, e.consumed): + del self._buffer[:e.consumed + seplen] + else: + self._buffer.clear() + self._maybe_resume_transport() + raise ValueError(e.args[0]) + return line + + @coroutine + def readuntil(self, separator=b'\n'): + """Read chunk of data from the stream until `separator` is found. + + On success, chunk and its separator will be removed from internal buffer + (i.e. consumed). Returned chunk will include separator at the end. + + Configured stream limit is used to check result. Limit means maximal + length of chunk that can be returned, not counting the separator. + + If EOF occurs and complete separator still not found, + IncompleteReadError(, None) will be raised and internal + buffer becomes empty. This partial data may contain a partial separator. + + If chunk cannot be read due to overlimit, LimitOverrunError will be raised + and data will be left in internal buffer, so it can be read again, in + some different way. + + If stream was paused, this function will automatically resume it if + needed. + """ + seplen = len(separator) + if seplen == 0: + raise ValueError('Separator should be at least one-byte string') + if self._exception is not None: raise self._exception - line = bytearray() - not_enough = True + # Consume whole buffer except last bytes, which length is + # one less than seplen. Let's check corner cases with + # separator='SEPARATOR': + # * we have received almost complete separator (without last + # byte). i.e buffer='some textSEPARATO'. In this case we + # can safely consume len(separator) - 1 bytes. + # * last byte of buffer is first byte of separator, i.e. + # buffer='abcdefghijklmnopqrS'. We may safely consume + # everything except that last byte, but this require to + # analyze bytes of buffer that match partial separator. + # This is slow and/or require FSM. For this case our + # implementation is not optimal, since require rescanning + # of data that is known to not belong to separator. In + # real world, separator will not be so long to notice + # performance problems. Even when reading MIME-encoded + # messages :) - while not_enough: - while self._buffer and not_enough: - ichar = self._buffer.find(b'\n') - if ichar < 0: - line.extend(self._buffer) - self._buffer.clear() - else: - ichar += 1 - line.extend(self._buffer[:ichar]) - del self._buffer[:ichar] - not_enough = False + # `offset` is the number of bytes from the beginning of the buffer where + # is no occurrence of `separator`. + offset = 0 - if len(line) > self._limit: - self._maybe_resume_transport() - raise ValueError('Line is too long') + # Loop until we find `separator` in the buffer, exceed the buffer size, + # or an EOF has happened. + while True: + buflen = len(self._buffer) + # Check if we now have enough data in the buffer for `separator` to + # fit. + if buflen - offset >= seplen: + isep = self._buffer.find(separator, offset) + + if isep != -1: + # `separator` is in the buffer. `isep` will be used later to + # retrieve the data. + break + + # see upper comment for explanation. + offset = buflen + 1 - seplen + if offset > self._limit: + raise LimitOverrunError('Separator is not found, and chunk exceed the limit', offset) + + # Complete message (with full separator) may be present in buffer + # even when EOF flag is set. This may happen when the last chunk + # adds data which makes separator be found. That's why we check for + # EOF *ater* inspecting the buffer. if self._eof: - break + chunk = bytes(self._buffer) + self._buffer.clear() + raise IncompleteReadError(chunk, None) - if not_enough: - yield from self._wait_for_data('readline') + # _wait_for_data() will resume reading if stream was paused. + yield from self._wait_for_data('readuntil') + if isep > self._limit: + raise LimitOverrunError('Separator is found, but chunk is longer than limit', isep) + + chunk = self._buffer[:isep + seplen] + del self._buffer[:isep + seplen] self._maybe_resume_transport() - return bytes(line) + return bytes(chunk) @coroutine def read(self, n=-1): + """Read up to `n` bytes from the stream. + + If n is not provided, or set to -1, read until EOF and return all read + bytes. If the EOF was received and the internal buffer is empty, return + an empty bytes object. + + If n is zero, return empty bytes object immediatelly. + + If n is positive, this function try to read `n` bytes, and may return + less or equal bytes than requested, but at least one byte. If EOF was + received before any byte is read, this function returns empty byte + object. + + Returned value is not limited with limit, configured at stream creation. + + If stream was paused, this function will automatically resume it if + needed. + """ + if self._exception is not None: raise self._exception - if not n: + if n == 0: return b'' if n < 0: @@ -477,29 +612,41 @@ break blocks.append(block) return b''.join(blocks) - else: - if not self._buffer and not self._eof: - yield from self._wait_for_data('read') - if n < 0 or len(self._buffer) <= n: - data = bytes(self._buffer) - self._buffer.clear() - else: - # n > 0 and len(self._buffer) > n - data = bytes(self._buffer[:n]) - del self._buffer[:n] + if not self._buffer and not self._eof: + yield from self._wait_for_data('read') + + # This will work right even if buffer is less than n bytes + data = bytes(self._buffer[:n]) + del self._buffer[:n] self._maybe_resume_transport() return data @coroutine def readexactly(self, n): + """Read exactly `n` bytes. + + Raise an `IncompleteReadError` if EOF is reached before `n` bytes can be + read. The `IncompleteReadError.partial` attribute of the exception will + contain the partial read bytes. + + if n is zero, return empty bytes object. + + Returned value is not limited with limit, configured at stream creation. + + If stream was paused, this function will automatically resume it if + needed. + """ if n < 0: raise ValueError('readexactly size can not be less than zero') if self._exception is not None: raise self._exception + if n == 0: + return b'' + # There used to be "optimized" code here. It created its own # Future and waited until self._buffer had at least the n # bytes, then called read(n). Unfortunately, this could pause @@ -516,6 +663,8 @@ blocks.append(block) n -= len(block) + assert n == 0 + return b''.join(blocks) if compat.PY35: diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -203,6 +203,20 @@ self.assertRaises( ValueError, self.loop.run_until_complete, stream.read(2)) + def test_invalid_limit(self): + with self.assertRaisesRegex(ValueError, 'imit'): + asyncio.StreamReader(limit=0, loop=self.loop) + + with self.assertRaisesRegex(ValueError, 'imit'): + asyncio.StreamReader(limit=-1, loop=self.loop) + + def test_read_limit(self): + stream = asyncio.StreamReader(limit=3, loop=self.loop) + stream.feed_data(b'chunk') + data = self.loop.run_until_complete(stream.read(5)) + self.assertEqual(b'chunk', data) + self.assertEqual(b'', stream._buffer) + def test_readline(self): # Read one line. 'readline' will need to wait for the data # to come from 'cb' @@ -292,6 +306,23 @@ ValueError, self.loop.run_until_complete, stream.readline()) self.assertEqual(b'chunk3\n', stream._buffer) + # check strictness of the limit + stream = asyncio.StreamReader(limit=7, loop=self.loop) + stream.feed_data(b'1234567\n') + line = self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'1234567\n', line) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'12345678\n') + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'12345678') + with self.assertRaises(ValueError) as cm: + self.loop.run_until_complete(stream.readline()) + self.assertEqual(b'', stream._buffer) + def test_readline_nolimit_nowait(self): # All needed data for the first 'readline' call will be # in the buffer. @@ -342,6 +373,92 @@ ValueError, self.loop.run_until_complete, stream.readline()) self.assertEqual(b'', stream._buffer) + def test_readuntil_separator(self): + stream = asyncio.StreamReader(loop=self.loop) + with self.assertRaisesRegex(ValueError, 'Separator should be'): + self.loop.run_until_complete(stream.readuntil(separator=b'')) + + def test_readuntil_multi_chunks(self): + stream = asyncio.StreamReader(loop=self.loop) + + stream.feed_data(b'lineAAA') + data = self.loop.run_until_complete(stream.readuntil(separator=b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'lineAAA') + data = self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'lineAAAxxx') + data = self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(b'lineAAA', data) + self.assertEqual(b'xxx', stream._buffer) + + def test_readuntil_multi_chunks_1(self): + stream = asyncio.StreamReader(loop=self.loop) + + stream.feed_data(b'QWEaa') + stream.feed_data(b'XYaa') + stream.feed_data(b'a') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'QWEaaXYaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'QWEaa') + stream.feed_data(b'XYa') + stream.feed_data(b'aa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'QWEaaXYaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'aaa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'aaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'Xaaa') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'Xaaa', data) + self.assertEqual(b'', stream._buffer) + + stream.feed_data(b'XXX') + stream.feed_data(b'a') + stream.feed_data(b'a') + stream.feed_data(b'a') + data = self.loop.run_until_complete(stream.readuntil(b'aaa')) + self.assertEqual(b'XXXaaa', data) + self.assertEqual(b'', stream._buffer) + + def test_readuntil_eof(self): + stream = asyncio.StreamReader(loop=self.loop) + stream.feed_data(b'some dataAA') + stream.feed_eof() + + with self.assertRaises(asyncio.IncompleteReadError) as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + self.assertEqual(cm.exception.partial, b'some dataAA') + self.assertIsNone(cm.exception.expected) + self.assertEqual(b'', stream._buffer) + + def test_readuntil_limit_found_sep(self): + stream = asyncio.StreamReader(loop=self.loop, limit=3) + stream.feed_data(b'some dataAA') + + with self.assertRaisesRegex(asyncio.LimitOverrunError, + 'not found') as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + + self.assertEqual(b'some dataAA', stream._buffer) + + stream.feed_data(b'A') + with self.assertRaisesRegex(asyncio.LimitOverrunError, + 'is found') as cm: + self.loop.run_until_complete(stream.readuntil(b'AAA')) + + self.assertEqual(b'some dataAAA', stream._buffer) + def test_readexactly_zero_or_less(self): # Read exact number of bytes (zero or less). stream = asyncio.StreamReader(loop=self.loop) @@ -372,6 +489,13 @@ self.assertEqual(self.DATA + self.DATA, data) self.assertEqual(self.DATA, stream._buffer) + def test_readexactly_limit(self): + stream = asyncio.StreamReader(limit=3, loop=self.loop) + stream.feed_data(b'chunk') + data = self.loop.run_until_complete(stream.readexactly(5)) + self.assertEqual(b'chunk', data) + self.assertEqual(b'', stream._buffer) + def test_readexactly_eof(self): # Read exact number of bytes (eof). stream = asyncio.StreamReader(loop=self.loop) @@ -657,7 +781,9 @@ @asyncio.coroutine def client(host, port): - reader, writer = yield from asyncio.open_connection(host, port, loop=self.loop) + reader, writer = yield from asyncio.open_connection( + host, port, loop=self.loop) + while True: writer.write(b"foo\n") yield from writer.drain() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,6 +30,9 @@ - Add asyncio.timeout() context manager. +- Issue #26050: Add asyncio.StreamReader.readuntil() method. + Patch by ???? ?????????. + What's New in Python 3.4.4? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 12:37:08 2016 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 11 Jan 2016 17:37:08 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_PEP_484_with_recommend?= =?utf-8?q?ed_Python_2_alternative=2E?= Message-ID: <20160111173707.47417.25750@psf.io> https://hg.python.org/peps/rev/06f8470390c2 changeset: 6166:06f8470390c2 user: Guido van Rossum date: Mon Jan 11 09:36:24 2016 -0800 summary: Update PEP 484 with recommended Python 2 alternative. files: pep-0484.txt | 46 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diff --git a/pep-0484.txt b/pep-0484.txt --- a/pep-0484.txt +++ b/pep-0484.txt @@ -1372,6 +1372,52 @@ results (generic over ``AnyStr``) +Suggested syntax for Python 2.7 and straddling code +=================================================== + +Some tools may want to support type annotations in code that must be +compatible with Python 2.7. For this purpose this PEP has a suggested +(but not mandatory) extension where function annotations are placed in +a ``# type:`` comment. Such a comment must be placed immediately +following the function header (before the docstring). An example: the +following Python 3 code:: + + def embezzle(self, account: str, funds: int = 1000000, *fake_receipts: str) -> None: + """Embezzle funds from account using fake receipts.""" + + +is equivalent to the following:: + + def embezzle(self, account, funds=1000000, *fake_receipts): + # type: (str, int, *str) -> None + """Embezzle funds from account using fake receipts.""" + + +Notes: + +- Tools that support this syntax should support it regardless of the + Python version being checked. This is necessary in order to support + code that straddles Python 2 and Python 3. + +- Every argument must be accounted for, except the first argument of + instance and class methods. + +- The return type is mandatory. If in Python 3 you would omit some + argument or the return type, the Python 2 notation should use + ``Any``. + +- For ``*args`` and ``**kwds``, put 1 or 2 stars in front of the + corresponding type annotation. (As with Python 3 annotations, the + annotation here denotes the type of the individual argument values, + not of the tuple/dict that you receive as the special argument value + ``args`` or ``kwds``.) + +- Like other type comments, any names used in the annotations must be + imported or defined by the module containing the annotation. + +- The entire annotation must be one line. + + Rejected Alternatives ===================== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 13:49:52 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 18:49:52 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIyOTk1?= =?utf-8?q?=3A_=5BUPDATE=5D_Comment_out_the_one_of_the_pickleability_tests?= =?utf-8?q?_in?= Message-ID: <20160111184946.59568.82853@psf.io> https://hg.python.org/cpython/rev/e79eddcdff63 changeset: 99848:e79eddcdff63 branch: 3.5 parent: 99846:a6288fe5420f user: Barry Warsaw date: Mon Jan 11 13:24:02 2016 -0500 summary: Issue #22995: [UPDATE] Comment out the one of the pickleability tests in _PyObject_GetState() due to regressions observed in Cython-based projects. files: Misc/NEWS | 3 +++ Objects/typeobject.c | 9 +++++++++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in + _PyObject_GetState() due to regressions observed in Cython-based projects. + - Issue #25961: Disallowed null characters in the type name. - Issue #25973: Fix segfault when an invalid nonlocal statement binds a name diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3866,6 +3866,14 @@ } assert(slotnames == Py_None || PyList_Check(slotnames)); +#if 0 + /* 2016-01-11 barry - This clause breaks at least three packages which + rely on Cython: kivy, pysam, and s3ql. Cython may be doing + something funny under the hood, but as this is clearly a regression + and the rationale for this prohibition is suspect, I am commenting + this out. Perhaps it should just be removed. See issue #22995 for + details. + */ if (required) { Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; if (obj->ob_type->tp_dictoffset) @@ -3883,6 +3891,7 @@ return NULL; } } +#endif if (slotnames != Py_None && Py_SIZE(slotnames) > 0) { PyObject *slots; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 13:49:53 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 18:49:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2322995=3A_=5BUPDATE=5D_Comment_out_the_one_of_th?= =?utf-8?q?e_pickleability_tests_in?= Message-ID: <20160111184946.70883.59373@psf.io> https://hg.python.org/cpython/rev/927fd0e14d49 changeset: 99849:927fd0e14d49 parent: 99847:c2cd3b435afe parent: 99848:e79eddcdff63 user: Barry Warsaw date: Mon Jan 11 13:49:40 2016 -0500 summary: Issue #22995: [UPDATE] Comment out the one of the pickleability tests in _PyObject_GetState() due to regressions observed in Cython-based projects. files: Misc/NEWS | 3 +++ Objects/typeobject.c | 9 +++++++++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in + _PyObject_GetState() due to regressions observed in Cython-based projects. + - Issue #25961: Disallowed null characters in the type name. - Issue #25973: Fix segfault when an invalid nonlocal statement binds a name diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3872,6 +3872,14 @@ } assert(slotnames == Py_None || PyList_Check(slotnames)); +#if 0 + /* 2016-01-11 barry - This clause breaks at least three packages which + rely on Cython: kivy, pysam, and s3ql. Cython may be doing + something funny under the hood, but as this is clearly a regression + and the rationale for this prohibition is suspect, I am commenting + this out. Perhaps it should just be removed. See issue #22995 for + details. + */ if (required) { Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; if (obj->ob_type->tp_dictoffset) @@ -3889,6 +3897,7 @@ return NULL; } } +#endif if (slotnames != Py_None && Py_SIZE(slotnames) > 0) { PyObject *slots; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 14:45:04 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 19:45:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIyOTk1?= =?utf-8?q?=3A_=5BUPDATE=5D_Comment_out_the_one_of_the_pickleability_tests?= =?utf-8?q?_in?= Message-ID: <20160111194504.11399.8572@psf.io> https://hg.python.org/cpython/rev/9d3ac16b78dc changeset: 99850:9d3ac16b78dc branch: 2.7 parent: 99825:84fdbdd483a3 user: Barry Warsaw date: Mon Jan 11 13:52:23 2016 -0500 summary: Issue #22995: [UPDATE] Comment out the one of the pickleability tests in _PyObject_GetState() due to regressions observed in Cython-based projects. files: Misc/NEWS | 3 +++ Objects/typeobject.c | 9 +++++++++ 2 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in + _PyObject_GetState() due to regressions observed in Cython-based projects. + - Issue #25961: Disallowed null characters in the type name. - Issue #22995: Instances of extension types with a state that aren't diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3279,6 +3279,14 @@ if (names == NULL) goto end; assert(names == Py_None || PyList_Check(names)); +#if 0 + /* 2016-01-11 barry - This clause breaks at least three packages which + rely on Cython: kivy, pysam, and s3ql. Cython may be doing + something funny under the hood, but as this is clearly a regression + and the rationale for this prohibition is suspect, I am commenting + this out. Perhaps it should just be removed. See issue #22995 for + details. + */ if (required_state) { Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; if (obj->ob_type->tp_dictoffset) @@ -3294,6 +3302,7 @@ goto end; } } +#endif if (names != Py_None) { slots = PyDict_New(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 14:45:04 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 19:45:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Comment_out_tw?= =?utf-8?q?o_tests_that_won=27t_pass_now_after_reverting_the_typeobject=2E?= =?utf-8?q?c?= Message-ID: <20160111194504.51234.73801@psf.io> https://hg.python.org/cpython/rev/6081db594907 changeset: 99851:6081db594907 branch: 2.7 user: Barry Warsaw date: Mon Jan 11 14:44:59 2016 -0500 summary: Comment out two tests that won't pass now after reverting the typeobject.c change. Also, as per further discussion, we'll just remove the regressing code in typeobject.c files: Lib/test/test_buffer.py | 11 ++++++----- Lib/test/test_memoryview.py | 11 ++++++----- Misc/NEWS | 2 +- Objects/typeobject.c | 24 ------------------------ 4 files changed, 13 insertions(+), 35 deletions(-) diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py --- a/Lib/test/test_buffer.py +++ b/Lib/test/test_buffer.py @@ -42,11 +42,12 @@ with self.assertRaises(TypeError): copy.copy(buf) - def test_pickle(self): - buf = buffer(b'abc') - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - with self.assertRaises(TypeError): - pickle.dumps(buf, proto) + # See issue #22995 + ## def test_pickle(self): + ## buf = buffer(b'abc') + ## for proto in range(pickle.HIGHEST_PROTOCOL + 1): + ## with self.assertRaises(TypeError): + ## pickle.dumps(buf, proto) def test_main(): diff --git a/Lib/test/test_memoryview.py b/Lib/test/test_memoryview.py --- a/Lib/test/test_memoryview.py +++ b/Lib/test/test_memoryview.py @@ -362,11 +362,12 @@ with self.assertRaises(TypeError): copy.copy(m) - def test_pickle(self): - m = memoryview(b'abc') - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - with self.assertRaises(TypeError): - pickle.dumps(m, proto) + # See issue #22995 + ## def test_pickle(self): + ## m = memoryview(b'abc') + ## for proto in range(pickle.HIGHEST_PROTOCOL + 1): + ## with self.assertRaises(TypeError): + ## pickle.dumps(m, proto) def test_main(): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,7 +10,7 @@ Core and Builtins ----------------- -- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in +- Issue #22995: [UPDATE] Remove the one of the pickleability tests in _PyObject_GetState() due to regressions observed in Cython-based projects. - Issue #25961: Disallowed null characters in the type name. diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3279,30 +3279,6 @@ if (names == NULL) goto end; assert(names == Py_None || PyList_Check(names)); -#if 0 - /* 2016-01-11 barry - This clause breaks at least three packages which - rely on Cython: kivy, pysam, and s3ql. Cython may be doing - something funny under the hood, but as this is clearly a regression - and the rationale for this prohibition is suspect, I am commenting - this out. Perhaps it should just be removed. See issue #22995 for - details. - */ - if (required_state) { - Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; - if (obj->ob_type->tp_dictoffset) - basicsize += sizeof(PyObject *); - if (obj->ob_type->tp_weaklistoffset) - basicsize += sizeof(PyObject *); - if (names != Py_None) - basicsize += sizeof(PyObject *) * Py_SIZE(names); - if (obj->ob_type->tp_basicsize > basicsize) { - PyErr_Format(PyExc_TypeError, - "can't pickle %.200s objects", - Py_TYPE(obj)->tp_name); - goto end; - } - } -#endif if (names != Py_None) { slots = PyDict_New(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 14:49:43 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 19:49:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Comment_out_an?= =?utf-8?q?other_test_that_won=27t_pass_after_reverting_the_picklability?= Message-ID: <20160111194940.69161.49852@psf.io> https://hg.python.org/cpython/rev/609226a23086 changeset: 99852:609226a23086 branch: 2.7 user: Barry Warsaw date: Mon Jan 11 14:49:34 2016 -0500 summary: Comment out another test that won't pass after reverting the picklability regression. files: Lib/test/test_csv.py | 19 ++++++++++--------- 1 files changed, 10 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -468,16 +468,17 @@ self.assertRaises(TypeError, csv.reader, [], quoting = -1) self.assertRaises(TypeError, csv.reader, [], quoting = 100) - def test_copy(self): - for name in csv.list_dialects(): - dialect = csv.get_dialect(name) - self.assertRaises(TypeError, copy.copy, dialect) + # See issue #22995 + ## def test_copy(self): + ## for name in csv.list_dialects(): + ## dialect = csv.get_dialect(name) + ## self.assertRaises(TypeError, copy.copy, dialect) - def test_pickle(self): - for name in csv.list_dialects(): - dialect = csv.get_dialect(name) - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - self.assertRaises(TypeError, pickle.dumps, dialect, proto) + ## def test_pickle(self): + ## for name in csv.list_dialects(): + ## dialect = csv.get_dialect(name) + ## for proto in range(pickle.HIGHEST_PROTOCOL + 1): + ## self.assertRaises(TypeError, pickle.dumps, dialect, proto) class TestCsvBase(unittest.TestCase): def readerAssertEqual(self, input, expected_result): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:16:05 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 11 Jan 2016 20:16:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325486=3A_Resurrec?= =?utf-8?q?t_inspect=2Egetargspec_in_3=2E6=2E_Backout_a565aad5d6e1=2E?= Message-ID: <20160111201602.59566.44833@psf.io> https://hg.python.org/cpython/rev/32c8bdcd66cc changeset: 99853:32c8bdcd66cc parent: 99849:927fd0e14d49 user: Yury Selivanov date: Mon Jan 11 15:15:01 2016 -0500 summary: Issue #25486: Resurrect inspect.getargspec in 3.6. Backout a565aad5d6e1. The decision is that we shouldn't remove popular APIs (however long they are depreacted) from Python 3, while 2.7 is still around and supported. files: Doc/library/inspect.rst | 18 ++++++++++++ Doc/whatsnew/3.6.rst | 3 -- Lib/inspect.py | 25 +++++++++++++++++ Lib/test/test_inspect.py | 40 ++++++++++++++++++++++++--- Misc/NEWS | 3 +- 5 files changed, 79 insertions(+), 10 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -796,6 +796,24 @@ classes using multiple inheritance and their descendants will appear multiple times. + +.. function:: getargspec(func) + + Get the names and default values of a Python function's arguments. A + :term:`named tuple` ``ArgSpec(args, varargs, keywords, defaults)`` is + returned. *args* is a list of the argument names. *varargs* and *keywords* + are the names of the ``*`` and ``**`` arguments or ``None``. *defaults* is a + tuple of default argument values or ``None`` if there are no default + arguments; if this tuple has *n* elements, they correspond to the last + *n* elements listed in *args*. + + .. deprecated:: 3.0 + Use :func:`signature` and + :ref:`Signature Object `, which provide a + better introspecting API for callables. This function will be removed + in Python 3.6. + + .. function:: getfullargspec(func) Get the names and default values of a Python function's arguments. A diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -218,9 +218,6 @@ API and Feature Removals ------------------------ -* ``inspect.getargspec()`` was removed (was deprecated since CPython 3.0). - :func:`inspect.getfullargspec` is an almost drop in replacement. - * ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3). :func:`inspect.getmodulename` should be used for obtaining the module name for a given path. diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -1004,6 +1004,31 @@ varkw = co.co_varnames[nargs] return args, varargs, kwonlyargs, varkw + +ArgSpec = namedtuple('ArgSpec', 'args varargs keywords defaults') + +def getargspec(func): + """Get the names and default values of a function's arguments. + + A tuple of four things is returned: (args, varargs, keywords, defaults). + 'args' is a list of the argument names, including keyword-only argument names. + 'varargs' and 'keywords' are the names of the * and ** arguments or None. + 'defaults' is an n-tuple of the default values of the last n arguments. + + Use the getfullargspec() API for Python 3 code, as annotations + and keyword arguments are supported. getargspec() will raise ValueError + if the func has either annotations or keyword arguments. + """ + warnings.warn("inspect.getargspec() is deprecated, " + "use inspect.signature() instead", DeprecationWarning, + stacklevel=2) + args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, ann = \ + getfullargspec(func) + if kwonlyargs or ann: + raise ValueError("Function has keyword-only arguments or annotations" + ", use getfullargspec() API which can support them") + return ArgSpec(args, varargs, varkw, defaults) + FullArgSpec = namedtuple('FullArgSpec', 'args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, annotations') diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -628,6 +628,18 @@ got = inspect.getmro(D) self.assertEqual(expected, got) + def assertArgSpecEquals(self, routine, args_e, varargs_e=None, + varkw_e=None, defaults_e=None, formatted=None): + with self.assertWarns(DeprecationWarning): + args, varargs, varkw, defaults = inspect.getargspec(routine) + self.assertEqual(args, args_e) + self.assertEqual(varargs, varargs_e) + self.assertEqual(varkw, varkw_e) + self.assertEqual(defaults, defaults_e) + if formatted is not None: + self.assertEqual(inspect.formatargspec(args, varargs, varkw, defaults), + formatted) + def assertFullArgSpecEquals(self, routine, args_e, varargs_e=None, varkw_e=None, defaults_e=None, kwonlyargs_e=[], kwonlydefaults_e=None, @@ -646,6 +658,23 @@ kwonlyargs, kwonlydefaults, ann), formatted) + def test_getargspec(self): + self.assertArgSpecEquals(mod.eggs, ['x', 'y'], formatted='(x, y)') + + self.assertArgSpecEquals(mod.spam, + ['a', 'b', 'c', 'd', 'e', 'f'], + 'g', 'h', (3, 4, 5), + '(a, b, c, d=3, e=4, f=5, *g, **h)') + + self.assertRaises(ValueError, self.assertArgSpecEquals, + mod2.keyworded, []) + + self.assertRaises(ValueError, self.assertArgSpecEquals, + mod2.annotated, []) + self.assertRaises(ValueError, self.assertArgSpecEquals, + mod2.keyword_only_arg, []) + + def test_getfullargspec(self): self.assertFullArgSpecEquals(mod2.keyworded, [], varargs_e='arg1', kwonlyargs_e=['arg2'], @@ -659,19 +688,20 @@ kwonlyargs_e=['arg'], formatted='(*, arg)') - def test_fullargspec_api_ignores_wrapped(self): + def test_argspec_api_ignores_wrapped(self): # Issue 20684: low level introspection API must ignore __wrapped__ @functools.wraps(mod.spam) def ham(x, y): pass # Basic check + self.assertArgSpecEquals(ham, ['x', 'y'], formatted='(x, y)') self.assertFullArgSpecEquals(ham, ['x', 'y'], formatted='(x, y)') self.assertFullArgSpecEquals(functools.partial(ham), ['x', 'y'], formatted='(x, y)') # Other variants def check_method(f): - self.assertFullArgSpecEquals(f, ['self', 'x', 'y'], - formatted='(self, x, y)') + self.assertArgSpecEquals(f, ['self', 'x', 'y'], + formatted='(self, x, y)') class C: @functools.wraps(mod.spam) def ham(self, x, y): @@ -749,11 +779,11 @@ with self.assertRaises(TypeError): inspect.getfullargspec(builtin) - def test_getfullargspec_method(self): + def test_getargspec_method(self): class A(object): def m(self): pass - self.assertFullArgSpecEquals(A.m, ['self']) + self.assertArgSpecEquals(A.m, ['self']) def test_classify_newstyle(self): class A(object): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -429,8 +429,7 @@ - Issue #23661: unittest.mock side_effects can now be exceptions again. This was a regression vs Python 3.4. Patch from Ignacio Rossi -- Issue #13248: Remove deprecated inspect.getargspec and inspect.getmoduleinfo - functions. +- Issue #13248: Remove deprecated inspect.getmoduleinfo function. - Issue #25578: Fix (another) memory leak in SSLSocket.getpeercer(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:43:35 2016 From: python-checkins at python.org (matthias.klose) Date: Mon, 11 Jan 2016 20:43:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogLSBJc3N1ZSAjMjQ3?= =?utf-8?q?05=3A_Fix_sysconfig=2E=5Fparse=5Fmakefile_not_expanding_=24=7B?= =?utf-8?q?=7D_vars?= Message-ID: <20160111204335.70885.45252@psf.io> https://hg.python.org/cpython/rev/ef84d21f5292 changeset: 99854:ef84d21f5292 branch: 3.5 parent: 99848:e79eddcdff63 user: doko at ubuntu.com date: Mon Jan 11 21:41:40 2016 +0100 summary: - Issue #24705: Fix sysconfig._parse_makefile not expanding ${} vars appearing before $() vars. files: Lib/sysconfig.py | 7 ++++++- Misc/NEWS | 3 +++ 2 files changed, 9 insertions(+), 1 deletions(-) diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -260,7 +260,12 @@ while len(variables) > 0: for name in tuple(variables): value = notdone[name] - m = _findvar1_rx.search(value) or _findvar2_rx.search(value) + m1 = _findvar1_rx.search(value) + m2 = _findvar2_rx.search(value) + if m1 and m2: + m = m1 if m1.start() < m2.start() else m2 + else: + m = m1 if m1 else m2 if m is not None: n = m.group(1) found = True diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,9 @@ Library ------- +- Issue #24705: Fix sysconfig._parse_makefile not expanding ${} vars + appearing before $() vars. + - Issue #22138: Fix mock.patch behavior when patching descriptors. Restore original values after patching. Patch contributed by Sean McCully. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:43:35 2016 From: python-checkins at python.org (matthias.klose) Date: Mon, 11 Jan 2016 20:43:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160111204335.11407.57616@psf.io> https://hg.python.org/cpython/rev/2b0ff9e16955 changeset: 99855:2b0ff9e16955 parent: 99853:32c8bdcd66cc parent: 99854:ef84d21f5292 user: doko at ubuntu.com date: Mon Jan 11 21:43:29 2016 +0100 summary: Merge 3.5 files: Lib/sysconfig.py | 7 ++++++- Misc/NEWS | 3 +++ 2 files changed, 9 insertions(+), 1 deletions(-) diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -260,7 +260,12 @@ while len(variables) > 0: for name in tuple(variables): value = notdone[name] - m = _findvar1_rx.search(value) or _findvar2_rx.search(value) + m1 = _findvar1_rx.search(value) + m2 = _findvar2_rx.search(value) + if m1 and m2: + m = m1 if m1.start() < m2.start() else m2 + else: + m = m1 if m1 else m2 if m is not None: n = m.group(1) found = True diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,9 @@ Library ------- +- Issue #24705: Fix sysconfig._parse_makefile not expanding ${} vars + appearing before $() vars. + - Issue #26069: Remove the deprecated apis in the trace module. - Issue #22138: Fix mock.patch behavior when patching descriptors. Restore -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:53:57 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 20:53:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Comment_out_so?= =?utf-8?q?me_tests_that_won=27t_pass_now_that_we=27ve_reverted_the?= Message-ID: <20160111205356.11407.68316@psf.io> https://hg.python.org/cpython/rev/54dd5c105334 changeset: 99856:54dd5c105334 branch: 3.5 parent: 99848:e79eddcdff63 user: Barry Warsaw date: Mon Jan 11 15:14:53 2016 -0500 summary: Comment out some tests that won't pass now that we've reverted the picklability regression. Also, as per further discussion, remove the regressing code. files: Lib/test/test_csv.py | 19 ++++++++++--------- Objects/typeobject.c | 27 --------------------------- 2 files changed, 10 insertions(+), 36 deletions(-) diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -426,16 +426,17 @@ self.assertRaises(TypeError, csv.reader, [], quoting = -1) self.assertRaises(TypeError, csv.reader, [], quoting = 100) - def test_copy(self): - for name in csv.list_dialects(): - dialect = csv.get_dialect(name) - self.assertRaises(TypeError, copy.copy, dialect) + # See issue #22995 + ## def test_copy(self): + ## for name in csv.list_dialects(): + ## dialect = csv.get_dialect(name) + ## self.assertRaises(TypeError, copy.copy, dialect) - def test_pickle(self): - for name in csv.list_dialects(): - dialect = csv.get_dialect(name) - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - self.assertRaises(TypeError, pickle.dumps, dialect, proto) + ## def test_pickle(self): + ## for name in csv.list_dialects(): + ## dialect = csv.get_dialect(name) + ## for proto in range(pickle.HIGHEST_PROTOCOL + 1): + ## self.assertRaises(TypeError, pickle.dumps, dialect, proto) class TestCsvBase(unittest.TestCase): def readerAssertEqual(self, input, expected_result): diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3866,33 +3866,6 @@ } assert(slotnames == Py_None || PyList_Check(slotnames)); -#if 0 - /* 2016-01-11 barry - This clause breaks at least three packages which - rely on Cython: kivy, pysam, and s3ql. Cython may be doing - something funny under the hood, but as this is clearly a regression - and the rationale for this prohibition is suspect, I am commenting - this out. Perhaps it should just be removed. See issue #22995 for - details. - */ - if (required) { - Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; - if (obj->ob_type->tp_dictoffset) - basicsize += sizeof(PyObject *); - if (obj->ob_type->tp_weaklistoffset) - basicsize += sizeof(PyObject *); - if (slotnames != Py_None) - basicsize += sizeof(PyObject *) * Py_SIZE(slotnames); - if (obj->ob_type->tp_basicsize > basicsize) { - Py_DECREF(slotnames); - Py_DECREF(state); - PyErr_Format(PyExc_TypeError, - "can't pickle %.200s objects", - Py_TYPE(obj)->tp_name); - return NULL; - } - } -#endif - if (slotnames != Py_None && Py_SIZE(slotnames) > 0) { PyObject *slots; Py_ssize_t slotnames_size, i; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:53:57 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 20:53:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_As_per_further_discussion=2C_re-enable_the_typeobject=2E?= =?utf-8?q?c_guard_for_picklability=2E?= Message-ID: <20160111205357.27362.43184@psf.io> https://hg.python.org/cpython/rev/f1cfac294af2 changeset: 99857:f1cfac294af2 parent: 99849:927fd0e14d49 parent: 99856:54dd5c105334 user: Barry Warsaw date: Mon Jan 11 15:51:27 2016 -0500 summary: As per further discussion, re-enable the typeobject.c guard for picklability. upstreams such as Cython will have to adjust. files: Misc/NEWS | 3 --- Objects/typeobject.c | 9 --------- 2 files changed, 0 insertions(+), 12 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,9 +10,6 @@ Core and Builtins ----------------- -- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in - _PyObject_GetState() due to regressions observed in Cython-based projects. - - Issue #25961: Disallowed null characters in the type name. - Issue #25973: Fix segfault when an invalid nonlocal statement binds a name diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3872,14 +3872,6 @@ } assert(slotnames == Py_None || PyList_Check(slotnames)); -#if 0 - /* 2016-01-11 barry - This clause breaks at least three packages which - rely on Cython: kivy, pysam, and s3ql. Cython may be doing - something funny under the hood, but as this is clearly a regression - and the rationale for this prohibition is suspect, I am commenting - this out. Perhaps it should just be removed. See issue #22995 for - details. - */ if (required) { Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; if (obj->ob_type->tp_dictoffset) @@ -3897,7 +3889,6 @@ return NULL; } } -#endif if (slotnames != Py_None && Py_SIZE(slotnames) > 0) { PyObject *slots; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:53:57 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 20:53:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_Merge_heads=2E?= Message-ID: <20160111205357.31091.39064@psf.io> https://hg.python.org/cpython/rev/1e9ec2d9f2d6 changeset: 99858:1e9ec2d9f2d6 branch: 3.5 parent: 99854:ef84d21f5292 parent: 99856:54dd5c105334 user: Barry Warsaw date: Mon Jan 11 15:53:19 2016 -0500 summary: Merge heads. files: Lib/test/test_csv.py | 19 ++++++++++--------- Objects/typeobject.c | 27 --------------------------- 2 files changed, 10 insertions(+), 36 deletions(-) diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -426,16 +426,17 @@ self.assertRaises(TypeError, csv.reader, [], quoting = -1) self.assertRaises(TypeError, csv.reader, [], quoting = 100) - def test_copy(self): - for name in csv.list_dialects(): - dialect = csv.get_dialect(name) - self.assertRaises(TypeError, copy.copy, dialect) + # See issue #22995 + ## def test_copy(self): + ## for name in csv.list_dialects(): + ## dialect = csv.get_dialect(name) + ## self.assertRaises(TypeError, copy.copy, dialect) - def test_pickle(self): - for name in csv.list_dialects(): - dialect = csv.get_dialect(name) - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - self.assertRaises(TypeError, pickle.dumps, dialect, proto) + ## def test_pickle(self): + ## for name in csv.list_dialects(): + ## dialect = csv.get_dialect(name) + ## for proto in range(pickle.HIGHEST_PROTOCOL + 1): + ## self.assertRaises(TypeError, pickle.dumps, dialect, proto) class TestCsvBase(unittest.TestCase): def readerAssertEqual(self, input, expected_result): diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3866,33 +3866,6 @@ } assert(slotnames == Py_None || PyList_Check(slotnames)); -#if 0 - /* 2016-01-11 barry - This clause breaks at least three packages which - rely on Cython: kivy, pysam, and s3ql. Cython may be doing - something funny under the hood, but as this is clearly a regression - and the rationale for this prohibition is suspect, I am commenting - this out. Perhaps it should just be removed. See issue #22995 for - details. - */ - if (required) { - Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; - if (obj->ob_type->tp_dictoffset) - basicsize += sizeof(PyObject *); - if (obj->ob_type->tp_weaklistoffset) - basicsize += sizeof(PyObject *); - if (slotnames != Py_None) - basicsize += sizeof(PyObject *) * Py_SIZE(slotnames); - if (obj->ob_type->tp_basicsize > basicsize) { - Py_DECREF(slotnames); - Py_DECREF(state); - PyErr_Format(PyExc_TypeError, - "can't pickle %.200s objects", - Py_TYPE(obj)->tp_name); - return NULL; - } - } -#endif - if (slotnames != Py_None && Py_SIZE(slotnames) > 0) { PyObject *slots; Py_ssize_t slotnames_size, i; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 15:53:58 2016 From: python-checkins at python.org (barry.warsaw) Date: Mon, 11 Jan 2016 20:53:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgaGVhZHMu?= Message-ID: <20160111205357.59574.55739@psf.io> https://hg.python.org/cpython/rev/6ab135134013 changeset: 99859:6ab135134013 parent: 99855:2b0ff9e16955 parent: 99857:f1cfac294af2 user: Barry Warsaw date: Mon Jan 11 15:53:51 2016 -0500 summary: Merge heads. files: Misc/NEWS | 3 --- Objects/typeobject.c | 9 --------- 2 files changed, 0 insertions(+), 12 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,9 +10,6 @@ Core and Builtins ----------------- -- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in - _PyObject_GetState() due to regressions observed in Cython-based projects. - - Issue #25961: Disallowed null characters in the type name. - Issue #25973: Fix segfault when an invalid nonlocal statement binds a name diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3872,14 +3872,6 @@ } assert(slotnames == Py_None || PyList_Check(slotnames)); -#if 0 - /* 2016-01-11 barry - This clause breaks at least three packages which - rely on Cython: kivy, pysam, and s3ql. Cython may be doing - something funny under the hood, but as this is clearly a regression - and the rationale for this prohibition is suspect, I am commenting - this out. Perhaps it should just be removed. See issue #22995 for - details. - */ if (required) { Py_ssize_t basicsize = PyBaseObject_Type.tp_basicsize; if (obj->ob_type->tp_dictoffset) @@ -3897,7 +3889,6 @@ return NULL; } } -#endif if (slotnames != Py_None && Py_SIZE(slotnames) > 0) { PyObject *slots; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 16:31:52 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 21:31:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIzI1OTkxOiBtZXJnZSB3aXRoIDMuNS4=?= Message-ID: <20160111213148.69141.85949@psf.io> https://hg.python.org/cpython/rev/cb08e5271cc0 changeset: 99862:cb08e5271cc0 parent: 99859:6ab135134013 parent: 99861:416db1a2fb81 user: Ezio Melotti date: Mon Jan 11 23:31:38 2016 +0200 summary: #25991: merge with 3.5. files: Doc/library/readline.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst --- a/Doc/library/readline.rst +++ b/Doc/library/readline.rst @@ -209,6 +209,8 @@ histfile = os.path.join(os.path.expanduser("~"), ".python_history") try: readline.read_history_file(histfile) + # default history len is -1 (infinite), which may grow unruly + readline.set_history_length(1000) except FileNotFoundError: pass @@ -234,6 +236,7 @@ def save(prev_h_len, histfile): new_h_len = readline.get_history_length() + readline.set_history_length(1000) readline.append_history_file(new_h_len - prev_h_len, histfile) atexit.register(save, h_len, histfile) @@ -261,4 +264,5 @@ atexit.register(self.save_history, histfile) def save_history(self, histfile): + readline.set_history_length(1000) readline.write_history_file(histfile) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 16:31:53 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 21:31:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI1OTkxOiBmaXgg?= =?utf-8?q?readline_example_to_limit_history_size=2E__Patch_by_Daniel_Dye?= =?utf-8?q?=2E?= Message-ID: <20160111213148.51234.62332@psf.io> https://hg.python.org/cpython/rev/416db1a2fb81 changeset: 99861:416db1a2fb81 branch: 3.5 parent: 99858:1e9ec2d9f2d6 user: Ezio Melotti date: Mon Jan 11 23:30:56 2016 +0200 summary: #25991: fix readline example to limit history size. Patch by Daniel Dye. files: Doc/library/readline.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst --- a/Doc/library/readline.rst +++ b/Doc/library/readline.rst @@ -209,6 +209,8 @@ histfile = os.path.join(os.path.expanduser("~"), ".python_history") try: readline.read_history_file(histfile) + # default history len is -1 (infinite), which may grow unruly + readline.set_history_length(1000) except FileNotFoundError: pass @@ -234,6 +236,7 @@ def save(prev_h_len, histfile): new_h_len = readline.get_history_length() + readline.set_history_length(1000) readline.append_history_file(new_h_len - prev_h_len, histfile) atexit.register(save, h_len, histfile) @@ -261,4 +264,5 @@ atexit.register(self.save_history, histfile) def save_history(self, histfile): + readline.set_history_length(1000) readline.write_history_file(histfile) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 16:31:53 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 21:31:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI1OTkxOiBmaXgg?= =?utf-8?q?readline_example_to_limit_history_size=2E__Patch_by_Daniel_Dye?= =?utf-8?q?=2E?= Message-ID: <20160111213148.47403.77916@psf.io> https://hg.python.org/cpython/rev/9a2c891a4333 changeset: 99860:9a2c891a4333 branch: 2.7 parent: 99852:609226a23086 user: Ezio Melotti date: Mon Jan 11 23:30:15 2016 +0200 summary: #25991: fix readline example to limit history size. Patch by Daniel Dye. files: Doc/library/readline.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst --- a/Doc/library/readline.rst +++ b/Doc/library/readline.rst @@ -214,6 +214,8 @@ histfile = os.path.join(os.path.expanduser("~"), ".pyhist") try: readline.read_history_file(histfile) + # default history len is -1 (infinite), which may grow unruly + readline.set_history_length(1000) except IOError: pass import atexit @@ -244,5 +246,6 @@ atexit.register(self.save_history, histfile) def save_history(self, histfile): + readline.set_history_length(1000) readline.write_history_file(histfile) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 16:56:59 2016 From: python-checkins at python.org (gregory.p.smith) Date: Mon, 11 Jan 2016 21:56:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fixes_issue_?= =?utf-8?q?=2326083=3A_Workaround_a_subprocess_bug_that_raised_an_incorrec?= =?utf-8?q?t?= Message-ID: <20160111215658.27380.63722@psf.io> https://hg.python.org/cpython/rev/268d13c7e939 changeset: 99863:268d13c7e939 branch: 2.7 parent: 99860:9a2c891a4333 user: Gregory P. Smith date: Mon Jan 11 13:56:42 2016 -0800 summary: Fixes issue #26083: Workaround a subprocess bug that raised an incorrect "ValueError: insecure string pickle" exception instead of the actual exception on some platforms such as Mac OS X when an exception raised in the forked child process prior to the exec() was large enough that it overflowed the internal errpipe_read pipe buffer. files: Lib/subprocess.py | 6 +++++- Misc/NEWS | 6 ++++++ 2 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -1313,8 +1313,12 @@ os.close(errpipe_write) # Wait for exec to fail or succeed; possibly raising exception - # Exception limited to 1M data = _eintr_retry_call(os.read, errpipe_read, 1048576) + pickle_bits = [data] + while data: + pickle_bits.append(data) + data = _eintr_retry_call(os.read, errpipe_read, 1048576) + data = "".join(pickle_bits) finally: if p2cread is not None and p2cwrite is not None: _close_in_parent(p2cread) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,12 @@ Library ------- +- Issue #26083: Workaround a subprocess bug that raises an incorrect + "ValueError: insecure string pickle" exception instead of the actual + exception on some platforms such as Mac OS X when an exception raised + in the forked child process prior to the exec() was large enough that + it overflowed the internal errpipe_read pipe buffer. + - Issue #24103: Fixed possible use after free in ElementTree.iterparse(). - Issue #20954: _args_from_interpreter_flags used by multiprocessing and some -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 17:00:51 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 22:00:51 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_=2326063=3A_future-proof_?= =?utf-8?q?copyright_updating=2E?= Message-ID: <20160111220048.51238.79041@psf.io> https://hg.python.org/devguide/rev/a4c18139d9b0 changeset: 781:a4c18139d9b0 user: Ezio Melotti date: Tue Jan 12 00:00:43 2016 +0200 summary: #26063: future-proof copyright updating. files: conf.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/conf.py b/conf.py --- a/conf.py +++ b/conf.py @@ -11,7 +11,9 @@ # All configuration values have a default; values that are commented out # serve to show the default. -import sys, os +import os +import sys +import time # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -43,7 +45,7 @@ # General information about the project. project = u'Python Developer\'s Guide' -copyright = u'2011-2016, Python Software Foundation' +copyright = u'2011-%s, Python Software Foundation' % time.strftime('%Y') # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Mon Jan 11 17:11:39 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 22:11:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI1NTE3OiBmaXgg?= =?utf-8?q?regex_in_the_regex_howto=2E__Patch_by_Elena_Oat=2E?= Message-ID: <20160111221138.31083.44160@psf.io> https://hg.python.org/cpython/rev/6bd4a4907f66 changeset: 99865:6bd4a4907f66 branch: 3.5 parent: 99861:416db1a2fb81 user: Ezio Melotti date: Tue Jan 12 00:09:13 2016 +0200 summary: #25517: fix regex in the regex howto. Patch by Elena Oat. files: Doc/howto/regex.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -1004,17 +1004,18 @@ A negative lookahead cuts through all this confusion: -``.*[.](?!bat$).*$`` The negative lookahead means: if the expression ``bat`` +``.*[.](?!bat$)[^.]*$`` The negative lookahead means: if the expression ``bat`` doesn't match at this point, try the rest of the pattern; if ``bat$`` does match, the whole pattern will fail. The trailing ``$`` is required to ensure that something like ``sample.batch``, where the extension only starts with -``bat``, will be allowed. +``bat``, will be allowed. The ``[^.]*`` makes sure that the pattern works +when there are multiple dots in the filename. Excluding another filename extension is now easy; simply add it as an alternative inside the assertion. The following pattern excludes filenames that end in either ``bat`` or ``exe``: -``.*[.](?!bat$|exe$).*$`` +``.*[.](?!bat$|exe$)[^.]*$`` Modifying Strings -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 17:11:39 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 22:11:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI1NTE3OiBmaXgg?= =?utf-8?q?regex_in_the_regex_howto=2E__Patch_by_Elena_Oat=2E?= Message-ID: <20160111221138.11399.45577@psf.io> https://hg.python.org/cpython/rev/c6b5c03183e3 changeset: 99864:c6b5c03183e3 branch: 2.7 parent: 99860:9a2c891a4333 user: Ezio Melotti date: Tue Jan 12 00:07:23 2016 +0200 summary: #25517: fix regex in the regex howto. Patch by Elena Oat. files: Doc/howto/regex.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -1015,17 +1015,18 @@ A negative lookahead cuts through all this confusion: -``.*[.](?!bat$).*$`` The negative lookahead means: if the expression ``bat`` +``.*[.](?!bat$)[^.]*$`` The negative lookahead means: if the expression ``bat`` doesn't match at this point, try the rest of the pattern; if ``bat$`` does match, the whole pattern will fail. The trailing ``$`` is required to ensure that something like ``sample.batch``, where the extension only starts with -``bat``, will be allowed. +``bat``, will be allowed. The ``[^.]*`` makes sure that the pattern works +when there are multiple dots in the filename. Excluding another filename extension is now easy; simply add it as an alternative inside the assertion. The following pattern excludes filenames that end in either ``bat`` or ``exe``: -``.*[.](?!bat$|exe$).*$`` +``.*[.](?!bat$|exe$)[^.]*$`` Modifying Strings -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 17:11:40 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 22:11:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIzI1NTE3OiBtZXJnZSB3aXRoIDMuNS4=?= Message-ID: <20160111221139.47409.50695@psf.io> https://hg.python.org/cpython/rev/48e2f5915d49 changeset: 99866:48e2f5915d49 parent: 99862:cb08e5271cc0 parent: 99865:6bd4a4907f66 user: Ezio Melotti date: Tue Jan 12 00:09:43 2016 +0200 summary: #25517: merge with 3.5. files: Doc/howto/regex.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -1004,17 +1004,18 @@ A negative lookahead cuts through all this confusion: -``.*[.](?!bat$).*$`` The negative lookahead means: if the expression ``bat`` +``.*[.](?!bat$)[^.]*$`` The negative lookahead means: if the expression ``bat`` doesn't match at this point, try the rest of the pattern; if ``bat$`` does match, the whole pattern will fail. The trailing ``$`` is required to ensure that something like ``sample.batch``, where the extension only starts with -``bat``, will be allowed. +``bat``, will be allowed. The ``[^.]*`` makes sure that the pattern works +when there are multiple dots in the filename. Excluding another filename extension is now easy; simply add it as an alternative inside the assertion. The following pattern excludes filenames that end in either ``bat`` or ``exe``: -``.*[.](?!bat$|exe$).*$`` +``.*[.](?!bat$|exe$)[^.]*$`` Modifying Strings -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 17:11:40 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 22:11:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMi43IC0+IDIuNyk6?= =?utf-8?q?_Merge_heads=2E?= Message-ID: <20160111221139.59584.1819@psf.io> https://hg.python.org/cpython/rev/d208a2ada67c changeset: 99867:d208a2ada67c branch: 2.7 parent: 99863:268d13c7e939 parent: 99864:c6b5c03183e3 user: Ezio Melotti date: Tue Jan 12 00:11:24 2016 +0200 summary: Merge heads. files: Doc/howto/regex.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -1015,17 +1015,18 @@ A negative lookahead cuts through all this confusion: -``.*[.](?!bat$).*$`` The negative lookahead means: if the expression ``bat`` +``.*[.](?!bat$)[^.]*$`` The negative lookahead means: if the expression ``bat`` doesn't match at this point, try the rest of the pattern; if ``bat$`` does match, the whole pattern will fail. The trailing ``$`` is required to ensure that something like ``sample.batch``, where the extension only starts with -``bat``, will be allowed. +``bat``, will be allowed. The ``[^.]*`` makes sure that the pattern works +when there are multiple dots in the filename. Excluding another filename extension is now easy; simply add it as an alternative inside the assertion. The following pattern excludes filenames that end in either ``bat`` or ``exe``: -``.*[.](?!bat$|exe$).*$`` +``.*[.](?!bat$|exe$)[^.]*$`` Modifying Strings -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 17:15:21 2016 From: python-checkins at python.org (ezio.melotti) Date: Mon, 11 Jan 2016 22:15:21 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_=2325967=3A_add_note_abou?= =?utf-8?q?t_updating_lib2to3_when_the_grammar_is_updated=2E__Patch_by?= Message-ID: <20160111221521.4224.41814@psf.io> https://hg.python.org/devguide/rev/367486fbab84 changeset: 782:367486fbab84 user: Ezio Melotti date: Tue Jan 12 00:15:14 2016 +0200 summary: #25967: add note about updating lib2to3 when the grammar is updated. Patch by Chris Thomas. files: grammar.rst | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/grammar.rst b/grammar.rst --- a/grammar.rst +++ b/grammar.rst @@ -58,6 +58,8 @@ * Certain changes may require tweaks to the library module pyclbr. +* Lib/lib2to3/Grammar.txt may need changes to match the Grammar. + * Documentation must be written! * After everything has been checked in, you're likely to see a new -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Mon Jan 11 18:16:25 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 23:16:25 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510?= Message-ID: <20160111231625.11393.66903@psf.io> https://hg.python.org/peps/rev/849525efbf8a changeset: 6167:849525efbf8a user: Victor Stinner date: Mon Jan 11 23:55:32 2016 +0100 summary: PEP 510 * make new methods private * make Guard type private and rename it to PyFuncGuard * elaborate potential changes on the Python semantics * explain than other implementations of Python are free to not implement new methods, or implement them as no-op files: pep-0510.txt | 86 ++++++++++++++++++++++++++++----------- 1 files changed, 62 insertions(+), 24 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -13,8 +13,8 @@ Abstract ======== -Add an API to add specialized functions with guards to functions, to -support static optimizers respecting the Python semantics. +Add a private API to CPython to add specialized functions with guards to +functions, to support static optimizers respecting the Python semantics. Rationale @@ -29,10 +29,15 @@ semantics requires to detect when "something changes", we will call these checks "guards". -This PEP proposes to add a ``specialize()`` method to functions to add a -specialized functions with guards. When the function is called, the -specialized function is used if nothing changed, otherwise use the -original bytecode. +This PEP proposes to add an API to add specialized functions with guards +to a function. When the function is called, the specialized function is +used if nothing changed, otherwise use the original bytecode. + +Even if guards help to respect most parts of the Python semantics, it's +really hard to optimize Python without making subtle changes on the +exact behaviour. CPython has a long history and many applications rely +on implementation details. A compromise must be found between +"everything is mutable" and performance. Writing an optimizer is out of the scope of this PEP. @@ -112,18 +117,18 @@ def fast_func(): return "A" - func.specialize(fast_func.__code__, [myoptimizer.GuardBuiltins("chr")]) + func._specialize(fast_func.__code__, [myoptimizer.GuardBuiltins("chr")]) del fast_func print("func(): %s" % func()) - print("#specialized: %s" % len(func.get_specialized())) + print("#specialized: %s" % len(func._get_specialized())) print() import builtins builtins.chr = lambda obj: "mock" print("func(): %s" % func()) - print("#specialized: %s" % len(func.get_specialized())) + print("#specialized: %s" % len(func._get_specialized())) Output:: @@ -157,10 +162,10 @@ def func(arg): return chr(arg) - func.specialize(chr, [myoptimizer.GuardBuiltins("chr")]) + func._specialize(chr, [myoptimizer.GuardBuiltins("chr")]) print("func(65): %s" % func(65)) - print("#specialized: %s" % len(func.get_specialized())) + print("#specialized: %s" % len(func._get_specialized())) print() import builtins @@ -222,24 +227,43 @@ Changes ======= -* Add two new methods to functions: +* Add two new private methods to functions: - - ``specialize(code, guards: list)``: add specialized + * ``_specialize(code, guards: list)``: add specialized function with guard. `code` is a code object (ex: - ``func2.__code__``) or any callable object (ex: ``len``). - The specialization can be ignored if a guard already fails. - - ``get_specialized()``: get the list of specialized functions with - guards + ``func2.__code__``) or any callable object (ex: the builtin + ``len()`` function). The specialization can be ignored if a guard + already fails or for other reasons (ex: the implementation of Python + does not implement this feature). Return ``False`` is the + specialized function was ignored, return ``True`` otherwise. -* Base ``Guard`` type which can be used as parent type to implement - guards. It requires to implement a ``check()`` function, with an - optional ``first_check()`` function. API: + * ``_get_specialized()``: get the list of specialized functions with + guards. Return a list of ``(func, guards)`` tuples where func is the + specialized function and guards is a list of guards. Return an empty + list if the function was never specialized. - * ``int first_check(PyObject *guard, PyObject *func)``: return 0 on - success, -1 if the guard will always fail +* Add a private ``PyFuncGuard`` Python type. It requires to implement a + C ``check()`` function, with an optional C ``init()`` function. API: + + * ``int init(PyObject *guard, PyObject *func)``: initialize a guard, + *func* is the function to which the specialized function will be + attached. Result: + + * return ``1`` on success + * return ``0`` if the guard will always fail (the specialization must be + ignored) + * raise an exception and return ``-1`` on error + * ``int check(PyObject *guard, PyObject **stack, int na, int nk)``: - return 1 on success, 0 if the guard failed temporarely, -1 if the - guard will always fail + check the guard. Result: + + * return 2 on success + * return 1 if the guard failed temporarely + * return 0 if the guard will always fail + * raise an exception and return -1 on error + + * A guard can be called in Python with parameters, it returns the + result of the guard check. Microbenchmark on ``python3.6 -m timeit -s 'def f(): pass' 'f()'`` (best of 3 runs): @@ -263,6 +287,20 @@ when a module is loaded. +Other implementations of Python +=============================== + +This PEP is designed to be implemented in C for CPython. + +Other implementations of Python are free to not implement added private +function methods. + +Or they can implement a ``_specialize()`` method which always ignores +the specialized function (in short, do nothing and always return +``False``) and a ``_get_specialized()`` method which always returns an +empty list. + + Discussion ========== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 18:19:35 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 11 Jan 2016 23:19:35 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_be_more_explicit_o?= =?utf-8?q?n_when_the_version_is_incremented_or_not?= Message-ID: <20160111231933.70883.37214@psf.io> https://hg.python.org/peps/rev/e732b21668d6 changeset: 6168:e732b21668d6 user: Victor Stinner date: Tue Jan 12 00:19:28 2016 +0100 summary: PEP 509: be more explicit on when the version is incremented or not files: pep-0509.txt | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -153,9 +153,10 @@ * ``setdefault(key, value)`` if the `key` does not exist * ``__detitem__(key)`` if the key exists * ``__setitem__(key, value)`` if the `key` doesn't exist or if the value - is different -* ``update(...)`` if new values are different than existing values (the - version can be incremented multiple times) + is not ``dict[key]`` +* ``update(...)`` if new values are different than existing values: + values are compared by identity, not by their content; the version can + be incremented multiple times Example using an hypothetical ``dict_get_version(dict)`` function:: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 11 21:05:21 2016 From: python-checkins at python.org (yury.selivanov) Date: Tue, 12 Jan 2016 02:05:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpc3N1ZSAjMjU0ODYp?= Message-ID: <20160112020521.1009.75611@psf.io> https://hg.python.org/cpython/rev/4f1a0e5deca6 changeset: 99869:4f1a0e5deca6 parent: 99866:48e2f5915d49 parent: 99868:36a54e30bcbd user: Yury Selivanov date: Mon Jan 11 21:05:18 2016 -0500 summary: Merge 3.5 (issue #25486) files: Doc/library/inspect.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -810,8 +810,7 @@ .. deprecated:: 3.0 Use :func:`signature` and :ref:`Signature Object `, which provide a - better introspecting API for callables. This function will be removed - in Python 3.6. + better introspecting API for callables. .. function:: getfullargspec(func) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 11 21:05:21 2016 From: python-checkins at python.org (yury.selivanov) Date: Tue, 12 Jan 2016 02:05:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1NDg2?= =?utf-8?q?=3A_Drop_=22removed_in_3=2E6=22_wording_from_inspect=2Egetargsp?= =?utf-8?q?ec_docs=2E?= Message-ID: <20160112020520.993.28160@psf.io> https://hg.python.org/cpython/rev/36a54e30bcbd changeset: 99868:36a54e30bcbd branch: 3.5 parent: 99865:6bd4a4907f66 user: Yury Selivanov date: Mon Jan 11 21:04:50 2016 -0500 summary: Issue #25486: Drop "removed in 3.6" wording from inspect.getargspec docs. files: Doc/library/inspect.rst | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -829,8 +829,7 @@ .. deprecated:: 3.0 Use :func:`signature` and :ref:`Signature Object `, which provide a - better introspecting API for callables. This function will be removed - in Python 3.6. + better introspecting API for callables. .. function:: getfullargspec(func) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 02:32:04 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 12 Jan 2016 07:32:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MzQ4?= =?utf-8?q?=3A_Add_--pgo_and_--pgo-job_flags_to_PCbuild=5Cbuild=2Ebat?= Message-ID: <20160112073201.47399.47549@psf.io> https://hg.python.org/cpython/rev/01ab138f4133 changeset: 99871:01ab138f4133 branch: 3.5 parent: 99868:36a54e30bcbd user: Zachary Ware date: Tue Jan 12 01:26:50 2016 -0600 summary: Issue #25348: Add --pgo and --pgo-job flags to PCbuild\build.bat files: Misc/NEWS | 5 ++ PCbuild/build.bat | 57 ++++++++++++++++++++++++++++-- PCbuild/build_pgo.bat | 50 ++------------------------- 3 files changed, 61 insertions(+), 51 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -151,6 +151,11 @@ Build ----- +- Issue #25348: Added ``--pgo`` and ``--pgo-job`` arguments to + ``PCbuild\build.bat`` for building with Profile-Guided Optimization. The + old ``PCbuild\build_pgo.bat`` script is now deprecated, and simply calls + ``PCbuild\build.bat --pgo %*``. + - Issue #25827: Add support for building with ICC to ``configure``, including a new ``--with-icc`` flag. diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -25,6 +25,9 @@ echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) +echo. --pgo Build with Profile-Guided Optimization. This flag +echo. overrides -c and -d +echo. --test-marker Enable the test marker within the build. echo. echo.Available flags to avoid building certain modules. echo.These flags have no effect if '-e' is not given: @@ -38,7 +41,8 @@ echo. Set the platform (default: Win32) echo. -t Build ^| Rebuild ^| Clean ^| CleanAll echo. Set the target manually -echo. --test-marker Enable the test marker within the build. +echo. --pgo-job The job to use for PGO training; implies --pgo +echo. (default: "-m test --pgo") exit /b 127 :Run @@ -51,6 +55,12 @@ set parallel=/m set verbose=/nologo /v:m set kill= +set do_pgo= +set pgo_job=-m test --pgo +set on_64_bit=true + +rem This may not be 100% accurate, but close enough. +if "%ProgramFiles(x86)%"=="" (set on_64_bit=false) :CheckOpts if "%~1"=="-h" goto Usage @@ -63,6 +73,8 @@ if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts +if "%~1"=="--pgo" (set do_pgo=true) & shift & goto CheckOpts +if "%~1"=="--pgo-job" (set do_pgo=true) & (set pgo_job=%~2) & shift & shift & goto CheckOpts if "%~1"=="--test-marker" (set UseTestMarker=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version rem These use the actual property names used by MSBuild. We could just let @@ -78,15 +90,49 @@ if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" -if "%platf%"=="x64" (set vs_platf=x86_amd64) +if "%platf%"=="x64" ( + if "%on_64_bit%"=="true" ( + rem This ought to always be correct these days... + set vs_platf=amd64 + ) else ( + if "%do_pgo%"=="true" ( + echo.ERROR: Cannot cross-compile with PGO + echo. 32bit operating system detected, if this is incorrect, + echo. make sure the ProgramFiles(x86^) environment variable is set + exit /b 1 + ) + set vs_platf=x86_amd64 + ) +) rem Setup the environment call "%dir%env.bat" %vs_platf% >nul -if "%kill%"=="true" ( - msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true +if "%kill%"=="true" call :Kill + +if "%do_pgo%"=="true" ( + set conf=PGInstrument + call :Build + del /s "%dir%\*.pgc" + del /s "%dir%\..\Lib\*.pyc" + echo on + call "%dir%\..\python.bat" %pgo_job% + @echo off + call :Kill + set conf=PGUpdate ) +goto Build +:Kill +echo on +msbuild "%dir%\pythoncore.vcxproj" /t:KillPython %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:KillPython=true + + at echo off +goto :eof + +:Build rem Call on MSBuild to do the work, echo the command. rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" @@ -98,7 +144,8 @@ /p:UseTestMarker=%UseTestMarker%^ %1 %2 %3 %4 %5 %6 %7 %8 %9 - at goto :eof + at echo off +goto :eof :Version rem Display the current build version information diff --git a/PCbuild/build_pgo.bat b/PCbuild/build_pgo.bat --- a/PCbuild/build_pgo.bat +++ b/PCbuild/build_pgo.bat @@ -1,48 +1,6 @@ @echo off -rem A batch program to build PGO (Profile guided optimization) by first -rem building instrumented binaries, then running the testsuite, and -rem finally building the optimized code. -rem Note, after the first instrumented run, one can just keep on -rem building the PGUpdate configuration while developing. +echo.DeprecationWarning: +echo. This script is deprecated, use `build.bat --pgo` instead. +echo. -setlocal -set platf=Win32 -set parallel=/m -set dir=%~dp0 - -rem use the performance testsuite. This is quick and simple -set job1="%dir%..\tools\pybench\pybench.py" -n 1 -C 1 --with-gc -set path1="%dir%..\tools\pybench" - -rem or the whole testsuite for more thorough testing -set job2="%dir%..\lib\test\regrtest.py" -set path2="%dir%..\lib" - -set job=%job1% -set clrpath=%path1% - -:CheckOpts -if "%1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts -if "%1"=="-2" (set job=%job2%) & (set clrpath=%path2%) & shift & goto CheckOpts -if "%1"=="-M" (set parallel=) & shift & goto CheckOpts - - -rem We cannot cross compile PGO builds, as the optimization needs to be run natively -set vs_platf=x86 -set PGO=%dir%win32-pgo - -if "%platf%"=="x64" (set vs_platf=amd64) & (set PGO=%dir%amd64-pgo) -rem Setup the environment -call "%dir%env.bat" %vs_platf% - - -rem build the instrumented version -msbuild "%dir%pcbuild.proj" %parallel% /t:Build /p:Configuration=PGInstrument /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 - -rem remove .pyc files, .pgc files and execute the job -"%PGO%\python.exe" "%dir%rmpyc.py" %clrpath% -del "%PGO%\*.pgc" -"%PGO%\python.exe" %job% - -rem build optimized version -msbuild "%dir%pcbuild.proj" %parallel% /t:Build /p:Configuration=PGUpdate /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +call "%~dp0build.bat" --pgo %* -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 02:32:04 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 12 Jan 2016 07:32:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2325348=3A_Merge_with_3=2E5?= Message-ID: <20160112073201.51226.24966@psf.io> https://hg.python.org/cpython/rev/9442b4a60379 changeset: 99872:9442b4a60379 parent: 99869:4f1a0e5deca6 parent: 99871:01ab138f4133 user: Zachary Ware date: Tue Jan 12 01:31:21 2016 -0600 summary: Closes #25348: Merge with 3.5 files: Misc/NEWS | 4 ++ PCbuild/build.bat | 57 ++++++++++++++++++++++++++++-- PCbuild/build_pgo.bat | 48 -------------------------- 3 files changed, 56 insertions(+), 53 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -581,6 +581,10 @@ Build ----- +- Issue #25348: Added ``--pgo`` and ``--pgo-job`` arguments to + ``PCbuild\build.bat`` for building with Profile-Guided Optimization. The + old ``PCbuild\build_pgo.bat`` script is removed. + - Issue #25827: Add support for building with ICC to ``configure``, including a new ``--with-icc`` flag. diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -25,6 +25,9 @@ echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) +echo. --pgo Build with Profile-Guided Optimization. This flag +echo. overrides -c and -d +echo. --test-marker Enable the test marker within the build. echo. echo.Available flags to avoid building certain modules. echo.These flags have no effect if '-e' is not given: @@ -38,7 +41,8 @@ echo. Set the platform (default: Win32) echo. -t Build ^| Rebuild ^| Clean ^| CleanAll echo. Set the target manually -echo. --test-marker Enable the test marker within the build. +echo. --pgo-job The job to use for PGO training; implies --pgo +echo. (default: "-m test --pgo") exit /b 127 :Run @@ -51,6 +55,12 @@ set parallel=/m set verbose=/nologo /v:m set kill= +set do_pgo= +set pgo_job=-m test --pgo +set on_64_bit=true + +rem This may not be 100% accurate, but close enough. +if "%ProgramFiles(x86)%"=="" (set on_64_bit=false) :CheckOpts if "%~1"=="-h" goto Usage @@ -63,6 +73,8 @@ if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts +if "%~1"=="--pgo" (set do_pgo=true) & shift & goto CheckOpts +if "%~1"=="--pgo-job" (set do_pgo=true) & (set pgo_job=%~2) & shift & shift & goto CheckOpts if "%~1"=="--test-marker" (set UseTestMarker=true) & shift & goto CheckOpts if "%~1"=="-V" shift & goto Version rem These use the actual property names used by MSBuild. We could just let @@ -78,15 +90,49 @@ if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" -if "%platf%"=="x64" (set vs_platf=x86_amd64) +if "%platf%"=="x64" ( + if "%on_64_bit%"=="true" ( + rem This ought to always be correct these days... + set vs_platf=amd64 + ) else ( + if "%do_pgo%"=="true" ( + echo.ERROR: Cannot cross-compile with PGO + echo. 32bit operating system detected, if this is incorrect, + echo. make sure the ProgramFiles(x86^) environment variable is set + exit /b 1 + ) + set vs_platf=x86_amd64 + ) +) rem Setup the environment call "%dir%env.bat" %vs_platf% >nul -if "%kill%"=="true" ( - msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true +if "%kill%"=="true" call :Kill + +if "%do_pgo%"=="true" ( + set conf=PGInstrument + call :Build + del /s "%dir%\*.pgc" + del /s "%dir%\..\Lib\*.pyc" + echo on + call "%dir%\..\python.bat" %pgo_job% + @echo off + call :Kill + set conf=PGUpdate ) +goto Build +:Kill +echo on +msbuild "%dir%\pythoncore.vcxproj" /t:KillPython %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:KillPython=true + + at echo off +goto :eof + +:Build rem Call on MSBuild to do the work, echo the command. rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" @@ -98,7 +144,8 @@ /p:UseTestMarker=%UseTestMarker%^ %1 %2 %3 %4 %5 %6 %7 %8 %9 - at goto :eof + at echo off +goto :eof :Version rem Display the current build version information diff --git a/PCbuild/build_pgo.bat b/PCbuild/build_pgo.bat deleted file mode 100644 --- a/PCbuild/build_pgo.bat +++ /dev/null @@ -1,48 +0,0 @@ - at echo off -rem A batch program to build PGO (Profile guided optimization) by first -rem building instrumented binaries, then running the testsuite, and -rem finally building the optimized code. -rem Note, after the first instrumented run, one can just keep on -rem building the PGUpdate configuration while developing. - -setlocal -set platf=Win32 -set parallel=/m -set dir=%~dp0 - -rem use the performance testsuite. This is quick and simple -set job1="%dir%..\tools\pybench\pybench.py" -n 1 -C 1 --with-gc -set path1="%dir%..\tools\pybench" - -rem or the whole testsuite for more thorough testing -set job2="%dir%..\lib\test\regrtest.py" -set path2="%dir%..\lib" - -set job=%job1% -set clrpath=%path1% - -:CheckOpts -if "%1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts -if "%1"=="-2" (set job=%job2%) & (set clrpath=%path2%) & shift & goto CheckOpts -if "%1"=="-M" (set parallel=) & shift & goto CheckOpts - - -rem We cannot cross compile PGO builds, as the optimization needs to be run natively -set vs_platf=x86 -set PGO=%dir%win32-pgo - -if "%platf%"=="x64" (set vs_platf=amd64) & (set PGO=%dir%amd64-pgo) -rem Setup the environment -call "%dir%env.bat" %vs_platf% - - -rem build the instrumented version -msbuild "%dir%pcbuild.proj" %parallel% /t:Build /p:Configuration=PGInstrument /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 - -rem remove .pyc files, .pgc files and execute the job -"%PGO%\python.exe" "%dir%rmpyc.py" %clrpath% -del "%PGO%\*.pgc" -"%PGO%\python.exe" %job% - -rem build optimized version -msbuild "%dir%pcbuild.proj" %parallel% /t:Build /p:Configuration=PGUpdate /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 02:32:04 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 12 Jan 2016 07:32:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1MzQ4?= =?utf-8?q?=3A_Add_--pgo_and_--pgo-job_flags_to_PCbuild=5Cbuild=2Ebat?= Message-ID: <20160112073201.69139.38849@psf.io> https://hg.python.org/cpython/rev/113c524aed93 changeset: 99870:113c524aed93 branch: 2.7 parent: 99867:d208a2ada67c user: Zachary Ware date: Tue Jan 12 01:20:33 2016 -0600 summary: Issue #25348: Add --pgo and --pgo-job flags to PCbuild\build.bat files: Misc/NEWS | 5 ++ PCbuild/build.bat | 55 +++++++++++++++++++++++++++++- PCbuild/build_pgo.bat | 50 ++------------------------- PCbuild/env.bat | 6 +++ 4 files changed, 67 insertions(+), 49 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -85,6 +85,11 @@ Build ----- +- Issue #25348: Added ``--pgo`` and ``--pgo-job`` arguments to + ``PCbuild\build.bat`` for building with Profile-Guided Optimization. The + old ``PCbuild\build_pgo.bat`` script is now deprecated, and simply calls + ``PCbuild\build.bat --pgo %*``. + - Issue #25827: Add support for building with ICC to ``configure``, including a new ``--with-icc`` flag. diff --git a/PCbuild/build.bat b/PCbuild/build.bat --- a/PCbuild/build.bat +++ b/PCbuild/build.bat @@ -24,6 +24,8 @@ echo. -v Increased output messages echo. -k Attempt to kill any running Pythons before building (usually done echo. automatically by the pythoncore project) +echo. --pgo Build with Profile-Guided Optimization. This flag +echo. overrides -c and -d echo. echo.Available flags to avoid building certain modules. echo.These flags have no effect if '-e' is not given: @@ -38,6 +40,8 @@ echo. Set the platform (default: Win32) echo. -t Build ^| Rebuild ^| Clean ^| CleanAll echo. Set the target manually +echo. --pgo-job The job to use for PGO training; implies --pgo +echo. (default: "-m test.regrtest --pgo") exit /b 127 :Run @@ -50,6 +54,12 @@ set parallel= set verbose=/nologo /v:m set kill= +set do_pgo= +set pgo_job=-m test.regrtest --pgo +set on_64_bit=true + +rem This may not be 100% accurate, but close enough. +if "%ProgramFiles(x86)%"=="" (set on_64_bit=false) :CheckOpts if "%~1"=="-h" goto Usage @@ -62,6 +72,8 @@ if "%~1"=="-M" (set parallel=) & shift & goto CheckOpts if "%~1"=="-v" (set verbose=/v:n) & shift & goto CheckOpts if "%~1"=="-k" (set kill=true) & shift & goto CheckOpts +if "%~1"=="--pgo" (set do_pgo=true) & shift & goto CheckOpts +if "%~1"=="--pgo-job" (set do_pgo=true) & (set pgo_job=%~2) & shift & shift & goto CheckOpts rem These use the actual property names used by MSBuild. We could just let rem them in through the environment, but we specify them on the command line rem anyway for visibility so set defaults after this @@ -77,15 +89,49 @@ if "%IncludeExternals%"=="true" call "%dir%get_externals.bat" -if "%platf%"=="x64" (set vs_platf=x86_amd64) +if "%platf%"=="x64" ( + if "%on_64_bit%"=="true" ( + rem This ought to always be correct these days... + set vs_platf=amd64 + ) else ( + if "%do_pgo%"=="true" ( + echo.ERROR: Cannot cross-compile with PGO + echo. 32bit operating system detected, if this is incorrect, + echo. make sure the ProgramFiles(x86^) environment variable is set + exit /b 1 + ) + set vs_platf=x86_amd64 + ) +) rem Setup the environment call "%dir%env.bat" %vs_platf% >nul -if "%kill%"=="true" ( - msbuild /v:m /nologo /target:KillPython "%dir%\pythoncore.vcxproj" /p:Configuration=%conf% /p:Platform=%platf% /p:KillPython=true +if "%kill%"=="true" call :Kill + +if "%do_pgo%"=="true" ( + set conf=PGInstrument + call :Build + del /s "%dir%\*.pgc" + del /s "%dir%\..\Lib\*.pyc" + echo on + call "%dir%\..\python.bat" %pgo_job% + @echo off + call :Kill + set conf=PGUpdate ) +goto Build +:Kill +echo on +msbuild "%dir%\pythoncore.vcxproj" /t:KillPython %verbose%^ + /p:Configuration=%conf% /p:Platform=%platf%^ + /p:KillPython=true + + at echo off +goto :eof + +:Build rem Call on MSBuild to do the work, echo the command. rem Passing %1-9 is not the preferred option, but argument parsing in rem batch is, shall we say, "lackluster" @@ -96,3 +142,6 @@ /p:IncludeSSL=%IncludeSSL% /p:IncludeTkinter=%IncludeTkinter%^ /p:IncludeBsddb=%IncludeBsddb%^ %1 %2 %3 %4 %5 %6 %7 %8 %9 + + at echo off +goto :eof diff --git a/PCbuild/build_pgo.bat b/PCbuild/build_pgo.bat --- a/PCbuild/build_pgo.bat +++ b/PCbuild/build_pgo.bat @@ -1,48 +1,6 @@ @echo off -rem A batch program to build PGO (Profile guided optimization) by first -rem building instrumented binaries, then running the testsuite, and -rem finally building the optimized code. -rem Note, after the first instrumented run, one can just keep on -rem building the PGUpdate configuration while developing. +echo.DeprecationWarning: +echo. This script is deprecated, use `build.bat --pgo` instead. +echo. -setlocal -set platf=Win32 -set parallel=/m -set dir=%~dp0 - -rem use the performance testsuite. This is quick and simple -set job1="%dir%..\tools\pybench\pybench.py" -n 1 -C 1 --with-gc -set path1="%dir%..\tools\pybench" - -rem or the whole testsuite for more thorough testing -set job2="%dir%..\lib\test\regrtest.py" -set path2="%dir%..\lib" - -set job=%job1% -set clrpath=%path1% - -:CheckOpts -if "%1"=="-p" (set platf=%2) & shift & shift & goto CheckOpts -if "%1"=="-2" (set job=%job2%) & (set clrpath=%path2%) & shift & goto CheckOpts -if "%1"=="-M" (set parallel=) & shift & goto CheckOpts - - -rem We cannot cross compile PGO builds, as the optimization needs to be run natively -set vs_platf=x86 -set PGO=%dir%win32-pgo - -if "%platf%"=="x64" (set vs_platf=amd64) & (set PGO=%dir%amd64-pgo) -rem Setup the environment -call "%dir%env.bat" %vs_platf% - - -rem build the instrumented version -msbuild "%dir%pcbuild.proj" %parallel% /t:Build /p:Configuration=PGInstrument /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 - -rem remove .pyc files, .pgc files and execute the job -"%PGO%\python.exe" "%dir%rmpyc.py" %clrpath% -del "%PGO%\*.pgc" -"%PGO%\python.exe" %job% - -rem build optimized version -msbuild "%dir%pcbuild.proj" %parallel% /t:Build /p:Configuration=PGUpdate /p:Platform=%platf% %1 %2 %3 %4 %5 %6 %7 %8 %9 +call "%~dp0build.bat" --pgo %* diff --git a/PCbuild/env.bat b/PCbuild/env.bat --- a/PCbuild/env.bat +++ b/PCbuild/env.bat @@ -10,6 +10,12 @@ echo Build environments: x86, amd64, x86_amd64 echo. + +rem Set up the v90 tools first. This is mostly needed to allow PGInstrument +rem builds to find the PGO DLL. Do it first so the newer MSBuild is found +rem before the one from v90 (vcvarsall.bat prepends to PATH). +call "%VS90COMNTOOLS%..\..\VC\vcvarsall.bat" %* + set VSTOOLS=%VS140COMNTOOLS% if "%VSTOOLS%"=="" set VSTOOLS=%VS120COMNTOOLS% if "%VSTOOLS%"=="" set VSTOOLS=%VS110COMNTOOLS% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 03:33:42 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 08:33:42 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_=2319316=3A_fix_wording_i?= =?utf-8?q?n_the_compiler=2Erst_page=2E?= Message-ID: <20160112083342.4226.49662@psf.io> https://hg.python.org/devguide/rev/43d842419188 changeset: 783:43d842419188 user: Ezio Melotti date: Tue Jan 12 10:33:37 2016 +0200 summary: #19316: fix wording in the compiler.rst page. files: compiler.rst | 36 +++++++++++++----------------------- 1 files changed, 13 insertions(+), 23 deletions(-) diff --git a/compiler.rst b/compiler.rst --- a/compiler.rst +++ b/compiler.rst @@ -39,10 +39,10 @@ implementation laid out in the Dragon Book [Aho86]_. The grammar file for Python can be found in Grammar/Grammar with the -numeric value of grammar rules are stored in Include/graminit.h. The +numeric value of grammar rules stored in Include/graminit.h. The numeric values for types of tokens (literal tokens, such as ``:``, -numbers, etc.) are kept in Include/token.h). The parse tree made up of -``node *`` structs (as defined in Include/node.h). +numbers, etc.) are kept in Include/token.h. The parse tree is made up +of ``node *`` structs (as defined in Include/node.h). Querying data from the node structs can be done with the following macros (which are all defined in Include/node.h): @@ -65,14 +65,14 @@ retrieve the line number of the source code that led to the creation of the parse rule; defined in Python/ast.c -To tie all of this example, consider the rule for 'while':: +For example, consider the rule for 'while':: while_stmt: 'while' test ':' suite ['else' ':' suite] The node representing this will have ``TYPE(node) == while_stmt`` and -the number of children can be 4 or 7 depending on if there is an 'else' -statement. To access what should be the first ':' and require it be an -actual ':' token, `(REQ(CHILD(node, 2), COLON)``. +the number of children can be 4 or 7 depending on whether there is an +'else' statement. ``REQ(CHILD(node, 2), COLON)`` can be used to access +what should be the first ``:`` and require it be an actual ``:`` token. Abstract Syntax Trees (AST) @@ -175,7 +175,7 @@ PyArena_New() will create a new arena. The returned PyArena structure will store pointers to all memory given to it. This does the bookkeeping of what memory needs to be freed when the compiler is finished with the memory it used. -That freeing is done with PyArena_Free(). This needs to only be called in +That freeing is done with PyArena_Free(). This only needs to be called in strategic areas where the compiler exits. As stated above, in general you should not have to worry about memory @@ -209,7 +209,7 @@ you need to watch out for the ':' token to find the end of the conditional). The functions called to generate AST nodes from the parse tree all have -the name ast_for_xx where xx is what the grammar rule that the function +the name ast_for_xx where xx is the grammar rule that the function handles (alias_for_import_name is the exception to this). These in turn call the constructor functions as defined by the ASDL grammar and contained in Python/Python-ast.c (which was generated by @@ -218,7 +218,7 @@ Function and macros for creating and using ``asdl_seq *`` types as found -in Python/asdl.c and Include/asdl.h: +in Python/asdl.c and Include/asdl.h are as follows: ``asdl_seq_new()`` Allocate memory for an asdl_seq for the specified length @@ -435,8 +435,8 @@ Python-ast.c Creates C structs corresponding to the ASDL types. Also - contains code for marshaling AST nodes (core ASDL types have - marshaling code in asdl.c). "File automatically generated by + contains code for marshalling AST nodes (core ASDL types have + marshalling code in asdl.c). "File automatically generated by Parser/asdl_c.py". This file must be committed separately after every grammar change is committed since the __version__ value is set to the latest grammar change revision number. @@ -444,7 +444,7 @@ asdl.c Contains code to handle the ASDL sequence type. Also has code to handle marshalling the core ASDL types, such as number and - identifier. used by Python-ast.c for marshaling AST nodes. + identifier. Used by Python-ast.c for marshalling AST nodes. ast.c Converts Python's parse tree into the abstract syntax tree. @@ -504,16 +504,6 @@ opcode.py One of the files that must be modified if Include/opcode.h is. - compiler/ - - pyassem.py - One of the files that must be modified if Include/opcode.h is - changed. - - pycodegen.py - One of the files that must be modified if Include/opcode.h is - changed. - Known Compiler-related Experiments ---------------------------------- -- Repository URL: https://hg.python.org/devguide From solipsis at pitrou.net Tue Jan 12 03:42:20 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 12 Jan 2016 08:42:20 +0000 Subject: [Python-checkins] Daily reference leaks (4f1a0e5deca6): sum=6 Message-ID: <20160112084220.27386.39495@psf.io> results for 4f1a0e5deca6 on branch "default" -------------------------------------------- test_collections leaked [2, 0, 0] references, sum=2 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogRSN6pV', '--timeout', '7200'] From python-checkins at python.org Tue Jan 12 04:04:02 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 09:04:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE5MDA2OiBmaXgg?= =?utf-8?q?wording_in_unittest_docs=2E?= Message-ID: <20160112090402.27374.85428@psf.io> https://hg.python.org/cpython/rev/ddb8fa84a119 changeset: 99873:ddb8fa84a119 branch: 2.7 parent: 99870:113c524aed93 user: Ezio Melotti date: Tue Jan 12 11:03:10 2016 +0200 summary: #19006: fix wording in unittest docs. files: Doc/library/unittest.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -783,8 +783,9 @@ .. _assert-methods: - The :class:`TestCase` class provides a number of methods to check for and - report failures, such as: + The :class:`TestCase` class provides several assert methods to check for and + report failures. The following table lists the most commonly used methods + (see the tables below for more assert methods): +-----------------------------------------+-----------------------------+---------------+ | Method | Checks that | New in | -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 04:04:02 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 09:04:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzE5MDA2OiBmaXgg?= =?utf-8?q?wording_in_unittest_docs=2E?= Message-ID: <20160112090402.27382.49932@psf.io> https://hg.python.org/cpython/rev/84f34e8685de changeset: 99874:84f34e8685de branch: 3.5 parent: 99871:01ab138f4133 user: Ezio Melotti date: Tue Jan 12 11:03:31 2016 +0200 summary: #19006: fix wording in unittest docs. files: Doc/library/unittest.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -762,8 +762,9 @@ .. _assert-methods: - The :class:`TestCase` class provides a number of methods to check for and - report failures, such as: + The :class:`TestCase` class provides several assert methods to check for and + report failures. The following table lists the most commonly used methods + (see the tables below for more assert methods): +-----------------------------------------+-----------------------------+---------------+ | Method | Checks that | New in | @@ -884,7 +885,7 @@ - It is also possible to check the production of exceptions, warnings and + It is also possible to check the production of exceptions, warnings, and log messages using the following methods: +---------------------------------------------------------+--------------------------------------+------------+ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 04:04:03 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 09:04:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIzE5MDA2OiBtZXJnZSB3aXRoIDMuNS4=?= Message-ID: <20160112090402.59576.53335@psf.io> https://hg.python.org/cpython/rev/86c8d65d188f changeset: 99875:86c8d65d188f parent: 99872:9442b4a60379 parent: 99874:84f34e8685de user: Ezio Melotti date: Tue Jan 12 11:03:54 2016 +0200 summary: #19006: merge with 3.5. files: Doc/library/unittest.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -762,8 +762,9 @@ .. _assert-methods: - The :class:`TestCase` class provides a number of methods to check for and - report failures, such as: + The :class:`TestCase` class provides several assert methods to check for and + report failures. The following table lists the most commonly used methods + (see the tables below for more assert methods): +-----------------------------------------+-----------------------------+---------------+ | Method | Checks that | New in | @@ -884,7 +885,7 @@ - It is also possible to check the production of exceptions, warnings and + It is also possible to check the production of exceptions, warnings, and log messages using the following methods: +---------------------------------------------------------+--------------------------------------+------------+ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 04:28:08 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 09:28:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIzI2MDAxOiBtZXJnZSB3aXRoIDMuNS4=?= Message-ID: <20160112092808.27374.17657@psf.io> https://hg.python.org/cpython/rev/4b434a4770a9 changeset: 99877:4b434a4770a9 parent: 99875:86c8d65d188f parent: 99876:5a2692911a43 user: Ezio Melotti date: Tue Jan 12 11:27:58 2016 +0200 summary: #26001: merge with 3.5. files: Doc/tutorial/inputoutput.rst | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -271,10 +271,11 @@ ``f`` has already been created. To read a file's contents, call ``f.read(size)``, which reads some quantity of -data and returns it as a string or bytes object. *size* is an optional numeric -argument. When *size* is omitted or negative, the entire contents of the file -will be read and returned; it's your problem if the file is twice as large as -your machine's memory. Otherwise, at most *size* bytes are read and returned. +data and returns it as a string (in text mode) or bytes object (in binary mode). +*size* is an optional numeric argument. When *size* is omitted or negative, the +entire contents of the file will be read and returned; it's your problem if the +file is twice as large as your machine's memory. Otherwise, at most *size* bytes +are read and returned. If the end of the file has been reached, ``f.read()`` will return an empty string (``''``). :: @@ -315,11 +316,11 @@ >>> f.write('This is a test\n') 15 -To write something other than a string, it needs to be converted to a string -first:: +Other types of objects need to be converted -- either to a string (in text mode) +or a bytes object (in binary mode) -- before writing them:: >>> value = ('the answer', 42) - >>> s = str(value) + >>> s = str(value) # convert the tuple to string >>> f.write(s) 18 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 04:28:09 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 09:28:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI2MDAxOiBtZW50?= =?utf-8?q?ion_in_the_tutorial_that_files_in_binary_mode_expect_bytes=2C_n?= =?utf-8?q?ot_str=2E?= Message-ID: <20160112092807.59574.56552@psf.io> https://hg.python.org/cpython/rev/5a2692911a43 changeset: 99876:5a2692911a43 branch: 3.5 parent: 99874:84f34e8685de user: Ezio Melotti date: Tue Jan 12 11:27:30 2016 +0200 summary: #26001: mention in the tutorial that files in binary mode expect bytes, not str. files: Doc/tutorial/inputoutput.rst | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/tutorial/inputoutput.rst b/Doc/tutorial/inputoutput.rst --- a/Doc/tutorial/inputoutput.rst +++ b/Doc/tutorial/inputoutput.rst @@ -271,10 +271,11 @@ ``f`` has already been created. To read a file's contents, call ``f.read(size)``, which reads some quantity of -data and returns it as a string or bytes object. *size* is an optional numeric -argument. When *size* is omitted or negative, the entire contents of the file -will be read and returned; it's your problem if the file is twice as large as -your machine's memory. Otherwise, at most *size* bytes are read and returned. +data and returns it as a string (in text mode) or bytes object (in binary mode). +*size* is an optional numeric argument. When *size* is omitted or negative, the +entire contents of the file will be read and returned; it's your problem if the +file is twice as large as your machine's memory. Otherwise, at most *size* bytes +are read and returned. If the end of the file has been reached, ``f.read()`` will return an empty string (``''``). :: @@ -315,11 +316,11 @@ >>> f.write('This is a test\n') 15 -To write something other than a string, it needs to be converted to a string -first:: +Other types of objects need to be converted -- either to a string (in text mode) +or a bytes object (in binary mode) -- before writing them:: >>> value = ('the answer', 42) - >>> s = str(value) + >>> s = str(value) # convert the tuple to string >>> f.write(s) 18 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 04:36:37 2016 From: python-checkins at python.org (ezio.melotti) Date: Tue, 12 Jan 2016 09:36:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI1NTc0OiBiYWNr?= =?utf-8?q?port_glossary_doc_fix_about_hash_and_id_of_user-defined_classes?= =?utf-8?q?=2E?= Message-ID: <20160112093634.69139.46905@psf.io> https://hg.python.org/cpython/rev/499ee454eecc changeset: 99878:499ee454eecc branch: 2.7 parent: 99873:ddb8fa84a119 user: Ezio Melotti date: Tue Jan 12 11:36:25 2016 +0200 summary: #25574: backport glossary doc fix about hash and id of user-defined classes. files: Doc/glossary.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -346,8 +346,8 @@ All of Python's immutable built-in objects are hashable, while no mutable containers (such as lists or dictionaries) are. Objects which are instances of user-defined classes are hashable by default; they all - compare unequal (except with themselves), and their hash value is their - :func:`id`. + compare unequal (except with themselves), and their hash value is derived + from their :func:`id`. IDLE An Integrated Development Environment for Python. IDLE is a basic editor -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 12 06:23:53 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 11:23:53 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510?= Message-ID: <20160112112353.27378.90296@psf.io> https://hg.python.org/peps/rev/20c7dec855dc changeset: 6169:20c7dec855dc user: Victor Stinner date: Tue Jan 12 12:11:53 2016 +0100 summary: PEP 510 * convert Python functions to C functions * replace "specialized functions" with "specialized codes" to avoid confusion * guard functions now use 0 result as success to simplify the code files: pep-0510.txt | 311 +++++++++++++++++++++++++------------- 1 files changed, 201 insertions(+), 110 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -1,5 +1,5 @@ PEP: 510 -Title: Specialized functions with guards +Title: Specialize functions Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner @@ -13,8 +13,9 @@ Abstract ======== -Add a private API to CPython to add specialized functions with guards to -functions, to support static optimizers respecting the Python semantics. +Add functions to the Python C API to specialize pure Python functions: +add specialized codes with guards. It allows to implement static +optimizers respecting the Python semantics. Rationale @@ -29,15 +30,16 @@ semantics requires to detect when "something changes", we will call these checks "guards". -This PEP proposes to add an API to add specialized functions with guards -to a function. When the function is called, the specialized function is -used if nothing changed, otherwise use the original bytecode. +This PEP proposes to add a public API to the Python C API to add +specialized codes with guards to a function. When the function is +called, a specialized code is used if nothing changed, otherwise use the +original bytecode. Even if guards help to respect most parts of the Python semantics, it's -really hard to optimize Python without making subtle changes on the -exact behaviour. CPython has a long history and many applications rely -on implementation details. A compromise must be found between -"everything is mutable" and performance. +hard to optimize Python without making subtle changes on the exact +behaviour. CPython has a long history and many applications rely on +implementation details. A compromise must be found between "everything +is mutable" and performance. Writing an optimizer is out of the scope of this PEP. @@ -101,13 +103,35 @@ 2011. -Example -======= +Examples +======== + +Following examples are not written to show powerful optimizations +promising important speedup, but to be short and easy to understand, +just to explain the principle. + +Hypothetical myoptimizer module +------------------------------- + +Examples in this PEP uses an hypothetical ``myoptimizer`` module which +provides the following functions and types: + +* ``specialize(func, code, guards)``: add the specialized code `code` + with guards `guards` to the function `func` +* ``get_specialized(func)``: get the list of specialized codes as a list + of ``(code, guards)`` tuples where `code` is a callable or code object + and `guards` is a list of a guards +* ``GuardBuiltins(name)``: guard watching for + ``builtins.__dict__[name]`` and ``globals()[name]``. The guard fails + if ``builtins.__dict__[name]`` is replaced, or if ``globals()[name]`` + is set. + Using bytecode -------------- -Replace ``chr(65)`` with ``"A"``:: +Add specialized bytecode where the call to the pure builtin function +``chr(65)`` is replaced with its result ``"A"``:: import myoptimizer @@ -117,18 +141,21 @@ def fast_func(): return "A" - func._specialize(fast_func.__code__, [myoptimizer.GuardBuiltins("chr")]) + myoptimizer.specialize(func, fast_func.__code__, + [myoptimizer.GuardBuiltins("chr")]) del fast_func +Example showing the behaviour of the guard:: + print("func(): %s" % func()) - print("#specialized: %s" % len(func._get_specialized())) + print("#specialized: %s" % len(myoptimizer.get_specialized(func))) print() import builtins builtins.chr = lambda obj: "mock" print("func(): %s" % func()) - print("#specialized: %s" % len(func._get_specialized())) + print("#specialized: %s" % len(myoptimizer.get_specialized(func))) Output:: @@ -138,41 +165,40 @@ func(): mock #specialized: 0 -The hypothetical ``myoptimizer.GuardBuiltins("len")`` is a guard on the -builtin ``len()`` function and the ``len`` name in the global namespace. -The guard fails if the builtin function is replaced or if a ``len`` name -is defined in the global namespace. +The first call uses the specialized bytecode which returns the string +``"A"``. The second call removes the specialized code because the +builtin ``chr()`` function was replaced, and executes the original +bytecode calling ``chr(65)``. -The first call returns directly the string ``"A"``. The second call -removes the specialized function because the builtin ``chr()`` function -was replaced, and executes the original bytecode - -On a microbenchmark, calling the specialized function takes 88 ns, -whereas the original bytecode takes 145 ns (+57 ns): 1.6 times as fast. +On a microbenchmark, calling the specialized bytecode takes 88 ns, +whereas the original function takes 145 ns (+57 ns): 1.6 times as fast. Using builtin function ---------------------- -Replace a slow Python function calling ``chr(obj)`` with a direct call -to the builtin ``chr()`` function:: +Add the C builtin ``chr()`` function as the specialized code instead of +a bytecode calling ``chr(obj)``:: import myoptimizer def func(arg): return chr(arg) - func._specialize(chr, [myoptimizer.GuardBuiltins("chr")]) + myoptimizer.specialize(func, chr, + [myoptimizer.GuardBuiltins("chr")]) + +Example showing the behaviour of the guard:: print("func(65): %s" % func(65)) - print("#specialized: %s" % len(func._get_specialized())) + print("#specialized: %s" % len(myoptimizer.get_specialized(func))) print() import builtins builtins.chr = lambda obj: "mock" print("func(65): %s" % func(65)) - print("#specialized: %s" % len(func.get_specialized())) + print("#specialized: %s" % len(myoptimizer.get_specialized(func))) Output:: @@ -182,88 +208,165 @@ func(): mock #specialized: 0 -The first call returns directly the builtin ``chr()`` function (without -creating a Python frame). The second call removes the specialized -function because the builtin ``chr()`` function was replaced, and -executes the original bytecode. +The first call calls the C builtin ``chr()`` function (without creating +a Python frame). The second call removes the specialized code because +the builtin ``chr()`` function was replaced, and executes the original +bytecode. -On a microbenchmark, calling the specialized function takes 95 ns, -whereas the original bytecode takes 155 ns (+60 ns): 1.6 times as fast. -Calling directly ``chr(65)`` takes 76 ns. +On a microbenchmark, calling the C builtin takes 95 ns, whereas the +original bytecode takes 155 ns (+60 ns): 1.6 times as fast. Calling +directly ``chr(65)`` takes 76 ns. -Python Function Call -==================== +Choose the specialized code +=========================== -Pseudo-code to call a Python function having specialized functions with -guards:: +Pseudo-code to choose the specialized code to call a pure Python +function:: - def call_func(func, *args, **kwargs): - # by default, call the regular bytecode - code = func.__code__.co_code - specialized = func.get_specialized() + def call_func(func, args, kwargs): + specialized = myoptimizer.get_specialized(func) nspecialized = len(specialized) - index = 0 while index < nspecialized: - guard = specialized[index].guard - # pass arguments, some guards need them - check = guard(args, kwargs) - if check == 1: - # guard succeeded: we can use the specialized function - code = specialized[index].code - break - elif check == -1: - # guard will always fail: remove the specialized function + specialized_code, guards = specialized[index] + + for guard in guards: + check = guard(args, kwargs) + if check: + break + + if not check: + # all guards succeeded: + # use the specialized code + return specialized_code + elif check == 1: + # a guard failed temporarely: + # try the next specialized code + index += 1 + else: + assert check == 2 + # a guard will always fail: + # remove the specialized code del specialized[index] - elif check == 0: - # guard failed temporarely - index += 1 - # code can be a code object or any callable object - execute_code(code, args, kwargs) + # if a guard of each specialized code failed, or if the function + # has no specialized code, use original bytecode + code = func.__code__ + Changes ======= -* Add two new private methods to functions: +Changes to the Python C API: - * ``_specialize(code, guards: list)``: add specialized - function with guard. `code` is a code object (ex: - ``func2.__code__``) or any callable object (ex: the builtin - ``len()`` function). The specialization can be ignored if a guard - already fails or for other reasons (ex: the implementation of Python - does not implement this feature). Return ``False`` is the - specialized function was ignored, return ``True`` otherwise. +* Add a ``PyFuncGuardObject`` object and a ``PyFuncGuard_Type`` type +* Add a ``PySpecializedFunc`` structure +* Add the following fields to the ``PyFunctionObject`` structure:: - * ``_get_specialized()``: get the list of specialized functions with - guards. Return a list of ``(func, guards)`` tuples where func is the - specialized function and guards is a list of guards. Return an empty - list if the function was never specialized. + Py_ssize_t nb_specialized; + PyObject *specialized; /* array of PySpecializedFunc objects */ -* Add a private ``PyFuncGuard`` Python type. It requires to implement a - C ``check()`` function, with an optional C ``init()`` function. API: +* Add function methods: - * ``int init(PyObject *guard, PyObject *func)``: initialize a guard, - *func* is the function to which the specialized function will be - attached. Result: + * ``PyFunction_Specialize()`` + * ``PyFunction_GetSpecializedCodes()`` + * ``PyFunction_GetSpecializedCode()`` - * return ``1`` on success - * return ``0`` if the guard will always fail (the specialization must be - ignored) - * raise an exception and return ``-1`` on error +None of these function and types are exposed at the Python level. - * ``int check(PyObject *guard, PyObject **stack, int na, int nk)``: - check the guard. Result: +All these additions are explicitly excluded of the stable ABI. - * return 2 on success - * return 1 if the guard failed temporarely - * return 0 if the guard will always fail - * raise an exception and return -1 on error +When a function code is replaced (``func.__code__ = new_code``), all +specialized codes and guards are removed. - * A guard can be called in Python with parameters, it returns the - result of the guard check. +When a function is serialized ``pickle``, specialized codes and guards are +ignored (not serialized). Specialized codes and guards are not stored in +``.pyc`` files but created and registered at runtime, when a module is +loaded. + + +Function guard +-------------- + +Add a function guard object:: + + typedef struct { + PyObject ob_base; + int (*init) (PyObject *guard, PyObject *func); + int (*check) (PyObject *guard, PyObject **stack, int na, int nk); + } PyFuncGuardObject; + +The ``init()`` function initializes a guard: + +* Return ``0`` on success +* Return ``1`` if the guard will always fail: ``PyFunction_Specialize()`` + must ignore the specialized code +* Raise an exception and return ``-1`` on error + + +The ``check()`` function checks a guard: + +* Return ``0`` on success +* Return ``1`` if the guard failed temporarely +* Return ``2`` if the guard will always fail: the specialized code must + be removed +* Raise an exception and return ``-1`` on error + +*stack* is an array of arguments: indexed arguments followed by (*key*, +*value*) pairs of keyword arguments. *na* is the number of indexed +arguments. *nk* is the number of keyword arguments: the number of (*key*, +*value*) pairs. `stack` contains ``na + nk * 2`` objects. + + +Specialized code +---------------- + +Add a specialized code structure:: + + typedef struct { + PyObject *code; /* callable or code object */ + Py_ssize_t nb_guard; + PyObject **guards; /* PyFuncGuardObject objects */ + } PySpecializedCode; + + +Function methods +---------------- + +Add a function method to specialize the function, add a specialized code +with guards:: + + int PyFunction_Specialize(PyObject *func, + PyObject *code, PyObject *guards) + +Result: + +* Return ``0`` on success +* Return ``1`` if the specialization has been ignored +* Raise an exception and return ``-1`` on error + +Add a function method to get the list of specialized codes:: + + PyObject* PyFunction_GetSpecializedCodes(PyObject *func) + +Return a list of (*code*, *guards*) tuples where *code* is a callable or +code object and *guards* is a list of ``PyFuncGuard`` objects. Raise an +exception and return ``NULL`` on error. + +Add a function method to get the specialized code:: + + PyObject* PyFunction_GetSpecializedCode(PyObject *func, + PyObject **stack, + int na, int nk) + +See ``check()`` function of guards for *stack*, *na* and *nk* arguments. +Return a callable or a code object on success. Raise an exception and +return ``NULL`` on error. + +Benchmark +--------- Microbenchmark on ``python3.6 -m timeit -s 'def f(): pass' 'f()'`` (best of 3 runs): @@ -275,30 +378,18 @@ a Python function without specialization. -Behaviour -========= - -When a function code is replaced (``func.__code__ = new_code``), all -specialized functions are removed. - -When a function is serialized ``pickle``, specialized functions and -guards are ignored (not serialized). Specialized functions and guards -are not stored in ``.pyc`` files but created and registered at runtime, -when a module is loaded. - - Other implementations of Python =============================== -This PEP is designed to be implemented in C for CPython. +This PEP only contains changes to the Python C API, the Python API is +unchanged. Other implementations of Python are free to not implement new +additions, or implement added functions as no-op: -Other implementations of Python are free to not implement added private -function methods. - -Or they can implement a ``_specialize()`` method which always ignores -the specialized function (in short, do nothing and always return -``False``) and a ``_get_specialized()`` method which always returns an -empty list. +* ``PyFunction_Specialize()``: always return ``1`` (the specialization + has been ignored) +* ``PyFunction_GetSpecializedCodes()``: always return an empty list +* ``PyFunction_GetSpecializedCode()``: return the function code object, + as the existing ``PyFunction_GET_CODE()`` macro Discussion -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 08:57:45 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 13:57:45 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_reserve_PEP_number_511_for_th?= =?utf-8?q?e_API_of_AST_transformers?= Message-ID: <20160112135738.59592.38348@psf.io> https://hg.python.org/peps/rev/3525ca62966d changeset: 6170:3525ca62966d user: Victor Stinner date: Tue Jan 12 14:57:02 2016 +0100 summary: reserve PEP number 511 for the API of AST transformers files: pep-0511.txt | 28 ++++++++++++++++++++++++++++ 1 files changed, 28 insertions(+), 0 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt new file mode 100644 --- /dev/null +++ b/pep-0511.txt @@ -0,0 +1,28 @@ +PEP: 511 +Title: API for AST transformers +Version: $Revision$ +Last-Modified: $Date$ +Author: Victor Stinner +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 4-January-2016 +Python-Version: 3.6 + +Abstract +======== + +Propose an API to support AST transformers. + + +Work In Progress +================ + +This document is just to reserve a PEP number, the PEP is not ready for a +public review yet. + + +Copyright +========= + +This document has been placed in the public domain. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 09:18:44 2016 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 12 Jan 2016 14:18:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue25347_-_Format_the_er?= =?utf-8?q?ror_message_output_of_mock=27s_assert=5Fhas=5Fcalls_method=2E?= Message-ID: <20160112141844.11385.38848@psf.io> https://hg.python.org/cpython/rev/77d24f51effc changeset: 99879:77d24f51effc parent: 99877:4b434a4770a9 user: Senthil Kumaran date: Tue Jan 12 06:18:32 2016 -0800 summary: Issue25347 - Format the error message output of mock's assert_has_calls method. Patch contributed by Robert Zimmerman. files: Lib/unittest/mock.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -820,7 +820,7 @@ if expected not in all_calls: raise AssertionError( 'Calls not found.\nExpected: %r\n' - 'Actual: %r' % (calls, self.mock_calls) + 'Actual: %r' % (_CallList(calls), self.mock_calls) ) from cause return -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Jan 12 09:22:53 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 12 Jan 2016 14:22:53 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-12 Message-ID: Results for project Python 2.7, build date 2016-01-12 04:01:46 +0000 commit: d208a2ada67c1fca3f7311330e1a6afb60cd69ee previous commit: 84fdbdd483a3551c3342c2a9c5a8aee83731a096 revision date: 2016-01-11 22:11:24 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.38% -0.07% 4.60% 4.14% :-) pybench 0.14% 0.39% 6.31% 4.60% :-( regex_v8 1.07% 0.17% -2.27% 9.87% :-) nbody 0.12% -0.71% 7.16% 3.64% :-) json_dump_v2 0.22% 0.25% 5.15% 9.55% :-( normal_startup 1.86% -0.24% -5.65% 2.57% :-| ssbench 0.21% -0.17% 0.44% 1.42% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-2-7-2016-01-12/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Jan 12 09:21:36 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 12 Jan 2016 14:21:36 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-12 Message-ID: <11cadef6-69f4-4c88-9a3a-e1792b468141@irsmsx151.ger.corp.intel.com> Results for project Python default, build date 2016-01-12 03:08:17 +0000 commit: 4f1a0e5deca6ec0b0af7439862a22ba35b30abe0 previous commit: 8ee02ec9b6340d4da9a6ee0a8a2e9fbcfa296372 revision date: 2016-01-12 02:05:18 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.23% -0.32% 11.43% 14.45% :-| pybench 0.07% -0.14% -1.79% 5.59% :-( regex_v8 2.58% 0.21% -4.93% 0.42% :-| nbody 0.20% 0.15% 1.67% 5.64% :-( json_dump_v2 0.22% -0.75% -2.70% 11.79% :-| normal_startup 0.73% 0.14% -1.19% 4.73% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-default-2016-01-12/ Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Jan 12 09:32:39 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 14:32:39 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_title?= Message-ID: <20160112143239.27376.83766@psf.io> https://hg.python.org/peps/rev/649f49a07c1e changeset: 6171:649f49a07c1e user: Victor Stinner date: Tue Jan 12 15:32:35 2016 +0100 summary: PEP 510: title files: pep-0510.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -1,5 +1,5 @@ PEP: 510 -Title: Specialize functions +Title: Specialize functions with guards Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 15:04:05 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 20:04:05 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_fix_typos_reported?= =?utf-8?q?_by_Jim_J=2E_Jewett?= Message-ID: <20160112200402.105167.67812@psf.io> https://hg.python.org/peps/rev/bfb63df08552 changeset: 6172:bfb63df08552 user: Victor Stinner date: Tue Jan 12 21:03:57 2016 +0100 summary: PEP 509: fix typos reported by Jim J. Jewett files: pep-0509.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -128,7 +128,7 @@ Unladen Swallow --------------- -Even if dictionary version was not explicitly mentionned, optimization +Even if dictionary version was not explicitly mentioned, optimization globals and builtins lookup was part of the Unladen Swallow plan: "Implement one of the several proposed schemes for speeding lookups of globals and builtins." Source: `Unladen Swallow ProjectPlan @@ -212,7 +212,7 @@ any overhead on dictionary operations. When the version does not change, ``PyDict_GetItem()`` takes 14.8 ns for -a dictioanry lookup, whereas a guard check only takes 3.8 ns. Moreover, +a dictionary lookup, whereas a guard check only takes 3.8 ns. Moreover, a guard can watch for multiple keys. For example, for an optimization using 10 global variables in a function, 10 dictionary lookups costs 148 ns, whereas the guard still only costs 3.8 ns when the version does not -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 16:47:01 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 21:47:01 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_509=3A_mention_explicitly?= =?utf-8?q?_that_PyDictObject_is_not_part_of_the_stable_ABI?= Message-ID: <20160112214701.27366.64280@psf.io> https://hg.python.org/peps/rev/655f4e82d7b2 changeset: 6174:655f4e82d7b2 user: Victor Stinner date: Tue Jan 12 22:46:56 2016 +0100 summary: PEP 509: mention explicitly that PyDictObject is not part of the stable ABI files: pep-0509.txt | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/pep-0509.txt b/pep-0509.txt --- a/pep-0509.txt +++ b/pep-0509.txt @@ -158,6 +158,9 @@ values are compared by identity, not by their content; the version can be incremented multiple times +.. note:: + The ``PyDictObject`` structure is not part of the stable ABI. + Example using an hypothetical ``dict_get_version(dict)`` function:: >>> d = {} -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 16:47:06 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 21:47:06 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_minor_fixes?= Message-ID: <20160112214701.105173.93561@psf.io> https://hg.python.org/peps/rev/5e7bdef3f3d3 changeset: 6173:5e7bdef3f3d3 user: Victor Stinner date: Tue Jan 12 22:46:36 2016 +0100 summary: PEP 510: minor fixes files: pep-0510.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -262,11 +262,11 @@ Changes to the Python C API: * Add a ``PyFuncGuardObject`` object and a ``PyFuncGuard_Type`` type -* Add a ``PySpecializedFunc`` structure +* Add a ``PySpecializedCode`` structure * Add the following fields to the ``PyFunctionObject`` structure:: Py_ssize_t nb_specialized; - PyObject *specialized; /* array of PySpecializedFunc objects */ + PySpecializedCode *specialized; * Add function methods: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 18:02:38 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 12 Jan 2016 23:02:38 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_rephrase_introduct?= =?utf-8?q?ion_on_PyPy?= Message-ID: <20160112230234.121154.92423@psf.io> https://hg.python.org/peps/rev/b463ab37c04d changeset: 6175:b463ab37c04d user: Victor Stinner date: Wed Jan 13 00:02:21 2016 +0100 summary: PEP 510: rephrase introduction on PyPy suggestion of Yury Selivanov files: pep-0510.txt | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -55,10 +55,11 @@ * `Pyjion `_ Numba is specific to numerical computation. Pyston and Pyjion are still -young. PyPy is the most complete Python interpreter, it is much faster -than CPython and has a very good compatibility with CPython (it respects -the Python semantics). There are still issues with Python JIT compilers -which avoid them to be widely used instead of CPython. +young. PyPy is the most complete Python interpreter, it is generally +faster than CPython in micro- and many macro-benchmarks and has a very +good compatibility with CPython (it respects the Python semantics). +There are still issues with Python JIT compilers which avoid them to be +widely used instead of CPython. Many popular libraries like numpy, PyGTK, PyQt, PySide and wxPython are implemented in C or C++ and use the Python C API. To have a small memory -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 12 19:26:36 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 13 Jan 2016 00:26:36 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_add_the_text_of_th?= =?utf-8?q?e_PEP?= Message-ID: <20160113002636.27386.66495@psf.io> https://hg.python.org/peps/rev/3a461b846ace changeset: 6176:3a461b846ace user: Victor Stinner date: Wed Jan 13 00:57:21 2016 +0100 summary: PEP 511: add the text of the PEP files: pep-0511.txt | 297 ++++++++++++++++++++++++++++++++++++++- 1 files changed, 292 insertions(+), 5 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -12,14 +12,301 @@ Abstract ======== -Propose an API to support AST transformers. +Propose an API to support AST transformers. Add also ``-o OPTIM_TAG`` +command line option to change ``.pyc`` filenames. Raise an +``ImportError`` exception on import if the ``.pyc`` file is missing and +the AST transformers required to transform the code are missing. +AST transformers are not needed code transformed ahead of time (loaded +from ``.pyc`` files). -Work In Progress -================ +Rationale +========= -This document is just to reserve a PEP number, the PEP is not ready for a -public review yet. +Python does not provide a standard way to transform the code. Projects +transforming the code use various hooks. The MacroPy project uses an +import hook: it adds its own module finder in ``sys.meta_path`` to +hook its AST transformer. Another option is to monkey-patch the +builtin ``compile()`` function. There are even more options to +hook a code transformer. + +Transforming the code allows to extend the Python language for specific +use cases. Transforming an Abstract Syntax Tree (AST) is a convenient +way to implement an optimizer. It's easier to work on the AST than +working on the bytecode, AST contains more information and is more high +level. + +Python 3.6 optimizes the code using a peephole optimizer. By +definition, a peephole optimizer has a narrow view of the code and so +can only implement basic optimizations. The optimizer rewrites the +bytecode. It is difficult to enhance it, because it written in C. + +This PEP proposes to add an API to register AST transformers. + +A new ``-o OPTIM_TAG`` command line option is added to only load +transformed code: it changes the name of searched ``.pyc`` files. If the +``.pyc`` file of a module is missing and the ``.py`` is available, an +``ImportError`` exception is raised import if the AST transformers +required to transform the code are missing. The import behaviour with +the default optimizer tag (``'opt'``) is unchanged. + +The transformation can done ahead of time. It allows to implement +powerful but expensive transformations. + + +Use Cases +========= + +This section give examples of use cases explaining when and how AST +transformers will be used. + +Interactive interpreter +----------------------- + +It will be possible to use AST transformers with the interactive +interpreter which is popular in Python and commonly used to demonstrate +Python. + +The code is transformed at runtime and so the interpreter can be slower +when expensive AST transformers are used. + +Build a transformed package +--------------------------- + +It will be possible to build a package of the transformed code. + +A transformer can have a configuration. The configuration is not stored +in the package. + +All ``.pyc`` files of the package must be transformed with the same AST +transformers and the same transformers configuration. + +It is possible to build different ``.pyc`` files using different +optimizer tags. Example: ``fat`` for the default configuration and +``fat_inline`` for a different configuration with function inlining +enabled. + +A package can contain ``.pyc`` files with different optimizer tags. + + +Install a package containing transformed .pyc files +--------------------------------------------------- + +It will be possible to install a package which contains transformed +``.pyc`` files. + +All ``.pyc`` files with any optimizer tag contained in the package are +installed, not only for the current optimizer tag. + + +Build .pyc files when installing a package +------------------------------------------ + +If a package does not contain any ``.pyc`` files of the current +optimizer tag (or some ``.pyc`` files are missing), the ``.pyc`` are +created during the installation. + +AST transformers of the optimizer tag are required. Otherwise, the +installation fails with an error. + + +Execute transformed code +------------------------ + +It will be possible to execute transformed code. + +Raise an ``ImportError`` exception on import if the ``.pyc`` file of the +current optimizer tag is missing and the AST transformers required to +transform the code are missing. + +The interesting point here is that AST transformers are not needed to +execute the transformed code if all required ``.pyc`` files are already +available. + + +Changes +======= + +API to support AST transformers: + +* Add ``sys.ast_transformers``: list of AST transformers used to rewrite + an AST tree. The list is empty by default: no AST transformer. +* Add ``sys.implementation.ast_transformers``: name of AST + transformers registered in ``sys.ast_transformers`` +* Add ``sys.implementation.optim_tag`` (``str``): optimization tag. + The default optimization tag is ``'opt'``. +* Use the optimizer tag in ``.pyc`` filenames in ``importlib``. + Remove also the special case for the optimizer level ``0`` with the + default optimizer tag ``'opt'`` to simplify the code. +* Add a new ``-o OPTIM_TAG`` command line option to set + ``sys.implementation.optim_tag`` + +.. note:: + FIXME: There is a design issue: ``sys.ast_transformers`` and + ``sys.implementation.ast_transformers`` can be inconsistent. + ``sys.implementation.ast_transformers`` is required at runtime in + some corner cases to have specific code depending if a specific AST + transformer was used. Do you have a better suggestion? + +API of an AST transformer (from ``sys.ast_transformers``): + +* An AST transformer is a callable object with the prototype:: + + def ast_transformer(tree, context): + ... + return tree + + where *tree* is an AST tree and *context* is an object with a + ``filename`` attribute (``str``). New attributes may be added to + *context* in the future. + +* It must return an AST tree. +* It must have a ``name`` attribute (``str``): short string used to identify an + optimizer. The name must not contain ``.`` (dot) nor ``-`` (dash) characters: + ``.`` is used to separated fields in a ``.pyc`` filename and ``-`` is used + to join AST transformer names to build the optimizer tag. +* The transformer is called after the creation of the AST by the parser + and before the compilation to bytecode +* It can modify the AST tree in place, or create a new AST tree. + +.. note:: + It would be nice to pass the fully qualified name of a module in the + *context* when an AST transformer is used to transform a module, but + it looks like the information is not available currently. + +AST transformer changes: + +* Add a new compiler flag ``PyCF_TRANSFORMED_AST`` to get the + transformed AST. ``PyCF_ONLY_AST`` returns the AST before the + transformers. +* Add ``ast.Constant``: this type is not emited by the compiler, but + can be used in an AST transformer to simplify the code. It does not + contain line number and column offset informations on tuple or + frozenset items. +* ``PyCodeObject.co_lnotab``: line number delta becomes signed to + support moving instructions (note: need to modify MAGIC_NUMBER in + importlib). +* Enhance the bytecode compiler to support ``tuple`` and ``frozenset`` + constants. Currently, ``tuple`` and ``frozenset`` constants are + created by the peephole transformer, after the bytecode compilation. +* ``marshal`` module: fix serialization of the empty frozenset singleton +* update ``Tools/parser/unparse.py`` to support the new ``ast.Constant`` + node type + + +Example +======= + +.pyc filenames +-------------- + +Example of ``.pyc`` filenames of the ``os`` module. + +With the default optimizer tag ``'opt'``: + +=========================== ================== +.pyc filename Optimization level +=========================== ================== +``os.cpython-36.opt-0.pyc`` 0 +``os.cpython-36.opt-1.pyc`` 1 +``os.cpython-36.opt-2.pyc`` 2 +=========================== ================== + +With the ``'fat'`` optimizer tag: + +=========================== ================== +.pyc filename Optimization level +=========================== ================== +``os.cpython-36.fat-0.pyc`` 0 +``os.cpython-36.fat-1.pyc`` 1 +``os.cpython-36.fat-2.pyc`` 2 +=========================== ================== + + +AST transformer +---------------- + +Scary AST transformer replacing all strings with ``"Ni! Ni! Ni!"``:: + + import ast + import sys + + + class KnightsWhoSayNi(ast.NodeTransformer): + def visit_Str(self, node): + node.s = 'Ni! Ni! Ni!' + return node + + + class ASTTransformer: + name = "knights_who_say_ni" + + def __call__(self, tree, context): + KnightsWhoSayNi().visit(tree) + return tree + + + # register the AST transformer + sys.ast_transformers.append(ASTTransformer()) + + # execute code which will be transformed by ast_transformer() + exec("print('Hello World!')") + +Output:: + + Ni! Ni! Ni! + + +Prior Art +========= + +AST optimizers +-------------- + +In 2011, Eugene Toder proposed to rewrite some peephole optimizations in +a new AST optimizer: issue #11549, `Build-out an AST optimizer, moving +some functionality out of the peephole optimizer +`_. The patch adds ``ast.Lit`` (it +was proposed to rename it to ``ast.Literal``). + +In 2012, Victor Stinner wrote the `astoptimizer +`_ project, an AST optimizer +implementing various optimizations. Most interesting optimizations break +the Python semantics since no guard is used to disable optimization if +something changes. + +Issue #17515: `Add sys.setasthook() to allow to use a custom AST +optimizer `_. + + +Python Preprocessors +-------------------- + +* `MacroPy `_: MacroPy is an + implementation of Syntactic Macros in the Python Programming Language. + MacroPy provides a mechanism for user-defined functions (macros) to + perform transformations on the abstract syntax tree (AST) of a Python + program at import time. +* `pypreprocessor `_: C-style + preprocessor directives in Python, like ``#define`` and ``#ifdef`` + + +Modify the bytecode +------------------- + +* `codetransformer `_: + Bytecode transformers for CPython inspired by the ``ast`` module?s + ``NodeTransformer``. +* `byteplay `_: Byteplay lets you + convert Python code objects into equivalent objects which are easy to + play with, and lets you convert those objects back into living Python + code objects. It's useful for applying crazy transformations on Python + functions, and is also useful in learning Python byte code + intricacies. See `byteplay documentation + `_. + +See also `BytecodeAssembler +`_. Copyright -- Repository URL: https://hg.python.org/peps From tjreedy at udel.edu Tue Jan 12 22:14:47 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 12 Jan 2016 22:14:47 -0500 Subject: [Python-checkins] peps: PEP 511: add the text of the PEP In-Reply-To: <20160113002636.27386.66495@psf.io> References: <20160113002636.27386.66495@psf.io> Message-ID: <5695C127.4020603@udel.edu> On 1/12/2016 7:26 PM, victor.stinner wrote: > PEP 511: add the text of the PEP > Abstract > ======== > > -Propose an API to support AST transformers. > +Propose an API to support AST transformers. Add also ``-o OPTIM_TAG`` > +command line option to change ``.pyc`` filenames. Raise an > +``ImportError`` exception on import if the ``.pyc`` file is missing and > +the AST transformers required to transform the code are missing. > +AST transformers are not needed code transformed ahead of time (loaded > +from ``.pyc`` files). This sentence is garbled. > +Transforming the code allows to extend the Python language for specific > +use cases. Transforming an Abstract Syntax Tree (AST) is a convenient > +way to implement an optimizer. It's easier to work on the AST than > +working on the bytecode, AST contains more information and is more high > +level. For many things, it should also be easier to work on the AST than the source code. But, it would be nice to have an AST tree pattern matcher, with capture groups, equivalent to re for linear text. > +This PEP proposes to add an API to register AST transformers. > + > +A new ``-o OPTIM_TAG`` command line option is added to only load > +transformed code: it changes the name of searched ``.pyc`` files. If the > +``.pyc`` file of a module is missing and the ``.py`` is available, an > +``ImportError`` exception is raised import if the AST transformers 'raised on import' > +required to transform the code are missing. The import behaviour with > +the default optimizer tag (``'opt'``) is unchanged. > + > +The transformation can done ahead of time. I presume you mean during the one-time compile and generate .pyc step. > It allows to implement powerful but expensive transformations. For many transformations, the expensive part is finding subtrees matching the applicable pattern. The Windows installer optionally runs compileall. Another minute for an install would not be too bad. In any case, stdlib modules are stable for months at a time. Ditto for installations of most 3rd party packages. Getting months of use from one time costs, added on to installs that may proceed while one does something else, is an important point. > +Use Cases > +========= > + > +This section give examples of use cases explaining when and how AST > +transformers will be used. > + > +Interactive interpreter > +----------------------- > + > +It will be possible to use AST transformers with the interactive > +interpreter which is popular in Python and commonly used to demonstrate > +Python. Possible, but limited to default inexpensive transformations with immediate payoff, such as the current peephole optimizer. > +The code is transformed at runtime and so the interpreter can be slower > +when expensive AST transformers are used. Expensive transformers should be turned off. I am not sure what you are saying here. Now for some possible dumb quesitons. > +Build a transformed package > +--------------------------- I am probably missing some the import differences between this and +Install a package containing transformed .pyc files +--------------------------------------------------- and +Build .pyc files when installing a package +------------------------------------------ > +It will be possible to build a package of the transformed code. > + > +A transformer can have a configuration. The configuration is not stored > +in the package. > + > +All ``.pyc`` files of the package must be transformed with the same AST > +transformers and the same transformers configuration. Why? is unclear to me. The issues I have: Suppose I write (as I might) one or more recursion-to-iteration transformers (within one transformer module). Each function transformation would be independent of all others, so could potentially be applied on a function-to-function basis. (For instance, decorators might be used to indicate which transformer, if any, to apply to a particular function.) For simple linear recursive functions (such as factorial), eliminating the repeated function calls might double the speed. The grouping of modules into packages is somewhat arbitrary. An application often uses modules from multiple packages, so it must anyway execute with different modules having different optimizations. A transform may not be 100% reliable, or may not be believed to be so by the programmer, or may depend on information supplies by the programmer. So a careful programmer only wants it applied after testing. The boundaries of what has been tested may not coincide with package boundaries. A related issue is that AST transformations may affect tracebacks, and a programmer might want to temporarily disable a transformation for one function, without having every module in a large package re-compiled to yet another set of .pyc files. I am partly worried here about combinatorial explosion of space usage. Suppose a transformer looks for certain decorators. Running that transformer over files that do not contain the trigger decorators is harmless, in that nothing will be changed, but is also irrelevant and a waste of time. > +API to support AST transformers: > +* Add ``sys.ast_transformers``: list of AST transformers used to rewrite > + an AST tree. The list is empty by default: no AST transformer. I do not understand how this would work, or rather it seems to conflict with 'all modules in each package must have a matching set of optimized .pycs'. > +* Add ``sys.implementation.ast_transformers``: name of AST > + transformers registered in ``sys.ast_transformers`` How is this different from sys.ast_transformers? ... > +AST transformer > +---------------- > + > +Scary AST transformer replacing all strings with ``"Ni! Ni! Ni!"``:: This helps a lot. > + import ast > + import sys > + > + > + class KnightsWhoSayNi(ast.NodeTransformer): > + def visit_Str(self, node): > + node.s = 'Ni! Ni! Ni!' > + return node This I can do today. Tree pattern recognition has to be hard-coded and substring capture done manually. > + class ASTTransformer: > + name = "knights_who_say_ni" > + > + def __call__(self, tree, context): > + KnightsWhoSayNi().visit(tree) > + return tree > + > + > + # register the AST transformer > + sys.ast_transformers.append(ASTTransformer()) This is new, with the result > + # execute code which will be transformed by ast_transformer() > + exec("print('Hello World!')") > + > +Output:: > + > + Ni! Ni! Ni! that this happens automatically, without having to write out exec(compile(Transformer.visit(compile-to-ast(code)))) More important, the transformation happens automatically for import statements without having to split up the import process to dig out and similarly augment the compile process. That will tjr From python-checkins at python.org Wed Jan 13 00:00:17 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 13 Jan 2016 05:00:17 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_update_2=2E7_schedule_=28clos?= =?utf-8?q?es_=2326087=29?= Message-ID: <20160113050017.31081.55424@psf.io> https://hg.python.org/peps/rev/1046a40e086f changeset: 6177:1046a40e086f user: Benjamin Peterson date: Tue Jan 12 21:00:13 2016 -0800 summary: update 2.7 schedule (closes #26087) files: pep-0373.txt | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/pep-0373.txt b/pep-0373.txt --- a/pep-0373.txt +++ b/pep-0373.txt @@ -56,8 +56,7 @@ Planned future release dates: -- 2.7.11rc1 2015-11-21 -- 2.7.11 2015-12-05 +- 2.7.12 May 2016ish - beyond this date, releases as needed Dates of previous maintenance releases: @@ -79,6 +78,8 @@ - 2.7.9 2014-12-10 - 2.7.10rc1 2015-05-09 - 2.7.10 2015-05-23 +- 2.7.11rc1 2015-11-21 +- 2.7.11 2015-12-05 2.7.0 Release Schedule ====================== -- Repository URL: https://hg.python.org/peps From solipsis at pitrou.net Wed Jan 13 03:44:18 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 13 Jan 2016 08:44:18 +0000 Subject: [Python-checkins] Daily reference leaks (77d24f51effc): sum=4 Message-ID: <20160113084418.121150.50943@psf.io> results for 77d24f51effc on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogZQfoo_', '--timeout', '7200'] From lp_benchmark_robot at intel.com Wed Jan 13 09:21:03 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 13 Jan 2016 14:21:03 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-13 Message-ID: Results for project Python default, build date 2016-01-13 03:08:15 +0000 commit: 77d24f51effc97d02d51a80db201b37fa79443c3 previous commit: 4f1a0e5deca6ec0b0af7439862a22ba35b30abe0 revision date: 2016-01-12 14:18:32 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.40% 1.45% 12.72% 12.45% :-| pybench 0.07% 0.02% -1.77% 5.44% :-( regex_v8 2.61% -0.01% -4.94% 5.72% :-) nbody 0.09% 0.55% 2.21% 4.57% :-( json_dump_v2 0.16% 0.24% -2.45% 11.05% :-| normal_startup 0.78% 0.09% -1.82% 4.80% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-default-2016-01-13/ Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Jan 13 09:21:54 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 13 Jan 2016 14:21:54 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-13 Message-ID: <7024d41d-80e9-4930-a76b-f4c8408defb2@irsmsx153.ger.corp.intel.com> Results for project Python 2.7, build date 2016-01-13 03:59:30 +0000 commit: 499ee454eeccbcb3bc2cba0108e3e268fb8a4797 previous commit: d208a2ada67c1fca3f7311330e1a6afb60cd69ee revision date: 2016-01-12 09:36:25 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.37% -0.39% 4.22% 2.48% :-) pybench 0.17% -0.11% 6.21% 4.90% :-( regex_v8 1.08% -0.05% -2.32% 10.46% :-) nbody 0.15% -0.01% 7.15% 6.52% :-) json_dump_v2 0.20% 0.26% 5.39% 9.31% :-( normal_startup 1.80% -0.31% -5.98% 2.89% :-| ssbench 0.13% 0.12% 0.56% 1.29% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-2-7-2016-01-13/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Jan 13 10:47:02 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 13 Jan 2016 15:47:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue22642_-_Convert_trace?= =?utf-8?q?_module=27s_option_handling_mechanism_from_getopt_to?= Message-ID: <20160113154701.106539.72364@psf.io> https://hg.python.org/cpython/rev/0aa46b9ffba3 changeset: 99880:0aa46b9ffba3 user: Senthil Kumaran date: Wed Jan 13 07:46:54 2016 -0800 summary: Issue22642 - Convert trace module's option handling mechanism from getopt to argparse. Patch contributed by SilentGhost. files: Lib/test/test_trace.py | 22 + Lib/trace.py | 335 +++++++++++----------------- 2 files changed, 154 insertions(+), 203 deletions(-) diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py --- a/Lib/test/test_trace.py +++ b/Lib/test/test_trace.py @@ -1,6 +1,7 @@ import os import sys from test.support import TESTFN, rmtree, unlink, captured_stdout +from test.support.script_helper import assert_python_ok, assert_python_failure import unittest import trace @@ -364,6 +365,27 @@ # Matched before. self.assertTrue(ignore.names(jn('bar', 'baz.py'), 'baz')) +class TestCommandLine(unittest.TestCase): + + def test_failures(self): + _errors = ( + (b'filename is missing: required with the main options', '-l', '-T'), + (b'cannot specify both --listfuncs and (--trace or --count)', '-lc'), + (b'argument -R/--no-report: not allowed with argument -r/--report', '-rR'), + (b'must specify one of --trace, --count, --report, --listfuncs, or --trackcalls', '-g'), + (b'-r/--report requires -f/--file', '-r'), + (b'--summary can only be used with --count or --report', '-sT'), + (b'unrecognized arguments: -y', '-y')) + for message, *args in _errors: + *_, stderr = assert_python_failure('-m', 'trace', *args) + self.assertIn(message, stderr) + + def test_listfuncs_flag_success(self): + with open(TESTFN, 'w') as fd: + self.addCleanup(unlink, TESTFN) + fd.write("a = 1\n") + status, stdout, stderr = assert_python_ok('-m', 'trace', '-l', TESTFN) + self.assertIn(b'functions called:', stdout) if __name__ == '__main__': unittest.main() diff --git a/Lib/trace.py b/Lib/trace.py --- a/Lib/trace.py +++ b/Lib/trace.py @@ -48,6 +48,7 @@ r.write_results(show_missing=True, coverdir="/tmp") """ __all__ = ['Trace', 'CoverageResults'] +import argparse import linecache import os import re @@ -76,51 +77,6 @@ sys.settrace(None) threading.settrace(None) -def _usage(outfile): - outfile.write("""Usage: %s [OPTIONS] [ARGS] - -Meta-options: ---help Display this help then exit. ---version Output version information then exit. - -Otherwise, exactly one of the following three options must be given: --t, --trace Print each line to sys.stdout before it is executed. --c, --count Count the number of times each line is executed - and write the counts to .cover for each - module executed, in the module's directory. - See also `--coverdir', `--file', `--no-report' below. --l, --listfuncs Keep track of which functions are executed at least - once and write the results to sys.stdout after the - program exits. --T, --trackcalls Keep track of caller/called pairs and write the - results to sys.stdout after the program exits. --r, --report Generate a report from a counts file; do not execute - any code. `--file' must specify the results file to - read, which must have been created in a previous run - with `--count --file=FILE'. - -Modifiers: --f, --file= File to accumulate counts over several runs. --R, --no-report Do not generate the coverage report files. - Useful if you want to accumulate over several runs. --C, --coverdir= Directory where the report files. The coverage - report for . is written to file - //.cover. --m, --missing Annotate executable lines that were not executed - with '>>>>>> '. --s, --summary Write a brief summary on stdout for each file. - (Can only be used with --count or --report.) --g, --timing Prefix each line with the time since the program started. - Only used while tracing. - -Filters, may be repeated multiple times: ---ignore-module= Ignore the given module(s) and its submodules - (if it is a package). Accepts comma separated - list of module names ---ignore-dir= Ignore files in the given directory (multiple - directories can be joined by os.pathsep). -""" % sys.argv[0]) - PRAGMA_NOCOVER = "#pragma NO COVER" # Simple rx to find lines with no code. @@ -264,7 +220,13 @@ def write_results(self, show_missing=True, summary=False, coverdir=None): """ - @param coverdir + Write the coverage results. + + :param show_missing: Show lines that had no hits. + :param summary: Include coverage summary per module. + :param coverdir: If None, the results of each module are placed in it's + directory, otherwise it is included in the directory + specified. """ if self.calledfuncs: print() @@ -646,168 +608,135 @@ calledfuncs=self._calledfuncs, callers=self._callers) -def _err_exit(msg): - sys.stderr.write("%s: %s\n" % (sys.argv[0], msg)) - sys.exit(1) +def main(): -def main(argv=None): - import getopt + parser = argparse.ArgumentParser() + parser.add_argument('--version', action='version', version='trace 2.0') - if argv is None: - argv = sys.argv + grp = parser.add_argument_group('Main options', + 'One of these (or --report) must be given') + + grp.add_argument('-c', '--count', action='store_true', + help='Count the number of times each line is executed and write ' + 'the counts to .cover for each module executed, in ' + 'the module\'s directory. See also --coverdir, --file, ' + '--no-report below.') + grp.add_argument('-t', '--trace', action='store_true', + help='Print each line to sys.stdout before it is executed') + grp.add_argument('-l', '--listfuncs', action='store_true', + help='Keep track of which functions are executed at least once ' + 'and write the results to sys.stdout after the program exits. ' + 'Cannot be specified alongside --trace or --count.') + grp.add_argument('-T', '--trackcalls', action='store_true', + help='Keep track of caller/called pairs and write the results to ' + 'sys.stdout after the program exits.') + + grp = parser.add_argument_group('Modifiers') + + _grp = grp.add_mutually_exclusive_group() + _grp.add_argument('-r', '--report', action='store_true', + help='Generate a report from a counts file; does not execute any ' + 'code. --file must specify the results file to read, which ' + 'must have been created in a previous run with --count ' + '--file=FILE') + _grp.add_argument('-R', '--no-report', action='store_true', + help='Do not generate the coverage report files. ' + 'Useful if you want to accumulate over several runs.') + + grp.add_argument('-f', '--file', + help='File to accumulate counts over several runs') + grp.add_argument('-C', '--coverdir', + help='Directory where the report files go. The coverage report ' + 'for . will be written to file ' + '//.cover') + grp.add_argument('-m', '--missing', action='store_true', + help='Annotate executable lines that were not executed with ' + '">>>>>> "') + grp.add_argument('-s', '--summary', action='store_true', + help='Write a brief summary for each file to sys.stdout. ' + 'Can only be used with --count or --report') + grp.add_argument('-g', '--timing', action='store_true', + help='Prefix each line with the time since the program started. ' + 'Only used while tracing') + + grp = parser.add_argument_group('Filters', + 'Can be specified multiple times') + grp.add_argument('--ignore-module', action='append', default=[], + help='Ignore the given module(s) and its submodules' + '(if it is a package). Accepts comma separated list of ' + 'module names.') + grp.add_argument('--ignore-dir', action='append', default=[], + help='Ignore files in the given directory ' + '(multiple directories can be joined by os.pathsep).') + + parser.add_argument('filename', nargs='?', + help='file to run as main program') + parser.add_argument('arguments', nargs=argparse.REMAINDER, + help='arguments to the program') + + opts = parser.parse_args() + + if opts.ignore_dir: + rel_path = 'lib', 'python{0.major}.{0.minor}'.format(sys.version_info) + _prefix = os.path.join(sys.base_prefix, *rel_path) + _exec_prefix = os.path.join(sys.base_exec_prefix, *rel_path) + + def parse_ignore_dir(s): + s = os.path.expanduser(os.path.expandvars(s)) + s = s.replace('$prefix', _prefix).replace('$exec_prefix', _exec_prefix) + return os.path.normpath(s) + + opts.ignore_module = [mod.strip() + for i in opts.ignore_module for mod in i.split(',')] + opts.ignore_dir = [parse_ignore_dir(s) + for i in opts.ignore_dir for s in i.split(os.pathsep)] + + if opts.report: + if not opts.file: + parser.error('-r/--report requires -f/--file') + results = CoverageResults(infile=opts.file, outfile=opts.file) + return results.write_results(opts.missing, opts.summary, opts.coverdir) + + if not any([opts.trace, opts.count, opts.listfuncs, opts.trackcalls]): + parser.error('must specify one of --trace, --count, --report, ' + '--listfuncs, or --trackcalls') + + if opts.listfuncs and (opts.count or opts.trace): + parser.error('cannot specify both --listfuncs and (--trace or --count)') + + if opts.summary and not opts.count: + parser.error('--summary can only be used with --count or --report') + + if opts.filename is None: + parser.error('filename is missing: required with the main options') + + sys.argv = opts.filename, *opts.arguments + sys.path[0] = os.path.dirname(opts.filename) + + t = Trace(opts.count, opts.trace, countfuncs=opts.listfuncs, + countcallers=opts.trackcalls, ignoremods=opts.ignore_module, + ignoredirs=opts.ignore_dir, infile=opts.file, + outfile=opts.file, timing=opts.timing) try: - opts, prog_argv = getopt.getopt(argv[1:], "tcrRf:d:msC:lTg", - ["help", "version", "trace", "count", - "report", "no-report", "summary", - "file=", "missing", - "ignore-module=", "ignore-dir=", - "coverdir=", "listfuncs", - "trackcalls", "timing"]) + with open(opts.filename) as fp: + code = compile(fp.read(), opts.filename, 'exec') + # try to emulate __main__ namespace as much as possible + globs = { + '__file__': opts.filename, + '__name__': '__main__', + '__package__': None, + '__cached__': None, + } + t.runctx(code, globs, globs) + except OSError as err: + sys.exit("Cannot run file %r because: %s" % (sys.argv[0], err)) + except SystemExit: + pass - except getopt.error as msg: - sys.stderr.write("%s: %s\n" % (sys.argv[0], msg)) - sys.stderr.write("Try `%s --help' for more information\n" - % sys.argv[0]) - sys.exit(1) + results = t.results() - trace = 0 - count = 0 - report = 0 - no_report = 0 - counts_file = None - missing = 0 - ignore_modules = [] - ignore_dirs = [] - coverdir = None - summary = 0 - listfuncs = False - countcallers = False - timing = False - - for opt, val in opts: - if opt == "--help": - _usage(sys.stdout) - sys.exit(0) - - if opt == "--version": - sys.stdout.write("trace 2.0\n") - sys.exit(0) - - if opt == "-T" or opt == "--trackcalls": - countcallers = True - continue - - if opt == "-l" or opt == "--listfuncs": - listfuncs = True - continue - - if opt == "-g" or opt == "--timing": - timing = True - continue - - if opt == "-t" or opt == "--trace": - trace = 1 - continue - - if opt == "-c" or opt == "--count": - count = 1 - continue - - if opt == "-r" or opt == "--report": - report = 1 - continue - - if opt == "-R" or opt == "--no-report": - no_report = 1 - continue - - if opt == "-f" or opt == "--file": - counts_file = val - continue - - if opt == "-m" or opt == "--missing": - missing = 1 - continue - - if opt == "-C" or opt == "--coverdir": - coverdir = val - continue - - if opt == "-s" or opt == "--summary": - summary = 1 - continue - - if opt == "--ignore-module": - for mod in val.split(","): - ignore_modules.append(mod.strip()) - continue - - if opt == "--ignore-dir": - for s in val.split(os.pathsep): - s = os.path.expandvars(s) - # should I also call expanduser? (after all, could use $HOME) - - s = s.replace("$prefix", - os.path.join(sys.base_prefix, "lib", - "python" + sys.version[:3])) - s = s.replace("$exec_prefix", - os.path.join(sys.base_exec_prefix, "lib", - "python" + sys.version[:3])) - s = os.path.normpath(s) - ignore_dirs.append(s) - continue - - assert 0, "Should never get here" - - if listfuncs and (count or trace): - _err_exit("cannot specify both --listfuncs and (--trace or --count)") - - if not (count or trace or report or listfuncs or countcallers): - _err_exit("must specify one of --trace, --count, --report, " - "--listfuncs, or --trackcalls") - - if report and no_report: - _err_exit("cannot specify both --report and --no-report") - - if report and not counts_file: - _err_exit("--report requires a --file") - - if no_report and len(prog_argv) == 0: - _err_exit("missing name of file to run") - - # everything is ready - if report: - results = CoverageResults(infile=counts_file, outfile=counts_file) - results.write_results(missing, summary=summary, coverdir=coverdir) - else: - sys.argv = prog_argv - progname = prog_argv[0] - sys.path[0] = os.path.split(progname)[0] - - t = Trace(count, trace, countfuncs=listfuncs, - countcallers=countcallers, ignoremods=ignore_modules, - ignoredirs=ignore_dirs, infile=counts_file, - outfile=counts_file, timing=timing) - try: - with open(progname) as fp: - code = compile(fp.read(), progname, 'exec') - # try to emulate __main__ namespace as much as possible - globs = { - '__file__': progname, - '__name__': '__main__', - '__package__': None, - '__cached__': None, - } - t.runctx(code, globs, globs) - except OSError as err: - _err_exit("Cannot run file %r because: %s" % (sys.argv[0], err)) - except SystemExit: - pass - - results = t.results() - - if not no_report: - results.write_results(missing, summary=summary, coverdir=coverdir) + if not opts.no_report: + results.write_results(opts.missing, opts.summary, opts.coverdir) if __name__=='__main__': main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 13 10:49:02 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 13 Jan 2016 15:49:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_NEWS_entry_for_Issue?= =?utf-8?q?_=2322642=2E?= Message-ID: <20160113154901.4238.83057@psf.io> https://hg.python.org/cpython/rev/69aa17b1f894 changeset: 99881:69aa17b1f894 user: Senthil Kumaran date: Wed Jan 13 07:48:57 2016 -0800 summary: Add a NEWS entry for Issue #22642. files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #22642: Convert trace module option parsing mechanism to argparse. + Patch contributed by SilentGhost. + - Issue #24705: Fix sysconfig._parse_makefile not expanding ${} vars appearing before $() vars. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 13 11:14:49 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 13 Jan 2016 16:14:49 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510?= Message-ID: <20160113161449.11403.57758@psf.io> https://hg.python.org/peps/rev/8aaa222b14b2 changeset: 6178:8aaa222b14b2 user: Victor Stinner date: Wed Jan 13 15:16:46 2016 +0100 summary: PEP 510 * complete PyFunction_Specialize() description * add link to the implementation files: pep-0510.txt | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -342,6 +342,16 @@ int PyFunction_Specialize(PyObject *func, PyObject *code, PyObject *guards) +If *code* is a Python function, the code object of the *code* function +is used as the specialized code. The specialized Python function must +have the same parameter defaults, the same keyword parameter defaults, +and must not have specialized code. + +If *code* is a Python function or a code object, a new code object is +created and the code name and first number number of the code object of +*func* are copied. The specialized code must have the same cell +variables and the same free variables. + Result: * Return ``0`` on success @@ -379,6 +389,14 @@ a Python function without specialization. +Implementation +============== + +The `issue #26098: PEP 510: Specialize functions with guards +`_ contains a patch which implements +this PEP. + + Other implementations of Python =============================== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Jan 13 12:38:27 2016 From: python-checkins at python.org (ezio.melotti) Date: Wed, 13 Jan 2016 17:38:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Convert_test?= =?utf-8?q?=5Ftarfile_to_ASCII_=28backport_c1ed490bd266=29=2E?= Message-ID: <20160113173824.99613.70884@psf.io> https://hg.python.org/cpython/rev/d16d9f10bb80 changeset: 99882:d16d9f10bb80 branch: 2.7 parent: 99878:499ee454eecc user: Ezio Melotti date: Wed Jan 13 19:36:49 2016 +0200 summary: Convert test_tarfile to ASCII (backport c1ed490bd266). files: Lib/test/test_tarfile.py | 40 +++++++++++++-------------- 1 files changed, 19 insertions(+), 21 deletions(-) diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1,5 +1,3 @@ -# -*- coding: iso-8859-15 -*- - import sys import os import shutil @@ -640,7 +638,7 @@ self._test_member(tarinfo, size=86016, chksum=md5_sparse) def test_find_umlauts(self): - tarinfo = self.tar.getmember("ustar/umlauts-???????") + tarinfo = self.tar.getmember("ustar/umlauts-\xc4\xd6\xdc\xe4\xf6\xfc\xdf") self._test_member(tarinfo, size=7011, chksum=md5_regtype) def test_find_ustar_longname(self): @@ -653,7 +651,7 @@ def test_find_pax_umlauts(self): self.tar = tarfile.open(self.tarname, mode=self.mode, encoding="iso8859-1") - tarinfo = self.tar.getmember("pax/umlauts-???????") + tarinfo = self.tar.getmember("pax/umlauts-\xc4\xd6\xdc\xe4\xf6\xfc\xdf") self._test_member(tarinfo, size=7011, chksum=md5_regtype) @@ -721,17 +719,17 @@ tarinfo = tar.getmember("pax/regtype1") self.assertEqual(tarinfo.uname, "foo") self.assertEqual(tarinfo.gname, "bar") - self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"???????") + self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") tarinfo = tar.getmember("pax/regtype2") self.assertEqual(tarinfo.uname, "") self.assertEqual(tarinfo.gname, "bar") - self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"???????") + self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") tarinfo = tar.getmember("pax/regtype3") self.assertEqual(tarinfo.uname, "tarfile") self.assertEqual(tarinfo.gname, "tarfile") - self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"???????") + self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") def test_pax_number_fields(self): # All following number fields are read from the pax header. @@ -1288,8 +1286,8 @@ u"foo": u"bar", u"uid": u"0", u"mtime": u"1.23", - u"test": u"???", - u"???": u"test"} + u"test": u"\xe4\xf6\xfc", + u"\xe4\xf6\xfc": u"test"} tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT, pax_headers=pax_headers) @@ -1318,7 +1316,7 @@ tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT, encoding="iso8859-1") t = tarfile.TarInfo() - t.name = u"???" # non-ASCII + t.name = u"\xe4\xf6\xfc" # non-ASCII t.uid = 8**8 # too large t.pax_headers = pax_headers tar.addfile(t) @@ -1347,7 +1345,7 @@ def _test_unicode_filename(self, encoding): tar = tarfile.open(tmpname, "w", format=self.format, encoding=encoding, errors="strict") - name = u"???" + name = u"\xe4\xf6\xfc" tar.addfile(tarfile.TarInfo(name)) tar.close() @@ -1360,17 +1358,17 @@ tar = tarfile.open(tmpname, "w", format=self.format, encoding="ascii", errors="strict") tarinfo = tarfile.TarInfo() - tarinfo.name = "???" + tarinfo.name = "\xe4\xf6\xfc" if self.format == tarfile.PAX_FORMAT: self.assertRaises(UnicodeError, tar.addfile, tarinfo) else: tar.addfile(tarinfo) - tarinfo.name = u"???" + tarinfo.name = u"\xe4\xf6\xfc" self.assertRaises(UnicodeError, tar.addfile, tarinfo) tarinfo.name = "foo" - tarinfo.uname = u"???" + tarinfo.uname = u"\xe4\xf6\xfc" self.assertRaises(UnicodeError, tar.addfile, tarinfo) def test_unicode_argument(self): @@ -1383,7 +1381,7 @@ tar.close() def test_uname_unicode(self): - for name in (u"???", "???"): + for name in (u"\xe4\xf6\xfc", "\xe4\xf6\xfc"): t = tarfile.TarInfo("foo") t.uname = name t.gname = name @@ -1396,8 +1394,8 @@ tar = tarfile.open("foo.tar", fileobj=fobj, encoding="iso8859-1") t = tar.getmember("foo") - self.assertEqual(t.uname, "???") - self.assertEqual(t.gname, "???") + self.assertEqual(t.uname, "\xe4\xf6\xfc") + self.assertEqual(t.gname, "\xe4\xf6\xfc") class GNUUnicodeTest(UstarUnicodeTest): @@ -1419,9 +1417,9 @@ def test_error_handlers(self): # Test if the unicode error handlers work correctly for characters # that cannot be expressed in a given encoding. - self._create_unicode_name(u"???") + self._create_unicode_name(u"\xe4\xf6\xfc") - for handler, name in (("utf-8", u"???".encode("utf8")), + for handler, name in (("utf-8", u"\xe4\xf6\xfc".encode("utf8")), ("replace", "???"), ("ignore", "")): tar = tarfile.open(tmpname, format=self.format, encoding="ascii", errors=handler) @@ -1433,11 +1431,11 @@ def test_error_handler_utf8(self): # Create a pathname that has one component representable using # iso8859-1 and the other only in iso8859-15. - self._create_unicode_name(u"???/?") + self._create_unicode_name(u"\xe4\xf6\xfc/\u20ac") tar = tarfile.open(tmpname, format=self.format, encoding="iso8859-1", errors="utf-8") - self.assertEqual(tar.getnames()[0], "???/" + u"?".encode("utf8")) + self.assertEqual(tar.getnames()[0], "\xe4\xf6\xfc/" + u"\u20ac".encode("utf8")) class AppendTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 13 15:21:27 2016 From: python-checkins at python.org (ezio.melotti) Date: Wed, 13 Jan 2016 20:21:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzc5NDQ6IGNsb3Nl?= =?utf-8?q?_files_explicitly_in_test=5Ftarfile_=28backport_d560eece0857=29?= =?utf-8?q?=2E?= Message-ID: <20160113202127.31097.91431@psf.io> https://hg.python.org/cpython/rev/e3763d98c46e changeset: 99883:e3763d98c46e branch: 2.7 user: Ezio Melotti date: Wed Jan 13 22:21:21 2016 +0200 summary: #7944: close files explicitly in test_tarfile (backport d560eece0857). files: Lib/test/test_tarfile.py | 633 +++++++++++++++----------- 1 files changed, 360 insertions(+), 273 deletions(-) diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -60,9 +60,10 @@ self.tar.extract("ustar/regtype", TEMPDIR) tarinfo = self.tar.getmember("ustar/regtype") fobj1 = open(os.path.join(TEMPDIR, "ustar/regtype"), "rU") + with open(os.path.join(TEMPDIR, "ustar/regtype"), "rU") as fobj1: + lines1 = fobj1.readlines() fobj2 = self.tar.extractfile(tarinfo) - lines1 = fobj1.readlines() lines2 = fobj2.readlines() self.assertTrue(lines1 == lines2, "fileobj.readlines() failed") @@ -75,18 +76,17 @@ def test_fileobj_iter(self): self.tar.extract("ustar/regtype", TEMPDIR) tarinfo = self.tar.getmember("ustar/regtype") - fobj1 = open(os.path.join(TEMPDIR, "ustar/regtype"), "rU") + with open(os.path.join(TEMPDIR, "ustar/regtype"), "rU") as fobj1: + lines1 = fobj1.readlines() fobj2 = self.tar.extractfile(tarinfo) - lines1 = fobj1.readlines() lines2 = [line for line in fobj2] self.assertTrue(lines1 == lines2, "fileobj.__iter__() failed") def test_fileobj_seek(self): self.tar.extract("ustar/regtype", TEMPDIR) - fobj = open(os.path.join(TEMPDIR, "ustar/regtype"), "rb") - data = fobj.read() - fobj.close() + with open(os.path.join(TEMPDIR, "ustar/regtype"), "rb") as fobj: + data = fobj.read() tarinfo = self.tar.getmember("ustar/regtype") fobj = self.tar.extractfile(tarinfo) @@ -238,19 +238,24 @@ # This test checks if tarfile.open() is able to open an empty tar # archive successfully. Note that an empty tar archive is not the # same as an empty file! - tarfile.open(tmpname, self.mode.replace("r", "w")).close() + with tarfile.open(tmpname, self.mode.replace("r", "w")): + pass try: tar = tarfile.open(tmpname, self.mode) tar.getnames() except tarfile.ReadError: self.fail("tarfile.open() failed on empty archive") - self.assertListEqual(tar.getmembers(), []) + else: + self.assertListEqual(tar.getmembers(), []) + finally: + tar.close() def test_null_tarfile(self): # Test for issue6123: Allow opening empty archives. # This test guarantees that tarfile.open() does not treat an empty # file as an empty tar archive. - open(tmpname, "wb").close() + with open(tmpname, "wb"): + pass self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, self.mode) self.assertRaises(tarfile.ReadError, tarfile.open, tmpname) @@ -274,15 +279,16 @@ for char in ('\0', 'a'): # Test if EOFHeaderError ('\0') and InvalidHeaderError ('a') # are ignored correctly. - fobj = _open(tmpname, "wb") - fobj.write(char * 1024) - fobj.write(tarfile.TarInfo("foo").tobuf()) - fobj.close() + with _open(tmpname, "wb") as fobj: + fobj.write(char * 1024) + fobj.write(tarfile.TarInfo("foo").tobuf()) tar = tarfile.open(tmpname, mode="r", ignore_zeros=True) - self.assertListEqual(tar.getnames(), ["foo"], + try: + self.assertListEqual(tar.getnames(), ["foo"], "ignore_zeros=True should have skipped the %r-blocks" % char) - tar.close() + finally: + tar.close() def test_premature_end_of_archive(self): for size in (512, 600, 1024, 1200): @@ -313,19 +319,21 @@ taropen = tarfile.TarFile.taropen def test_no_name_argument(self): - fobj = open(self.tarname, "rb") - tar = tarfile.open(fileobj=fobj, mode=self.mode) - self.assertEqual(tar.name, os.path.abspath(fobj.name)) + with open(self.tarname, "rb") as fobj: + tar = tarfile.open(fileobj=fobj, mode=self.mode) + self.assertEqual(tar.name, os.path.abspath(fobj.name)) def test_no_name_attribute(self): - data = open(self.tarname, "rb").read() + with open(self.tarname, "rb") as fobj: + data = fobj.read() fobj = StringIO.StringIO(data) self.assertRaises(AttributeError, getattr, fobj, "name") tar = tarfile.open(fileobj=fobj, mode=self.mode) self.assertEqual(tar.name, None) def test_empty_name_attribute(self): - data = open(self.tarname, "rb").read() + with open(self.tarname, "rb") as fobj: + data = fobj.read() fobj = StringIO.StringIO(data) fobj.name = "" tar = tarfile.open(fileobj=fobj, mode=self.mode) @@ -346,12 +354,14 @@ # Skip the first member and store values from the second member # of the testtar. tar = tarfile.open(self.tarname, mode=self.mode) - tar.next() - t = tar.next() - name = t.name - offset = t.offset - data = tar.extractfile(t).read() - tar.close() + try: + tar.next() + t = tar.next() + name = t.name + offset = t.offset + data = tar.extractfile(t).read() + finally: + tar.close() # Open the testtar and seek to the offset of the second member. if self.mode.endswith(":gz"): @@ -361,26 +371,30 @@ else: _open = open fobj = _open(self.tarname, "rb") - fobj.seek(offset) + try: + fobj.seek(offset) - # Test if the tarfile starts with the second member. - tar = tar.open(self.tarname, mode="r:", fileobj=fobj) - t = tar.next() - self.assertEqual(t.name, name) - # Read to the end of fileobj and test if seeking back to the - # beginning works. - tar.getmembers() - self.assertEqual(tar.extractfile(t).read(), data, - "seek back did not work") - tar.close() + # Test if the tarfile starts with the second member. + tar = tar.open(self.tarname, mode="r:", fileobj=fobj) + t = tar.next() + self.assertEqual(t.name, name) + # Read to the end of fileobj and test if seeking back to the + # beginning works. + tar.getmembers() + self.assertEqual(tar.extractfile(t).read(), data, + "seek back did not work") + tar.close() + finally: + fobj.close() def test_fail_comp(self): # For Gzip and Bz2 Tests: fail with a ReadError on an uncompressed file. if self.mode == "r:": self.skipTest('needs a gz or bz2 mode') self.assertRaises(tarfile.ReadError, tarfile.open, tarname, self.mode) - fobj = open(tarname, "rb") - self.assertRaises(tarfile.ReadError, tarfile.open, fileobj=fobj, mode=self.mode) + with open(tarname, "rb") as fobj: + self.assertRaises(tarfile.ReadError, tarfile.open, + fileobj=fobj, mode=self.mode) def test_v7_dirtype(self): # Test old style dirtype member (bug #1336623): @@ -434,22 +448,25 @@ # Test if extractall() correctly restores directory permissions # and times (see issue1735). tar = tarfile.open(tarname, encoding="iso8859-1") - directories = [t for t in tar if t.isdir()] - tar.extractall(TEMPDIR, directories) - for tarinfo in directories: - path = os.path.join(TEMPDIR, tarinfo.name) - if sys.platform != "win32": - # Win32 has no support for fine grained permissions. - self.assertEqual(tarinfo.mode & 0777, os.stat(path).st_mode & 0777) - self.assertEqual(tarinfo.mtime, os.path.getmtime(path)) - tar.close() + try: + directories = [t for t in tar if t.isdir()] + tar.extractall(TEMPDIR, directories) + for tarinfo in directories: + path = os.path.join(TEMPDIR, tarinfo.name) + if sys.platform != "win32": + # Win32 has no support for fine grained permissions. + self.assertEqual(tarinfo.mode & 0777, os.stat(path).st_mode & 0777) + self.assertEqual(tarinfo.mtime, os.path.getmtime(path)) + finally: + tar.close() def test_init_close_fobj(self): # Issue #7341: Close the internal file object in the TarFile # constructor in case of an error. For the test we rely on # the fact that opening an empty file raises a ReadError. empty = os.path.join(TEMPDIR, "empty") - open(empty, "wb").write("") + with open(empty, "wb") as fobj: + fobj.write("") try: tar = object.__new__(tarfile.TarFile) @@ -460,7 +477,7 @@ else: self.fail("ReadError not raised") finally: - os.remove(empty) + support.unlink(empty) def test_parallel_iteration(self): # Issue #16601: Restarting iteration over tarfile continued @@ -489,42 +506,47 @@ def test_compare_members(self): tar1 = tarfile.open(tarname, encoding="iso8859-1") - tar2 = self.tar + try: + tar2 = self.tar - while True: - t1 = tar1.next() - t2 = tar2.next() - if t1 is None: - break - self.assertTrue(t2 is not None, "stream.next() failed.") + while True: + t1 = tar1.next() + t2 = tar2.next() + if t1 is None: + break + self.assertTrue(t2 is not None, "stream.next() failed.") - if t2.islnk() or t2.issym(): - self.assertRaises(tarfile.StreamError, tar2.extractfile, t2) - continue + if t2.islnk() or t2.issym(): + self.assertRaises(tarfile.StreamError, tar2.extractfile, t2) + continue - v1 = tar1.extractfile(t1) - v2 = tar2.extractfile(t2) - if v1 is None: - continue - self.assertTrue(v2 is not None, "stream.extractfile() failed") - self.assertTrue(v1.read() == v2.read(), "stream extraction failed") - - tar1.close() + v1 = tar1.extractfile(t1) + v2 = tar2.extractfile(t2) + if v1 is None: + continue + self.assertTrue(v2 is not None, "stream.extractfile() failed") + self.assertTrue(v1.read() == v2.read(), "stream extraction failed") + finally: + tar1.close() class DetectReadTest(unittest.TestCase): def _testfunc_file(self, name, mode): try: - tarfile.open(name, mode) + tar = tarfile.open(name, mode) except tarfile.ReadError: self.fail() + else: + tar.close() def _testfunc_fileobj(self, name, mode): try: - tarfile.open(name, mode, fileobj=open(name, "rb")) + tar = tarfile.open(name, mode, fileobj=open(name, "rb")) except tarfile.ReadError: self.fail() + else: + tar.close() def _test_modes(self, testfunc): testfunc(tarname, "r") @@ -715,33 +737,39 @@ def test_pax_global_headers(self): tar = tarfile.open(tarname, encoding="iso8859-1") + try: - tarinfo = tar.getmember("pax/regtype1") - self.assertEqual(tarinfo.uname, "foo") - self.assertEqual(tarinfo.gname, "bar") - self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") + tarinfo = tar.getmember("pax/regtype1") + self.assertEqual(tarinfo.uname, "foo") + self.assertEqual(tarinfo.gname, "bar") + self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") - tarinfo = tar.getmember("pax/regtype2") - self.assertEqual(tarinfo.uname, "") - self.assertEqual(tarinfo.gname, "bar") - self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") + tarinfo = tar.getmember("pax/regtype2") + self.assertEqual(tarinfo.uname, "") + self.assertEqual(tarinfo.gname, "bar") + self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") - tarinfo = tar.getmember("pax/regtype3") - self.assertEqual(tarinfo.uname, "tarfile") - self.assertEqual(tarinfo.gname, "tarfile") - self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") + tarinfo = tar.getmember("pax/regtype3") + self.assertEqual(tarinfo.uname, "tarfile") + self.assertEqual(tarinfo.gname, "tarfile") + self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), u"\xc4\xd6\xdc\xe4\xf6\xfc\xdf") + finally: + tar.close() def test_pax_number_fields(self): # All following number fields are read from the pax header. tar = tarfile.open(tarname, encoding="iso8859-1") - tarinfo = tar.getmember("pax/regtype4") - self.assertEqual(tarinfo.size, 7011) - self.assertEqual(tarinfo.uid, 123) - self.assertEqual(tarinfo.gid, 123) - self.assertEqual(tarinfo.mtime, 1041808783.0) - self.assertEqual(type(tarinfo.mtime), float) - self.assertEqual(float(tarinfo.pax_headers["atime"]), 1041808783.0) - self.assertEqual(float(tarinfo.pax_headers["ctime"]), 1041808783.0) + try: + tarinfo = tar.getmember("pax/regtype4") + self.assertEqual(tarinfo.size, 7011) + self.assertEqual(tarinfo.uid, 123) + self.assertEqual(tarinfo.gid, 123) + self.assertEqual(tarinfo.mtime, 1041808783.0) + self.assertEqual(type(tarinfo.mtime), float) + self.assertEqual(float(tarinfo.pax_headers["atime"]), 1041808783.0) + self.assertEqual(float(tarinfo.pax_headers["ctime"]), 1041808783.0) + finally: + tar.close() class WriteTestBase(unittest.TestCase): @@ -773,52 +801,60 @@ # a trailing '\0'. name = "0123456789" * 10 tar = tarfile.open(tmpname, self.mode) - t = tarfile.TarInfo(name) - tar.addfile(t) - tar.close() + try: + t = tarfile.TarInfo(name) + tar.addfile(t) + finally: + tar.close() tar = tarfile.open(tmpname) - self.assertTrue(tar.getnames()[0] == name, - "failed to store 100 char filename") - tar.close() + try: + self.assertTrue(tar.getnames()[0] == name, + "failed to store 100 char filename") + finally: + tar.close() def test_tar_size(self): # Test for bug #1013882. tar = tarfile.open(tmpname, self.mode) - path = os.path.join(TEMPDIR, "file") - fobj = open(path, "wb") - fobj.write("aaa") - fobj.close() - tar.add(path) - tar.close() + try: + path = os.path.join(TEMPDIR, "file") + with open(path, "wb") as fobj: + fobj.write("aaa") + tar.add(path) + finally: + tar.close() self.assertTrue(os.path.getsize(tmpname) > 0, "tarfile is empty") # The test_*_size tests test for bug #1167128. def test_file_size(self): tar = tarfile.open(tmpname, self.mode) + try: - path = os.path.join(TEMPDIR, "file") - fobj = open(path, "wb") - fobj.close() - tarinfo = tar.gettarinfo(path) - self.assertEqual(tarinfo.size, 0) + path = os.path.join(TEMPDIR, "file") + with open(path, "wb"): + pass + tarinfo = tar.gettarinfo(path) + self.assertEqual(tarinfo.size, 0) - fobj = open(path, "wb") - fobj.write("aaa") - fobj.close() - tarinfo = tar.gettarinfo(path) - self.assertEqual(tarinfo.size, 3) - - tar.close() + with open(path, "wb") as fobj: + fobj.write("aaa") + tarinfo = tar.gettarinfo(path) + self.assertEqual(tarinfo.size, 3) + finally: + tar.close() def test_directory_size(self): path = os.path.join(TEMPDIR, "directory") os.mkdir(path) try: tar = tarfile.open(tmpname, self.mode) - tarinfo = tar.gettarinfo(path) - self.assertEqual(tarinfo.size, 0) + try: + tarinfo = tar.gettarinfo(path) + self.assertEqual(tarinfo.size, 0) + finally: + tar.close() finally: os.rmdir(path) @@ -826,16 +862,18 @@ if hasattr(os, "link"): link = os.path.join(TEMPDIR, "link") target = os.path.join(TEMPDIR, "link_target") - fobj = open(target, "wb") - fobj.write("aaa") - fobj.close() + with open(target, "wb") as fobj: + fobj.write("aaa") os.link(target, link) try: tar = tarfile.open(tmpname, self.mode) - # Record the link target in the inodes list. - tar.gettarinfo(target) - tarinfo = tar.gettarinfo(link) - self.assertEqual(tarinfo.size, 0) + try: + # Record the link target in the inodes list. + tar.gettarinfo(target) + tarinfo = tar.gettarinfo(link) + self.assertEqual(tarinfo.size, 0) + finally: + tar.close() finally: os.remove(target) os.remove(link) @@ -846,26 +884,30 @@ os.symlink("link_target", path) try: tar = tarfile.open(tmpname, self.mode) - tarinfo = tar.gettarinfo(path) - self.assertEqual(tarinfo.size, 0) + try: + tarinfo = tar.gettarinfo(path) + self.assertEqual(tarinfo.size, 0) + finally: + tar.close() finally: os.remove(path) def test_add_self(self): # Test for #1257255. dstname = os.path.abspath(tmpname) + tar = tarfile.open(tmpname, self.mode) + try: + self.assertTrue(tar.name == dstname, "archive name must be absolute") + tar.add(dstname) + self.assertTrue(tar.getnames() == [], "added the archive to itself") - tar = tarfile.open(tmpname, self.mode) - self.assertTrue(tar.name == dstname, "archive name must be absolute") - - tar.add(dstname) - self.assertTrue(tar.getnames() == [], "added the archive to itself") - - cwd = os.getcwd() - os.chdir(TEMPDIR) - tar.add(dstname) - os.chdir(cwd) - self.assertTrue(tar.getnames() == [], "added the archive to itself") + cwd = os.getcwd() + os.chdir(TEMPDIR) + tar.add(dstname) + os.chdir(cwd) + self.assertTrue(tar.getnames() == [], "added the archive to itself") + finally: + tar.close() def test_exclude(self): tempdir = os.path.join(TEMPDIR, "exclude") @@ -878,14 +920,19 @@ exclude = os.path.isfile tar = tarfile.open(tmpname, self.mode, encoding="iso8859-1") - with test_support.check_warnings(("use the filter argument", - DeprecationWarning)): - tar.add(tempdir, arcname="empty_dir", exclude=exclude) - tar.close() + try: + with test_support.check_warnings(("use the filter argument", + DeprecationWarning)): + tar.add(tempdir, arcname="empty_dir", exclude=exclude) + finally: + tar.close() tar = tarfile.open(tmpname, "r") - self.assertEqual(len(tar.getmembers()), 1) - self.assertEqual(tar.getnames()[0], "empty_dir") + try: + self.assertEqual(len(tar.getmembers()), 1) + self.assertEqual(tar.getnames()[0], "empty_dir") + finally: + tar.close() finally: shutil.rmtree(tempdir) @@ -905,15 +952,19 @@ return tarinfo tar = tarfile.open(tmpname, self.mode, encoding="iso8859-1") - tar.add(tempdir, arcname="empty_dir", filter=filter) - tar.close() + try: + tar.add(tempdir, arcname="empty_dir", filter=filter) + finally: + tar.close() tar = tarfile.open(tmpname, "r") - for tarinfo in tar: - self.assertEqual(tarinfo.uid, 123) - self.assertEqual(tarinfo.uname, "foo") - self.assertEqual(len(tar.getmembers()), 3) - tar.close() + try: + for tarinfo in tar: + self.assertEqual(tarinfo.uid, 123) + self.assertEqual(tarinfo.uname, "foo") + self.assertEqual(len(tar.getmembers()), 3) + finally: + tar.close() finally: shutil.rmtree(tempdir) @@ -931,12 +982,16 @@ os.mkdir(foo) tar = tarfile.open(tmpname, self.mode) - tar.add(foo, arcname=path) - tar.close() + try: + tar.add(foo, arcname=path) + finally: + tar.close() tar = tarfile.open(tmpname, "r") - t = tar.next() - tar.close() + try: + t = tar.next() + finally: + tar.close() if not dir: os.remove(foo) @@ -972,16 +1027,18 @@ def test_cwd(self): # Test adding the current working directory. with support.change_cwd(TEMPDIR): - open("foo", "w").close() - tar = tarfile.open(tmpname, self.mode) - tar.add(".") - tar.close() + try: + tar.add(".") + finally: + tar.close() tar = tarfile.open(tmpname, "r") - for t in tar: - self.assertTrue(t.name == "." or t.name.startswith("./")) - tar.close() + try: + for t in tar: + self.assertTrue(t.name == "." or t.name.startswith("./")) + finally: + tar.close() @unittest.skipUnless(hasattr(os, 'symlink'), "needs os.symlink") def test_extractall_symlinks(self): @@ -1098,19 +1155,18 @@ tar.close() if self.mode.endswith("gz"): - fobj = gzip.GzipFile(tmpname) - data = fobj.read() - fobj.close() + with gzip.GzipFile(tmpname) as fobj: + data = fobj.read() elif self.mode.endswith("bz2"): dec = bz2.BZ2Decompressor() - data = open(tmpname, "rb").read() + with open(tmpname, "rb") as fobj: + data = fobj.read() data = dec.decompress(data) self.assertTrue(len(dec.unused_data) == 0, "found trailing data") else: - fobj = open(tmpname, "rb") - data = fobj.read() - fobj.close() + with open(tmpname, "rb") as fobj: + data = fobj.read() self.assertTrue(data.count("\0") == tarfile.RECORDSIZE, "incorrect zero padding") @@ -1171,23 +1227,27 @@ tarinfo.type = tarfile.LNKTYPE tar = tarfile.open(tmpname, "w") - tar.format = tarfile.GNU_FORMAT - tar.addfile(tarinfo) + try: + tar.format = tarfile.GNU_FORMAT + tar.addfile(tarinfo) - v1 = self._calc_size(name, link) - v2 = tar.offset - self.assertTrue(v1 == v2, "GNU longname/longlink creation failed") - - tar.close() + v1 = self._calc_size(name, link) + v2 = tar.offset + self.assertTrue(v1 == v2, "GNU longname/longlink creation failed") + finally: + tar.close() tar = tarfile.open(tmpname) - member = tar.next() - self.assertIsNotNone(member, - "unable to read longname member") - self.assertEqual(tarinfo.name, member.name, - "unable to read longname member") - self.assertEqual(tarinfo.linkname, member.linkname, - "unable to read longname member") + try: + member = tar.next() + self.assertIsNotNone(member, + "unable to read longname member") + self.assertEqual(tarinfo.name, member.name, + "unable to read longname member") + self.assertEqual(tarinfo.linkname, member.linkname, + "unable to read longname member") + finally: + tar.close() def test_longname_1023(self): self._test(("longnam/" * 127) + "longnam") @@ -1227,9 +1287,8 @@ self.foo = os.path.join(TEMPDIR, "foo") self.bar = os.path.join(TEMPDIR, "bar") - fobj = open(self.foo, "wb") - fobj.write("foo") - fobj.close() + with open(self.foo, "wb") as fobj: + fobj.write("foo") os.link(self.foo, self.bar) @@ -1238,8 +1297,8 @@ def tearDown(self): self.tar.close() - os.remove(self.foo) - os.remove(self.bar) + support.unlink(self.foo) + support.unlink(self.bar) def test_add_twice(self): # The same name will be added as a REGTYPE every @@ -1270,16 +1329,21 @@ tarinfo.type = tarfile.LNKTYPE tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT) - tar.addfile(tarinfo) - tar.close() + try: + tar.addfile(tarinfo) + finally: + tar.close() tar = tarfile.open(tmpname) - if link: - l = tar.getmembers()[0].linkname - self.assertTrue(link == l, "PAX longlink creation failed") - else: - n = tar.getmembers()[0].name - self.assertTrue(name == n, "PAX longname creation failed") + try: + if link: + l = tar.getmembers()[0].linkname + self.assertTrue(link == l, "PAX longlink creation failed") + else: + n = tar.getmembers()[0].name + self.assertTrue(name == n, "PAX longname creation failed") + finally: + tar.close() def test_pax_global_header(self): pax_headers = { @@ -1291,23 +1355,28 @@ tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT, pax_headers=pax_headers) - tar.addfile(tarfile.TarInfo("test")) - tar.close() + try: + tar.addfile(tarfile.TarInfo("test")) + finally: + tar.close() # Test if the global header was written correctly. tar = tarfile.open(tmpname, encoding="iso8859-1") - self.assertEqual(tar.pax_headers, pax_headers) - self.assertEqual(tar.getmembers()[0].pax_headers, pax_headers) + try: + self.assertEqual(tar.pax_headers, pax_headers) + self.assertEqual(tar.getmembers()[0].pax_headers, pax_headers) - # Test if all the fields are unicode. - for key, val in tar.pax_headers.iteritems(): - self.assertTrue(type(key) is unicode) - self.assertTrue(type(val) is unicode) - if key in tarfile.PAX_NUMBER_FIELDS: - try: - tarfile.PAX_NUMBER_FIELDS[key](val) - except (TypeError, ValueError): - self.fail("unable to convert pax header field") + # Test if all the fields are unicode. + for key, val in tar.pax_headers.iteritems(): + self.assertTrue(type(key) is unicode) + self.assertTrue(type(val) is unicode) + if key in tarfile.PAX_NUMBER_FIELDS: + try: + tarfile.PAX_NUMBER_FIELDS[key](val) + except (TypeError, ValueError): + self.fail("unable to convert pax header field") + finally: + tar.close() def test_pax_extended_header(self): # The fields from the pax header have priority over the @@ -1315,18 +1384,23 @@ pax_headers = {u"path": u"foo", u"uid": u"123"} tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT, encoding="iso8859-1") - t = tarfile.TarInfo() - t.name = u"\xe4\xf6\xfc" # non-ASCII - t.uid = 8**8 # too large - t.pax_headers = pax_headers - tar.addfile(t) - tar.close() + try: + t = tarfile.TarInfo() + t.name = u"\xe4\xf6\xfc" # non-ASCII + t.uid = 8**8 # too large + t.pax_headers = pax_headers + tar.addfile(t) + finally: + tar.close() tar = tarfile.open(tmpname, encoding="iso8859-1") - t = tar.getmembers()[0] - self.assertEqual(t.pax_headers, pax_headers) - self.assertEqual(t.name, "foo") - self.assertEqual(t.uid, 123) + try: + t = tar.getmembers()[0] + self.assertEqual(t.pax_headers, pax_headers) + self.assertEqual(t.name, "foo") + self.assertEqual(t.uid, 123) + finally: + tar.close() class UstarUnicodeTest(unittest.TestCase): @@ -1345,40 +1419,49 @@ def _test_unicode_filename(self, encoding): tar = tarfile.open(tmpname, "w", format=self.format, encoding=encoding, errors="strict") - name = u"\xe4\xf6\xfc" - tar.addfile(tarfile.TarInfo(name)) - tar.close() + try: + name = u"\xe4\xf6\xfc" + tar.addfile(tarfile.TarInfo(name)) + finally: + tar.close() tar = tarfile.open(tmpname, encoding=encoding) - self.assertTrue(type(tar.getnames()[0]) is not unicode) - self.assertEqual(tar.getmembers()[0].name, name.encode(encoding)) - tar.close() + try: + self.assertTrue(type(tar.getnames()[0]) is not unicode) + self.assertEqual(tar.getmembers()[0].name, name.encode(encoding)) + finally: + tar.close() def test_unicode_filename_error(self): tar = tarfile.open(tmpname, "w", format=self.format, encoding="ascii", errors="strict") - tarinfo = tarfile.TarInfo() + try: + tarinfo = tarfile.TarInfo() - tarinfo.name = "\xe4\xf6\xfc" - if self.format == tarfile.PAX_FORMAT: + tarinfo.name = "\xe4\xf6\xfc" + if self.format == tarfile.PAX_FORMAT: + self.assertRaises(UnicodeError, tar.addfile, tarinfo) + else: + tar.addfile(tarinfo) + + tarinfo.name = u"\xe4\xf6\xfc" self.assertRaises(UnicodeError, tar.addfile, tarinfo) - else: - tar.addfile(tarinfo) - tarinfo.name = u"\xe4\xf6\xfc" - self.assertRaises(UnicodeError, tar.addfile, tarinfo) - - tarinfo.name = "foo" - tarinfo.uname = u"\xe4\xf6\xfc" - self.assertRaises(UnicodeError, tar.addfile, tarinfo) + tarinfo.name = "foo" + tarinfo.uname = u"\xe4\xf6\xfc" + self.assertRaises(UnicodeError, tar.addfile, tarinfo) + finally: + tar.close() def test_unicode_argument(self): tar = tarfile.open(tarname, "r", encoding="iso8859-1", errors="strict") - for t in tar: - self.assertTrue(type(t.name) is str) - self.assertTrue(type(t.linkname) is str) - self.assertTrue(type(t.uname) is str) - self.assertTrue(type(t.gname) is str) - tar.close() + try: + for t in tar: + self.assertTrue(type(t.name) is str) + self.assertTrue(type(t.linkname) is str) + self.assertTrue(type(t.uname) is str) + self.assertTrue(type(t.gname) is str) + finally: + tar.close() def test_uname_unicode(self): for name in (u"\xe4\xf6\xfc", "\xe4\xf6\xfc"): @@ -1388,8 +1471,10 @@ fobj = StringIO.StringIO() tar = tarfile.open("foo.tar", mode="w", fileobj=fobj, format=self.format, encoding="iso8859-1") - tar.addfile(t) - tar.close() + try: + tar.addfile(t) + finally: + tar.close() fobj.seek(0) tar = tarfile.open("foo.tar", fileobj=fobj, encoding="iso8859-1") @@ -1447,22 +1532,20 @@ os.remove(self.tarname) def _add_testfile(self, fileobj=None): - tar = tarfile.open(self.tarname, "a", fileobj=fileobj) - tar.addfile(tarfile.TarInfo("bar")) - tar.close() + with tarfile.open(self.tarname, "a", fileobj=fileobj) as tar: + tar.addfile(tarfile.TarInfo("bar")) def _create_testtar(self, mode="w:"): - src = tarfile.open(tarname, encoding="iso8859-1") - t = src.getmember("ustar/regtype") - t.name = "foo" - f = src.extractfile(t) - tar = tarfile.open(self.tarname, mode) - tar.addfile(t, f) - tar.close() + with tarfile.open(tarname, encoding="iso8859-1") as src: + t = src.getmember("ustar/regtype") + t.name = "foo" + f = src.extractfile(t) + with tarfile.open(self.tarname, mode) as tar: + tar.addfile(t, f) def _test(self, names=["bar"], fileobj=None): - tar = tarfile.open(self.tarname, fileobj=fileobj) - self.assertEqual(tar.getnames(), names) + with tarfile.open(self.tarname, fileobj=fileobj) as tar: + self.assertEqual(tar.getnames(), names) def test_non_existing(self): self._add_testfile() @@ -1481,7 +1564,8 @@ def test_fileobj(self): self._create_testtar() - data = open(self.tarname).read() + with open(self.tarname) as fobj: + data = fobj.read() fobj = StringIO.StringIO(data) self._add_testfile(fobj) fobj.seek(0) @@ -1505,7 +1589,8 @@ # Append mode is supposed to fail if the tarfile to append to # does not end with a zero block. def _test_error(self, data): - open(self.tarname, "wb").write(data) + with open(self.tarname, "wb") as fobj: + fobj.write(data) self.assertRaises(tarfile.ReadError, self._add_testfile) def test_null(self): @@ -1641,15 +1726,14 @@ def test_fileobj(self): # Test that __exit__() did not close the external file # object. - fobj = open(tmpname, "wb") - try: - with tarfile.open(fileobj=fobj, mode="w") as tar: - raise Exception - except: - pass - self.assertFalse(fobj.closed, "external file object was closed") - self.assertTrue(tar.closed, "context manager failed") - fobj.close() + with open(tmpname, "wb") as fobj: + try: + with tarfile.open(fileobj=fobj, mode="w") as tar: + raise Exception + except: + pass + self.assertFalse(fobj.closed, "external file object was closed") + self.assertTrue(tar.closed, "context manager failed") class LinkEmulationTest(ReadTest): @@ -1737,6 +1821,7 @@ def test_main(): + support.unlink(TEMPDIR) os.makedirs(TEMPDIR) tests = [ @@ -1766,15 +1851,14 @@ else: tests.append(LinkEmulationTest) - fobj = open(tarname, "rb") - data = fobj.read() - fobj.close() + with open(tarname, "rb") as fobj: + data = fobj.read() if gzip: # Create testtar.tar.gz and add gzip-specific tests. - tar = gzip.open(gzipname, "wb") - tar.write(data) - tar.close() + support.unlink(gzipname) + with gzip.open(gzipname, "wb") as tar: + tar.write(data) tests += [ GzipMiscReadTest, @@ -1787,9 +1871,12 @@ if bz2: # Create testtar.tar.bz2 and add bz2-specific tests. + support.unlink(bz2name) tar = bz2.BZ2File(bz2name, "wb") - tar.write(data) - tar.close() + try: + tar.write(data) + finally: + tar.close() tests += [ Bz2MiscReadTest, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 13 15:36:47 2016 From: python-checkins at python.org (chris.angelico) Date: Wed, 13 Jan 2016 20:36:47 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Apply_Oleg=27s_edits_to_PEP_1?= =?utf-8?q?03=2C_per_request?= Message-ID: <20160113203647.31093.60783@psf.io> https://hg.python.org/peps/rev/7aeecf213809 changeset: 6179:7aeecf213809 user: Chris Angelico date: Thu Jan 14 07:36:39 2016 +1100 summary: Apply Oleg's edits to PEP 103, per request files: pep-0103.txt | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/pep-0103.txt b/pep-0103.txt --- a/pep-0103.txt +++ b/pep-0103.txt @@ -62,6 +62,8 @@ `Git Wiki `_. +`Git Buch `_ (German). + Offline documentation --------------------- @@ -886,8 +888,9 @@ `_ and `pagure `_, both are written in Python; pagure was written by Fedora developers and is being used to develop some Fedora projects. `Gogs -`_ is written in Go; there is a fork `Gitea -`_. +`_ is written in Go; there is a fork `Gitea +`_. `GitBucket +`_ is written in Scala. And last but not least, `Gitlab `_. It's perhaps the most advanced web-based development environment for git. -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 03:11:46 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 14 Jan 2016 08:11:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325822=3A_Add_docs?= =?utf-8?q?trings_to_the_fields_of_urllib=2Eparse_results=2E?= Message-ID: <20160114081146.99631.67043@psf.io> https://hg.python.org/cpython/rev/7bfcb8b75ad9 changeset: 99884:7bfcb8b75ad9 parent: 99881:69aa17b1f894 user: Senthil Kumaran date: Thu Jan 14 00:11:39 2016 -0800 summary: Issue #25822: Add docstrings to the fields of urllib.parse results. Patch contributed by Swati Jaiswal. files: Lib/urllib/parse.py | 67 ++++++++++++++++++++++++++++++++- Misc/NEWS | 3 + 2 files changed, 68 insertions(+), 2 deletions(-) diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py --- a/Lib/urllib/parse.py +++ b/Lib/urllib/parse.py @@ -224,8 +224,71 @@ from collections import namedtuple _DefragResultBase = namedtuple('DefragResult', 'url fragment') -_SplitResultBase = namedtuple('SplitResult', 'scheme netloc path query fragment') -_ParseResultBase = namedtuple('ParseResult', 'scheme netloc path params query fragment') +_SplitResultBase = namedtuple( + 'SplitResult', 'scheme netloc path query fragment') +_ParseResultBase = namedtuple( + 'ParseResult', 'scheme netloc path params query fragment') + +_DefragResultBase.__doc__ = """ +DefragResult(url, fragment) + +A 2-tuple that contains the url without fragment identifier and the fragment +identifier as a separate argument. +""" + +_DefragResultBase.url.__doc__ = """The URL with no fragment identifier.""" + +_DefragResultBase.fragment.__doc__ = """ +Fragment identifier separated from URL, that allows indirect identification of a +secondary resource by reference to a primary resource and additional identifying +information. +""" + +_SplitResultBase.__doc__ = """ +SplitResult(scheme, netloc, path, query, fragment) + +A 5-tuple that contains the different components of a URL. Similar to +ParseResult, but does not split params. +""" + +_SplitResultBase.scheme.__doc__ = """Specifies URL scheme for the request.""" + +_SplitResultBase.netloc.__doc__ = """ +Network location where the request is made to. +""" + +_SplitResultBase.path.__doc__ = """ +The hierarchical path, such as the path to a file to download. +""" + +_SplitResultBase.query.__doc__ = """ +The query component, that contains non-hierarchical data, that along with data +in path component, identifies a resource in the scope of URI's scheme and +network location. +""" + +_SplitResultBase.fragment.__doc__ = """ +Fragment identifier, that allows indirect identification of a secondary resource +by reference to a primary resource and additional identifying information. +""" + +_ParseResultBase.__doc__ = """ +ParseResult(scheme, netloc, path, params, query, fragment) + +A 6-tuple that contains components of a parsed URL. +""" + +_ParseResultBase.scheme.__doc__ = _SplitResultBase.scheme.__doc__ +_ParseResultBase.netloc.__doc__ = _SplitResultBase.netloc.__doc__ +_ParseResultBase.path.__doc__ = _SplitResultBase.path.__doc__ +_ParseResultBase.params.__doc__ = """ +Parameters for last path element used to dereference the URI in order to provide +access to perform some operation on the resource. +""" + +_ParseResultBase.query.__doc__ = _SplitResultBase.query.__doc__ +_ParseResultBase.fragment.__doc__ = _SplitResultBase.fragment.__doc__ + # For backwards compatibility, alias _NetlocResultMixinStr # ResultBase is no longer part of the documented API, but it is diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -128,6 +128,9 @@ Library ------- +- Issue #25822: Add docstrings to the fields of urllib.parse results. + Patch contributed by Swati Jaiswal. + - Issue #22642: Convert trace module option parsing mechanism to argparse. Patch contributed by SilentGhost. -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Thu Jan 14 03:44:41 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 14 Jan 2016 08:44:41 +0000 Subject: [Python-checkins] Daily reference leaks (69aa17b1f894): sum=4 Message-ID: <20160114084441.14081.58756@psf.io> results for 69aa17b1f894 on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogzK03qJ', '--timeout', '7200'] From python-checkins at python.org Thu Jan 14 03:45:46 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 14 Jan 2016 08:45:46 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_pickle_doesn=27t_s?= =?utf-8?q?erialize_code_object=2C_remove_nonsense_paragraph=2E?= Message-ID: <20160114084546.99639.71619@psf.io> https://hg.python.org/peps/rev/92324b60dbb7 changeset: 6180:92324b60dbb7 user: Victor Stinner date: Thu Jan 14 09:44:15 2016 +0100 summary: PEP 510: pickle doesn't serialize code object, remove nonsense paragraph. files: pep-0510.txt | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -282,11 +282,6 @@ When a function code is replaced (``func.__code__ = new_code``), all specialized codes and guards are removed. -When a function is serialized ``pickle``, specialized codes and guards are -ignored (not serialized). Specialized codes and guards are not stored in -``.pyc`` files but created and registered at runtime, when a module is -loaded. - Function guard -------------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 03:50:30 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 14 Jan 2016 08:50:30 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_510=3A_Add_PyFunction=5FR?= =?utf-8?q?emoveSpecialized=28=29_and?= Message-ID: <20160114085030.18732.19458@psf.io> https://hg.python.org/peps/rev/04b071bdb4c1 changeset: 6181:04b071bdb4c1 user: Victor Stinner date: Thu Jan 14 09:50:22 2016 +0100 summary: PEP 510: Add PyFunction_RemoveSpecialized() and PyFunction_RemoveAllSpecialized() files: pep-0510.txt | 36 +++++++++++++++++++++++++++++++++++- 1 files changed, 35 insertions(+), 1 deletions(-) diff --git a/pep-0510.txt b/pep-0510.txt --- a/pep-0510.txt +++ b/pep-0510.txt @@ -274,6 +274,8 @@ * ``PyFunction_Specialize()`` * ``PyFunction_GetSpecializedCodes()`` * ``PyFunction_GetSpecializedCode()`` + * ``PyFunction_RemoveSpecialized()`` + * ``PyFunction_RemoveAllSpecialized()`` None of these function and types are exposed at the Python level. @@ -331,6 +333,9 @@ Function methods ---------------- +PyFunction_Specialize +^^^^^^^^^^^^^^^^^^^^^ + Add a function method to specialize the function, add a specialized code with guards:: @@ -353,6 +358,10 @@ * Return ``1`` if the specialization has been ignored * Raise an exception and return ``-1`` on error + +PyFunction_GetSpecializedCodes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Add a function method to get the list of specialized codes:: PyObject* PyFunction_GetSpecializedCodes(PyObject *func) @@ -361,7 +370,11 @@ code object and *guards* is a list of ``PyFuncGuard`` objects. Raise an exception and return ``NULL`` on error. -Add a function method to get the specialized code:: + +PyFunction_GetSpecializedCode +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add a function method checking guards to choose a specialized code:: PyObject* PyFunction_GetSpecializedCode(PyObject *func, PyObject **stack, @@ -371,6 +384,27 @@ Return a callable or a code object on success. Raise an exception and return ``NULL`` on error. + +PyFunction_RemoveSpecialized +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add a function method to remove a specialized code with its guards by +its index:: + + int PyFunction_RemoveSpecialized(PyObject *func, Py_ssize_t index) + +Return ``0`` on sucess. Raise an exception and return ``-1`` on error. + + +PyFunction_RemoveAllSpecialized +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Add a function method to remove all specialized codes and guards of a +function:: + + void PyFunction_RemoveAllSpecialized(PyObject *func) + + Benchmark --------- -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 04:15:40 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 14 Jan 2016 09:15:40 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_add_link_to_issue_?= =?utf-8?q?=2326107?= Message-ID: <20160114091540.24750.84832@psf.io> https://hg.python.org/peps/rev/c4a826184937 changeset: 6182:c4a826184937 user: Victor Stinner date: Thu Jan 14 10:15:32 2016 +0100 summary: PEP 511: add link to issue #26107 files: pep-0511.txt | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -185,7 +185,9 @@ frozenset items. * ``PyCodeObject.co_lnotab``: line number delta becomes signed to support moving instructions (note: need to modify MAGIC_NUMBER in - importlib). + importlib). Implemented in the `issue #26107: code.co_lnotab: use + signed line number delta to support moving instructions in an + optimizer `_ * Enhance the bytecode compiler to support ``tuple`` and ``frozenset`` constants. Currently, ``tuple`` and ``frozenset`` constants are created by the peephole transformer, after the bytecode compilation. -- Repository URL: https://hg.python.org/peps From lp_benchmark_robot at intel.com Thu Jan 14 05:00:23 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 14 Jan 2016 10:00:23 +0000 Subject: [Python-checkins] Benchmark Results for Python Default 2016-01-14 Message-ID: <72b56ed9-1b39-49bb-9bec-fbc7e60e8025@irsmsx105.ger.corp.intel.com> Results for project Python default, build date 2016-01-14 03:08:21 +0000 commit: 69aa17b1f894 previous commit: 77d24f51effc revision date: 2016-01-13 15:48:57 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.26% -1.45% 11.45% 14.52% :-| pybench 0.08% -0.03% -1.80% 5.77% :-( regex_v8 2.58% 0.09% -4.85% 4.82% :-| nbody 0.18% -0.67% 1.56% 4.08% :-( json_dump_v2 0.19% -0.32% -2.78% 12.14% :-| normal_startup 0.98% 0.04% -0.02% 5.10% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-default-2016-01-14/ Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Jan 14 05:04:56 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 14 Jan 2016 10:04:56 +0000 Subject: [Python-checkins] Benchmark Results for Python 2.7 2016-01-14 Message-ID: Results for project Python 2.7, build date 2016-01-14 03:59:31 +0000 commit: e3763d98c46e previous commit: 499ee454eecc revision date: 2016-01-13 20:21:21 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.36% -0.78% 3.48% 2.90% :-) pybench 0.18% 0.12% 6.32% 4.45% :-( regex_v8 1.09% -0.09% -2.42% 10.97% :-) nbody 0.14% 0.04% 7.19% 6.02% :-) json_dump_v2 0.19% -0.16% 5.24% 9.57% :-( normal_startup 2.01% 0.03% -5.94% 3.01% :-| ssbench 0.17% -0.33% 0.23% 1.64% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/benchmark-results-for-python-2-7-2016-01-14/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Jan 14 07:31:26 2016 From: python-checkins at python.org (nick.coghlan) Date: Thu, 14 Jan 2016 12:31:26 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_432_reference_impl_is_up_?= =?utf-8?q?to_date?= Message-ID: <20160114123124.77976.52033@psf.io> https://hg.python.org/peps/rev/3596a8498c2d changeset: 6183:3596a8498c2d user: Nick Coghlan date: Thu Jan 14 22:30:50 2016 +1000 summary: PEP 432 reference impl is up to date files: pep-0432.txt | 23 +++++++++++++++++++---- 1 files changed, 19 insertions(+), 4 deletions(-) diff --git a/pep-0432.txt b/pep-0432.txt --- a/pep-0432.txt +++ b/pep-0432.txt @@ -1031,11 +1031,26 @@ Implementation ============== -A reference implementation for an earlier design was developed as a feature -branch in my BitBucket sandbox [2_]. +A reference implementation is available as a feature branch in my BitBucket +sandbox [2_]. -There is not yet a reference implementation for the design currently -described in the PEP. +It implements the ``_Py_InitializeCore`` and ``_Py_InitializeMainInterpreter`` +aspects of the refactoring as a private API. All ``_PyCoreConfig`` settings +are included, but currently only the "install signal handlers" setting is +implemented for the main interpreter configuration. + +The currently available interfaces in the reference implementation: + +* ``_Py_IsCoreInitialized`` +* ``_Py_InitializeCore`` +* ``_PyCoreConfig`` +* ``_PyCoreConfig_INIT`` +* ``_Py_ReadHashSeed`` +* ``_Py_InitializeMainInterpreter`` +* ``_PyMainInterpreterConfig`` +* ``_PyMainInterpreterConfig_INIT`` +* ``_Py_ReadMainInterpreterConfig`` + The Status Quo ============== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 08:05:30 2016 From: python-checkins at python.org (matthias.klose) Date: Thu, 14 Jan 2016 13:05:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogIC0gbWVyZ2UgMy41?= Message-ID: <20160114130528.14814.7920@psf.io> https://hg.python.org/cpython/rev/121fa7d22e22 changeset: 99886:121fa7d22e22 parent: 99884:7bfcb8b75ad9 parent: 99885:d48948c13473 user: doko at ubuntu.com date: Thu Jan 14 14:05:21 2016 +0100 summary: - merge 3.5 files: Objects/dictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2579,7 +2579,7 @@ return sizeof(PyDictKeysObject) + (DK_SIZE(keys)-1) * sizeof(PyDictKeyEntry); } -PyObject * +static PyObject * dict_sizeof(PyDictObject *mp) { return PyLong_FromSsize_t(_PyDict_SizeOf(mp)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:05:32 2016 From: python-checkins at python.org (matthias.klose) Date: Thu, 14 Jan 2016 13:05:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogZGljdG9iamVjdC5j?= =?utf-8?q?=28dict=5Fsizeof=29=3A_Make_it_static_again=2E?= Message-ID: <20160114130528.106561.69472@psf.io> https://hg.python.org/cpython/rev/d48948c13473 changeset: 99885:d48948c13473 branch: 3.5 parent: 99876:5a2692911a43 user: doko at ubuntu.com date: Thu Jan 14 14:04:59 2016 +0100 summary: dictobject.c(dict_sizeof): Make it static again. files: Objects/dictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2576,7 +2576,7 @@ return sizeof(PyDictKeysObject) + (DK_SIZE(keys)-1) * sizeof(PyDictKeyEntry); } -PyObject * +static PyObject * dict_sizeof(PyDictObject *mp) { return PyLong_FromSsize_t(_PyDict_SizeOf(mp)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:15 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzI1OTQw?= =?utf-8?q?=3A_Backport_self-signed=2Epythontest=2Enet_testing_for_test=5F?= =?utf-8?q?httplib?= Message-ID: <20160114133914.14071.74745@psf.io> https://hg.python.org/cpython/rev/81b3beea7e99 changeset: 99887:81b3beea7e99 branch: 3.2 parent: 99451:36be496963d1 user: Georg Brandl date: Wed Nov 05 20:37:40 2014 +0100 summary: Issue #25940: Backport self-signed.pythontest.net testing for test_httplib The svn.python.org server recently changed root certificate, causing the test to fail. This backports revision 4985375db40f. files: Lib/test/selfsigned_pythontestdotnet.pem | 16 ++++++++++++ Lib/test/test_httplib.py | 15 ++++++----- 2 files changed, 24 insertions(+), 7 deletions(-) diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem new file mode 100644 --- /dev/null +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIChzCCAfCgAwIBAgIJAKGU95wKR8pSMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjKTAnMCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MA0GCSqGSIb3DQEBBQUAA4GBAIOXmdtM +eG9qzP9TiXW/Gc/zI4cBfdCpC+Y4gOfC9bQUC7hefix4iO3+iZjgy3X/FaRxUUoV +HKiXcXIaWqTSUWp45cSh0MbwZXudp6JIAptzdAhvvCrPKeC9i9GvxsPD4LtDAL97 +vSaxQBezA7hdxZd90/EeyMgVZgAnTCnvAWX9 +-----END CERTIFICATE----- diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -15,8 +15,8 @@ CERT_localhost = os.path.join(here, 'keycert.pem') # Self-signed cert file for 'fakehostname' CERT_fakehostname = os.path.join(here, 'keycert2.pem') -# Root cert file (CA) for svn.python.org's cert -CACERT_svn_python_org = os.path.join(here, 'https_svn_python_org_root.pem') +# Self-signed cert file for self-signed.pythontest.net +CERT_selfsigned_pythontestdotnet = os.path.join(here, 'selfsigned_pythontestdotnet.pem') HOST = support.HOST @@ -570,17 +570,18 @@ self._check_svn_python_org(resp) def test_networked_good_cert(self): - # We feed a CA cert that validates the server's cert + # We feed the server's cert as a validating cert import ssl support.requires('network') - with support.transient_internet('svn.python.org'): + with support.transient_internet('self-signed.pythontest.net'): context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) context.verify_mode = ssl.CERT_REQUIRED - context.load_verify_locations(CACERT_svn_python_org) - h = client.HTTPSConnection('svn.python.org', 443, context=context) + context.load_verify_locations(CERT_selfsigned_pythontestdotnet) + h = client.HTTPSConnection('self-signed.pythontest.net', 443, context=context) h.request('GET', '/') resp = h.getresponse() - self._check_svn_python_org(resp) + server_string = resp.getheader('server') + self.assertIn('nginx', server_string) def test_networked_bad_cert(self): # We feed a "CA" cert that is unrelated to the server's cert -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:15 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzI1OTQw?= =?utf-8?q?=3A_Use_self-signed=2Epythontest=2Enet_in_SSL_tests?= Message-ID: <20160114133914.24746.25802@psf.io> https://hg.python.org/cpython/rev/adf750b1252d changeset: 99888:adf750b1252d branch: 3.2 user: Martin Panter date: Thu Jan 14 09:36:00 2016 +0000 summary: Issue #25940: Use self-signed.pythontest.net in SSL tests This is instead of svn.python.org, whose certificate recently expired, and whose new certificate uses a different root certificate. The certificate used at the pythontest server was modifed to set the "basic constraints" CA flag. This flag seems to be required for test_get_ca_certs_ capath() to work (in Python 3.4+). Added the new self-signed certificate to capath with the following commands: cp Lib/test/{selfsigned_pythontestdotnet.pem,capath/} c_rehash -v Lib/test/capath/ c_rehash -v -old Lib/test/capath/ # Note the generated file names cp Lib/test/capath/{selfsigned_pythontestdotnet.pem,0e4015b9.0} mv Lib/test/capath/{selfsigned_pythontestdotnet.pem,ce7b8643.0} The new server responds with "No route to host" when connecting to port 444. files: Lib/test/capath/0e4015b9.0 | 16 + Lib/test/capath/ce7b8643.0 | 16 + Lib/test/https_svn_python_org_root.pem | 41 ---- Lib/test/selfsigned_pythontestdotnet.pem | 12 +- Lib/test/test_ssl.py | 101 +++++----- Misc/NEWS | 7 + 6 files changed, 97 insertions(+), 96 deletions(-) diff --git a/Lib/test/capath/0e4015b9.0 b/Lib/test/capath/0e4015b9.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/0e4015b9.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/capath/ce7b8643.0 b/Lib/test/capath/ce7b8643.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/ce7b8643.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/https_svn_python_org_root.pem b/Lib/test/https_svn_python_org_root.pem deleted file mode 100644 --- a/Lib/test/https_svn_python_org_root.pem +++ /dev/null @@ -1,41 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 -IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB -IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA -Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO -BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi -MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ -ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC -CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ -8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 -zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y -fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 -w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc -G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k -epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q -laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ -QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU -fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 -YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w -ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY -gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe -MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 -IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy -dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw -czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 -dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl -aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC -AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg -b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB -ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc -nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg -18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c -gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl -Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY -sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T -SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF -CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum -GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk -zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW -omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD ------END CERTIFICATE----- diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem --- a/Lib/test/selfsigned_pythontestdotnet.pem +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -1,5 +1,5 @@ -----BEGIN CERTIFICATE----- -MIIChzCCAfCgAwIBAgIJAKGU95wKR8pSMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG @@ -8,9 +8,9 @@ aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv -EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjKTAnMCUGA1UdEQQeMByCGnNl -bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MA0GCSqGSIb3DQEBBQUAA4GBAIOXmdtM -eG9qzP9TiXW/Gc/zI4cBfdCpC+Y4gOfC9bQUC7hefix4iO3+iZjgy3X/FaRxUUoV -HKiXcXIaWqTSUWp45cSh0MbwZXudp6JIAptzdAhvvCrPKeC9i9GvxsPD4LtDAL97 -vSaxQBezA7hdxZd90/EeyMgVZgAnTCnvAWX9 +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= -----END CERTIFICATE----- diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -29,7 +29,8 @@ HOST = support.HOST -data_file = lambda name: os.path.join(os.path.dirname(__file__), name) +def data_file(*name): + return os.path.join(os.path.dirname(__file__), *name) # The custom key and certificate files used in test_ssl are generated # using Lib/test/make_ssl_certs.py. @@ -44,8 +45,10 @@ BYTES_ONLYKEY = os.fsencode(ONLYKEY) CAPATH = data_file("capath") BYTES_CAPATH = os.fsencode(CAPATH) +CAFILE_CACERT = data_file("capath", "5ed36f99.0") -SVN_PYTHON_ORG_ROOT_CERT = data_file("https_svn_python_org_root.pem") +REMOTE_HOST = "self-signed.pythontest.net" +REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") EMPTYCERT = data_file("nullcert.pem") BADCERT = data_file("badcert.pem") @@ -163,7 +166,7 @@ ) def test_DER_to_PEM(self): - with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f: + with open(CAFILE_CACERT, 'r') as f: pem = f.read() d1 = ssl.PEM_cert_to_DER_cert(pem) p2 = ssl.DER_cert_to_PEM_cert(d1) @@ -462,7 +465,7 @@ # Mismatching key and cert ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with self.assertRaisesRegex(ssl.SSLError, "key values mismatch"): - ctx.load_cert_chain(SVN_PYTHON_ORG_ROOT_CERT, ONLYKEY) + ctx.load_cert_chain(CAFILE_CACERT, ONLYKEY) def test_load_verify_locations(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) @@ -511,11 +514,11 @@ class NetworkedTests(unittest.TestCase): def test_connect(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertEqual({}, s.getpeercert()) finally: s.close() @@ -524,27 +527,27 @@ s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # this should succeed because we specify the root cert s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertTrue(s.getpeercert()) finally: s.close() def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(0, s.connect_ex(("svn.python.org", 443))) + self.assertEqual(0, s.connect_ex((REMOTE_HOST, 443))) self.assertTrue(s.getpeercert()) finally: s.close() @@ -552,14 +555,14 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.setblocking(False) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) # EWOULDBLOCK under Windows, EINPROGRESS elsewhere self.assertIn(rc, (0, errno.EINPROGRESS, errno.EWOULDBLOCK)) # Wait for connect to finish @@ -584,59 +587,59 @@ def test_timeout_connect_ex(self): # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.settimeout(0.0000001) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) if rc == 0: - self.skipTest("svn.python.org responded too quickly") + self.skipTest("REMOTE_HOST responded too quickly") self.assertIn(rc, (errno.EAGAIN, errno.EWOULDBLOCK)) finally: s.close() def test_connect_ex_error(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(errno.ECONNREFUSED, - s.connect_ex(("svn.python.org", 444))) + rc = s.connect_ex((REMOTE_HOST, 444)) + self.assertIn(rc, (errno.ECONNREFUSED, errno.EHOSTUNREACH)) finally: s.close() def test_connect_with_context(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): # Same as test_connect, but with a separately created context ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: self.assertEqual({}, s.getpeercert()) finally: s.close() # Same with a server hostname s = ctx.wrap_socket(socket.socket(socket.AF_INET), - server_hostname="svn.python.org") + server_hostname=REMOTE_HOST) if ssl.HAS_SNI: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.close() else: - self.assertRaises(ValueError, s.connect, ("svn.python.org", 443)) + self.assertRaises(ValueError, s.connect, (REMOTE_HOST, 443)) # This should fail because we have no verification certs ctx.verify_mode = ssl.CERT_REQUIRED s = ctx.wrap_socket(socket.socket(socket.AF_INET)) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # This should succeed because we specify the root cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -649,12 +652,12 @@ # OpenSSL 0.9.8n and 1.0.0, as a result the capath directory must # contain both versions of each certificate (same content, different # filename) for this test to be portable across OpenSSL releases. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -665,7 +668,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=BYTES_CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -677,9 +680,9 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) fd = ss.fileno() f = ss.makefile() f.close() @@ -693,9 +696,9 @@ self.assertEqual(e.exception.errno, errno.EBADF) def test_non_blocking_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = socket.socket(socket.AF_INET) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.setblocking(False) s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE, @@ -718,14 +721,14 @@ sys.stdout.write("\nNeeded %d calls to do_handshake() to establish session.\n" % count) def test_get_server_certificate(self): - with support.transient_internet("svn.python.org"): - pem = ssl.get_server_certificate(("svn.python.org", 443), + with support.transient_internet(REMOTE_HOST): + pem = ssl.get_server_certificate((REMOTE_HOST, 443), ssl.PROTOCOL_SSLv23) if not pem: - self.fail("No server certificate on svn.python.org:443!") + self.fail("No server certificate on REMOTE_HOST:443!") try: - pem = ssl.get_server_certificate(("svn.python.org", 443), + pem = ssl.get_server_certificate((REMOTE_HOST, 443), ssl.PROTOCOL_SSLv23, ca_certs=CERTFILE) except ssl.SSLError as x: @@ -733,18 +736,18 @@ if support.verbose: sys.stdout.write("%s\n" % x) else: - self.fail("Got server certificate %s for svn.python.org!" % pem) + self.fail("Got server certificate %s for REMOTE_HOST!" % pem) - pem = ssl.get_server_certificate(("svn.python.org", 443), + pem = ssl.get_server_certificate((REMOTE_HOST, 443), ssl.PROTOCOL_SSLv23, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) if not pem: - self.fail("No server certificate on svn.python.org:443!") + self.fail("No server certificate on REMOTE_HOST:443!") if support.verbose: - sys.stdout.write("\nVerified certificate for svn.python.org:443 is\n%s\n" % pem) + sys.stdout.write("\nVerified certificate for REMOTE_HOST:443 is\n%s\n" % pem) def test_ciphers(self): - remote = ("svn.python.org", 443) + remote = (REMOTE_HOST, 443) with support.transient_internet(remote[0]): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL") @@ -1745,7 +1748,7 @@ print(" HAS_SNI = %r" % ssl.HAS_SNI) for filename in [ - CERTFILE, SVN_PYTHON_ORG_ROOT_CERT, BYTES_CERTFILE, + CERTFILE, REMOTE_ROOT_CERT, BYTES_CERTFILE, ONLYCERT, ONLYKEY, BYTES_ONLYCERT, BYTES_ONLYKEY, BADCERT, BADKEY, EMPTYCERT]: if not os.path.exists(filename): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -21,6 +21,13 @@ - Issue #22931: Allow '[' and ']' in cookie values. +Tests +----- + +- Issue #25940: Changed test_ssl and test_httplib to use + self-signed.pythontest.net. This avoids relying on svn.python.org, which + recently changed root certificate. + What's New in Python 3.2.6? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:15 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:15 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_Issue_=2325940=3A_Merge_self-signed=2Epythontest=2Enet_testing?= =?utf-8?q?_from_3=2E2_into_3=2E3?= Message-ID: <20160114133915.77988.47551@psf.io> https://hg.python.org/cpython/rev/e167a9296947 changeset: 99889:e167a9296947 branch: 3.3 parent: 99741:cd8cbe513597 parent: 99888:adf750b1252d user: Martin Panter date: Thu Jan 14 12:21:02 2016 +0000 summary: Issue #25940: Merge self-signed.pythontest.net testing from 3.2 into 3.3 files: Lib/test/capath/0e4015b9.0 | 16 ++ Lib/test/capath/ce7b8643.0 | 16 ++ Lib/test/https_svn_python_org_root.pem | 41 ----- Lib/test/selfsigned_pythontestdotnet.pem | 16 ++ Lib/test/test_httplib.py | 15 +- Lib/test/test_ssl.py | 89 ++++++----- Misc/NEWS | 7 + 7 files changed, 111 insertions(+), 89 deletions(-) diff --git a/Lib/test/capath/0e4015b9.0 b/Lib/test/capath/0e4015b9.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/0e4015b9.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/capath/ce7b8643.0 b/Lib/test/capath/ce7b8643.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/ce7b8643.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/https_svn_python_org_root.pem b/Lib/test/https_svn_python_org_root.pem deleted file mode 100644 --- a/Lib/test/https_svn_python_org_root.pem +++ /dev/null @@ -1,41 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 -IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB -IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA -Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO -BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi -MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ -ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC -CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ -8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 -zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y -fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 -w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc -G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k -epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q -laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ -QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU -fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 -YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w -ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY -gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe -MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 -IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy -dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw -czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 -dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl -aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC -AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg -b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB -ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc -nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg -18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c -gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl -Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY -sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T -SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF -CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum -GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk -zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW -omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD ------END CERTIFICATE----- diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem new file mode 100644 --- /dev/null +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -15,8 +15,8 @@ CERT_localhost = os.path.join(here, 'keycert.pem') # Self-signed cert file for 'fakehostname' CERT_fakehostname = os.path.join(here, 'keycert2.pem') -# Root cert file (CA) for svn.python.org's cert -CACERT_svn_python_org = os.path.join(here, 'https_svn_python_org_root.pem') +# Self-signed cert file for self-signed.pythontest.net +CERT_selfsigned_pythontestdotnet = os.path.join(here, 'selfsigned_pythontestdotnet.pem') HOST = support.HOST @@ -758,17 +758,18 @@ self._check_svn_python_org(resp) def test_networked_good_cert(self): - # We feed a CA cert that validates the server's cert + # We feed the server's cert as a validating cert import ssl support.requires('network') - with support.transient_internet('svn.python.org'): + with support.transient_internet('self-signed.pythontest.net'): context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) context.verify_mode = ssl.CERT_REQUIRED - context.load_verify_locations(CACERT_svn_python_org) - h = client.HTTPSConnection('svn.python.org', 443, context=context) + context.load_verify_locations(CERT_selfsigned_pythontestdotnet) + h = client.HTTPSConnection('self-signed.pythontest.net', 443, context=context) h.request('GET', '/') resp = h.getresponse() - self._check_svn_python_org(resp) + server_string = resp.getheader('server') + self.assertIn('nginx', server_string) def test_networked_bad_cert(self): # We feed a "CA" cert that is unrelated to the server's cert diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -29,7 +29,8 @@ HOST = support.HOST -data_file = lambda name: os.path.join(os.path.dirname(__file__), name) +def data_file(*name): + return os.path.join(os.path.dirname(__file__), *name) # The custom key and certificate files used in test_ssl are generated # using Lib/test/make_ssl_certs.py. @@ -47,8 +48,10 @@ KEY_PASSWORD = "somepass" CAPATH = data_file("capath") BYTES_CAPATH = os.fsencode(CAPATH) +CAFILE_CACERT = data_file("capath", "5ed36f99.0") -SVN_PYTHON_ORG_ROOT_CERT = data_file("https_svn_python_org_root.pem") +REMOTE_HOST = "self-signed.pythontest.net" +REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") EMPTYCERT = data_file("nullcert.pem") BADCERT = data_file("badcert.pem") @@ -229,7 +232,7 @@ self.assertEqual(p['subjectAltName'], san) def test_DER_to_PEM(self): - with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f: + with open(CAFILE_CACERT, 'r') as f: pem = f.read() d1 = ssl.PEM_cert_to_DER_cert(pem) p2 = ssl.DER_cert_to_PEM_cert(d1) @@ -592,7 +595,7 @@ # Mismatching key and cert ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with self.assertRaisesRegex(ssl.SSLError, "key values mismatch"): - ctx.load_cert_chain(SVN_PYTHON_ORG_ROOT_CERT, ONLYKEY) + ctx.load_cert_chain(CAFILE_CACERT, ONLYKEY) # Password protected key and cert ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD) ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD.encode()) @@ -759,11 +762,11 @@ class NetworkedTests(unittest.TestCase): def test_connect(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertEqual({}, s.getpeercert()) finally: s.close() @@ -772,27 +775,27 @@ s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # this should succeed because we specify the root cert s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertTrue(s.getpeercert()) finally: s.close() def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(0, s.connect_ex(("svn.python.org", 443))) + self.assertEqual(0, s.connect_ex((REMOTE_HOST, 443))) self.assertTrue(s.getpeercert()) finally: s.close() @@ -800,14 +803,14 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.setblocking(False) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) # EWOULDBLOCK under Windows, EINPROGRESS elsewhere self.assertIn(rc, (0, errno.EINPROGRESS, errno.EWOULDBLOCK)) # Wait for connect to finish @@ -829,61 +832,65 @@ def test_timeout_connect_ex(self): # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.settimeout(0.0000001) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) if rc == 0: - self.skipTest("svn.python.org responded too quickly") + self.skipTest("REMOTE_HOST responded too quickly") self.assertIn(rc, (errno.EAGAIN, errno.EWOULDBLOCK)) finally: s.close() def test_connect_ex_error(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - rc = s.connect_ex(("svn.python.org", 444)) + rc = s.connect_ex((REMOTE_HOST, 444)) # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. - self.assertIn(rc, (errno.ECONNREFUSED, errno.EWOULDBLOCK)) + errors = ( + errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.EWOULDBLOCK, + ) + self.assertIn(rc, errors) finally: s.close() def test_connect_with_context(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): # Same as test_connect, but with a separately created context ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: self.assertEqual({}, s.getpeercert()) finally: s.close() # Same with a server hostname s = ctx.wrap_socket(socket.socket(socket.AF_INET), - server_hostname="svn.python.org") + server_hostname=REMOTE_HOST) if ssl.HAS_SNI: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.close() else: - self.assertRaises(ValueError, s.connect, ("svn.python.org", 443)) + self.assertRaises(ValueError, s.connect, (REMOTE_HOST, 443)) # This should fail because we have no verification certs ctx.verify_mode = ssl.CERT_REQUIRED s = ctx.wrap_socket(socket.socket(socket.AF_INET)) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # This should succeed because we specify the root cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -896,12 +903,12 @@ # OpenSSL 0.9.8n and 1.0.0, as a result the capath directory must # contain both versions of each certificate (same content, different # filename) for this test to be portable across OpenSSL releases. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -912,7 +919,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=BYTES_CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -924,9 +931,9 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) fd = ss.fileno() f = ss.makefile() f.close() @@ -940,9 +947,9 @@ self.assertEqual(e.exception.errno, errno.EBADF) def test_non_blocking_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = socket.socket(socket.AF_INET) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.setblocking(False) s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE, @@ -988,12 +995,12 @@ if support.verbose: sys.stdout.write("\nVerified certificate for %s:%s is\n%s\n" % (host, port ,pem)) - _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT) + _test_get_server_certificate(REMOTE_HOST, 443, REMOTE_ROOT_CERT) if support.IPV6_ENABLED: _test_get_server_certificate('ipv6.google.com', 443) def test_ciphers(self): - remote = ("svn.python.org", 443) + remote = (REMOTE_HOST, 443) with support.transient_internet(remote[0]): with ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL") as s: @@ -2150,7 +2157,7 @@ print(" HAS_SNI = %r" % ssl.HAS_SNI) for filename in [ - CERTFILE, SVN_PYTHON_ORG_ROOT_CERT, BYTES_CERTFILE, + CERTFILE, REMOTE_ROOT_CERT, BYTES_CERTFILE, ONLYCERT, ONLYKEY, BYTES_ONLYCERT, BYTES_ONLYKEY, BADCERT, BADKEY, EMPTYCERT]: if not os.path.exists(filename): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -52,6 +52,13 @@ - Issue #23998: PyImport_ReInitLock() now checks for lock allocation error +Tests +----- + +- Issue #25940: Changed test_ssl and test_httplib to use + self-signed.pythontest.net. This avoids relying on svn.python.org, which + recently changed root certificate. + What's New in Python 3.3.6? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:16 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_Issue_=2325940=3A_Merge_self-signed=2Epythontest=2Enet_testing?= =?utf-8?q?_from_3=2E3_into_3=2E4?= Message-ID: <20160114133915.106553.8787@psf.io> https://hg.python.org/cpython/rev/32bcb9aa286e changeset: 99890:32bcb9aa286e branch: 3.4 parent: 99845:db11d0db1598 parent: 99889:e167a9296947 user: Martin Panter date: Thu Jan 14 12:46:49 2016 +0000 summary: Issue #25940: Merge self-signed.pythontest.net testing from 3.3 into 3.4 files: Lib/test/capath/0e4015b9.0 | 16 ++ Lib/test/capath/ce7b8643.0 | 16 ++ Lib/test/https_svn_python_org_root.pem | 41 ----- Lib/test/selfsigned_pythontestdotnet.pem | 12 +- Lib/test/test_ssl.py | 83 ++++++----- Misc/NEWS | 6 + 6 files changed, 88 insertions(+), 86 deletions(-) diff --git a/Lib/test/capath/0e4015b9.0 b/Lib/test/capath/0e4015b9.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/0e4015b9.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/capath/ce7b8643.0 b/Lib/test/capath/ce7b8643.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/ce7b8643.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/https_svn_python_org_root.pem b/Lib/test/https_svn_python_org_root.pem deleted file mode 100644 --- a/Lib/test/https_svn_python_org_root.pem +++ /dev/null @@ -1,41 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 -IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB -IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA -Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO -BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi -MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ -ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC -CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ -8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 -zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y -fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 -w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc -G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k -epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q -laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ -QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU -fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 -YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w -ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY -gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe -MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 -IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy -dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw -czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 -dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl -aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC -AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg -b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB -ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc -nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg -18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c -gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl -Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY -sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T -SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF -CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum -GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk -zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW -omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD ------END CERTIFICATE----- diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem --- a/Lib/test/selfsigned_pythontestdotnet.pem +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -1,5 +1,5 @@ -----BEGIN CERTIFICATE----- -MIIChzCCAfCgAwIBAgIJAKGU95wKR8pSMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG @@ -8,9 +8,9 @@ aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv -EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjKTAnMCUGA1UdEQQeMByCGnNl -bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MA0GCSqGSIb3DQEBBQUAA4GBAIOXmdtM -eG9qzP9TiXW/Gc/zI4cBfdCpC+Y4gOfC9bQUC7hefix4iO3+iZjgy3X/FaRxUUoV -HKiXcXIaWqTSUWp45cSh0MbwZXudp6JIAptzdAhvvCrPKeC9i9GvxsPD4LtDAL97 -vSaxQBezA7hdxZd90/EeyMgVZgAnTCnvAWX9 +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= -----END CERTIFICATE----- diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -55,7 +55,8 @@ SIGNED_CERTFILE2 = data_file("keycert4.pem") SIGNING_CA = data_file("pycacert.pem") -SVN_PYTHON_ORG_ROOT_CERT = data_file("https_svn_python_org_root.pem") +REMOTE_HOST = "self-signed.pythontest.net" +REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") EMPTYCERT = data_file("nullcert.pem") BADCERT = data_file("badcert.pem") @@ -260,7 +261,7 @@ self.assertEqual(p['subjectAltName'], san) def test_DER_to_PEM(self): - with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f: + with open(CAFILE_CACERT, 'r') as f: pem = f.read() d1 = ssl.PEM_cert_to_DER_cert(pem) p2 = ssl.DER_cert_to_PEM_cert(d1) @@ -752,7 +753,7 @@ # Mismatching key and cert ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with self.assertRaisesRegex(ssl.SSLError, "key values mismatch"): - ctx.load_cert_chain(SVN_PYTHON_ORG_ROOT_CERT, ONLYKEY) + ctx.load_cert_chain(CAFILE_CACERT, ONLYKEY) # Password protected key and cert ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD) ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD.encode()) @@ -1172,11 +1173,11 @@ class NetworkedTests(unittest.TestCase): def test_connect(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertEqual({}, s.getpeercert()) finally: s.close() @@ -1185,27 +1186,27 @@ s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # this should succeed because we specify the root cert s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertTrue(s.getpeercert()) finally: s.close() def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(0, s.connect_ex(("svn.python.org", 443))) + self.assertEqual(0, s.connect_ex((REMOTE_HOST, 443))) self.assertTrue(s.getpeercert()) finally: s.close() @@ -1213,14 +1214,14 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.setblocking(False) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) # EWOULDBLOCK under Windows, EINPROGRESS elsewhere self.assertIn(rc, (0, errno.EINPROGRESS, errno.EWOULDBLOCK)) # Wait for connect to finish @@ -1242,58 +1243,62 @@ def test_timeout_connect_ex(self): # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.settimeout(0.0000001) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) if rc == 0: - self.skipTest("svn.python.org responded too quickly") + self.skipTest("REMOTE_HOST responded too quickly") self.assertIn(rc, (errno.EAGAIN, errno.EWOULDBLOCK)) finally: s.close() def test_connect_ex_error(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - rc = s.connect_ex(("svn.python.org", 444)) + rc = s.connect_ex((REMOTE_HOST, 444)) # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. - self.assertIn(rc, (errno.ECONNREFUSED, errno.EWOULDBLOCK)) + errors = ( + errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.EWOULDBLOCK, + ) + self.assertIn(rc, errors) finally: s.close() def test_connect_with_context(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): # Same as test_connect, but with a separately created context ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: self.assertEqual({}, s.getpeercert()) finally: s.close() # Same with a server hostname s = ctx.wrap_socket(socket.socket(socket.AF_INET), - server_hostname="svn.python.org") - s.connect(("svn.python.org", 443)) + server_hostname=REMOTE_HOST) + s.connect((REMOTE_HOST, 443)) s.close() # This should fail because we have no verification certs ctx.verify_mode = ssl.CERT_REQUIRED s = ctx.wrap_socket(socket.socket(socket.AF_INET)) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # This should succeed because we specify the root cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1306,12 +1311,12 @@ # OpenSSL 0.9.8n and 1.0.0, as a result the capath directory must # contain both versions of each certificate (same content, different # filename) for this test to be portable across OpenSSL releases. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1322,7 +1327,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=BYTES_CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1356,9 +1361,9 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) fd = ss.fileno() f = ss.makefile() f.close() @@ -1372,9 +1377,9 @@ self.assertEqual(e.exception.errno, errno.EBADF) def test_non_blocking_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = socket.socket(socket.AF_INET) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.setblocking(False) s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE, @@ -1420,12 +1425,12 @@ if support.verbose: sys.stdout.write("\nVerified certificate for %s:%s is\n%s\n" % (host, port ,pem)) - _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT) + _test_get_server_certificate(REMOTE_HOST, 443, REMOTE_ROOT_CERT) if support.IPV6_ENABLED: _test_get_server_certificate('ipv6.google.com', 443) def test_ciphers(self): - remote = ("svn.python.org", 443) + remote = (REMOTE_HOST, 443) with support.transient_internet(remote[0]): with ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL") as s: @@ -2913,7 +2918,7 @@ pass for filename in [ - CERTFILE, SVN_PYTHON_ORG_ROOT_CERT, BYTES_CERTFILE, + CERTFILE, REMOTE_ROOT_CERT, BYTES_CERTFILE, ONLYCERT, ONLYKEY, BYTES_ONLYCERT, BYTES_ONLYKEY, SIGNED_CERTFILE, SIGNED_CERTFILE2, SIGNING_CA, BADCERT, BADKEY, EMPTYCERT]: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -33,6 +33,12 @@ - Issue #26050: Add asyncio.StreamReader.readuntil() method. Patch by ???? ?????????. +Tests +----- + +- Issue #25940: Changed test_ssl to use self-signed.pythontest.net. This + avoids relying on svn.python.org, which recently changed root certificate. + What's New in Python 3.4.4? =========================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:16 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1OTQw?= =?utf-8?q?=3A_Update_new_SSL_tests_for_self-signed=2Epythontest=2Enet?= Message-ID: <20160114133916.19328.36182@psf.io> https://hg.python.org/cpython/rev/f5f14d65297e changeset: 99893:f5f14d65297e branch: 3.5 user: Martin Panter date: Thu Jan 14 13:05:46 2016 +0000 summary: Issue #25940: Update new SSL tests for self-signed.pythontest.net Removed SSL_ERROR_SYSCALL checking from ssl_io_loop() so that the loop can terminate when unwrap() raises that error. files: Lib/test/test_ssl.py | 27 ++++++++++++++------------- 1 files changed, 14 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1688,13 +1688,8 @@ try: ret = func(*args) except ssl.SSLError as e: - # Note that we get a spurious -1/SSL_ERROR_SYSCALL for - # non-blocking IO. The SSL_shutdown manpage hints at this. - # It *should* be safe to just ignore SYS_ERROR_SYSCALL because - # with a Memory BIO there's no syscalls (for IO at least). if e.errno not in (ssl.SSL_ERROR_WANT_READ, - ssl.SSL_ERROR_WANT_WRITE, - ssl.SSL_ERROR_SYSCALL): + ssl.SSL_ERROR_WANT_WRITE): raise errno = e.errno # Get any data from the outgoing BIO irrespective of any error, and @@ -1717,16 +1712,16 @@ return ret def test_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): sock = socket.socket(socket.AF_INET) - sock.connect(("svn.python.org", 443)) + sock.connect((REMOTE_HOST, 443)) incoming = ssl.MemoryBIO() outgoing = ssl.MemoryBIO() ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) ctx.check_hostname = True - sslobj = ctx.wrap_bio(incoming, outgoing, False, 'svn.python.org') + sslobj = ctx.wrap_bio(incoming, outgoing, False, REMOTE_HOST) self.assertIs(sslobj._sslobj.owner, sslobj) self.assertIsNone(sslobj.cipher()) self.assertIsNone(sslobj.shared_ciphers()) @@ -1739,14 +1734,20 @@ self.assertTrue(sslobj.getpeercert()) if 'tls-unique' in ssl.CHANNEL_BINDING_TYPES: self.assertTrue(sslobj.get_channel_binding('tls-unique')) - self.ssl_io_loop(sock, incoming, outgoing, sslobj.unwrap) + try: + self.ssl_io_loop(sock, incoming, outgoing, sslobj.unwrap) + except ssl.SSLSyscallError: + # self-signed.pythontest.net probably shuts down the TCP + # connection without sending a secure shutdown message, and + # this is reported as SSL_ERROR_SYSCALL + pass self.assertRaises(ssl.SSLError, sslobj.write, b'foo') sock.close() def test_read_write_data(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): sock = socket.socket(socket.AF_INET) - sock.connect(("svn.python.org", 443)) + sock.connect((REMOTE_HOST, 443)) incoming = ssl.MemoryBIO() outgoing = ssl.MemoryBIO() ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:18 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325940=3A_test=5Fs?= =?utf-8?q?sl_is_working_again?= Message-ID: <20160114133917.14796.31655@psf.io> https://hg.python.org/cpython/rev/ac94418299bd changeset: 99895:ac94418299bd user: Martin Panter date: Thu Jan 14 13:26:43 2016 +0000 summary: Issue #25940: test_ssl is working again files: Lib/test/test_ssl.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1343,7 +1343,6 @@ self.assertRaises(TypeError, bio.write, 1) - at unittest.skipIf(True, "temporarily disabled: see #25940") class NetworkedTests(unittest.TestCase): def test_connect(self): @@ -1712,7 +1711,6 @@ % (count, func.__name__)) return ret - @unittest.skipIf(True, "temporarily disabled: see #25940") def test_handshake(self): with support.transient_internet(REMOTE_HOST): sock = socket.socket(socket.AF_INET) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:18 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI1OTQw?= =?utf-8?q?=3A_Update_new_SSL_tests_for_self-signed=2Epythontest=2Enet?= Message-ID: <20160114133916.14808.11221@psf.io> https://hg.python.org/cpython/rev/0585c0089466 changeset: 99891:0585c0089466 branch: 3.4 user: Martin Panter date: Thu Jan 14 12:53:56 2016 +0000 summary: Issue #25940: Update new SSL tests for self-signed.pythontest.net files: Lib/test/test_ssl.py | 24 ++++++++++++------------ 1 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -971,7 +971,7 @@ ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 0, 'crl': 0, 'x509': 1}) - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 1, 'crl': 0, 'x509': 2}) @@ -981,8 +981,8 @@ # CERTFILE is not flagged as X509v3 Basic Constraints: CA:TRUE ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.get_ca_certs(), []) - # but SVN_PYTHON_ORG_ROOT_CERT is a CA cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + # but CAFILE_CACERT is a CA cert + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.get_ca_certs(), [{'issuer': ((('organizationName', 'Root CA'),), (('organizationalUnitName', 'http://www.cacert.org'),), @@ -998,7 +998,7 @@ (('emailAddress', 'support at cacert.org'),)), 'version': 3}]) - with open(SVN_PYTHON_ORG_ROOT_CERT) as f: + with open(CAFILE_CACERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) self.assertEqual(ctx.get_ca_certs(True), [der]) @@ -1335,15 +1335,15 @@ s.close() def test_connect_cadata(self): - with open(CAFILE_CACERT) as f: + with open(REMOTE_ROOT_CERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=pem) with ctx.wrap_socket(socket.socket(socket.AF_INET)) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1352,7 +1352,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=der) with ctx.wrap_socket(socket.socket(socket.AF_INET)) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1475,13 +1475,13 @@ def test_get_ca_certs_capath(self): # capath certs are loaded on request - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) self.assertEqual(ctx.get_ca_certs(), []) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1492,12 +1492,12 @@ @needs_sni def test_context_setget(self): # Check that the context of a connected socket can be replaced. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx1 = ssl.SSLContext(ssl.PROTOCOL_TLSv1) ctx2 = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = socket.socket(socket.AF_INET) with ctx1.wrap_socket(s) as ss: - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) self.assertIs(ss.context, ctx1) self.assertIs(ss._sslobj.context, ctx1) ss.context = ctx2 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:18 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325940=3A_Merge_self-signed=2Epythontest=2Enet_testing?= =?utf-8?q?_from_3=2E4_into_3=2E5?= Message-ID: <20160114133916.77976.43950@psf.io> https://hg.python.org/cpython/rev/c3aa4b48b905 changeset: 99892:c3aa4b48b905 branch: 3.5 parent: 99885:d48948c13473 parent: 99891:0585c0089466 user: Martin Panter date: Thu Jan 14 13:22:29 2016 +0000 summary: Issue #25940: Merge self-signed.pythontest.net testing from 3.4 into 3.5 files: Lib/test/capath/0e4015b9.0 | 16 + Lib/test/capath/ce7b8643.0 | 16 + Lib/test/https_svn_python_org_root.pem | 41 ---- Lib/test/selfsigned_pythontestdotnet.pem | 12 +- Lib/test/test_ssl.py | 107 +++++----- Misc/NEWS | 3 + 6 files changed, 97 insertions(+), 98 deletions(-) diff --git a/Lib/test/capath/0e4015b9.0 b/Lib/test/capath/0e4015b9.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/0e4015b9.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/capath/ce7b8643.0 b/Lib/test/capath/ce7b8643.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/ce7b8643.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/https_svn_python_org_root.pem b/Lib/test/https_svn_python_org_root.pem deleted file mode 100644 --- a/Lib/test/https_svn_python_org_root.pem +++ /dev/null @@ -1,41 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 -IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB -IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA -Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO -BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi -MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ -ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC -CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ -8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 -zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y -fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 -w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc -G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k -epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q -laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ -QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU -fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 -YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w -ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY -gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe -MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 -IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy -dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw -czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 -dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl -aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC -AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg -b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB -ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc -nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg -18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c -gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl -Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY -sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T -SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF -CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum -GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk -zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW -omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD ------END CERTIFICATE----- diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem --- a/Lib/test/selfsigned_pythontestdotnet.pem +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -1,5 +1,5 @@ -----BEGIN CERTIFICATE----- -MIIChzCCAfCgAwIBAgIJAKGU95wKR8pSMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG @@ -8,9 +8,9 @@ aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv -EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjKTAnMCUGA1UdEQQeMByCGnNl -bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MA0GCSqGSIb3DQEBBQUAA4GBAIOXmdtM -eG9qzP9TiXW/Gc/zI4cBfdCpC+Y4gOfC9bQUC7hefix4iO3+iZjgy3X/FaRxUUoV -HKiXcXIaWqTSUWp45cSh0MbwZXudp6JIAptzdAhvvCrPKeC9i9GvxsPD4LtDAL97 -vSaxQBezA7hdxZd90/EeyMgVZgAnTCnvAWX9 +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= -----END CERTIFICATE----- diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -55,7 +55,8 @@ SIGNED_CERTFILE2 = data_file("keycert4.pem") SIGNING_CA = data_file("pycacert.pem") -SVN_PYTHON_ORG_ROOT_CERT = data_file("https_svn_python_org_root.pem") +REMOTE_HOST = "self-signed.pythontest.net" +REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") EMPTYCERT = data_file("nullcert.pem") BADCERT = data_file("badcert.pem") @@ -276,7 +277,7 @@ self.assertEqual(p['subjectAltName'], san) def test_DER_to_PEM(self): - with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f: + with open(CAFILE_CACERT, 'r') as f: pem = f.read() d1 = ssl.PEM_cert_to_DER_cert(pem) p2 = ssl.DER_cert_to_PEM_cert(d1) @@ -862,7 +863,7 @@ # Mismatching key and cert ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with self.assertRaisesRegex(ssl.SSLError, "key values mismatch"): - ctx.load_cert_chain(SVN_PYTHON_ORG_ROOT_CERT, ONLYKEY) + ctx.load_cert_chain(CAFILE_CACERT, ONLYKEY) # Password protected key and cert ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD) ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD.encode()) @@ -1080,7 +1081,7 @@ ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 0, 'crl': 0, 'x509': 1}) - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 1, 'crl': 0, 'x509': 2}) @@ -1090,8 +1091,8 @@ # CERTFILE is not flagged as X509v3 Basic Constraints: CA:TRUE ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.get_ca_certs(), []) - # but SVN_PYTHON_ORG_ROOT_CERT is a CA cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + # but CAFILE_CACERT is a CA cert + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.get_ca_certs(), [{'issuer': ((('organizationName', 'Root CA'),), (('organizationalUnitName', 'http://www.cacert.org'),), @@ -1107,7 +1108,7 @@ (('emailAddress', 'support at cacert.org'),)), 'version': 3}]) - with open(SVN_PYTHON_ORG_ROOT_CERT) as f: + with open(CAFILE_CACERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) self.assertEqual(ctx.get_ca_certs(True), [der]) @@ -1345,11 +1346,11 @@ class NetworkedTests(unittest.TestCase): def test_connect(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertEqual({}, s.getpeercert()) finally: s.close() @@ -1358,27 +1359,27 @@ s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # this should succeed because we specify the root cert s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertTrue(s.getpeercert()) finally: s.close() def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(0, s.connect_ex(("svn.python.org", 443))) + self.assertEqual(0, s.connect_ex((REMOTE_HOST, 443))) self.assertTrue(s.getpeercert()) finally: s.close() @@ -1386,14 +1387,14 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.setblocking(False) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) # EWOULDBLOCK under Windows, EINPROGRESS elsewhere self.assertIn(rc, (0, errno.EINPROGRESS, errno.EWOULDBLOCK)) # Wait for connect to finish @@ -1415,58 +1416,62 @@ def test_timeout_connect_ex(self): # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.settimeout(0.0000001) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) if rc == 0: - self.skipTest("svn.python.org responded too quickly") + self.skipTest("REMOTE_HOST responded too quickly") self.assertIn(rc, (errno.EAGAIN, errno.EWOULDBLOCK)) finally: s.close() def test_connect_ex_error(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - rc = s.connect_ex(("svn.python.org", 444)) + rc = s.connect_ex((REMOTE_HOST, 444)) # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. - self.assertIn(rc, (errno.ECONNREFUSED, errno.EWOULDBLOCK)) + errors = ( + errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.EWOULDBLOCK, + ) + self.assertIn(rc, errors) finally: s.close() def test_connect_with_context(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): # Same as test_connect, but with a separately created context ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: self.assertEqual({}, s.getpeercert()) finally: s.close() # Same with a server hostname s = ctx.wrap_socket(socket.socket(socket.AF_INET), - server_hostname="svn.python.org") - s.connect(("svn.python.org", 443)) + server_hostname=REMOTE_HOST) + s.connect((REMOTE_HOST, 443)) s.close() # This should fail because we have no verification certs ctx.verify_mode = ssl.CERT_REQUIRED s = ctx.wrap_socket(socket.socket(socket.AF_INET)) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # This should succeed because we specify the root cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1479,12 +1484,12 @@ # OpenSSL 0.9.8n and 1.0.0, as a result the capath directory must # contain both versions of each certificate (same content, different # filename) for this test to be portable across OpenSSL releases. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1495,7 +1500,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=BYTES_CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1503,15 +1508,15 @@ s.close() def test_connect_cadata(self): - with open(CAFILE_CACERT) as f: + with open(REMOTE_ROOT_CERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=pem) with ctx.wrap_socket(socket.socket(socket.AF_INET)) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1520,7 +1525,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=der) with ctx.wrap_socket(socket.socket(socket.AF_INET)) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1529,9 +1534,9 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) fd = ss.fileno() f = ss.makefile() f.close() @@ -1545,9 +1550,9 @@ self.assertEqual(e.exception.errno, errno.EBADF) def test_non_blocking_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = socket.socket(socket.AF_INET) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.setblocking(False) s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE, @@ -1590,12 +1595,12 @@ if support.verbose: sys.stdout.write("\nVerified certificate for %s:%s is\n%s\n" % (host, port ,pem)) - _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT) + _test_get_server_certificate(REMOTE_HOST, 443, REMOTE_ROOT_CERT) if support.IPV6_ENABLED: _test_get_server_certificate('ipv6.google.com', 443) def test_ciphers(self): - remote = ("svn.python.org", 443) + remote = (REMOTE_HOST, 443) with support.transient_internet(remote[0]): with ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL") as s: @@ -1640,13 +1645,13 @@ def test_get_ca_certs_capath(self): # capath certs are loaded on request - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) self.assertEqual(ctx.get_ca_certs(), []) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1657,12 +1662,12 @@ @needs_sni def test_context_setget(self): # Check that the context of a connected socket can be replaced. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx1 = ssl.SSLContext(ssl.PROTOCOL_TLSv1) ctx2 = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = socket.socket(socket.AF_INET) with ctx1.wrap_socket(s) as ss: - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) self.assertIs(ss.context, ctx1) self.assertIs(ss._sslobj.context, ctx1) ss.context = ctx2 @@ -3325,7 +3330,7 @@ pass for filename in [ - CERTFILE, SVN_PYTHON_ORG_ROOT_CERT, BYTES_CERTFILE, + CERTFILE, REMOTE_ROOT_CERT, BYTES_CERTFILE, ONLYCERT, ONLYKEY, BYTES_ONLYCERT, BYTES_ONLYKEY, SIGNED_CERTFILE, SIGNED_CERTFILE2, SIGNING_CA, BADCERT, BADKEY, EMPTYCERT]: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -145,6 +145,9 @@ Tests ----- +- Issue #25940: Changed test_ssl to use self-signed.pythontest.net. This + avoids relying on svn.python.org, which recently changed root certificate. + - Issue #25616: Tests for OrderedDict are extracted from test_collections into separate file test_ordered_dict. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 08:39:18 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 14 Jan 2016 13:39:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325940=3A_Merge_self-signed=2Epythontest=2Enet_t?= =?utf-8?q?esting_from_3=2E5?= Message-ID: <20160114133916.106541.7746@psf.io> https://hg.python.org/cpython/rev/2f2b42a8b34d changeset: 99894:2f2b42a8b34d parent: 99886:121fa7d22e22 parent: 99893:f5f14d65297e user: Martin Panter date: Thu Jan 14 13:25:06 2016 +0000 summary: Issue #25940: Merge self-signed.pythontest.net testing from 3.5 files: Lib/test/capath/0e4015b9.0 | 16 + Lib/test/capath/ce7b8643.0 | 16 + Lib/test/https_svn_python_org_root.pem | 41 --- Lib/test/selfsigned_pythontestdotnet.pem | 12 +- Lib/test/test_ssl.py | 134 +++++----- Misc/NEWS | 3 + 6 files changed, 111 insertions(+), 111 deletions(-) diff --git a/Lib/test/capath/0e4015b9.0 b/Lib/test/capath/0e4015b9.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/0e4015b9.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/capath/ce7b8643.0 b/Lib/test/capath/ce7b8643.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/ce7b8643.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/https_svn_python_org_root.pem b/Lib/test/https_svn_python_org_root.pem deleted file mode 100644 --- a/Lib/test/https_svn_python_org_root.pem +++ /dev/null @@ -1,41 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 -IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB -IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA -Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO -BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi -MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ -ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC -CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ -8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 -zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y -fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 -w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc -G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k -epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q -laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ -QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU -fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 -YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w -ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY -gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe -MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 -IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy -dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw -czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 -dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl -aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC -AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg -b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB -ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc -nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg -18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c -gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl -Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY -sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T -SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF -CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum -GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk -zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW -omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD ------END CERTIFICATE----- diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem --- a/Lib/test/selfsigned_pythontestdotnet.pem +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -1,5 +1,5 @@ -----BEGIN CERTIFICATE----- -MIIChzCCAfCgAwIBAgIJAKGU95wKR8pSMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG @@ -8,9 +8,9 @@ aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv -EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjKTAnMCUGA1UdEQQeMByCGnNl -bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MA0GCSqGSIb3DQEBBQUAA4GBAIOXmdtM -eG9qzP9TiXW/Gc/zI4cBfdCpC+Y4gOfC9bQUC7hefix4iO3+iZjgy3X/FaRxUUoV -HKiXcXIaWqTSUWp45cSh0MbwZXudp6JIAptzdAhvvCrPKeC9i9GvxsPD4LtDAL97 -vSaxQBezA7hdxZd90/EeyMgVZgAnTCnvAWX9 +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= -----END CERTIFICATE----- diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -55,7 +55,8 @@ SIGNED_CERTFILE2 = data_file("keycert4.pem") SIGNING_CA = data_file("pycacert.pem") -SVN_PYTHON_ORG_ROOT_CERT = data_file("https_svn_python_org_root.pem") +REMOTE_HOST = "self-signed.pythontest.net" +REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") EMPTYCERT = data_file("nullcert.pem") BADCERT = data_file("badcert.pem") @@ -276,7 +277,7 @@ self.assertEqual(p['subjectAltName'], san) def test_DER_to_PEM(self): - with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f: + with open(CAFILE_CACERT, 'r') as f: pem = f.read() d1 = ssl.PEM_cert_to_DER_cert(pem) p2 = ssl.DER_cert_to_PEM_cert(d1) @@ -862,7 +863,7 @@ # Mismatching key and cert ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with self.assertRaisesRegex(ssl.SSLError, "key values mismatch"): - ctx.load_cert_chain(SVN_PYTHON_ORG_ROOT_CERT, ONLYKEY) + ctx.load_cert_chain(CAFILE_CACERT, ONLYKEY) # Password protected key and cert ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD) ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD.encode()) @@ -1080,7 +1081,7 @@ ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 0, 'crl': 0, 'x509': 1}) - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 1, 'crl': 0, 'x509': 2}) @@ -1090,8 +1091,8 @@ # CERTFILE is not flagged as X509v3 Basic Constraints: CA:TRUE ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.get_ca_certs(), []) - # but SVN_PYTHON_ORG_ROOT_CERT is a CA cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + # but CAFILE_CACERT is a CA cert + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.get_ca_certs(), [{'issuer': ((('organizationName', 'Root CA'),), (('organizationalUnitName', 'http://www.cacert.org'),), @@ -1107,7 +1108,7 @@ (('emailAddress', 'support at cacert.org'),)), 'version': 3}]) - with open(SVN_PYTHON_ORG_ROOT_CERT) as f: + with open(CAFILE_CACERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) self.assertEqual(ctx.get_ca_certs(True), [der]) @@ -1346,11 +1347,11 @@ class NetworkedTests(unittest.TestCase): def test_connect(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertEqual({}, s.getpeercert()) finally: s.close() @@ -1359,27 +1360,27 @@ s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # this should succeed because we specify the root cert s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertTrue(s.getpeercert()) finally: s.close() def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(0, s.connect_ex(("svn.python.org", 443))) + self.assertEqual(0, s.connect_ex((REMOTE_HOST, 443))) self.assertTrue(s.getpeercert()) finally: s.close() @@ -1387,14 +1388,14 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.setblocking(False) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) # EWOULDBLOCK under Windows, EINPROGRESS elsewhere self.assertIn(rc, (0, errno.EINPROGRESS, errno.EWOULDBLOCK)) # Wait for connect to finish @@ -1416,58 +1417,62 @@ def test_timeout_connect_ex(self): # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.settimeout(0.0000001) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) if rc == 0: - self.skipTest("svn.python.org responded too quickly") + self.skipTest("REMOTE_HOST responded too quickly") self.assertIn(rc, (errno.EAGAIN, errno.EWOULDBLOCK)) finally: s.close() def test_connect_ex_error(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - rc = s.connect_ex(("svn.python.org", 444)) + rc = s.connect_ex((REMOTE_HOST, 444)) # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. - self.assertIn(rc, (errno.ECONNREFUSED, errno.EWOULDBLOCK)) + errors = ( + errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.EWOULDBLOCK, + ) + self.assertIn(rc, errors) finally: s.close() def test_connect_with_context(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): # Same as test_connect, but with a separately created context ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: self.assertEqual({}, s.getpeercert()) finally: s.close() # Same with a server hostname s = ctx.wrap_socket(socket.socket(socket.AF_INET), - server_hostname="svn.python.org") - s.connect(("svn.python.org", 443)) + server_hostname=REMOTE_HOST) + s.connect((REMOTE_HOST, 443)) s.close() # This should fail because we have no verification certs ctx.verify_mode = ssl.CERT_REQUIRED s = ctx.wrap_socket(socket.socket(socket.AF_INET)) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # This should succeed because we specify the root cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1480,12 +1485,12 @@ # OpenSSL 0.9.8n and 1.0.0, as a result the capath directory must # contain both versions of each certificate (same content, different # filename) for this test to be portable across OpenSSL releases. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1496,7 +1501,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=BYTES_CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1504,15 +1509,15 @@ s.close() def test_connect_cadata(self): - with open(CAFILE_CACERT) as f: + with open(REMOTE_ROOT_CERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=pem) with ctx.wrap_socket(socket.socket(socket.AF_INET)) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1521,7 +1526,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=der) with ctx.wrap_socket(socket.socket(socket.AF_INET)) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1530,9 +1535,9 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) fd = ss.fileno() f = ss.makefile() f.close() @@ -1546,9 +1551,9 @@ self.assertEqual(e.exception.errno, errno.EBADF) def test_non_blocking_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = socket.socket(socket.AF_INET) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.setblocking(False) s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE, @@ -1591,12 +1596,12 @@ if support.verbose: sys.stdout.write("\nVerified certificate for %s:%s is\n%s\n" % (host, port ,pem)) - _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT) + _test_get_server_certificate(REMOTE_HOST, 443, REMOTE_ROOT_CERT) if support.IPV6_ENABLED: _test_get_server_certificate('ipv6.google.com', 443) def test_ciphers(self): - remote = ("svn.python.org", 443) + remote = (REMOTE_HOST, 443) with support.transient_internet(remote[0]): with ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL") as s: @@ -1641,13 +1646,13 @@ def test_get_ca_certs_capath(self): # capath certs are loaded on request - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) self.assertEqual(ctx.get_ca_certs(), []) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1658,12 +1663,12 @@ @needs_sni def test_context_setget(self): # Check that the context of a connected socket can be replaced. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx1 = ssl.SSLContext(ssl.PROTOCOL_TLSv1) ctx2 = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = socket.socket(socket.AF_INET) with ctx1.wrap_socket(s) as ss: - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) self.assertIs(ss.context, ctx1) self.assertIs(ss._sslobj.context, ctx1) ss.context = ctx2 @@ -1684,13 +1689,8 @@ try: ret = func(*args) except ssl.SSLError as e: - # Note that we get a spurious -1/SSL_ERROR_SYSCALL for - # non-blocking IO. The SSL_shutdown manpage hints at this. - # It *should* be safe to just ignore SYS_ERROR_SYSCALL because - # with a Memory BIO there's no syscalls (for IO at least). if e.errno not in (ssl.SSL_ERROR_WANT_READ, - ssl.SSL_ERROR_WANT_WRITE, - ssl.SSL_ERROR_SYSCALL): + ssl.SSL_ERROR_WANT_WRITE): raise errno = e.errno # Get any data from the outgoing BIO irrespective of any error, and @@ -1714,16 +1714,16 @@ @unittest.skipIf(True, "temporarily disabled: see #25940") def test_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): sock = socket.socket(socket.AF_INET) - sock.connect(("svn.python.org", 443)) + sock.connect((REMOTE_HOST, 443)) incoming = ssl.MemoryBIO() outgoing = ssl.MemoryBIO() ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) ctx.check_hostname = True - sslobj = ctx.wrap_bio(incoming, outgoing, False, 'svn.python.org') + sslobj = ctx.wrap_bio(incoming, outgoing, False, REMOTE_HOST) self.assertIs(sslobj._sslobj.owner, sslobj) self.assertIsNone(sslobj.cipher()) self.assertIsNone(sslobj.shared_ciphers()) @@ -1736,14 +1736,20 @@ self.assertTrue(sslobj.getpeercert()) if 'tls-unique' in ssl.CHANNEL_BINDING_TYPES: self.assertTrue(sslobj.get_channel_binding('tls-unique')) - self.ssl_io_loop(sock, incoming, outgoing, sslobj.unwrap) + try: + self.ssl_io_loop(sock, incoming, outgoing, sslobj.unwrap) + except ssl.SSLSyscallError: + # self-signed.pythontest.net probably shuts down the TCP + # connection without sending a secure shutdown message, and + # this is reported as SSL_ERROR_SYSCALL + pass self.assertRaises(ssl.SSLError, sslobj.write, b'foo') sock.close() def test_read_write_data(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): sock = socket.socket(socket.AF_INET) - sock.connect(("svn.python.org", 443)) + sock.connect((REMOTE_HOST, 443)) incoming = ssl.MemoryBIO() outgoing = ssl.MemoryBIO() ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) @@ -3327,7 +3333,7 @@ pass for filename in [ - CERTFILE, SVN_PYTHON_ORG_ROOT_CERT, BYTES_CERTFILE, + CERTFILE, REMOTE_ROOT_CERT, BYTES_CERTFILE, ONLYCERT, ONLYKEY, BYTES_ONLYCERT, BYTES_ONLYKEY, SIGNED_CERTFILE, SIGNED_CERTFILE2, SIGNING_CA, BADCERT, BADKEY, EMPTYCERT]: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -564,6 +564,9 @@ Tests ----- +- Issue #25940: Changed test_ssl to use self-signed.pythontest.net. This + avoids relying on svn.python.org, which recently changed root certificate. + - Issue #25616: Tests for OrderedDict are extracted from test_collections into separate file test_ordered_dict. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 19:01:10 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 00:01:10 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_add_usages_to_the_?= =?utf-8?q?rationale?= Message-ID: <20160115000109.41348.71434@psf.io> https://hg.python.org/peps/rev/e4d6c57dc832 changeset: 6184:e4d6c57dc832 user: Victor Stinner date: Fri Jan 15 01:01:05 2016 +0100 summary: PEP 511: add usages to the rationale files: pep-0511.txt | 52 +++++++++++++++++++++++++++++++-------- 1 files changed, 41 insertions(+), 11 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -30,8 +30,14 @@ builtin ``compile()`` function. There are even more options to hook a code transformer. -Transforming the code allows to extend the Python language for specific -use cases. Transforming an Abstract Syntax Tree (AST) is a convenient +Python 3.4 added a ``compile_source()`` method to +``importlib.abc.SourceLoader``. But code transformation is wider than just +importing modules, see described use cases below. + +Usage 1: AST optimizer +---------------------- + +Transforming an Abstract Syntax Tree (AST) is a convenient way to implement an optimizer. It's easier to work on the AST than working on the bytecode, AST contains more information and is more high level. @@ -41,17 +47,29 @@ can only implement basic optimizations. The optimizer rewrites the bytecode. It is difficult to enhance it, because it written in C. -This PEP proposes to add an API to register AST transformers. +Usage 2: Preprocessor +--------------------- -A new ``-o OPTIM_TAG`` command line option is added to only load -transformed code: it changes the name of searched ``.pyc`` files. If the -``.pyc`` file of a module is missing and the ``.py`` is available, an -``ImportError`` exception is raised import if the AST transformers -required to transform the code are missing. The import behaviour with -the default optimizer tag (``'opt'``) is unchanged. +A preprocessor can be easily implemented with an AST transformer. A +preprocessor has various and different usages. Examples: -The transformation can done ahead of time. It allows to implement -powerful but expensive transformations. +* Remove debug code (like assertions and logs) to make the code faster to run + it for production. +* `Tail-call Optimization `_ +* Add profiling code +* Lazy macro create a memoizing thunk. + +Examples extending or changing the Python language: + +* Domain Specific Language (DSL) like SQL queries. The + Python language itself doesn't need to be modified. Previous attempts to + implement DSL for SQL like `PEP 335 - Overloadable Boolean Operators + `_ was rejected. +* Pattern Matching of functional languages +* String Interpolation, but `PEP 498 -- Literal String Interpolation + `_ was merged into Python 3.6. + +MacroPy has a much longer list of examples and use cases. Use Cases @@ -127,6 +145,18 @@ Changes ======= +This PEP proposes to add an API to register AST transformers. + +A new ``-o OPTIM_TAG`` command line option is added to only load +transformed code: it changes the name of searched ``.pyc`` files. If the +``.pyc`` file of a module is missing and the ``.py`` is available, an +``ImportError`` exception is raised import if the AST transformers +required to transform the code are missing. The import behaviour with +the default optimizer tag (``'opt'``) is unchanged. + +The transformation can done ahead of time. It allows to implement +powerful but expensive transformations. + API to support AST transformers: * Add ``sys.ast_transformers``: list of AST transformers used to rewrite -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 19:45:20 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 00:45:20 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511?= Message-ID: <20160115004518.25971.11015@psf.io> https://hg.python.org/peps/rev/10df1af7e4a1 changeset: 6185:10df1af7e4a1 user: Victor Stinner date: Fri Jan 15 01:44:58 2016 +0100 summary: PEP 511 * Fix the API: use sys.get/set_ast_transformers() * elaborate changes on importlib, try to be more concrete * add more examples of usage of AST transformers files: pep-0511.txt | 150 +++++++++++++++++++++++++++----------- 1 files changed, 107 insertions(+), 43 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -37,15 +37,40 @@ Usage 1: AST optimizer ---------------------- +Python 3.6 optimizes the code using a peephole optimizer. By +definition, a peephole optimizer has a narrow view of the code and so +can only implement basic optimizations. The optimizer rewrites the +bytecode. It is difficult to enhance it, because it written in C. + Transforming an Abstract Syntax Tree (AST) is a convenient way to implement an optimizer. It's easier to work on the AST than working on the bytecode, AST contains more information and is more high level. -Python 3.6 optimizes the code using a peephole optimizer. By -definition, a peephole optimizer has a narrow view of the code and so -can only implement basic optimizations. The optimizer rewrites the -bytecode. It is difficult to enhance it, because it written in C. +Example of optimizations which can be implemented with an AST optimizer: + +* `Copy propagation + `_: + replace ``x=1; y=x`` with ``x=1; y=1`` +* `Constant folding + `_: + replace ``1+1`` with ``2`` +* `Dead code elimination + `_ + +Using guards (see the PEP 510), it is possible to implement a much wider choice +of optimizations. Examples: + +* Simplify iterable: replace ``range(3)`` with ``(0, 1, 2)`` when used + as iterable +* `Loop unrolling `_ +* Call pure builtins: replace ``len("abc")`` with ``3`` +* Copy used builtin symbols to constants + +See also `optimizations implemented in fatoptimizer +`_, a +static optimizer for Python 3.6. + Usage 2: Preprocessor --------------------- @@ -53,14 +78,17 @@ A preprocessor can be easily implemented with an AST transformer. A preprocessor has various and different usages. Examples: -* Remove debug code (like assertions and logs) to make the code faster to run +* Remove debug code like assertions and logs to make the code faster to run it for production. * `Tail-call Optimization `_ * Add profiling code -* Lazy macro create a memoizing thunk. - -Examples extending or changing the Python language: - +* `Lazy evaluation `_: + see `lazy_python `_ + (bytecode transformer) and `lazy macro of MacroPy + `_ (AST transformer) +* Change dictionary literals into collection.OrderedDict instances +* Declare constants: see `@asconstants of codetransformer + `_ * Domain Specific Language (DSL) like SQL queries. The Python language itself doesn't need to be modified. Previous attempts to implement DSL for SQL like `PEP 335 - Overloadable Boolean Operators @@ -69,7 +97,8 @@ * String Interpolation, but `PEP 498 -- Literal String Interpolation `_ was merged into Python 3.6. -MacroPy has a much longer list of examples and use cases. +`MacroPy `_ has a long list of +examples and use cases. Use Cases @@ -147,38 +176,25 @@ This PEP proposes to add an API to register AST transformers. -A new ``-o OPTIM_TAG`` command line option is added to only load -transformed code: it changes the name of searched ``.pyc`` files. If the -``.pyc`` file of a module is missing and the ``.py`` is available, an -``ImportError`` exception is raised import if the AST transformers -required to transform the code are missing. The import behaviour with -the default optimizer tag (``'opt'``) is unchanged. - The transformation can done ahead of time. It allows to implement powerful but expensive transformations. -API to support AST transformers: -* Add ``sys.ast_transformers``: list of AST transformers used to rewrite - an AST tree. The list is empty by default: no AST transformer. -* Add ``sys.implementation.ast_transformers``: name of AST - transformers registered in ``sys.ast_transformers`` -* Add ``sys.implementation.optim_tag`` (``str``): optimization tag. - The default optimization tag is ``'opt'``. -* Use the optimizer tag in ``.pyc`` filenames in ``importlib``. - Remove also the special case for the optimizer level ``0`` with the - default optimizer tag ``'opt'`` to simplify the code. -* Add a new ``-o OPTIM_TAG`` command line option to set - ``sys.implementation.optim_tag`` +API for AST transformers +------------------------ -.. note:: - FIXME: There is a design issue: ``sys.ast_transformers`` and - ``sys.implementation.ast_transformers`` can be inconsistent. - ``sys.implementation.ast_transformers`` is required at runtime in - some corner cases to have specific code depending if a specific AST - transformer was used. Do you have a better suggestion? +Add new functions to register AST transformers: -API of an AST transformer (from ``sys.ast_transformers``): +* ``sys.set_ast_transformers(transformers)``: set the list of AST + transformers +* ``sys.get_ast_transformers()``: get the list of AST + transformers. + +The order of AST transformers matter. Running transformer A and then +transformer B can give a different output than running transformer B an +then transformer A. + +API of an AST transformer: * An AST transformer is a callable object with the prototype:: @@ -202,9 +218,55 @@ .. note:: It would be nice to pass the fully qualified name of a module in the *context* when an AST transformer is used to transform a module, but - it looks like the information is not available currently. + it looks like the information is not available in + ``PyParser_ASTFromStringObject()``. -AST transformer changes: + +Optimizer tag +------------- + +Changes: + +* Add ``sys.implementation.optim_tag`` (``str``): optimization tag. + The default optimization tag is ``'opt'``. +* Add a new ``-o OPTIM_TAG`` command line option to set + ``sys.implementation.optim_tag`` + +Changes on ``importlib``: + +* ``importlib`` uses ``sys.implementation.optim_tag`` to build the + ``.pyc`` filename to importing modules, instead of always using + ``opt``. Remove also the special case for the optimizer level ``0`` + with the default optimizer tag ``'opt'`` to simplify the code. +* When loading a module, if the ``.pyc`` file is missing but the ``.py`` + is available, the ``.py`` is only used if AST optimizers have the same + optimizer tag than the current tag, otherwise an ``ImportError`` + exception is raised. + +Pseudo-code of a ``use_py()`` function to decide if a ``.py`` file can +be compiled to import a module:: + + def get_ast_optim_tag(): + transformers = sys.get_ast_transformers() + if not transformers: + return 'opt' + return '-'.join(transformer.name for transformer in transformers) + + def use_py(): + return (get_ast_transformers() == sys.implementation.optim_tag) + +The order of ``sys.get_ast_transformers()`` matter. For example, the +``fat`` transformer followed by the ``pythran`` transformer gives the +optimizer tag ``fat-pythran``. + +The behaviour of the ``importlib`` module is unchanged with the default +optimizer tag (``'opt'``). + + +AST enhancements +---------------- + +Enhancements to simplify the implementation of AST transformers: * Add a new compiler flag ``PyCF_TRANSFORMED_AST`` to get the transformed AST. ``PyCF_ONLY_AST`` returns the AST before the @@ -215,9 +277,8 @@ frozenset items. * ``PyCodeObject.co_lnotab``: line number delta becomes signed to support moving instructions (note: need to modify MAGIC_NUMBER in - importlib). Implemented in the `issue #26107: code.co_lnotab: use - signed line number delta to support moving instructions in an - optimizer `_ + importlib). Implemented in the `issue #26107 + `_ * Enhance the bytecode compiler to support ``tuple`` and ``frozenset`` constants. Currently, ``tuple`` and ``frozenset`` constants are created by the peephole transformer, after the bytecode compilation. @@ -273,13 +334,16 @@ class ASTTransformer: name = "knights_who_say_ni" + def __init__(self): + self.transformer = KnightsWhoSayNi() + def __call__(self, tree, context): - KnightsWhoSayNi().visit(tree) + self.transformer.visit(tree) return tree # register the AST transformer - sys.ast_transformers.append(ASTTransformer()) + sys.set_ast_transformers([ASTTransformer()]) # execute code which will be transformed by ast_transformer() exec("print('Hello World!')") -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 20:09:40 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 15 Jan 2016 01:09:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1OTQw?= =?utf-8?q?=3A_Use_self-signed=2Epythontest=2Enet_in_SSL_tests?= Message-ID: <20160115010940.96379.60821@psf.io> https://hg.python.org/cpython/rev/fb7131939508 changeset: 99896:fb7131939508 branch: 2.7 parent: 99883:e3763d98c46e user: Martin Panter date: Fri Jan 15 00:25:29 2016 +0000 summary: Issue #25940: Use self-signed.pythontest.net in SSL tests This is instead of svn.python.org, whose certificate recently expired, and whose new certificate uses a different root certificate. The certificate used at the pythontest server was modifed to set the "basic constraints" CA flag. This flag seems to be required for test_get_ca_certs_ capath() to work. Added the new self-signed certificate to capath with the following commands: cp Lib/test/{selfsigned_pythontestdotnet.pem,capath/} c_rehash -v Lib/test/capath/ c_rehash -v -old Lib/test/capath/ # Note the generated file names cp Lib/test/capath/{selfsigned_pythontestdotnet.pem,0e4015b9.0} mv Lib/test/capath/{selfsigned_pythontestdotnet.pem,ce7b8643.0} When attempting to connect to port 444 on the new server, the resulting error code is EHOSTUNREACH on Linux, and ETIMEDOUT on Windows. files: Lib/test/capath/0e4015b9.0 | 16 + Lib/test/capath/ce7b8643.0 | 16 + Lib/test/https_svn_python_org_root.pem | 41 ---- Lib/test/selfsigned_pythontestdotnet.pem | 12 +- Lib/test/test_ssl.py | 107 +++++----- Misc/NEWS | 3 + 6 files changed, 97 insertions(+), 98 deletions(-) diff --git a/Lib/test/capath/0e4015b9.0 b/Lib/test/capath/0e4015b9.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/0e4015b9.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/capath/ce7b8643.0 b/Lib/test/capath/ce7b8643.0 new file mode 100644 --- /dev/null +++ b/Lib/test/capath/ce7b8643.0 @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv +bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG +A1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMwIQYDVQQKDBpQeXRo +b24gU29mdHdhcmUgRm91bmRhdGlvbjEjMCEGA1UEAwwac2VsZi1zaWduZWQucHl0 +aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ +Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm +Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= +-----END CERTIFICATE----- diff --git a/Lib/test/https_svn_python_org_root.pem b/Lib/test/https_svn_python_org_root.pem deleted file mode 100644 --- a/Lib/test/https_svn_python_org_root.pem +++ /dev/null @@ -1,41 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 -IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB -IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA -Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO -BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi -MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ -ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC -CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ -8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 -zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y -fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 -w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc -G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k -epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q -laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ -QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU -fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 -YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w -ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY -gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe -MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 -IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy -dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw -czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 -dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl -aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC -AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg -b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB -ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc -nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg -18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c -gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl -Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY -sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T -SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF -CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum -GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk -zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW -omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD ------END CERTIFICATE----- diff --git a/Lib/test/selfsigned_pythontestdotnet.pem b/Lib/test/selfsigned_pythontestdotnet.pem --- a/Lib/test/selfsigned_pythontestdotnet.pem +++ b/Lib/test/selfsigned_pythontestdotnet.pem @@ -1,5 +1,5 @@ -----BEGIN CERTIFICATE----- -MIIChzCCAfCgAwIBAgIJAKGU95wKR8pSMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV +MIIClTCCAf6gAwIBAgIJAKGU95wKR8pTMA0GCSqGSIb3DQEBBQUAMHAxCzAJBgNV BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u IFNvZnR3YXJlIEZvdW5kYXRpb24xIzAhBgNVBAMMGnNlbGYtc2lnbmVkLnB5dGhv bnRlc3QubmV0MB4XDTE0MTEwMjE4MDkyOVoXDTI0MTAzMDE4MDkyOVowcDELMAkG @@ -8,9 +8,9 @@ aG9udGVzdC5uZXQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANDXQXW9tjyZ Xt0Iv2tLL1+jinr4wGg36ioLDLFkMf+2Y1GL0v0BnKYG4N1OKlAU15LXGeGer8vm Sv/yIvmdrELvhAbbo3w4a9TMYQA4XkIVLdvu3mvNOAet+8PMJxn26dbDhG809ALv -EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjKTAnMCUGA1UdEQQeMByCGnNl -bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MA0GCSqGSIb3DQEBBQUAA4GBAIOXmdtM -eG9qzP9TiXW/Gc/zI4cBfdCpC+Y4gOfC9bQUC7hefix4iO3+iZjgy3X/FaRxUUoV -HKiXcXIaWqTSUWp45cSh0MbwZXudp6JIAptzdAhvvCrPKeC9i9GvxsPD4LtDAL97 -vSaxQBezA7hdxZd90/EeyMgVZgAnTCnvAWX9 +EHY57lQsBS3G59RZyBPVqAqmImWNJnVzAgMBAAGjNzA1MCUGA1UdEQQeMByCGnNl +bGYtc2lnbmVkLnB5dGhvbnRlc3QubmV0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcN +AQEFBQADgYEAIuzAhgMouJpNdf3URCHIineyoSt6WK/9+eyUcjlKOrDoXNZaD72h +TXMeKYoWvJyVcSLKL8ckPtDobgP2OTt0UkyAaj0n+ZHaqq1lH2yVfGUA1ILJv515 +C8BqbvVZuqm3i7ygmw3bqE/lYMgOrYtXXnqOrz6nvsE6Yc9V9rFflOM= -----END CERTIFICATE----- diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -57,7 +57,8 @@ SIGNED_CERTFILE2 = data_file("keycert4.pem") SIGNING_CA = data_file("pycacert.pem") -SVN_PYTHON_ORG_ROOT_CERT = data_file("https_svn_python_org_root.pem") +REMOTE_HOST = "self-signed.pythontest.net" +REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") EMPTYCERT = data_file("nullcert.pem") BADCERT = data_file("badcert.pem") @@ -244,7 +245,7 @@ self.assertEqual(p['subjectAltName'], san) def test_DER_to_PEM(self): - with open(SVN_PYTHON_ORG_ROOT_CERT, 'r') as f: + with open(CAFILE_CACERT, 'r') as f: pem = f.read() d1 = ssl.PEM_cert_to_DER_cert(pem) p2 = ssl.DER_cert_to_PEM_cert(d1) @@ -792,7 +793,7 @@ # Mismatching key and cert ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with self.assertRaisesRegexp(ssl.SSLError, "key values mismatch"): - ctx.load_cert_chain(SVN_PYTHON_ORG_ROOT_CERT, ONLYKEY) + ctx.load_cert_chain(CAFILE_CACERT, ONLYKEY) # Password protected key and cert ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD) ctx.load_cert_chain(CERTFILE_PROTECTED, password=KEY_PASSWORD.encode()) @@ -1013,7 +1014,7 @@ ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 0, 'crl': 0, 'x509': 1}) - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.cert_store_stats(), {'x509_ca': 1, 'crl': 0, 'x509': 2}) @@ -1023,8 +1024,8 @@ # CERTFILE is not flagged as X509v3 Basic Constraints: CA:TRUE ctx.load_verify_locations(CERTFILE) self.assertEqual(ctx.get_ca_certs(), []) - # but SVN_PYTHON_ORG_ROOT_CERT is a CA cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + # but CAFILE_CACERT is a CA cert + ctx.load_verify_locations(CAFILE_CACERT) self.assertEqual(ctx.get_ca_certs(), [{'issuer': ((('organizationName', 'Root CA'),), (('organizationalUnitName', 'http://www.cacert.org'),), @@ -1040,7 +1041,7 @@ (('emailAddress', 'support at cacert.org'),)), 'version': 3}]) - with open(SVN_PYTHON_ORG_ROOT_CERT) as f: + with open(CAFILE_CACERT) as f: pem = f.read() der = ssl.PEM_cert_to_DER_cert(pem) self.assertEqual(ctx.get_ca_certs(True), [der]) @@ -1215,11 +1216,11 @@ class NetworkedTests(unittest.TestCase): def test_connect(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertEqual({}, s.getpeercert()) finally: s.close() @@ -1228,27 +1229,27 @@ s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.assertRaisesRegexp(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # this should succeed because we specify the root cert s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) self.assertTrue(s.getpeercert()) finally: s.close() def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - self.assertEqual(0, s.connect_ex(("svn.python.org", 443))) + self.assertEqual(0, s.connect_ex((REMOTE_HOST, 443))) self.assertTrue(s.getpeercert()) finally: s.close() @@ -1256,14 +1257,14 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.setblocking(False) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) # EWOULDBLOCK under Windows, EINPROGRESS elsewhere self.assertIn(rc, (0, errno.EINPROGRESS, errno.EWOULDBLOCK)) # Wait for connect to finish @@ -1285,58 +1286,62 @@ def test_timeout_connect_ex(self): # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT, + ca_certs=REMOTE_ROOT_CERT, do_handshake_on_connect=False) try: s.settimeout(0.0000001) - rc = s.connect_ex(('svn.python.org', 443)) + rc = s.connect_ex((REMOTE_HOST, 443)) if rc == 0: - self.skipTest("svn.python.org responded too quickly") + self.skipTest("REMOTE_HOST responded too quickly") self.assertIn(rc, (errno.EAGAIN, errno.EWOULDBLOCK)) finally: s.close() def test_connect_ex_error(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, - ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + ca_certs=REMOTE_ROOT_CERT) try: - rc = s.connect_ex(("svn.python.org", 444)) + rc = s.connect_ex((REMOTE_HOST, 444)) # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. - self.assertIn(rc, (errno.ECONNREFUSED, errno.EWOULDBLOCK)) + errors = ( + errno.ECONNREFUSED, errno.EHOSTUNREACH, errno.ETIMEDOUT, + errno.EWOULDBLOCK, + ) + self.assertIn(rc, errors) finally: s.close() def test_connect_with_context(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): # Same as test_connect, but with a separately created context ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: self.assertEqual({}, s.getpeercert()) finally: s.close() # Same with a server hostname s = ctx.wrap_socket(socket.socket(socket.AF_INET), - server_hostname="svn.python.org") - s.connect(("svn.python.org", 443)) + server_hostname=REMOTE_HOST) + s.connect((REMOTE_HOST, 443)) s.close() # This should fail because we have no verification certs ctx.verify_mode = ssl.CERT_REQUIRED s = ctx.wrap_socket(socket.socket(socket.AF_INET)) self.assertRaisesRegexp(ssl.SSLError, "certificate verify failed", - s.connect, ("svn.python.org", 443)) + s.connect, (REMOTE_HOST, 443)) s.close() # This should succeed because we specify the root cert - ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT) + ctx.load_verify_locations(REMOTE_ROOT_CERT) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1349,12 +1354,12 @@ # OpenSSL 0.9.8n and 1.0.0, as a result the capath directory must # contain both versions of each certificate (same content, different # filename) for this test to be portable across OpenSSL releases. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1365,7 +1370,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=BYTES_CAPATH) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1373,15 +1378,15 @@ s.close() def test_connect_cadata(self): - with open(CAFILE_CACERT) as f: + with open(REMOTE_ROOT_CERT) as f: pem = f.read().decode('ascii') der = ssl.PEM_cert_to_DER_cert(pem) - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=pem) with closing(ctx.wrap_socket(socket.socket(socket.AF_INET))) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1390,7 +1395,7 @@ ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(cadata=der) with closing(ctx.wrap_socket(socket.socket(socket.AF_INET))) as s: - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) cert = s.getpeercert() self.assertTrue(cert) @@ -1399,9 +1404,9 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) fd = ss.fileno() f = ss.makefile() f.close() @@ -1415,9 +1420,9 @@ self.assertEqual(e.exception.errno, errno.EBADF) def test_non_blocking_handshake(self): - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): s = socket.socket(socket.AF_INET) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) s.setblocking(False) s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE, @@ -1460,12 +1465,12 @@ if support.verbose: sys.stdout.write("\nVerified certificate for %s:%s is\n%s\n" % (host, port ,pem)) - _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT) + _test_get_server_certificate(REMOTE_HOST, 443, REMOTE_ROOT_CERT) if support.IPV6_ENABLED: _test_get_server_certificate('ipv6.google.com', 443) def test_ciphers(self): - remote = ("svn.python.org", 443) + remote = (REMOTE_HOST, 443) with support.transient_internet(remote[0]): with closing(ssl.wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL")) as s: @@ -1510,13 +1515,13 @@ def test_get_ca_certs_capath(self): # capath certs are loaded on request - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_verify_locations(capath=CAPATH) self.assertEqual(ctx.get_ca_certs(), []) s = ctx.wrap_socket(socket.socket(socket.AF_INET)) - s.connect(("svn.python.org", 443)) + s.connect((REMOTE_HOST, 443)) try: cert = s.getpeercert() self.assertTrue(cert) @@ -1527,12 +1532,12 @@ @needs_sni def test_context_setget(self): # Check that the context of a connected socket can be replaced. - with support.transient_internet("svn.python.org"): + with support.transient_internet(REMOTE_HOST): ctx1 = ssl.SSLContext(ssl.PROTOCOL_TLSv1) ctx2 = ssl.SSLContext(ssl.PROTOCOL_SSLv23) s = socket.socket(socket.AF_INET) with closing(ctx1.wrap_socket(s)) as ss: - ss.connect(("svn.python.org", 443)) + ss.connect((REMOTE_HOST, 443)) self.assertIs(ss.context, ctx1) self.assertIs(ss._sslobj.context, ctx1) ss.context = ctx2 @@ -3026,7 +3031,7 @@ pass for filename in [ - CERTFILE, SVN_PYTHON_ORG_ROOT_CERT, BYTES_CERTFILE, + CERTFILE, REMOTE_ROOT_CERT, BYTES_CERTFILE, ONLYCERT, ONLYKEY, BYTES_ONLYCERT, BYTES_ONLYKEY, SIGNED_CERTFILE, SIGNED_CERTFILE2, SIGNING_CA, BADCERT, BADKEY, EMPTYCERT]: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -79,6 +79,9 @@ Tests ----- +- Issue #25940: Changed test_ssl to use self-signed.pythontest.net. This + avoids relying on svn.python.org, which recently changed root certificate. + - Issue #25616: Tests for OrderedDict are extracted from test_collections into separate file test_ordered_dict. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 20:14:52 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 01:14:52 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_add_list_of_AST_op?= =?utf-8?q?timizer_issues?= Message-ID: <20160115011452.41316.74026@psf.io> https://hg.python.org/peps/rev/2eef070a4602 changeset: 6186:2eef070a4602 user: Victor Stinner date: Fri Jan 15 02:14:44 2016 +0100 summary: PEP 511: add list of AST optimizer issues files: pep-0511.txt | 35 ++++++++++++++++++++++++++++++----- 1 files changed, 30 insertions(+), 5 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -34,6 +34,8 @@ ``importlib.abc.SourceLoader``. But code transformation is wider than just importing modules, see described use cases below. +Writing an optimizer or a preprocessor is out of the scope of this PEP. + Usage 1: AST optimizer ---------------------- @@ -58,18 +60,41 @@ * `Dead code elimination `_ -Using guards (see the PEP 510), it is possible to implement a much wider choice -of optimizations. Examples: +Using guards (see the `PEP 510 +`_), it is possible to +implement a much wider choice of optimizations. Examples: * Simplify iterable: replace ``range(3)`` with ``(0, 1, 2)`` when used as iterable * `Loop unrolling `_ * Call pure builtins: replace ``len("abc")`` with ``3`` * Copy used builtin symbols to constants +* See also `optimizations implemented in fatoptimizer + `_, + a static optimizer for Python 3.6. -See also `optimizations implemented in fatoptimizer -`_, a -static optimizer for Python 3.6. +The following issues can be implemented with an AST optimizer: + +* `Issue #1346238 + `_: A constant folding + optimization pass for the AST +* `Issue #2181 `_: + optimize out local variables at end of function +* `Issue #2499 `_: + Fold unary + and not on constants +* `Issue #4264 `_: + Patch: optimize code to use LIST_APPEND instead of calling list.append +* `Issue #7682 `_: + Optimisation of if with constant expression +* `Issue #10399 `_: AST + Optimization: inlining of function calls +* `Issue #11549 `_: + Build-out an AST optimizer, moving some functionality out of the + peephole optimizer +* `Issue #17068 `_: + peephole optimization for constant strings +* `Issue #17430 `_: + missed peephole optimization Usage 2: Preprocessor -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 20:39:08 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 01:39:08 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A?= Message-ID: <20160115013908.121304.97579@psf.io> https://hg.python.org/peps/rev/12438d762862 changeset: 6187:12438d762862 user: Victor Stinner date: Fri Jan 15 02:39:03 2016 +0100 summary: PEP 511: * usage: obfuscator! * impact on other python implementations * disable peephole optimizer * link to the old python-dev discussion files: pep-0511.txt | 32 ++++++++++++++++++++++++++++++-- 1 files changed, 30 insertions(+), 2 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -125,6 +125,9 @@ `MacroPy `_ has a long list of examples and use cases. +See also `PyXfuscator `_: Python +obfuscator, deobfuscator, and user-assisted decompiler. + Use Cases ========= @@ -378,6 +381,26 @@ Ni! Ni! Ni! +Other Python implementations +============================ + +The PEP 511 should be implemented be all Python implementation. The AST +emited by the parser is not specified. + +By the way, even between minor version of CPython, there are changes on +the AST API. There are differences, but only minor differences. It is +quite easy to write an AST transformer which works on Python 2.7 and +Python 3.5 for example. + + +Discussion +========== + +* `[Python-Dev] AST optimizer implemented in Python + `_ + (August 2012) + + Prior Art ========= @@ -426,8 +449,13 @@ intricacies. See `byteplay documentation `_. -See also `BytecodeAssembler -`_. +See also: + +* `BytecodeAssembler `_ +* `Issue #2506 `_: Add mechanism to + disable optimizations +* `[Python-ideas] Disable all peephole optimizations + `_ Copyright -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 14 21:46:47 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 15 Jan 2016 02:46:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_Issue_=2325940=3A_Merge_ETIMEDOUT_fix_from_3=2E2_into_3=2E3?= Message-ID: <20160115024647.96356.99392@psf.io> https://hg.python.org/cpython/rev/c5cae7366835 changeset: 99898:c5cae7366835 branch: 3.3 parent: 99889:e167a9296947 parent: 99897:b2036b717028 user: Martin Panter date: Fri Jan 15 02:08:13 2016 +0000 summary: Issue #25940: Merge ETIMEDOUT fix from 3.2 into 3.3 files: Lib/test/test_ssl.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -856,7 +856,7 @@ # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. errors = ( - errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.ECONNREFUSED, errno.EHOSTUNREACH, errno.ETIMEDOUT, errno.EWOULDBLOCK, ) self.assertIn(rc, errors) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 21:46:47 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 15 Jan 2016 02:46:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_Issue_=2325940=3A_Merge_ETIMEDOUT_fix_from_3=2E3_into_3=2E4?= Message-ID: <20160115024647.18405.94344@psf.io> https://hg.python.org/cpython/rev/a806ed21bd82 changeset: 99899:a806ed21bd82 branch: 3.4 parent: 99891:0585c0089466 parent: 99898:c5cae7366835 user: Martin Panter date: Fri Jan 15 02:18:31 2016 +0000 summary: Issue #25940: Merge ETIMEDOUT fix from 3.3 into 3.4 files: Lib/test/test_ssl.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1267,7 +1267,7 @@ # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. errors = ( - errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.ECONNREFUSED, errno.EHOSTUNREACH, errno.ETIMEDOUT, errno.EWOULDBLOCK, ) self.assertIn(rc, errors) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 21:46:47 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 15 Jan 2016 02:46:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2325940=3A_Merge_ETIMEDOUT_fix_from_3=2E4_into_3=2E5?= Message-ID: <20160115024647.14183.73131@psf.io> https://hg.python.org/cpython/rev/bd9d5bc8bc95 changeset: 99900:bd9d5bc8bc95 branch: 3.5 parent: 99893:f5f14d65297e parent: 99899:a806ed21bd82 user: Martin Panter date: Fri Jan 15 02:28:59 2016 +0000 summary: Issue #25940: Merge ETIMEDOUT fix from 3.4 into 3.5 files: Lib/test/test_ssl.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1440,7 +1440,7 @@ # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. errors = ( - errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.ECONNREFUSED, errno.EHOSTUNREACH, errno.ETIMEDOUT, errno.EWOULDBLOCK, ) self.assertIn(rc, errors) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 21:46:47 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 15 Jan 2016 02:46:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325940=3A_Merge_ETIMEDOUT_fix_from_3=2E5?= Message-ID: <20160115024647.5459.6646@psf.io> https://hg.python.org/cpython/rev/32a4e7b337c9 changeset: 99901:32a4e7b337c9 parent: 99895:ac94418299bd parent: 99900:bd9d5bc8bc95 user: Martin Panter date: Fri Jan 15 02:37:21 2016 +0000 summary: Issue #25940: Merge ETIMEDOUT fix from 3.5 files: Lib/test/test_ssl.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1440,7 +1440,7 @@ # Issue #19919: Windows machines or VMs hosted on Windows # machines sometimes return EWOULDBLOCK. errors = ( - errno.ECONNREFUSED, errno.EHOSTUNREACH, + errno.ECONNREFUSED, errno.EHOSTUNREACH, errno.ETIMEDOUT, errno.EWOULDBLOCK, ) self.assertIn(rc, errors) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Jan 14 21:46:47 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 15 Jan 2016 02:46:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4yKTogSXNzdWUgIzI1OTQw?= =?utf-8?q?=3A_On_Windows=2C_connecting_to_port_444_returns_ETIMEDOUT?= Message-ID: <20160115024647.25983.96834@psf.io> https://hg.python.org/cpython/rev/b2036b717028 changeset: 99897:b2036b717028 branch: 3.2 parent: 99888:adf750b1252d user: Martin Panter date: Fri Jan 15 01:16:41 2016 +0000 summary: Issue #25940: On Windows, connecting to port 444 returns ETIMEDOUT files: Lib/test/test_ssl.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -608,7 +608,9 @@ ca_certs=REMOTE_ROOT_CERT) try: rc = s.connect_ex((REMOTE_HOST, 444)) - self.assertIn(rc, (errno.ECONNREFUSED, errno.EHOSTUNREACH)) + errors = ( + errno.ECONNREFUSED, errno.EHOSTUNREACH, errno.ETIMEDOUT) + self.assertIn(rc, errors) finally: s.close() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 00:58:03 2016 From: python-checkins at python.org (senthil.kumaran) Date: Fri, 15 Jan 2016 05:58:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue26097_-_F?= =?utf-8?q?ix_the_TextTestRunner_documentation=2C_mention_all_the_argument?= =?utf-8?q?s_of?= Message-ID: <20160115055802.41316.35028@psf.io> https://hg.python.org/cpython/rev/687f0d4bc188 changeset: 99902:687f0d4bc188 branch: 2.7 parent: 99896:fb7131939508 user: Senthil Kumaran date: Thu Jan 14 21:57:57 2016 -0800 summary: Issue26097 - Fix the TextTestRunner documentation, mention all the arguments of the class. Patch contributed by Nicolas ?vrard. files: Doc/library/unittest.rst | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1764,9 +1764,10 @@ instead of repeatedly creating new instances. -.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1) - - A basic test runner implementation which prints results on standard error. It +.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1, \ + failfast=False, buffer=False, resultclass=None) + + A basic test runner implementation which prints results on standard error. It has a few configurable parameters, but is essentially very simple. Graphical applications which run test suites should provide alternate implementations. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 02:47:07 2016 From: python-checkins at python.org (nick.coghlan) Date: Fri, 15 Jan 2016 07:47:07 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_493=3A_link_to_issue_with?= =?utf-8?q?_the_implementation?= Message-ID: <20160115074706.41320.10204@psf.io> https://hg.python.org/peps/rev/6a8f5a92aa13 changeset: 6188:6a8f5a92aa13 user: Nick Coghlan date: Fri Jan 15 17:46:40 2016 +1000 summary: PEP 493: link to issue with the implementation files: pep-0493.txt | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/pep-0493.txt b/pep-0493.txt --- a/pep-0493.txt +++ b/pep-0493.txt @@ -256,6 +256,13 @@ inside an activated Python virtual environment. +Reference Implementation +======================== + +A patch for Python 2.7 implementing the above two features is attached to +the `relevant tracker issue `__. + + Backporting this PEP to earlier Python versions =============================================== -- Repository URL: https://hg.python.org/peps From solipsis at pitrou.net Fri Jan 15 03:42:03 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 15 Jan 2016 08:42:03 +0000 Subject: [Python-checkins] Daily reference leaks (32a4e7b337c9): sum=7 Message-ID: <20160115084202.14189.23970@psf.io> results for 32a4e7b337c9 on branch "default" -------------------------------------------- test_asyncio leaked [0, 3, 0] memory blocks, sum=3 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogA0nANU', '--timeout', '7200'] From lp_benchmark_robot at intel.com Fri Jan 15 04:18:52 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 15 Jan 2016 09:18:52 +0000 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-01-15 Message-ID: Results for project Python default, build date 2016-01-15 03:08:00 +0000 commit: 32a4e7b337c9 previous commit: 69aa17b1f894 revision date: 2016-01-15 02:37:21 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.25% 0.14% 11.58% 14.91% :-| pybench 0.10% 0.01% -1.79% 5.70% :-( regex_v8 2.58% -0.05% -4.91% 5.37% :-| nbody 0.09% -0.46% 1.10% 6.27% :-| json_dump_v2 0.16% 1.05% -1.70% 12.06% :-| normal_startup 0.90% -0.20% 0.01% 5.45% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-01-15/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Jan 15 04:33:32 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 15 Jan 2016 09:33:32 +0000 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-01-15 Message-ID: <8d6b977a-09eb-4b1f-8ea9-593e3a534b1a@irsmsx102.ger.corp.intel.com> Results for project Python 2.7, build date 2016-01-15 04:01:23 +0000 commit: fb7131939508 previous commit: e3763d98c46e revision date: 2016-01-15 00:25:29 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.41% 0.88% 4.33% 4.34% :-) pybench 0.15% -0.06% 6.26% 3.72% :-( regex_v8 1.09% 0.11% -2.30% 10.45% :-) nbody 0.13% -0.02% 7.17% 4.07% :-) json_dump_v2 0.20% 0.19% 5.42% 9.84% :-( normal_startup 1.72% 0.20% -5.73% 2.49% :-| ssbench 0.17% -0.13% 0.10% 1.99% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-01-15/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Jan 15 07:11:30 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 12:11:30 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_add_bytecode_trans?= =?utf-8?q?formers?= Message-ID: <20160115121129.41320.28259@psf.io> https://hg.python.org/peps/rev/95de80b56938 changeset: 6189:95de80b56938 user: Victor Stinner date: Fri Jan 15 12:54:09 2016 +0100 summary: PEP 511: add bytecode transformers files: pep-0511.txt | 107 +++++++++++++++++++++----------------- 1 files changed, 60 insertions(+), 47 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -1,5 +1,5 @@ PEP: 511 -Title: API for AST transformers +Title: API for code transformers Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner @@ -12,12 +12,12 @@ Abstract ======== -Propose an API to support AST transformers. Add also ``-o OPTIM_TAG`` -command line option to change ``.pyc`` filenames. Raise an +Propose an API to register AST and bytecode transformers. Add also ``-o +OPTIM_TAG`` command line option to change ``.pyc`` filenames. Raise an ``ImportError`` exception on import if the ``.pyc`` file is missing and -the AST transformers required to transform the code are missing. -AST transformers are not needed code transformed ahead of time (loaded -from ``.pyc`` files). +the code transformers required to transform the code are missing. code +transformers are not needed code transformed ahead of time (loaded from +``.pyc`` files). Rationale @@ -132,18 +132,18 @@ Use Cases ========= -This section give examples of use cases explaining when and how AST +This section give examples of use cases explaining when and how code transformers will be used. Interactive interpreter ----------------------- -It will be possible to use AST transformers with the interactive +It will be possible to use code transformers with the interactive interpreter which is popular in Python and commonly used to demonstrate Python. The code is transformed at runtime and so the interpreter can be slower -when expensive AST transformers are used. +when expensive code transformers are used. Build a transformed package --------------------------- @@ -153,7 +153,7 @@ A transformer can have a configuration. The configuration is not stored in the package. -All ``.pyc`` files of the package must be transformed with the same AST +All ``.pyc`` files of the package must be transformed with the same code transformers and the same transformers configuration. It is possible to build different ``.pyc`` files using different @@ -181,7 +181,7 @@ optimizer tag (or some ``.pyc`` files are missing), the ``.pyc`` are created during the installation. -AST transformers of the optimizer tag are required. Otherwise, the +Code transformers of the optimizer tag are required. Otherwise, the installation fails with an error. @@ -191,10 +191,10 @@ It will be possible to execute transformed code. Raise an ``ImportError`` exception on import if the ``.pyc`` file of the -current optimizer tag is missing and the AST transformers required to +current optimizer tag is missing and the code transformers required to transform the code are missing. -The interesting point here is that AST transformers are not needed to +The interesting point here is that code transformers are not needed to execute the transformed code if all required ``.pyc`` files are already available. @@ -202,46 +202,57 @@ Changes ======= -This PEP proposes to add an API to register AST transformers. +This PEP proposes to add an API to register code transformers. The transformation can done ahead of time. It allows to implement powerful but expensive transformations. -API for AST transformers ------------------------- +API for code transformers +------------------------- -Add new functions to register AST transformers: +Add new functions to register code transformers: -* ``sys.set_ast_transformers(transformers)``: set the list of AST - transformers -* ``sys.get_ast_transformers()``: get the list of AST +* ``sys.set_code_transformers(transformers)``: set the list of code + transformers and update ``sys.implementation.optim_tag`` +* ``sys.get_code_transformers()``: get the list of code transformers. -The order of AST transformers matter. Running transformer A and then +The order of code transformers matter. Running transformer A and then transformer B can give a different output than running transformer B an then transformer A. -API of an AST transformer: +API of an code transformer: -* An AST transformer is a callable object with the prototype:: +* An code transformer is a class with ``ast_transformer()`` and/or + ``bytecode_transformer()`` methods and a ``name`` attribute. Example:: - def ast_transformer(tree, context): - ... - return tree + class MyTransformer: + name = 'my_transformer' - where *tree* is an AST tree and *context* is an object with a - ``filename`` attribute (``str``). New attributes may be added to - *context* in the future. + def bytecode_transformer(self, bytecode, ...): + ... + return tree -* It must return an AST tree. -* It must have a ``name`` attribute (``str``): short string used to identify an - optimizer. The name must not contain ``.`` (dot) nor ``-`` (dash) characters: - ``.`` is used to separated fields in a ``.pyc`` filename and ``-`` is used - to join AST transformer names to build the optimizer tag. -* The transformer is called after the creation of the AST by the parser - and before the compilation to bytecode -* It can modify the AST tree in place, or create a new AST tree. + def ast_transformer(self, tree, context): + ... + return tree + + ``bytecode_transformer()``: *bytecode* is bytes objects. + + ``ast_transformer()``: *tree* is an AST tree and *context* is an + object with a ``filename`` attribute (``str``). New attributes may be + added to *context* in the future. It must return an AST tree. It can + modify the AST tree in place, or create a new AST tree. + +* The ``name`` attribute (``str``) must be a short string used to + identify an optimizer. The name must not contain ``.`` (dot), ``-`` + (dash) characters or directory separators: ``.`` is used to separated + fields in a ``.pyc`` filename and ``-`` is used to join AST + transformer names to build the optimizer tag. +* The AST transformer is called after the creation of the AST by the + parser and before the compilation to bytecode +* The bytecode transformer is after the compilation to bytecode .. note:: It would be nice to pass the fully qualified name of a module in the @@ -267,23 +278,23 @@ ``opt``. Remove also the special case for the optimizer level ``0`` with the default optimizer tag ``'opt'`` to simplify the code. * When loading a module, if the ``.pyc`` file is missing but the ``.py`` - is available, the ``.py`` is only used if AST optimizers have the same - optimizer tag than the current tag, otherwise an ``ImportError`` + is available, the ``.py`` is only used if code optimizers have the + same optimizer tag than the current tag, otherwise an ``ImportError`` exception is raised. Pseudo-code of a ``use_py()`` function to decide if a ``.py`` file can be compiled to import a module:: - def get_ast_optim_tag(): - transformers = sys.get_ast_transformers() + def transformers_tag(): + transformers = sys.get_code_transformers() if not transformers: return 'opt' return '-'.join(transformer.name for transformer in transformers) def use_py(): - return (get_ast_transformers() == sys.implementation.optim_tag) + return (transformers_tag() == sys.implementation.optim_tag) -The order of ``sys.get_ast_transformers()`` matter. For example, the +The order of ``sys.get_code_transformers()`` matter. For example, the ``fat`` transformer followed by the ``pythran`` transformer gives the optimizer tag ``fat-pythran``. @@ -345,7 +356,7 @@ AST transformer ----------------- +--------------- Scary AST transformer replacing all strings with ``"Ni! Ni! Ni!"``:: @@ -365,13 +376,15 @@ def __init__(self): self.transformer = KnightsWhoSayNi() - def __call__(self, tree, context): + def ast_transformer(self, tree, context): self.transformer.visit(tree) return tree - # register the AST transformer - sys.set_ast_transformers([ASTTransformer()]) + # append our AST transformer after existing transformers + transformers = sys.get_code_transformers() + transformers.append(ASTTransformer()) + sys.set_code_transformers(transformers) # execute code which will be transformed by ast_transformer() exec("print('Hello World!')") -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Jan 15 10:52:44 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 15:52:44 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511?= Message-ID: <20160115155242.14195.8983@psf.io> https://hg.python.org/peps/rev/f49d35c25c49 changeset: 6190:f49d35c25c49 user: Victor Stinner date: Fri Jan 15 16:18:00 2016 +0100 summary: PEP 511 * bytecode => code * more fun with bytecode * enhance the Changes section files: pep-0511.txt | 258 ++++++++++++++++++++++++++++---------- 1 files changed, 190 insertions(+), 68 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -12,10 +12,11 @@ Abstract ======== -Propose an API to register AST and bytecode transformers. Add also ``-o -OPTIM_TAG`` command line option to change ``.pyc`` filenames. Raise an -``ImportError`` exception on import if the ``.pyc`` file is missing and -the code transformers required to transform the code are missing. code +Propose an API to register bytecode and AST transformers. Add also ``-o +OPTIM_TAG`` command line option to change ``.pyc`` filenames, ``-o +noopt`` disables the peephole optimizer. Raise an ``ImportError`` +exception on import if the ``.pyc`` file is missing and the code +transformers required to transform the code are missing. code transformers are not needed code transformed ahead of time (loaded from ``.pyc`` files). @@ -39,16 +40,14 @@ Usage 1: AST optimizer ---------------------- -Python 3.6 optimizes the code using a peephole optimizer. By -definition, a peephole optimizer has a narrow view of the code and so -can only implement basic optimizations. The optimizer rewrites the -bytecode. It is difficult to enhance it, because it written in C. - Transforming an Abstract Syntax Tree (AST) is a convenient way to implement an optimizer. It's easier to work on the AST than working on the bytecode, AST contains more information and is more high level. +Since the optimization can done ahead of time, complex but slow +optimizations can be implemented. + Example of optimizations which can be implemented with an AST optimizer: * `Copy propagation @@ -101,7 +100,9 @@ --------------------- A preprocessor can be easily implemented with an AST transformer. A -preprocessor has various and different usages. Examples: +preprocessor has various and different usages. + +Some examples: * Remove debug code like assertions and logs to make the code faster to run it for production. @@ -125,10 +126,49 @@ `MacroPy `_ has a long list of examples and use cases. +This PEP does not add any new code transformer. Using a code transformer +will require an external module and to register it manually. + See also `PyXfuscator `_: Python obfuscator, deobfuscator, and user-assisted decompiler. +Usage 3: Disable all optimization +--------------------------------- + +Ned Batchelder asked to add an option to disable the peephole optimizer +because it makes code coverage more difficult to implement. See the +discussion on the python-ideas mailing list: `Disable all peephole +optimizations +`_. + +This PEP adds a new ``-o noopt`` command line option to disable the +peephole optimizer. In Python, it's as easy as:: + + sys.set_code_transformers([]) + +It will fix the `Issue #2506 `_: Add +mechanism to disable optimizations. + + +Usage 4: Write new bytecode optimizers in Python +------------------------------------------------ + +Python 3.6 optimizes the code using a peephole optimizer. By +definition, a peephole optimizer has a narrow view of the code and so +can only implement basic optimizations. The optimizer rewrites the +bytecode. It is difficult to enhance it, because it written in C. + +With this PEP, it becomes possible to implement a new bytecode optimizer +in pure Python and experiment new optimizations. + +Some optimizations are easier to implement on the AST like constant +folding, but optimizations on the bytecode are still useful. For +example, when the AST is compiled to bytecode, useless jumps can be +emited because the compiler is naive and does not try to optimize +anything. + + Use Cases ========= @@ -199,17 +239,85 @@ available. +Code transformer API +==================== + +A code transformer is a class with ``ast_transformer()`` and/or +``code_transformer()`` methods (API described below) and a ``name`` +attribute. + +For efficiency, do not define a ``code_transformer()`` or +``ast_transformer()`` method if it does nothing. + +The ``name`` attribute (``str``) must be a short string used to identify +an optimizer. It is used to build a ``.pyc`` filename. The name must not +contain dots (``'.'``), dashes (``'-'``) or directory separators: dots +are used to separated fields in a ``.pyc`` filename and dashes areused +to join code transformer names to build the optimizer tag. + +.. note:: + It would be nice to pass the fully qualified name of a module in the + *context* when an AST transformer is used to transform a module on + import, but it looks like the information is not available in + ``PyParser_ASTFromStringObject()``. + + +code_transformer() +------------------ + +Prototype:: + + def code_transformer(code, consts, names, lnotab, context): + ... + return (code, consts, names, lnotab) + +Parameters: + +* *code*: the bytecode (``bytes``) +* *consts*: a sequence of constants +* *names*: tuple of variable names +* *lnotab*: table mapping instruction offsets to line numbers (``bytes``) + +The code transformer is run after the compilation to bytecode + + +ast_transformer() +------------------ + +Prototype:: + + def ast_transformer(tree, context): + ... + return tree + +Parameters: + +* *tree*: an AST tree +* *context*: an object with a ``filename`` attribute (``str``) + +It must return an AST tree. It can modify the AST tree in place, or +create a new AST tree. + +The AST transformer is called after the creation of the AST by the +parser and before the compilation to bytecode. New attributes may be +added to *context* in the future. + + Changes ======= -This PEP proposes to add an API to register code transformers. +In short, add: -The transformation can done ahead of time. It allows to implement -powerful but expensive transformations. +* ``-o OPTIM_TAG`` command line option +* ``ast.Constant`` +* ``ast.PyCF_TRANSFORMED_AST`` +* ``sys.get_code_transformers()`` +* ``sys.implementation.optim_tag`` +* ``sys.set_code_transformers(transformers)`` -API for code transformers -------------------------- +API to get/set code transformers +-------------------------------- Add new functions to register code transformers: @@ -222,43 +330,15 @@ transformer B can give a different output than running transformer B an then transformer A. -API of an code transformer: +Example to prepend a new code transformer:: -* An code transformer is a class with ``ast_transformer()`` and/or - ``bytecode_transformer()`` methods and a ``name`` attribute. Example:: + transformers = sys.get_code_transformers() + transformers.insert(0, new_cool_transformer) + sys.set_code_transformers(transformers) - class MyTransformer: - name = 'my_transformer' - - def bytecode_transformer(self, bytecode, ...): - ... - return tree - - def ast_transformer(self, tree, context): - ... - return tree - - ``bytecode_transformer()``: *bytecode* is bytes objects. - - ``ast_transformer()``: *tree* is an AST tree and *context* is an - object with a ``filename`` attribute (``str``). New attributes may be - added to *context* in the future. It must return an AST tree. It can - modify the AST tree in place, or create a new AST tree. - -* The ``name`` attribute (``str``) must be a short string used to - identify an optimizer. The name must not contain ``.`` (dot), ``-`` - (dash) characters or directory separators: ``.`` is used to separated - fields in a ``.pyc`` filename and ``-`` is used to join AST - transformer names to build the optimizer tag. -* The AST transformer is called after the creation of the AST by the - parser and before the compilation to bytecode -* The bytecode transformer is after the compilation to bytecode - -.. note:: - It would be nice to pass the fully qualified name of a module in the - *context* when an AST transformer is used to transform a module, but - it looks like the information is not available in - ``PyParser_ASTFromStringObject()``. +All AST tranformers are run sequentially (ex: the second transformer +gets the input of the first transformer), and then all bytecode +transformers are run sequentially. Optimizer tag @@ -269,7 +349,7 @@ * Add ``sys.implementation.optim_tag`` (``str``): optimization tag. The default optimization tag is ``'opt'``. * Add a new ``-o OPTIM_TAG`` command line option to set - ``sys.implementation.optim_tag`` + ``sys.implementation.optim_tag``. Changes on ``importlib``: @@ -302,6 +382,19 @@ optimizer tag (``'opt'``). +Peephole optimizer +------------------ + +By default, ``sys.implementation.optim_tag`` is ``opt`` and +``sys.get_code_transformers()`` returns a list of one code transformer: +the peephole optimizer (optimize the bytecode). + +Use ``-o noopt`` to disable the peephole optimizer. In this case, the +optimizer tag is ``noopt`` and no code transformer is registered. + +Using the ``-o opt`` option has not effect. + + AST enhancements ---------------- @@ -326,8 +419,8 @@ node type -Example -======= +Examples +======== .pyc filenames -------------- @@ -355,10 +448,40 @@ =========================== ================== +Bytecode transformer +-------------------- + +Scary bytecode transformer replacing all strings with +``"Ni! Ni! Ni!"``:: + + import sys + + + class BytecodeTransformer: + name = "knights_who_say_ni" + + def code_transformer(self, code, consts, names, lnotab, context): + consts = ['Ni! Ni! Ni!' if isinstance(const, str) else const + for const in consts] + return (code, consts, names, lnotab) + + + # replace existing code transformers with our bytecode transformer + sys.set_code_transformers([BytecodeTransformer()]) + + # execute code which will be transformed by ast_transformer() + exec("print('Hello World!')") + +Output:: + + Ni! Ni! Ni! + + AST transformer --------------- -Scary AST transformer replacing all strings with ``"Ni! Ni! Ni!"``:: +Similary to the bytecode transformer example, the AST transformer also +replaces all strings with ``"Ni! Ni! Ni!"``:: import ast import sys @@ -381,10 +504,8 @@ return tree - # append our AST transformer after existing transformers - transformers = sys.get_code_transformers() - transformers.append(ASTTransformer()) - sys.set_code_transformers(transformers) + # replace existing code transformers with the new AST transformer + sys.set_code_transformers([ASTTransformer()]) # execute code which will be transformed by ast_transformer() exec("print('Hello World!')") @@ -397,8 +518,8 @@ Other Python implementations ============================ -The PEP 511 should be implemented be all Python implementation. The AST -emited by the parser is not specified. +The PEP 511 should be implemented by all Python implementation, but the +bytecode and the AST are not standardized. By the way, even between minor version of CPython, there are changes on the AST API. There are differences, but only minor differences. It is @@ -432,8 +553,13 @@ the Python semantics since no guard is used to disable optimization if something changes. -Issue #17515: `Add sys.setasthook() to allow to use a custom AST -optimizer `_. +In 2015, Victor Stinner wrote the `fatoptimizer +`_ project, an AST optimizer +specializing functions using guards. + +The Issue #17515 `"Add sys.setasthook() to allow to use a custom AST" +optimizer `_ was a first attempt of +API for code transformers, but specific to AST. Python Preprocessors @@ -448,8 +574,8 @@ preprocessor directives in Python, like ``#define`` and ``#ifdef`` -Modify the bytecode -------------------- +Bytecode transformers +--------------------- * `codetransformer `_: Bytecode transformers for CPython inspired by the ``ast`` module?s @@ -465,10 +591,6 @@ See also: * `BytecodeAssembler `_ -* `Issue #2506 `_: Add mechanism to - disable optimizations -* `[Python-ideas] Disable all peephole optimizations - `_ Copyright -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Jan 15 11:12:37 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 16:12:37 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_reformat_to_send_b?= =?utf-8?q?y_email?= Message-ID: <20160115161235.111056.66040@psf.io> https://hg.python.org/peps/rev/4dca70ef4b97 changeset: 6191:4dca70ef4b97 user: Victor Stinner date: Fri Jan 15 17:12:31 2016 +0100 summary: PEP 511: reformat to send by email files: pep-0511.txt | 34 ++++++++++++++++------------------ 1 files changed, 16 insertions(+), 18 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -32,8 +32,8 @@ hook a code transformer. Python 3.4 added a ``compile_source()`` method to -``importlib.abc.SourceLoader``. But code transformation is wider than just -importing modules, see described use cases below. +``importlib.abc.SourceLoader``. But code transformation is wider than +just importing modules, see described use cases below. Writing an optimizer or a preprocessor is out of the scope of this PEP. @@ -104,8 +104,8 @@ Some examples: -* Remove debug code like assertions and logs to make the code faster to run - it for production. +* Remove debug code like assertions and logs to make the code faster to + run it for production. * `Tail-call Optimization `_ * Add profiling code * `Lazy evaluation `_: @@ -116,12 +116,13 @@ * Declare constants: see `@asconstants of codetransformer `_ * Domain Specific Language (DSL) like SQL queries. The - Python language itself doesn't need to be modified. Previous attempts to - implement DSL for SQL like `PEP 335 - Overloadable Boolean Operators - `_ was rejected. + Python language itself doesn't need to be modified. Previous attempts + to implement DSL for SQL like `PEP 335 - Overloadable Boolean + Operators `_ was rejected. * Pattern Matching of functional languages * String Interpolation, but `PEP 498 -- Literal String Interpolation - `_ was merged into Python 3.6. + `_ was merged into Python + 3.6. `MacroPy `_ has a long list of examples and use cases. @@ -276,7 +277,8 @@ * *code*: the bytecode (``bytes``) * *consts*: a sequence of constants * *names*: tuple of variable names -* *lnotab*: table mapping instruction offsets to line numbers (``bytes``) +* *lnotab*: table mapping instruction offsets to line numbers + (``bytes``) The code transformer is run after the compilation to bytecode @@ -368,8 +370,9 @@ def transformers_tag(): transformers = sys.get_code_transformers() if not transformers: - return 'opt' - return '-'.join(transformer.name for transformer in transformers) + return 'noopt' + return '-'.join(transformer.name + for transformer in transformers) def use_py(): return (transformers_tag() == sys.implementation.optim_tag) @@ -456,7 +459,6 @@ import sys - class BytecodeTransformer: name = "knights_who_say_ni" @@ -465,11 +467,10 @@ for const in consts] return (code, consts, names, lnotab) - - # replace existing code transformers with our bytecode transformer + # replace existing code transformers with the new bytecode transformer sys.set_code_transformers([BytecodeTransformer()]) - # execute code which will be transformed by ast_transformer() + # execute code which will be transformed by code_transformer() exec("print('Hello World!')") Output:: @@ -486,13 +487,11 @@ import ast import sys - class KnightsWhoSayNi(ast.NodeTransformer): def visit_Str(self, node): node.s = 'Ni! Ni! Ni!' return node - class ASTTransformer: name = "knights_who_say_ni" @@ -503,7 +502,6 @@ self.transformer.visit(tree) return tree - # replace existing code transformers with the new AST transformer sys.set_code_transformers([ASTTransformer()]) -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Jan 15 12:39:13 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 17:39:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MTE0?= =?utf-8?q?=3A_Remove_a_reference_to_=27Numerical_Recipes=27=2E?= Message-ID: <20160115173911.41332.88245@psf.io> https://hg.python.org/cpython/rev/76eb752e5447 changeset: 99903:76eb752e5447 branch: 3.5 parent: 99900:bd9d5bc8bc95 user: Brett Cannon date: Fri Jan 15 09:38:24 2016 -0800 summary: Issue #26114: Remove a reference to 'Numerical Recipes'. While no copyright violation occurred, the license which 'Numerical Recipes' operates under is not amenable to Python, so to prevent confusion it's easier to simply remove its mention. files: Modules/mathmodule.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -400,9 +400,8 @@ Implementations of the error function erf(x) and the complementary error function erfc(x). - Method: following 'Numerical Recipes' by Flannery, Press et. al. (2nd ed., - Cambridge University Press), we use a series approximation for erf for - small x, and a continued fraction approximation for erfc(x) for larger x; + Method: we use a series approximation for erf for small x, and a continued + fraction approximation for erfc(x) for larger x; combined with the relations erf(-x) = -erf(x) and erfc(x) = 1.0 - erf(x), this gives us erf(x) and erfc(x) for all x. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 12:39:13 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 17:39:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_issue_=2326114?= Message-ID: <20160115173911.96366.59003@psf.io> https://hg.python.org/cpython/rev/8ad701463cd7 changeset: 99904:8ad701463cd7 parent: 99901:32a4e7b337c9 parent: 99903:76eb752e5447 user: Brett Cannon date: Fri Jan 15 09:39:06 2016 -0800 summary: Merge for issue #26114 files: Modules/mathmodule.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -400,9 +400,8 @@ Implementations of the error function erf(x) and the complementary error function erfc(x). - Method: following 'Numerical Recipes' by Flannery, Press et. al. (2nd ed., - Cambridge University Press), we use a series approximation for erf for - small x, and a continued fraction approximation for erfc(x) for larger x; + Method: we use a series approximation for erf for small x, and a continued + fraction approximation for erfc(x) for larger x; combined with the relations erf(-x) = -erf(x) and erfc(x) = 1.0 - erf(x), this gives us erf(x) and erfc(x) for all x. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 12:41:56 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 17:41:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MTE0?= =?utf-8?q?=3A_Remove_mention_of_=27Numerical_Recipes=27=2E?= Message-ID: <20160115174155.25904.95217@psf.io> https://hg.python.org/cpython/rev/faac8f09020d changeset: 99905:faac8f09020d branch: 2.7 parent: 99902:687f0d4bc188 user: Brett Cannon date: Fri Jan 15 09:41:49 2016 -0800 summary: Issue #26114: Remove mention of 'Numerical Recipes'. files: Modules/mathmodule.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -379,9 +379,8 @@ Implementations of the error function erf(x) and the complementary error function erfc(x). - Method: following 'Numerical Recipes' by Flannery, Press et. al. (2nd ed., - Cambridge University Press), we use a series approximation for erf for - small x, and a continued fraction approximation for erfc(x) for larger x; + Method: we use a series approximation for erf for small x, and a continued + fraction approximation for erfc(x) for larger x; combined with the relations erf(-x) = -erf(x) and erfc(x) = 1.0 - erf(x), this gives us erf(x) and erfc(x) for all x. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 12:54:21 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 17:54:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_=27used_with_permission=27_additions?= Message-ID: <20160115175421.5463.22781@psf.io> https://hg.python.org/cpython/rev/92442e434f88 changeset: 99907:92442e434f88 parent: 99904:8ad701463cd7 parent: 99906:ca8419fdb3c2 user: Brett Cannon date: Fri Jan 15 09:54:16 2016 -0800 summary: Merge 'used with permission' additions files: Lib/datetime.py | 1 + Lib/encodings/hp_roman8.py | 2 ++ Lib/unittest/__init__.py | 2 +- 3 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -897,6 +897,7 @@ ISO calendar algorithm taken from http://www.phys.uu.nl/~vgent/calendar/isocalendar.htm + (used with permission) """ year = self._year week1monday = _isoweek1monday(year) diff --git a/Lib/encodings/hp_roman8.py b/Lib/encodings/hp_roman8.py --- a/Lib/encodings/hp_roman8.py +++ b/Lib/encodings/hp_roman8.py @@ -5,6 +5,8 @@ Original source: LaserJet IIP Printer User's Manual HP part no 33471-90901, Hewlet-Packard, June 1989. + (Used with permission) + """#" import codecs diff --git a/Lib/unittest/__init__.py b/Lib/unittest/__init__.py --- a/Lib/unittest/__init__.py +++ b/Lib/unittest/__init__.py @@ -1,6 +1,6 @@ """ Python unit testing framework, based on Erich Gamma's JUnit and Kent Beck's -Smalltalk testing framework. +Smalltalk testing framework (used with permission). This module contains the core framework classes that form the basis of specific test cases and suites (TestCase, TestSuite etc.), and also a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 12:54:21 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 17:54:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQWRkIHNvbWUgInVz?= =?utf-8?q?ed_with_permission=22_mentions_where_external_resources_are?= Message-ID: <20160115175421.14203.78745@psf.io> https://hg.python.org/cpython/rev/ca8419fdb3c2 changeset: 99906:ca8419fdb3c2 branch: 3.5 parent: 99903:76eb752e5447 user: Brett Cannon date: Fri Jan 15 09:53:51 2016 -0800 summary: Add some "used with permission" mentions where external resources are referenced. Permission was validated prior to adding these markings. files: Lib/datetime.py | 1 + Lib/encodings/hp_roman8.py | 2 ++ Lib/unittest/__init__.py | 2 +- 3 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/datetime.py b/Lib/datetime.py --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -896,6 +896,7 @@ ISO calendar algorithm taken from http://www.phys.uu.nl/~vgent/calendar/isocalendar.htm + (used with permission) """ year = self._year week1monday = _isoweek1monday(year) diff --git a/Lib/encodings/hp_roman8.py b/Lib/encodings/hp_roman8.py --- a/Lib/encodings/hp_roman8.py +++ b/Lib/encodings/hp_roman8.py @@ -5,6 +5,8 @@ Original source: LaserJet IIP Printer User's Manual HP part no 33471-90901, Hewlet-Packard, June 1989. + (Used with permission) + """#" import codecs diff --git a/Lib/unittest/__init__.py b/Lib/unittest/__init__.py --- a/Lib/unittest/__init__.py +++ b/Lib/unittest/__init__.py @@ -1,6 +1,6 @@ """ Python unit testing framework, based on Erich Gamma's JUnit and Kent Beck's -Smalltalk testing framework. +Smalltalk testing framework (used with permission). This module contains the core framework classes that form the basis of specific test cases and suites (TestCase, TestSuite etc.), and also a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 14:40:13 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 19:40:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_issue_=2317633?= Message-ID: <20160115194013.25987.715@psf.io> https://hg.python.org/cpython/rev/87f87673af7b changeset: 99909:87f87673af7b parent: 99907:92442e434f88 parent: 99908:07a615a8f9ad user: Brett Cannon date: Fri Jan 15 11:23:46 2016 -0800 summary: Merge for issue #17633 files: Lib/test/test_zipimport.py | 248 +++++++++++++++++++++--- Modules/zipimport.c | 48 +++- 2 files changed, 248 insertions(+), 48 deletions(-) diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -1,6 +1,7 @@ import sys import os import marshal +import importlib import importlib.util import struct import time @@ -48,6 +49,7 @@ TESTMOD = "ziptestmodule" TESTPACK = "ziptestpackage" TESTPACK2 = "ziptestpackage2" +TEMP_DIR = os.path.abspath("junk95142") TEMP_ZIP = os.path.abspath("junk95142.zip") pyc_file = importlib.util.cache_from_source(TESTMOD + '.py') @@ -77,45 +79,64 @@ def setUp(self): # We're reusing the zip archive path, so we must clear the - # cached directory info and linecache + # cached directory info and linecache. linecache.clearcache() zipimport._zip_directory_cache.clear() ImportHooksBaseTestCase.setUp(self) - def doTest(self, expected_ext, files, *modules, **kw): - z = ZipFile(TEMP_ZIP, "w") - try: + def makeTree(self, files, dirName=TEMP_DIR): + # Create a filesystem based set of modules/packages + # defined by files under the directory dirName. + self.addCleanup(support.rmtree, dirName) + + for name, (mtime, data) in files.items(): + path = os.path.join(dirName, name) + if path[-1] == os.sep: + if not os.path.isdir(path): + os.makedirs(path) + else: + dname = os.path.dirname(path) + if not os.path.isdir(dname): + os.makedirs(dname) + with open(path, 'wb') as fp: + fp.write(data) + + def makeZip(self, files, zipName=TEMP_ZIP, **kw): + # Create a zip archive based set of modules/packages + # defined by files in the zip file zipName. If the + # key 'stuff' exists in kw it is prepended to the archive. + self.addCleanup(support.unlink, zipName) + + with ZipFile(zipName, "w") as z: for name, (mtime, data) in files.items(): zinfo = ZipInfo(name, time.localtime(mtime)) zinfo.compress_type = self.compression z.writestr(zinfo, data) - z.close() - stuff = kw.get("stuff", None) - if stuff is not None: - # Prepend 'stuff' to the start of the zipfile - with open(TEMP_ZIP, "rb") as f: - data = f.read() - with open(TEMP_ZIP, "wb") as f: - f.write(stuff) - f.write(data) + stuff = kw.get("stuff", None) + if stuff is not None: + # Prepend 'stuff' to the start of the zipfile + with open(zipName, "rb") as f: + data = f.read() + with open(zipName, "wb") as f: + f.write(stuff) + f.write(data) - sys.path.insert(0, TEMP_ZIP) + def doTest(self, expected_ext, files, *modules, **kw): + self.makeZip(files, **kw) - mod = __import__(".".join(modules), globals(), locals(), - ["__dummy__"]) + sys.path.insert(0, TEMP_ZIP) - call = kw.get('call') - if call is not None: - call(mod) + mod = importlib.import_module(".".join(modules)) - if expected_ext: - file = mod.get_file() - self.assertEqual(file, os.path.join(TEMP_ZIP, + call = kw.get('call') + if call is not None: + call(mod) + + if expected_ext: + file = mod.get_file() + self.assertEqual(file, os.path.join(TEMP_ZIP, *modules) + expected_ext) - finally: - z.close() - os.remove(TEMP_ZIP) def testAFakeZlib(self): # @@ -201,7 +222,9 @@ packdir + TESTMOD + pyc_ext: (NOW, test_pyc)} self.doTest(pyc_ext, files, TESTPACK, TESTMOD) - def testDeepPackage(self): + def testSubPackage(self): + # Test that subpackages function when loaded from zip + # archives. packdir = TESTPACK + os.sep packdir2 = packdir + TESTPACK2 + os.sep files = {packdir + "__init__" + pyc_ext: (NOW, test_pyc), @@ -209,6 +232,167 @@ packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} self.doTest(pyc_ext, files, TESTPACK, TESTPACK2, TESTMOD) + def testSubNamespacePackage(self): + # Test that implicit namespace subpackages function + # when loaded from zip archives. + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + # The first two files are just directory entries (so have no data). + files = {packdir: (NOW, ""), + packdir2: (NOW, ""), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + self.doTest(pyc_ext, files, TESTPACK, TESTPACK2, TESTMOD) + + def testMixedNamespacePackage(self): + # Test implicit namespace packages spread between a + # real filesystem and a zip archive. + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + packdir3 = packdir2 + TESTPACK + '3' + os.sep + files1 = {packdir: (NOW, ""), + packdir + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir3: (NOW, ""), + packdir3 + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + '3' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + files2 = {packdir: (NOW, ""), + packdir + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir2 + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + + zip1 = os.path.abspath("path1.zip") + self.makeZip(files1, zip1) + + zip2 = TEMP_DIR + self.makeTree(files2, zip2) + + # zip2 should override zip1. + sys.path.insert(0, zip1) + sys.path.insert(0, zip2) + + mod = importlib.import_module(TESTPACK) + + # if TESTPACK is functioning as a namespace pkg then + # there should be two entries in the __path__. + # First should be path2 and second path1. + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual(os.path.basename(TEMP_DIR), p1.split(os.sep)[-2]) + self.assertEqual("path1.zip", p2.split(os.sep)[-2]) + + # packdir3 should import as a namespace package. + # Its __path__ is an iterable of 1 element from zip1. + mod = importlib.import_module(packdir3.replace(os.sep, '.')[:-1]) + self.assertEqual(1, len(mod.__path__)) + mpath = list(mod.__path__)[0].split('path1.zip' + os.sep)[1] + self.assertEqual(packdir3[:-1], mpath) + + # TESTPACK/TESTMOD only exists in path1. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD))) + self.assertEqual("path1.zip", mod.__file__.split(os.sep)[-3]) + + # And TESTPACK/(TESTMOD + '2') only exists in path2. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD + '2'))) + self.assertEqual(os.path.basename(TEMP_DIR), + mod.__file__.split(os.sep)[-3]) + + # One level deeper... + subpkg = '.'.join((TESTPACK, TESTPACK2)) + mod = importlib.import_module(subpkg) + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual(os.path.basename(TEMP_DIR), p1.split(os.sep)[-3]) + self.assertEqual("path1.zip", p2.split(os.sep)[-3]) + + # subpkg.TESTMOD exists in both zips should load from zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD))) + self.assertEqual(os.path.basename(TEMP_DIR), + mod.__file__.split(os.sep)[-4]) + + # subpkg.TESTMOD + '2' only exists in zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '2'))) + self.assertEqual(os.path.basename(TEMP_DIR), + mod.__file__.split(os.sep)[-4]) + + # Finally subpkg.TESTMOD + '3' only exists in zip1. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '3'))) + self.assertEqual('path1.zip', mod.__file__.split(os.sep)[-4]) + + def testNamespacePackage(self): + # Test implicit namespace packages spread between multiple zip + # archives. + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + packdir3 = packdir2 + TESTPACK + '3' + os.sep + files1 = {packdir: (NOW, ""), + packdir + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir3: (NOW, ""), + packdir3 + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + '3' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + zip1 = os.path.abspath("path1.zip") + self.makeZip(files1, zip1) + + files2 = {packdir: (NOW, ""), + packdir + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir2 + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + zip2 = os.path.abspath("path2.zip") + self.makeZip(files2, zip2) + + # zip2 should override zip1. + sys.path.insert(0, zip1) + sys.path.insert(0, zip2) + + mod = importlib.import_module(TESTPACK) + + # if TESTPACK is functioning as a namespace pkg then + # there should be two entries in the __path__. + # First should be path2 and second path1. + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual("path2.zip", p1.split(os.sep)[-2]) + self.assertEqual("path1.zip", p2.split(os.sep)[-2]) + + # packdir3 should import as a namespace package. + # Tts __path__ is an iterable of 1 element from zip1. + mod = importlib.import_module(packdir3.replace(os.sep, '.')[:-1]) + self.assertEqual(1, len(mod.__path__)) + mpath = list(mod.__path__)[0].split('path1.zip' + os.sep)[1] + self.assertEqual(packdir3[:-1], mpath) + + # TESTPACK/TESTMOD only exists in path1. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD))) + self.assertEqual("path1.zip", mod.__file__.split(os.sep)[-3]) + + # And TESTPACK/(TESTMOD + '2') only exists in path2. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD + '2'))) + self.assertEqual("path2.zip", mod.__file__.split(os.sep)[-3]) + + # One level deeper... + subpkg = '.'.join((TESTPACK, TESTPACK2)) + mod = importlib.import_module(subpkg) + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual("path2.zip", p1.split(os.sep)[-3]) + self.assertEqual("path1.zip", p2.split(os.sep)[-3]) + + # subpkg.TESTMOD exists in both zips should load from zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD))) + self.assertEqual('path2.zip', mod.__file__.split(os.sep)[-4]) + + # subpkg.TESTMOD + '2' only exists in zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '2'))) + self.assertEqual('path2.zip', mod.__file__.split(os.sep)[-4]) + + # Finally subpkg.TESTMOD + '3' only exists in zip1. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '3'))) + self.assertEqual('path1.zip', mod.__file__.split(os.sep)[-4]) + def testZipImporterMethods(self): packdir = TESTPACK + os.sep packdir2 = packdir + TESTPACK2 + os.sep @@ -240,7 +424,7 @@ mod = zi.load_module(TESTPACK) self.assertEqual(zi.get_filename(TESTPACK), mod.__file__) - existing_pack_path = __import__(TESTPACK).__path__[0] + existing_pack_path = importlib.import_module(TESTPACK).__path__[0] expected_path_path = os.path.join(TEMP_ZIP, TESTPACK) self.assertEqual(existing_pack_path, expected_path_path) @@ -250,8 +434,8 @@ mod_path = packdir2 + TESTMOD mod_name = module_path_to_dotted_name(mod_path) - __import__(mod_name) - mod = sys.modules[mod_name] + mod = importlib.import_module(mod_name) + self.assertTrue(mod_name in sys.modules) self.assertEqual(zi.get_source(TESTPACK), None) self.assertEqual(zi.get_source(mod_path), None) self.assertEqual(zi.get_filename(mod_path), mod.__file__) @@ -308,13 +492,13 @@ mod_path = TESTPACK2 + os.sep + TESTMOD mod_name = module_path_to_dotted_name(mod_path) - __import__(mod_name) - mod = sys.modules[mod_name] + mod = importlib.import_module(mod_name) + self.assertTrue(mod_name in sys.modules) self.assertEqual(zi.get_source(TESTPACK2), None) self.assertEqual(zi.get_source(mod_path), None) self.assertEqual(zi.get_filename(mod_path), mod.__file__) # To pass in the module name instead of the path, we must use the - # right importer + # right importer. loader = mod.__loader__ self.assertEqual(loader.get_source(mod_name), None) self.assertEqual(loader.get_filename(mod_name), mod.__file__) diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -324,17 +324,14 @@ } typedef enum { - FL_ERROR, - FL_NOT_FOUND, - FL_MODULE_FOUND, - FL_NS_FOUND + FL_ERROR = -1, /* error */ + FL_NOT_FOUND, /* no loader or namespace portions found */ + FL_MODULE_FOUND, /* module/package found */ + FL_NS_FOUND /* namespace portion found: */ + /* *namespace_portion will point to the name */ } find_loader_result; -/* The guts of "find_loader" and "find_module". Return values: - -1: error - 0: no loader or namespace portions found - 1: module/package found - 2: namespace portion found: *namespace_portion will point to the name +/* The guts of "find_loader" and "find_module". */ static find_loader_result find_loader(ZipImporter *self, PyObject *fullname, PyObject **namespace_portion) @@ -349,21 +346,34 @@ if (mi == MI_NOT_FOUND) { /* Not a module or regular package. See if this is a directory, and therefore possibly a portion of a namespace package. */ - int is_dir = check_is_directory(self, self->prefix, fullname); + find_loader_result result = FL_NOT_FOUND; + PyObject *subname; + int is_dir; + + /* We're only interested in the last path component of fullname; + earlier components are recorded in self->prefix. */ + subname = get_subname(fullname); + if (subname == NULL) { + return FL_ERROR; + } + + is_dir = check_is_directory(self, self->prefix, subname); if (is_dir < 0) - return -1; - if (is_dir) { + result = FL_ERROR; + else if (is_dir) { /* This is possibly a portion of a namespace package. Return the string representing its path, without a trailing separator. */ *namespace_portion = PyUnicode_FromFormat("%U%c%U%U", self->archive, SEP, - self->prefix, fullname); + self->prefix, subname); if (*namespace_portion == NULL) - return FL_ERROR; - return FL_NS_FOUND; + result = FL_ERROR; + else + result = FL_NS_FOUND; } - return FL_NOT_FOUND; + Py_DECREF(subname); + return result; } /* This is a module or package. */ return FL_MODULE_FOUND; @@ -397,6 +407,9 @@ case FL_MODULE_FOUND: result = (PyObject *)self; break; + default: + PyErr_BadInternalCall(); + return NULL; } Py_INCREF(result); return result; @@ -433,6 +446,9 @@ result = Py_BuildValue("O[O]", Py_None, namespace_portion); Py_DECREF(namespace_portion); return result; + default: + PyErr_BadInternalCall(); + return NULL; } return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 14:40:13 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 19:40:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE3NjMz?= =?utf-8?q?=3A_Improve_support_for_namespace_packages_with_zipimport=2E?= Message-ID: <20160115194013.5451.97662@psf.io> https://hg.python.org/cpython/rev/07a615a8f9ad changeset: 99908:07a615a8f9ad branch: 3.5 parent: 99906:ca8419fdb3c2 user: Brett Cannon date: Fri Jan 15 11:22:19 2016 -0800 summary: Issue #17633: Improve support for namespace packages with zipimport. Previously zipimport mistakenly limited namespace support to only the top-level of the zipfile when it should have supported an arbitrary depth. Thanks to Phil Connel for the bug report and initial patch and Mike Romberg for the final patch. files: Lib/test/test_zipimport.py | 248 +++++++++++++++++++++--- Misc/NEWS | 2 + Modules/zipimport.c | 48 +++- 3 files changed, 250 insertions(+), 48 deletions(-) diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -1,6 +1,7 @@ import sys import os import marshal +import importlib import importlib.util import struct import time @@ -48,6 +49,7 @@ TESTMOD = "ziptestmodule" TESTPACK = "ziptestpackage" TESTPACK2 = "ziptestpackage2" +TEMP_DIR = os.path.abspath("junk95142") TEMP_ZIP = os.path.abspath("junk95142.zip") pyc_file = importlib.util.cache_from_source(TESTMOD + '.py') @@ -77,45 +79,64 @@ def setUp(self): # We're reusing the zip archive path, so we must clear the - # cached directory info and linecache + # cached directory info and linecache. linecache.clearcache() zipimport._zip_directory_cache.clear() ImportHooksBaseTestCase.setUp(self) - def doTest(self, expected_ext, files, *modules, **kw): - z = ZipFile(TEMP_ZIP, "w") - try: + def makeTree(self, files, dirName=TEMP_DIR): + # Create a filesystem based set of modules/packages + # defined by files under the directory dirName. + self.addCleanup(support.rmtree, dirName) + + for name, (mtime, data) in files.items(): + path = os.path.join(dirName, name) + if path[-1] == os.sep: + if not os.path.isdir(path): + os.makedirs(path) + else: + dname = os.path.dirname(path) + if not os.path.isdir(dname): + os.makedirs(dname) + with open(path, 'wb') as fp: + fp.write(data) + + def makeZip(self, files, zipName=TEMP_ZIP, **kw): + # Create a zip archive based set of modules/packages + # defined by files in the zip file zipName. If the + # key 'stuff' exists in kw it is prepended to the archive. + self.addCleanup(support.unlink, zipName) + + with ZipFile(zipName, "w") as z: for name, (mtime, data) in files.items(): zinfo = ZipInfo(name, time.localtime(mtime)) zinfo.compress_type = self.compression z.writestr(zinfo, data) - z.close() - stuff = kw.get("stuff", None) - if stuff is not None: - # Prepend 'stuff' to the start of the zipfile - with open(TEMP_ZIP, "rb") as f: - data = f.read() - with open(TEMP_ZIP, "wb") as f: - f.write(stuff) - f.write(data) + stuff = kw.get("stuff", None) + if stuff is not None: + # Prepend 'stuff' to the start of the zipfile + with open(zipName, "rb") as f: + data = f.read() + with open(zipName, "wb") as f: + f.write(stuff) + f.write(data) - sys.path.insert(0, TEMP_ZIP) + def doTest(self, expected_ext, files, *modules, **kw): + self.makeZip(files, **kw) - mod = __import__(".".join(modules), globals(), locals(), - ["__dummy__"]) + sys.path.insert(0, TEMP_ZIP) - call = kw.get('call') - if call is not None: - call(mod) + mod = importlib.import_module(".".join(modules)) - if expected_ext: - file = mod.get_file() - self.assertEqual(file, os.path.join(TEMP_ZIP, + call = kw.get('call') + if call is not None: + call(mod) + + if expected_ext: + file = mod.get_file() + self.assertEqual(file, os.path.join(TEMP_ZIP, *modules) + expected_ext) - finally: - z.close() - os.remove(TEMP_ZIP) def testAFakeZlib(self): # @@ -201,7 +222,9 @@ packdir + TESTMOD + pyc_ext: (NOW, test_pyc)} self.doTest(pyc_ext, files, TESTPACK, TESTMOD) - def testDeepPackage(self): + def testSubPackage(self): + # Test that subpackages function when loaded from zip + # archives. packdir = TESTPACK + os.sep packdir2 = packdir + TESTPACK2 + os.sep files = {packdir + "__init__" + pyc_ext: (NOW, test_pyc), @@ -209,6 +232,167 @@ packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} self.doTest(pyc_ext, files, TESTPACK, TESTPACK2, TESTMOD) + def testSubNamespacePackage(self): + # Test that implicit namespace subpackages function + # when loaded from zip archives. + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + # The first two files are just directory entries (so have no data). + files = {packdir: (NOW, ""), + packdir2: (NOW, ""), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + self.doTest(pyc_ext, files, TESTPACK, TESTPACK2, TESTMOD) + + def testMixedNamespacePackage(self): + # Test implicit namespace packages spread between a + # real filesystem and a zip archive. + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + packdir3 = packdir2 + TESTPACK + '3' + os.sep + files1 = {packdir: (NOW, ""), + packdir + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir3: (NOW, ""), + packdir3 + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + '3' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + files2 = {packdir: (NOW, ""), + packdir + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir2 + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + + zip1 = os.path.abspath("path1.zip") + self.makeZip(files1, zip1) + + zip2 = TEMP_DIR + self.makeTree(files2, zip2) + + # zip2 should override zip1. + sys.path.insert(0, zip1) + sys.path.insert(0, zip2) + + mod = importlib.import_module(TESTPACK) + + # if TESTPACK is functioning as a namespace pkg then + # there should be two entries in the __path__. + # First should be path2 and second path1. + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual(os.path.basename(TEMP_DIR), p1.split(os.sep)[-2]) + self.assertEqual("path1.zip", p2.split(os.sep)[-2]) + + # packdir3 should import as a namespace package. + # Its __path__ is an iterable of 1 element from zip1. + mod = importlib.import_module(packdir3.replace(os.sep, '.')[:-1]) + self.assertEqual(1, len(mod.__path__)) + mpath = list(mod.__path__)[0].split('path1.zip' + os.sep)[1] + self.assertEqual(packdir3[:-1], mpath) + + # TESTPACK/TESTMOD only exists in path1. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD))) + self.assertEqual("path1.zip", mod.__file__.split(os.sep)[-3]) + + # And TESTPACK/(TESTMOD + '2') only exists in path2. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD + '2'))) + self.assertEqual(os.path.basename(TEMP_DIR), + mod.__file__.split(os.sep)[-3]) + + # One level deeper... + subpkg = '.'.join((TESTPACK, TESTPACK2)) + mod = importlib.import_module(subpkg) + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual(os.path.basename(TEMP_DIR), p1.split(os.sep)[-3]) + self.assertEqual("path1.zip", p2.split(os.sep)[-3]) + + # subpkg.TESTMOD exists in both zips should load from zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD))) + self.assertEqual(os.path.basename(TEMP_DIR), + mod.__file__.split(os.sep)[-4]) + + # subpkg.TESTMOD + '2' only exists in zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '2'))) + self.assertEqual(os.path.basename(TEMP_DIR), + mod.__file__.split(os.sep)[-4]) + + # Finally subpkg.TESTMOD + '3' only exists in zip1. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '3'))) + self.assertEqual('path1.zip', mod.__file__.split(os.sep)[-4]) + + def testNamespacePackage(self): + # Test implicit namespace packages spread between multiple zip + # archives. + packdir = TESTPACK + os.sep + packdir2 = packdir + TESTPACK2 + os.sep + packdir3 = packdir2 + TESTPACK + '3' + os.sep + files1 = {packdir: (NOW, ""), + packdir + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir3: (NOW, ""), + packdir3 + TESTMOD + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + '3' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + zip1 = os.path.abspath("path1.zip") + self.makeZip(files1, zip1) + + files2 = {packdir: (NOW, ""), + packdir + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2: (NOW, ""), + packdir2 + TESTMOD + '2' + pyc_ext: (NOW, test_pyc), + packdir2 + TESTMOD + pyc_ext: (NOW, test_pyc)} + zip2 = os.path.abspath("path2.zip") + self.makeZip(files2, zip2) + + # zip2 should override zip1. + sys.path.insert(0, zip1) + sys.path.insert(0, zip2) + + mod = importlib.import_module(TESTPACK) + + # if TESTPACK is functioning as a namespace pkg then + # there should be two entries in the __path__. + # First should be path2 and second path1. + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual("path2.zip", p1.split(os.sep)[-2]) + self.assertEqual("path1.zip", p2.split(os.sep)[-2]) + + # packdir3 should import as a namespace package. + # Tts __path__ is an iterable of 1 element from zip1. + mod = importlib.import_module(packdir3.replace(os.sep, '.')[:-1]) + self.assertEqual(1, len(mod.__path__)) + mpath = list(mod.__path__)[0].split('path1.zip' + os.sep)[1] + self.assertEqual(packdir3[:-1], mpath) + + # TESTPACK/TESTMOD only exists in path1. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD))) + self.assertEqual("path1.zip", mod.__file__.split(os.sep)[-3]) + + # And TESTPACK/(TESTMOD + '2') only exists in path2. + mod = importlib.import_module('.'.join((TESTPACK, TESTMOD + '2'))) + self.assertEqual("path2.zip", mod.__file__.split(os.sep)[-3]) + + # One level deeper... + subpkg = '.'.join((TESTPACK, TESTPACK2)) + mod = importlib.import_module(subpkg) + self.assertEqual(2, len(mod.__path__)) + p1, p2 = mod.__path__ + self.assertEqual("path2.zip", p1.split(os.sep)[-3]) + self.assertEqual("path1.zip", p2.split(os.sep)[-3]) + + # subpkg.TESTMOD exists in both zips should load from zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD))) + self.assertEqual('path2.zip', mod.__file__.split(os.sep)[-4]) + + # subpkg.TESTMOD + '2' only exists in zip2. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '2'))) + self.assertEqual('path2.zip', mod.__file__.split(os.sep)[-4]) + + # Finally subpkg.TESTMOD + '3' only exists in zip1. + mod = importlib.import_module('.'.join((subpkg, TESTMOD + '3'))) + self.assertEqual('path1.zip', mod.__file__.split(os.sep)[-4]) + def testZipImporterMethods(self): packdir = TESTPACK + os.sep packdir2 = packdir + TESTPACK2 + os.sep @@ -231,7 +415,7 @@ mod = zi.load_module(TESTPACK) self.assertEqual(zi.get_filename(TESTPACK), mod.__file__) - existing_pack_path = __import__(TESTPACK).__path__[0] + existing_pack_path = importlib.import_module(TESTPACK).__path__[0] expected_path_path = os.path.join(TEMP_ZIP, TESTPACK) self.assertEqual(existing_pack_path, expected_path_path) @@ -241,8 +425,8 @@ mod_path = packdir2 + TESTMOD mod_name = module_path_to_dotted_name(mod_path) - __import__(mod_name) - mod = sys.modules[mod_name] + mod = importlib.import_module(mod_name) + self.assertTrue(mod_name in sys.modules) self.assertEqual(zi.get_source(TESTPACK), None) self.assertEqual(zi.get_source(mod_path), None) self.assertEqual(zi.get_filename(mod_path), mod.__file__) @@ -289,13 +473,13 @@ mod_path = TESTPACK2 + os.sep + TESTMOD mod_name = module_path_to_dotted_name(mod_path) - __import__(mod_name) - mod = sys.modules[mod_name] + mod = importlib.import_module(mod_name) + self.assertTrue(mod_name in sys.modules) self.assertEqual(zi.get_source(TESTPACK2), None) self.assertEqual(zi.get_source(mod_path), None) self.assertEqual(zi.get_filename(mod_path), mod.__file__) # To pass in the module name instead of the path, we must use the - # right importer + # right importer. loader = mod.__loader__ self.assertEqual(loader.get_source(mod_name), None) self.assertEqual(loader.get_filename(mod_name), mod.__file__) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,8 @@ Library ------- +- Issue #17633: Improve zipimport's support for namespace packages. + - Issue #24705: Fix sysconfig._parse_makefile not expanding ${} vars appearing before $() vars. diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -324,17 +324,14 @@ } typedef enum { - FL_ERROR, - FL_NOT_FOUND, - FL_MODULE_FOUND, - FL_NS_FOUND + FL_ERROR = -1, /* error */ + FL_NOT_FOUND, /* no loader or namespace portions found */ + FL_MODULE_FOUND, /* module/package found */ + FL_NS_FOUND /* namespace portion found: */ + /* *namespace_portion will point to the name */ } find_loader_result; -/* The guts of "find_loader" and "find_module". Return values: - -1: error - 0: no loader or namespace portions found - 1: module/package found - 2: namespace portion found: *namespace_portion will point to the name +/* The guts of "find_loader" and "find_module". */ static find_loader_result find_loader(ZipImporter *self, PyObject *fullname, PyObject **namespace_portion) @@ -349,21 +346,34 @@ if (mi == MI_NOT_FOUND) { /* Not a module or regular package. See if this is a directory, and therefore possibly a portion of a namespace package. */ - int is_dir = check_is_directory(self, self->prefix, fullname); + find_loader_result result = FL_NOT_FOUND; + PyObject *subname; + int is_dir; + + /* We're only interested in the last path component of fullname; + earlier components are recorded in self->prefix. */ + subname = get_subname(fullname); + if (subname == NULL) { + return FL_ERROR; + } + + is_dir = check_is_directory(self, self->prefix, subname); if (is_dir < 0) - return -1; - if (is_dir) { + result = FL_ERROR; + else if (is_dir) { /* This is possibly a portion of a namespace package. Return the string representing its path, without a trailing separator. */ *namespace_portion = PyUnicode_FromFormat("%U%c%U%U", self->archive, SEP, - self->prefix, fullname); + self->prefix, subname); if (*namespace_portion == NULL) - return FL_ERROR; - return FL_NS_FOUND; + result = FL_ERROR; + else + result = FL_NS_FOUND; } - return FL_NOT_FOUND; + Py_DECREF(subname); + return result; } /* This is a module or package. */ return FL_MODULE_FOUND; @@ -397,6 +407,9 @@ case FL_MODULE_FOUND: result = (PyObject *)self; break; + default: + PyErr_BadInternalCall(); + return NULL; } Py_INCREF(result); return result; @@ -433,6 +446,9 @@ result = Py_BuildValue("O[O]", Py_None, namespace_portion); Py_DECREF(namespace_portion); return result; + default: + PyErr_BadInternalCall(); + return NULL; } return result; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 16:33:10 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 15 Jan 2016 21:33:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325791=3A_Raise_an?= =?utf-8?q?_ImportWarning_when_=5F=5Fspec=5F=5F_or_=5F=5Fpackage=5F=5F_are?= Message-ID: <20160115213308.111030.10805@psf.io> https://hg.python.org/cpython/rev/6908b2c9a404 changeset: 99910:6908b2c9a404 user: Brett Cannon date: Fri Jan 15 13:33:03 2016 -0800 summary: Issue #25791: Raise an ImportWarning when __spec__ or __package__ are not defined for a relative import. This is the start of work to try and clean up import semantics to rely more on a module's spec than on the myriad attributes that get set on a module. Thanks to Rose Ames for the patch. files: Doc/reference/import.rst | 3 +- Doc/whatsnew/3.6.rst | 11 +- Lib/importlib/_bootstrap.py | 6 + Lib/test/test_importlib/import_/test___package__.py | 40 +- Misc/NEWS | 3 + Python/import.c | 66 +- Python/importlib.h | 559 +++++---- 7 files changed, 377 insertions(+), 311 deletions(-) diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -554,7 +554,8 @@ details. This attribute is used instead of ``__name__`` to calculate explicit - relative imports for main modules, as defined in :pep:`366`. + relative imports for main modules -- as defined in :pep:`366` -- + when ``__spec__`` is not defined. .. attribute:: __spec__ diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -209,7 +209,12 @@ * The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``. This prevents confusion as to what Python interpreter ``pyvenv`` is connected to and thus what Python interpreter will be used by the virtual - environment. See :issue:`25154`. + environment. (Contributed by Brett Cannon in :issue:`25154`.) + +* When performing a relative import, falling back on ``__name__`` and + ``__path__`` from the calling module when ``__spec__`` or + ``__package__`` are not defined now raises an :exc:`ImportWarning`. + (Contributed by Rose Ames in :issue:`25791`.) Removed @@ -251,6 +256,10 @@ :mod:`wave`. This means they will export new symbols when ``import *`` is used. See :issue:`23883`. +* When performing a relative import, ``__spec__.parent`` is used + is ``__spec__`` is defined instead of ``__package__``. + (Contributed by Rose Ames in :issue:`25791`.) + Changes in the C API -------------------- diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -1032,8 +1032,14 @@ to represent that its proper value is unknown. """ + spec = globals.get('__spec__') + if spec is not None: + return spec.parent package = globals.get('__package__') if package is None: + _warnings.warn("can't resolve package from __spec__ or __package__, " + "falling back on __name__ and __path__", + ImportWarning, stacklevel=3) package = globals['__name__'] if '__path__' not in globals: package = package.rpartition('.')[0] diff --git a/Lib/test/test_importlib/import_/test___package__.py b/Lib/test/test_importlib/import_/test___package__.py --- a/Lib/test/test_importlib/import_/test___package__.py +++ b/Lib/test/test_importlib/import_/test___package__.py @@ -33,31 +33,39 @@ """ - def test_using___package__(self): - # [__package__] + def import_module(self, globals_): with self.mock_modules('pkg.__init__', 'pkg.fake') as importer: with util.import_state(meta_path=[importer]): self.__import__('pkg.fake') module = self.__import__('', - globals={'__package__': 'pkg.fake'}, + globals=globals_, fromlist=['attr'], level=2) + return module + + def test_using___package__(self): + # [__package__] + module = self.import_module({'__package__': 'pkg.fake'}) self.assertEqual(module.__name__, 'pkg') - def test_using___name__(self, package_as_None=False): + def test_using___name__(self): # [__name__] - globals_ = {'__name__': 'pkg.fake', '__path__': []} - if package_as_None: - globals_['__package__'] = None - with self.mock_modules('pkg.__init__', 'pkg.fake') as importer: - with util.import_state(meta_path=[importer]): - self.__import__('pkg.fake') - module = self.__import__('', globals= globals_, - fromlist=['attr'], level=2) - self.assertEqual(module.__name__, 'pkg') + module = self.import_module({'__name__': 'pkg.fake', '__path__': []}) + self.assertEqual(module.__name__, 'pkg') + + def test_warn_when_using___name__(self): + with self.assertWarns(ImportWarning): + self.import_module({'__name__': 'pkg.fake', '__path__': []}) def test_None_as___package__(self): # [None] - self.test_using___name__(package_as_None=True) + module = self.import_module({ + '__name__': 'pkg.fake', '__path__': [], '__package__': None }) + self.assertEqual(module.__name__, 'pkg') + + def test_prefers___spec__(self): + globals = {'__spec__': FakeSpec()} + with self.assertRaises(SystemError): + self.__import__('', globals, {}, ['relimport'], 1) def test_bad__package__(self): globals = {'__package__': ''} @@ -70,6 +78,10 @@ self.__import__('', globals, {}, ['relimport'], 1) +class FakeSpec: + parent = '' + + class Using__package__PEP302(Using__package__): mock_modules = util.mock_modules diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25791: Trying to resolve a relative import without __spec__ or + __package__ defined now raises an ImportWarning + - Issue #25961: Disallowed null characters in the type name. - Issue #25973: Fix segfault when an invalid nonlocal statement binds a name diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1359,6 +1359,7 @@ PyObject *final_mod = NULL; PyObject *mod = NULL; PyObject *package = NULL; + PyObject *spec = NULL; PyObject *globals = NULL; PyObject *fromlist = NULL; PyInterpreterState *interp = PyThreadState_GET()->interp; @@ -1414,39 +1415,60 @@ goto error; } else if (level > 0) { - package = _PyDict_GetItemId(globals, &PyId___package__); + spec = _PyDict_GetItemId(globals, &PyId___spec__); + if (spec != NULL) { + package = PyObject_GetAttrString(spec, "parent"); + } if (package != NULL && package != Py_None) { - Py_INCREF(package); if (!PyUnicode_Check(package)) { - PyErr_SetString(PyExc_TypeError, "package must be a string"); + PyErr_SetString(PyExc_TypeError, + "__spec__.parent must be a string"); goto error; } } else { - package = _PyDict_GetItemId(globals, &PyId___name__); - if (package == NULL) { - PyErr_SetString(PyExc_KeyError, "'__name__' not in globals"); - goto error; - } - else if (!PyUnicode_Check(package)) { - PyErr_SetString(PyExc_TypeError, "__name__ must be a string"); - } - Py_INCREF(package); - - if (_PyDict_GetItemId(globals, &PyId___path__) == NULL) { - PyObject *partition = NULL; - PyObject *borrowed_dot = _PyUnicode_FromId(&single_dot); - if (borrowed_dot == NULL) { + package = _PyDict_GetItemId(globals, &PyId___package__); + if (package != NULL && package != Py_None) { + Py_INCREF(package); + if (!PyUnicode_Check(package)) { + PyErr_SetString(PyExc_TypeError, "package must be a string"); goto error; } - partition = PyUnicode_RPartition(package, borrowed_dot); - Py_DECREF(package); - if (partition == NULL) { + } + else { + if (PyErr_WarnEx(PyExc_ImportWarning, + "can't resolve package from __spec__ or __package__, " + "falling back on __name__ and __path__", 1) < 0) { goto error; } - package = PyTuple_GET_ITEM(partition, 0); + + package = _PyDict_GetItemId(globals, &PyId___name__); + if (package == NULL) { + PyErr_SetString(PyExc_KeyError, "'__name__' not in globals"); + goto error; + } + Py_INCREF(package); - Py_DECREF(partition); + if (!PyUnicode_Check(package)) { + PyErr_SetString(PyExc_TypeError, "__name__ must be a string"); + goto error; + } + + if (_PyDict_GetItemId(globals, &PyId___path__) == NULL) { + PyObject *partition = NULL; + PyObject *borrowed_dot = _PyUnicode_FromId(&single_dot); + if (borrowed_dot == NULL) { + goto error; + } + partition = PyUnicode_RPartition(package, borrowed_dot); + Py_DECREF(package); + if (partition == NULL) { + goto error; + } + package = PyTuple_GET_ITEM(partition, 0); + Py_INCREF(package); + Py_DECREF(partition); + } } } diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 17:20:03 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 15 Jan 2016 22:20:03 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_511=3A_reorganize?= Message-ID: <20160115222003.25979.43116@psf.io> https://hg.python.org/peps/rev/adc60967a6c5 changeset: 6192:adc60967a6c5 user: Victor Stinner date: Fri Jan 15 23:19:59 2016 +0100 summary: PEP 511: reorganize files: pep-0511.txt | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/pep-0511.txt b/pep-0511.txt --- a/pep-0511.txt +++ b/pep-0511.txt @@ -310,12 +310,12 @@ In short, add: -* ``-o OPTIM_TAG`` command line option -* ``ast.Constant`` -* ``ast.PyCF_TRANSFORMED_AST`` -* ``sys.get_code_transformers()`` -* ``sys.implementation.optim_tag`` -* ``sys.set_code_transformers(transformers)`` +* -o OPTIM_TAG command line option +* sys.implementation.optim_tag +* sys.get_code_transformers() +* sys.set_code_transformers(transformers) +* ast.Constant +* ast.PyCF_TRANSFORMED_AST API to get/set code transformers -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Fri Jan 15 18:03:22 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 15 Jan 2016 23:03:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_revert_change_87a9dff5106c?= =?utf-8?q?=3A_pure_Enum_members_again_evaluate_to_True=3B?= Message-ID: <20160115230322.14193.84270@psf.io> https://hg.python.org/cpython/rev/52dc28ee3c88 changeset: 99911:52dc28ee3c88 parent: 99909:87f87673af7b user: Ethan Furman date: Fri Jan 15 15:01:33 2016 -0800 summary: revert change 87a9dff5106c: pure Enum members again evaluate to True; update Finer Points section of docs to cover boolean evaluation; add more tests for pure and mixed boolean evaluation files: Doc/library/enum.rst | 11 ++++++++++- Lib/enum.py | 3 --- Lib/test/test_enum.py | 17 ++++++++++++++++- 3 files changed, 26 insertions(+), 5 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -257,7 +257,7 @@ >>> Color.red < Color.blue Traceback (most recent call last): File "", line 1, in - TypeError: '<' not supported between instances of 'Color' and 'Color' + TypeError: unorderable types: Color() < Color() Equality comparisons are defined though:: @@ -747,6 +747,15 @@ .. versionchanged:: 3.5 +Boolean evaluation: Enum classes that are mixed with non-Enum types (such as +:class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in +type's rules; otherwise, all members evaluate as ``True``. To make your own +Enum's boolean evaluation depend on the member's value add the following to +your class:: + + def __bool__(self): + return bool(self._value_) + The :attr:`__members__` attribute is only available on the class. If you give your :class:`Enum` subclass extra methods, like the `Planet`_ diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -482,9 +482,6 @@ def __str__(self): return "%s.%s" % (self.__class__.__name__, self._name_) - def __bool__(self): - return bool(self._value_) - def __dir__(self): added_behavior = [ m diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -272,11 +272,26 @@ _any_name_ = 9 def test_bool(self): + # plain Enum members are always True class Logic(Enum): true = True false = False self.assertTrue(Logic.true) - self.assertFalse(Logic.false) + self.assertTrue(Logic.false) + # unless overridden + class RealLogic(Enum): + true = True + false = False + def __bool__(self): + return bool(self._value_) + self.assertTrue(RealLogic.true) + self.assertFalse(RealLogic.false) + # mixed Enums depend on mixed-in type + class IntLogic(int, Enum): + true = 1 + false = 0 + self.assertTrue(IntLogic.true) + self.assertFalse(IntLogic.false) def test_contains(self): Season = self.Season -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 18:03:23 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 15 Jan 2016 23:03:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_branch_merge?= Message-ID: <20160115230323.121312.14091@psf.io> https://hg.python.org/cpython/rev/4b1bca0b560f changeset: 99912:4b1bca0b560f parent: 99910:6908b2c9a404 parent: 99911:52dc28ee3c88 user: Ethan Furman date: Fri Jan 15 15:03:12 2016 -0800 summary: branch merge files: Doc/library/enum.rst | 11 ++++++++++- Lib/enum.py | 3 --- Lib/test/test_enum.py | 17 ++++++++++++++++- 3 files changed, 26 insertions(+), 5 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -257,7 +257,7 @@ >>> Color.red < Color.blue Traceback (most recent call last): File "", line 1, in - TypeError: '<' not supported between instances of 'Color' and 'Color' + TypeError: unorderable types: Color() < Color() Equality comparisons are defined though:: @@ -747,6 +747,15 @@ .. versionchanged:: 3.5 +Boolean evaluation: Enum classes that are mixed with non-Enum types (such as +:class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in +type's rules; otherwise, all members evaluate as ``True``. To make your own +Enum's boolean evaluation depend on the member's value add the following to +your class:: + + def __bool__(self): + return bool(self._value_) + The :attr:`__members__` attribute is only available on the class. If you give your :class:`Enum` subclass extra methods, like the `Planet`_ diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -482,9 +482,6 @@ def __str__(self): return "%s.%s" % (self.__class__.__name__, self._name_) - def __bool__(self): - return bool(self._value_) - def __dir__(self): added_behavior = [ m diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -272,11 +272,26 @@ _any_name_ = 9 def test_bool(self): + # plain Enum members are always True class Logic(Enum): true = True false = False self.assertTrue(Logic.true) - self.assertFalse(Logic.false) + self.assertTrue(Logic.false) + # unless overridden + class RealLogic(Enum): + true = True + false = False + def __bool__(self): + return bool(self._value_) + self.assertTrue(RealLogic.true) + self.assertFalse(RealLogic.false) + # mixed Enums depend on mixed-in type + class IntLogic(int, Enum): + true = 1 + false = 0 + self.assertTrue(IntLogic.true) + self.assertFalse(IntLogic.false) def test_contains(self): Season = self.Season -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 20:31:11 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 16 Jan 2016 01:31:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Make_the_error_message_regex_more_lenient_so_that_it_mat?= =?utf-8?q?ches_both?= Message-ID: <20160116013110.18419.10704@psf.io> https://hg.python.org/cpython/rev/0eba22d30fd2 changeset: 99915:0eba22d30fd2 parent: 99912:4b1bca0b560f parent: 99914:fe2ad01aad02 user: Gregory P. Smith date: Fri Jan 15 17:30:42 2016 -0800 summary: Make the error message regex more lenient so that it matches both "certificate verify failed " and "CERTIFICATE_VERIFY_FAILED " as some SSL libraries use different text than OpenSSL. files: Lib/test/test_asyncio/test_events.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -999,7 +999,7 @@ with mock.patch.object(self.loop, 'call_exception_handler'): with test_utils.disable_logger(): with self.assertRaisesRegex(ssl.SSLError, - 'certificate verify failed '): + '(?i)certificate.verify.failed '): self.loop.run_until_complete(f_c) # execute the loop to log the connection error @@ -1033,7 +1033,7 @@ with mock.patch.object(self.loop, 'call_exception_handler'): with test_utils.disable_logger(): with self.assertRaisesRegex(ssl.SSLError, - 'certificate verify failed '): + '(?i)certificate.verify.failed '): self.loop.run_until_complete(f_c) # execute the loop to log the connection error -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 20:31:11 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 16 Jan 2016 01:31:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Make_the_error?= =?utf-8?q?_message_regex_more_lenient_so_that_it_matches_both?= Message-ID: <20160116013110.41316.44201@psf.io> https://hg.python.org/cpython/rev/0364c4a74b96 changeset: 99913:0364c4a74b96 branch: 3.4 parent: 99899:a806ed21bd82 user: Gregory P. Smith date: Fri Jan 15 17:29:45 2016 -0800 summary: Make the error message regex more lenient so that it matches both "certificate verify failed " and "CERTIFICATE_VERIFY_FAILED " as some SSL libraries use different text than OpenSSL. files: Lib/test/test_asyncio/test_events.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -999,7 +999,7 @@ with mock.patch.object(self.loop, 'call_exception_handler'): with test_utils.disable_logger(): with self.assertRaisesRegex(ssl.SSLError, - 'certificate verify failed '): + '(?i)certificate.verify.failed '): self.loop.run_until_complete(f_c) # execute the loop to log the connection error @@ -1033,7 +1033,7 @@ with mock.patch.object(self.loop, 'call_exception_handler'): with test_utils.disable_logger(): with self.assertRaisesRegex(ssl.SSLError, - 'certificate verify failed '): + '(?i)certificate.verify.failed '): self.loop.run_until_complete(f_c) # execute the loop to log the connection error -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 20:31:11 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 16 Jan 2016 01:31:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Make_the_error_message_regex_more_lenient_so_that_it_matches_b?= =?utf-8?q?oth?= Message-ID: <20160116013110.96370.17387@psf.io> https://hg.python.org/cpython/rev/fe2ad01aad02 changeset: 99914:fe2ad01aad02 branch: 3.5 parent: 99908:07a615a8f9ad parent: 99913:0364c4a74b96 user: Gregory P. Smith date: Fri Jan 15 17:30:24 2016 -0800 summary: Make the error message regex more lenient so that it matches both "certificate verify failed " and "CERTIFICATE_VERIFY_FAILED " as some SSL libraries use different text than OpenSSL. files: Lib/test/test_asyncio/test_events.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -999,7 +999,7 @@ with mock.patch.object(self.loop, 'call_exception_handler'): with test_utils.disable_logger(): with self.assertRaisesRegex(ssl.SSLError, - 'certificate verify failed '): + '(?i)certificate.verify.failed '): self.loop.run_until_complete(f_c) # execute the loop to log the connection error @@ -1033,7 +1033,7 @@ with mock.patch.object(self.loop, 'call_exception_handler'): with test_utils.disable_logger(): with self.assertRaisesRegex(ssl.SSLError, - 'certificate verify failed '): + '(?i)certificate.verify.failed '): self.loop.run_until_complete(f_c) # execute the loop to log the connection error -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 23:48:34 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 04:48:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MTI3?= =?utf-8?q?=3A_Fix_links_in_tokenize_documentation?= Message-ID: <20160116044834.14201.35546@psf.io> https://hg.python.org/cpython/rev/8b09d71fbb6b changeset: 99918:8b09d71fbb6b branch: 2.7 parent: 99905:faac8f09020d user: Martin Panter date: Sat Jan 16 04:32:52 2016 +0000 summary: Issue #26127: Fix links in tokenize documentation files: Doc/library/tokenize.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -28,7 +28,7 @@ The :func:`generate_tokens` generator requires one argument, *readline*, which must be a callable object which provides the same interface as the - :meth:`readline` method of built-in file objects (see section + :meth:`~file.readline` method of built-in file objects (see section :ref:`bltin-file-objects`). Each call to the function should return one line of input as a string. Alternately, *readline* may be a callable object that signals completion by raising :exc:`StopIteration`. @@ -47,11 +47,11 @@ .. function:: tokenize(readline[, tokeneater]) - The :func:`tokenize` function accepts two parameters: one representing the input - stream, and one providing an output mechanism for :func:`tokenize`. + The :func:`.tokenize` function accepts two parameters: one representing the input + stream, and one providing an output mechanism for :func:`.tokenize`. The first parameter, *readline*, must be a callable object which provides the - same interface as the :meth:`readline` method of built-in file objects (see + same interface as the :meth:`~file.readline` method of built-in file objects (see section :ref:`bltin-file-objects`). Each call to the function should return one line of input as a string. Alternately, *readline* may be a callable object that signals completion by raising :exc:`StopIteration`. @@ -65,7 +65,7 @@ All constants from the :mod:`token` module are also exported from :mod:`tokenize`, as are two additional token type values that might be passed to -the *tokeneater* function by :func:`tokenize`: +the *tokeneater* function by :func:`.tokenize`: .. data:: COMMENT -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 23:48:34 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 04:48:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MTI3?= =?utf-8?q?=3A_Fix_links_in_tokenize_documentation=3B_patch_by_Silent_Ghos?= =?utf-8?q?t?= Message-ID: <20160116044834.14173.93624@psf.io> https://hg.python.org/cpython/rev/4b4dbc90c6a0 changeset: 99916:4b4dbc90c6a0 branch: 3.5 parent: 99914:fe2ad01aad02 user: Martin Panter date: Sat Jan 16 04:32:52 2016 +0000 summary: Issue #26127: Fix links in tokenize documentation; patch by Silent Ghost files: Doc/library/tokenize.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -27,7 +27,7 @@ .. function:: tokenize(readline) - The :func:`tokenize` generator requires one argument, *readline*, which + The :func:`.tokenize` generator requires one argument, *readline*, which must be a callable object which provides the same interface as the :meth:`io.IOBase.readline` method of file objects. Each call to the function should return one line of input as bytes. @@ -52,7 +52,7 @@ .. versionchanged:: 3.3 Added support for ``exact_type``. - :func:`tokenize` determines the source encoding of the file by looking for a + :func:`.tokenize` determines the source encoding of the file by looking for a UTF-8 BOM or encoding cookie, according to :pep:`263`. @@ -74,7 +74,7 @@ .. data:: ENCODING Token value that indicates the encoding used to decode the source bytes - into text. The first token returned by :func:`tokenize` will always be an + into text. The first token returned by :func:`.tokenize` will always be an ENCODING token. @@ -96,17 +96,17 @@ positions) may change. It returns bytes, encoded using the ENCODING token, which is the first - token sequence output by :func:`tokenize`. + token sequence output by :func:`.tokenize`. -:func:`tokenize` needs to detect the encoding of source files it tokenizes. The +:func:`.tokenize` needs to detect the encoding of source files it tokenizes. The function it uses to do this is available: .. function:: detect_encoding(readline) The :func:`detect_encoding` function is used to detect the encoding that should be used to decode a Python source file. It requires one argument, - readline, in the same way as the :func:`tokenize` generator. + readline, in the same way as the :func:`.tokenize` generator. It will call readline a maximum of twice, and return the encoding used (as a string) and a list of any lines (not decoded from bytes) it has read @@ -120,7 +120,7 @@ If no encoding is specified, then the default of ``'utf-8'`` will be returned. - Use :func:`open` to open Python source files: it uses + Use :func:`.open` to open Python source files: it uses :func:`detect_encoding` to detect the file encoding. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Jan 15 23:48:34 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 04:48:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326127=3A_Merge_tokenize_documentation?= Message-ID: <20160116044834.25963.66126@psf.io> https://hg.python.org/cpython/rev/fe360700d1ae changeset: 99917:fe360700d1ae parent: 99915:0eba22d30fd2 parent: 99916:4b4dbc90c6a0 user: Martin Panter date: Sat Jan 16 04:34:50 2016 +0000 summary: Issue #26127: Merge tokenize documentation files: Doc/library/tokenize.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -27,7 +27,7 @@ .. function:: tokenize(readline) - The :func:`tokenize` generator requires one argument, *readline*, which + The :func:`.tokenize` generator requires one argument, *readline*, which must be a callable object which provides the same interface as the :meth:`io.IOBase.readline` method of file objects. Each call to the function should return one line of input as bytes. @@ -52,7 +52,7 @@ .. versionchanged:: 3.3 Added support for ``exact_type``. - :func:`tokenize` determines the source encoding of the file by looking for a + :func:`.tokenize` determines the source encoding of the file by looking for a UTF-8 BOM or encoding cookie, according to :pep:`263`. @@ -74,7 +74,7 @@ .. data:: ENCODING Token value that indicates the encoding used to decode the source bytes - into text. The first token returned by :func:`tokenize` will always be an + into text. The first token returned by :func:`.tokenize` will always be an ENCODING token. @@ -96,17 +96,17 @@ positions) may change. It returns bytes, encoded using the ENCODING token, which is the first - token sequence output by :func:`tokenize`. + token sequence output by :func:`.tokenize`. -:func:`tokenize` needs to detect the encoding of source files it tokenizes. The +:func:`.tokenize` needs to detect the encoding of source files it tokenizes. The function it uses to do this is available: .. function:: detect_encoding(readline) The :func:`detect_encoding` function is used to detect the encoding that should be used to decode a Python source file. It requires one argument, - readline, in the same way as the :func:`tokenize` generator. + readline, in the same way as the :func:`.tokenize` generator. It will call readline a maximum of twice, and return the encoding used (as a string) and a list of any lines (not decoded from bytes) it has read @@ -120,7 +120,7 @@ If no encoding is specified, then the default of ``'utf-8'`` will be returned. - Use :func:`open` to open Python source files: it uses + Use :func:`.open` to open Python source files: it uses :func:`detect_encoding` to detect the file encoding. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 00:21:39 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 05:21:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE0Nzcx?= =?utf-8?q?=3A_Redirect_GDB=27s_stdin_to_avoid_messing_the_terminal_settin?= =?utf-8?q?gs?= Message-ID: <20160116052139.14189.80300@psf.io> https://hg.python.org/cpython/rev/8b8ac7adbf49 changeset: 99919:8b8ac7adbf49 branch: 2.7 user: Martin Panter date: Sat Jan 16 05:07:35 2016 +0000 summary: Issue #14771: Redirect GDB's stdin to avoid messing the terminal settings Otherwise, GDB seems to affect the terminal's foreground process group, interfering with test_ioctl, which does not expect the foreground process to change during the test. This change also solves the problem of the tests being stopped in the shell if test_gdb is run twice in parallel. files: Lib/test/test_gdb.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -71,6 +71,8 @@ if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) out, err = subprocess.Popen(base_cmd + args, + # Redirect stdin to prevent GDB from messing with terminal settings + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env, ).communicate() return out, err -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 00:21:40 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 05:21:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326127=3A_Merge_test=5Fgdb_fix_from_3=2E5?= Message-ID: <20160116052139.14195.87307@psf.io> https://hg.python.org/cpython/rev/620a37dbc686 changeset: 99921:620a37dbc686 parent: 99917:fe360700d1ae parent: 99920:c87cc05af8e7 user: Martin Panter date: Sat Jan 16 05:20:57 2016 +0000 summary: Issue #26127: Merge test_gdb fix from 3.5 files: Lib/test/test_gdb.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -75,6 +75,9 @@ if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) proc = subprocess.Popen(base_cmd + args, + # Redirect stdin to prevent GDB from messing with + # the terminal settings + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 00:21:39 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 05:21:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE0Nzcx?= =?utf-8?q?=3A_Redirect_GDB=27s_stdin_to_avoid_messing_the_terminal_settin?= =?utf-8?q?gs?= Message-ID: <20160116052139.14185.79258@psf.io> https://hg.python.org/cpython/rev/c87cc05af8e7 changeset: 99920:c87cc05af8e7 branch: 3.5 parent: 99916:4b4dbc90c6a0 user: Martin Panter date: Sat Jan 16 05:18:47 2016 +0000 summary: Issue #14771: Redirect GDB's stdin to avoid messing the terminal settings Otherwise, GDB seems to affect the terminal's foreground process group, interfering with test_ioctl, which does not expect the foreground process to change during the test. This change also solves the problem of the tests being stopped in the shell if test_gdb is run twice in parallel. files: Lib/test/test_gdb.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -75,6 +75,9 @@ if (gdb_major_version, gdb_minor_version) >= (7, 4): base_cmd += ('-iex', 'add-auto-load-safe-path ' + checkout_hook_path) proc = subprocess.Popen(base_cmd + args, + # Redirect stdin to prevent GDB from messing with + # the terminal settings + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 00:46:14 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 16 Jan 2016 05:46:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A__Issue26035_-_?= =?utf-8?q?Correct_the_argument_names_used_in_the_docs_of_the_traceback?= Message-ID: <20160116054614.121298.58339@psf.io> https://hg.python.org/cpython/rev/e40f6c3dc114 changeset: 99922:e40f6c3dc114 branch: 3.5 parent: 99920:c87cc05af8e7 user: Senthil Kumaran date: Fri Jan 15 21:45:17 2016 -0800 summary: Issue26035 - Correct the argument names used in the docs of the traceback module. Make it consistent with module args. Patch contributed by Upendra Kumar. files: Doc/library/traceback.rst | 69 +++++++++++++------------- 1 files changed, 35 insertions(+), 34 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -20,29 +20,29 @@ The module defines the following functions: -.. function:: print_tb(traceback, limit=None, file=None) +.. function:: print_tb(tb, limit=None, file=None) - Print up to *limit* stack trace entries from *traceback* (starting from - the caller's frame) if *limit* is positive. Otherwise, print the last - ``abs(limit)`` entries. If *limit* is omitted or ``None``, all entries - are printed. If *file* is omitted or ``None``, the output goes to - ``sys.stderr``; otherwise it should be an open file or file-like object - to receive the output. + Print up to *limit* stack trace entries from traceback object *tb* (starting + from the caller's frame) if *limit* is positive. Otherwise, print the last + ``abs(limit)`` entries. If *limit* is omitted or ``None``, all entries are + printed. If *file* is omitted or ``None``, the output goes to + ``sys.stderr``; otherwise it should be an open file or file-like object to + receive the output. .. versionchanged:: 3.5 Added negative *limit* support. -.. function:: print_exception(type, value, traceback, limit=None, file=None, chain=True) +.. function:: print_exception(etype, value, tb, limit=None, file=None, chain=True) - Print exception information and stack trace entries from - *traceback* to *file*. This differs from :func:`print_tb` in the following + Print exception information and stack trace entries from traceback object + *tb* to *file*. This differs from :func:`print_tb` in the following ways: - * if *traceback* is not ``None``, it prints a header ``Traceback (most recent + * if *tb* is not ``None``, it prints a header ``Traceback (most recent call last):`` - * it prints the exception *type* and *value* after the stack trace - * if *type* is :exc:`SyntaxError` and *value* has the appropriate format, it + * it prints the exception *etype* and *value* after the stack trace + * if *etype* is :exc:`SyntaxError` and *value* has the appropriate format, it prints the line where the syntax error occurred with a caret indicating the approximate position of the error. @@ -80,10 +80,10 @@ Added negative *limit* support. -.. function:: extract_tb(traceback, limit=None) +.. function:: extract_tb(tb, limit=None) Return a list of "pre-processed" stack trace entries extracted from the - traceback object *traceback*. It is useful for alternate formatting of + traceback object *tb*. It is useful for alternate formatting of stack traces. The optional *limit* argument has the same meaning as for :func:`print_tb`. A "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, *function name*, *text*) representing the @@ -99,39 +99,40 @@ arguments have the same meaning as for :func:`print_stack`. -.. function:: format_list(list) +.. function:: format_list(extracted_list) Given a list of tuples as returned by :func:`extract_tb` or - :func:`extract_stack`, return a list of strings ready for printing. Each string - in the resulting list corresponds to the item with the same index in the - argument list. Each string ends in a newline; the strings may contain internal - newlines as well, for those items whose source text line is not ``None``. + :func:`extract_stack`, return a list of strings ready for printing. Each + string in the resulting list corresponds to the item with the same index in + the argument list. Each string ends in a newline; the strings may contain + internal newlines as well, for those items whose source text line is not + ``None``. -.. function:: format_exception_only(type, value) +.. function:: format_exception_only(etype, value) - Format the exception part of a traceback. The arguments are the exception type - and value such as given by ``sys.last_type`` and ``sys.last_value``. The return - value is a list of strings, each ending in a newline. Normally, the list - contains a single string; however, for :exc:`SyntaxError` exceptions, it - contains several lines that (when printed) display detailed information about - where the syntax error occurred. The message indicating which exception - occurred is the always last string in the list. + Format the exception part of a traceback. The arguments are the exception + type and value such as given by ``sys.last_type`` and ``sys.last_value``. + The return value is a list of strings, each ending in a newline. Normally, + the list contains a single string; however, for :exc:`SyntaxError` + exceptions, it contains several lines that (when printed) display detailed + information about where the syntax error occurred. The message indicating + which exception occurred is the always last string in the list. -.. function:: format_exception(type, value, tb, limit=None, chain=True) +.. function:: format_exception(etype, value, tb, limit=None, chain=True) Format a stack trace and the exception information. The arguments have the same meaning as the corresponding arguments to :func:`print_exception`. The - return value is a list of strings, each ending in a newline and some containing - internal newlines. When these lines are concatenated and printed, exactly the - same text is printed as does :func:`print_exception`. + return value is a list of strings, each ending in a newline and some + containing internal newlines. When these lines are concatenated and printed, + exactly the same text is printed as does :func:`print_exception`. .. function:: format_exc(limit=None, chain=True) - This is like ``print_exc(limit)`` but returns a string instead of printing to a - file. + This is like ``print_exc(limit)`` but returns a string instead of printing to + a file. .. function:: format_tb(tb, limit=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 00:46:14 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 16 Jan 2016 05:46:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160116054614.96360.82791@psf.io> https://hg.python.org/cpython/rev/e96c1491896d changeset: 99923:e96c1491896d parent: 99921:620a37dbc686 parent: 99922:e40f6c3dc114 user: Senthil Kumaran date: Fri Jan 15 21:46:08 2016 -0800 summary: Merge from 3.5 Issue26035 - Correct the argument names used in the docs of the traceback module. Make it consistent with module args. Patch contributed by Upendra Kumar. files: Doc/library/traceback.rst | 69 +++++++++++++------------- 1 files changed, 35 insertions(+), 34 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -20,29 +20,29 @@ The module defines the following functions: -.. function:: print_tb(traceback, limit=None, file=None) +.. function:: print_tb(tb, limit=None, file=None) - Print up to *limit* stack trace entries from *traceback* (starting from - the caller's frame) if *limit* is positive. Otherwise, print the last - ``abs(limit)`` entries. If *limit* is omitted or ``None``, all entries - are printed. If *file* is omitted or ``None``, the output goes to - ``sys.stderr``; otherwise it should be an open file or file-like object - to receive the output. + Print up to *limit* stack trace entries from traceback object *tb* (starting + from the caller's frame) if *limit* is positive. Otherwise, print the last + ``abs(limit)`` entries. If *limit* is omitted or ``None``, all entries are + printed. If *file* is omitted or ``None``, the output goes to + ``sys.stderr``; otherwise it should be an open file or file-like object to + receive the output. .. versionchanged:: 3.5 Added negative *limit* support. -.. function:: print_exception(type, value, traceback, limit=None, file=None, chain=True) +.. function:: print_exception(etype, value, tb, limit=None, file=None, chain=True) - Print exception information and stack trace entries from - *traceback* to *file*. This differs from :func:`print_tb` in the following + Print exception information and stack trace entries from traceback object + *tb* to *file*. This differs from :func:`print_tb` in the following ways: - * if *traceback* is not ``None``, it prints a header ``Traceback (most recent + * if *tb* is not ``None``, it prints a header ``Traceback (most recent call last):`` - * it prints the exception *type* and *value* after the stack trace - * if *type* is :exc:`SyntaxError` and *value* has the appropriate format, it + * it prints the exception *etype* and *value* after the stack trace + * if *etype* is :exc:`SyntaxError` and *value* has the appropriate format, it prints the line where the syntax error occurred with a caret indicating the approximate position of the error. @@ -80,10 +80,10 @@ Added negative *limit* support. -.. function:: extract_tb(traceback, limit=None) +.. function:: extract_tb(tb, limit=None) Return a list of "pre-processed" stack trace entries extracted from the - traceback object *traceback*. It is useful for alternate formatting of + traceback object *tb*. It is useful for alternate formatting of stack traces. The optional *limit* argument has the same meaning as for :func:`print_tb`. A "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, *function name*, *text*) representing the @@ -99,39 +99,40 @@ arguments have the same meaning as for :func:`print_stack`. -.. function:: format_list(list) +.. function:: format_list(extracted_list) Given a list of tuples as returned by :func:`extract_tb` or - :func:`extract_stack`, return a list of strings ready for printing. Each string - in the resulting list corresponds to the item with the same index in the - argument list. Each string ends in a newline; the strings may contain internal - newlines as well, for those items whose source text line is not ``None``. + :func:`extract_stack`, return a list of strings ready for printing. Each + string in the resulting list corresponds to the item with the same index in + the argument list. Each string ends in a newline; the strings may contain + internal newlines as well, for those items whose source text line is not + ``None``. -.. function:: format_exception_only(type, value) +.. function:: format_exception_only(etype, value) - Format the exception part of a traceback. The arguments are the exception type - and value such as given by ``sys.last_type`` and ``sys.last_value``. The return - value is a list of strings, each ending in a newline. Normally, the list - contains a single string; however, for :exc:`SyntaxError` exceptions, it - contains several lines that (when printed) display detailed information about - where the syntax error occurred. The message indicating which exception - occurred is the always last string in the list. + Format the exception part of a traceback. The arguments are the exception + type and value such as given by ``sys.last_type`` and ``sys.last_value``. + The return value is a list of strings, each ending in a newline. Normally, + the list contains a single string; however, for :exc:`SyntaxError` + exceptions, it contains several lines that (when printed) display detailed + information about where the syntax error occurred. The message indicating + which exception occurred is the always last string in the list. -.. function:: format_exception(type, value, tb, limit=None, chain=True) +.. function:: format_exception(etype, value, tb, limit=None, chain=True) Format a stack trace and the exception information. The arguments have the same meaning as the corresponding arguments to :func:`print_exception`. The - return value is a list of strings, each ending in a newline and some containing - internal newlines. When these lines are concatenated and printed, exactly the - same text is printed as does :func:`print_exception`. + return value is a list of strings, each ending in a newline and some + containing internal newlines. When these lines are concatenated and printed, + exactly the same text is printed as does :func:`print_exception`. .. function:: format_exc(limit=None, chain=True) - This is like ``print_exc(limit)`` but returns a string instead of printing to a - file. + This is like ``print_exc(limit)`` but returns a string instead of printing to + a file. .. function:: format_tb(tb, limit=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 01:13:22 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sat, 16 Jan 2016 06:13:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue26035_-_C?= =?utf-8?q?orrect_the_argument_names_used_in_the_docs_of_the_traceback?= Message-ID: <20160116061322.41332.58380@psf.io> https://hg.python.org/cpython/rev/daff4ced1b32 changeset: 99924:daff4ced1b32 branch: 2.7 parent: 99919:8b8ac7adbf49 user: Senthil Kumaran date: Fri Jan 15 22:13:16 2016 -0800 summary: Issue26035 - Correct the argument names used in the docs of the traceback module. Make it consistent with module args. files: Doc/library/traceback.rst | 99 +++++++++++++------------- 1 files changed, 51 insertions(+), 48 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -14,29 +14,30 @@ .. index:: object: traceback The module uses traceback objects --- this is the object type that is stored in -the variables :data:`sys.exc_traceback` (deprecated) and :data:`sys.last_traceback` and -returned as the third item from :func:`sys.exc_info`. +the variables :data:`sys.exc_traceback` (deprecated) and +:data:`sys.last_traceback` and returned as the third item from +:func:`sys.exc_info`. The module defines the following functions: -.. function:: print_tb(traceback[, limit[, file]]) +.. function:: print_tb(tb[, limit[, file]]) - Print up to *limit* stack trace entries from *traceback*. If *limit* is omitted - or ``None``, all entries are printed. If *file* is omitted or ``None``, the - output goes to ``sys.stderr``; otherwise it should be an open file or file-like - object to receive the output. + Print up to *limit* stack trace entries from the traceback object *tb*. If + *limit* is omitted or ``None``, all entries are printed. If *file* is omitted + or ``None``, the output goes to ``sys.stderr``; otherwise it should be an + open file or file-like object to receive the output. -.. function:: print_exception(type, value, traceback[, limit[, file]]) +.. function:: print_exception(etype, value, tb[, limit[, file]]) - Print exception information and up to *limit* stack trace entries from - *traceback* to *file*. This differs from :func:`print_tb` in the following ways: - (1) if *traceback* is not ``None``, it prints a header ``Traceback (most recent - call last):``; (2) it prints the exception *type* and *value* after the stack - trace; (3) if *type* is :exc:`SyntaxError` and *value* has the appropriate - format, it prints the line where the syntax error occurred with a caret - indicating the approximate position of the error. + Print exception information and up to *limit* stack trace entries from the + traceback *tb* to *file*. This differs from :func:`print_tb` in the following + ways: (1) if *tb* is not ``None``, it prints a header ``Traceback (most + recent call last):``; (2) it prints the exception *etype* and *value* after + the stack trace; (3) if *etype* is :exc:`SyntaxError` and *value* has the + appropriate format, it prints the line where the syntax error occurred with a + caret indicating the approximate position of the error. .. function:: print_exc([limit[, file]]) @@ -49,8 +50,8 @@ .. function:: format_exc([limit]) - This is like ``print_exc(limit)`` but returns a string instead of printing to a - file. + This is like ``print_exc(limit)`` but returns a string instead of printing to + a file. .. versionadded:: 2.4 @@ -64,21 +65,21 @@ .. function:: print_stack([f[, limit[, file]]]) - This function prints a stack trace from its invocation point. The optional *f* - argument can be used to specify an alternate stack frame to start. The optional - *limit* and *file* arguments have the same meaning as for + This function prints a stack trace from its invocation point. The optional + *f* argument can be used to specify an alternate stack frame to start. The + optional limit* and *file* arguments have the same meaning as for :func:`print_exception`. -.. function:: extract_tb(traceback[, limit]) +.. function:: extract_tb(tb[, limit]) Return a list of up to *limit* "pre-processed" stack trace entries extracted - from the traceback object *traceback*. It is useful for alternate formatting of - stack traces. If *limit* is omitted or ``None``, all entries are extracted. A - "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, - *function name*, *text*) representing the information that is usually printed - for a stack trace. The *text* is a string with leading and trailing whitespace - stripped; if the source is not available it is ``None``. + from the traceback object *tb*. It is useful for alternate formatting of + stack traces. If *limit* is omitted or ``None``, all entries are extracted. + A "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, + function name*, *text*) representing the information that is usually printed + for a stack trace. The *text* is a string with leading and trailing + whitespace stripped; if the source is not available it is ``None``. .. function:: extract_stack([f[, limit]]) @@ -88,33 +89,35 @@ arguments have the same meaning as for :func:`print_stack`. -.. function:: format_list(list) +.. function:: format_list(extracted_list) Given a list of tuples as returned by :func:`extract_tb` or - :func:`extract_stack`, return a list of strings ready for printing. Each string - in the resulting list corresponds to the item with the same index in the - argument list. Each string ends in a newline; the strings may contain internal - newlines as well, for those items whose source text line is not ``None``. + :func:`extract_stack`, return a list of strings ready for printing. Each + string in the resulting list corresponds to the item with the same index in + the argument list. Each string ends in a newline; the strings may contain + internal newlines as well, for those items whose source text line is not + ``None``. -.. function:: format_exception_only(type, value) +.. function:: format_exception_only(etype, value) - Format the exception part of a traceback. The arguments are the exception type - and value such as given by ``sys.last_type`` and ``sys.last_value``. The return - value is a list of strings, each ending in a newline. Normally, the list - contains a single string; however, for :exc:`SyntaxError` exceptions, it - contains several lines that (when printed) display detailed information about - where the syntax error occurred. The message indicating which exception - occurred is the always last string in the list. + Format the exception part of a traceback. The arguments are the exception + type, *etype* and *value* such as given by ``sys.last_type`` and + ``sys.last_value``. The return value is a list of strings, each ending in a + newline. Normally, the list contains a single string; however, for + :exc:`SyntaxError` exceptions, it contains several lines that (when printed) + display detailed information about where the syntax error occurred. The + message indicating which exception occurred is the always last string in the + list. -.. function:: format_exception(type, value, tb[, limit]) +.. function:: format_exception(etype, value, tb[, limit]) Format a stack trace and the exception information. The arguments have the same meaning as the corresponding arguments to :func:`print_exception`. The - return value is a list of strings, each ending in a newline and some containing - internal newlines. When these lines are concatenated and printed, exactly the - same text is printed as does :func:`print_exception`. + return value is a list of strings, each ending in a newline and some + containing internal newlines. When these lines are concatenated and printed, + exactly the same text is printed as does :func:`print_exception`. .. function:: format_tb(tb[, limit]) @@ -129,10 +132,10 @@ .. function:: tb_lineno(tb) - This function returns the current line number set in the traceback object. This - function was necessary because in versions of Python prior to 2.3 when the - :option:`-O` flag was passed to Python the ``tb.tb_lineno`` was not updated - correctly. This function has no use in versions past 2.3. + This function returns the current line number set in the traceback object. + This function was necessary because in versions of Python prior to 2.3 when + the :option:`-O` flag was passed to Python the ``tb.tb_lineno`` was not + updated correctly. This function has no use in versions past 2.3. .. _traceback-example: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 02:39:22 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 07:39:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323883=3A_Add_miss?= =?utf-8?q?ing_APIs_to_calendar=2E=5F=5Fall=5F=5F?= Message-ID: <20160116073922.14201.40170@psf.io> https://hg.python.org/cpython/rev/a2ffa9eedb1b changeset: 99926:a2ffa9eedb1b user: Martin Panter date: Sat Jan 16 06:49:30 2016 +0000 summary: Issue #23883: Add missing APIs to calendar.__all__ Patch by Joel Taddei and Jacek Ko?odziej. files: Lib/calendar.py | 4 +++- Lib/test/test_calendar.py | 9 +++++++++ Misc/ACKS | 1 + 3 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/calendar.py b/Lib/calendar.py --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -12,7 +12,9 @@ __all__ = ["IllegalMonthError", "IllegalWeekdayError", "setfirstweekday", "firstweekday", "isleap", "leapdays", "weekday", "monthrange", "monthcalendar", "prmonth", "month", "prcal", "calendar", - "timegm", "month_name", "month_abbr", "day_name", "day_abbr"] + "timegm", "month_name", "month_abbr", "day_name", "day_abbr", + "Calendar", "TextCalendar", "HTMLCalendar", "LocaleTextCalendar", + "LocaleHTMLCalendar", "weekheader"] # Exception raised for bad input (with string parameter for details) error = ValueError diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -815,5 +815,14 @@ b'href="custom.css" />', stdout) +class MiscTestCase(unittest.TestCase): + def test__all__(self): + blacklist = {'error', 'mdays', 'January', 'February', 'EPOCH', + 'MONDAY', 'TUESDAY', 'WEDNESDAY', 'THURSDAY', 'FRIDAY', + 'SATURDAY', 'SUNDAY', 'different_locale', 'c', + 'prweek', 'week', 'format', 'formatstring', 'main'} + support.check__all__(self, calendar, blacklist=blacklist) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1419,6 +1419,7 @@ John Szakmeister Amir Szekely Maciej Szulik +Joel Taddei Arfrever Frehtes Taifersar Arahesis Hideaki Takahashi Takase Arihiro -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 02:39:23 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 07:39:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323883=3A_Add_miss?= =?utf-8?b?aW5nIEFQSXMgdG8gdGFyZmlsZS5fX2FsbF9f?= Message-ID: <20160116073922.14203.77464@psf.io> https://hg.python.org/cpython/rev/48090e08e367 changeset: 99927:48090e08e367 user: Martin Panter date: Sat Jan 16 06:59:13 2016 +0000 summary: Issue #23883: Add missing APIs to tarfile.__all__ Patch by Joel Taddei and Jacek Ko?odziej. files: Lib/tarfile.py | 5 ++++- Lib/test/test_tarfile.py | 18 ++++++++++++++++++ 2 files changed, 22 insertions(+), 1 deletions(-) diff --git a/Lib/tarfile.py b/Lib/tarfile.py --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -64,7 +64,10 @@ pass # from tarfile import * -__all__ = ["TarFile", "TarInfo", "is_tarfile", "TarError"] +__all__ = ["TarFile", "TarInfo", "is_tarfile", "TarError", "ReadError", + "CompressionError", "StreamError", "ExtractError", "HeaderError", + "ENCODING", "USTAR_FORMAT", "GNU_FORMAT", "PAX_FORMAT", + "DEFAULT_FORMAT", "open"] #--------------------------------------------------------- # tar constants diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1997,6 +1997,24 @@ with self.assertRaises(ValueError): tarfile.itn(0x10000000000, 6, tarfile.GNU_FORMAT) + def test__all__(self): + blacklist = {'version', 'bltn_open', 'symlink_exception', + 'NUL', 'BLOCKSIZE', 'RECORDSIZE', 'GNU_MAGIC', + 'POSIX_MAGIC', 'LENGTH_NAME', 'LENGTH_LINK', + 'LENGTH_PREFIX', 'REGTYPE', 'AREGTYPE', 'LNKTYPE', + 'SYMTYPE', 'CHRTYPE', 'BLKTYPE', 'DIRTYPE', 'FIFOTYPE', + 'CONTTYPE', 'GNUTYPE_LONGNAME', 'GNUTYPE_LONGLINK', + 'GNUTYPE_SPARSE', 'XHDTYPE', 'XGLTYPE', 'SOLARIS_XHDTYPE', + 'SUPPORTED_TYPES', 'REGULAR_TYPES', 'GNU_TYPES', + 'PAX_FIELDS', 'PAX_NAME_FIELDS', 'PAX_NUMBER_FIELDS', + 'stn', 'nts', 'nti', 'itn', 'calc_chksums', 'copyfileobj', + 'filemode', + 'EmptyHeaderError', 'TruncatedHeaderError', + 'EOFHeaderError', 'InvalidHeaderError', + 'SubsequentHeaderError', 'ExFileObject', 'TarIter', + 'main'} + support.check__all__(self, tarfile, blacklist=blacklist) + class CommandLineTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 02:39:22 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 07:39:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323883=3A_Missing_?= =?utf-8?q?fileinput=2E=5F=5Fall=5F=5F_APIs=3B_patch_by_Mauro_SM_Rodrigues?= Message-ID: <20160116073922.111026.52545@psf.io> https://hg.python.org/cpython/rev/571632315c36 changeset: 99925:571632315c36 parent: 99923:e96c1491896d user: Martin Panter date: Sat Jan 16 06:26:54 2016 +0000 summary: Issue #23883: Missing fileinput.__all__ APIs; patch by Mauro SM Rodrigues files: Lib/fileinput.py | 3 ++- Lib/test/test_fileinput.py | 8 ++++++++ Misc/ACKS | 1 + 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/fileinput.py b/Lib/fileinput.py --- a/Lib/fileinput.py +++ b/Lib/fileinput.py @@ -82,7 +82,8 @@ import sys, os __all__ = ["input", "close", "nextfile", "filename", "lineno", "filelineno", - "isfirstline", "isstdin", "FileInput"] + "fileno", "isfirstline", "isstdin", "FileInput", "hook_compressed", + "hook_encoded"] _state = None diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -24,6 +24,7 @@ from test.support import verbose, TESTFN, run_unittest, check_warnings from test.support import unlink as safe_unlink +from test import support from unittest import mock @@ -913,5 +914,12 @@ check('rb', ['A\n', 'B\r\n', 'C\r', 'D\u20ac']) +class MiscTest(unittest.TestCase): + + def test_all(self): + blacklist = {'DEFAULT_BUFSIZE'} + support.check__all__(self, fileinput, blacklist=blacklist) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1219,6 +1219,7 @@ Kevin Rodgers Sean Rodman Giampaolo Rodola +Mauro S. M. Rodrigues Elson Rodriguez Adi Roiban Luis Rojas -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 02:39:28 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 07:39:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323883=3A_Update_n?= =?utf-8?q?ews?= Message-ID: <20160116073928.94200.97719@psf.io> https://hg.python.org/cpython/rev/a5d3ebb6ad2a changeset: 99928:a5d3ebb6ad2a user: Martin Panter date: Sat Jan 16 07:01:46 2016 +0000 summary: Issue #23883: Update news files: Doc/whatsnew/3.6.rst | 5 +++-- Misc/NEWS | 7 ++++--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -251,8 +251,9 @@ :exc:`PendingDeprecationWarning`. * The following modules have had missing APIs added to their :attr:`__all__` - attributes to match the documented APIs: :mod:`csv`, :mod:`enum`, - :mod:`ftplib`, :mod:`logging`, :mod:`optparse`, :mod:`threading` and + attributes to match the documented APIs: :mod:`calendar`, :mod:`csv`, + :mod:`enum`, :mod:`fileinput`, :mod:`ftplib`, :mod:`logging`, + :mod:`optparse`, :mod:`tarfile`, :mod:`threading` and :mod:`wave`. This means they will export new symbols when ``import *`` is used. See :issue:`23883`. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -275,9 +275,10 @@ anything as they could potentially signal a different process. - Issue #23883: Added missing APIs to __all__ to match the documented APIs - for the following modules: csv, enum, ftplib, logging, optparse, threading - and wave. Also added a test.support.check__all__() helper. Patches by - Jacek Ko?odziej. + for the following modules: calendar, csv, enum, fileinput, ftplib, logging, + optparse, tarfile, threading and wave. Also added a + test.support.check__all__() helper. Patches by Jacek Ko?odziej, Mauro + S. M. Rodrigues and Joel Taddei. - Issue #25590: In the Readline completer, only call getattr() once per attribute. Also complete names of attributes such as properties and slots -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sat Jan 16 03:46:36 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 16 Jan 2016 08:46:36 +0000 Subject: [Python-checkins] Daily reference leaks (620a37dbc686): sum=13 Message-ID: <20160116084636.5449.52394@psf.io> results for 620a37dbc686 on branch "default" -------------------------------------------- test_asyncio leaked [0, 3, 0] memory blocks, sum=3 test_collections leaked [4, 0, 0] references, sum=4 test_collections leaked [2, 0, 0] memory blocks, sum=2 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogzWe76L', '--timeout', '7200'] From python-checkins at python.org Sat Jan 16 04:05:45 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 16 Jan 2016 09:05:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323883=3A_Removed_?= =?utf-8?q?redundant_names_from_blacklists=2E?= Message-ID: <20160116090545.111038.52671@psf.io> https://hg.python.org/cpython/rev/62e925be0aff changeset: 99929:62e925be0aff user: Serhiy Storchaka date: Sat Jan 16 11:05:11 2016 +0200 summary: Issue #23883: Removed redundant names from blacklists. files: Lib/test/test_calendar.py | 2 +- Lib/test/test_tarfile.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -817,7 +817,7 @@ class MiscTestCase(unittest.TestCase): def test__all__(self): - blacklist = {'error', 'mdays', 'January', 'February', 'EPOCH', + blacklist = {'mdays', 'January', 'February', 'EPOCH', 'MONDAY', 'TUESDAY', 'WEDNESDAY', 'THURSDAY', 'FRIDAY', 'SATURDAY', 'SUNDAY', 'different_locale', 'c', 'prweek', 'week', 'format', 'formatstring', 'main'} diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1998,7 +1998,7 @@ tarfile.itn(0x10000000000, 6, tarfile.GNU_FORMAT) def test__all__(self): - blacklist = {'version', 'bltn_open', 'symlink_exception', + blacklist = {'version', 'symlink_exception', 'NUL', 'BLOCKSIZE', 'RECORDSIZE', 'GNU_MAGIC', 'POSIX_MAGIC', 'LENGTH_NAME', 'LENGTH_LINK', 'LENGTH_PREFIX', 'REGTYPE', 'AREGTYPE', 'LNKTYPE', @@ -2011,7 +2011,7 @@ 'filemode', 'EmptyHeaderError', 'TruncatedHeaderError', 'EOFHeaderError', 'InvalidHeaderError', - 'SubsequentHeaderError', 'ExFileObject', 'TarIter', + 'SubsequentHeaderError', 'ExFileObject', 'main'} support.check__all__(self, tarfile, blacklist=blacklist) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 06:02:50 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 16 Jan 2016 11:02:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323883=3A_grp_and_?= =?utf-8?q?pwd_are_None_on_Windows?= Message-ID: <20160116110248.94216.76961@psf.io> https://hg.python.org/cpython/rev/bd6127a6354f changeset: 99930:bd6127a6354f user: Martin Panter date: Sat Jan 16 11:01:14 2016 +0000 summary: Issue #23883: grp and pwd are None on Windows files: Lib/test/test_tarfile.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1998,7 +1998,7 @@ tarfile.itn(0x10000000000, 6, tarfile.GNU_FORMAT) def test__all__(self): - blacklist = {'version', 'symlink_exception', + blacklist = {'version', 'grp', 'pwd', 'symlink_exception', 'NUL', 'BLOCKSIZE', 'RECORDSIZE', 'GNU_MAGIC', 'POSIX_MAGIC', 'LENGTH_NAME', 'LENGTH_LINK', 'LENGTH_PREFIX', 'REGTYPE', 'AREGTYPE', 'LNKTYPE', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 14:59:38 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 19:59:38 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MDg5?= =?utf-8?q?=3A_Adds_short_documentation_section_for_modifying_an_install?= =?utf-8?q?=2E?= Message-ID: <20160116195938.41314.96805@psf.io> https://hg.python.org/cpython/rev/5fc9bd33a712 changeset: 99931:5fc9bd33a712 branch: 3.5 parent: 99922:e40f6c3dc114 user: Steve Dower date: Sat Jan 16 11:58:30 2016 -0800 summary: Issue #25089: Adds short documentation section for modifying an install. files: Doc/using/windows.rst | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -219,6 +219,23 @@ You may also specify the ``/quiet`` option to hide the progress display. +Modifying an install +-------------------- + +Once Python has been installed, you can add or remove features through the +Programs and Features tool that is part of Windows. Select the Python entry and +choose "Uninstall/Change" to open the installer in maintenance mode. + +"Modify" allows you to add or remove features by modifying the checkboxes - +unchanged checkboxes will not install or remove anything. Some options cannot be +changed in this mode, such as the install directory; to modify these, you will +need to remove and then reinstall Python completely. + +"Repair" will verify all the files that should be installed using the current +settings and replace any that have been removed or modified. + +"Uninstall" will remove Python entirely, with the exception of the +:ref:`launcher`, which has its own entry in Programs and Features. Other Platforms --------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 14:59:39 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 19:59:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325089=3A_Adds_short_documentation_section_for_m?= =?utf-8?q?odifying_an_install=2E?= Message-ID: <20160116195938.41342.40700@psf.io> https://hg.python.org/cpython/rev/4b06490cca47 changeset: 99932:4b06490cca47 parent: 99930:bd6127a6354f parent: 99931:5fc9bd33a712 user: Steve Dower date: Sat Jan 16 11:59:02 2016 -0800 summary: Issue #25089: Adds short documentation section for modifying an install. files: Doc/using/windows.rst | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -219,6 +219,23 @@ You may also specify the ``/quiet`` option to hide the progress display. +Modifying an install +-------------------- + +Once Python has been installed, you can add or remove features through the +Programs and Features tool that is part of Windows. Select the Python entry and +choose "Uninstall/Change" to open the installer in maintenance mode. + +"Modify" allows you to add or remove features by modifying the checkboxes - +unchanged checkboxes will not install or remove anything. Some options cannot be +changed in this mode, such as the install directory; to modify these, you will +need to remove and then reinstall Python completely. + +"Repair" will verify all the files that should be installed using the current +settings and replace any that have been removed or modified. + +"Uninstall" will remove Python entirely, with the exception of the +:ref:`launcher`, which has its own entry in Programs and Features. Other Platforms --------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 15:40:34 2016 From: python-checkins at python.org (ethan.furman) Date: Sat, 16 Jan 2016 20:40:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_public_=27value=27?= Message-ID: <20160116204034.111040.19573@psf.io> https://hg.python.org/cpython/rev/e4c22eadc25c changeset: 99933:e4c22eadc25c user: Ethan Furman date: Sat Jan 16 12:39:53 2016 -0800 summary: use public 'value' files: Doc/library/enum.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -754,7 +754,7 @@ your class:: def __bool__(self): - return bool(self._value_) + return bool(self.value) The :attr:`__members__` attribute is only available on the class. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 15:41:17 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 20:41:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1ODUw?= =?utf-8?q?=3A_Use_cross-compilation_by_default_for_64-bit_Windows=2E?= Message-ID: <20160116204117.41330.73996@psf.io> https://hg.python.org/cpython/rev/fc117df27143 changeset: 99934:fc117df27143 branch: 3.5 parent: 99931:5fc9bd33a712 user: Steve Dower date: Sat Jan 16 12:39:10 2016 -0800 summary: Issue #25850: Use cross-compilation by default for 64-bit Windows. files: Lib/distutils/_msvccompiler.py | 21 +++++---------------- Misc/NEWS | 2 ++ 2 files changed, 7 insertions(+), 16 deletions(-) diff --git a/Lib/distutils/_msvccompiler.py b/Lib/distutils/_msvccompiler.py --- a/Lib/distutils/_msvccompiler.py +++ b/Lib/distutils/_msvccompiler.py @@ -125,11 +125,11 @@ return exe # A map keyed by get_platform() return values to values accepted by -# 'vcvarsall.bat'. Note a cross-compile may combine these (eg, 'x86_amd64' is -# the param to cross-compile on x86 targetting amd64.) +# 'vcvarsall.bat'. Always cross-compile from x86 to work with the +# lighter-weight MSVC installs that do not include native 64-bit tools. PLAT_TO_VCVARS = { 'win32' : 'x86', - 'win-amd64' : 'amd64', + 'win-amd64' : 'x86_amd64', } # A map keyed by get_platform() return values to the file under @@ -193,19 +193,8 @@ raise DistutilsPlatformError("--plat-name must be one of {}" .format(tuple(PLAT_TO_VCVARS))) - # On x86, 'vcvarsall.bat amd64' creates an env that doesn't work; - # to cross compile, you use 'x86_amd64'. - # On AMD64, 'vcvarsall.bat amd64' is a native build env; to cross - # compile use 'x86' (ie, it runs the x86 compiler directly) - if plat_name == get_platform() or plat_name == 'win32': - # native build or cross-compile to win32 - plat_spec = PLAT_TO_VCVARS[plat_name] - else: - # cross compile from win32 -> some 64bit - plat_spec = '{}_{}'.format( - PLAT_TO_VCVARS[get_platform()], - PLAT_TO_VCVARS[plat_name] - ) + # Get the vcvarsall.bat spec for the requested platform. + plat_spec = PLAT_TO_VCVARS[plat_name] vc_env = _get_vc_env(plat_spec) if not vc_env: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,8 @@ Library ------- +- Issue #25850: Use cross-compilation by default for 64-bit Windows. + - Issue #17633: Improve zipimport's support for namespace packages. - Issue #24705: Fix sysconfig._parse_makefile not expanding ${} vars -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 15:41:18 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 20:41:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325850=3A_Use_cross-compilation_by_default_for_6?= =?utf-8?q?4-bit_Windows=2E?= Message-ID: <20160116204117.111050.83537@psf.io> https://hg.python.org/cpython/rev/37dc870175be changeset: 99935:37dc870175be parent: 99932:4b06490cca47 parent: 99934:fc117df27143 user: Steve Dower date: Sat Jan 16 12:40:19 2016 -0800 summary: Issue #25850: Use cross-compilation by default for 64-bit Windows. files: Lib/distutils/_msvccompiler.py | 21 +++++---------------- Misc/NEWS | 2 ++ 2 files changed, 7 insertions(+), 16 deletions(-) diff --git a/Lib/distutils/_msvccompiler.py b/Lib/distutils/_msvccompiler.py --- a/Lib/distutils/_msvccompiler.py +++ b/Lib/distutils/_msvccompiler.py @@ -125,11 +125,11 @@ return exe # A map keyed by get_platform() return values to values accepted by -# 'vcvarsall.bat'. Note a cross-compile may combine these (eg, 'x86_amd64' is -# the param to cross-compile on x86 targetting amd64.) +# 'vcvarsall.bat'. Always cross-compile from x86 to work with the +# lighter-weight MSVC installs that do not include native 64-bit tools. PLAT_TO_VCVARS = { 'win32' : 'x86', - 'win-amd64' : 'amd64', + 'win-amd64' : 'x86_amd64', } # A map keyed by get_platform() return values to the file under @@ -193,19 +193,8 @@ raise DistutilsPlatformError("--plat-name must be one of {}" .format(tuple(PLAT_TO_VCVARS))) - # On x86, 'vcvarsall.bat amd64' creates an env that doesn't work; - # to cross compile, you use 'x86_amd64'. - # On AMD64, 'vcvarsall.bat amd64' is a native build env; to cross - # compile use 'x86' (ie, it runs the x86 compiler directly) - if plat_name == get_platform() or plat_name == 'win32': - # native build or cross-compile to win32 - plat_spec = PLAT_TO_VCVARS[plat_name] - else: - # cross compile from win32 -> some 64bit - plat_spec = '{}_{}'.format( - PLAT_TO_VCVARS[get_platform()], - PLAT_TO_VCVARS[plat_name] - ) + # Get the vcvarsall.bat spec for the requested platform. + plat_spec = PLAT_TO_VCVARS[plat_name] vc_env = _get_vc_env(plat_spec) if not vc_env: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,8 @@ Library ------- +- Issue #25850: Use cross-compilation by default for 64-bit Windows. + - Issue #25822: Add docstrings to the fields of urllib.parse results. Patch contributed by Swati Jaiswal. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 15:41:23 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 20:41:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160116204123.111026.91840@psf.io> https://hg.python.org/cpython/rev/a092030f1f9e changeset: 99936:a092030f1f9e parent: 99935:37dc870175be parent: 99933:e4c22eadc25c user: Steve Dower date: Sat Jan 16 12:41:03 2016 -0800 summary: Merge files: Doc/library/enum.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -754,7 +754,7 @@ your class:: def __bool__(self): - return bool(self._value_) + return bool(self.value) The :attr:`__members__` attribute is only available on the class. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:42:03 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:42:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1ODI0?= =?utf-8?q?=3A_Fixes_sys=2Ewinver_to_not_include_any_architecture_suffix?= =?utf-8?q?=2E?= Message-ID: <20160116214203.25914.6432@psf.io> https://hg.python.org/cpython/rev/b91c323a605e changeset: 99937:b91c323a605e branch: 2.7 parent: 99924:daff4ced1b32 user: Steve Dower date: Sat Jan 16 13:41:48 2016 -0800 summary: Issue #25824: Fixes sys.winver to not include any architecture suffix. files: Misc/NEWS | 2 ++ PCbuild/python.props | 1 - 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -88,6 +88,8 @@ Build ----- +- Issue #25824: Fixes sys.winver to not include any architecture suffix. + - Issue #25348: Added ``--pgo`` and ``--pgo-job`` arguments to ``PCbuild\build.bat`` for building with Profile-Guided Optimization. The old ``PCbuild\build_pgo.bat`` script is now deprecated, and simply calls diff --git a/PCbuild/python.props b/PCbuild/python.props --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -102,7 +102,6 @@ $(MajorVersionNumber).$(MinorVersionNumber) - $(SysWinVer)-32 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:45:47 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:45:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDY1?= =?utf-8?q?=3A_Excludes_venv_from_library_when_generating_embeddable_distr?= =?utf-8?q?o=2E?= Message-ID: <20160116214546.25971.19067@psf.io> https://hg.python.org/cpython/rev/84101e587f47 changeset: 99938:84101e587f47 branch: 3.5 parent: 99934:fc117df27143 user: Steve Dower date: Sat Jan 16 13:44:43 2016 -0800 summary: Issue #26065: Excludes venv from library when generating embeddable distro. files: Misc/NEWS | 5 +++++ Tools/msi/make_zip.py | 1 + 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -170,6 +170,11 @@ - Issue #25798: Update OS X 10.5 installer to use OpenSSL 1.0.2e. +Windows +------- + +- Issue #26065: Excludes venv from library when generating embeddable + distro. What's New in Python 3.5.1 final? ================================= diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -23,6 +23,7 @@ 'site-packages', 'tkinter', 'turtledemo', + 'venv', } EXCLUDE_FILE_FROM_LIBRARY = { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:45:47 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:45:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326065=3A_Excludes_venv_from_library_when_genera?= =?utf-8?q?ting_embeddable_distro=2E?= Message-ID: <20160116214547.41342.77424@psf.io> https://hg.python.org/cpython/rev/d3422b0e0cee changeset: 99939:d3422b0e0cee parent: 99936:a092030f1f9e parent: 99938:84101e587f47 user: Steve Dower date: Sat Jan 16 13:45:32 2016 -0800 summary: Issue #26065: Excludes venv from library when generating embeddable distro. files: Misc/NEWS | 3 +++ Tools/msi/make_zip.py | 1 + 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -613,6 +613,9 @@ Windows ------- +- Issue #26065: Excludes venv from library when generating embeddable + distro. + - Issue #25022: Removed very outdated PC/example_nt/ directory. Tools/Demos diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -23,6 +23,7 @@ 'site-packages', 'tkinter', 'turtledemo', + 'venv', } EXCLUDE_FILE_FROM_LIBRARY = { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:48:49 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:48:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326073=3A_Update_the_list_of_magic_numbers_in_la?= =?utf-8?q?uncher?= Message-ID: <20160116214849.18417.44925@psf.io> https://hg.python.org/cpython/rev/cf868f77b400 changeset: 99941:cf868f77b400 parent: 99939:d3422b0e0cee parent: 99940:1a2cfa52f749 user: Steve Dower date: Sat Jan 16 13:48:31 2016 -0800 summary: Issue #26073: Update the list of magic numbers in launcher files: Misc/NEWS | 2 ++ PC/launcher.c | 5 ++++- 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -613,6 +613,8 @@ Windows ------- +- Issue #26073: Update the list of magic numbers in launcher + - Issue #26065: Excludes venv from library when generating embeddable distro. diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1079,7 +1079,10 @@ { 0x0bb8, 0x0c3b, L"3.0" }, { 0x0c45, 0x0c4f, L"3.1" }, { 0x0c58, 0x0c6c, L"3.2" }, - { 0x0c76, 0x0c76, L"3.3" }, + { 0x0c76, 0x0c9e, L"3.3" }, + { 0x0cb2, 0x0cee, L"3.4" }, + { 0x0cf8, 0x0d16, L"3.5" }, + { 0x0d20, 0x0d20, L"3.6" }, { 0 } }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:48:49 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:48:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDcz?= =?utf-8?q?=3A_Update_the_list_of_magic_numbers_in_launcher?= Message-ID: <20160116214849.25977.12775@psf.io> https://hg.python.org/cpython/rev/1a2cfa52f749 changeset: 99940:1a2cfa52f749 branch: 3.5 parent: 99938:84101e587f47 user: Steve Dower date: Sat Jan 16 13:48:06 2016 -0800 summary: Issue #26073: Update the list of magic numbers in launcher files: Misc/NEWS | 2 ++ PC/launcher.c | 5 ++++- 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -173,6 +173,8 @@ Windows ------- +- Issue #26073: Update the list of magic numbers in launcher + - Issue #26065: Excludes venv from library when generating embeddable distro. diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1079,7 +1079,10 @@ { 0x0bb8, 0x0c3b, L"3.0" }, { 0x0c45, 0x0c4f, L"3.1" }, { 0x0c58, 0x0c6c, L"3.2" }, - { 0x0c76, 0x0c76, L"3.3" }, + { 0x0c76, 0x0c9e, L"3.3" }, + { 0x0cb2, 0x0cee, L"3.4" }, + { 0x0cf8, 0x0d16, L"3.5" }, + { 0x0d20, 0x0d20, L"3.6" }, { 0 } }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:55:28 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:55:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326071=3A_bdist=5Fwininst_created_binaries_fail_?= =?utf-8?q?to_start_and_find_32bit_Python?= Message-ID: <20160116215528.18431.96213@psf.io> https://hg.python.org/cpython/rev/db74f3a4cbeb changeset: 99943:db74f3a4cbeb parent: 99941:cf868f77b400 parent: 99942:daa6fdaf9835 user: Steve Dower date: Sat Jan 16 13:55:05 2016 -0800 summary: Issue #26071: bdist_wininst created binaries fail to start and find 32bit Python files: Lib/distutils/command/wininst-14.0-amd64.exe | Bin Lib/distutils/command/wininst-14.0.exe | Bin Misc/NEWS | 3 + PC/bdist_wininst/bdist_wininst.vcxproj | 2 + PC/bdist_wininst/install.c | 17 ++++++++- 5 files changed, 19 insertions(+), 3 deletions(-) diff --git a/Lib/distutils/command/wininst-14.0-amd64.exe b/Lib/distutils/command/wininst-14.0-amd64.exe index 7a5e78d52bcc5db0be1baff6cdad41e172a5b81f..22299543a97ffc1525a3b1c778cb158d6c6430ad GIT binary patch [stripped] diff --git a/Lib/distutils/command/wininst-14.0.exe b/Lib/distutils/command/wininst-14.0.exe index cc43296b677a5b995899bb60bbe0e632163e64ae..0dac1103d98db0af1e9027c41fe921136c5f6396 GIT binary patch [stripped] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -613,6 +613,9 @@ Windows ------- +- Issue #26071: bdist_wininst created binaries fail to start and find + 32bit Python + - Issue #26073: Update the list of magic numbers in launcher - Issue #26065: Excludes venv from library when generating embeddable diff --git a/PC/bdist_wininst/bdist_wininst.vcxproj b/PC/bdist_wininst/bdist_wininst.vcxproj --- a/PC/bdist_wininst/bdist_wininst.vcxproj +++ b/PC/bdist_wininst/bdist_wininst.vcxproj @@ -70,6 +70,8 @@ MinSpace $(PySourcePath)Modules\zlib;%(AdditionalIncludeDirectories) _CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;%(PreprocessorDefinitions) + MultiThreadedDebug + MultiThreaded $(PySourcePath)PC\bdist_wininst;%(AdditionalIncludeDirectories) diff --git a/PC/bdist_wininst/install.c b/PC/bdist_wininst/install.c --- a/PC/bdist_wininst/install.c +++ b/PC/bdist_wininst/install.c @@ -153,6 +153,13 @@ HANDLE hBitmap; char *bitmap_bytes; +static const char *REGISTRY_SUFFIX_6432 = +#ifdef MS_WIN64 + ""; +#else + "-32"; +#endif + #define WM_NUMFILES WM_USER+1 /* wParam: 0, lParam: total number of files */ @@ -657,8 +664,8 @@ if (h) return h; wsprintf(subkey_name, - "SOFTWARE\\Python\\PythonCore\\%d.%d\\InstallPath", - py_major, py_minor); + "SOFTWARE\\Python\\PythonCore\\%d.%d%s\\InstallPath", + py_major, py_minor, REGISTRY_SUFFIX_6432); if (ERROR_SUCCESS != RegQueryValue(HKEY_CURRENT_USER, subkey_name, fullpath, &size) && ERROR_SUCCESS != RegQueryValue(HKEY_LOCAL_MACHINE, subkey_name, @@ -666,7 +673,9 @@ return NULL; strcat(fullpath, "\\"); strcat(fullpath, fname); - return LoadLibrary(fullpath); + // We use LOAD_WITH_ALTERED_SEARCH_PATH to ensure any dependent DLLs + // next to the Python DLL (eg, the CRT DLL) are also loaded. + return LoadLibraryEx(fullpath, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); } static int prepare_script_environment(HINSTANCE hPython) @@ -2254,6 +2263,8 @@ GetPrivateProfileString("Setup", "user_access_control", "", user_access_control, sizeof(user_access_control), ini_file); + strcat(target_version, REGISTRY_SUFFIX_6432); + // See if we need to do the Vista UAC magic. if (strcmp(user_access_control, "force")==0) { if (!MyIsUserAnAdmin()) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 16:55:28 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 16 Jan 2016 21:55:28 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDcx?= =?utf-8?q?=3A_bdist=5Fwininst_created_binaries_fail_to_start_and_find_32b?= =?utf-8?q?it_Python?= Message-ID: <20160116215528.111050.52483@psf.io> https://hg.python.org/cpython/rev/daa6fdaf9835 changeset: 99942:daa6fdaf9835 branch: 3.5 parent: 99940:1a2cfa52f749 user: Steve Dower date: Sat Jan 16 13:54:53 2016 -0800 summary: Issue #26071: bdist_wininst created binaries fail to start and find 32bit Python files: Lib/distutils/command/wininst-14.0-amd64.exe | Bin Lib/distutils/command/wininst-14.0.exe | Bin Misc/NEWS | 3 + PC/bdist_wininst/bdist_wininst.vcxproj | 2 + PC/bdist_wininst/install.c | 17 ++++++++- 5 files changed, 19 insertions(+), 3 deletions(-) diff --git a/Lib/distutils/command/wininst-14.0-amd64.exe b/Lib/distutils/command/wininst-14.0-amd64.exe index 7a5e78d52bcc5db0be1baff6cdad41e172a5b81f..22299543a97ffc1525a3b1c778cb158d6c6430ad GIT binary patch [stripped] diff --git a/Lib/distutils/command/wininst-14.0.exe b/Lib/distutils/command/wininst-14.0.exe index cc43296b677a5b995899bb60bbe0e632163e64ae..0dac1103d98db0af1e9027c41fe921136c5f6396 GIT binary patch [stripped] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -173,6 +173,9 @@ Windows ------- +- Issue #26071: bdist_wininst created binaries fail to start and find + 32bit Python + - Issue #26073: Update the list of magic numbers in launcher - Issue #26065: Excludes venv from library when generating embeddable diff --git a/PC/bdist_wininst/bdist_wininst.vcxproj b/PC/bdist_wininst/bdist_wininst.vcxproj --- a/PC/bdist_wininst/bdist_wininst.vcxproj +++ b/PC/bdist_wininst/bdist_wininst.vcxproj @@ -70,6 +70,8 @@ MinSpace $(PySourcePath)Modules\zlib;%(AdditionalIncludeDirectories) _CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;%(PreprocessorDefinitions) + MultiThreadedDebug + MultiThreaded $(PySourcePath)PC\bdist_wininst;%(AdditionalIncludeDirectories) diff --git a/PC/bdist_wininst/install.c b/PC/bdist_wininst/install.c --- a/PC/bdist_wininst/install.c +++ b/PC/bdist_wininst/install.c @@ -153,6 +153,13 @@ HANDLE hBitmap; char *bitmap_bytes; +static const char *REGISTRY_SUFFIX_6432 = +#ifdef MS_WIN64 + ""; +#else + "-32"; +#endif + #define WM_NUMFILES WM_USER+1 /* wParam: 0, lParam: total number of files */ @@ -657,8 +664,8 @@ if (h) return h; wsprintf(subkey_name, - "SOFTWARE\\Python\\PythonCore\\%d.%d\\InstallPath", - py_major, py_minor); + "SOFTWARE\\Python\\PythonCore\\%d.%d%s\\InstallPath", + py_major, py_minor, REGISTRY_SUFFIX_6432); if (ERROR_SUCCESS != RegQueryValue(HKEY_CURRENT_USER, subkey_name, fullpath, &size) && ERROR_SUCCESS != RegQueryValue(HKEY_LOCAL_MACHINE, subkey_name, @@ -666,7 +673,9 @@ return NULL; strcat(fullpath, "\\"); strcat(fullpath, fname); - return LoadLibrary(fullpath); + // We use LOAD_WITH_ALTERED_SEARCH_PATH to ensure any dependent DLLs + // next to the Python DLL (eg, the CRT DLL) are also loaded. + return LoadLibraryEx(fullpath, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); } static int prepare_script_environment(HINSTANCE hPython) @@ -2250,6 +2259,8 @@ GetPrivateProfileString("Setup", "user_access_control", "", user_access_control, sizeof(user_access_control), ini_file); + strcat(target_version, REGISTRY_SUFFIX_6432); + // See if we need to do the Vista UAC magic. if (strcmp(user_access_control, "force")==0) { if (!MyIsUserAnAdmin()) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 21:44:02 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 17 Jan 2016 02:44:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160117024402.18421.83220@psf.io> https://hg.python.org/cpython/rev/6636d688bbd3 changeset: 99945:6636d688bbd3 parent: 99943:db74f3a4cbeb parent: 99944:31e6d4d25b8e user: Senthil Kumaran date: Sat Jan 16 18:43:57 2016 -0800 summary: merge from 3.5 Issue26135 - In the tutorial section on modules, reference importlib.reload instead of imp.reload. files: Doc/tutorial/modules.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -117,7 +117,8 @@ For efficiency reasons, each module is only imported once per interpreter session. Therefore, if you change your modules, you must restart the interpreter -- or, if it's just one module you want to test interactively, - use :func:`imp.reload`, e.g. ``import imp; imp.reload(modulename)``. + use :func:`importlib.reload`, e.g. ``import importlib; + importlib.reload(modulename)``. .. _tut-modulesasscripts: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 21:44:03 2016 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 17 Jan 2016 02:44:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Issue26135_-_I?= =?utf-8?q?n_the_tutorial_section_on_modules=2C_reference_importlib=2Erelo?= =?utf-8?q?ad?= Message-ID: <20160117024402.111042.73861@psf.io> https://hg.python.org/cpython/rev/31e6d4d25b8e changeset: 99944:31e6d4d25b8e branch: 3.5 parent: 99942:daa6fdaf9835 user: Senthil Kumaran date: Sat Jan 16 18:43:24 2016 -0800 summary: Issue26135 - In the tutorial section on modules, reference importlib.reload instead of imp.reload. files: Doc/tutorial/modules.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -117,7 +117,8 @@ For efficiency reasons, each module is only imported once per interpreter session. Therefore, if you change your modules, you must restart the interpreter -- or, if it's just one module you want to test interactively, - use :func:`imp.reload`, e.g. ``import imp; imp.reload(modulename)``. + use :func:`importlib.reload`, e.g. ``import importlib; + importlib.reload(modulename)``. .. _tut-modulesasscripts: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 23:44:42 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 17 Jan 2016 04:44:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1OTA1?= =?utf-8?q?=3A_Specify_=27ascii=27_encoding_for_README=2Etxt_and_NEWS=2Etx?= =?utf-8?q?t=2E?= Message-ID: <20160117044442.5455.55192@psf.io> https://hg.python.org/cpython/rev/21356a5b8a5e changeset: 99946:21356a5b8a5e branch: 2.7 parent: 99937:b91c323a605e user: Terry Jan Reedy date: Sat Jan 16 23:43:58 2016 -0500 summary: Issue #25905: Specify 'ascii' encoding for README.txt and NEWS.txt. Re-encode CREDITS.txt to utf-8 and open it with 'utf-8'. files: Lib/idlelib/CREDITS.txt | 2 +- Lib/idlelib/aboutDialog.py | 9 ++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/CREDITS.txt b/Lib/idlelib/CREDITS.txt --- a/Lib/idlelib/CREDITS.txt +++ b/Lib/idlelib/CREDITS.txt @@ -24,7 +24,7 @@ integration, debugger integration and persistent breakpoints). Scott David Daniels, Tal Einat, Hernan Foffani, Christos Georgiou, -Jim Jewett, Martin v. L?wis, Jason Orendorff, Guilherme Polo, Josh Robb, +Jim Jewett, Martin v. L??wis, Jason Orendorff, Guilherme Polo, Josh Robb, Nigel Rowe, Bruce Sherwood, Jeff Shute, and Weeble have submitted useful patches. Thanks, guys! diff --git a/Lib/idlelib/aboutDialog.py b/Lib/idlelib/aboutDialog.py --- a/Lib/idlelib/aboutDialog.py +++ b/Lib/idlelib/aboutDialog.py @@ -110,6 +110,7 @@ command=self.ShowIDLECredits) idle_credits_b.pack(side=LEFT, padx=10, pady=10) + # License, et all, are of type _sitebuiltins._Printer def ShowLicense(self): self.display_printer_text('About - License', license) @@ -119,14 +120,16 @@ def ShowPythonCredits(self): self.display_printer_text('About - Python Credits', credits) + # Encode CREDITS.txt to utf-8 for proper version of Loewis. + # Specify others as ascii until need utf-8, so catch errors. def ShowIDLECredits(self): - self.display_file_text('About - Credits', 'CREDITS.txt', 'iso-8859-1') + self.display_file_text('About - Credits', 'CREDITS.txt', 'utf-8') def ShowIDLEAbout(self): - self.display_file_text('About - Readme', 'README.txt') + self.display_file_text('About - Readme', 'README.txt', 'ascii') def ShowIDLENEWS(self): - self.display_file_text('About - NEWS', 'NEWS.txt') + self.display_file_text('About - NEWS', 'NEWS.txt', 'ascii') def display_printer_text(self, title, printer): printer._Printer__setup() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 23:44:43 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 17 Jan 2016 04:44:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160117044442.25900.39060@psf.io> https://hg.python.org/cpython/rev/e48bd90262b5 changeset: 99948:e48bd90262b5 parent: 99945:6636d688bbd3 parent: 99947:59852a79b508 user: Terry Jan Reedy date: Sat Jan 16 23:44:16 2016 -0500 summary: Merge with 3.5 files: Lib/idlelib/CREDITS.txt | 2 +- Lib/idlelib/aboutDialog.py | 9 ++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/CREDITS.txt b/Lib/idlelib/CREDITS.txt --- a/Lib/idlelib/CREDITS.txt +++ b/Lib/idlelib/CREDITS.txt @@ -24,7 +24,7 @@ integration, debugger integration and persistent breakpoints). Scott David Daniels, Tal Einat, Hernan Foffani, Christos Georgiou, -Jim Jewett, Martin v. L?wis, Jason Orendorff, Guilherme Polo, Josh Robb, +Jim Jewett, Martin v. L??wis, Jason Orendorff, Guilherme Polo, Josh Robb, Nigel Rowe, Bruce Sherwood, Jeff Shute, and Weeble have submitted useful patches. Thanks, guys! diff --git a/Lib/idlelib/aboutDialog.py b/Lib/idlelib/aboutDialog.py --- a/Lib/idlelib/aboutDialog.py +++ b/Lib/idlelib/aboutDialog.py @@ -111,6 +111,7 @@ command=self.ShowIDLECredits) idle_credits_b.pack(side=LEFT, padx=10, pady=10) + # License, et all, are of type _sitebuiltins._Printer def ShowLicense(self): self.display_printer_text('About - License', license) @@ -120,14 +121,16 @@ def ShowPythonCredits(self): self.display_printer_text('About - Python Credits', credits) + # Encode CREDITS.txt to utf-8 for proper version of Loewis. + # Specify others as ascii until need utf-8, so catch errors. def ShowIDLECredits(self): - self.display_file_text('About - Credits', 'CREDITS.txt', 'iso-8859-1') + self.display_file_text('About - Credits', 'CREDITS.txt', 'utf-8') def ShowIDLEAbout(self): - self.display_file_text('About - Readme', 'README.txt') + self.display_file_text('About - Readme', 'README.txt', 'ascii') def ShowIDLENEWS(self): - self.display_file_text('About - NEWS', 'NEWS.txt') + self.display_file_text('About - NEWS', 'NEWS.txt', 'ascii') def display_printer_text(self, title, printer): printer._Printer__setup() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Jan 16 23:44:43 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 17 Jan 2016 04:44:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1OTA1?= =?utf-8?q?=3A_Specify_=27ascii=27_encoding_for_README=2Etxt_and_NEWS=2Etx?= =?utf-8?q?t=2E?= Message-ID: <20160117044442.121284.62809@psf.io> https://hg.python.org/cpython/rev/59852a79b508 changeset: 99947:59852a79b508 branch: 3.5 parent: 99944:31e6d4d25b8e user: Terry Jan Reedy date: Sat Jan 16 23:44:04 2016 -0500 summary: Issue #25905: Specify 'ascii' encoding for README.txt and NEWS.txt. Re-encode CREDITS.txt to utf-8 and open it with 'utf-8'. files: Lib/idlelib/CREDITS.txt | 2 +- Lib/idlelib/aboutDialog.py | 9 ++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/Lib/idlelib/CREDITS.txt b/Lib/idlelib/CREDITS.txt --- a/Lib/idlelib/CREDITS.txt +++ b/Lib/idlelib/CREDITS.txt @@ -24,7 +24,7 @@ integration, debugger integration and persistent breakpoints). Scott David Daniels, Tal Einat, Hernan Foffani, Christos Georgiou, -Jim Jewett, Martin v. L?wis, Jason Orendorff, Guilherme Polo, Josh Robb, +Jim Jewett, Martin v. L??wis, Jason Orendorff, Guilherme Polo, Josh Robb, Nigel Rowe, Bruce Sherwood, Jeff Shute, and Weeble have submitted useful patches. Thanks, guys! diff --git a/Lib/idlelib/aboutDialog.py b/Lib/idlelib/aboutDialog.py --- a/Lib/idlelib/aboutDialog.py +++ b/Lib/idlelib/aboutDialog.py @@ -111,6 +111,7 @@ command=self.ShowIDLECredits) idle_credits_b.pack(side=LEFT, padx=10, pady=10) + # License, et all, are of type _sitebuiltins._Printer def ShowLicense(self): self.display_printer_text('About - License', license) @@ -120,14 +121,16 @@ def ShowPythonCredits(self): self.display_printer_text('About - Python Credits', credits) + # Encode CREDITS.txt to utf-8 for proper version of Loewis. + # Specify others as ascii until need utf-8, so catch errors. def ShowIDLECredits(self): - self.display_file_text('About - Credits', 'CREDITS.txt', 'iso-8859-1') + self.display_file_text('About - Credits', 'CREDITS.txt', 'utf-8') def ShowIDLEAbout(self): - self.display_file_text('About - Readme', 'README.txt') + self.display_file_text('About - Readme', 'README.txt', 'ascii') def ShowIDLENEWS(self): - self.display_file_text('About - NEWS', 'NEWS.txt') + self.display_file_text('About - NEWS', 'NEWS.txt', 'ascii') def display_printer_text(self, title, printer): printer._Printer__setup() -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Sun Jan 17 03:43:50 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 17 Jan 2016 08:43:50 +0000 Subject: [Python-checkins] Daily reference leaks (e48bd90262b5): sum=2 Message-ID: <20160117084350.25918.44212@psf.io> results for e48bd90262b5 on branch "default" -------------------------------------------- test_collections leaked [-2, 0, 0] references, sum=-2 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogrAjLCB', '--timeout', '7200'] From python-checkins at python.org Sun Jan 17 06:29:03 2016 From: python-checkins at python.org (stefan.krah) Date: Sun, 17 Jan 2016 11:29:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326139=3A_libmpdec?= =?utf-8?q?=3A_disable_/W4_warning_=28non-standard_dllimport_behavior=29?= =?utf-8?q?=2E?= Message-ID: <20160117112903.25987.29951@psf.io> https://hg.python.org/cpython/rev/df78978dacab changeset: 99949:df78978dacab user: Stefan Krah date: Sun Jan 17 12:28:43 2016 +0100 summary: Issue #26139: libmpdec: disable /W4 warning (non-standard dllimport behavior). files: Modules/_decimal/libmpdec/memory.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Modules/_decimal/libmpdec/memory.c b/Modules/_decimal/libmpdec/memory.c --- a/Modules/_decimal/libmpdec/memory.c +++ b/Modules/_decimal/libmpdec/memory.c @@ -33,6 +33,11 @@ #include "memory.h" +#if defined(_MSC_VER) + #pragma warning(disable : 4232) +#endif + + /* Guaranteed minimum allocation for a coefficient. May be changed once at program start using mpd_setminalloc(). */ mpd_ssize_t MPD_MINALLOC = MPD_MINALLOC_MIN; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 17 15:28:21 2016 From: python-checkins at python.org (brett.cannon) Date: Sun, 17 Jan 2016 20:28:21 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_PEP_502=3A_Migrating_from?= =?utf-8?q?_hg=2Epython=2Eorg_to_GitHub?= Message-ID: <20160117202821.94220.33044@psf.io> https://hg.python.org/peps/rev/721edb923497 changeset: 6193:721edb923497 user: Brett Cannon date: Sun Jan 17 12:28:17 2016 -0800 summary: Add PEP 502: Migrating from hg.python.org to GitHub files: pep-0512.txt | 782 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 782 insertions(+), 0 deletions(-) diff --git a/pep-0512.txt b/pep-0512.txt new file mode 100644 --- /dev/null +++ b/pep-0512.txt @@ -0,0 +1,782 @@ +PEP: 512 +Title: Migrating from hg.python.org to GitHub +Version: $Revision$ +Last-Modified: $Date$ +Author: Brett Cannon +Status: Active +Type: Process +Content-Type: text/x-rst +Created: +Post-History: 17-Jan-2015 + +Abstract +======== +This PEP outlines the steps required to migrate Python's development +process from Mercurial [#hg]_ as hosted at +hg.python.org [#h.p.o]_ to Git [#git]_ on GitHub [#GitHub]_. Meeting +the minimum goals of this PEP should allow for the development +process of Python to be as productive as it currently is, and meeting +its extended goals should improve it. + +Rationale +========= +In 2014, it became obvious that Python's custom development +process was becoming a hindrance. As an example, for an external +contributor to submit a fix for a bug that eventually was committed, +the basic steps were: + +1. Open an issue for the bug at bugs.python.org [#b.p.o]_. +2. Checkout out the CPython source code from hg.python.org [#h.p.o]_. +3. Make the fix. +4. Upload a patch. +5. Have a core developer review the patch using our fork of the + Rietveld code review tool [#rietveld]_. +6. Download the patch to make sure it still applies cleanly. +7. Run the test suite manually. +8. Commit the change manually. +9. If the change was for a bugfix release, merge into the + in-development branch. +10. Run the test suite manually again. +11. Commit the merge. +12. Push the changes. + +This is a very heavy, manual process for core developers. Even in the +simple case, you could only possibly skip the code review step, as you +would still need to build the documentation. This led to patches +languishing on the issue tracker due to core developers not being +able to work through the backlog fast enough to keep up with +submissions. In turn, that led to a side-effect issue of discouraging +outside contribution due to frustration from lack of attention, which +is dangerous problem for an open source project as it runs counter to +having a viable future for the project. Simply accepting patches +uploaded to bugs.python.org [#b.p.o]_ is potentially simple for an +external contributor, it is as slow and burdensome as it gets for +a core developer to work with. + +Hence the decision was made in late 2014 that a move to a new +development process was needed. A request for PEPs +proposing new workflows was made, in the end leading to two: +PEP 481 and PEP 507 proposing GitHub [#github]_ and +GitLab [#gitlab]_, respectively. + +The year 2015 was spent off-and-on working on those proposals and +trying to tease out details of what made them different from each +other on the core-workflow mailing list [#core-workflow]_. +PyCon US 2015 also showed that the community was a bit frustrated +with our process due to both cognitive overhead for new contributors +and how long it was taking for core developers to +look at a patch (see the end of Guido van Rossum's +keynote at PyCon US 2015 [#guido-keynote]_ as an example of the +frustration). + +On January 1, 2016, the decision was made by Brett Cannon to move the +development process to GitHub. The key reasons for choosing GitHub +were [#reasons]_: + +* Maintaining custom infrastructure has been a burden on volunteers + (e.g., a custom fork of Rietveld [#rietveld]_ + that is not being maintained is currently being used). +* The custom workflow is very time-consuming for core developers + (not enough automated tooling built to help support it). +* The custom workflow is a hindrance to external contributors + (acts as a barrier of entry due to time required to ramp up on + development process). +* There is no feature differentiating GitLab from GitHub beyond + GitLab being open source. +* Familiarity with GitHub is far higher amongst core developers and + external contributors than with GitLab. +* Our BDFL prefers GitHub (who would be the first person to tell + you that his opinion shouldn't matter, but the person making the + decision felt it was important that the BDFL feel comfortable with + the workflow of his own programming language to encourage his + continued participation). + +There's even already an unofficial image to use to represent the +migration to GitHub [#pythocat]_. + +The overarching goal of this migration is to improve the development +process to the extent that a core developer can go from external +contribution submission through all the steps leading to committing +said contribution all from within a browser on a tablet with WiFi. +All of this will be done in such a way that if an external +contributor chooses not to use GitHub then they will continue to have +that option. + +Repositories to Migrate +======================= +While hg.python.org [#h.p.o]_ hosts many repositories, there are only +six key repositories that must move: + +1. devinabox [#devinabox-repo]_ +2. benchmarks [#benchmarks-repo]_ +3. tracker [#tracker-repo]_ +4. peps [#peps-repo]_ +5. devguide [#devguide-repo]_ +6. cpython [#cpython-repo]_ + +The devinabox, benchmarksm and tracker repositories are code-only. +The peps and devguide repositories involve the generation of webpages. +And the cpython repository has special requirements for integration +with bugs.python.org [#b.p.o]_. + +Migration Plan +============== +The migration plan is separated into sections based on what is +required to migrate the repositories listed in the +`Repositories to Migrate`_ section. Completion of requirements +outlined in each section should unblock the migration of the related +repositories. The sections are expected to be completed in order, but +not necessarily the requirements within a section. + +Requirements for Code-Only Repositories +--------------------------------------- +Completion of the requirements in this section will allow the +devinabox, benchmarks, and tracker repositories to move to +GitHub. While devinabox has a sufficiently descriptive name, the +benchmarks repository does not; therefore, it will be named +"python-benchmark-suite". The tracker repo also has a misleading name +and will be renamed "bugs.python.org". + +Create a 'python-dev' team +'''''''''''''''''''''''''' +To manage permissions, a 'python-dev' team will be created as part of +the python organization [#github-python-org]_. Any repository that is +moved will have the 'python-dev' team added to it with write +permissions [#github-org-perms]_. Anyone who previously had rights to +manage SSH keys on hg.python.org will become a team maintainer for the +'python-dev' team. + +Define commands to move a Mercurial repository to Git +''''''''''''''''''''''''''''''''''''''''''''''''''''' +Since moving to GitHub also entails moving to Git [#git]_, we must +decide what tools and commands we will run to translate a Mercurial +repository to Git. The exact tools and steps to use are an +open issue; see `Tools and commands to move from Mercurial to Git`_. + +CLA enforcement +''''''''''''''' +A key part of any open source project is making sure that its source +code can be properly licensed. This requires making sure all people +making contributions have signed a contributor license agreement +(CLA) [#cla]_. Up until now, enforcement of CLA signing of +contributed code has been enforced by core developers checking +whether someone had an ``*`` by their username on +bugs.python.org [#b.p.o]_. With this migration, the plan is to start +off with automated checking and enforcement of contributors signing +the CLA. + +Adding GitHub username support to bugs.python.org ++++++++++++++++++++++++++++++++++++++++++++++++++ +To keep tracking of CLA signing under the direct control of the PSF, +tracking who has signed the PSF CLA will be continued by marking that +fact as part of someone's bugs.python.org user profile. What this +means is that an association will be needed between a person's +bugs.python.org [#b.p.o]_ account and their GitHub account, which +will be done through a new field in a user's profile. + +A bot to enforce CLA signing +++++++++++++++++++++++++++++ +With an association between someone's GitHub account and their +bugs.python.org [#b.p.o]_ account, which has the data as to whether +someone has signed the CLA, a bot can monitor pull requests on +GitHub and denote whether the contributor has signed the CLA. + +If the user has signed the CLA, the bot will add a positive label to +the issue to denote the pull request has no CLA issues (e.g., a green +label stating, "CLA: ?"). If the contributor has not signed a CLA, +a negative label will be added to the pull request will be blocked +using GitHub's status API (e.g., a red label stating, "CLA: ?"). If a +contributor lacks a bugs.python.org account, that will lead to +another label (e.g., "CLA: ? (no account)"). Using a label for both +positive and negative cases provides a fallback notification if the +bot happens to fail, preventing potential false-positives or +false-negatives. It also allows for an easy way to trigger the bot +again by simply removing a CLA-related label. + +Requirements for Web-Related Repositories +----------------------------------------- +Due to their use for generating webpages, the +devguide [#devguide-repo]_ and peps [#peps-repo]_ repositories need +their respective processes updated to pull from their new Git +repositories. + +The devguide repository might also need to be named +``python-devguide`` to make sure the repository is not ambiguous +when viewed in isolation from the +python organization [#github-python-org]_. + +Requirements for the cpython Repository +--------------------------------------- +Obviously the most active and important repository currently hosted +at hg.python.org [#h.p.o]_ is the cpython +repository [#cpython-repo]_. Because of its importance and high- +frequency use, it requires more tooling before being moved to GitHub +compared to the other repositories mentioned in this PEP. + +Document steps to commit a pull request +''''''''''''''''''''''''''''''''''''''' +During the process of choosing a new development workflow, it was +decided that a linear history is desired. This means that the +convenient "Merge" button in GitHub pull requests is undesireable, as +it creates a merge commit along with all of the contributor's +individual commits (this does not affect the other repositories where +the desire for a linear history doesn't exist). + +Luckily, Git [#git]_ does not require GitHub's workflow and so one can +be chosen which gives us a linear history by using Git's CLI. The +expectation is that all pull requests will be fast-forwarded and +rebased before being pushed to the master repository. This should +give proper attribution to the pull request author in the Git +history. + +A second set of recommended commands will also be written for +committing a contribution from a patch file uploaded to +bugs.python.org [#b.p.o]_. This will obviously help keep the linear +history, but it will need to be made to have attribution to the patch +author. + +The exact sequence of commands that will be given as guidelines to +core developers is an open issue: +`Git CLI commands for committing a pull request to cpython`_. + +Handling Misc/NEWS +'''''''''''''''''' +Traditionally the ``Misc/NEWS`` file [#news-file]_ has been problematic +for changes which spanned Python releases. Often times there will be +merge conflicts when committing a change between e.g., 3.5 and 3.6 +only in the ``Misc/NEWS`` file. It's so common, in fact, that the +example instructions in the devguide explicitly mention how to +resolve conflicts in the ``Misc/NEWS`` file +[#devguide-merge-across-branches]_. As part of our tool +modernization, working with the ``Misc/NEWS`` file will be +simplified. + +There are currently two competing approaches to solving the +``Misc/NEWS`` problem which are discussed in an open issue: +`How to handle the Misc/NEWS file`_. + +Handling Misc/ACKS +'''''''''''''''''' +Traditionally the ``Misc/ACKS`` file [#acks-file]_ has been managed +by hand. But thanks to Git supporting an ``author`` value as well as +a ``committer`` value per commit, authorship of a commit can be part +of the history of the code itself. + +As such, manual management of ``Misc/ACKS`` will become optional. A +script will be written that will collect all author and committer +names and merge them into ``Misc/ACKS`` with all of the names listed +prior to the move to Git. Running this script will become part of the +release process. + +Linking pull requests to issues +''''''''''''''''''''''''''''''' +Historically, external contributions were attached to an issue on +bugs.python.org [#b.p.o]_ thanks to the fact that all external +contributions were uploaded as a file. For changes committed by a +core developer who committed a change directly, the specifying of an +issue number in the commit message of the format ``Issue #`` at the +start of the message led to a comment being posted to the issue +linking to the commit. + +Linking a pull request to an issue +++++++++++++++++++++++++++++++++++ +An association between a pull request and an issue is needed to track +when a fix has been proposed. The association needs to be many-to-one +as there can take multiple pull requests to solve a single issue +(technically it should be a many-to-many association for when a +single fix solves multiple issues, but this is fairly rare and issues +can be merged into one using the ``Superceder`` field on the issue +tracker). + +Association between a pull request and an issue will be done based on +detecting the regular expression``[Ii]ssue #(?P\d+)``. If +this is specified in either the title or in the body of a message on +a pull request then connection will be made on +bugs.python.org [#b.p.o]_. A label will also be added to the pull +request to signify that the connection was made successfully. This +could lead to incorrect associations if the wrong issue or +referencing another issue was done, but these are rare occasions. + +Notify the issue if the pull request is committed ++++++++++++++++++++++++++++++++++++++++++++++++++ +Once a pull request is closed (merged or not), the issue should be +updated to reflect this fact. + +Update linking service for mapping commit IDs to URLs +''''''''''''''''''''''''''''''''''''''''''''''''''''' +Currently you can use https://hg.python.org/lookup/ with a revision +ID from either the Subversion or Mercurial copies of the +cpython repo [#cpython-repo]_ to get redirected to the URL for that +revision in the Mercurial repository. The URL rewriter will need to +be updated to redirect to the Git repository and to support the new +revision IDs created for the Git repository. + +Create https://git.python.org +''''''''''''''''''''''''''''' +Just as hg.python.org [#h.p.o]_ currently points to the Mercurial +repository for Python, git.python.org should do the equivalent for +the Git repository. + +Backup of pull request data +''''''''''''''''''''''''''' +Since GitHub [#github]_ is going to be used for code hosting and code +review, those two things need to be backed up. In the case of code +hosting, the backup is implicit as all non-shallow Git [#git]_ clones +contain the full history of the repository, hence there will be many +backups of the repository. + +The code review history does not have the same implicit backup +mechanism as the repository itself. That means a daily backup of code +review history should be done so that it is not lost in case of any +issues with GitHub. It also helps guarantee that a migration from +GitHub to some other code review system is feasible were GitHub to +disappear overnight. + +Change ``sys._mercurial`` +''''''''''''''''''''''''' +Once Python is no longer kept in Mercurial, the ``sys._mercurial`` +attribute will need to be removed. An equivalent ``sys._git`` +attribute will be needed to take its place. + +Optional, Planned Features +-------------------------- +Once the cpython repository [#cpython-repo]_ is migrated, all +repositories will have been moved to GitHub [#github]_ and the +development process should be on equal footing as before. But a key +reason for this migration is to improve the development process, +making it better than it has ever been. This section outlines some +plans on how to improve things. + +It should be mentioned that overall feature planning for +bugs.python.org [#b.p.o]_ -- which includes plans independent of this +migration -- are tracked on their own wiki page [#tracker-plans]_. + +Bot to handle pull request merging +'''''''''''''''''''''''''''''''''' +As stated in the section entitled +"`Document steps to commit a pull request`_", the desire is to +maintain a linear history for cpython. Unfortunately, +Github's [#github]_ web-based workflow does not support a linear +history. Because of this, a bot should be written to substitute for +GitHub's in-browser commit abilities. + +To start, the bot should accept commands to commit a pull request +against a list of branches. This allows for committing a pull request +that fixes a bug in multiple versions of Python. + +More advanced features such as a commit queue can come later. This +would linearly apply accepted pull requests and verify that the +commits did not interfere with each other by running the test suite +and backing out commits if the test run failed. To help facilitate +the speed of testing, all patches committed since the last test run +can be applied and run in a single test run as the optimistic +assumption is that the patches will work in tandem. + +Inspiration or basis of the bot could be taken from pre-existig bots +such as Homu [#homu]_ or Zuul [#zuul]_. + +The name given to this bot in order to give it commands is an open +issue: `Naming the commit bot`_. + +Continuous integration per pull request +''''''''''''''''''''''''''''''''''''''' +To help speed up pull request approvals, continuous integration +testing should be used. This helps mitigate the need for a core +developer to download a patch simply to run the test suite against +the patch. + +Which free CI service to use is an open issue: +`Choosing a CI service`_. + +Test coverage report +'''''''''''''''''''' +Getting an up-to-date test coverage report for Python's standard +library would be extremely beneficial as generating such a report can +take quite a while to produce. + +There are a couple pre-existing services that provide free test +coverage for open source projects. Which option is best is an open +issue: `Choosing a test coverage service`_. + +Notifying issues of pull request comments +''''''''''''''''''''''''''''''''''''''''' +The current development process does not include notifying an issue +on bugs.python.org [#b.p.o]_ when a review comment is left on +Rietveld [#rietveld]_. It would be nice to fix this so that people +can subscribe only to comments at bugs.python.org and not +GitHub [#github]_ and yet still know when something occurs on GitHub +in terms of review comments on relevant pull requests. Current +thinking is to post a comment to bugs.python.org to the relevant +issue when at least one review comment has been made over a certain +period of time (e.g., 15 or 30 minutes). This keeps the email volume +down for those that receive both GitHub and bugs.python.org email +notifications while still making sure that those only following +bugs.python.org know when there might be a review comment to address. + +Allow bugs.python.org to use GitHub as a login provider +''''''''''''''''''''''''''''''''''''''''''''''''''''''' +As of right now, bugs.python.org [#b.p.o]_ allows people to log in +using Google, Launchpad, or OpenID credentials. It would be good to +expand this to GitHub credentials. + +Web hooks for re-generating web content +''''''''''''''''''''''''''''''''''''''' +The content at https://docs.python.org/, +https://docs.python.org/devguide, and +https://www.python.org/dev/peps/ are all derived from files kept in +one of the repositories to be moved as part of this migration. As +such, it would be nice to set up appropriate webhooks to trigger +rebuilding the appropriate web content when the files they are based +on change instead of having to wait for, e.g., a cronjob to trigger. + +Link web content back to files that it is generated from +'''''''''''''''''''''''''''''''''''''''''''''''''''''''' +It would be helpful for people who find issues with any of the +documentation that is generated from a file to have a link on each +page which points back to the file on GitHub [#github]_ that stores +the content of the page. That would allow for quick pull requests to +fix simple things such as spelling mistakes. + +Splitting out parts of the documentation into their own repositories +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' +While certain parts of the documentation at https://docs.python.org +change with the code, other parts are fairly static and are not +tightly bound to the CPython code itself. The following sections of +the documentation fit this category of slow-changing, +loosely-coupled: + +* `Tutorial `__ +* `Python Setup and Usage `__ +* `HOWTOs `__ +* `Installing Python Modules `__ +* `Distributing Python Modules `__ +* `Extending and Embedding `__ +* `FAQs `__ + +These parts of the documentation could be broken out into their own +repositories to simplify their maintenance and to expand who has +commit rights to them to ease in their maintenance. + +Status +====== +XXX list all sections (w/ backlinks) stating the status + +Open Issues +=========== +For this PEP, open issues are ones where a decision needs to be made +to how to approach or solve a problem. Open issues do not entail +coordination issues such as who is going to write a certain bit of +code. + +The fate of hg.python.org +------------------------- +With the code repositories moving over to Git [#git]_, there is no +technical need to keep hg.python.org [#h.p.o]_ running. Having said +that, some in the community would like to have it stay functioning as +a Mercurial [#hg]_ mirror of the Git repositories. Others have said +that they still want a mirror, but one using Git. + +As maintaining hg.python.org is not necessary, it will be up to the +PSF infrastructure committee to decide if they want to spend the +time and resources to keep it running. They may also choose whether +they want to host a Git mirror on PSF infrastructure. + +Tools and commands to move from Mercurial to Git +------------------------------------------------ +A decision needs to be made on exactly what tooling and what commands +involving those tools will be used to convert a Mercurial repository +to Git. Currently a suggestion has been made to use +https://github.com/frej/fast-export. Another suggestion is to use +https://github.com/felipec/git-remote-hg. Finally, +http://hg-git.github.io/ has been suggested. + +Git CLI commands for committing a pull request to cpython +--------------------------------------------------------- +Because Git [#git]_ may be a new version control system for core +developers, the commands people are expected to run will need to be +written down. These commands also need to keep a linear history while +giving proper attribution to the pull request author. + +Another set of commands will also be necessary for when working with +a patch file uploaded to bugs.python.org [#b.p.o]_. Here the linear +history will be kept implicitly, but it will need to make sure to +keep/add attribution. + +How to handle the Misc/NEWS file +-------------------------------- +There are two competing approaches to handling +``Misc/NEWS`` [#news-file]_. One is to add a news entry for issues on +bugs.python.org [#b.p.o]_. This would mean an issue that is marked +as "resolved" could not be closed until a news entry is added in the +"news" field in the issue tracker. The benefit of tying the news +entry to the issue is it makes sure that all changes worthy of a news +entry have an accompanying issue. It also makes classifying a news +entry automatic thanks to the Component field of the issue. The +Versions field of the issue also ties the news entry to which Python +releases were affected. A script would be written to query +bugs.python.org for relevant new entries for a release and to produce +the output needed to be checked into the code repository. This +approach is agnostic to whether a commit was done by CLI or bot. + +The competing approach is to use an individual file per news entry, +containg the text for the entry. In this scenario each feature +release would have its own directory for news entries and a separate +file would be created in that directory that was either named after +the issue it closed or a timestamp value (which prevents collisions). +Merges across branches would have no issue as the news entry file +would still be uniqeuely named and in the directory of the latest +version that contained the fix. A script would collect all news entry +files no matter what directory they reside in and create an +appropriate news file (the release directory can be ignored as the +mere fact that the file exists is enough to represent that the entry +belongs to the release). Classification can either be done by keyword +in the new entry file itself or by using subdirectories representing +each news entry classification in each release directory (or +classification of news entries could be dropped since critical +information is captured by the "What's New" documents which are +organized). The benefit of this approach is that it keeps the changes +with the code that was actually changed. It also ties the message to +being part of the commit which introduced the change. For a commit +made through the CLI, a script will be provided to help generate the +file. In a bot-driven scenario, the merge bot will have a way to +specify a specific news entry and create the file as part of its +flattened commit (while most likely also supporting using the first +line of the commit message if no specific news entry was specified). +Code for this approach has been written previously for the Mercurial +workflow at http://bugs.python.org/issue18967. There is also a tool +from the community at https://pypi.python.org/pypi/towncrier. + +Naming the commit bot +--------------------- +As naming things can lead to bikeshedding of epic proportions, Brett +Cannon will choose the final name of the commit bot (the name of the +project for the bot itself can be anything, this is purely for the +name used in giving commands to the bot). The name will come from +Monty Python, which is only fitting since Python is named after the +comedy troupe. It will most likely come from +'Monty Python and the Holy Grail' [#holy-grail]_ (which happens to be +how Brett was introduced to Monty Python). Current ideas on the name +include: + +"Black Knight" sketch [#black-knight-sketch]_: + +* black-knight +* none-shall-pass +* just-a-flesh-wound + +"Bridge of Death" sketch [#bridge-of-death-sketch]_: + +* bridge-keeper +* man-from-scene-24 +* five-questions +* what-is-your-quest +* blue-no-green +* air-speed-velocity +* your-favourite-colour + (and that specific spelling; Monty Python is British, after all) + +"Killer rabbit" sketch [#killer-rabbit-sketch]_: + +* killer-rabbit +* holy-hand-grenade +* 5-is-right-out + +"Witch Village" sketch [#witch-village-sketch]_: + +* made-of-wood +* burn-her + +"French Taunter" sketch [#french-taunter-sketch]_: + +* elderberries +* kanigget + +"Constitutional Peasants" sketch [#constitutional-peasants-sketch]_: + +* dennis +* from-the-masses +* watery-tart + +"Knights Who Say Ni" sketch [#ni-sketch]_: + +* shubbery +* ni + +From "Monty Python and the Holy Grail" in general: + +* brave-sir-robin + +Choosing a CI service +--------------------- +There are various CI services that provide free support for open +source projects hosted on GitHub [#github]_. Two such examples are +Travis [#travis]_ and Codeship [#codeship]_. Whatever solution is +chosen will need to not time out in the time it takes to execute +Python's test suite. It should optimally provide access to multiple C +compilers for more thorough testing. Network access is also +beneficial. + +The current CI service for Python is Pypatcher [#pypatcher]_. A +request can be made in IRC to try a patch from +bugs.python.org [#b.p.o]_. The results can be viewed at +https://ci.centos.org/job/cPython-build-patch/ . + +Choosing a test coverage service +-------------------------------- +Getting basic test coverage of Python's standard library can be +created simply by using coverage.py [#coverage]_. Getting +thorough test coverage is actually quite tricky, with the details +outlined in the devinabox's README [#devinabox-repo]_. It would be +best if a service could be found that would allow for thorough test +coverage, but it might not be feasible. + +Free test coverage services include Coveralls [#coveralls]_ and +Codecov [#codecov]_. + +Rejected Ideas +============== +Separate Python 2 and Python 3 repositories +------------------------------------------- +It was discussed whether separate repositories for Python 2 and +Python 3 were desired. The thinking was that this would shrink the +overall repository size which benefits people with slow Internet +connections or small bandwidth caps. + +In the end it was decided that it was easier logistically to simply +keep all of CPython's history in a single repository. + +Commit multi-release changes in bugfix branch first +--------------------------------------------------- +As the current development process has changes committed in the +oldest branch first and then merged up to the default branch, the +question came up as to whether this workflow should be perpetuated. +In the end it was decided that committing in the newest branch and +then cherrypicking changes into older branches would work best as +most people will instinctively work off the newest branch and it is a +more common workflow when using Git [#git]_. + +Deriving ``Misc/NEWS`` from the commit logs +------------------------------------------- +As part of the discussion surrounding `Handling Misc/NEWS`_, the +suggestion has come up of deriving the file from the commit logs +itself. In this scenario, the first line of a commit message would be +taken to represent the news entry for the change. Some heuristic to +tie in whether a change warranted a news entry would be used, e.g., +whether an issue number is listed. + +This idea has been rejected due to some core developers preferring to +write a news entry separate from the commit message. The argument is +the first line of a commit message compared to that of a news entry +have different requirements in terms of brevity, what should be said, +etc. + +References +========== +.. [#h.p.o] https://hg.python.org + +.. [#GitHub] GitHub (https://github.com) + +.. [#hg] Mercurial (https://www.mercurial-scm.org/) + +.. [#git] Git (https://git-scm.com/) + +.. [#b.p.o] https://bugs.python.org + +.. [#rietveld] Rietveld (https://github.com/rietveld-codereview/rietveld) + +.. [#gitlab] GitLab (https://about.gitlab.com/) + +.. [#core-workflow] core-workflow mailing list (https://mail.python.org/mailman/listinfo/core-workflow) + +.. [#guido-keynote] Guido van Rossum's keynote at PyCon US (https://www.youtube.com/watch?v=G-uKNd5TSBw) + +.. [#reasons] Email to core-workflow outlining reasons why GitHub was selected + (https://mail.python.org/pipermail/core-workflow/2016-January/000345.html) + +.. [#benchmarks-repo] Mercurial repository for the Unified Benchmark Suite + (https://hg.python.org/benchmarks/) + +.. [#devinabox-repo] Mercurial repository for devinabox (https://hg.python.org/devinabox/) + +.. [#peps-repo] Mercurial repository of the Python Enhancement Proposals (https://hg.python.org/peps/) + +.. [#tracker-repo] bugs.python.org code repository (https://hg.python.org/tracker/python-dev/) + +.. [#devguide-repo] Mercurial repository for the Python Developer's Guide (https://hg.python.org/devguide/) + +.. [#cpython-repo] Mercurial repository for CPython (https://hg.python.org/cpython/) + +.. [#github-python-org] Python organization on GitHub (https://github.com/python) + +.. [#github-org-perms] GitHub repository permission levels + (https://help.github.com/enterprise/2.4/user/articles/repository-permission-levels-for-an-organization/) + +.. [#cla] Python Software Foundation Contributor Agreement (https://www.python.org/psf/contrib/contrib-form/) + +.. [#news-file] ``Misc/NEWS`` (https://hg.python.org/cpython/file/default/Misc/NEWS) + +.. [#acks-file] ``Misc/ACKS`` (https://hg.python.org/cpython/file/default/Misc/ACKS) + +.. [#devguide-merge-across-branches] Devguide instructions on how to merge across branches + (https://docs.python.org/devguide/committing.html#merging-between-different-branches-within-the-same-major-version) + +.. [#pythocat] Pythocat (https://octodex.github.com/pythocat/) + +.. [#tracker-plans] Wiki page for bugs.python.org feature development + (https://wiki.python.org/moin/TrackerDevelopmentPlanning) + +.. [#black-knight-sketch] The "Black Knight" sketch from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=dhRUe-gz690) + +.. [#bridge-of-death-sketch] The "Bridge of Death" sketch from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=cV0tCphFMr8) + +.. [#holy-grail] "Monty Python and the Holy Grail" sketches + (https://www.youtube.com/playlist?list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp) + +.. [#killer-rabbit-sketch] "Killer rabbit" sketch from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=Nvs5pqf-DMA&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=11) + +.. [#witch-village-sketch] "Witch Village" from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=k3jt5ibfRzw&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=12) + +.. [#french-taunter-sketch] "French Taunter" from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=A8yjNbcKkNY&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=13) + +.. [#constitutional-peasants-sketch] "Constitutional Peasants" from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=JvKIWjnEPNY&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=14) + +.. [#ni-sketch] "Knights Who Say Ni" from "Monty Python and the Holy Grail" + (https://www.youtube.com/watch?v=zIV4poUZAQo&list=PL-Qryc-SVnnu1MvN3r94Y9atpaRuIoGmp&index=15) + +.. [#homu] Homu (http://homu.io/) + +.. [#zuul] Zuul (http://docs.openstack.org/infra/zuul/) + +.. [#travis] Travis (https://travis-ci.org/) + +.. [#codeship] Codeship (https://codeship.com/) + +.. [#coverage] coverage.py (https://pypi.python.org/pypi/coverage) + +.. [#coveralls] Coveralls (https://coveralls.io/) + +.. [#codecov] Codecov (https://codecov.io/) + +.. [#pypatcher] Pypatcher (https://github.com/kushaldas/pypatcher) + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 17 15:43:21 2016 From: python-checkins at python.org (brett.cannon) Date: Sun, 17 Jan 2016 20:43:21 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_a_Status_section_to_PEP_5?= =?utf-8?q?12?= Message-ID: <20160117204321.14183.78538@psf.io> https://hg.python.org/peps/rev/6f55514cf66c changeset: 6194:6f55514cf66c user: Brett Cannon date: Sun Jan 17 12:43:18 2016 -0800 summary: Add a Status section to PEP 512 files: pep-0512.txt | 81 ++++++++++++++++++++++++++++++++++++++- 1 files changed, 78 insertions(+), 3 deletions(-) diff --git a/pep-0512.txt b/pep-0512.txt --- a/pep-0512.txt +++ b/pep-0512.txt @@ -332,8 +332,8 @@ GitHub to some other code review system is feasible were GitHub to disappear overnight. -Change ``sys._mercurial`` -''''''''''''''''''''''''' +Change sys._mercurial +''''''''''''''''''''' Once Python is no longer kept in Mercurial, the ``sys._mercurial`` attribute will need to be removed. An equivalent ``sys._git`` attribute will be needed to take its place. @@ -459,7 +459,82 @@ Status ====== -XXX list all sections (w/ backlinks) stating the status +Requirements for migrating the devinabox [#devinabox-repo]_, +benchmarks [#benchmarks-repo]_, and tracker [#tracker-repo]_ +repositories: + +* Not started + + - `Create a 'python-dev' team`_ + - `Define commands to move a Mercurial repository to Git`_ + - `Adding GitHub username support to bugs.python.org`_ + - `A bot to enforce CLA signing`_ + +* In progress + + - None + +* Completed + + - None + + Repositories whose build steps need updating: + + * Not started + + - peps [#peps-repo]_ + - devguide [#devguide-repo]_ + + * In progress + + - None + + * Completed + + - None + + Requirements to move over the cpython repo [#cpython-repo]_: + + * Not started + + - `Document steps to commit a pull request`_ + - `Handling Misc/NEWS`_ + - `Handling Misc/ACKS`_ + - `Linking a pull request to an issue`_ + - `Notify the issue if the pull request is committed`_ + - `Update linking service for mapping commit IDs to URLs`_ + - `Create https://git.python.org`_ + - `Backup of pull request data`_ + - `Change sys._mercurial`_ + + * In progress + + - None + + * Completed + + - None + + Optional features: + + * Not started + + - `Bot to handle pull request merging`_ + - `Continuous integration per pull request`_ + - `Test coverage report`_ + - `Notifying issues of pull request comments`_ + - `Allow bugs.python.org to use GitHub as a login provider`_ + - `Web hooks for re-generating web content`_ + - `Link web content back to files that it is generated from`_ + - `Splitting out parts of the documentation into their own repositories`_ + + * In progress + + - None + + * Completed + + - None Open Issues =========== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 17 18:12:45 2016 From: python-checkins at python.org (brett.cannon) Date: Sun, 17 Jan 2016 23:12:45 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Remove_a_bot_name_from_PEP_51?= =?utf-8?q?2_that_might_be_taken_as_sexist?= Message-ID: <20160117231244.121306.54988@psf.io> https://hg.python.org/peps/rev/d0a09f54ad0d changeset: 6195:d0a09f54ad0d user: Brett Cannon date: Sun Jan 17 15:12:36 2016 -0800 summary: Remove a bot name from PEP 512 that might be taken as sexist files: pep-0512.txt | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/pep-0512.txt b/pep-0512.txt --- a/pep-0512.txt +++ b/pep-0512.txt @@ -670,7 +670,6 @@ * dennis * from-the-masses -* watery-tart "Knights Who Say Ni" sketch [#ni-sketch]_: -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Sun Jan 17 20:05:19 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 18 Jan 2016 01:05:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue26142_-_F?= =?utf-8?q?ix_a_formatting_bug_in_c-api_intro_doc=2E?= Message-ID: <20160118010519.25967.49402@psf.io> https://hg.python.org/cpython/rev/4f1ed606410b changeset: 99950:4f1ed606410b branch: 2.7 parent: 99946:21356a5b8a5e user: Senthil Kumaran date: Sun Jan 17 17:05:14 2016 -0800 summary: Issue26142 - Fix a formatting bug in c-api intro doc. files: Doc/c-api/intro.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -258,7 +258,7 @@ if (PyObject_SetItem(target, index, item) < 0) { Py_DECREF(index); return -1; - } + } Py_DECREF(index); } return 0; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 17 21:42:57 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 18 Jan 2016 02:42:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160118024256.94200.60908@psf.io> https://hg.python.org/cpython/rev/18ae23b69e08 changeset: 99952:18ae23b69e08 parent: 99949:df78978dacab parent: 99951:31debd0dc0dc user: Senthil Kumaran date: Sun Jan 17 18:42:50 2016 -0800 summary: Merge from 3.5 Issue26017 - Suggest enclosing command args in double quotes when using characters which get interpreted by shell. files: Doc/installing/index.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst --- a/Doc/installing/index.rst +++ b/Doc/installing/index.rst @@ -84,10 +84,12 @@ Python. It's also possible to specify an exact or minimum version directly on the -command line:: +command line. When using comparator operators such as ``>``, ``<`` or some other +special character which get interpreted by shell, the package name and the +version should be enclosed within double quotes:: python -m pip install SomePackage==1.0.4 # specific version - python -m pip install 'SomePackage>=1.0.4' # minimum version + python -m pip install "SomePackage>=1.0.4" # minimum version Normally, if a suitable module is already installed, attempting to install it again will have no effect. Upgrading existing modules must be requested -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 17 21:42:56 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 18 Jan 2016 02:42:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Issue26017_-_S?= =?utf-8?q?uggest_enclosing_command_args_in_double_quotes_when_using?= Message-ID: <20160118024256.111036.93298@psf.io> https://hg.python.org/cpython/rev/31debd0dc0dc changeset: 99951:31debd0dc0dc branch: 3.5 parent: 99947:59852a79b508 user: Senthil Kumaran date: Sun Jan 17 18:42:13 2016 -0800 summary: Issue26017 - Suggest enclosing command args in double quotes when using characters which get interpreted by shell. files: Doc/installing/index.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst --- a/Doc/installing/index.rst +++ b/Doc/installing/index.rst @@ -84,10 +84,12 @@ Python. It's also possible to specify an exact or minimum version directly on the -command line:: +command line. When using comparator operators such as ``>``, ``<`` or some other +special character which get interpreted by shell, the package name and the +version should be enclosed within double quotes:: python -m pip install SomePackage==1.0.4 # specific version - python -m pip install 'SomePackage>=1.0.4' # minimum version + python -m pip install "SomePackage>=1.0.4" # minimum version Normally, if a suitable module is already installed, attempting to install it again will have no effect. Upgrading existing modules must be requested -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 17 21:45:15 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 18 Jan 2016 02:45:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue26017_-_S?= =?utf-8?q?uggest_enclosing_command_args_in_double_quotes_when_using?= Message-ID: <20160118024515.25973.46826@psf.io> https://hg.python.org/cpython/rev/668827be66d6 changeset: 99953:668827be66d6 branch: 2.7 parent: 99950:4f1ed606410b user: Senthil Kumaran date: Sun Jan 17 18:45:11 2016 -0800 summary: Issue26017 - Suggest enclosing command args in double quotes when using characters which get interpreted by shell. files: Doc/installing/index.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst --- a/Doc/installing/index.rst +++ b/Doc/installing/index.rst @@ -82,10 +82,12 @@ Python. It's also possible to specify an exact or minimum version directly on the -command line:: +command line. When using comparator operators such as ``>``, ``<`` or some other +special character which get interpreted by shell, the package name and the +version should be enclosed within double quotes:: python -m pip install SomePackage==1.0.4 # specific version - python -m pip install 'SomePackage>=1.0.4' # minimum version + python -m pip install "SomePackage>=1.0.4" # minimum version Normally, if a suitable module is already installed, attempting to install it again will have no effect. Upgrading existing modules must be requested -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Jan 17 23:12:22 2016 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 18 Jan 2016 04:12:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue26069_-_Update_whatsn?= =?utf-8?q?ew/3=2E6=2Erst_on_traceback_module=27s_api_removals=2E?= Message-ID: <20160118041222.96364.69297@psf.io> https://hg.python.org/cpython/rev/715d8a739f3b changeset: 99954:715d8a739f3b parent: 99952:18ae23b69e08 user: Senthil Kumaran date: Sun Jan 17 20:12:16 2016 -0800 summary: Issue26069 - Update whatsnew/3.6.rst on traceback module's api removals. files: Doc/whatsnew/3.6.rst | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -227,6 +227,13 @@ :func:`inspect.getmodulename` should be used for obtaining the module name for a given path. +* ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``, + ``traceback.fullmodname``, ``traceback.find_lines_from_code``, + ``traceback.find_lines``, ``traceback.find_strings``, + ``traceback.find_executable_lines`` methods were removed from the + :mod:`traceback` module. They were undocumented methods deprecated since + Python 3.2 and equivalent functionality is available from private methods. + Porting to Python 3.6 ===================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 01:56:50 2016 From: python-checkins at python.org (georg.brandl) Date: Mon, 18 Jan 2016 06:56:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_two_instances_of_wrong?= =?utf-8?q?_indentation=2E?= Message-ID: <20160118065650.14183.98526@psf.io> https://hg.python.org/cpython/rev/1ececa34b748 changeset: 99955:1ececa34b748 user: Georg Brandl date: Mon Jan 18 07:53:59 2016 +0100 summary: Fix two instances of wrong indentation. files: Python/ast.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -3214,14 +3214,14 @@ alias_ty import_alias = alias_for_import_name(c, n, 1); if (!import_alias) return NULL; - asdl_seq_SET(aliases, 0, import_alias); + asdl_seq_SET(aliases, 0, import_alias); } else { for (i = 0; i < NCH(n); i += 2) { alias_ty import_alias = alias_for_import_name(c, CHILD(n, i), 1); if (!import_alias) return NULL; - asdl_seq_SET(aliases, i / 2, import_alias); + asdl_seq_SET(aliases, i / 2, import_alias); } } if (mod != NULL) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 02:00:25 2016 From: python-checkins at python.org (georg.brandl) Date: Mon, 18 Jan 2016 07:00:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_indentation_of_continu?= =?utf-8?q?ation_lines=2E?= Message-ID: <20160118070025.25959.6880@psf.io> https://hg.python.org/cpython/rev/bc625fce0aee changeset: 99956:bc625fce0aee user: Georg Brandl date: Mon Jan 18 08:00:15 2016 +0100 summary: Fix indentation of continuation lines. files: Modules/_testcapimodule.c | 4 ++-- Python/pylifecycle.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -1047,7 +1047,7 @@ value = PyLong_AsUnsignedLongMask(num); if (value != ULONG_MAX) return raiseTestError("test_k_code", - "PyLong_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF"); + "PyLong_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF"); PyTuple_SET_ITEM(tuple, 0, num); @@ -1066,7 +1066,7 @@ value = PyLong_AsUnsignedLongMask(num); if (value != (unsigned long)-0x42) return raiseTestError("test_k_code", - "PyLong_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF"); + "PyLong_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF"); PyTuple_SET_ITEM(tuple, 0, num); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -796,7 +796,7 @@ if (initstdio() < 0) Py_FatalError( - "Py_Initialize: can't initialize sys standard streams"); + "Py_Initialize: can't initialize sys standard streams"); initmain(interp); if (!Py_NoSiteFlag) initsite(); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 03:40:47 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 08:40:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzY1MDA6?= =?utf-8?q?_Fixed_infinite_recursion_in_urllib2=2ERequest=2E=5F=5Fgetattr?= =?utf-8?b?X18oKS4=?= Message-ID: <20160118084047.94196.18064@psf.io> https://hg.python.org/cpython/rev/d34fdd1736f2 changeset: 99957:d34fdd1736f2 branch: 2.7 parent: 99953:668827be66d6 user: Serhiy Storchaka date: Mon Jan 18 10:35:40 2016 +0200 summary: Issue #6500: Fixed infinite recursion in urllib2.Request.__getattr__(). files: Lib/test/test_urllib2.py | 5 +++++ Lib/urllib2.py | 8 +++----- Misc/NEWS | 2 ++ 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1350,6 +1350,11 @@ req = Request(url) self.assertEqual(req.get_full_url(), url) + def test_private_attributes(self): + self.assertFalse(hasattr(self.get, '_Request__r_xxx')) + # Issue #6500: infinite recursion + self.assertFalse(hasattr(self.get, '_Request__r_method')) + def test_HTTPError_interface(self): """ Issue 13211 reveals that HTTPError didn't implement the URLError diff --git a/Lib/urllib2.py b/Lib/urllib2.py --- a/Lib/urllib2.py +++ b/Lib/urllib2.py @@ -248,11 +248,9 @@ # methods getting called in a non-standard order. this may be # too complicated and/or unnecessary. # XXX should the __r_XXX attributes be public? - if attr[:12] == '_Request__r_': - name = attr[12:] - if hasattr(Request, 'get_' + name): - getattr(self, 'get_' + name)() - return getattr(self, attr) + if attr in ('_Request__r_type', '_Request__r_host'): + getattr(self, 'get_' + attr[12:])() + return self.__dict__[attr] raise AttributeError, attr def get_method(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,8 @@ Library ------- +- Issue #6500: Fixed infinite recursion in urllib2.Request.__getattr__(). + - Issue #26083: Workaround a subprocess bug that raises an incorrect "ValueError: insecure string pickle" exception instead of the actual exception on some platforms such as Mac OS X when an exception raised -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Mon Jan 18 03:44:08 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 18 Jan 2016 08:44:08 +0000 Subject: [Python-checkins] Daily reference leaks (715d8a739f3b): sum=-41 Message-ID: <20160118084408.96376.84749@psf.io> results for 715d8a739f3b on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, -34] references, sum=-34 test_asyncio leaked [0, 0, -11] memory blocks, sum=-11 test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogX_dgCl', '--timeout', '7200'] From python-checkins at python.org Mon Jan 18 05:26:52 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 18 Jan 2016 10:26:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_test=5Fcompilepath=28?= =?utf-8?q?=29_of_test=5Fcompileall?= Message-ID: <20160118102652.18413.70558@psf.io> https://hg.python.org/cpython/rev/c6a0f424837a changeset: 99958:c6a0f424837a parent: 99956:bc625fce0aee user: Victor Stinner date: Mon Jan 18 11:25:50 2016 +0100 summary: Fix test_compilepath() of test_compileall Issue #26101: Exclude Lib/test/ from sys.path in test_compilepath(). The directory contains invalid Python files like Lib/test/badsyntax_pep3120.py, whereas the test ensures that all files can be compiled. files: Lib/test/test_compileall.py | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_compileall.py b/Lib/test/test_compileall.py --- a/Lib/test/test_compileall.py +++ b/Lib/test/test_compileall.py @@ -103,6 +103,18 @@ force=False, quiet=2)) def test_compile_path(self): + # Exclude Lib/test/ which contains invalid Python files like + # Lib/test/badsyntax_pep3120.py + testdir = os.path.realpath(os.path.dirname(__file__)) + if testdir in sys.path: + self.addCleanup(setattr, sys, 'path', sys.path) + + sys.path = list(sys.path) + try: + sys.path.remove(testdir) + except ValueError: + pass + self.assertTrue(compileall.compile_path(quiet=2)) with test.test_importlib.util.import_state(path=[self.directory]): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 06:16:00 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 18 Jan 2016 11:16:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_subprocess=2E=5Foptim=5Far?= =?utf-8?b?Z3NfZnJvbV9pbnRlcnByZXRlcl9mbGFncygp?= Message-ID: <20160118111559.94196.45244@psf.io> https://hg.python.org/cpython/rev/ac6d33317eda changeset: 99959:ac6d33317eda user: Victor Stinner date: Mon Jan 18 12:15:08 2016 +0100 summary: subprocess._optim_args_from_interpreter_flags() Issue #26100: * Add subprocess._optim_args_from_interpreter_flags() * Add test.support.optim_args_from_interpreter_flags() * Use new functions in distutils, test_cmd_line_script, test_compileall and test_inspect The change enables test_details() test of test_inspect when -O or -OO command line option is used. files: Lib/distutils/util.py | 15 +++++++++------ Lib/subprocess.py | 14 ++++++++++++-- Lib/test/support/__init__.py | 5 +++++ Lib/test/test_cmd_line_script.py | 5 ++--- Lib/test/test_compileall.py | 7 +++---- Lib/test/test_inspect.py | 7 ++++--- 6 files changed, 35 insertions(+), 18 deletions(-) diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py --- a/Lib/distutils/util.py +++ b/Lib/distutils/util.py @@ -7,8 +7,8 @@ import os import re import importlib.util +import string import sys -import string from distutils.errors import DistutilsPlatformError from distutils.dep_util import newer from distutils.spawn import spawn @@ -350,6 +350,11 @@ generated in indirect mode; unless you know what you're doing, leave it set to None. """ + + # Late import to fix a bootstrap issue: _posixsubprocess is built by + # setup.py, but setup.py uses distutils. + import subprocess + # nothing is done if sys.dont_write_bytecode is True if sys.dont_write_bytecode: raise DistutilsByteCompileError('byte-compiling is disabled.') @@ -412,11 +417,9 @@ script.close() - cmd = [sys.executable, script_name] - if optimize == 1: - cmd.insert(1, "-O") - elif optimize == 2: - cmd.insert(1, "-OO") + cmd = [sys.executable] + cmd.extend(subprocess._optim_args_from_interpreter_flags()) + cmd.append(script_name) spawn(cmd, dry_run=dry_run) execute(os.remove, (script_name,), "removing %s" % script_name, dry_run=dry_run) diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -520,6 +520,16 @@ # but it's here so that it can be imported when Python is compiled without # threads. +def _optim_args_from_interpreter_flags(): + """Return a list of command-line arguments reproducing the current + optimization settings in sys.flags.""" + args = [] + value = sys.flags.optimize + if value > 0: + args.append('-' + 'O' * value) + return args + + def _args_from_interpreter_flags(): """Return a list of command-line arguments reproducing the current settings in sys.flags and sys.warnoptions.""" @@ -527,7 +537,6 @@ 'debug': 'd', # 'inspect': 'i', # 'interactive': 'i', - 'optimize': 'O', 'dont_write_bytecode': 'B', 'no_user_site': 's', 'no_site': 'S', @@ -535,8 +544,9 @@ 'verbose': 'v', 'bytes_warning': 'b', 'quiet': 'q', + # -O is handled in _optim_args_from_interpreter_flags() } - args = [] + args = _optim_args_from_interpreter_flags() for flag, opt in flag_opt_map.items(): v = getattr(sys.flags, flag) if v > 0: diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -2053,6 +2053,11 @@ settings in sys.flags and sys.warnoptions.""" return subprocess._args_from_interpreter_flags() +def optim_args_from_interpreter_flags(): + """Return a list of command-line arguments reproducing the current + optimization settings in sys.flags.""" + return subprocess._optim_args_from_interpreter_flags() + #============================================================ # Support for assertions about logging. #============================================================ diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -138,9 +138,8 @@ expected_argv0, expected_path0, expected_package, expected_loader, *cmd_line_switches): - if not __debug__: - cmd_line_switches += ('-' + 'O' * sys.flags.optimize,) - run_args = cmd_line_switches + (script_name,) + tuple(example_args) + run_args = [*support.optim_args_from_interpreter_flags(), + *cmd_line_switches, script_name, *example_args] rc, out, err = assert_python_ok(*run_args, __isolated=False) self._check_output(script_name, rc, out + err, expected_file, expected_argv0, expected_path0, diff --git a/Lib/test/test_compileall.py b/Lib/test/test_compileall.py --- a/Lib/test/test_compileall.py +++ b/Lib/test/test_compileall.py @@ -231,10 +231,9 @@ raise unittest.SkipTest('not all entries on sys.path are writable') def _get_run_args(self, args): - interp_args = ['-S'] - if sys.flags.optimize: - interp_args.append({1 : '-O', 2 : '-OO'}[sys.flags.optimize]) - return interp_args + ['-m', 'compileall'] + list(args) + return [*support.optim_args_from_interpreter_flags(), + '-S', '-m', 'compileall', + *args] def assertRunOK(self, *args, **env_vars): rc, out, err = script_helper.assert_python_ok( diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -30,6 +30,7 @@ from test.support.script_helper import assert_python_ok, assert_python_failure from test import inspect_fodder as mod from test import inspect_fodder2 as mod2 +from test import support from test.test_import import _ready_to_import @@ -3536,14 +3537,14 @@ def test_details(self): module = importlib.import_module('unittest') - rc, out, err = assert_python_ok('-m', 'inspect', + args = support.optim_args_from_interpreter_flags() + rc, out, err = assert_python_ok(*args, '-m', 'inspect', 'unittest', '--details') output = out.decode() # Just a quick sanity check on the output self.assertIn(module.__name__, output) self.assertIn(module.__file__, output) - if not sys.flags.optimize: - self.assertIn(module.__cached__, output) + self.assertIn(module.__cached__, output) self.assertEqual(err, b'') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 11:51:17 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 16:51:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326129=3A_Deprecat?= =?utf-8?q?ed_accepting_non-integers_in_grp=2Egetgrgid=28=29=2E?= Message-ID: <20160118165117.18419.36612@psf.io> https://hg.python.org/cpython/rev/65e0e06b70b6 changeset: 99960:65e0e06b70b6 user: Serhiy Storchaka date: Mon Jan 18 18:49:57 2016 +0200 summary: Issue #26129: Deprecated accepting non-integers in grp.getgrgid(). files: Doc/library/grp.rst | 3 +++ Lib/test/test_grp.py | 10 ++++++++++ Misc/NEWS | 2 ++ Modules/grpmodule.c | 21 ++++++++++++++++----- 4 files changed, 31 insertions(+), 5 deletions(-) diff --git a/Doc/library/grp.rst b/Doc/library/grp.rst --- a/Doc/library/grp.rst +++ b/Doc/library/grp.rst @@ -42,6 +42,9 @@ Return the group database entry for the given numeric group ID. :exc:`KeyError` is raised if the entry asked for cannot be found. + .. deprecated:: 3.6 + Since Python 3.6 the support of non-integer arguments like floats or + strings in :func:`getgrgid` is deprecated. .. function:: getgrnam(name) diff --git a/Lib/test/test_grp.py b/Lib/test/test_grp.py --- a/Lib/test/test_grp.py +++ b/Lib/test/test_grp.py @@ -92,5 +92,15 @@ self.assertRaises(KeyError, grp.getgrgid, fakegid) + def test_noninteger_gid(self): + entries = grp.getgrall() + if not entries: + self.skipTest('no groups') + # Choose an existent gid. + gid = entries[0][2] + self.assertWarns(DeprecationWarning, grp.getgrgid, float(gid)) + self.assertWarns(DeprecationWarning, grp.getgrgid, str(gid)) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,8 @@ Library ------- +- Issue #26129: Deprecated accepting non-integers in grp.getgrgid(). + - Issue #25850: Use cross-compilation by default for 64-bit Windows. - Issue #25822: Add docstrings to the fields of urllib.parse results. diff --git a/Modules/grpmodule.c b/Modules/grpmodule.c --- a/Modules/grpmodule.c +++ b/Modules/grpmodule.c @@ -100,14 +100,25 @@ gid_t gid; struct group *p; - py_int_id = PyNumber_Long(id); - if (!py_int_id) + if (!_Py_Gid_Converter(id, &gid)) { + if (!PyErr_ExceptionMatches(PyExc_TypeError)) { return NULL; - if (!_Py_Gid_Converter(py_int_id, &gid)) { + } + PyErr_Clear(); + if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, + "group id must be int, not %.200", + id->ob_type->tp_name) < 0) { + return NULL; + } + py_int_id = PyNumber_Long(id); + if (!py_int_id) + return NULL; + if (!_Py_Gid_Converter(py_int_id, &gid)) { + Py_DECREF(py_int_id); + return NULL; + } Py_DECREF(py_int_id); - return NULL; } - Py_DECREF(py_int_id); if ((p = getgrgid(gid)) == NULL) { PyObject *gid_obj = _PyLong_FromGid(gid); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Jan 18 12:04:04 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 18 Jan 2016 17:04:04 +0000 Subject: [Python-checkins] BAD Benchmark Results for Python Default 2016-01-18 Message-ID: <9f507c34-8df4-424f-8b70-3ec156790411@irsmsx103.ger.corp.intel.com> Results for project Python default, build date 2016-01-18 07:58:03 +0000 commit: bc625fce0aee previous commit: 32a4e7b337c9 revision date: 2016-01-18 07:00:15 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.42% -1.32% 10.41% 13.11% :-| pybench 0.14% 0.09% -1.70% 6.18% :-( regex_v8 2.72% 0.33% -4.57% 4.04% :-| nbody 0.08% -0.00% 1.10% 6.19% :-| json_dump_v2 0.20% 0.16% -1.54% 11.45% :-| normal_startup 0.90% 0.20% -0.22% 4.83% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-01-18/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Mon Jan 18 12:05:33 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 18 Jan 2016 17:05:33 +0000 Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-01-18 Message-ID: <65c97ba2-7005-4f15-a699-da904522aed1@irsmsx103.ger.corp.intel.com> Results for project Python 2.7, build date 2016-01-18 12:30:14 +0000 commit: d34fdd1736f2 previous commit: fb7131939508 revision date: 2016-01-18 08:35:40 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.15% 1.09% 5.37% 3.01% :-) pybench 0.16% 0.02% 6.28% 4.20% :-( regex_v8 0.81% 0.02% -2.28% 2.19% :-) nbody 0.14% 0.26% 7.41% 3.55% :-) json_dump_v2 0.18% 0.09% 5.50% 9.68% :-( normal_startup 1.84% -0.14% -5.87% 2.08% :-| ssbench 0.17% 0.19% 0.29% 1.22% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-01-18/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Jan 18 12:40:48 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 17:40:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=239006=3A_Added_tests_for_XML_RPC_with_non-UTF-8_?= =?utf-8?q?encoding=2E?= Message-ID: <20160118174048.25975.51238@psf.io> https://hg.python.org/cpython/rev/96a7603d25ea changeset: 99964:96a7603d25ea parent: 99960:65e0e06b70b6 parent: 99963:59cb8811286a user: Serhiy Storchaka date: Mon Jan 18 19:40:23 2016 +0200 summary: Issue #9006: Added tests for XML RPC with non-UTF-8 encoding. files: Lib/test/test_xmlrpc.py | 56 +++++++++++++++++++++++++++- 1 files changed, 53 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -183,6 +183,20 @@ xmlrpclib.loads(strg)[0][0]) self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,)) + def test_dump_encoding(self): + value = '\u20ac' + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15') + strg = "" + strg + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + strg = strg.encode('iso-8859-15') + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', + methodresponse=True) + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + strg = strg.encode('iso-8859-15') + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + def test_dump_bytes(self): sample = b"my dog has fleas" self.assertEqual(sample, xmlrpclib.Binary(sample)) @@ -371,7 +385,7 @@ # The evt is set twice. First when the server is ready to serve. # Second when the server has been shutdown. The user must clear # the event after it has been set the first time to catch the second set. -def http_server(evt, numrequests, requestHandler=None): +def http_server(evt, numrequests, requestHandler=None, encoding=None): class TestInstanceClass: def div(self, x, y): return x // y @@ -400,6 +414,7 @@ if not requestHandler: requestHandler = xmlrpc.server.SimpleXMLRPCRequestHandler serv = MyXMLRPCServer(("localhost", 0), requestHandler, + encoding=encoding, logRequests=False, bind_and_activate=False) try: serv.server_bind() @@ -582,6 +597,20 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + def test_client_encoding(self): + start_string = '\u20ac' + end_string = '\xa3' + + try: + p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') + self.assertEqual(p.add(start_string, end_string), + start_string + end_string) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): # send POST with http.client, it should return 404 header and @@ -731,6 +760,26 @@ (None, None)) +class SimpleServerEncodingTestCase(BaseServerTestCase): + @staticmethod + def threadFunc(evt, numrequests, requestHandler=None, encoding=None): + http_server(evt, numrequests, requestHandler, 'iso-8859-15') + + def test_server_encoding(self): + start_string = '\u20ac' + end_string = '\xa3' + + try: + p = xmlrpclib.ServerProxy(URL) + self.assertEqual(p.add(start_string, end_string), + start_string + end_string) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + + class MultiPathServerTestCase(BaseServerTestCase): threadFunc = staticmethod(http_multi_server) request_count = 2 @@ -1143,8 +1192,9 @@ def test_main(): support.run_unittest(XMLRPCTestCase, HelperTestCase, DateTimeTestCase, BinaryTestCase, FaultTestCase, UseBuiltinTypesTestCase, - SimpleServerTestCase, KeepaliveServerTestCase1, - KeepaliveServerTestCase2, GzipServerTestCase, GzipUtilTestCase, + SimpleServerTestCase, SimpleServerEncodingTestCase, + KeepaliveServerTestCase1, KeepaliveServerTestCase2, + GzipServerTestCase, GzipUtilTestCase, MultiPathServerTestCase, ServerProxyTestCase, FailingServerTestCase, CGIHandlerTestCase) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 12:40:49 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 17:40:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzkwMDY6?= =?utf-8?q?_Added_tests_for_XML_RPC_with_non-UTF-8_encoding=2E?= Message-ID: <20160118174048.94198.73178@psf.io> https://hg.python.org/cpython/rev/04e95f05aafe changeset: 99962:04e95f05aafe branch: 2.7 user: Serhiy Storchaka date: Mon Jan 18 19:38:53 2016 +0200 summary: Issue #9006: Added tests for XML RPC with non-UTF-8 encoding. files: Lib/test/test_xmlrpc.py | 56 +++++++++++++++++++++++++++- 1 files changed, 53 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -148,6 +148,17 @@ self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,)) @test_support.requires_unicode + def test_dump_encoding(self): + value = unichr(0x20ac) + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15') + strg = "" + strg + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', + methodresponse=True) + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + + @test_support.requires_unicode def test_default_encoding_issues(self): # SF bug #1115989: wrong decoding in '_stringify' utf8 = """ @@ -280,7 +291,7 @@ # The evt is set twice. First when the server is ready to serve. # Second when the server has been shutdown. The user must clear # the event after it has been set the first time to catch the second set. -def http_server(evt, numrequests, requestHandler=None): +def http_server(evt, numrequests, requestHandler=None, encoding=None): class TestInstanceClass: def div(self, x, y): return x // y @@ -304,6 +315,7 @@ if not requestHandler: requestHandler = SimpleXMLRPCServer.SimpleXMLRPCRequestHandler serv = MyXMLRPCServer(("localhost", 0), requestHandler, + encoding=encoding, logRequests=False, bind_and_activate=False) try: serv.socket.settimeout(3) @@ -461,9 +473,10 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + @test_support.requires_unicode def test_nonascii(self): - start_string = 'P\N{LATIN SMALL LETTER Y WITH CIRCUMFLEX}t' - end_string = 'h\N{LATIN SMALL LETTER O WITH HORN}n' + start_string = test_support.u(r'P\N{LATIN SMALL LETTER Y WITH CIRCUMFLEX}t') + end_string = test_support.u(r'h\N{LATIN SMALL LETTER O WITH HORN}n') try: p = xmlrpclib.ServerProxy(URL) @@ -480,6 +493,21 @@ server = xmlrpclib.ServerProxy(u"http://%s:%d/RPC2"%(ADDR, PORT)) self.assertEqual(server.add("a", u"\xe9"), u"a\xe9") + @test_support.requires_unicode + def test_client_encoding(self): + start_string = unichr(0x20ac) + end_string = unichr(0xa3) + + try: + p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') + self.assertEqual(p.add(start_string, end_string), + start_string + end_string) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): # send POST with httplib, it should return 404 header and @@ -599,6 +627,27 @@ conn.request('POST', '/RPC2 HTTP/1.0\r\nContent-Length: 100\r\n\r\nbye') conn.close() +class SimpleServerEncodingTestCase(BaseServerTestCase): + @staticmethod + def threadFunc(evt, numrequests, requestHandler=None, encoding=None): + http_server(evt, numrequests, requestHandler, 'iso-8859-15') + + @test_support.requires_unicode + def test_server_encoding(self): + start_string = unichr(0x20ac) + end_string = unichr(0xa3) + + try: + p = xmlrpclib.ServerProxy(URL) + self.assertEqual(p.add(start_string, end_string), + start_string + end_string) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + + class MultiPathServerTestCase(BaseServerTestCase): threadFunc = staticmethod(http_multi_server) request_count = 2 @@ -1031,6 +1080,7 @@ xmlrpc_tests = [XMLRPCTestCase, HelperTestCase, DateTimeTestCase, BinaryTestCase, FaultTestCase, TransportSubclassTestCase] xmlrpc_tests.append(SimpleServerTestCase) + xmlrpc_tests.append(SimpleServerEncodingTestCase) xmlrpc_tests.append(KeepaliveServerTestCase1) xmlrpc_tests.append(KeepaliveServerTestCase2) xmlrpc_tests.append(GzipServerTestCase) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 12:40:49 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 17:40:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzkwMDY6?= =?utf-8?q?_Added_tests_for_XML_RPC_with_non-UTF-8_encoding=2E?= Message-ID: <20160118174048.96370.5404@psf.io> https://hg.python.org/cpython/rev/59cb8811286a changeset: 99963:59cb8811286a branch: 3.5 parent: 99951:31debd0dc0dc user: Serhiy Storchaka date: Mon Jan 18 19:39:26 2016 +0200 summary: Issue #9006: Added tests for XML RPC with non-UTF-8 encoding. files: Lib/test/test_xmlrpc.py | 56 +++++++++++++++++++++++++++- 1 files changed, 53 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -183,6 +183,20 @@ xmlrpclib.loads(strg)[0][0]) self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,)) + def test_dump_encoding(self): + value = '\u20ac' + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15') + strg = "" + strg + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + strg = strg.encode('iso-8859-15') + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', + methodresponse=True) + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + strg = strg.encode('iso-8859-15') + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + def test_dump_bytes(self): sample = b"my dog has fleas" self.assertEqual(sample, xmlrpclib.Binary(sample)) @@ -371,7 +385,7 @@ # The evt is set twice. First when the server is ready to serve. # Second when the server has been shutdown. The user must clear # the event after it has been set the first time to catch the second set. -def http_server(evt, numrequests, requestHandler=None): +def http_server(evt, numrequests, requestHandler=None, encoding=None): class TestInstanceClass: def div(self, x, y): return x // y @@ -400,6 +414,7 @@ if not requestHandler: requestHandler = xmlrpc.server.SimpleXMLRPCRequestHandler serv = MyXMLRPCServer(("localhost", 0), requestHandler, + encoding=encoding, logRequests=False, bind_and_activate=False) try: serv.server_bind() @@ -582,6 +597,20 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + def test_client_encoding(self): + start_string = '\u20ac' + end_string = '\xa3' + + try: + p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') + self.assertEqual(p.add(start_string, end_string), + start_string + end_string) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): # send POST with http.client, it should return 404 header and @@ -731,6 +760,26 @@ (None, None)) +class SimpleServerEncodingTestCase(BaseServerTestCase): + @staticmethod + def threadFunc(evt, numrequests, requestHandler=None, encoding=None): + http_server(evt, numrequests, requestHandler, 'iso-8859-15') + + def test_server_encoding(self): + start_string = '\u20ac' + end_string = '\xa3' + + try: + p = xmlrpclib.ServerProxy(URL) + self.assertEqual(p.add(start_string, end_string), + start_string + end_string) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + + class MultiPathServerTestCase(BaseServerTestCase): threadFunc = staticmethod(http_multi_server) request_count = 2 @@ -1143,8 +1192,9 @@ def test_main(): support.run_unittest(XMLRPCTestCase, HelperTestCase, DateTimeTestCase, BinaryTestCase, FaultTestCase, UseBuiltinTypesTestCase, - SimpleServerTestCase, KeepaliveServerTestCase1, - KeepaliveServerTestCase2, GzipServerTestCase, GzipUtilTestCase, + SimpleServerTestCase, SimpleServerEncodingTestCase, + KeepaliveServerTestCase1, KeepaliveServerTestCase2, + GzipServerTestCase, GzipUtilTestCase, MultiPathServerTestCase, ServerProxyTestCase, FailingServerTestCase, CGIHandlerTestCase) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 12:40:49 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 17:40:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxODQ3?= =?utf-8?q?=3A_Fixed_xmlrpclib_and_tests_on_Unicode-disabled_builds=2E?= Message-ID: <20160118174047.94216.17100@psf.io> https://hg.python.org/cpython/rev/f602dfd35cd4 changeset: 99961:f602dfd35cd4 branch: 2.7 parent: 99957:d34fdd1736f2 user: Serhiy Storchaka date: Mon Jan 18 19:35:23 2016 +0200 summary: Issue #21847: Fixed xmlrpclib and tests on Unicode-disabled builds. files: Lib/test/test_xmlrpc.py | 19 +++++++++---------- Lib/xmlrpclib.py | 4 ++-- Misc/NEWS | 2 ++ 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -23,13 +23,6 @@ except ImportError: gzip = None -try: - unicode -except NameError: - have_unicode = False -else: - have_unicode = True - alist = [{'astring': 'foo at bar.baz.spam', 'afloat': 7283.43, 'anint': 2**20, @@ -37,8 +30,6 @@ 'anotherlist': ['.zyx.41'], 'abase64': xmlrpclib.Binary("my dog has fleas"), 'boolean': xmlrpclib.False, - 'unicode': u'\u4000\u6000\u8000', - u'ukey\u4000': 'regular value', 'datetime1': xmlrpclib.DateTime('20050210T11:41:23'), 'datetime2': xmlrpclib.DateTime( (2005, 02, 10, 11, 41, 23, 0, 1, -1)), @@ -46,6 +37,12 @@ datetime.datetime(2005, 02, 10, 11, 41, 23)), }] +if test_support.have_unicode: + alist[0].update({ + 'unicode': test_support.u(r'\u4000\u6000\u8000'), + test_support.u(r'ukey\u4000'): 'regular value', + }) + class XMLRPCTestCase(unittest.TestCase): def test_dump_load(self): @@ -150,6 +147,7 @@ xmlrpclib.loads(strg)[0][0]) self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,)) + @test_support.requires_unicode def test_default_encoding_issues(self): # SF bug #1115989: wrong decoding in '_stringify' utf8 = """ @@ -182,7 +180,7 @@ temp_sys.setdefaultencoding(old_encoding) items = d.items() - if have_unicode: + if test_support.have_unicode: self.assertEqual(s, u"abc \x95") self.assertIsInstance(s, unicode) self.assertEqual(items, [(u"def \x96", u"ghi \x97")]) @@ -477,6 +475,7 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + @test_support.requires_unicode def test_unicode_host(self): server = xmlrpclib.ServerProxy(u"http://%s:%d/RPC2"%(ADDR, PORT)) self.assertEqual(server.add("a", u"\xe9"), u"a\xe9") diff --git a/Lib/xmlrpclib.py b/Lib/xmlrpclib.py --- a/Lib/xmlrpclib.py +++ b/Lib/xmlrpclib.py @@ -393,7 +393,7 @@ elif datetime and isinstance(other, datetime.datetime): s = self.value o = other.strftime("%Y%m%dT%H:%M:%S") - elif isinstance(other, (str, unicode)): + elif isinstance(other, basestring): s = self.value o = other elif hasattr(other, "timetuple"): @@ -1560,7 +1560,7 @@ allow_none=0, use_datetime=0, context=None): # establish a "logical" server connection - if isinstance(uri, unicode): + if unicode and isinstance(uri, unicode): uri = uri.encode('ISO-8859-1') # get the url diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,8 @@ Library ------- +- Issue #21847: Fixed xmlrpclib on Unicode-disabled builds. + - Issue #6500: Fixed infinite recursion in urllib2.Request.__getattr__(). - Issue #26083: Workaround a subprocess bug that raises an incorrect -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 13:53:01 2016 From: python-checkins at python.org (brett.cannon) Date: Mon, 18 Jan 2016 18:53:01 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Various_changes_to_PEP_512_ba?= =?utf-8?q?sed_on_feedback_from_core-workflow?= Message-ID: <20160118185301.14181.11179@psf.io> https://hg.python.org/peps/rev/633f51d10a67 changeset: 6196:633f51d10a67 user: Brett Cannon date: Mon Jan 18 10:52:57 2016 -0800 summary: Various changes to PEP 512 based on feedback from core-workflow files: pep-0512.txt | 179 ++++++++++++++++++++++++-------------- 1 files changed, 114 insertions(+), 65 deletions(-) diff --git a/pep-0512.txt b/pep-0512.txt --- a/pep-0512.txt +++ b/pep-0512.txt @@ -97,24 +97,24 @@ The overarching goal of this migration is to improve the development process to the extent that a core developer can go from external contribution submission through all the steps leading to committing -said contribution all from within a browser on a tablet with WiFi. -All of this will be done in such a way that if an external -contributor chooses not to use GitHub then they will continue to have -that option. +said contribution all from within a browser on a tablet with WiFi +using *some* development process (this does not inherently mean +GitHub's default workflow). All of this will be done in such a way +that if an external contributor chooses not to use GitHub then they +will continue to have that option. Repositories to Migrate ======================= While hg.python.org [#h.p.o]_ hosts many repositories, there are only -six key repositories that must move: +five key repositories that should move: 1. devinabox [#devinabox-repo]_ 2. benchmarks [#benchmarks-repo]_ -3. tracker [#tracker-repo]_ 4. peps [#peps-repo]_ 5. devguide [#devguide-repo]_ 6. cpython [#cpython-repo]_ -The devinabox, benchmarksm and tracker repositories are code-only. +The devinabox and benchmarks repositories are code-only. The peps and devguide repositories involve the generation of webpages. And the cpython repository has special requirements for integration with bugs.python.org [#b.p.o]_. @@ -131,11 +131,10 @@ Requirements for Code-Only Repositories --------------------------------------- Completion of the requirements in this section will allow the -devinabox, benchmarks, and tracker repositories to move to +devinabox and benchmarks repositories to move to GitHub. While devinabox has a sufficiently descriptive name, the benchmarks repository does not; therefore, it will be named -"python-benchmark-suite". The tracker repo also has a misleading name -and will be renamed "bugs.python.org". +"python-benchmark-suite". Create a 'python-dev' team '''''''''''''''''''''''''' @@ -174,6 +173,10 @@ bugs.python.org [#b.p.o]_ account and their GitHub account, which will be done through a new field in a user's profile. +This does implicitly require that contributors will need both a +GitHub [#github]_ and bugs.python.org account in order to sign the +CLA and contribute through GitHub. + A bot to enforce CLA signing ++++++++++++++++++++++++++++ With an association between someone's GitHub account and their @@ -216,11 +219,14 @@ Document steps to commit a pull request ''''''''''''''''''''''''''''''''''''''' During the process of choosing a new development workflow, it was -decided that a linear history is desired. This means that the -convenient "Merge" button in GitHub pull requests is undesireable, as -it creates a merge commit along with all of the contributor's -individual commits (this does not affect the other repositories where -the desire for a linear history doesn't exist). +decided that a linear history is desired. People preferred having a +single commit representing a single change instead of having a set of +unrelated commits lead to a merge commit that represented a single +change. This means that the convenient "Merge" button in GitHub pull +requests is undesirable, as it creates a merge commit along with all +of the contributor's individual commits (this does not affect the +other repositories where the desire for a linear history doesn't +exist). Luckily, Git [#git]_ does not require GitHub's workflow and so one can be chosen which gives us a linear history by using Git's CLI. The @@ -285,7 +291,7 @@ as there can take multiple pull requests to solve a single issue (technically it should be a many-to-many association for when a single fix solves multiple issues, but this is fairly rare and issues -can be merged into one using the ``Superceder`` field on the issue +can be merged into one using the ``Superseder`` field on the issue tracker). Association between a pull request and an issue will be done based on @@ -338,6 +344,16 @@ attribute will need to be removed. An equivalent ``sys._git`` attribute will be needed to take its place. +Update the devguide +''''''''''''''''''' +The devguide will need to be updated with details of the new +workflow. Mostly likely work will take place in a separate branch +until the migration actually occurs. + +Update PEP 101 +'''''''''''''' +The release process will need to be updated as necessary. + Optional, Planned Features -------------------------- Once the cpython repository [#cpython-repo]_ is migrated, all @@ -457,11 +473,24 @@ repositories to simplify their maintenance and to expand who has commit rights to them to ease in their maintenance. +It has also been suggested to split out the +`What's New `__ +documents. That would require deciding whether a workflow could be +developed where it would be difficult to forget to update +What's New (potentially through a label added to PRs, like +"What's New needed"). + +Backup of Git repositories +'''''''''''''''''''''''''' +While not necessary, it would be good to have official backups of the +various Git repositories for disaster protection. It will be up to +the PSF infrastructure committee to decide if this is worthwhile or +unnecessary. + Status ====== -Requirements for migrating the devinabox [#devinabox-repo]_, -benchmarks [#benchmarks-repo]_, and tracker [#tracker-repo]_ -repositories: +Requirements for migrating the devinabox [#devinabox-repo]_ and +benchmarks [#benchmarks-repo]_ repositories: * Not started @@ -478,63 +507,66 @@ - None - Repositories whose build steps need updating: +Repositories whose build steps need updating: - * Not started +* Not started - - peps [#peps-repo]_ - - devguide [#devguide-repo]_ + - peps [#peps-repo]_ + - devguide [#devguide-repo]_ - * In progress +* In progress - - None + - None - * Completed +* Completed - - None + - None - Requirements to move over the cpython repo [#cpython-repo]_: +Requirements to move over the cpython repo [#cpython-repo]_: - * Not started +* Not started - - `Document steps to commit a pull request`_ - - `Handling Misc/NEWS`_ - - `Handling Misc/ACKS`_ - - `Linking a pull request to an issue`_ - - `Notify the issue if the pull request is committed`_ - - `Update linking service for mapping commit IDs to URLs`_ - - `Create https://git.python.org`_ - - `Backup of pull request data`_ - - `Change sys._mercurial`_ + - `Document steps to commit a pull request`_ + - `Handling Misc/NEWS`_ + - `Handling Misc/ACKS`_ + - `Linking a pull request to an issue`_ + - `Notify the issue if the pull request is committed`_ + - `Update linking service for mapping commit IDs to URLs`_ + - `Create https://git.python.org`_ + - `Backup of pull request data`_ + - `Change sys._mercurial`_ + - `Update the devguide`_ + - `Update PEP 101`_ - * In progress +* In progress - - None + - None - * Completed +* Completed - - None + - None - Optional features: +Optional features: - * Not started +* Not started - - `Bot to handle pull request merging`_ - - `Continuous integration per pull request`_ - - `Test coverage report`_ - - `Notifying issues of pull request comments`_ - - `Allow bugs.python.org to use GitHub as a login provider`_ - - `Web hooks for re-generating web content`_ - - `Link web content back to files that it is generated from`_ - - `Splitting out parts of the documentation into their own repositories`_ + - `Bot to handle pull request merging`_ + - `Continuous integration per pull request`_ + - `Test coverage report`_ + - `Notifying issues of pull request comments`_ + - `Allow bugs.python.org to use GitHub as a login provider`_ + - `Web hooks for re-generating web content`_ + - `Link web content back to files that it is generated from`_ + - `Splitting out parts of the documentation into their own repositories`_ + - `Backup of Git repositories`_ - * In progress +* In progress - - None + - None - * Completed +* Completed - - None + - None Open Issues =========== @@ -556,6 +588,10 @@ time and resources to keep it running. They may also choose whether they want to host a Git mirror on PSF infrastructure. +Depending on the decision reached, other ancillary repositories will +either be forced to migration or they can choose to simply stay on +hg.python.org. + Tools and commands to move from Mercurial to Git ------------------------------------------------ A decision needs to be made on exactly what tooling and what commands @@ -579,7 +615,7 @@ How to handle the Misc/NEWS file -------------------------------- -There are two competing approaches to handling +There are three competing approaches to handling ``Misc/NEWS`` [#news-file]_. One is to add a news entry for issues on bugs.python.org [#b.p.o]_. This would mean an issue that is marked as "resolved" could not be closed until a news entry is added in the @@ -593,13 +629,13 @@ the output needed to be checked into the code repository. This approach is agnostic to whether a commit was done by CLI or bot. -The competing approach is to use an individual file per news entry, -containg the text for the entry. In this scenario each feature +A competing approach is to use an individual file per news entry, +containing the text for the entry. In this scenario each feature release would have its own directory for news entries and a separate file would be created in that directory that was either named after the issue it closed or a timestamp value (which prevents collisions). Merges across branches would have no issue as the news entry file -would still be uniqeuely named and in the directory of the latest +would still be uniquely named and in the directory of the latest version that contained the fix. A script would collect all news entry files no matter what directory they reside in and create an appropriate news file (the release directory can be ignored as the @@ -618,8 +654,14 @@ flattened commit (while most likely also supporting using the first line of the commit message if no specific news entry was specified). Code for this approach has been written previously for the Mercurial -workflow at http://bugs.python.org/issue18967. There is also a tool -from the community at https://pypi.python.org/pypi/towncrier. +workflow at http://bugs.python.org/issue18967. There is also tools +from the community like https://pypi.python.org/pypi/towncrier and +https://github.com/twisted/newsbuilder . + +A yet third option is a merge script to handle the conflicts. This +approach allows for keeping the NEWS file as a single file. It does +run the risk, though, of failure and thus blocking a commit until it +can be manually resolved. Naming the commit bot --------------------- @@ -725,10 +767,19 @@ oldest branch first and then merged up to the default branch, the question came up as to whether this workflow should be perpetuated. In the end it was decided that committing in the newest branch and -then cherrypicking changes into older branches would work best as +then cherry-picking changes into older branches would work best as most people will instinctively work off the newest branch and it is a more common workflow when using Git [#git]_. +Cherry-picking is also more bot-friendly for an in-browser workflow. +In the merge-up scenario, if you were to request a bot to do a merge +and it failed, then you would have to make sure to immediately solve +the merge conflicts if you still allowed the main commit, else you +would need to postpone the entire commit until all merges could be +handled. With a cherry-picking workflow, the main commit could +proceed while postponing the merge-failing cherry-picks. This allows +for possibly distributing the work of managing conflicting merges. + Deriving ``Misc/NEWS`` from the commit logs ------------------------------------------- As part of the discussion surrounding `Handling Misc/NEWS`_, the @@ -774,8 +825,6 @@ .. [#peps-repo] Mercurial repository of the Python Enhancement Proposals (https://hg.python.org/peps/) -.. [#tracker-repo] bugs.python.org code repository (https://hg.python.org/tracker/python-dev/) - .. [#devguide-repo] Mercurial repository for the Python Developer's Guide (https://hg.python.org/devguide/) .. [#cpython-repo] Mercurial repository for CPython (https://hg.python.org/cpython/) -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Mon Jan 18 14:36:31 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 19:36:31 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDEz?= =?utf-8?q?=3A_Added_compatibility_with_broken_protocol_2_pickles_created?= Message-ID: <20160118193630.94212.41483@psf.io> https://hg.python.org/cpython/rev/270add17f203 changeset: 99965:270add17f203 branch: 3.5 parent: 99963:59cb8811286a user: Serhiy Storchaka date: Mon Jan 18 21:35:22 2016 +0200 summary: Issue #26013: Added compatibility with broken protocol 2 pickles created in old Python 3 versions (3.4.3 and lower). files: Lib/_compat_pickle.py | 7 +++++++ Misc/NEWS | 3 +++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/_compat_pickle.py b/Lib/_compat_pickle.py --- a/Lib/_compat_pickle.py +++ b/Lib/_compat_pickle.py @@ -177,6 +177,13 @@ 'DocXMLRPCServer': 'xmlrpc.server', 'SimpleHTTPServer': 'http.server', 'CGIHTTPServer': 'http.server', + # For compatibility with broken pickles saved in old Python 3 versions + 'UserDict': 'collections', + 'UserList': 'collections', + 'UserString': 'collections', + 'whichdb': 'dbm', + 'StringIO': 'io', + 'cStringIO': 'io', }) REVERSE_IMPORT_MAPPING.update({ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,6 +44,9 @@ Library ------- +- Issue #26013: Added compatibility with broken protocol 2 pickles created + in old Python 3 versions (3.4.3 and lower). + - Issue #25850: Use cross-compilation by default for 64-bit Windows. - Issue #17633: Improve zipimport's support for namespace packages. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 14:36:31 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 19:36:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326013=3A_Added_compatibility_with_broken_protoc?= =?utf-8?q?ol_2_pickles_created?= Message-ID: <20160118193630.121296.54327@psf.io> https://hg.python.org/cpython/rev/35ff0976b211 changeset: 99966:35ff0976b211 parent: 99964:96a7603d25ea parent: 99965:270add17f203 user: Serhiy Storchaka date: Mon Jan 18 21:36:03 2016 +0200 summary: Issue #26013: Added compatibility with broken protocol 2 pickles created in old Python 3 versions (3.4.3 and lower). files: Lib/_compat_pickle.py | 7 +++++++ Misc/NEWS | 3 +++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/_compat_pickle.py b/Lib/_compat_pickle.py --- a/Lib/_compat_pickle.py +++ b/Lib/_compat_pickle.py @@ -177,6 +177,13 @@ 'DocXMLRPCServer': 'xmlrpc.server', 'SimpleHTTPServer': 'http.server', 'CGIHTTPServer': 'http.server', + # For compatibility with broken pickles saved in old Python 3 versions + 'UserDict': 'collections', + 'UserList': 'collections', + 'UserString': 'collections', + 'whichdb': 'dbm', + 'StringIO': 'io', + 'cStringIO': 'io', }) REVERSE_IMPORT_MAPPING.update({ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,9 @@ Library ------- +- Issue #26013: Added compatibility with broken protocol 2 pickles created + in old Python 3 versions (3.4.3 and lower). + - Issue #26129: Deprecated accepting non-integers in grp.getgrgid(). - Issue #25850: Use cross-compilation by default for 64-bit Windows. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 15:34:40 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 20:34:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Added_exceptin?= =?utf-8?q?s_for_testing_non-reversible_import_mapping_for_Issue_=2326013?= =?utf-8?q?=2E?= Message-ID: <20160118203440.25920.80485@psf.io> https://hg.python.org/cpython/rev/7215d13baa2d changeset: 99967:7215d13baa2d branch: 3.5 parent: 99965:270add17f203 user: Serhiy Storchaka date: Mon Jan 18 22:33:44 2016 +0200 summary: Added exceptins for testing non-reversible import mapping for Issue #26013. files: Lib/test/test_pickle.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -244,6 +244,8 @@ ALT_IMPORT_MAPPING = { ('_elementtree', 'xml.etree.ElementTree'), ('cPickle', 'pickle'), + ('StringIO', 'io'), + ('cStringIO', 'io'), } ALT_NAME_MAPPING = { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 15:34:40 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Mon, 18 Jan 2016 20:34:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Added_exceptins_for_testing_non-reversible_import_mappin?= =?utf-8?q?g_for_Issue_=2326013=2E?= Message-ID: <20160118203440.18433.75091@psf.io> https://hg.python.org/cpython/rev/16cfc1652844 changeset: 99968:16cfc1652844 parent: 99966:35ff0976b211 parent: 99967:7215d13baa2d user: Serhiy Storchaka date: Mon Jan 18 22:34:19 2016 +0200 summary: Added exceptins for testing non-reversible import mapping for Issue #26013. files: Lib/test/test_pickle.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -239,6 +239,8 @@ ALT_IMPORT_MAPPING = { ('_elementtree', 'xml.etree.ElementTree'), ('cPickle', 'pickle'), + ('StringIO', 'io'), + ('cStringIO', 'io'), } ALT_NAME_MAPPING = { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 19:02:09 2016 From: python-checkins at python.org (berker.peksag) Date: Tue, 19 Jan 2016 00:02:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325366=3A_Skip_test=5Fwith=5Fpip_when_threading_?= =?utf-8?q?module_is_not_available?= Message-ID: <20160119000209.25975.37627@psf.io> https://hg.python.org/cpython/rev/27d4f06c3720 changeset: 99970:27d4f06c3720 parent: 99968:16cfc1652844 parent: 99969:2a201272daed user: Berker Peksag date: Tue Jan 19 02:02:13 2016 +0200 summary: Issue #25366: Skip test_with_pip when threading module is not available Some dependencies of pip import threading module unconditionally so we need to skip the test to make buildbots happy. files: Lib/test/test_venv.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -25,6 +25,11 @@ except ImportError: ssl = None +try: + import threading +except ImportError: + threading = None + skipInVenv = unittest.skipIf(sys.prefix != sys.base_prefix, 'Test not appropriate in a venv') @@ -319,6 +324,8 @@ # Requesting pip fails without SSL (http://bugs.python.org/issue19744) @unittest.skipIf(ssl is None, ensurepip._MISSING_SSL_MESSAGE) + @unittest.skipUnless(threading, 'some dependencies of pip import threading' + ' module unconditionally') def test_with_pip(self): rmtree(self.env_dir) with EnvironmentVarGuard() as envvars: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 19:02:10 2016 From: python-checkins at python.org (berker.peksag) Date: Tue, 19 Jan 2016 00:02:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MzY2?= =?utf-8?q?=3A_Skip_test=5Fwith=5Fpip_when_threading_module_is_not_availab?= =?utf-8?q?le?= Message-ID: <20160119000209.41322.9070@psf.io> https://hg.python.org/cpython/rev/2a201272daed changeset: 99969:2a201272daed branch: 3.5 parent: 99967:7215d13baa2d user: Berker Peksag date: Tue Jan 19 02:01:53 2016 +0200 summary: Issue #25366: Skip test_with_pip when threading module is not available Some dependencies of pip import threading module unconditionally so we need to skip the test to make buildbots happy. files: Lib/test/test_venv.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -25,6 +25,11 @@ except ImportError: ssl = None +try: + import threading +except ImportError: + threading = None + skipInVenv = unittest.skipIf(sys.prefix != sys.base_prefix, 'Test not appropriate in a venv') @@ -319,6 +324,8 @@ # Requesting pip fails without SSL (http://bugs.python.org/issue19744) @unittest.skipIf(ssl is None, ensurepip._MISSING_SSL_MESSAGE) + @unittest.skipUnless(threading, 'some dependencies of pip import threading' + ' module unconditionally') def test_with_pip(self): rmtree(self.env_dir) with EnvironmentVarGuard() as envvars: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 20:38:19 2016 From: python-checkins at python.org (martin.panter) Date: Tue, 19 Jan 2016 01:38:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325859=3A_Merge_NNTP_test_server_from_3=2E5?= Message-ID: <20160119013819.25961.15476@psf.io> https://hg.python.org/cpython/rev/c44bfa733526 changeset: 99972:c44bfa733526 parent: 99970:27d4f06c3720 parent: 99971:a9fca95f119a user: Martin Panter date: Tue Jan 19 01:35:44 2016 +0000 summary: Issue #25859: Merge NNTP test server from 3.5 files: Lib/test/test_nntplib.py | 82 +++++++++++++++++++++------ 1 files changed, 64 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_nntplib.py b/Lib/test/test_nntplib.py --- a/Lib/test/test_nntplib.py +++ b/Lib/test/test_nntplib.py @@ -5,6 +5,7 @@ import unittest import functools import contextlib +import os.path from test import support from nntplib import NNTP, GroupInfo import nntplib @@ -13,8 +14,13 @@ import ssl except ImportError: ssl = None +try: + import threading +except ImportError: + threading = None TIMEOUT = 30 +certfile = os.path.join(os.path.dirname(__file__), 'keycert3.pem') # TODO: # - test the `file` arg to more commands @@ -202,24 +208,6 @@ resp, caps = self.server.capabilities() _check_caps(caps) - @unittest.skipUnless(ssl, 'requires SSL support') - def test_starttls(self): - file = self.server.file - sock = self.server.sock - try: - self.server.starttls() - except nntplib.NNTPPermanentError: - self.skipTest("STARTTLS not supported by server.") - else: - # Check that the socket and internal pseudo-file really were - # changed. - self.assertNotEqual(file, self.server.file) - self.assertNotEqual(sock, self.server.sock) - # Check that the new socket really is an SSL one - self.assertIsInstance(self.server.sock, ssl.SSLSocket) - # Check that trying starttls when it's already active fails. - self.assertRaises(ValueError, self.server.starttls) - def test_zlogin(self): # This test must be the penultimate because further commands will be # refused. @@ -1520,6 +1508,64 @@ def nntp_class(*pos, **kw): return nntplib.NNTP_SSL(*pos, ssl_context=bypass_context, **kw) + at unittest.skipUnless(threading, 'requires multithreading') +class LocalServerTests(unittest.TestCase): + def setUp(self): + sock = socket.socket() + port = support.bind_port(sock) + sock.listen() + self.background = threading.Thread( + target=self.run_server, args=(sock,)) + self.background.start() + self.addCleanup(self.background.join) + + self.nntp = NNTP(support.HOST, port, usenetrc=False).__enter__() + self.addCleanup(self.nntp.__exit__, None, None, None) + + def run_server(self, sock): + # Could be generalized to handle more commands in separate methods + with sock: + [client, _] = sock.accept() + with contextlib.ExitStack() as cleanup: + cleanup.enter_context(client) + reader = cleanup.enter_context(client.makefile('rb')) + client.sendall(b'200 Server ready\r\n') + while True: + cmd = reader.readline() + if cmd == b'CAPABILITIES\r\n': + client.sendall( + b'101 Capability list:\r\n' + b'VERSION 2\r\n' + b'STARTTLS\r\n' + b'.\r\n' + ) + elif cmd == b'STARTTLS\r\n': + reader.close() + client.sendall(b'382 Begin TLS negotiation now\r\n') + client = ssl.wrap_socket( + client, server_side=True, certfile=certfile) + cleanup.enter_context(client) + reader = cleanup.enter_context(client.makefile('rb')) + elif cmd == b'QUIT\r\n': + client.sendall(b'205 Bye!\r\n') + break + else: + raise ValueError('Unexpected command {!r}'.format(cmd)) + + @unittest.skipUnless(ssl, 'requires SSL support') + def test_starttls(self): + file = self.nntp.file + sock = self.nntp.sock + self.nntp.starttls() + # Check that the socket and internal pseudo-file really were + # changed. + self.assertNotEqual(file, self.nntp.file) + self.assertNotEqual(sock, self.nntp.sock) + # Check that the new socket really is an SSL one + self.assertIsInstance(self.nntp.sock, ssl.SSLSocket) + # Check that trying starttls when it's already active fails. + self.assertRaises(ValueError, self.nntp.starttls) + if __name__ == "__main__": unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 20:38:19 2016 From: python-checkins at python.org (martin.panter) Date: Tue, 19 Jan 2016 01:38:19 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1ODU5?= =?utf-8?q?=3A_Reimplement_NNTP_test=5Fstarttls=28=29_using_local_server?= Message-ID: <20160119013819.121284.23252@psf.io> https://hg.python.org/cpython/rev/a9fca95f119a changeset: 99971:a9fca95f119a branch: 3.5 parent: 99969:2a201272daed user: Martin Panter date: Tue Jan 19 01:10:58 2016 +0000 summary: Issue #25859: Reimplement NNTP test_starttls() using local server The previous test relied on a remote server, which currently seems to be shutting the connection down once TLS has been set up, causing an EOFError. Now the test is implemented using a minimal NNTP server running in a background thread. files: Lib/test/test_nntplib.py | 82 +++++++++++++++++++++------ 1 files changed, 64 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_nntplib.py b/Lib/test/test_nntplib.py --- a/Lib/test/test_nntplib.py +++ b/Lib/test/test_nntplib.py @@ -5,6 +5,7 @@ import unittest import functools import contextlib +import os.path from test import support from nntplib import NNTP, GroupInfo import nntplib @@ -13,8 +14,13 @@ import ssl except ImportError: ssl = None +try: + import threading +except ImportError: + threading = None TIMEOUT = 30 +certfile = os.path.join(os.path.dirname(__file__), 'keycert3.pem') # TODO: # - test the `file` arg to more commands @@ -202,24 +208,6 @@ resp, caps = self.server.capabilities() _check_caps(caps) - @unittest.skipUnless(ssl, 'requires SSL support') - def test_starttls(self): - file = self.server.file - sock = self.server.sock - try: - self.server.starttls() - except nntplib.NNTPPermanentError: - self.skipTest("STARTTLS not supported by server.") - else: - # Check that the socket and internal pseudo-file really were - # changed. - self.assertNotEqual(file, self.server.file) - self.assertNotEqual(sock, self.server.sock) - # Check that the new socket really is an SSL one - self.assertIsInstance(self.server.sock, ssl.SSLSocket) - # Check that trying starttls when it's already active fails. - self.assertRaises(ValueError, self.server.starttls) - def test_zlogin(self): # This test must be the penultimate because further commands will be # refused. @@ -1520,6 +1508,64 @@ def nntp_class(*pos, **kw): return nntplib.NNTP_SSL(*pos, ssl_context=bypass_context, **kw) + at unittest.skipUnless(threading, 'requires multithreading') +class LocalServerTests(unittest.TestCase): + def setUp(self): + sock = socket.socket() + port = support.bind_port(sock) + sock.listen() + self.background = threading.Thread( + target=self.run_server, args=(sock,)) + self.background.start() + self.addCleanup(self.background.join) + + self.nntp = NNTP(support.HOST, port, usenetrc=False).__enter__() + self.addCleanup(self.nntp.__exit__, None, None, None) + + def run_server(self, sock): + # Could be generalized to handle more commands in separate methods + with sock: + [client, _] = sock.accept() + with contextlib.ExitStack() as cleanup: + cleanup.enter_context(client) + reader = cleanup.enter_context(client.makefile('rb')) + client.sendall(b'200 Server ready\r\n') + while True: + cmd = reader.readline() + if cmd == b'CAPABILITIES\r\n': + client.sendall( + b'101 Capability list:\r\n' + b'VERSION 2\r\n' + b'STARTTLS\r\n' + b'.\r\n' + ) + elif cmd == b'STARTTLS\r\n': + reader.close() + client.sendall(b'382 Begin TLS negotiation now\r\n') + client = ssl.wrap_socket( + client, server_side=True, certfile=certfile) + cleanup.enter_context(client) + reader = cleanup.enter_context(client.makefile('rb')) + elif cmd == b'QUIT\r\n': + client.sendall(b'205 Bye!\r\n') + break + else: + raise ValueError('Unexpected command {!r}'.format(cmd)) + + @unittest.skipUnless(ssl, 'requires SSL support') + def test_starttls(self): + file = self.nntp.file + sock = self.nntp.sock + self.nntp.starttls() + # Check that the socket and internal pseudo-file really were + # changed. + self.assertNotEqual(file, self.nntp.file) + self.assertNotEqual(sock, self.nntp.sock) + # Check that the new socket really is an SSL one + self.assertIsInstance(self.nntp.sock, ssl.SSLSocket) + # Check that trying starttls when it's already active fails. + self.assertRaises(ValueError, self.nntp.starttls) + if __name__ == "__main__": unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 21:46:00 2016 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 19 Jan 2016 02:46:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_issue23962_-_R?= =?utf-8?q?eference_the_correct_TimeoutError_in_concurrent=2Efutures?= Message-ID: <20160119024600.14189.10958@psf.io> https://hg.python.org/cpython/rev/673d1ccea050 changeset: 99973:673d1ccea050 branch: 3.5 parent: 99971:a9fca95f119a user: Senthil Kumaran date: Mon Jan 18 18:45:00 2016 -0800 summary: issue23962 - Reference the correct TimeoutError in concurrent.futures documentation. Patch contributed by Ryder Lewis. files: Doc/library/concurrent.futures.rst | 43 +++++++++++------ 1 files changed, 27 insertions(+), 16 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -42,7 +42,7 @@ Equivalent to :func:`map(func, *iterables) ` except *func* is executed asynchronously and several calls to *func* may be made concurrently. The - returned iterator raises a :exc:`TimeoutError` if + returned iterator raises a :exc:`concurrent.futures.TimeoutError` if :meth:`~iterator.__next__` is called and the result isn't available after *timeout* seconds from the original call to :meth:`Executor.map`. *timeout* can be an int or a float. If *timeout* is not specified or @@ -274,11 +274,12 @@ Return the value returned by the call. If the call hasn't yet completed then this method will wait up to *timeout* seconds. If the call hasn't - completed in *timeout* seconds, then a :exc:`TimeoutError` will be - raised. *timeout* can be an int or float. If *timeout* is not specified - or ``None``, there is no limit to the wait time. + completed in *timeout* seconds, then a + :exc:`concurrent.futures.TimeoutError` will be raised. *timeout* can be + an int or float. If *timeout* is not specified or ``None``, there is no + limit to the wait time. - If the future is cancelled before completing then :exc:`CancelledError` + If the future is cancelled before completing then :exc:`.CancelledError` will be raised. If the call raised, this method will raise the same exception. @@ -287,11 +288,12 @@ Return the exception raised by the call. If the call hasn't yet completed then this method will wait up to *timeout* seconds. If the - call hasn't completed in *timeout* seconds, then a :exc:`TimeoutError` - will be raised. *timeout* can be an int or float. If *timeout* is not - specified or ``None``, there is no limit to the wait time. + call hasn't completed in *timeout* seconds, then a + :exc:`concurrent.futures.TimeoutError` will be raised. *timeout* can be + an int or float. If *timeout* is not specified or ``None``, there is no + limit to the wait time. - If the future is cancelled before completing then :exc:`CancelledError` + If the future is cancelled before completing then :exc:`.CancelledError` will be raised. If the call completed without raising, ``None`` is returned. @@ -391,13 +393,12 @@ Returns an iterator over the :class:`Future` instances (possibly created by different :class:`Executor` instances) given by *fs* that yields futures as they complete (finished or were cancelled). Any futures given by *fs* that - are duplicated will be returned once. Any futures that completed - before :func:`as_completed` is called will be yielded first. The returned - iterator raises a :exc:`TimeoutError` if :meth:`~iterator.__next__` is - called and the result isn't available after *timeout* seconds from the - original call to :func:`as_completed`. *timeout* can be an int or float. - If *timeout* is not specified or ``None``, there is no limit to the wait - time. + are duplicated will be returned once. Any futures that completed before + :func:`as_completed` is called will be yielded first. The returned iterator + raises a :exc:`concurrent.futures.TimeoutError` if :meth:`~iterator.__next__` + is called and the result isn't available after *timeout* seconds from the + original call to :func:`as_completed`. *timeout* can be an int or float. If + *timeout* is not specified or ``None``, there is no limit to the wait time. .. seealso:: @@ -410,6 +411,16 @@ Exception classes ----------------- +.. currentmodule:: concurrent.futures + +.. exception:: CancelledError + + Raised when a future is cancelled. + +.. exception:: TimeoutError + + Raised when a future operation exceeds the given timeout. + .. currentmodule:: concurrent.futures.process .. exception:: BrokenProcessPool -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Jan 18 21:46:00 2016 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 19 Jan 2016 02:46:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160119024600.121300.84726@psf.io> https://hg.python.org/cpython/rev/2b0c7b67eca5 changeset: 99974:2b0c7b67eca5 parent: 99972:c44bfa733526 parent: 99973:673d1ccea050 user: Senthil Kumaran date: Mon Jan 18 18:45:54 2016 -0800 summary: merge from 3.5 issue23962 - Reference the correct TimeoutError in concurrent.futures documentation. Patch contributed by Ryder Lewis. files: Doc/library/concurrent.futures.rst | 43 +++++++++++------ 1 files changed, 27 insertions(+), 16 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -42,7 +42,7 @@ Equivalent to :func:`map(func, *iterables) ` except *func* is executed asynchronously and several calls to *func* may be made concurrently. The - returned iterator raises a :exc:`TimeoutError` if + returned iterator raises a :exc:`concurrent.futures.TimeoutError` if :meth:`~iterator.__next__` is called and the result isn't available after *timeout* seconds from the original call to :meth:`Executor.map`. *timeout* can be an int or a float. If *timeout* is not specified or @@ -274,11 +274,12 @@ Return the value returned by the call. If the call hasn't yet completed then this method will wait up to *timeout* seconds. If the call hasn't - completed in *timeout* seconds, then a :exc:`TimeoutError` will be - raised. *timeout* can be an int or float. If *timeout* is not specified - or ``None``, there is no limit to the wait time. + completed in *timeout* seconds, then a + :exc:`concurrent.futures.TimeoutError` will be raised. *timeout* can be + an int or float. If *timeout* is not specified or ``None``, there is no + limit to the wait time. - If the future is cancelled before completing then :exc:`CancelledError` + If the future is cancelled before completing then :exc:`.CancelledError` will be raised. If the call raised, this method will raise the same exception. @@ -287,11 +288,12 @@ Return the exception raised by the call. If the call hasn't yet completed then this method will wait up to *timeout* seconds. If the - call hasn't completed in *timeout* seconds, then a :exc:`TimeoutError` - will be raised. *timeout* can be an int or float. If *timeout* is not - specified or ``None``, there is no limit to the wait time. + call hasn't completed in *timeout* seconds, then a + :exc:`concurrent.futures.TimeoutError` will be raised. *timeout* can be + an int or float. If *timeout* is not specified or ``None``, there is no + limit to the wait time. - If the future is cancelled before completing then :exc:`CancelledError` + If the future is cancelled before completing then :exc:`.CancelledError` will be raised. If the call completed without raising, ``None`` is returned. @@ -391,13 +393,12 @@ Returns an iterator over the :class:`Future` instances (possibly created by different :class:`Executor` instances) given by *fs* that yields futures as they complete (finished or were cancelled). Any futures given by *fs* that - are duplicated will be returned once. Any futures that completed - before :func:`as_completed` is called will be yielded first. The returned - iterator raises a :exc:`TimeoutError` if :meth:`~iterator.__next__` is - called and the result isn't available after *timeout* seconds from the - original call to :func:`as_completed`. *timeout* can be an int or float. - If *timeout* is not specified or ``None``, there is no limit to the wait - time. + are duplicated will be returned once. Any futures that completed before + :func:`as_completed` is called will be yielded first. The returned iterator + raises a :exc:`concurrent.futures.TimeoutError` if :meth:`~iterator.__next__` + is called and the result isn't available after *timeout* seconds from the + original call to :func:`as_completed`. *timeout* can be an int or float. If + *timeout* is not specified or ``None``, there is no limit to the wait time. .. seealso:: @@ -410,6 +411,16 @@ Exception classes ----------------- +.. currentmodule:: concurrent.futures + +.. exception:: CancelledError + + Raised when a future is cancelled. + +.. exception:: TimeoutError + + Raised when a future operation exceeds the given timeout. + .. currentmodule:: concurrent.futures.process .. exception:: BrokenProcessPool -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 00:18:42 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 19 Jan 2016 05:18:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogc2V0IHRwX25ldyBm?= =?utf-8?q?rom_the_class_in_the_hierarchy_that_actually_owns_the_descripto?= =?utf-8?q?r?= Message-ID: <20160119051842.25979.49139@psf.io> https://hg.python.org/cpython/rev/3c9512d8ac0d changeset: 99975:3c9512d8ac0d branch: 3.5 parent: 99973:673d1ccea050 user: Benjamin Peterson date: Mon Jan 18 21:11:18 2016 -0800 summary: set tp_new from the class in the hierarchy that actually owns the descriptor (closes #25731) Debugging by Eryk Sun. files: Lib/test/test_descr.py | 8 ++++++++ Misc/NEWS | 2 ++ Objects/typeobject.c | 2 +- 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -4564,6 +4564,14 @@ self.assertRegex(repr(method), r">") + def test_deleting_new_in_subclasses(self): + class X: + def __init__(self, a): + pass + X.__new__ = None + del X.__new__ + X(1) # should work + class DictProxyTests(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #25731: Fix set and deleting __new__ on a class. + - Issue #22995: [UPDATE] Comment out the one of the pickleability tests in _PyObject_GetState() due to regressions observed in Cython-based projects. diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6777,7 +6777,7 @@ sanity checks and constructing a new argument list. Cut all that nonsense short -- this speeds up instance creation tremendously. */ - specific = (void *)type->tp_new; + specific = (void *)((PyTypeObject *)PyCFunction_GET_SELF(descr))->tp_new; /* XXX I'm not 100% sure that there isn't a hole in this reasoning that requires additional sanity checks. I'll buy the first person to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 00:18:43 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 19 Jan 2016 05:18:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjU3MzEp?= Message-ID: <20160119051843.94210.39202@psf.io> https://hg.python.org/cpython/rev/a7953ee29f1c changeset: 99977:a7953ee29f1c parent: 99974:2b0c7b67eca5 parent: 99975:3c9512d8ac0d user: Benjamin Peterson date: Mon Jan 18 21:18:35 2016 -0800 summary: merge 3.5 (#25731) files: Lib/test/test_descr.py | 8 ++++++++ Misc/NEWS | 2 ++ Objects/typeobject.c | 2 +- 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -4564,6 +4564,14 @@ self.assertRegex(repr(method), r">") + def test_deleting_new_in_subclasses(self): + class X: + def __init__(self, a): + pass + X.__new__ = None + del X.__new__ + X(1) # should work + class DictProxyTests(unittest.TestCase): def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,8 @@ - Issue #25791: Trying to resolve a relative import without __spec__ or __package__ defined now raises an ImportWarning +- Issue #25731: Fix set and deleting __new__ on a class. + - Issue #25961: Disallowed null characters in the type name. - Issue #25973: Fix segfault when an invalid nonlocal statement binds a name diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6791,7 +6791,7 @@ sanity checks and constructing a new argument list. Cut all that nonsense short -- this speeds up instance creation tremendously. */ - specific = (void *)type->tp_new; + specific = (void *)((PyTypeObject *)PyCFunction_GET_SELF(descr))->tp_new; /* XXX I'm not 100% sure that there isn't a hole in this reasoning that requires additional sanity checks. I'll buy the first person to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 00:18:43 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 19 Jan 2016 05:18:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogc2V0IHRwX25ldyBm?= =?utf-8?q?rom_the_class_in_the_hierarchy_that_actually_owns_the_descripto?= =?utf-8?q?r?= Message-ID: <20160119051842.94214.50341@psf.io> https://hg.python.org/cpython/rev/e7062dd9085e changeset: 99976:e7062dd9085e branch: 2.7 parent: 99962:04e95f05aafe user: Benjamin Peterson date: Mon Jan 18 21:17:54 2016 -0800 summary: set tp_new from the class in the hierarchy that actually owns the descriptor (closes #25731) Debugging by Eryk Sun. files: Objects/typeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6239,7 +6239,7 @@ sanity checks and constructing a new argument list. Cut all that nonsense short -- this speeds up instance creation tremendously. */ - specific = (void *)type->tp_new; + specific = (void *)((PyTypeObject *)PyCFunction_GET_SELF(descr))->tp_new; /* XXX I'm not 100% sure that there isn't a hole in this reasoning that requires additional sanity checks. I'll buy the first person to -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 00:18:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 19 Jan 2016 05:18:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_add_space?= Message-ID: <20160119051855.5455.78805@psf.io> https://hg.python.org/cpython/rev/b452c89bff53 changeset: 99978:b452c89bff53 branch: 2.7 parent: 99976:e7062dd9085e user: Benjamin Peterson date: Mon Jan 18 21:18:49 2016 -0800 summary: add space files: Lib/test/test_descr.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -4699,6 +4699,7 @@ for o in gc.get_objects(): self.assertIsNot(type(o), X) + class DictProxyTests(unittest.TestCase): def setUp(self): class C(object): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 02:51:18 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 19 Jan 2016 07:51:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_BytecodeTe?= =?utf-8?q?stCase=2EassertNotInBytecode=28=29?= Message-ID: <20160119075117.14181.99685@psf.io> https://hg.python.org/cpython/rev/bf997b22df06 changeset: 99979:bf997b22df06 branch: 3.5 parent: 99975:3c9512d8ac0d user: Victor Stinner date: Tue Jan 19 08:48:48 2016 +0100 summary: Fix BytecodeTestCase.assertNotInBytecode() Issue #11816: Fix bytecode_helper to handle correctly errors. Don't use unassigned variables. files: Lib/test/bytecode_helper.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/bytecode_helper.py b/Lib/test/bytecode_helper.py --- a/Lib/test/bytecode_helper.py +++ b/Lib/test/bytecode_helper.py @@ -32,8 +32,8 @@ """Throws AssertionError if op is found""" for instr in dis.get_instructions(x): if instr.opname == opname: - disassembly = self.get_disassembly_as_string(co) - if opargval is _UNSPECIFIED: + disassembly = self.get_disassembly_as_string(x) + if argval is _UNSPECIFIED: msg = '%s occurs in bytecode:\n%s' % (opname, disassembly) elif instr.argval == argval: msg = '(%s,%r) occurs in bytecode:\n%s' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 02:51:18 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 19 Jan 2016 07:51:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160119075118.96354.10043@psf.io> https://hg.python.org/cpython/rev/1f003062d830 changeset: 99980:1f003062d830 parent: 99977:a7953ee29f1c parent: 99979:bf997b22df06 user: Victor Stinner date: Tue Jan 19 08:50:56 2016 +0100 summary: Merge 3.5 files: Lib/test/bytecode_helper.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/bytecode_helper.py b/Lib/test/bytecode_helper.py --- a/Lib/test/bytecode_helper.py +++ b/Lib/test/bytecode_helper.py @@ -32,8 +32,8 @@ """Throws AssertionError if op is found""" for instr in dis.get_instructions(x): if instr.opname == opname: - disassembly = self.get_disassembly_as_string(co) - if opargval is _UNSPECIFIED: + disassembly = self.get_disassembly_as_string(x) + if argval is _UNSPECIFIED: msg = '%s occurs in bytecode:\n%s' % (opname, disassembly) elif instr.argval == argval: msg = '(%s,%r) occurs in bytecode:\n%s' -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Tue Jan 19 03:44:03 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 19 Jan 2016 08:44:03 +0000 Subject: [Python-checkins] Daily reference leaks (a7953ee29f1c): sum=4 Message-ID: <20160119084403.25916.55096@psf.io> results for a7953ee29f1c on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogB6U36b', '--timeout', '7200'] From python-checkins at python.org Tue Jan 19 07:10:35 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 19 Jan 2016 12:10:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE2NjIw?= =?utf-8?q?=3A_Fixed_AttributeError_in_msilib=2EDirectory=2Eglob=28=29=2E?= Message-ID: <20160119121034.25918.33242@psf.io> https://hg.python.org/cpython/rev/652a50208e89 changeset: 99981:652a50208e89 branch: 3.5 parent: 99979:bf997b22df06 user: Serhiy Storchaka date: Tue Jan 19 13:55:36 2016 +0200 summary: Issue #16620: Fixed AttributeError in msilib.Directory.glob(). files: Lib/msilib/__init__.py | 6 +++++- Misc/NEWS | 2 ++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -1,7 +1,11 @@ # Copyright (C) 2005 Martin v. L?wis # Licensed to PSF under a Contributor Agreement. from _msi import * -import os, string, re, sys +import glob +import os +import re +import string +import sys AMD64 = "AMD64" in sys.version Itanium = "Itanium" in sys.version diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,8 @@ Library ------- +- Issue #16620: Fixed AttributeError in msilib.Directory.glob(). + - Issue #26013: Added compatibility with broken protocol 2 pickles created in old Python 3 versions (3.4.3 and lower). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 07:10:35 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 19 Jan 2016 12:10:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE2NjIw?= =?utf-8?q?=3A_Fixed_AttributeError_in_msilib=2EDirectory=2Eglob=28=29=2E?= Message-ID: <20160119121034.18429.24013@psf.io> https://hg.python.org/cpython/rev/158ae15f5809 changeset: 99982:158ae15f5809 branch: 2.7 parent: 99978:b452c89bff53 user: Serhiy Storchaka date: Tue Jan 19 13:55:36 2016 +0200 summary: Issue #16620: Fixed AttributeError in msilib.Directory.glob(). files: Lib/msilib/__init__.py | 6 +++++- Misc/NEWS | 2 ++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -2,7 +2,11 @@ # Copyright (C) 2005 Martin v. L?wis # Licensed to PSF under a Contributor Agreement. from _msi import * -import os, string, re, sys +import glob +import os +import re +import string +import sys AMD64 = "AMD64" in sys.version Itanium = "Itanium" in sys.version diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,8 @@ Library ------- +- Issue #16620: Fixed AttributeError in msilib.Directory.glob(). + - Issue #21847: Fixed xmlrpclib on Unicode-disabled builds. - Issue #6500: Fixed infinite recursion in urllib2.Request.__getattr__(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 07:10:35 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 19 Jan 2016 12:10:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316620=3A_Got_rid_?= =?utf-8?q?of_using_undocumented_function_glob=2Eglob1=28=29=2E?= Message-ID: <20160119121035.14193.32057@psf.io> https://hg.python.org/cpython/rev/f2586c381b0b changeset: 99984:f2586c381b0b user: Serhiy Storchaka date: Tue Jan 19 14:09:33 2016 +0200 summary: Issue #16620: Got rid of using undocumented function glob.glob1(). files: Lib/msilib/__init__.py | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -1,7 +1,7 @@ # Copyright (C) 2005 Martin v. L?wis # Licensed to PSF under a Contributor Agreement. from _msi import * -import glob +import fnmatch import os import re import string @@ -379,7 +379,13 @@ def glob(self, pattern, exclude = None): """Add a list of files to the current component as specified in the glob pattern. Individual files can be excluded in the exclude list.""" - files = glob.glob1(self.absolute, pattern) + try: + files = os.listdir(self.absolute) + except OSError: + return [] + if pattern[:1] != '.': + files = (f for f in files if f[0] != '.') + files = fnmatch.filter(files, pattern) for f in files: if exclude and f in exclude: continue self.add_file(f) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 07:10:39 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 19 Jan 2016 12:10:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2316620=3A_Fixed_AttributeError_in_msilib=2EDirec?= =?utf-8?b?dG9yeS5nbG9iKCku?= Message-ID: <20160119121035.94208.90338@psf.io> https://hg.python.org/cpython/rev/84a50f14a266 changeset: 99983:84a50f14a266 parent: 99980:1f003062d830 parent: 99981:652a50208e89 user: Serhiy Storchaka date: Tue Jan 19 14:06:13 2016 +0200 summary: Issue #16620: Fixed AttributeError in msilib.Directory.glob(). files: Lib/msilib/__init__.py | 6 +++++- Misc/NEWS | 2 ++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -1,7 +1,11 @@ # Copyright (C) 2005 Martin v. L?wis # Licensed to PSF under a Contributor Agreement. from _msi import * -import os, string, re, sys +import glob +import os +import re +import string +import sys AMD64 = "AMD64" in sys.version Itanium = "Itanium" in sys.version diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -133,6 +133,8 @@ Library ------- +- Issue #16620: Fixed AttributeError in msilib.Directory.glob(). + - Issue #26013: Added compatibility with broken protocol 2 pickles created in old Python 3 versions (3.4.3 and lower). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 07:49:01 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 19 Jan 2016 12:49:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1OTM1?= =?utf-8?q?=3A_Garbage_collector_now_breaks_reference_loops_with_OrderedDi?= =?utf-8?b?Y3Qu?= Message-ID: <20160119124901.25902.30931@psf.io> https://hg.python.org/cpython/rev/627575e9a7aa changeset: 99985:627575e9a7aa branch: 3.5 parent: 99981:652a50208e89 user: Serhiy Storchaka date: Tue Jan 19 14:46:25 2016 +0200 summary: Issue #25935: Garbage collector now breaks reference loops with OrderedDict. files: Lib/test/test_ordered_dict.py | 13 +++++++++++ Misc/NEWS | 2 + Objects/odictobject.c | 27 +++++++++++----------- 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_ordered_dict.py b/Lib/test/test_ordered_dict.py --- a/Lib/test/test_ordered_dict.py +++ b/Lib/test/test_ordered_dict.py @@ -1,10 +1,12 @@ import contextlib import copy +import gc import pickle from random import randrange, shuffle import struct import sys import unittest +import weakref from collections.abc import MutableMapping from test import mapping_tests, support @@ -585,6 +587,17 @@ dict.update(od, [('spam', 1)]) self.assertNotIn('NULL', repr(od)) + def test_reference_loop(self): + # Issue 25935 + OrderedDict = self.OrderedDict + class A: + od = OrderedDict() + A.od[A] = None + r = weakref.ref(A) + del A + gc.collect() + self.assertIsNone(r()) + class PurePythonOrderedDictTests(OrderedDictTests, unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,8 @@ Library ------- +- Issue #25935: Garbage collector now breaks reference loops with OrderedDict. + - Issue #16620: Fixed AttributeError in msilib.Directory.glob(). - Issue #26013: Added compatibility with broken protocol 2 pickles created diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -772,19 +772,17 @@ { _ODictNode *node, *next; - if (!_odict_EMPTY(od)) { - node = _odict_FIRST(od); - while (node != NULL) { - next = _odictnode_NEXT(node); - _odictnode_DEALLOC(node); - node = next; - } - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; - } - _odict_free_fast_nodes(od); od->od_fast_nodes = NULL; + + node = _odict_FIRST(od); + _odict_FIRST(od) = NULL; + _odict_LAST(od) = NULL; + while (node != NULL) { + next = _odictnode_NEXT(node); + _odictnode_DEALLOC(node); + node = next; + } } /* There isn't any memory management of nodes past this point. */ @@ -1233,8 +1231,6 @@ { PyDict_Clear((PyObject *)od); _odict_clear_nodes(od); - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; if (_odict_resize(od) < 0) return NULL; Py_RETURN_NONE; @@ -1556,8 +1552,13 @@ static int odict_traverse(PyODictObject *od, visitproc visit, void *arg) { + _ODictNode *node; + Py_VISIT(od->od_inst_dict); Py_VISIT(od->od_weakreflist); + _odict_FOREACH(od, node) { + Py_VISIT(_odictnode_KEY(node)); + } return PyDict_Type.tp_traverse((PyObject *)od, visit, arg); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 07:49:13 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 19 Jan 2016 12:49:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325935=3A_Garbage_collector_now_breaks_reference?= =?utf-8?q?_loops_with_OrderedDict=2E?= Message-ID: <20160119124901.41316.53753@psf.io> https://hg.python.org/cpython/rev/5f0a08cfc4f6 changeset: 99986:5f0a08cfc4f6 parent: 99984:f2586c381b0b parent: 99985:627575e9a7aa user: Serhiy Storchaka date: Tue Jan 19 14:48:42 2016 +0200 summary: Issue #25935: Garbage collector now breaks reference loops with OrderedDict. files: Lib/test/test_ordered_dict.py | 13 +++++++++++ Misc/NEWS | 2 + Objects/odictobject.c | 27 +++++++++++----------- 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_ordered_dict.py b/Lib/test/test_ordered_dict.py --- a/Lib/test/test_ordered_dict.py +++ b/Lib/test/test_ordered_dict.py @@ -1,10 +1,12 @@ import contextlib import copy +import gc import pickle from random import randrange, shuffle import struct import sys import unittest +import weakref from collections.abc import MutableMapping from test import mapping_tests, support @@ -593,6 +595,17 @@ dict.update(od, [('spam', 1)]) self.assertNotIn('NULL', repr(od)) + def test_reference_loop(self): + # Issue 25935 + OrderedDict = self.OrderedDict + class A: + od = OrderedDict() + A.od[A] = None + r = weakref.ref(A) + del A + gc.collect() + self.assertIsNone(r()) + class PurePythonOrderedDictTests(OrderedDictTests, unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -133,6 +133,8 @@ Library ------- +- Issue #25935: Garbage collector now breaks reference loops with OrderedDict. + - Issue #16620: Fixed AttributeError in msilib.Directory.glob(). - Issue #26013: Added compatibility with broken protocol 2 pickles created diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -772,19 +772,17 @@ { _ODictNode *node, *next; - if (!_odict_EMPTY(od)) { - node = _odict_FIRST(od); - while (node != NULL) { - next = _odictnode_NEXT(node); - _odictnode_DEALLOC(node); - node = next; - } - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; - } - _odict_free_fast_nodes(od); od->od_fast_nodes = NULL; + + node = _odict_FIRST(od); + _odict_FIRST(od) = NULL; + _odict_LAST(od) = NULL; + while (node != NULL) { + next = _odictnode_NEXT(node); + _odictnode_DEALLOC(node); + node = next; + } } /* There isn't any memory management of nodes past this point. */ @@ -1233,8 +1231,6 @@ { PyDict_Clear((PyObject *)od); _odict_clear_nodes(od); - _odict_FIRST(od) = NULL; - _odict_LAST(od) = NULL; if (_odict_resize(od) < 0) return NULL; Py_RETURN_NONE; @@ -1556,8 +1552,13 @@ static int odict_traverse(PyODictObject *od, visitproc visit, void *arg) { + _ODictNode *node; + Py_VISIT(od->od_inst_dict); Py_VISIT(od->od_weakreflist); + _odict_FOREACH(od, node) { + Py_VISIT(_odictnode_KEY(node)); + } return PyDict_Type.tp_traverse((PyObject *)od, visit, arg); } -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Jan 19 10:54:38 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 19 Jan 2016 15:54:38 +0000 Subject: [Python-checkins] BAD Benchmark Results for Python Default 2016-01-19 Message-ID: Results for project Python default, build date 2016-01-19 03:07:50 +0000 commit: 2b0c7b67eca5 previous commit: bc625fce0aee revision date: 2016-01-19 02:45:54 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.28% 0.26% 10.64% 15.84% :-| pybench 0.13% 0.22% -1.47% 5.50% :-( regex_v8 2.66% -0.26% -4.84% 4.25% :-| nbody 0.07% -2.00% -0.88% 6.89% :-| json_dump_v2 0.16% 0.47% -1.06% 11.45% :-| normal_startup 0.79% 0.02% -0.19% 4.83% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-01-19/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Jan 19 10:55:44 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 19 Jan 2016 15:55:44 +0000 Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-01-19 Message-ID: Results for project Python 2.7, build date 2016-01-19 08:52:56 +0000 commit: 04e95f05aafe previous commit: d34fdd1736f2 revision date: 2016-01-18 17:38:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.08% -2.04% 3.44% 4.83% :-) pybench 0.19% 0.11% 6.39% 5.35% :-( regex_v8 0.90% -0.53% -2.82% 10.79% :-) nbody 0.26% 1.99% 9.25% 1.95% :-) json_dump_v2 0.27% -1.58% 4.01% 11.37% :-( normal_startup 1.69% 0.70% -5.13% 2.00% :-| ssbench 0.23% 0.55% 0.85% 0.92% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-01-19/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Jan 19 15:17:38 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 19 Jan 2016 20:17:38 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Minor_clarification_for_PEP_5?= =?utf-8?q?12?= Message-ID: <20160119201732.96350.71654@psf.io> https://hg.python.org/peps/rev/d31623fc0670 changeset: 6197:d31623fc0670 user: Brett Cannon date: Tue Jan 19 12:17:28 2016 -0800 summary: Minor clarification for PEP 512 files: pep-0007.txt | 6 +++--- pep-0512.txt | 4 +++- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/pep-0007.txt b/pep-0007.txt --- a/pep-0007.txt +++ b/pep-0007.txt @@ -75,9 +75,9 @@ } * Code structure: one space between keywords like ``if``, ``for`` and - the following left paren; no spaces inside the paren; braces may be - omitted where C permits but when present, they should be formatted - as shown:: + the following left paren; no spaces inside the paren; braces are + strongly preferred but may be omitted where C permits, and they + should be formatted as shown:: if (mro != NULL) { ... diff --git a/pep-0512.txt b/pep-0512.txt --- a/pep-0512.txt +++ b/pep-0512.txt @@ -233,7 +233,9 @@ expectation is that all pull requests will be fast-forwarded and rebased before being pushed to the master repository. This should give proper attribution to the pull request author in the Git -history. +history. This does have the consequence of losing some GitHub +features such as automatic closing of pull requests, link generation, +etc. A second set of recommended commands will also be written for committing a contribution from a patch file uploaded to -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 19 19:22:16 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 20 Jan 2016 00:22:16 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_post-history_for_PEP_5?= =?utf-8?q?12?= Message-ID: <20160120002214.111030.82349@psf.io> https://hg.python.org/peps/rev/6f55ccb2ab00 changeset: 6198:6f55ccb2ab00 user: Brett Cannon date: Tue Jan 19 16:22:10 2016 -0800 summary: Update post-history for PEP 512 files: pep-0512.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0512.txt b/pep-0512.txt --- a/pep-0512.txt +++ b/pep-0512.txt @@ -7,7 +7,7 @@ Type: Process Content-Type: text/x-rst Created: -Post-History: 17-Jan-2015 +Post-History: 17-Jan-2015, 19-Jan-2015 Abstract ======== -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Tue Jan 19 20:14:44 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 20 Jan 2016 01:14:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_update_to_pip/setuptools?= Message-ID: <20160120011444.18417.46688@psf.io> https://hg.python.org/cpython/rev/abb05272bc90 changeset: 99988:abb05272bc90 branch: 3.5 parent: 99985:627575e9a7aa parent: 99987:3b2910d2225d user: Donald Stufft date: Tue Jan 19 20:14:39 2016 -0500 summary: Merge update to pip/setuptools files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,9 +8,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "18.2" +_SETUPTOOLS_VERSION = "19.4" -_PIP_VERSION = "7.1.2" +_PIP_VERSION = "8.0.0" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl deleted file mode 100644 index 5e490155f0ca7f4ddb64c93c39fb2efb8795cd08..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4e685be865845ae4911efd1e5c04a87b5bd01c99 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl deleted file mode 100644 index f4288d68e074466894d8a2342e113737df7b7649..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b58455f31ed6d10d450e22ae9fa57023891f758f GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 20:15:39 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 20 Jan 2016 01:15:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_update_to_pip/setuptools?= Message-ID: <20160120011537.96376.77527@psf.io> https://hg.python.org/cpython/rev/19e67e78e20e changeset: 99989:19e67e78e20e parent: 99986:5f0a08cfc4f6 parent: 99988:abb05272bc90 user: Donald Stufft date: Tue Jan 19 20:15:33 2016 -0500 summary: Merge update to pip/setuptools files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,9 +8,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "18.2" +_SETUPTOOLS_VERSION = "19.4" -_PIP_VERSION = "7.1.2" +_PIP_VERSION = "8.0.0" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl deleted file mode 100644 index 5e490155f0ca7f4ddb64c93c39fb2efb8795cd08..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4e685be865845ae4911efd1e5c04a87b5bd01c99 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl deleted file mode 100644 index f4288d68e074466894d8a2342e113737df7b7649..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b58455f31ed6d10d450e22ae9fa57023891f758f GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 20:17:12 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 20 Jan 2016 01:17:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Upgrade_pip_to?= =?utf-8?q?_8=2E0=2E0_and_setuptools_to_19=2E4?= Message-ID: <20160120011712.25926.38088@psf.io> https://hg.python.org/cpython/rev/65b7eb7dfb64 changeset: 99990:65b7eb7dfb64 branch: 2.7 parent: 99982:158ae15f5809 user: Donald Stufft date: Tue Jan 19 20:17:03 2016 -0500 summary: Upgrade pip to 8.0.0 and setuptools to 19.4 files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -12,9 +12,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "18.2" +_SETUPTOOLS_VERSION = "19.4" -_PIP_VERSION = "7.1.2" +_PIP_VERSION = "8.0.0" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl deleted file mode 100644 index 5e490155f0ca7f4ddb64c93c39fb2efb8795cd08..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4e685be865845ae4911efd1e5c04a87b5bd01c99 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl deleted file mode 100644 index f4288d68e074466894d8a2342e113737df7b7649..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b58455f31ed6d10d450e22ae9fa57023891f758f GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Jan 19 20:25:03 2016 From: python-checkins at python.org (donald.stufft) Date: Wed, 20 Jan 2016 01:25:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Update_pip_to_?= =?utf-8?q?8=2E0=2E0_and_setuptools_to_19=2E4?= Message-ID: <20160120011405.121300.30149@psf.io> https://hg.python.org/cpython/rev/3b2910d2225d changeset: 99987:3b2910d2225d branch: 3.4 parent: 99913:0364c4a74b96 user: Donald Stufft date: Tue Jan 19 20:13:51 2016 -0500 summary: Update pip to 8.0.0 and setuptools to 19.4 files: Lib/ensurepip/__init__.py | 4 ++-- Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl | Bin 5 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,9 +8,9 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "18.2" +_SETUPTOOLS_VERSION = "19.4" -_PIP_VERSION = "7.1.2" +_PIP_VERSION = "8.0.0" # pip currently requires ssl support, so we try to provide a nicer # error message when that is missing (http://bugs.python.org/issue19744) diff --git a/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl deleted file mode 100644 index 5e490155f0ca7f4ddb64c93c39fb2efb8795cd08..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-8.0.0-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4e685be865845ae4911efd1e5c04a87b5bd01c99 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-18.2-py2.py3-none-any.whl deleted file mode 100644 index f4288d68e074466894d8a2342e113737df7b7649..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-19.4-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b58455f31ed6d10d450e22ae9fa57023891f758f GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 00:14:51 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 20 Jan 2016 05:14:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326157=3A_Fix_typos_in_asyncio-eventloop=2Erst?= Message-ID: <20160120051451.14177.65815@psf.io> https://hg.python.org/cpython/rev/67f562cbd87b changeset: 99992:67f562cbd87b parent: 99989:19e67e78e20e parent: 99991:7415a9386048 user: Berker Peksag date: Wed Jan 20 07:14:43 2016 +0200 summary: Issue #26157: Fix typos in asyncio-eventloop.rst Initial patch by Carlo Beccarini. files: Doc/library/asyncio-eventloop.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -739,11 +739,11 @@ Stop serving: close listening sockets and set the :attr:`sockets` attribute to ``None``. - The sockets that represent existing incoming client connections are - leaved open. + The sockets that represent existing incoming client connections are left + open. - The server is closed asynchonously, use the :meth:`wait_closed` coroutine - to wait until the server is closed. + The server is closed asynchronously, use the :meth:`wait_closed` + coroutine to wait until the server is closed. .. coroutinemethod:: wait_closed() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 00:14:54 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 20 Jan 2016 05:14:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MTU3?= =?utf-8?q?=3A_Fix_typos_in_asyncio-eventloop=2Erst?= Message-ID: <20160120051451.121310.15412@psf.io> https://hg.python.org/cpython/rev/7415a9386048 changeset: 99991:7415a9386048 branch: 3.5 parent: 99988:abb05272bc90 user: Berker Peksag date: Wed Jan 20 07:14:22 2016 +0200 summary: Issue #26157: Fix typos in asyncio-eventloop.rst Initial patch by Carlo Beccarini. files: Doc/library/asyncio-eventloop.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -739,11 +739,11 @@ Stop serving: close listening sockets and set the :attr:`sockets` attribute to ``None``. - The sockets that represent existing incoming client connections are - leaved open. + The sockets that represent existing incoming client connections are left + open. - The server is closed asynchonously, use the :meth:`wait_closed` coroutine - to wait until the server is closed. + The server is closed asynchronously, use the :meth:`wait_closed` + coroutine to wait until the server is closed. .. coroutinemethod:: wait_closed() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 01:12:36 2016 From: python-checkins at python.org (zach.ware) Date: Wed, 20 Jan 2016 06:12:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1OTI1?= =?utf-8?q?=3A_Backport_C_coverage_reporting_Makefile_targets?= Message-ID: <20160120061230.94216.68937@psf.io> https://hg.python.org/cpython/rev/ed7b602a7400 changeset: 99993:ed7b602a7400 branch: 2.7 parent: 99990:65b7eb7dfb64 user: Zachary Ware date: Wed Jan 20 00:11:52 2016 -0600 summary: Issue #25925: Backport C coverage reporting Makefile targets Originally added by Christian Heimes in 85ec2b5bfcd2. Initial patch for the backport by Alecsandru Patrascu. files: .hgignore | 5 ++++ Makefile.pre.in | 42 ++++++++++++++++++++++++++++++++++++- 2 files changed, 46 insertions(+), 1 deletions(-) diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -30,6 +30,7 @@ Modules/ld_so_aix$ Parser/pgen$ Parser/pgen.stamp$ +^lcov-report/ ^core ^python-gdb.py ^pybuilddir.txt @@ -81,6 +82,10 @@ PCbuild/*.pdb PCbuild/amd64 PCbuild/obj +*.gcda +*.gcno +*.gcov +coverage.info .coverage coverage/ externals/ diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -211,6 +211,11 @@ # The task to run while instrument when building the profile-opt target PROFILE_TASK=-m test.regrtest --pgo +# report files for gcov / lcov coverage report +COVERAGE_INFO= $(abs_builddir)/coverage.info +COVERAGE_REPORT=$(abs_builddir)/lcov-report +COVERAGE_REPORT_OPTIONS=--no-branch-coverage --title "CPython lcov report" + # === Definitions added by makesetup === @@ -457,11 +462,46 @@ build_all_use_profile: $(MAKE) all CFLAGS="$(CFLAGS) $(PGO_PROF_USE_FLAG)" +# Compile and run with gcov +.PHONY=coverage coverage-lcov coverage-report coverage: @echo "Building with support for coverage checking:" - $(MAKE) clean + $(MAKE) clean profile-removal $(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" +coverage-lcov: + @echo "Creating Coverage HTML report with LCOV:" + @rm -f $(COVERAGE_INFO) + @rm -rf $(COVERAGE_REPORT) + @lcov --capture --directory $(abs_builddir) \ + --base-directory $(realpath $(abs_builddir)) \ + --path $(realpath $(abs_srcdir)) \ + --output-file $(COVERAGE_INFO) + : # remove 3rd party modules and system headers + @lcov --remove $(COVERAGE_INFO) \ + '*/Modules/_ctypes/libffi*/*' \ + '*/Modules/expat/*' \ + '*/Modules/zlib/*' \ + '*/Include/*' \ + '/usr/include/*' \ + '/usr/local/include/*' \ + --output-file $(COVERAGE_INFO) + @genhtml $(COVERAGE_INFO) --output-directory $(COVERAGE_REPORT) \ + $(COVERAGE_REPORT_OPTIONS) + @echo + @echo "lcov report at $(COVERAGE_REPORT)/index.html" + @echo + +coverage-report: + : # force rebuilding of parser + @touch $(GRAMMAR_INPUT) + : # build with coverage info + $(MAKE) coverage + : # run tests, ignore failures + $(TESTPYTHON) $(TESTPROG) $(TESTOPTS) || true + : # build lcov report + $(MAKE) coverage-lcov + # Build the interpreter $(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 01:45:58 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 20 Jan 2016 06:45:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=235626=3A_Remove_misleading_comment_from_socket?= =?utf-8?q?=2Egethostname=28=29_documentation?= Message-ID: <20160120064558.25985.55092@psf.io> https://hg.python.org/cpython/rev/689f881dd5d1 changeset: 99995:689f881dd5d1 parent: 99992:67f562cbd87b parent: 99994:e1a711808cfe user: Berker Peksag date: Wed Jan 20 08:45:54 2016 +0200 summary: Issue #5626: Remove misleading comment from socket.gethostname() documentation A machine can have more than one IP addresses so socket.gethostbyname(socket.gethostname()) probably won't return the correct one. files: Doc/library/socket.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -568,11 +568,6 @@ Return a string containing the hostname of the machine where the Python interpreter is currently executing. - If you want to know the current machine's IP address, you may want to use - ``gethostbyname(gethostname())``. This operation assumes that there is a - valid address-to-host mapping for the host, and the assumption does not - always hold. - Note: :func:`gethostname` doesn't always return the fully qualified domain name; use :func:`getfqdn` for that. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 01:46:03 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 20 Jan 2016 06:46:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzU2MjY6?= =?utf-8?q?_Remove_misleading_comment_from_socket=2Egethostname=28=29_docu?= =?utf-8?q?mentation?= Message-ID: <20160120064558.41314.53070@psf.io> https://hg.python.org/cpython/rev/e1a711808cfe changeset: 99994:e1a711808cfe branch: 3.5 parent: 99991:7415a9386048 user: Berker Peksag date: Wed Jan 20 08:45:37 2016 +0200 summary: Issue #5626: Remove misleading comment from socket.gethostname() documentation A machine can have more than one IP addresses so socket.gethostbyname(socket.gethostname()) probably won't return the correct one. files: Doc/library/socket.rst | 5 ----- 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -568,11 +568,6 @@ Return a string containing the hostname of the machine where the Python interpreter is currently executing. - If you want to know the current machine's IP address, you may want to use - ``gethostbyname(gethostname())``. This operation assumes that there is a - valid address-to-host mapping for the host, and the assumption does not - always hold. - Note: :func:`gethostname` doesn't always return the fully qualified domain name; use :func:`getfqdn` for that. -- Repository URL: https://hg.python.org/cpython From solipsis at pitrou.net Wed Jan 20 03:43:54 2016 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 20 Jan 2016 08:43:54 +0000 Subject: [Python-checkins] Daily reference leaks (67f562cbd87b): sum=4 Message-ID: <20160120084349.96348.91056@psf.io> results for 67f562cbd87b on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflog1KNEMD', '--timeout', '7200'] From python-checkins at python.org Wed Jan 20 04:09:59 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 20 Jan 2016 09:09:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MTQ3?= =?utf-8?q?=3A_xmlrpc_now_works_with_strings_not_encodable_with_used?= Message-ID: <20160120090959.18429.49943@psf.io> https://hg.python.org/cpython/rev/6c624ba1b61e changeset: 99997:6c624ba1b61e branch: 3.5 parent: 99994:e1a711808cfe user: Serhiy Storchaka date: Wed Jan 20 10:34:27 2016 +0200 summary: Issue #26147: xmlrpc now works with strings not encodable with used non-UTF-8 encoding. files: Lib/test/test_xmlrpc.py | 30 +++++++++++++++++++++++----- Lib/xmlrpc/client.py | 4 +-- Lib/xmlrpc/server.py | 4 +- Misc/NEWS | 3 ++ 4 files changed, 30 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -184,19 +184,26 @@ self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,)) def test_dump_encoding(self): - value = '\u20ac' + value = {'key\u20ac\xa4': + 'value\u20ac\xa4'} strg = xmlrpclib.dumps((value,), encoding='iso-8859-15') strg = "" + strg self.assertEqual(xmlrpclib.loads(strg)[0][0], value) - strg = strg.encode('iso-8859-15') + strg = strg.encode('iso-8859-15', 'xmlcharrefreplace') self.assertEqual(xmlrpclib.loads(strg)[0][0], value) strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', methodresponse=True) self.assertEqual(xmlrpclib.loads(strg)[0][0], value) - strg = strg.encode('iso-8859-15') + strg = strg.encode('iso-8859-15', 'xmlcharrefreplace') self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + methodname = 'method\u20ac\xa4' + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', + methodname=methodname) + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + self.assertEqual(xmlrpclib.loads(strg)[1], methodname) + def test_dump_bytes(self): sample = b"my dog has fleas" self.assertEqual(sample, xmlrpclib.Binary(sample)) @@ -430,6 +437,7 @@ serv.register_multicall_functions() serv.register_function(pow) serv.register_function(lambda x,y: x+y, 'add') + serv.register_function(lambda x: x, 't??t') serv.register_function(my_function) testInstance = TestInstanceClass() serv.register_instance(testInstance, allow_dotted_names=True) @@ -599,7 +607,7 @@ def test_client_encoding(self): start_string = '\u20ac' - end_string = '\xa3' + end_string = '\xa4' try: p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') @@ -611,6 +619,16 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + def test_nonascii_methodname(self): + try: + p = xmlrpclib.ServerProxy(URL, encoding='ascii') + self.assertEqual(p.t??t(42), 42) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): # send POST with http.client, it should return 404 header and @@ -624,7 +642,7 @@ self.assertEqual(response.reason, 'Not Found') def test_introspection1(self): - expected_methods = set(['pow', 'div', 'my_function', 'add', + expected_methods = set(['pow', 'div', 'my_function', 'add', 't??t', 'system.listMethods', 'system.methodHelp', 'system.methodSignature', 'system.multicall', 'Fixture']) @@ -767,7 +785,7 @@ def test_server_encoding(self): start_string = '\u20ac' - end_string = '\xa3' + end_string = '\xa4' try: p = xmlrpclib.ServerProxy(URL) diff --git a/Lib/xmlrpc/client.py b/Lib/xmlrpc/client.py --- a/Lib/xmlrpc/client.py +++ b/Lib/xmlrpc/client.py @@ -955,8 +955,6 @@ # standard XML-RPC wrappings if methodname: # a method call - if not isinstance(methodname, str): - methodname = methodname.encode(encoding) data = ( xmlheader, "\n" @@ -1422,7 +1420,7 @@ # call a method on the remote server request = dumps(params, methodname, encoding=self.__encoding, - allow_none=self.__allow_none).encode(self.__encoding) + allow_none=self.__allow_none).encode(self.__encoding, 'xmlcharrefreplace') response = self.__transport.request( self.__host, diff --git a/Lib/xmlrpc/server.py b/Lib/xmlrpc/server.py --- a/Lib/xmlrpc/server.py +++ b/Lib/xmlrpc/server.py @@ -269,7 +269,7 @@ encoding=self.encoding, allow_none=self.allow_none, ) - return response.encode(self.encoding) + return response.encode(self.encoding, 'xmlcharrefreplace') def system_listMethods(self): """system.listMethods() => ['add', 'subtract', 'multiple'] @@ -622,7 +622,7 @@ response = dumps( Fault(1, "%s:%s" % (exc_type, exc_value)), encoding=self.encoding, allow_none=self.allow_none) - response = response.encode(self.encoding) + response = response.encode(self.encoding, 'xmlcharrefreplace') return response class CGIXMLRPCRequestHandler(SimpleXMLRPCDispatcher): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #26147: xmlrpc now works with strings not encodable with used + non-UTF-8 encoding. + - Issue #25935: Garbage collector now breaks reference loops with OrderedDict. - Issue #16620: Fixed AttributeError in msilib.Directory.glob(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 04:09:59 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 20 Jan 2016 09:09:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MTQ3?= =?utf-8?q?=3A_xmlrpclib_now_works_with_unicode_not_encodable_with_used?= Message-ID: <20160120090959.14185.790@psf.io> https://hg.python.org/cpython/rev/bb2cb184eca9 changeset: 99996:bb2cb184eca9 branch: 2.7 parent: 99993:ed7b602a7400 user: Serhiy Storchaka date: Wed Jan 20 10:33:51 2016 +0200 summary: Issue #26147: xmlrpclib now works with unicode not encodable with used non-UTF-8 encoding. files: Lib/test/test_xmlrpc.py | 27 ++++++++++++++++++++++++--- Lib/xmlrpclib.py | 18 +++++++++--------- Misc/NEWS | 3 +++ 3 files changed, 36 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -149,7 +149,8 @@ @test_support.requires_unicode def test_dump_encoding(self): - value = unichr(0x20ac) + value = {test_support.u(r'key\u20ac\xa4'): + test_support.u(r'value\u20ac\xa4')} strg = xmlrpclib.dumps((value,), encoding='iso-8859-15') strg = "" + strg self.assertEqual(xmlrpclib.loads(strg)[0][0], value) @@ -158,6 +159,12 @@ methodresponse=True) self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + methodname = test_support.u(r'method\u20ac\xa4') + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', + methodname=methodname) + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + self.assertEqual(xmlrpclib.loads(strg)[1], methodname) + @test_support.requires_unicode def test_default_encoding_issues(self): # SF bug #1115989: wrong decoding in '_stringify' @@ -332,6 +339,7 @@ serv.register_multicall_functions() serv.register_function(pow) serv.register_function(lambda x,y: x+y, 'add') + serv.register_function(lambda x: x, test_support.u(r't\xea\u0161t')) serv.register_function(my_function) serv.register_instance(TestInstanceClass()) evt.set() @@ -496,7 +504,7 @@ @test_support.requires_unicode def test_client_encoding(self): start_string = unichr(0x20ac) - end_string = unichr(0xa3) + end_string = unichr(0xa4) try: p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') @@ -508,6 +516,18 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + @test_support.requires_unicode + def test_nonascii_methodname(self): + try: + p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') + m = getattr(p, 't\xea\xa8t') + self.assertEqual(m(42), 42) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): # send POST with httplib, it should return 404 header and @@ -525,6 +545,7 @@ p = xmlrpclib.ServerProxy(URL) meth = p.system.listMethods() expected_methods = set(['pow', 'div', 'my_function', 'add', + test_support.u(r't\xea\u0161t'), 'system.listMethods', 'system.methodHelp', 'system.methodSignature', 'system.multicall']) self.assertEqual(set(meth), expected_methods) @@ -635,7 +656,7 @@ @test_support.requires_unicode def test_server_encoding(self): start_string = unichr(0x20ac) - end_string = unichr(0xa3) + end_string = unichr(0xa4) try: p = xmlrpclib.ServerProxy(URL) diff --git a/Lib/xmlrpclib.py b/Lib/xmlrpclib.py --- a/Lib/xmlrpclib.py +++ b/Lib/xmlrpclib.py @@ -703,9 +703,8 @@ if unicode: def dump_unicode(self, value, write, escape=escape): - value = value.encode(self.encoding) write("") - write(escape(value)) + write(escape(value).encode(self.encoding, 'xmlcharrefreplace')) write("\n") dispatch[UnicodeType] = dump_unicode @@ -732,12 +731,13 @@ write("\n") for k, v in value.items(): write("\n") - if type(k) is not StringType: - if unicode and type(k) is UnicodeType: - k = k.encode(self.encoding) - else: - raise TypeError, "dictionary key must be string" - write("%s\n" % escape(k)) + if type(k) is StringType: + k = escape(k) + elif unicode and type(k) is UnicodeType: + k = escape(k).encode(self.encoding, 'xmlcharrefreplace') + else: + raise TypeError, "dictionary key must be string" + write("%s\n" % k) dump(v, write) write("\n") write("\n") @@ -1099,7 +1099,7 @@ if methodname: # a method call if not isinstance(methodname, StringType): - methodname = methodname.encode(encoding) + methodname = methodname.encode(encoding, 'xmlcharrefreplace') data = ( xmlheader, "\n" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #26147: xmlrpclib now works with unicode not encodable with used + non-UTF-8 encoding. + - Issue #16620: Fixed AttributeError in msilib.Directory.glob(). - Issue #21847: Fixed xmlrpclib on Unicode-disabled builds. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 04:10:01 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 20 Jan 2016 09:10:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326147=3A_xmlrpc_now_works_with_strings_not_enco?= =?utf-8?q?dable_with_used?= Message-ID: <20160120090959.96356.46862@psf.io> https://hg.python.org/cpython/rev/72034327022e changeset: 99998:72034327022e parent: 99995:689f881dd5d1 parent: 99997:6c624ba1b61e user: Serhiy Storchaka date: Wed Jan 20 10:41:33 2016 +0200 summary: Issue #26147: xmlrpc now works with strings not encodable with used non-UTF-8 encoding. files: Lib/test/test_xmlrpc.py | 30 +++++++++++++++++++++++----- Lib/xmlrpc/client.py | 4 +-- Lib/xmlrpc/server.py | 4 +- Misc/NEWS | 3 ++ 4 files changed, 30 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -184,19 +184,26 @@ self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,)) def test_dump_encoding(self): - value = '\u20ac' + value = {'key\u20ac\xa4': + 'value\u20ac\xa4'} strg = xmlrpclib.dumps((value,), encoding='iso-8859-15') strg = "" + strg self.assertEqual(xmlrpclib.loads(strg)[0][0], value) - strg = strg.encode('iso-8859-15') + strg = strg.encode('iso-8859-15', 'xmlcharrefreplace') self.assertEqual(xmlrpclib.loads(strg)[0][0], value) strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', methodresponse=True) self.assertEqual(xmlrpclib.loads(strg)[0][0], value) - strg = strg.encode('iso-8859-15') + strg = strg.encode('iso-8859-15', 'xmlcharrefreplace') self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + methodname = 'method\u20ac\xa4' + strg = xmlrpclib.dumps((value,), encoding='iso-8859-15', + methodname=methodname) + self.assertEqual(xmlrpclib.loads(strg)[0][0], value) + self.assertEqual(xmlrpclib.loads(strg)[1], methodname) + def test_dump_bytes(self): sample = b"my dog has fleas" self.assertEqual(sample, xmlrpclib.Binary(sample)) @@ -430,6 +437,7 @@ serv.register_multicall_functions() serv.register_function(pow) serv.register_function(lambda x,y: x+y, 'add') + serv.register_function(lambda x: x, 't??t') serv.register_function(my_function) testInstance = TestInstanceClass() serv.register_instance(testInstance, allow_dotted_names=True) @@ -599,7 +607,7 @@ def test_client_encoding(self): start_string = '\u20ac' - end_string = '\xa3' + end_string = '\xa4' try: p = xmlrpclib.ServerProxy(URL, encoding='iso-8859-15') @@ -611,6 +619,16 @@ # protocol error; provide additional information in test output self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + def test_nonascii_methodname(self): + try: + p = xmlrpclib.ServerProxy(URL, encoding='ascii') + self.assertEqual(p.t??t(42), 42) + except (xmlrpclib.ProtocolError, socket.error) as e: + # ignore failures due to non-blocking socket unavailable errors. + if not is_unavailable_exception(e): + # protocol error; provide additional information in test output + self.fail("%s\n%s" % (e, getattr(e, "headers", ""))) + # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): # send POST with http.client, it should return 404 header and @@ -624,7 +642,7 @@ self.assertEqual(response.reason, 'Not Found') def test_introspection1(self): - expected_methods = set(['pow', 'div', 'my_function', 'add', + expected_methods = set(['pow', 'div', 'my_function', 'add', 't??t', 'system.listMethods', 'system.methodHelp', 'system.methodSignature', 'system.multicall', 'Fixture']) @@ -767,7 +785,7 @@ def test_server_encoding(self): start_string = '\u20ac' - end_string = '\xa3' + end_string = '\xa4' try: p = xmlrpclib.ServerProxy(URL) diff --git a/Lib/xmlrpc/client.py b/Lib/xmlrpc/client.py --- a/Lib/xmlrpc/client.py +++ b/Lib/xmlrpc/client.py @@ -955,8 +955,6 @@ # standard XML-RPC wrappings if methodname: # a method call - if not isinstance(methodname, str): - methodname = methodname.encode(encoding) data = ( xmlheader, "\n" @@ -1422,7 +1420,7 @@ # call a method on the remote server request = dumps(params, methodname, encoding=self.__encoding, - allow_none=self.__allow_none).encode(self.__encoding) + allow_none=self.__allow_none).encode(self.__encoding, 'xmlcharrefreplace') response = self.__transport.request( self.__host, diff --git a/Lib/xmlrpc/server.py b/Lib/xmlrpc/server.py --- a/Lib/xmlrpc/server.py +++ b/Lib/xmlrpc/server.py @@ -269,7 +269,7 @@ encoding=self.encoding, allow_none=self.allow_none, ) - return response.encode(self.encoding) + return response.encode(self.encoding, 'xmlcharrefreplace') def system_listMethods(self): """system.listMethods() => ['add', 'subtract', 'multiple'] @@ -622,7 +622,7 @@ response = dumps( Fault(1, "%s:%s" % (exc_type, exc_value)), encoding=self.encoding, allow_none=self.allow_none) - response = response.encode(self.encoding) + response = response.encode(self.encoding, 'xmlcharrefreplace') return response class CGIXMLRPCRequestHandler(SimpleXMLRPCDispatcher): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -133,6 +133,9 @@ Library ------- +- Issue #26147: xmlrpc now works with strings not encodable with used + non-UTF-8 encoding. + - Issue #25935: Garbage collector now breaks reference loops with OrderedDict. - Issue #16620: Fixed AttributeError in msilib.Directory.glob(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 05:20:37 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 10:20:37 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQWRkIF9QeVRocmVh?= =?utf-8?q?dState=5FUncheckedGet=28=29?= Message-ID: <20160120102036.41348.19120@psf.io> https://hg.python.org/cpython/rev/f9461f1e0559 changeset: 99999:f9461f1e0559 branch: 3.5 parent: 99997:6c624ba1b61e user: Victor Stinner date: Wed Jan 20 11:12:38 2016 +0100 summary: Add _PyThreadState_UncheckedGet() Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which gets the current thread state, but don't call Py_FatalError() if it is NULL. Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to no more expose complex and private atomic types. Atomic types depends on the compiler or can even depend on compiler options. The new function _PyThreadState_UncheckedGet() allows to get the variable value without having to care of the exact implementation of atomic types. Changes: * Replace direct usage of the _PyThreadState_Current variable with a call to _PyThreadState_UncheckedGet(). * In pystate.c, replace direct usage of the _PyThreadState_Current variable with the PyThreadState_GET() macro for readability. * Document also PyThreadState_Get() in pystate.h files: Include/pystate.h | 10 +++++++++ Misc/NEWS | 7 ++++++ Modules/faulthandler.c | 2 +- Objects/dictobject.c | 6 +--- Python/errors.c | 8 +------ Python/pystate.c | 33 ++++++++++++++++++----------- Python/sysmodule.c | 2 +- 7 files changed, 42 insertions(+), 26 deletions(-) diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -168,7 +168,17 @@ PyAPI_FUNC(void) _PyGILState_Reinit(void); #endif +/* Return the current thread state. The global interpreter lock must be held. + * When the current thread state is NULL, this issues a fatal error (so that + * the caller needn't check for NULL). */ PyAPI_FUNC(PyThreadState *) PyThreadState_Get(void); + +#ifdef WITH_THREAD +/* Similar to PyThreadState_Get(), but don't issue a fatal error + * if it is NULL. */ +PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void); +#endif + PyAPI_FUNC(PyThreadState *) PyThreadState_Swap(PyThreadState *); PyAPI_FUNC(PyObject *) PyThreadState_GetDict(void); PyAPI_FUNC(int) PyThreadState_SetAsyncExc(long, PyObject *); diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,13 @@ Core and Builtins ----------------- +- Issue #26154: Add a new private _PyThreadState_UncheckedGet() function to get + the current Python thread state, but don't issue a fatal error if it is NULL. + This new function must be used instead of accessing directly the + _PyThreadState_Current variable. The variable is no more exposed since + Python 3.5.1 to hide the exact implementation of atomic C types, to avoid + compiler issues. + - Issue #25731: Fix set and deleting __new__ on a class. - Issue #22995: [UPDATE] Comment out the one of the pickleability tests in diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -490,7 +490,7 @@ assert(st == PY_LOCK_FAILURE); /* get the thread holding the GIL, NULL if no thread hold the GIL */ - current = (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current); + current = _PyThreadState_UncheckedGet(); _Py_write_noraise(thread.fd, thread.header, (int)thread.header_len); diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1064,8 +1064,7 @@ Let's just hope that no exception occurs then... This must be _PyThreadState_Current and not PyThreadState_GET() because in debug mode, the latter complains if tstate is NULL. */ - tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + tstate = _PyThreadState_UncheckedGet(); if (tstate != NULL && tstate->curexc_type != NULL) { /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; @@ -1102,8 +1101,7 @@ Let's just hope that no exception occurs then... This must be _PyThreadState_Current and not PyThreadState_GET() because in debug mode, the latter complains if tstate is NULL. */ - tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + tstate = _PyThreadState_UncheckedGet(); if (tstate != NULL && tstate->curexc_type != NULL) { /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; diff --git a/Python/errors.c b/Python/errors.c --- a/Python/errors.c +++ b/Python/errors.c @@ -152,13 +152,7 @@ PyObject * PyErr_Occurred(void) { - /* If there is no thread state, PyThreadState_GET calls - Py_FatalError, which calls PyErr_Occurred. To avoid the - resulting infinite loop, we inline PyThreadState_GET here and - treat no thread as no error. */ - PyThreadState *tstate = - ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)); - + PyThreadState *tstate = _PyThreadState_UncheckedGet(); return tstate == NULL ? NULL : tstate->curexc_type; } diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -3,6 +3,12 @@ #include "Python.h" +#ifndef Py_BUILD_CORE +/* ensure that PyThreadState_GET() is a macro, not an alias to + * PyThreadState_Get() */ +# error "pystate.c must be compiled with Py_BUILD_CORE defined" +#endif + /* -------------------------------------------------------------------------- CAUTION @@ -423,7 +429,7 @@ void PyThreadState_Delete(PyThreadState *tstate) { - if (tstate == (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) + if (tstate == PyThreadState_GET()) Py_FatalError("PyThreadState_Delete: tstate is still current"); #ifdef WITH_THREAD if (autoInterpreterState && PyThread_get_key_value(autoTLSkey) == tstate) @@ -437,8 +443,7 @@ void PyThreadState_DeleteCurrent() { - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); if (tstate == NULL) Py_FatalError( "PyThreadState_DeleteCurrent: no current tstate"); @@ -489,10 +494,16 @@ PyThreadState * +_PyThreadState_UncheckedGet(void) +{ + return PyThreadState_GET(); +} + + +PyThreadState * PyThreadState_Get(void) { - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); if (tstate == NULL) Py_FatalError("PyThreadState_Get: no current thread"); @@ -503,8 +514,7 @@ PyThreadState * PyThreadState_Swap(PyThreadState *newts) { - PyThreadState *oldts = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *oldts = PyThreadState_GET(); _Py_atomic_store_relaxed(&_PyThreadState_Current, newts); /* It should not be possible for more than one thread state @@ -535,8 +545,7 @@ PyObject * PyThreadState_GetDict(void) { - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); if (tstate == NULL) return NULL; @@ -682,7 +691,7 @@ { /* Must be the tstate for this thread */ assert(PyGILState_GetThisThreadState()==tstate); - return tstate == (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current); + return tstate == PyThreadState_GET(); } /* Internal initialization/finalization functions called by @@ -774,9 +783,7 @@ int PyGILState_Check(void) { - /* can't use PyThreadState_Get() since it will assert that it has the GIL */ - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); return tstate && (tstate == PyGILState_GetThisThreadState()); } diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1397,7 +1397,7 @@ Py_XDECREF(name); Py_XDECREF(value); /* No return value, therefore clear error state if possible */ - if (_Py_atomic_load_relaxed(&_PyThreadState_Current)) + if (_PyThreadState_UncheckedGet()) PyErr_Clear(); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 05:20:43 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 10:20:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160120102041.41340.10251@psf.io> https://hg.python.org/cpython/rev/d4f13c9a2b07 changeset: 100000:d4f13c9a2b07 parent: 99998:72034327022e parent: 99999:f9461f1e0559 user: Victor Stinner date: Wed Jan 20 11:19:46 2016 +0100 summary: Merge 3.5 Issue #26154: Add a new private _PyThreadState_UncheckedGet() function. files: Include/pystate.h | 10 +++++++++ Misc/NEWS | 7 ++++++ Modules/faulthandler.c | 2 +- Objects/dictobject.c | 6 +--- Python/errors.c | 8 +------ Python/pystate.c | 33 ++++++++++++++++++----------- Python/sysmodule.c | 2 +- 7 files changed, 42 insertions(+), 26 deletions(-) diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -168,7 +168,17 @@ PyAPI_FUNC(void) _PyGILState_Reinit(void); #endif +/* Return the current thread state. The global interpreter lock must be held. + * When the current thread state is NULL, this issues a fatal error (so that + * the caller needn't check for NULL). */ PyAPI_FUNC(PyThreadState *) PyThreadState_Get(void); + +#ifdef WITH_THREAD +/* Similar to PyThreadState_Get(), but don't issue a fatal error + * if it is NULL. */ +PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void); +#endif + PyAPI_FUNC(PyThreadState *) PyThreadState_Swap(PyThreadState *); PyAPI_FUNC(PyObject *) PyThreadState_GetDict(void); PyAPI_FUNC(int) PyThreadState_SetAsyncExc(long, PyObject *); diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,13 @@ Core and Builtins ----------------- +- Issue #26154: Add a new private _PyThreadState_UncheckedGet() function to get + the current Python thread state, but don't issue a fatal error if it is NULL. + This new function must be used instead of accessing directly the + _PyThreadState_Current variable. The variable is no more exposed since + Python 3.5.1 to hide the exact implementation of atomic C types, to avoid + compiler issues. + - Issue #25791: Trying to resolve a relative import without __spec__ or __package__ defined now raises an ImportWarning diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -490,7 +490,7 @@ assert(st == PY_LOCK_FAILURE); /* get the thread holding the GIL, NULL if no thread hold the GIL */ - current = (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current); + current = _PyThreadState_UncheckedGet(); _Py_write_noraise(thread.fd, thread.header, (int)thread.header_len); diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1064,8 +1064,7 @@ Let's just hope that no exception occurs then... This must be _PyThreadState_Current and not PyThreadState_GET() because in debug mode, the latter complains if tstate is NULL. */ - tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + tstate = _PyThreadState_UncheckedGet(); if (tstate != NULL && tstate->curexc_type != NULL) { /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; @@ -1102,8 +1101,7 @@ Let's just hope that no exception occurs then... This must be _PyThreadState_Current and not PyThreadState_GET() because in debug mode, the latter complains if tstate is NULL. */ - tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + tstate = _PyThreadState_UncheckedGet(); if (tstate != NULL && tstate->curexc_type != NULL) { /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; diff --git a/Python/errors.c b/Python/errors.c --- a/Python/errors.c +++ b/Python/errors.c @@ -152,13 +152,7 @@ PyObject * PyErr_Occurred(void) { - /* If there is no thread state, PyThreadState_GET calls - Py_FatalError, which calls PyErr_Occurred. To avoid the - resulting infinite loop, we inline PyThreadState_GET here and - treat no thread as no error. */ - PyThreadState *tstate = - ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)); - + PyThreadState *tstate = _PyThreadState_UncheckedGet(); return tstate == NULL ? NULL : tstate->curexc_type; } diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -3,6 +3,12 @@ #include "Python.h" +#ifndef Py_BUILD_CORE +/* ensure that PyThreadState_GET() is a macro, not an alias to + * PyThreadState_Get() */ +# error "pystate.c must be compiled with Py_BUILD_CORE defined" +#endif + /* -------------------------------------------------------------------------- CAUTION @@ -423,7 +429,7 @@ void PyThreadState_Delete(PyThreadState *tstate) { - if (tstate == (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) + if (tstate == PyThreadState_GET()) Py_FatalError("PyThreadState_Delete: tstate is still current"); #ifdef WITH_THREAD if (autoInterpreterState && PyThread_get_key_value(autoTLSkey) == tstate) @@ -437,8 +443,7 @@ void PyThreadState_DeleteCurrent() { - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); if (tstate == NULL) Py_FatalError( "PyThreadState_DeleteCurrent: no current tstate"); @@ -489,10 +494,16 @@ PyThreadState * +_PyThreadState_UncheckedGet(void) +{ + return PyThreadState_GET(); +} + + +PyThreadState * PyThreadState_Get(void) { - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); if (tstate == NULL) Py_FatalError("PyThreadState_Get: no current thread"); @@ -503,8 +514,7 @@ PyThreadState * PyThreadState_Swap(PyThreadState *newts) { - PyThreadState *oldts = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *oldts = PyThreadState_GET(); _Py_atomic_store_relaxed(&_PyThreadState_Current, newts); /* It should not be possible for more than one thread state @@ -535,8 +545,7 @@ PyObject * PyThreadState_GetDict(void) { - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); if (tstate == NULL) return NULL; @@ -682,7 +691,7 @@ { /* Must be the tstate for this thread */ assert(PyGILState_GetThisThreadState()==tstate); - return tstate == (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current); + return tstate == PyThreadState_GET(); } /* Internal initialization/finalization functions called by @@ -774,9 +783,7 @@ int PyGILState_Check(void) { - /* can't use PyThreadState_Get() since it will assert that it has the GIL */ - PyThreadState *tstate = (PyThreadState*)_Py_atomic_load_relaxed( - &_PyThreadState_Current); + PyThreadState *tstate = PyThreadState_GET(); return tstate && (tstate == PyGILState_GetThisThreadState()); } diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1396,7 +1396,7 @@ Py_XDECREF(name); Py_XDECREF(value); /* No return value, therefore clear error state if possible */ - if (_Py_atomic_load_relaxed(&_PyThreadState_Current)) + if (_PyThreadState_UncheckedGet()) PyErr_Clear(); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 06:12:07 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 20 Jan 2016 11:12:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E5?= Message-ID: <20160120111157.25987.13144@psf.io> https://hg.python.org/cpython/rev/f97da0952a2e changeset: 100002:f97da0952a2e parent: 100000:d4f13c9a2b07 parent: 100001:32ee5d197500 user: Senthil Kumaran date: Wed Jan 20 03:11:52 2016 -0800 summary: merge from 3.5 issue25982 - Add a class definition for managers.Namespace in the multiprocessing docs. files: Doc/library/multiprocessing.rst | 40 +++++++++++--------- 1 files changed, 21 insertions(+), 19 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -316,7 +316,7 @@ proxies. A manager returned by :func:`Manager` will support types - :class:`list`, :class:`dict`, :class:`Namespace`, :class:`Lock`, + :class:`list`, :class:`dict`, :class:`~managers.Namespace`, :class:`Lock`, :class:`RLock`, :class:`Semaphore`, :class:`BoundedSemaphore`, :class:`Condition`, :class:`Event`, :class:`Barrier`, :class:`Queue`, :class:`Value` and :class:`Array`. For example, :: @@ -1752,24 +1752,26 @@ lproxy[0] = d -Namespace objects ->>>>>>>>>>>>>>>>> - -A namespace object has no public methods, but does have writable attributes. -Its representation shows the values of its attributes. - -However, when using a proxy for a namespace object, an attribute beginning with -``'_'`` will be an attribute of the proxy and not an attribute of the referent: - -.. doctest:: - - >>> manager = multiprocessing.Manager() - >>> Global = manager.Namespace() - >>> Global.x = 10 - >>> Global.y = 'hello' - >>> Global._z = 12.3 # this is an attribute of the proxy - >>> print(Global) - Namespace(x=10, y='hello') +.. class:: Namespace + + A type that can register with :class:`SyncManager`. + + A namespace object has no public methods, but does have writable attributes. + Its representation shows the values of its attributes. + + However, when using a proxy for a namespace object, an attribute beginning + with ``'_'`` will be an attribute of the proxy and not an attribute of the + referent: + + .. doctest:: + + >>> manager = multiprocessing.Manager() + >>> Global = manager.Namespace() + >>> Global.x = 10 + >>> Global.y = 'hello' + >>> Global._z = 12.3 # this is an attribute of the proxy + >>> print(Global) + Namespace(x=10, y='hello') Customized managers -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 06:12:12 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 20 Jan 2016 11:12:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_issue25982_-_A?= =?utf-8?q?dd_a_class_definition_for_managers=2ENamespace_in_the?= Message-ID: <20160120111157.94218.36516@psf.io> https://hg.python.org/cpython/rev/32ee5d197500 changeset: 100001:32ee5d197500 branch: 3.5 parent: 99999:f9461f1e0559 user: Senthil Kumaran date: Wed Jan 20 03:10:13 2016 -0800 summary: issue25982 - Add a class definition for managers.Namespace in the multiprocessing docs. files: Doc/library/multiprocessing.rst | 40 +++++++++++--------- 1 files changed, 21 insertions(+), 19 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -316,7 +316,7 @@ proxies. A manager returned by :func:`Manager` will support types - :class:`list`, :class:`dict`, :class:`Namespace`, :class:`Lock`, + :class:`list`, :class:`dict`, :class:`~managers.Namespace`, :class:`Lock`, :class:`RLock`, :class:`Semaphore`, :class:`BoundedSemaphore`, :class:`Condition`, :class:`Event`, :class:`Barrier`, :class:`Queue`, :class:`Value` and :class:`Array`. For example, :: @@ -1747,24 +1747,26 @@ lproxy[0] = d -Namespace objects ->>>>>>>>>>>>>>>>> - -A namespace object has no public methods, but does have writable attributes. -Its representation shows the values of its attributes. - -However, when using a proxy for a namespace object, an attribute beginning with -``'_'`` will be an attribute of the proxy and not an attribute of the referent: - -.. doctest:: - - >>> manager = multiprocessing.Manager() - >>> Global = manager.Namespace() - >>> Global.x = 10 - >>> Global.y = 'hello' - >>> Global._z = 12.3 # this is an attribute of the proxy - >>> print(Global) - Namespace(x=10, y='hello') +.. class:: Namespace + + A type that can register with :class:`SyncManager`. + + A namespace object has no public methods, but does have writable attributes. + Its representation shows the values of its attributes. + + However, when using a proxy for a namespace object, an attribute beginning + with ``'_'`` will be an attribute of the proxy and not an attribute of the + referent: + + .. doctest:: + + >>> manager = multiprocessing.Manager() + >>> Global = manager.Namespace() + >>> Global.x = 10 + >>> Global.y = 'hello' + >>> Global._z = 12.3 # this is an attribute of the proxy + >>> print(Global) + Namespace(x=10, y='hello') Customized managers -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 06:19:01 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 20 Jan 2016 11:19:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_issue25982_-_A?= =?utf-8?q?dd_a_class_definition_for_managers=2ENamespace_in_the?= Message-ID: <20160120111859.94206.61612@psf.io> https://hg.python.org/cpython/rev/8d7aadd40d30 changeset: 100003:8d7aadd40d30 branch: 2.7 parent: 99996:bb2cb184eca9 user: Senthil Kumaran date: Wed Jan 20 03:18:48 2016 -0800 summary: issue25982 - Add a class definition for managers.Namespace in the multiprocessing docs. files: Doc/library/multiprocessing.rst | 39 ++++++++++---------- 1 files changed, 20 insertions(+), 19 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -215,7 +215,7 @@ proxies. A manager returned by :func:`Manager` will support types :class:`list`, - :class:`dict`, :class:`Namespace`, :class:`Lock`, :class:`RLock`, + :class:`dict`, :class:`~managers.Namespace`, :class:`Lock`, :class:`RLock`, :class:`Semaphore`, :class:`BoundedSemaphore`, :class:`Condition`, :class:`Event`, :class:`~multiprocessing.Queue`, :class:`Value` and :class:`Array`. For example, :: @@ -1512,24 +1512,25 @@ lproxy[0] = d -Namespace objects ->>>>>>>>>>>>>>>>> - -A namespace object has no public methods, but does have writable attributes. -Its representation shows the values of its attributes. - -However, when using a proxy for a namespace object, an attribute beginning with -``'_'`` will be an attribute of the proxy and not an attribute of the referent: - -.. doctest:: - - >>> manager = multiprocessing.Manager() - >>> Global = manager.Namespace() - >>> Global.x = 10 - >>> Global.y = 'hello' - >>> Global._z = 12.3 # this is an attribute of the proxy - >>> print Global - Namespace(x=10, y='hello') +.. class:: Namespace + + A type that can register with :class:`SyncManager`. + + A namespace object has no public methods, but does have writable attributes. + Its representation shows the values of its attributes. + + However, when using a proxy for a namespace object, an attribute beginning with + ``'_'`` will be an attribute of the proxy and not an attribute of the referent: + + .. doctest:: + + >>> manager = multiprocessing.Manager() + >>> Global = manager.Namespace() + >>> Global.x = 10 + >>> Global.y = 'hello' + >>> Global._z = 12.3 # this is an attribute of the proxy + >>> print Global + Namespace(x=10, y='hello') Customized managers -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 06:31:35 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 11:31:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_co=5Flnotab_supports_negat?= =?utf-8?q?ive_line_number_delta?= Message-ID: <20160120113134.5451.31741@psf.io> https://hg.python.org/cpython/rev/775b74e0e103 changeset: 100004:775b74e0e103 parent: 100002:f97da0952a2e user: Victor Stinner date: Wed Jan 20 12:16:21 2016 +0100 summary: co_lnotab supports negative line number delta Issue #26107: The format of the co_lnotab attribute of code objects changes to support negative line number delta. Changes: * assemble_lnotab(): if line number delta is less than -128 or greater than 127, emit multiple (offset_delta, lineno_delta) in co_lnotab * update functions decoding co_lnotab to use signed 8-bit integers - dis.findlinestarts() - PyCode_Addr2Line() - _PyCode_CheckLineNumber() - frame_setlineno() * update lnotab_notes.txt * increase importlib MAGIC_NUMBER to 3361 * document the change in What's New in Python 3.6 * cleanup also PyCode_Optimize() to use better variable names files: Doc/whatsnew/3.6.rst | 10 + Include/code.h | 2 +- Lib/dis.py | 7 +- Lib/importlib/_bootstrap_external.py | 5 +- Misc/NEWS | 3 + Objects/codeobject.c | 11 +- Objects/frameobject.c | 2 +- Objects/lnotab_notes.txt | 37 +- Python/compile.c | 25 +- Python/importlib_external.h | 211 +++++++------- Python/peephole.c | 51 +- 11 files changed, 203 insertions(+), 161 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -244,6 +244,16 @@ Changes in the Python API ------------------------- +* The format of the ``co_lnotab`` attribute of code objects changed to support + negative line number delta. By default, Python does not emit bytecode with + negative line number delta. Functions using ``frame.f_lineno``, + ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected. + Functions decoding directly ``co_lnotab`` should be updated to use a signed + 8-bit integer type for the line number delta, but it's only required to + support applications using negative line number delta. See + ``Objects/lnotab_notes.txt`` for the ``co_lnotab`` format and how to decode + it, and see the :pep:`511` for the rationale. + * The functions in the :mod:`compileall` module now return booleans instead of ``1`` or ``0`` to represent success or failure, respectively. Thanks to booleans being a subclass of integers, this should only be an issue if you diff --git a/Include/code.h b/Include/code.h --- a/Include/code.h +++ b/Include/code.h @@ -117,7 +117,7 @@ #endif PyAPI_FUNC(PyObject*) PyCode_Optimize(PyObject *code, PyObject* consts, - PyObject *names, PyObject *lineno_obj); + PyObject *names, PyObject *lnotab); #ifdef __cplusplus } diff --git a/Lib/dis.py b/Lib/dis.py --- a/Lib/dis.py +++ b/Lib/dis.py @@ -397,8 +397,8 @@ Generate pairs (offset, lineno) as described in Python/compile.c. """ - byte_increments = list(code.co_lnotab[0::2]) - line_increments = list(code.co_lnotab[1::2]) + byte_increments = code.co_lnotab[0::2] + line_increments = code.co_lnotab[1::2] lastlineno = None lineno = code.co_firstlineno @@ -409,6 +409,9 @@ yield (addr, lineno) lastlineno = lineno addr += byte_incr + if line_incr >= 0x80: + # line_increments is an array of 8-bit signed integers + line_incr -= 0x100 lineno += line_incr if lineno != lastlineno: yield (addr, lineno) diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -223,13 +223,14 @@ # Python 3.5b1 3330 (PEP 448: Additional Unpacking Generalizations) # Python 3.5b2 3340 (fix dictionary display evaluation order #11205) # Python 3.5b2 3350 (add GET_YIELD_FROM_ITER opcode #24400) -# Python 3.6a0 3360 (add FORMAT_VALUE opcode #25483) +# Python 3.6a0 3360 (add FORMAT_VALUE opcode #25483 +# Python 3.6a0 3361 (lineno delta of code.co_lnotab becomes signed) # # MAGIC must change whenever the bytecode emitted by the compiler may no # longer be understood by older implementations of the eval loop (usually # due to the addition of new opcodes). -MAGIC_NUMBER = (3360).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3361).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #26107: The format of the ``co_lnotab`` attribute of code objects + changes to support negative line number delta. + - Issue #26154: Add a new private _PyThreadState_UncheckedGet() function to get the current Python thread state, but don't issue a fatal error if it is NULL. This new function must be used instead of accessing directly the diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -557,7 +557,8 @@ addr += *p++; if (addr > addrq) break; - line += *p++; + line += (signed char)*p; + p++; } return line; } @@ -592,17 +593,19 @@ if (addr + *p > lasti) break; addr += *p++; - if (*p) + if ((signed char)*p) bounds->ap_lower = addr; - line += *p++; + line += (signed char)*p; + p++; --size; } if (size > 0) { while (--size >= 0) { addr += *p++; - if (*p++) + if ((signed char)*p) break; + p++; } bounds->ap_upper = addr; } diff --git a/Objects/frameobject.c b/Objects/frameobject.c --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -137,7 +137,7 @@ new_lasti = -1; for (offset = 0; offset < lnotab_len; offset += 2) { addr += lnotab[offset]; - line += lnotab[offset+1]; + line += (signed char)lnotab[offset+1]; if (line >= new_lineno) { new_lasti = addr; new_lineno = line; diff --git a/Objects/lnotab_notes.txt b/Objects/lnotab_notes.txt --- a/Objects/lnotab_notes.txt +++ b/Objects/lnotab_notes.txt @@ -12,42 +12,47 @@ 0 1 6 2 50 7 - 350 307 - 361 308 + 350 207 + 361 208 Instead of storing these numbers literally, we compress the list by storing only -the increments from one row to the next. Conceptually, the stored list might +the difference from one row to the next. Conceptually, the stored list might look like: - 0, 1, 6, 1, 44, 5, 300, 300, 11, 1 + 0, 1, 6, 1, 44, 5, 300, 200, 11, 1 -The above doesn't really work, but it's a start. Note that an unsigned byte -can't hold negative values, or values larger than 255, and the above example -contains two such values. So we make two tweaks: +The above doesn't really work, but it's a start. An unsigned byte (byte code +offset)) can't hold negative values, or values larger than 255, a signed byte +(line number) can't hold values larger than 127 or less than -128, and the +above example contains two such values. So we make two tweaks: - (a) there's a deep assumption that byte code offsets and their corresponding - line #s both increase monotonically, and - (b) if at least one column jumps by more than 255 from one row to the next, - more than one pair is written to the table. In case #b, there's no way to know - from looking at the table later how many were written. That's the delicate - part. A user of co_lnotab desiring to find the source line number - corresponding to a bytecode address A should do something like this + (a) there's a deep assumption that byte code offsets increase monotonically, + and + (b) if byte code offset jumps by more than 255 from one row to the next, or if + source code line number jumps by more than 127 or less than -128 from one row + to the next, more than one pair is written to the table. In case #b, + there's no way to know from looking at the table later how many were written. + That's the delicate part. A user of co_lnotab desiring to find the source + line number corresponding to a bytecode address A should do something like + this: lineno = addr = 0 for addr_incr, line_incr in co_lnotab: addr += addr_incr if addr > A: return lineno + if line_incr >= 0x80: + line_incr -= 0x100 lineno += line_incr (In C, this is implemented by PyCode_Addr2Line().) In order for this to work, when the addr field increments by more than 255, the line # increment in each pair generated must be 0 until the remaining addr increment is < 256. So, in the example above, assemble_lnotab in compile.c should not (as was actually done -until 2.2) expand 300, 300 to +until 2.2) expand 300, 200 to 255, 255, 45, 45, but to - 255, 0, 45, 255, 0, 45. + 255, 0, 45, 128, 0, 72. The above is sufficient to reconstruct line numbers for tracebacks, but not for line tracing. Tracing is handled by PyCode_CheckLineNumber() in codeobject.c diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -4452,7 +4452,6 @@ d_lineno = i->i_lineno - a->a_lineno; assert(d_bytecode >= 0); - assert(d_lineno >= 0); if(d_bytecode == 0 && d_lineno == 0) return 1; @@ -4482,9 +4481,21 @@ d_bytecode -= ncodes * 255; a->a_lnotab_off += ncodes * 2; } - assert(d_bytecode <= 255); - if (d_lineno > 255) { - int j, nbytes, ncodes = d_lineno / 255; + assert(0 <= d_bytecode && d_bytecode <= 255); + + if (d_lineno < -128 || 127 < d_lineno) { + int j, nbytes, ncodes, k; + if (d_lineno < 0) { + k = -128; + /* use division on positive numbers */ + ncodes = (-d_lineno) / 128; + } + else { + k = 127; + ncodes = d_lineno / 127; + } + d_lineno -= ncodes * k; + assert(ncodes >= 1); nbytes = a->a_lnotab_off + 2 * ncodes; len = PyBytes_GET_SIZE(a->a_lnotab); if (nbytes >= len) { @@ -4502,15 +4513,15 @@ lnotab = (unsigned char *) PyBytes_AS_STRING(a->a_lnotab) + a->a_lnotab_off; *lnotab++ = d_bytecode; - *lnotab++ = 255; + *lnotab++ = k; d_bytecode = 0; for (j = 1; j < ncodes; j++) { *lnotab++ = 0; - *lnotab++ = 255; + *lnotab++ = k; } - d_lineno -= ncodes * 255; a->a_lnotab_off += ncodes * 2; } + assert(-128 <= d_lineno && d_lineno <= 127); len = PyBytes_GET_SIZE(a->a_lnotab); if (a->a_lnotab_off + 2 >= len) { diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -258,7 +258,7 @@ 114,5,0,0,0,218,13,95,119,114,105,116,101,95,97,116, 111,109,105,99,99,0,0,0,115,26,0,0,0,0,5,24, 1,9,1,33,1,3,3,21,1,20,1,20,1,13,1,3, - 1,17,1,13,1,5,1,114,55,0,0,0,105,32,13,0, + 1,17,1,13,1,5,1,114,55,0,0,0,105,33,13,0, 0,233,2,0,0,0,114,13,0,0,0,115,2,0,0,0, 13,10,90,11,95,95,112,121,99,97,99,104,101,95,95,122, 4,111,112,116,45,122,3,46,112,121,122,4,46,112,121,99, @@ -368,7 +368,7 @@ 103,90,15,97,108,109,111,115,116,95,102,105,108,101,110,97, 109,101,114,4,0,0,0,114,4,0,0,0,114,5,0,0, 0,218,17,99,97,99,104,101,95,102,114,111,109,95,115,111, - 117,114,99,101,244,0,0,0,115,46,0,0,0,0,18,12, + 117,114,99,101,245,0,0,0,115,46,0,0,0,0,18,12, 1,9,1,7,1,12,1,6,1,12,1,18,1,18,1,24, 1,12,1,12,1,12,1,36,1,12,1,18,1,9,2,12, 1,12,1,12,1,12,1,21,1,21,1,114,79,0,0,0, @@ -447,7 +447,7 @@ 101,118,101,108,90,13,98,97,115,101,95,102,105,108,101,110, 97,109,101,114,4,0,0,0,114,4,0,0,0,114,5,0, 0,0,218,17,115,111,117,114,99,101,95,102,114,111,109,95, - 99,97,99,104,101,32,1,0,0,115,44,0,0,0,0,9, + 99,97,99,104,101,33,1,0,0,115,44,0,0,0,0,9, 18,1,12,1,18,1,18,1,12,1,9,1,15,1,15,1, 12,1,9,1,15,1,12,1,22,1,15,1,9,1,12,1, 22,1,12,1,9,1,12,1,19,1,114,85,0,0,0,99, @@ -484,7 +484,7 @@ 0,0,114,36,0,0,0,90,9,101,120,116,101,110,115,105, 111,110,218,11,115,111,117,114,99,101,95,112,97,116,104,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,15, - 95,103,101,116,95,115,111,117,114,99,101,102,105,108,101,65, + 95,103,101,116,95,115,111,117,114,99,101,102,105,108,101,66, 1,0,0,115,20,0,0,0,0,7,18,1,4,1,24,1, 35,1,4,1,3,1,16,1,19,1,21,1,114,91,0,0, 0,99,1,0,0,0,0,0,0,0,1,0,0,0,11,0, @@ -499,7 +499,7 @@ 0,0,114,79,0,0,0,114,66,0,0,0,114,74,0,0, 0,41,1,218,8,102,105,108,101,110,97,109,101,114,4,0, 0,0,114,4,0,0,0,114,5,0,0,0,218,11,95,103, - 101,116,95,99,97,99,104,101,100,84,1,0,0,115,16,0, + 101,116,95,99,97,99,104,101,100,85,1,0,0,115,16,0, 0,0,0,1,21,1,3,1,14,1,13,1,8,1,21,1, 4,2,114,95,0,0,0,99,1,0,0,0,0,0,0,0, 2,0,0,0,11,0,0,0,67,0,0,0,115,60,0,0, @@ -514,7 +514,7 @@ 114,39,0,0,0,114,41,0,0,0,114,40,0,0,0,41, 2,114,35,0,0,0,114,42,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,5,0,0,0,218,10,95,99,97,108, - 99,95,109,111,100,101,96,1,0,0,115,12,0,0,0,0, + 99,95,109,111,100,101,97,1,0,0,115,12,0,0,0,0, 2,3,1,19,1,13,1,11,3,10,1,114,97,0,0,0, 99,1,0,0,0,0,0,0,0,3,0,0,0,11,0,0, 0,3,0,0,0,115,84,0,0,0,100,1,0,135,0,0, @@ -554,7 +554,7 @@ 103,115,90,6,107,119,97,114,103,115,41,1,218,6,109,101, 116,104,111,100,114,4,0,0,0,114,5,0,0,0,218,19, 95,99,104,101,99,107,95,110,97,109,101,95,119,114,97,112, - 112,101,114,116,1,0,0,115,12,0,0,0,0,1,12,1, + 112,101,114,117,1,0,0,115,12,0,0,0,0,1,12,1, 12,1,15,1,6,1,25,1,122,40,95,99,104,101,99,107, 95,110,97,109,101,46,60,108,111,99,97,108,115,62,46,95, 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, @@ -573,7 +573,7 @@ 116,97,116,116,114,218,8,95,95,100,105,99,116,95,95,218, 6,117,112,100,97,116,101,41,3,90,3,110,101,119,90,3, 111,108,100,114,52,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,218,5,95,119,114,97,112,127,1, + 0,0,114,5,0,0,0,218,5,95,119,114,97,112,128,1, 0,0,115,8,0,0,0,0,1,25,1,15,1,29,1,122, 26,95,99,104,101,99,107,95,110,97,109,101,46,60,108,111, 99,97,108,115,62,46,95,119,114,97,112,41,3,218,10,95, @@ -581,7 +581,7 @@ 78,97,109,101,69,114,114,111,114,41,3,114,102,0,0,0, 114,103,0,0,0,114,113,0,0,0,114,4,0,0,0,41, 1,114,102,0,0,0,114,5,0,0,0,218,11,95,99,104, - 101,99,107,95,110,97,109,101,108,1,0,0,115,14,0,0, + 101,99,107,95,110,97,109,101,109,1,0,0,115,14,0,0, 0,0,8,21,7,3,1,13,1,13,2,17,5,13,1,114, 116,0,0,0,99,2,0,0,0,0,0,0,0,5,0,0, 0,4,0,0,0,67,0,0,0,115,84,0,0,0,124,0, @@ -611,7 +611,7 @@ 218,8,112,111,114,116,105,111,110,115,218,3,109,115,103,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,17, 95,102,105,110,100,95,109,111,100,117,108,101,95,115,104,105, - 109,136,1,0,0,115,10,0,0,0,0,10,21,1,24,1, + 109,137,1,0,0,115,10,0,0,0,0,10,21,1,24,1, 6,1,29,1,114,123,0,0,0,99,4,0,0,0,0,0, 0,0,11,0,0,0,19,0,0,0,67,0,0,0,115,252, 1,0,0,105,0,0,125,4,0,124,2,0,100,1,0,107, @@ -698,7 +698,7 @@ 218,11,115,111,117,114,99,101,95,115,105,122,101,114,4,0, 0,0,114,4,0,0,0,114,5,0,0,0,218,25,95,118, 97,108,105,100,97,116,101,95,98,121,116,101,99,111,100,101, - 95,104,101,97,100,101,114,153,1,0,0,115,76,0,0,0, + 95,104,101,97,100,101,114,154,1,0,0,115,76,0,0,0, 0,11,6,1,12,1,13,3,6,1,12,1,10,1,16,1, 16,1,16,1,12,1,18,1,16,1,18,1,18,1,15,1, 16,1,15,1,18,1,15,1,16,1,12,1,12,1,3,1, @@ -729,7 +729,7 @@ 5,114,53,0,0,0,114,98,0,0,0,114,89,0,0,0, 114,90,0,0,0,218,4,99,111,100,101,114,4,0,0,0, 114,4,0,0,0,114,5,0,0,0,218,17,95,99,111,109, - 112,105,108,101,95,98,121,116,101,99,111,100,101,208,1,0, + 112,105,108,101,95,98,121,116,101,99,111,100,101,209,1,0, 0,115,16,0,0,0,0,2,15,1,15,1,16,1,12,1, 16,1,4,2,18,1,114,141,0,0,0,114,59,0,0,0, 99,3,0,0,0,0,0,0,0,4,0,0,0,3,0,0, @@ -749,7 +749,7 @@ 100,117,109,112,115,41,4,114,140,0,0,0,114,126,0,0, 0,114,134,0,0,0,114,53,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,5,0,0,0,218,17,95,99,111,100, - 101,95,116,111,95,98,121,116,101,99,111,100,101,220,1,0, + 101,95,116,111,95,98,121,116,101,99,111,100,101,221,1,0, 0,115,10,0,0,0,0,3,12,1,19,1,19,1,22,1, 114,144,0,0,0,99,1,0,0,0,0,0,0,0,5,0, 0,0,4,0,0,0,67,0,0,0,115,89,0,0,0,100, @@ -778,7 +778,7 @@ 218,8,101,110,99,111,100,105,110,103,90,15,110,101,119,108, 105,110,101,95,100,101,99,111,100,101,114,114,4,0,0,0, 114,4,0,0,0,114,5,0,0,0,218,13,100,101,99,111, - 100,101,95,115,111,117,114,99,101,230,1,0,0,115,10,0, + 100,101,95,115,111,117,114,99,101,231,1,0,0,115,10,0, 0,0,0,5,12,1,18,1,15,1,18,1,114,149,0,0, 0,114,120,0,0,0,218,26,115,117,98,109,111,100,117,108, 101,95,115,101,97,114,99,104,95,108,111,99,97,116,105,111, @@ -843,7 +843,7 @@ 102,105,120,101,115,114,153,0,0,0,90,7,100,105,114,110, 97,109,101,114,4,0,0,0,114,4,0,0,0,114,5,0, 0,0,218,23,115,112,101,99,95,102,114,111,109,95,102,105, - 108,101,95,108,111,99,97,116,105,111,110,247,1,0,0,115, + 108,101,95,108,111,99,97,116,105,111,110,248,1,0,0,115, 60,0,0,0,0,12,12,4,6,1,15,2,3,1,19,1, 13,1,5,8,24,1,9,3,12,1,22,1,21,1,15,1, 9,1,5,2,4,3,12,2,15,1,3,1,19,1,13,1, @@ -883,7 +883,7 @@ 72,75,69,89,95,76,79,67,65,76,95,77,65,67,72,73, 78,69,41,2,218,3,99,108,115,218,3,107,101,121,114,4, 0,0,0,114,4,0,0,0,114,5,0,0,0,218,14,95, - 111,112,101,110,95,114,101,103,105,115,116,114,121,69,2,0, + 111,112,101,110,95,114,101,103,105,115,116,114,121,70,2,0, 0,115,8,0,0,0,0,2,3,1,23,1,13,1,122,36, 87,105,110,100,111,119,115,82,101,103,105,115,116,114,121,70, 105,110,100,101,114,46,95,111,112,101,110,95,114,101,103,105, @@ -910,7 +910,7 @@ 121,95,107,101,121,114,165,0,0,0,90,4,104,107,101,121, 218,8,102,105,108,101,112,97,116,104,114,4,0,0,0,114, 4,0,0,0,114,5,0,0,0,218,16,95,115,101,97,114, - 99,104,95,114,101,103,105,115,116,114,121,76,2,0,0,115, + 99,104,95,114,101,103,105,115,116,114,121,77,2,0,0,115, 22,0,0,0,0,2,9,1,12,2,9,1,15,1,22,1, 3,1,18,1,29,1,13,1,9,1,122,38,87,105,110,100, 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, @@ -935,7 +935,7 @@ 103,101,116,114,171,0,0,0,114,120,0,0,0,114,160,0, 0,0,114,158,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,5,0,0,0,218,9,102,105,110,100,95,115,112,101, - 99,91,2,0,0,115,26,0,0,0,0,2,15,1,12,1, + 99,92,2,0,0,115,26,0,0,0,0,2,15,1,12,1, 4,1,3,1,14,1,13,1,9,1,22,1,21,1,9,1, 15,1,9,1,122,31,87,105,110,100,111,119,115,82,101,103, 105,115,116,114,121,70,105,110,100,101,114,46,102,105,110,100, @@ -954,7 +954,7 @@ 175,0,0,0,114,120,0,0,0,41,4,114,164,0,0,0, 114,119,0,0,0,114,35,0,0,0,114,158,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,11, - 102,105,110,100,95,109,111,100,117,108,101,107,2,0,0,115, + 102,105,110,100,95,109,111,100,117,108,101,108,2,0,0,115, 8,0,0,0,0,7,18,1,12,1,7,2,122,33,87,105, 110,100,111,119,115,82,101,103,105,115,116,114,121,70,105,110, 100,101,114,46,102,105,110,100,95,109,111,100,117,108,101,41, @@ -963,7 +963,7 @@ 167,0,0,0,218,11,99,108,97,115,115,109,101,116,104,111, 100,114,166,0,0,0,114,172,0,0,0,114,175,0,0,0, 114,176,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,162,0,0,0,57,2, + 4,0,0,0,114,5,0,0,0,114,162,0,0,0,58,2, 0,0,115,20,0,0,0,12,2,6,3,6,3,6,2,6, 2,18,7,18,15,3,1,21,15,3,1,114,162,0,0,0, 99,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0, @@ -1001,7 +1001,7 @@ 100,0,0,0,114,119,0,0,0,114,94,0,0,0,90,13, 102,105,108,101,110,97,109,101,95,98,97,115,101,90,9,116, 97,105,108,95,110,97,109,101,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,153,0,0,0,126,2,0,0, + 0,0,114,5,0,0,0,114,153,0,0,0,127,2,0,0, 115,8,0,0,0,0,3,25,1,22,1,19,1,122,24,95, 76,111,97,100,101,114,66,97,115,105,99,115,46,105,115,95, 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, @@ -1012,7 +1012,7 @@ 111,110,46,78,114,4,0,0,0,41,2,114,100,0,0,0, 114,158,0,0,0,114,4,0,0,0,114,4,0,0,0,114, 5,0,0,0,218,13,99,114,101,97,116,101,95,109,111,100, - 117,108,101,134,2,0,0,115,0,0,0,0,122,27,95,76, + 117,108,101,135,2,0,0,115,0,0,0,0,122,27,95,76, 111,97,100,101,114,66,97,115,105,99,115,46,99,114,101,97, 116,101,95,109,111,100,117,108,101,99,2,0,0,0,0,0, 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,80, @@ -1033,7 +1033,7 @@ 101,99,114,111,0,0,0,41,3,114,100,0,0,0,218,6, 109,111,100,117,108,101,114,140,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,5,0,0,0,218,11,101,120,101,99, - 95,109,111,100,117,108,101,137,2,0,0,115,10,0,0,0, + 95,109,111,100,117,108,101,138,2,0,0,115,10,0,0,0, 0,2,18,1,12,1,9,1,15,1,122,25,95,76,111,97, 100,101,114,66,97,115,105,99,115,46,101,120,101,99,95,109, 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, @@ -1044,14 +1044,14 @@ 114,0,0,0,218,17,95,108,111,97,100,95,109,111,100,117, 108,101,95,115,104,105,109,41,2,114,100,0,0,0,114,119, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, - 0,0,218,11,108,111,97,100,95,109,111,100,117,108,101,145, + 0,0,218,11,108,111,97,100,95,109,111,100,117,108,101,146, 2,0,0,115,2,0,0,0,0,2,122,25,95,76,111,97, 100,101,114,66,97,115,105,99,115,46,108,111,97,100,95,109, 111,100,117,108,101,78,41,8,114,105,0,0,0,114,104,0, 0,0,114,106,0,0,0,114,107,0,0,0,114,153,0,0, 0,114,180,0,0,0,114,185,0,0,0,114,187,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,178,0,0,0,121,2,0,0,115,10,0, + 5,0,0,0,114,178,0,0,0,122,2,0,0,115,10,0, 0,0,12,3,6,2,12,8,12,3,12,8,114,178,0,0, 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, 0,0,64,0,0,0,115,106,0,0,0,101,0,0,90,1, @@ -1079,7 +1079,7 @@ 41,1,218,7,73,79,69,114,114,111,114,41,2,114,100,0, 0,0,114,35,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,5,0,0,0,218,10,112,97,116,104,95,109,116,105, - 109,101,152,2,0,0,115,2,0,0,0,0,6,122,23,83, + 109,101,153,2,0,0,115,2,0,0,0,0,6,122,23,83, 111,117,114,99,101,76,111,97,100,101,114,46,112,97,116,104, 95,109,116,105,109,101,99,2,0,0,0,0,0,0,0,2, 0,0,0,3,0,0,0,67,0,0,0,115,19,0,0,0, @@ -1114,7 +1114,7 @@ 32,32,32,32,114,126,0,0,0,41,1,114,190,0,0,0, 41,2,114,100,0,0,0,114,35,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,218,10,112,97,116, - 104,95,115,116,97,116,115,160,2,0,0,115,2,0,0,0, + 104,95,115,116,97,116,115,161,2,0,0,115,2,0,0,0, 0,11,122,23,83,111,117,114,99,101,76,111,97,100,101,114, 46,112,97,116,104,95,115,116,97,116,115,99,4,0,0,0, 0,0,0,0,4,0,0,0,3,0,0,0,67,0,0,0, @@ -1138,7 +1138,7 @@ 114,90,0,0,0,90,10,99,97,99,104,101,95,112,97,116, 104,114,53,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,5,0,0,0,218,15,95,99,97,99,104,101,95,98,121, - 116,101,99,111,100,101,173,2,0,0,115,2,0,0,0,0, + 116,101,99,111,100,101,174,2,0,0,115,2,0,0,0,0, 8,122,28,83,111,117,114,99,101,76,111,97,100,101,114,46, 95,99,97,99,104,101,95,98,121,116,101,99,111,100,101,99, 3,0,0,0,0,0,0,0,3,0,0,0,1,0,0,0, @@ -1155,7 +1155,7 @@ 32,32,32,32,32,32,32,78,114,4,0,0,0,41,3,114, 100,0,0,0,114,35,0,0,0,114,53,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,5,0,0,0,114,192,0, - 0,0,183,2,0,0,115,0,0,0,0,122,21,83,111,117, + 0,0,184,2,0,0,115,0,0,0,0,122,21,83,111,117, 114,99,101,76,111,97,100,101,114,46,115,101,116,95,100,97, 116,97,99,2,0,0,0,0,0,0,0,5,0,0,0,16, 0,0,0,67,0,0,0,115,105,0,0,0,124,0,0,106, @@ -1177,7 +1177,7 @@ 0,0,0,114,119,0,0,0,114,35,0,0,0,114,147,0, 0,0,218,3,101,120,99,114,4,0,0,0,114,4,0,0, 0,114,5,0,0,0,218,10,103,101,116,95,115,111,117,114, - 99,101,190,2,0,0,115,14,0,0,0,0,2,15,1,3, + 99,101,191,2,0,0,115,14,0,0,0,0,2,15,1,3, 1,19,1,18,1,9,1,31,1,122,23,83,111,117,114,99, 101,76,111,97,100,101,114,46,103,101,116,95,115,111,117,114, 99,101,218,9,95,111,112,116,105,109,105,122,101,114,29,0, @@ -1199,7 +1199,7 @@ 108,101,41,4,114,100,0,0,0,114,53,0,0,0,114,35, 0,0,0,114,197,0,0,0,114,4,0,0,0,114,4,0, 0,0,114,5,0,0,0,218,14,115,111,117,114,99,101,95, - 116,111,95,99,111,100,101,200,2,0,0,115,4,0,0,0, + 116,111,95,99,111,100,101,201,2,0,0,115,4,0,0,0, 0,5,21,1,122,27,83,111,117,114,99,101,76,111,97,100, 101,114,46,115,111,117,114,99,101,95,116,111,95,99,111,100, 101,99,2,0,0,0,0,0,0,0,10,0,0,0,43,0, @@ -1260,7 +1260,7 @@ 89,0,0,0,218,2,115,116,114,53,0,0,0,218,10,98, 121,116,101,115,95,100,97,116,97,114,147,0,0,0,90,11, 99,111,100,101,95,111,98,106,101,99,116,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,181,0,0,0,208, + 114,4,0,0,0,114,5,0,0,0,114,181,0,0,0,209, 2,0,0,115,78,0,0,0,0,7,15,1,6,1,3,1, 16,1,13,1,11,2,3,1,19,1,13,1,5,2,16,1, 3,1,19,1,13,1,5,2,3,1,9,1,12,1,13,1, @@ -1273,7 +1273,7 @@ 0,0,0,114,192,0,0,0,114,196,0,0,0,114,200,0, 0,0,114,181,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,188,0,0,0, - 150,2,0,0,115,14,0,0,0,12,2,12,8,12,13,12, + 151,2,0,0,115,14,0,0,0,12,2,12,8,12,13,12, 10,12,7,12,10,18,8,114,188,0,0,0,99,0,0,0, 0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0, 0,115,112,0,0,0,101,0,0,90,1,0,100,0,0,90, @@ -1301,7 +1301,7 @@ 32,32,32,32,32,102,105,110,100,101,114,46,78,41,2,114, 98,0,0,0,114,35,0,0,0,41,3,114,100,0,0,0, 114,119,0,0,0,114,35,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,179,0,0,0,9,3, + 4,0,0,0,114,5,0,0,0,114,179,0,0,0,10,3, 0,0,115,4,0,0,0,0,3,9,1,122,19,70,105,108, 101,76,111,97,100,101,114,46,95,95,105,110,105,116,95,95, 99,2,0,0,0,0,0,0,0,2,0,0,0,2,0,0, @@ -1311,7 +1311,7 @@ 2,218,9,95,95,99,108,97,115,115,95,95,114,111,0,0, 0,41,2,114,100,0,0,0,218,5,111,116,104,101,114,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,6, - 95,95,101,113,95,95,15,3,0,0,115,4,0,0,0,0, + 95,95,101,113,95,95,16,3,0,0,115,4,0,0,0,0, 1,18,1,122,17,70,105,108,101,76,111,97,100,101,114,46, 95,95,101,113,95,95,99,1,0,0,0,0,0,0,0,1, 0,0,0,3,0,0,0,67,0,0,0,115,26,0,0,0, @@ -1319,7 +1319,7 @@ 0,0,106,2,0,131,1,0,65,83,41,1,78,41,3,218, 4,104,97,115,104,114,98,0,0,0,114,35,0,0,0,41, 1,114,100,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,218,8,95,95,104,97,115,104,95,95,19, + 114,5,0,0,0,218,8,95,95,104,97,115,104,95,95,20, 3,0,0,115,2,0,0,0,0,1,122,19,70,105,108,101, 76,111,97,100,101,114,46,95,95,104,97,115,104,95,95,99, 2,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, @@ -1334,7 +1334,7 @@ 32,32,32,32,32,32,32,41,3,218,5,115,117,112,101,114, 114,204,0,0,0,114,187,0,0,0,41,2,114,100,0,0, 0,114,119,0,0,0,41,1,114,205,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,187,0,0,0,22,3,0,0, + 0,0,114,5,0,0,0,114,187,0,0,0,23,3,0,0, 115,2,0,0,0,0,10,122,22,70,105,108,101,76,111,97, 100,101,114,46,108,111,97,100,95,109,111,100,117,108,101,99, 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, @@ -1345,7 +1345,7 @@ 98,121,32,116,104,101,32,102,105,110,100,101,114,46,41,1, 114,35,0,0,0,41,2,114,100,0,0,0,114,119,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 114,151,0,0,0,34,3,0,0,115,2,0,0,0,0,3, + 114,151,0,0,0,35,3,0,0,115,2,0,0,0,0,3, 122,23,70,105,108,101,76,111,97,100,101,114,46,103,101,116, 95,102,105,108,101,110,97,109,101,99,2,0,0,0,0,0, 0,0,3,0,0,0,9,0,0,0,67,0,0,0,115,42, @@ -1358,14 +1358,14 @@ 0,0,0,114,50,0,0,0,90,4,114,101,97,100,41,3, 114,100,0,0,0,114,35,0,0,0,114,54,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,194, - 0,0,0,39,3,0,0,115,4,0,0,0,0,2,21,1, + 0,0,0,40,3,0,0,115,4,0,0,0,0,2,21,1, 122,19,70,105,108,101,76,111,97,100,101,114,46,103,101,116, 95,100,97,116,97,41,11,114,105,0,0,0,114,104,0,0, 0,114,106,0,0,0,114,107,0,0,0,114,179,0,0,0, 114,207,0,0,0,114,209,0,0,0,114,116,0,0,0,114, 187,0,0,0,114,151,0,0,0,114,194,0,0,0,114,4, 0,0,0,114,4,0,0,0,41,1,114,205,0,0,0,114, - 5,0,0,0,114,204,0,0,0,4,3,0,0,115,14,0, + 5,0,0,0,114,204,0,0,0,5,3,0,0,115,14,0, 0,0,12,3,6,2,12,6,12,4,12,3,24,12,18,5, 114,204,0,0,0,99,0,0,0,0,0,0,0,0,0,0, 0,0,4,0,0,0,64,0,0,0,115,64,0,0,0,101, @@ -1388,7 +1388,7 @@ 114,39,0,0,0,218,8,115,116,95,109,116,105,109,101,90, 7,115,116,95,115,105,122,101,41,3,114,100,0,0,0,114, 35,0,0,0,114,202,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,5,0,0,0,114,191,0,0,0,49,3,0, + 0,0,0,114,5,0,0,0,114,191,0,0,0,50,3,0, 0,115,4,0,0,0,0,2,12,1,122,27,83,111,117,114, 99,101,70,105,108,101,76,111,97,100,101,114,46,112,97,116, 104,95,115,116,97,116,115,99,4,0,0,0,0,0,0,0, @@ -1399,7 +1399,7 @@ 97,0,0,0,114,192,0,0,0,41,5,114,100,0,0,0, 114,90,0,0,0,114,89,0,0,0,114,53,0,0,0,114, 42,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,193,0,0,0,54,3,0,0,115,4,0,0, + 0,0,0,114,193,0,0,0,55,3,0,0,115,4,0,0, 0,0,2,12,1,122,32,83,111,117,114,99,101,70,105,108, 101,76,111,97,100,101,114,46,95,99,97,99,104,101,95,98, 121,116,101,99,111,100,101,114,214,0,0,0,105,182,1,0, @@ -1438,7 +1438,7 @@ 0,114,214,0,0,0,218,6,112,97,114,101,110,116,114,94, 0,0,0,114,27,0,0,0,114,23,0,0,0,114,195,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,192,0,0,0,59,3,0,0,115,42,0,0,0,0, + 0,114,192,0,0,0,60,3,0,0,115,42,0,0,0,0, 2,18,1,6,2,22,1,18,1,17,2,19,1,15,1,3, 1,17,1,13,2,7,1,18,3,9,1,10,1,27,1,3, 1,16,1,20,1,18,2,12,1,122,25,83,111,117,114,99, @@ -1447,7 +1447,7 @@ 0,114,106,0,0,0,114,107,0,0,0,114,191,0,0,0, 114,193,0,0,0,114,192,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,212, - 0,0,0,45,3,0,0,115,8,0,0,0,12,2,6,2, + 0,0,0,46,3,0,0,115,8,0,0,0,12,2,6,2, 12,5,12,5,114,212,0,0,0,99,0,0,0,0,0,0, 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,46, 0,0,0,101,0,0,90,1,0,100,0,0,90,2,0,100, @@ -1469,7 +1469,7 @@ 0,114,141,0,0,0,41,5,114,100,0,0,0,114,119,0, 0,0,114,35,0,0,0,114,53,0,0,0,114,203,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 114,181,0,0,0,94,3,0,0,115,8,0,0,0,0,1, + 114,181,0,0,0,95,3,0,0,115,8,0,0,0,0,1, 15,1,15,1,24,1,122,29,83,111,117,114,99,101,108,101, 115,115,70,105,108,101,76,111,97,100,101,114,46,103,101,116, 95,99,111,100,101,99,2,0,0,0,0,0,0,0,2,0, @@ -1479,13 +1479,13 @@ 111,32,115,111,117,114,99,101,32,99,111,100,101,46,78,114, 4,0,0,0,41,2,114,100,0,0,0,114,119,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,114, - 196,0,0,0,100,3,0,0,115,2,0,0,0,0,2,122, + 196,0,0,0,101,3,0,0,115,2,0,0,0,0,2,122, 31,83,111,117,114,99,101,108,101,115,115,70,105,108,101,76, 111,97,100,101,114,46,103,101,116,95,115,111,117,114,99,101, 78,41,6,114,105,0,0,0,114,104,0,0,0,114,106,0, 0,0,114,107,0,0,0,114,181,0,0,0,114,196,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,114,217,0,0,0,90,3,0,0,115,6, + 114,5,0,0,0,114,217,0,0,0,91,3,0,0,115,6, 0,0,0,12,2,6,2,12,6,114,217,0,0,0,99,0, 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,64, 0,0,0,115,136,0,0,0,101,0,0,90,1,0,100,0, @@ -1510,7 +1510,7 @@ 1,0,100,0,0,83,41,1,78,41,2,114,98,0,0,0, 114,35,0,0,0,41,3,114,100,0,0,0,114,98,0,0, 0,114,35,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,5,0,0,0,114,179,0,0,0,117,3,0,0,115,4, + 114,5,0,0,0,114,179,0,0,0,118,3,0,0,115,4, 0,0,0,0,1,9,1,122,28,69,120,116,101,110,115,105, 111,110,70,105,108,101,76,111,97,100,101,114,46,95,95,105, 110,105,116,95,95,99,2,0,0,0,0,0,0,0,2,0, @@ -1520,7 +1520,7 @@ 83,41,1,78,41,2,114,205,0,0,0,114,111,0,0,0, 41,2,114,100,0,0,0,114,206,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,207,0,0,0, - 121,3,0,0,115,4,0,0,0,0,1,18,1,122,26,69, + 122,3,0,0,115,4,0,0,0,0,1,18,1,122,26,69, 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, 101,114,46,95,95,101,113,95,95,99,1,0,0,0,0,0, 0,0,1,0,0,0,3,0,0,0,67,0,0,0,115,26, @@ -1528,7 +1528,7 @@ 0,0,124,0,0,106,2,0,131,1,0,65,83,41,1,78, 41,3,114,208,0,0,0,114,98,0,0,0,114,35,0,0, 0,41,1,114,100,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,209,0,0,0,125,3,0,0, + 0,0,114,5,0,0,0,114,209,0,0,0,126,3,0,0, 115,2,0,0,0,0,1,122,28,69,120,116,101,110,115,105, 111,110,70,105,108,101,76,111,97,100,101,114,46,95,95,104, 97,115,104,95,95,99,2,0,0,0,0,0,0,0,3,0, @@ -1546,7 +1546,7 @@ 97,109,105,99,114,129,0,0,0,114,98,0,0,0,114,35, 0,0,0,41,3,114,100,0,0,0,114,158,0,0,0,114, 184,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,180,0,0,0,128,3,0,0,115,10,0,0, + 0,0,0,114,180,0,0,0,129,3,0,0,115,10,0,0, 0,0,2,6,1,15,1,9,1,16,1,122,33,69,120,116, 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, 46,99,114,101,97,116,101,95,109,111,100,117,108,101,99,2, @@ -1564,7 +1564,7 @@ 97,109,105,99,114,129,0,0,0,114,98,0,0,0,114,35, 0,0,0,41,2,114,100,0,0,0,114,184,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,114,185, - 0,0,0,136,3,0,0,115,6,0,0,0,0,2,19,1, + 0,0,0,137,3,0,0,115,6,0,0,0,0,2,19,1, 9,1,122,31,69,120,116,101,110,115,105,111,110,70,105,108, 101,76,111,97,100,101,114,46,101,120,101,99,95,109,111,100, 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1582,7 +1582,7 @@ 41,2,114,179,0,0,0,78,114,4,0,0,0,41,2,114, 22,0,0,0,218,6,115,117,102,102,105,120,41,1,218,9, 102,105,108,101,95,110,97,109,101,114,4,0,0,0,114,5, - 0,0,0,250,9,60,103,101,110,101,120,112,114,62,145,3, + 0,0,0,250,9,60,103,101,110,101,120,112,114,62,146,3, 0,0,115,2,0,0,0,6,1,122,49,69,120,116,101,110, 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,105, 115,95,112,97,99,107,97,103,101,46,60,108,111,99,97,108, @@ -1591,7 +1591,7 @@ 88,84,69,78,83,73,79,78,95,83,85,70,70,73,88,69, 83,41,2,114,100,0,0,0,114,119,0,0,0,114,4,0, 0,0,41,1,114,220,0,0,0,114,5,0,0,0,114,153, - 0,0,0,142,3,0,0,115,6,0,0,0,0,2,19,1, + 0,0,0,143,3,0,0,115,6,0,0,0,0,2,19,1, 18,1,122,30,69,120,116,101,110,115,105,111,110,70,105,108, 101,76,111,97,100,101,114,46,105,115,95,112,97,99,107,97, 103,101,99,2,0,0,0,0,0,0,0,2,0,0,0,1, @@ -1602,7 +1602,7 @@ 101,97,116,101,32,97,32,99,111,100,101,32,111,98,106,101, 99,116,46,78,114,4,0,0,0,41,2,114,100,0,0,0, 114,119,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,181,0,0,0,148,3,0,0,115,2,0, + 5,0,0,0,114,181,0,0,0,149,3,0,0,115,2,0, 0,0,0,2,122,28,69,120,116,101,110,115,105,111,110,70, 105,108,101,76,111,97,100,101,114,46,103,101,116,95,99,111, 100,101,99,2,0,0,0,0,0,0,0,2,0,0,0,1, @@ -1612,7 +1612,7 @@ 117,108,101,115,32,104,97,118,101,32,110,111,32,115,111,117, 114,99,101,32,99,111,100,101,46,78,114,4,0,0,0,41, 2,114,100,0,0,0,114,119,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,196,0,0,0,152, + 114,4,0,0,0,114,5,0,0,0,114,196,0,0,0,153, 3,0,0,115,2,0,0,0,0,2,122,30,69,120,116,101, 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, @@ -1624,7 +1624,7 @@ 101,32,102,105,110,100,101,114,46,41,1,114,35,0,0,0, 41,2,114,100,0,0,0,114,119,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,151,0,0,0, - 156,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, + 157,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, 46,103,101,116,95,102,105,108,101,110,97,109,101,78,41,14, 114,105,0,0,0,114,104,0,0,0,114,106,0,0,0,114, @@ -1632,7 +1632,7 @@ 0,0,0,114,180,0,0,0,114,185,0,0,0,114,153,0, 0,0,114,181,0,0,0,114,196,0,0,0,114,116,0,0, 0,114,151,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,218,0,0,0,109, + 114,4,0,0,0,114,5,0,0,0,114,218,0,0,0,110, 3,0,0,115,20,0,0,0,12,6,6,2,12,4,12,4, 12,3,12,8,12,6,12,6,12,4,12,4,114,218,0,0, 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, @@ -1677,7 +1677,7 @@ 100,101,114,41,4,114,100,0,0,0,114,98,0,0,0,114, 35,0,0,0,218,11,112,97,116,104,95,102,105,110,100,101, 114,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 114,179,0,0,0,169,3,0,0,115,8,0,0,0,0,1, + 114,179,0,0,0,170,3,0,0,115,8,0,0,0,0,1, 9,1,9,1,21,1,122,23,95,78,97,109,101,115,112,97, 99,101,80,97,116,104,46,95,95,105,110,105,116,95,95,99, 1,0,0,0,0,0,0,0,4,0,0,0,3,0,0,0, @@ -1696,7 +1696,7 @@ 0,0,0,218,3,100,111,116,90,2,109,101,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,218,23,95,102,105, 110,100,95,112,97,114,101,110,116,95,112,97,116,104,95,110, - 97,109,101,115,175,3,0,0,115,8,0,0,0,0,2,27, + 97,109,101,115,176,3,0,0,115,8,0,0,0,0,2,27, 1,12,2,4,3,122,38,95,78,97,109,101,115,112,97,99, 101,80,97,116,104,46,95,102,105,110,100,95,112,97,114,101, 110,116,95,112,97,116,104,95,110,97,109,101,115,99,1,0, @@ -1709,7 +1709,7 @@ 0,90,18,112,97,114,101,110,116,95,109,111,100,117,108,101, 95,110,97,109,101,90,14,112,97,116,104,95,97,116,116,114, 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,227,0,0,0,185,3,0,0,115,4,0, + 5,0,0,0,114,227,0,0,0,186,3,0,0,115,4,0, 0,0,0,1,18,1,122,31,95,78,97,109,101,115,112,97, 99,101,80,97,116,104,46,95,103,101,116,95,112,97,114,101, 110,116,95,112,97,116,104,99,1,0,0,0,0,0,0,0, @@ -1727,7 +1727,7 @@ 226,0,0,0,41,3,114,100,0,0,0,90,11,112,97,114, 101,110,116,95,112,97,116,104,114,158,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,5,0,0,0,218,12,95,114, - 101,99,97,108,99,117,108,97,116,101,189,3,0,0,115,16, + 101,99,97,108,99,117,108,97,116,101,190,3,0,0,115,16, 0,0,0,0,2,18,1,15,1,21,3,27,1,9,1,12, 1,9,1,122,27,95,78,97,109,101,115,112,97,99,101,80, 97,116,104,46,95,114,101,99,97,108,99,117,108,97,116,101, @@ -1736,7 +1736,7 @@ 106,1,0,131,0,0,131,1,0,83,41,1,78,41,2,218, 4,105,116,101,114,114,234,0,0,0,41,1,114,100,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 218,8,95,95,105,116,101,114,95,95,202,3,0,0,115,2, + 218,8,95,95,105,116,101,114,95,95,203,3,0,0,115,2, 0,0,0,0,1,122,23,95,78,97,109,101,115,112,97,99, 101,80,97,116,104,46,95,95,105,116,101,114,95,95,99,1, 0,0,0,0,0,0,0,1,0,0,0,2,0,0,0,67, @@ -1744,7 +1744,7 @@ 0,131,0,0,131,1,0,83,41,1,78,41,2,114,31,0, 0,0,114,234,0,0,0,41,1,114,100,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,5,0,0,0,218,7,95, - 95,108,101,110,95,95,205,3,0,0,115,2,0,0,0,0, + 95,108,101,110,95,95,206,3,0,0,115,2,0,0,0,0, 1,122,22,95,78,97,109,101,115,112,97,99,101,80,97,116, 104,46,95,95,108,101,110,95,95,99,1,0,0,0,0,0, 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,16, @@ -1753,7 +1753,7 @@ 99,101,80,97,116,104,40,123,33,114,125,41,41,2,114,47, 0,0,0,114,226,0,0,0,41,1,114,100,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,8, - 95,95,114,101,112,114,95,95,208,3,0,0,115,2,0,0, + 95,95,114,101,112,114,95,95,209,3,0,0,115,2,0,0, 0,0,1,122,23,95,78,97,109,101,115,112,97,99,101,80, 97,116,104,46,95,95,114,101,112,114,95,95,99,2,0,0, 0,0,0,0,0,2,0,0,0,2,0,0,0,67,0,0, @@ -1761,7 +1761,7 @@ 0,0,107,6,0,83,41,1,78,41,1,114,234,0,0,0, 41,2,114,100,0,0,0,218,4,105,116,101,109,114,4,0, 0,0,114,4,0,0,0,114,5,0,0,0,218,12,95,95, - 99,111,110,116,97,105,110,115,95,95,211,3,0,0,115,2, + 99,111,110,116,97,105,110,115,95,95,212,3,0,0,115,2, 0,0,0,0,1,122,27,95,78,97,109,101,115,112,97,99, 101,80,97,116,104,46,95,95,99,111,110,116,97,105,110,115, 95,95,99,2,0,0,0,0,0,0,0,2,0,0,0,2, @@ -1769,7 +1769,7 @@ 0,0,106,1,0,124,1,0,131,1,0,1,100,0,0,83, 41,1,78,41,2,114,226,0,0,0,114,157,0,0,0,41, 2,114,100,0,0,0,114,239,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,157,0,0,0,214, + 114,4,0,0,0,114,5,0,0,0,114,157,0,0,0,215, 3,0,0,115,2,0,0,0,0,1,122,21,95,78,97,109, 101,115,112,97,99,101,80,97,116,104,46,97,112,112,101,110, 100,78,41,13,114,105,0,0,0,114,104,0,0,0,114,106, @@ -1777,7 +1777,7 @@ 0,0,114,227,0,0,0,114,234,0,0,0,114,236,0,0, 0,114,237,0,0,0,114,238,0,0,0,114,240,0,0,0, 114,157,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,5,0,0,0,114,224,0,0,0,162,3, + 4,0,0,0,114,5,0,0,0,114,224,0,0,0,163,3, 0,0,115,20,0,0,0,12,5,6,2,12,6,12,10,12, 4,12,13,12,3,12,3,12,3,12,3,114,224,0,0,0, 99,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0, @@ -1797,7 +1797,7 @@ 0,114,226,0,0,0,41,4,114,100,0,0,0,114,98,0, 0,0,114,35,0,0,0,114,230,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,179,0,0,0, - 220,3,0,0,115,2,0,0,0,0,1,122,25,95,78,97, + 221,3,0,0,115,2,0,0,0,0,1,122,25,95,78,97, 109,101,115,112,97,99,101,76,111,97,100,101,114,46,95,95, 105,110,105,116,95,95,99,2,0,0,0,0,0,0,0,2, 0,0,0,2,0,0,0,67,0,0,0,115,16,0,0,0, @@ -1814,21 +1814,21 @@ 41,62,41,2,114,47,0,0,0,114,105,0,0,0,41,2, 114,164,0,0,0,114,184,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,5,0,0,0,218,11,109,111,100,117,108, - 101,95,114,101,112,114,223,3,0,0,115,2,0,0,0,0, + 101,95,114,101,112,114,224,3,0,0,115,2,0,0,0,0, 7,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, 100,101,114,46,109,111,100,117,108,101,95,114,101,112,114,99, 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, 67,0,0,0,115,4,0,0,0,100,1,0,83,41,2,78, 84,114,4,0,0,0,41,2,114,100,0,0,0,114,119,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,153,0,0,0,232,3,0,0,115,2,0,0,0,0, + 0,114,153,0,0,0,233,3,0,0,115,2,0,0,0,0, 1,122,27,95,78,97,109,101,115,112,97,99,101,76,111,97, 100,101,114,46,105,115,95,112,97,99,107,97,103,101,99,2, 0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,67, 0,0,0,115,4,0,0,0,100,1,0,83,41,2,78,114, 30,0,0,0,114,4,0,0,0,41,2,114,100,0,0,0, 114,119,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,196,0,0,0,235,3,0,0,115,2,0, + 5,0,0,0,114,196,0,0,0,236,3,0,0,115,2,0, 0,0,0,1,122,27,95,78,97,109,101,115,112,97,99,101, 76,111,97,100,101,114,46,103,101,116,95,115,111,117,114,99, 101,99,2,0,0,0,0,0,0,0,2,0,0,0,6,0, @@ -1838,7 +1838,7 @@ 110,103,62,114,183,0,0,0,114,198,0,0,0,84,41,1, 114,199,0,0,0,41,2,114,100,0,0,0,114,119,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 114,181,0,0,0,238,3,0,0,115,2,0,0,0,0,1, + 114,181,0,0,0,239,3,0,0,115,2,0,0,0,0,1, 122,25,95,78,97,109,101,115,112,97,99,101,76,111,97,100, 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, @@ -1847,14 +1847,14 @@ 99,115,32,102,111,114,32,109,111,100,117,108,101,32,99,114, 101,97,116,105,111,110,46,78,114,4,0,0,0,41,2,114, 100,0,0,0,114,158,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,5,0,0,0,114,180,0,0,0,241,3,0, + 0,0,0,114,5,0,0,0,114,180,0,0,0,242,3,0, 0,115,0,0,0,0,122,30,95,78,97,109,101,115,112,97, 99,101,76,111,97,100,101,114,46,99,114,101,97,116,101,95, 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, 0,0,0,1,0,0,0,67,0,0,0,115,4,0,0,0, 100,0,0,83,41,1,78,114,4,0,0,0,41,2,114,100, 0,0,0,114,184,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,5,0,0,0,114,185,0,0,0,244,3,0,0, + 0,0,114,5,0,0,0,114,185,0,0,0,245,3,0,0, 115,2,0,0,0,0,1,122,28,95,78,97,109,101,115,112, 97,99,101,76,111,97,100,101,114,46,101,120,101,99,95,109, 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, @@ -1873,7 +1873,7 @@ 41,4,114,114,0,0,0,114,129,0,0,0,114,226,0,0, 0,114,186,0,0,0,41,2,114,100,0,0,0,114,119,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,114,187,0,0,0,247,3,0,0,115,6,0,0,0,0, + 0,114,187,0,0,0,248,3,0,0,115,6,0,0,0,0, 7,9,1,10,1,122,28,95,78,97,109,101,115,112,97,99, 101,76,111,97,100,101,114,46,108,111,97,100,95,109,111,100, 117,108,101,78,41,12,114,105,0,0,0,114,104,0,0,0, @@ -1881,7 +1881,7 @@ 242,0,0,0,114,153,0,0,0,114,196,0,0,0,114,181, 0,0,0,114,180,0,0,0,114,185,0,0,0,114,187,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,114,241,0,0,0,219,3,0,0,115, + 0,114,5,0,0,0,114,241,0,0,0,220,3,0,0,115, 16,0,0,0,12,1,12,3,18,9,12,3,12,3,12,3, 12,3,12,3,114,241,0,0,0,99,0,0,0,0,0,0, 0,0,0,0,0,0,5,0,0,0,64,0,0,0,115,160, @@ -1919,7 +1919,7 @@ 99,104,101,218,6,118,97,108,117,101,115,114,108,0,0,0, 114,244,0,0,0,41,2,114,164,0,0,0,218,6,102,105, 110,100,101,114,114,4,0,0,0,114,4,0,0,0,114,5, - 0,0,0,114,244,0,0,0,9,4,0,0,115,6,0,0, + 0,0,0,114,244,0,0,0,10,4,0,0,115,6,0,0, 0,0,4,22,1,15,1,122,28,80,97,116,104,70,105,110, 100,101,114,46,105,110,118,97,108,105,100,97,116,101,95,99, 97,99,104,101,115,99,2,0,0,0,0,0,0,0,3,0, @@ -1944,7 +1944,7 @@ 0,114,61,0,0,0,114,118,0,0,0,114,99,0,0,0, 41,3,114,164,0,0,0,114,35,0,0,0,90,4,104,111, 111,107,114,4,0,0,0,114,4,0,0,0,114,5,0,0, - 0,218,11,95,112,97,116,104,95,104,111,111,107,115,17,4, + 0,218,11,95,112,97,116,104,95,104,111,111,107,115,18,4, 0,0,115,16,0,0,0,0,7,25,1,16,1,16,1,3, 1,14,1,13,1,12,2,122,22,80,97,116,104,70,105,110, 100,101,114,46,95,112,97,116,104,95,104,111,111,107,115,99, @@ -1977,7 +1977,7 @@ 0,0,0,41,3,114,164,0,0,0,114,35,0,0,0,114, 247,0,0,0,114,4,0,0,0,114,4,0,0,0,114,5, 0,0,0,218,20,95,112,97,116,104,95,105,109,112,111,114, - 116,101,114,95,99,97,99,104,101,34,4,0,0,115,22,0, + 116,101,114,95,99,97,99,104,101,35,4,0,0,115,22,0, 0,0,0,8,12,1,3,1,16,1,13,3,9,1,3,1, 17,1,13,1,15,1,18,1,122,31,80,97,116,104,70,105, 110,100,101,114,46,95,112,97,116,104,95,105,109,112,111,114, @@ -1997,7 +1997,7 @@ 0,0,0,114,247,0,0,0,114,120,0,0,0,114,121,0, 0,0,114,158,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,5,0,0,0,218,16,95,108,101,103,97,99,121,95, - 103,101,116,95,115,112,101,99,56,4,0,0,115,18,0,0, + 103,101,116,95,115,112,101,99,57,4,0,0,115,18,0,0, 0,0,4,15,1,24,2,15,1,6,1,12,1,16,1,18, 1,9,1,122,27,80,97,116,104,70,105,110,100,101,114,46, 95,108,101,103,97,99,121,95,103,101,116,95,115,112,101,99, @@ -2033,7 +2033,7 @@ 99,101,95,112,97,116,104,90,5,101,110,116,114,121,114,247, 0,0,0,114,158,0,0,0,114,121,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,5,0,0,0,218,9,95,103, - 101,116,95,115,112,101,99,71,4,0,0,115,40,0,0,0, + 101,116,95,115,112,101,99,72,4,0,0,115,40,0,0,0, 0,5,6,1,13,1,21,1,3,1,15,1,12,1,15,1, 21,2,18,1,12,1,3,1,15,1,4,1,9,1,12,1, 12,5,17,2,18,1,9,1,122,20,80,97,116,104,70,105, @@ -2060,7 +2060,7 @@ 0,114,152,0,0,0,114,224,0,0,0,41,6,114,164,0, 0,0,114,119,0,0,0,114,35,0,0,0,114,174,0,0, 0,114,158,0,0,0,114,254,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,5,0,0,0,114,175,0,0,0,103, + 114,4,0,0,0,114,5,0,0,0,114,175,0,0,0,104, 4,0,0,115,26,0,0,0,0,4,12,1,9,1,21,1, 12,1,4,1,15,1,9,1,6,3,9,1,24,1,4,2, 7,2,122,20,80,97,116,104,70,105,110,100,101,114,46,102, @@ -2083,7 +2083,7 @@ 0,114,120,0,0,0,41,4,114,164,0,0,0,114,119,0, 0,0,114,35,0,0,0,114,158,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,176,0,0,0, - 125,4,0,0,115,8,0,0,0,0,8,18,1,12,1,4, + 126,4,0,0,115,8,0,0,0,0,8,18,1,12,1,4, 1,122,22,80,97,116,104,70,105,110,100,101,114,46,102,105, 110,100,95,109,111,100,117,108,101,41,12,114,105,0,0,0, 114,104,0,0,0,114,106,0,0,0,114,107,0,0,0,114, @@ -2091,7 +2091,7 @@ 0,0,0,114,252,0,0,0,114,255,0,0,0,114,175,0, 0,0,114,176,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,243,0,0,0, - 5,4,0,0,115,22,0,0,0,12,2,6,2,18,8,18, + 6,4,0,0,115,22,0,0,0,12,2,6,2,18,8,18, 17,18,22,18,15,3,1,18,31,3,1,21,21,3,1,114, 243,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, 0,3,0,0,0,64,0,0,0,115,133,0,0,0,101,0, @@ -2140,7 +2140,7 @@ 3,0,100,0,0,83,41,1,78,114,4,0,0,0,41,2, 114,22,0,0,0,114,219,0,0,0,41,1,114,120,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,221,0,0,0, - 154,4,0,0,115,2,0,0,0,6,0,122,38,70,105,108, + 155,4,0,0,115,2,0,0,0,6,0,122,38,70,105,108, 101,70,105,110,100,101,114,46,95,95,105,110,105,116,95,95, 46,60,108,111,99,97,108,115,62,46,60,103,101,110,101,120, 112,114,62,114,58,0,0,0,114,29,0,0,0,78,114,87, @@ -2152,7 +2152,7 @@ 100,0,0,0,114,35,0,0,0,218,14,108,111,97,100,101, 114,95,100,101,116,97,105,108,115,90,7,108,111,97,100,101, 114,115,114,160,0,0,0,114,4,0,0,0,41,1,114,120, - 0,0,0,114,5,0,0,0,114,179,0,0,0,148,4,0, + 0,0,0,114,5,0,0,0,114,179,0,0,0,149,4,0, 0,115,16,0,0,0,0,4,6,1,19,1,36,1,9,2, 15,1,9,1,12,1,122,19,70,105,108,101,70,105,110,100, 101,114,46,95,95,105,110,105,116,95,95,99,1,0,0,0, @@ -2163,7 +2163,7 @@ 116,105,109,101,46,114,29,0,0,0,78,114,87,0,0,0, 41,1,114,2,1,0,0,41,1,114,100,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,5,0,0,0,114,244,0, - 0,0,162,4,0,0,115,2,0,0,0,0,2,122,28,70, + 0,0,163,4,0,0,115,2,0,0,0,0,2,122,28,70, 105,108,101,70,105,110,100,101,114,46,105,110,118,97,108,105, 100,97,116,101,95,99,97,99,104,101,115,99,2,0,0,0, 0,0,0,0,3,0,0,0,2,0,0,0,67,0,0,0, @@ -2187,7 +2187,7 @@ 0,114,120,0,0,0,114,150,0,0,0,41,3,114,100,0, 0,0,114,119,0,0,0,114,158,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,117,0,0,0, - 168,4,0,0,115,8,0,0,0,0,7,15,1,12,1,10, + 169,4,0,0,115,8,0,0,0,0,7,15,1,12,1,10, 1,122,22,70,105,108,101,70,105,110,100,101,114,46,102,105, 110,100,95,108,111,97,100,101,114,99,6,0,0,0,0,0, 0,0,7,0,0,0,7,0,0,0,67,0,0,0,115,40, @@ -2198,7 +2198,7 @@ 7,114,100,0,0,0,114,159,0,0,0,114,119,0,0,0, 114,35,0,0,0,90,4,115,109,115,108,114,174,0,0,0, 114,120,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 5,0,0,0,114,255,0,0,0,180,4,0,0,115,6,0, + 5,0,0,0,114,255,0,0,0,181,4,0,0,115,6,0, 0,0,0,1,15,1,18,1,122,20,70,105,108,101,70,105, 110,100,101,114,46,95,103,101,116,95,115,112,101,99,78,99, 3,0,0,0,0,0,0,0,14,0,0,0,15,0,0,0, @@ -2262,7 +2262,7 @@ 110,105,116,95,102,105,108,101,110,97,109,101,90,9,102,117, 108,108,95,112,97,116,104,114,158,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,175,0,0,0, - 185,4,0,0,115,70,0,0,0,0,3,6,1,19,1,3, + 186,4,0,0,115,70,0,0,0,0,3,6,1,19,1,3, 1,34,1,13,1,11,1,15,1,10,1,9,2,9,1,9, 1,15,2,9,1,6,2,12,1,18,1,22,1,10,1,15, 1,12,1,32,4,12,2,22,1,22,1,22,1,16,1,12, @@ -2298,7 +2298,7 @@ 0,0,146,2,0,113,6,0,83,114,4,0,0,0,41,1, 114,88,0,0,0,41,2,114,22,0,0,0,90,2,102,110, 114,4,0,0,0,114,4,0,0,0,114,5,0,0,0,250, - 9,60,115,101,116,99,111,109,112,62,4,5,0,0,115,2, + 9,60,115,101,116,99,111,109,112,62,5,5,0,0,115,2, 0,0,0,9,0,122,41,70,105,108,101,70,105,110,100,101, 114,46,95,102,105,108,108,95,99,97,99,104,101,46,60,108, 111,99,97,108,115,62,46,60,115,101,116,99,111,109,112,62, @@ -2315,7 +2315,7 @@ 95,99,111,110,116,101,110,116,115,114,239,0,0,0,114,98, 0,0,0,114,231,0,0,0,114,219,0,0,0,90,8,110, 101,119,95,110,97,109,101,114,4,0,0,0,114,4,0,0, - 0,114,5,0,0,0,114,7,1,0,0,231,4,0,0,115, + 0,114,5,0,0,0,114,7,1,0,0,232,4,0,0,115, 34,0,0,0,0,2,9,1,3,1,31,1,22,3,11,3, 18,1,18,7,9,1,13,1,24,1,6,1,27,2,6,1, 17,1,9,1,18,1,122,22,70,105,108,101,70,105,110,100, @@ -2354,7 +2354,7 @@ 0,0,0,41,1,114,35,0,0,0,41,2,114,164,0,0, 0,114,6,1,0,0,114,4,0,0,0,114,5,0,0,0, 218,24,112,97,116,104,95,104,111,111,107,95,102,111,114,95, - 70,105,108,101,70,105,110,100,101,114,16,5,0,0,115,6, + 70,105,108,101,70,105,110,100,101,114,17,5,0,0,115,6, 0,0,0,0,2,12,1,18,1,122,54,70,105,108,101,70, 105,110,100,101,114,46,112,97,116,104,95,104,111,111,107,46, 60,108,111,99,97,108,115,62,46,112,97,116,104,95,104,111, @@ -2362,7 +2362,7 @@ 114,114,4,0,0,0,41,3,114,164,0,0,0,114,6,1, 0,0,114,12,1,0,0,114,4,0,0,0,41,2,114,164, 0,0,0,114,6,1,0,0,114,5,0,0,0,218,9,112, - 97,116,104,95,104,111,111,107,6,5,0,0,115,4,0,0, + 97,116,104,95,104,111,111,107,7,5,0,0,115,4,0,0, 0,0,10,21,6,122,20,70,105,108,101,70,105,110,100,101, 114,46,112,97,116,104,95,104,111,111,107,99,1,0,0,0, 0,0,0,0,1,0,0,0,2,0,0,0,67,0,0,0, @@ -2371,7 +2371,7 @@ 110,100,101,114,40,123,33,114,125,41,41,2,114,47,0,0, 0,114,35,0,0,0,41,1,114,100,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,5,0,0,0,114,238,0,0, - 0,24,5,0,0,115,2,0,0,0,0,1,122,19,70,105, + 0,25,5,0,0,115,2,0,0,0,0,1,122,19,70,105, 108,101,70,105,110,100,101,114,46,95,95,114,101,112,114,95, 95,41,15,114,105,0,0,0,114,104,0,0,0,114,106,0, 0,0,114,107,0,0,0,114,179,0,0,0,114,244,0,0, @@ -2379,7 +2379,7 @@ 114,255,0,0,0,114,175,0,0,0,114,7,1,0,0,114, 177,0,0,0,114,13,1,0,0,114,238,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,5,0, - 0,0,114,0,1,0,0,139,4,0,0,115,20,0,0,0, + 0,0,114,0,1,0,0,140,4,0,0,115,20,0,0,0, 12,7,6,2,12,14,12,4,6,2,12,12,12,5,15,46, 12,31,18,18,114,0,1,0,0,99,4,0,0,0,0,0, 0,0,6,0,0,0,11,0,0,0,67,0,0,0,115,195, @@ -2405,7 +2405,7 @@ 104,110,97,109,101,90,9,99,112,97,116,104,110,97,109,101, 114,120,0,0,0,114,158,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,5,0,0,0,218,14,95,102,105,120,95, - 117,112,95,109,111,100,117,108,101,30,5,0,0,115,34,0, + 117,112,95,109,111,100,117,108,101,31,5,0,0,115,34,0, 0,0,0,2,15,1,15,1,6,1,6,1,12,1,12,1, 18,2,15,1,6,1,21,1,3,1,10,1,10,1,10,1, 14,1,13,2,114,18,1,0,0,99,0,0,0,0,0,0, @@ -2426,7 +2426,7 @@ 0,41,3,90,10,101,120,116,101,110,115,105,111,110,115,90, 6,115,111,117,114,99,101,90,8,98,121,116,101,99,111,100, 101,114,4,0,0,0,114,4,0,0,0,114,5,0,0,0, - 114,155,0,0,0,53,5,0,0,115,8,0,0,0,0,5, + 114,155,0,0,0,54,5,0,0,115,8,0,0,0,0,5, 18,1,12,1,12,1,114,155,0,0,0,99,1,0,0,0, 0,0,0,0,12,0,0,0,12,0,0,0,67,0,0,0, 115,70,2,0,0,124,0,0,97,0,0,116,0,0,106,1, @@ -2488,7 +2488,7 @@ 83,41,2,114,29,0,0,0,78,41,1,114,31,0,0,0, 41,2,114,22,0,0,0,114,77,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,5,0,0,0,114,221,0,0,0, - 89,5,0,0,115,2,0,0,0,6,0,122,25,95,115,101, + 90,5,0,0,115,2,0,0,0,6,0,122,25,95,115,101, 116,117,112,46,60,108,111,99,97,108,115,62,46,60,103,101, 110,101,120,112,114,62,114,59,0,0,0,122,30,105,109,112, 111,114,116,108,105,98,32,114,101,113,117,105,114,101,115,32, @@ -2518,7 +2518,7 @@ 90,14,119,101,97,107,114,101,102,95,109,111,100,117,108,101, 90,13,119,105,110,114,101,103,95,109,111,100,117,108,101,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,6, - 95,115,101,116,117,112,64,5,0,0,115,82,0,0,0,0, + 95,115,101,116,117,112,65,5,0,0,115,82,0,0,0,0, 8,6,1,9,1,9,3,13,1,13,1,15,1,18,2,13, 1,20,3,33,1,19,2,31,1,10,1,15,1,13,1,4, 2,3,1,15,1,5,1,13,1,12,2,12,1,16,1,16, @@ -2544,7 +2544,7 @@ 114,212,0,0,0,41,2,114,26,1,0,0,90,17,115,117, 112,112,111,114,116,101,100,95,108,111,97,100,101,114,115,114, 4,0,0,0,114,4,0,0,0,114,5,0,0,0,218,8, - 95,105,110,115,116,97,108,108,132,5,0,0,115,16,0,0, + 95,105,110,115,116,97,108,108,133,5,0,0,115,16,0,0, 0,0,2,10,1,9,1,28,1,15,1,16,1,16,4,9, 1,114,29,1,0,0,41,3,122,3,119,105,110,114,1,0, 0,0,114,2,0,0,0,41,56,114,107,0,0,0,114,10, @@ -2571,11 +2571,12 @@ 114,18,1,0,0,114,155,0,0,0,114,27,1,0,0,114, 29,1,0,0,114,4,0,0,0,114,4,0,0,0,114,4, 0,0,0,114,5,0,0,0,218,8,60,109,111,100,117,108, - 101,62,8,0,0,0,115,98,0,0,0,6,17,6,3,12, + 101,62,8,0,0,0,115,102,0,0,0,6,17,6,3,12, 12,12,5,12,5,12,6,12,12,12,10,12,9,12,5,12, - 7,15,22,15,111,22,1,18,2,6,1,6,2,9,2,9, + 7,15,22,15,112,22,1,18,2,6,1,6,2,9,2,9, 2,10,2,21,44,12,33,12,19,12,12,12,12,12,28,12, 17,21,55,21,12,18,10,12,14,9,3,12,1,15,65,19, 64,19,29,22,110,19,41,25,45,25,16,6,3,25,53,19, - 57,19,42,19,134,19,147,15,23,12,11,12,68, + 57,19,42,19,127,0,7,19,127,0,20,15,23,12,11,12, + 68, }; diff --git a/Python/peephole.c b/Python/peephole.c --- a/Python/peephole.c +++ b/Python/peephole.c @@ -346,19 +346,19 @@ single basic block. All transformations keep the code size the same or smaller. For those that reduce size, the gaps are initially filled with NOPs. Later those NOPs are removed and the jump addresses retargeted in - a single pass. Line numbering is adjusted accordingly. */ + a single pass. Code offset is adjusted accordingly. */ PyObject * PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names, - PyObject *lineno_obj) + PyObject *lnotab_obj) { Py_ssize_t i, j, codelen; int nops, h, adj; int tgt, tgttgt, opcode; unsigned char *codestr = NULL; - unsigned char *lineno; + unsigned char *lnotab; int *addrmap = NULL; - int new_line, cum_orig_line, last_line; + int cum_orig_offset, last_offset; Py_ssize_t tabsiz; PyObject **const_stack = NULL; Py_ssize_t *load_const_stack = NULL; @@ -371,12 +371,17 @@ if (PyErr_Occurred()) goto exitError; - /* Bypass optimization when the lineno table is too complex */ - assert(PyBytes_Check(lineno_obj)); - lineno = (unsigned char*)PyBytes_AS_STRING(lineno_obj); - tabsiz = PyBytes_GET_SIZE(lineno_obj); - if (memchr(lineno, 255, tabsiz) != NULL) + /* Bypass optimization when the lnotab table is too complex */ + assert(PyBytes_Check(lnotab_obj)); + lnotab = (unsigned char*)PyBytes_AS_STRING(lnotab_obj); + tabsiz = PyBytes_GET_SIZE(lnotab_obj); + assert(tabsiz == 0 || Py_REFCNT(lnotab_obj) == 1); + if (memchr(lnotab, 255, tabsiz) != NULL) { + /* 255 value are used for multibyte bytecode instructions */ goto exitUnchanged; + } + /* Note: -128 and 127 special values for line number delta are ok, + the peephole optimizer doesn't modify line numbers. */ /* Avoid situations where jump retargeting could overflow */ assert(PyBytes_Check(code)); @@ -663,21 +668,24 @@ } } - /* Fixup linenotab */ + /* Fixup lnotab */ for (i=0, nops=0 ; i new code offset */ addrmap[i] = (int)(i - nops); if (codestr[i] == NOP) nops++; } - cum_orig_line = 0; - last_line = 0; + cum_orig_offset = 0; + last_offset = 0; for (i=0 ; i < tabsiz ; i+=2) { - cum_orig_line += lineno[i]; - new_line = addrmap[cum_orig_line]; - assert (new_line - last_line < 255); - lineno[i] =((unsigned char)(new_line - last_line)); - last_line = new_line; + int offset_delta, new_offset; + cum_orig_offset += lnotab[i]; + new_offset = addrmap[cum_orig_offset]; + offset_delta = new_offset - last_offset; + assert(0 <= offset_delta && offset_delta <= 255); + lnotab[i] = (unsigned char)offset_delta; + last_offset = new_offset; } /* Remove NOPs and fixup jump targets */ @@ -727,12 +735,9 @@ exitUnchanged: CONST_STACK_DELETE(); - if (blocks != NULL) - PyMem_Free(blocks); - if (addrmap != NULL) - PyMem_Free(addrmap); - if (codestr != NULL) - PyMem_Free(codestr); + PyMem_Free(blocks); + PyMem_Free(addrmap); + PyMem_Free(codestr); Py_XINCREF(code); return code; } -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Jan 20 12:23:29 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 20 Jan 2016 17:23:29 +0000 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-01-20 Message-ID: <49a866c3-a119-43f7-adc5-23a40ccaffd5@irsmsx153.ger.corp.intel.com> Results for project Python default, build date 2016-01-20 03:09:08 +0000 commit: 19e67e78e20e previous commit: 2b0c7b67eca5 revision date: 2016-01-20 01:15:33 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.42% -0.89% 9.85% 15.61% :-| pybench 0.08% -0.38% -1.85% 5.04% :-( regex_v8 3.62% 0.36% -4.46% 4.18% :-| nbody 0.11% 1.55% 0.69% 8.62% :-| json_dump_v2 0.23% -0.13% -1.19% 9.81% :-| normal_startup 1.08% -0.10% -0.02% 5.18% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-01-20/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Jan 20 12:27:23 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 20 Jan 2016 17:27:23 +0000 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-01-20 Message-ID: Results for project Python 2.7, build date 2016-01-20 04:00:32 +0000 commit: 65b7eb7dfb64 previous commit: 04e95f05aafe revision date: 2016-01-20 01:17:03 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.34% 1.02% 4.43% 2.08% :-) pybench 0.16% 0.06% 6.45% 3.68% :-( regex_v8 0.81% 0.09% -2.73% 11.39% :-) nbody 0.17% -0.09% 9.17% 1.05% :-) json_dump_v2 0.24% 0.26% 4.26% 9.68% :-( normal_startup 1.91% -0.33% -5.47% 2.05% :-| ssbench 0.22% 0.05% 0.90% 1.04% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-01-20/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Jan 20 16:16:41 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 21:16:41 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Add_a_table_=22Status_of_?= =?utf-8?q?Python_branches=22_to_the_front_page?= Message-ID: <20160120211641.25900.23461@psf.io> https://hg.python.org/devguide/rev/75b95ada261f changeset: 784:75b95ada261f user: Victor Stinner date: Wed Jan 20 22:11:02 2016 +0100 summary: Add a table "Status of Python branches" to the front page Issue #26165. files: devcycle.rst | 2 + index.rst | 55 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+), 0 deletions(-) diff --git a/devcycle.rst b/devcycle.rst --- a/devcycle.rst +++ b/devcycle.rst @@ -132,6 +132,8 @@ - the ``2.7`` branch holds bug fixes for future 2.7.x maintenance releases and descends from ``2.6`` (RM: Benjamin Peterson) +See also the :ref:`Status of Python branches `. + .. _stages: diff --git a/index.rst b/index.rst --- a/index.rst +++ b/index.rst @@ -58,6 +58,61 @@ * PEPs_ (Python Enhancement Proposals) * :doc:`gitdevs` +.. _branchstatus: + +Status of Python branches +------------------------- + ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| Branch | Schedule | Status | First release | End-of-life | Comment | ++==================+===============================================+=============+================+================+============================================================================+ +| default | `PEP 494 | features | *2016-12-16* | *2021-12-16* | The default branch is currently the future version Python 3.6. | +| | `_ | | | | | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.5 | `PEP 478 | bugfix | 2015-09-13 | *2020-09-13* | | +| | `_ | | | | | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 2.7 | `PEP 373 | bugfix | 2010-07-03 | *2020-01-01* | The support was extended to 2020. | +| | `_ | | | | | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.4 | `PEP 429 | security | 2014-03-16 | *2019-03-16* | `Last binary release: Python 3.4.4 | +| | `_ | | | | `_ | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.3 | `PEP 398 | security | 2012-09-29 | *2017-09-29* | `Last binary release: Python 3.3.5 | +| | `_ | | | | `_ | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.2 | `PEP 392 | security | 2011-02-20 | *2016-02-20* | `Last binary release: Python 3.2.5 | +| | `_ | | | | `_ | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.1 | `PEP 375 | end-of-life | 2009-06-27 | 2012-04-11 | `Last release: Python 3.1.5 | +| | `_ | | | | `_ | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.0 | `PEP 361 | end-of-life | 2008-12-03 | 2009-01-13 | `Last release: Python 3.0.1 | +| | `_ | | | | `_ | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 2.6 | `PEP 361 | end-of-life | 2008-10-01 | 2013-10-29 | `Last release: Python 2.6.9 | +| | `_ | | | | `_ | ++------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ + +Status: + +* features: new features are only added to the default branch, this branch + accepts any kind of change. +* bugfix: bugfixes and security fixes are accepted, new binaries are still + released. +* security: only security fixes are accepted and no more binaries are + released, but new source-only versions can be released +* end-of-line: branch no more maintained, no more change must be pushed + to these branches. + +Dates in italic are scheduled and can be adjusted. + +By default, the end-of-life is scheduled 5 years after the first release. It +can be adjusted by the release manager of each branch. Versions older than 2.6 +reached their end-of-line. + +See also :ref:`Security branches `. + .. _contributing: -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Wed Jan 20 16:31:30 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 21:31:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpc3N1ZSAjMjQ1MjAp?= Message-ID: <20160120213129.25961.25805@psf.io> https://hg.python.org/cpython/rev/6134d9ecab60 changeset: 100006:6134d9ecab60 parent: 100004:775b74e0e103 parent: 100005:76f35f35be50 user: Victor Stinner date: Wed Jan 20 22:28:10 2016 +0100 summary: Merge 3.5 (issue #24520) files: Programs/python.c | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Programs/python.c b/Programs/python.c --- a/Programs/python.c +++ b/Programs/python.c @@ -4,7 +4,7 @@ #include #ifdef __FreeBSD__ -#include +#include #endif #ifdef MS_WINDOWS @@ -23,9 +23,6 @@ wchar_t **argv_copy2; int i, res; char *oldloc; -#ifdef __FreeBSD__ - fp_except_t m; -#endif argv_copy = (wchar_t **)PyMem_RawMalloc(sizeof(wchar_t*) * (argc+1)); argv_copy2 = (wchar_t **)PyMem_RawMalloc(sizeof(wchar_t*) * (argc+1)); @@ -40,8 +37,7 @@ * exceptions by default. Here we disable them. */ #ifdef __FreeBSD__ - m = fpgetmask(); - fpsetmask(m & ~FP_X_OFL); + fedisableexcept(FE_OVERFLOW); #endif oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 16:31:30 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 21:31:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Replace_fpgetm?= =?utf-8?q?ask=28=29_with_fedisableexcept=28=29?= Message-ID: <20160120213129.5463.61773@psf.io> https://hg.python.org/cpython/rev/394ae9efc5c2 changeset: 100007:394ae9efc5c2 branch: 2.7 parent: 100003:8d7aadd40d30 user: Victor Stinner date: Wed Jan 20 22:30:58 2016 +0100 summary: Replace fpgetmask() with fedisableexcept() Issue #24520: On FreeBSD, fpgetmask() was deprecated long time ago. fedisableexcept() is now preferred. files: Modules/python.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/Modules/python.c b/Modules/python.c --- a/Modules/python.c +++ b/Modules/python.c @@ -3,7 +3,7 @@ #include "Python.h" #ifdef __FreeBSD__ -#include +#include #endif int @@ -15,10 +15,7 @@ * exceptions by default. Here we disable them. */ #ifdef __FreeBSD__ - fp_except_t m; - - m = fpgetmask(); - fpsetmask(m & ~FP_X_OFL); + fedisableexcept(FE_OVERFLOW); #endif return Py_Main(argc, argv); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 16:31:41 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 20 Jan 2016 21:31:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Replace_fpgetm?= =?utf-8?q?ask=28=29_with_fedisableexcept=28=29?= Message-ID: <20160120213129.121308.86328@psf.io> https://hg.python.org/cpython/rev/76f35f35be50 changeset: 100005:76f35f35be50 branch: 3.5 parent: 100001:32ee5d197500 user: Victor Stinner date: Wed Jan 20 22:27:34 2016 +0100 summary: Replace fpgetmask() with fedisableexcept() Issue #24520: On FreeBSD, fpgetmask() was deprecated long time ago. fedisableexcept() is now preferred. files: Programs/python.c | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Programs/python.c b/Programs/python.c --- a/Programs/python.c +++ b/Programs/python.c @@ -4,7 +4,7 @@ #include #ifdef __FreeBSD__ -#include +#include #endif #ifdef MS_WINDOWS @@ -23,9 +23,6 @@ wchar_t **argv_copy2; int i, res; char *oldloc; -#ifdef __FreeBSD__ - fp_except_t m; -#endif argv_copy = (wchar_t **)PyMem_RawMalloc(sizeof(wchar_t*) * (argc+1)); argv_copy2 = (wchar_t **)PyMem_RawMalloc(sizeof(wchar_t*) * (argc+1)); @@ -40,8 +37,7 @@ * exceptions by default. Here we disable them. */ #ifdef __FreeBSD__ - m = fpgetmask(); - fpsetmask(m & ~FP_X_OFL); + fedisableexcept(FE_OVERFLOW); #endif oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Jan 20 20:24:16 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 21 Jan 2016 01:24:16 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Issue_=2325704=3A_Update_?= =?utf-8?q?Python_version_to_3=2E5_where_appropriate?= Message-ID: <20160121012416.25961.37899@psf.io> https://hg.python.org/devguide/rev/32545ee8d673 changeset: 785:32545ee8d673 user: Berker Peksag date: Thu Jan 21 03:24:17 2016 +0200 summary: Issue #25704: Update Python version to 3.5 where appropriate Patch by St?phane Wirtel. files: faq.rst | 33 +++++++++++++++++---------------- patch.rst | 2 +- setup.rst | 4 ++-- 3 files changed, 20 insertions(+), 19 deletions(-) diff --git a/faq.rst b/faq.rst --- a/faq.rst +++ b/faq.rst @@ -271,6 +271,7 @@ $ hg branches default 93085:030fda7b1de8 2.7 93060:7ba47bbfe38d + 3.5 99283:4d5417444961 (inactive) 3.4 93082:5fd481150b35 (inactive) 3.3 93079:cda907a02a80 (inactive) 3.2 92975:eac54f7a8018 (inactive) @@ -282,9 +283,9 @@ $ hg branches default 93085:030fda7b1de8 - 3.4 93082:5fd481150b35 (inactive) + 3.5 99283:4d5417444961 (inactive) -This means all changesets in the "3.4" branch have been merged into the +This means all changesets in the "3.5" branch have been merged into the "default" branch (or any other branch, if such exists). @@ -317,10 +318,10 @@ $ hg branch default - $ hg update 3.4 + $ hg update 3.5 86 files updated, 0 files merged, 11 files removed, 0 files unresolved $ hg branch - 3.4 + 3.5 Adding the ``-v`` option to ``hg update`` will list all updated files. @@ -342,11 +343,11 @@ If you want to use the second way, you can do:: - $ hg clone cpython py34 + $ hg clone cpython py35 updating to branch default 3434 files updated, 0 files merged, 0 files removed, 0 files unresolved - $ cd py34 - $ hg update 3.4 + $ cd py35 + $ hg update 3.5 86 files updated, 0 files merged, 11 files removed, 0 files unresolved The current branch in a working copy is "sticky": if you pull in some new @@ -795,14 +796,14 @@ How do I make a null merge? ''''''''''''''''''''''''''' -If you committed something (e.g. on 3.4) that shouldn't be ported on newer +If you committed something (e.g. on 3.5) that shouldn't be ported on newer branches (e.g. on default), you have to do a *null merge*:: cd 3.x - hg merge 3.4 + hg merge 3.5 hg revert -ar default hg resolve -am # needed only if the merge created conflicts - hg ci -m '#12345: null merge with 3.4.' + hg ci -m '#12345: null merge with 3.5.' Before committing, ``hg status`` should list all the merged files as ``M``, but ``hg diff`` should produce no output. This will record the merge without @@ -841,10 +842,10 @@ If more than one branch has multiple heads, you have to repeat these steps for each branch. Since this creates new changesets, you will also have to -:ref:`merge them between branches `. For example, if both ``3.4`` -and ``default`` have multiple heads, you should first merge heads in ``3.4``, -then merge heads in ``default``, and finally merge ``3.4`` with ``default`` -using ``hg merge 3.4`` as usual. +:ref:`merge them between branches `. For example, if both ``3.5`` +and ``default`` have multiple heads, you should first merge heads in ``3.5``, +then merge heads in ``default``, and finally merge ``3.5`` with ``default`` +using ``hg merge 3.5`` as usual. In order to avoid this, you should *always remember to pull and update before committing*. @@ -870,7 +871,7 @@ SSH -======= +=== How do I generate an SSH-2 public key? -------------------------------------- @@ -930,7 +931,7 @@ Can I make commits from machines other than the one I generated the keys on? ------------------------------------------------------------------------------- +---------------------------------------------------------------------------- You can :ref:`make commits ` from any machine, since they will be recorded in your *local repository*. diff --git a/patch.rst b/patch.rst --- a/patch.rst +++ b/patch.rst @@ -118,7 +118,7 @@ your further contributions to Python. -.. _PSF license: http://docs.python.org/3.4/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python +.. _PSF license: http://docs.python.org/dev/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python .. _contributor form: http://www.python.org/psf/contrib/ .. _Python Software Foundation: http://www.python.org/psf/ diff --git a/setup.rst b/setup.rst --- a/setup.rst +++ b/setup.rst @@ -53,9 +53,9 @@ If you want a working copy of an already-released version of Python, i.e., a version in :ref:`maintenance mode `, you can update your -working copy. For instance, to update your working copy to Python 3.4, do:: +working copy. For instance, to update your working copy to Python 3.5, do:: - hg update 3.4 + hg update 3.5 You will need to re-compile CPython when you do such an update. -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Wed Jan 20 20:39:48 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 21 Jan 2016 01:39:48 +0000 Subject: [Python-checkins] =?utf-8?q?devguide=3A_Simplify_the_brach_status?= =?utf-8?q?_table?= Message-ID: <20160121013948.18409.86704@psf.io> https://hg.python.org/devguide/rev/29689a829d92 changeset: 786:29689a829d92 user: Berker Peksag date: Thu Jan 21 03:39:49 2016 +0200 summary: Simplify the brach status table files: index.rst | 75 ++++++++++++++++++++---------------------- 1 files changed, 36 insertions(+), 39 deletions(-) diff --git a/index.rst b/index.rst --- a/index.rst +++ b/index.rst @@ -63,49 +63,46 @@ Status of Python branches ------------------------- -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| Branch | Schedule | Status | First release | End-of-life | Comment | -+==================+===============================================+=============+================+================+============================================================================+ -| default | `PEP 494 | features | *2016-12-16* | *2021-12-16* | The default branch is currently the future version Python 3.6. | -| | `_ | | | | | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.5 | `PEP 478 | bugfix | 2015-09-13 | *2020-09-13* | | -| | `_ | | | | | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 2.7 | `PEP 373 | bugfix | 2010-07-03 | *2020-01-01* | The support was extended to 2020. | -| | `_ | | | | | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.4 | `PEP 429 | security | 2014-03-16 | *2019-03-16* | `Last binary release: Python 3.4.4 | -| | `_ | | | | `_ | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.3 | `PEP 398 | security | 2012-09-29 | *2017-09-29* | `Last binary release: Python 3.3.5 | -| | `_ | | | | `_ | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.2 | `PEP 392 | security | 2011-02-20 | *2016-02-20* | `Last binary release: Python 3.2.5 | -| | `_ | | | | `_ | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.1 | `PEP 375 | end-of-life | 2009-06-27 | 2012-04-11 | `Last release: Python 3.1.5 | -| | `_ | | | | `_ | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 3.0 | `PEP 361 | end-of-life | 2008-12-03 | 2009-01-13 | `Last release: Python 3.0.1 | -| | `_ | | | | `_ | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ -| 2.6 | `PEP 361 | end-of-life | 2008-10-01 | 2013-10-29 | `Last release: Python 2.6.9 | -| | `_ | | | | `_ | -+------------------+-----------------------------------------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| Branch | Schedule | Status | First release | End-of-life | Comment | ++==================+==============+=============+================+================+============================================================================+ +| default | :pep:`494` | features | *2016-12-16* | *2021-12-16* | The default branch is currently the future version Python 3.6. | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.5 | :pep:`478` | bugfix | 2015-09-13 | *2020-09-13* | | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 2.7 | :pep:`373` | bugfix | 2010-07-03 | *2020-01-01* | The support was extended to 2020. | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.4 | :pep:`429` | security | 2014-03-16 | *2019-03-16* | `Last binary release: Python 3.4.4 | +| | | | | | `_ | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.3 | :pep:`398` | security | 2012-09-29 | *2017-09-29* | `Last binary release: Python 3.3.5 | +| | | | | | `_ | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.2 | :pep:`392` | security | 2011-02-20 | *2016-02-20* | `Last binary release: Python 3.2.5 | +| | | | | | `_ | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.1 | :pep:`375` | end-of-life | 2009-06-27 | 2012-04-11 | `Last release: Python 3.1.5 | +| | | | | | `_ | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 3.0 | :pep:`361` | end-of-life | 2008-12-03 | 2009-01-13 | `Last release: Python 3.0.1 | +| | | | | | `_ | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ +| 2.6 | :pep:`361` | end-of-life | 2008-10-01 | 2013-10-29 | `Last release: Python 2.6.9 | +| | | | | | `_ | ++------------------+--------------+-------------+----------------+----------------+----------------------------------------------------------------------------+ Status: -* features: new features are only added to the default branch, this branch - accepts any kind of change. -* bugfix: bugfixes and security fixes are accepted, new binaries are still - released. -* security: only security fixes are accepted and no more binaries are - released, but new source-only versions can be released -* end-of-line: branch no more maintained, no more change must be pushed - to these branches. +:features: new features are only added to the default branch, this branch + accepts any kind of change. +:bugfix: bugfixes and security fixes are accepted, new binaries are still + released. +:security: only security fixes are accepted and no more binaries are released, + but new source-only versions can be released +:end-of-line: branch no more maintained, no more change must be pushed to these + branches. -Dates in italic are scheduled and can be adjusted. +Dates in *italic* are scheduled and can be adjusted. By default, the end-of-life is scheduled 5 years after the first release. It can be adjusted by the release manager of each branch. Versions older than 2.6 -- Repository URL: https://hg.python.org/devguide From python-checkins at python.org Wed Jan 20 22:19:44 2016 From: python-checkins at python.org (brett.cannon) Date: Thu, 21 Jan 2016 03:19:44 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_PEP_7_to_strongly_enco?= =?utf-8?q?urage_curly_braces_with_conditional?= Message-ID: <20160121031944.14189.13894@psf.io> https://hg.python.org/peps/rev/6fd405679546 changeset: 6200:6fd405679546 user: Brett Cannon date: Wed Jan 20 19:19:42 2016 -0800 summary: Update PEP 7 to strongly encourage curly braces with conditional statements. files: pep-0007.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0007.txt b/pep-0007.txt --- a/pep-0007.txt +++ b/pep-0007.txt @@ -75,9 +75,9 @@ } * Code structure: one space between keywords like ``if``, ``for`` and - the following left paren; no spaces inside the paren; braces may be - omitted where C permits but when present, they should be formatted - as shown:: + the following left paren; no spaces inside the paren; braces are + strongly preferred but may be omitted where C permits, and they + should be formatted as shown:: if (mro != NULL) { ... -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Wed Jan 20 22:19:44 2016 From: python-checkins at python.org (brett.cannon) Date: Thu, 21 Jan 2016 03:19:44 +0000 Subject: [Python-checkins] =?utf-8?q?peps=3A_Undo_accidental_part_of_a_com?= =?utf-8?q?mit?= Message-ID: <20160121031944.96370.63033@psf.io> https://hg.python.org/peps/rev/1960ea5fef23 changeset: 6199:1960ea5fef23 user: Brett Cannon date: Wed Jan 20 19:18:33 2016 -0800 summary: Undo accidental part of a commit files: pep-0007.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0007.txt b/pep-0007.txt --- a/pep-0007.txt +++ b/pep-0007.txt @@ -75,9 +75,9 @@ } * Code structure: one space between keywords like ``if``, ``for`` and - the following left paren; no spaces inside the paren; braces are - strongly preferred but may be omitted where C permits, and they - should be formatted as shown:: + the following left paren; no spaces inside the paren; braces may be + omitted where C permits but when present, they should be formatted + as shown:: if (mro != NULL) { ... -- Repository URL: https://hg.python.org/peps From python-checkins at python.org Thu Jan 21 01:03:35 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 21 Jan 2016 06:03:35 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_merge_3=2E4_=28=2326172=29?= Message-ID: <20160121060335.14181.71527@psf.io> https://hg.python.org/cpython/rev/56e68089ad65 changeset: 100009:56e68089ad65 branch: 3.5 parent: 100005:76f35f35be50 parent: 100008:562179b7912f user: Benjamin Peterson date: Wed Jan 20 22:02:47 2016 -0800 summary: merge 3.4 (#26172) files: Doc/tools/templates/layout.html | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -18,7 +18,7 @@ {% if not embedded %}{% endif %} {% if versionswitcher is defined and not embedded %}{% endif %} - {% if pagename == 'whatsnew/changelog' %} + {% if pagename == 'whatsnew/changelog' and not embedded %} {% endif %} {% if versionswitcher is defined and not embedded %}{% endif %} - {% if pagename == 'whatsnew/changelog' %} + {% if pagename == 'whatsnew/changelog' and not embedded %} {% endif %} {% if versionswitcher is defined and not embedded %}{% endif %} - {% if pagename == 'whatsnew/changelog' %} + {% if pagename == 'whatsnew/changelog' and not embedded %}