From python-checkins at python.org Tue Apr 1 01:08:54 2014 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 1 Apr 2014 01:08:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Get_rid_of_dep?= =?utf-8?q?recated_IOError_in_the_doc?= Message-ID: <3fyTr612pbz7LjT@mail.python.org> http://hg.python.org/cpython/rev/8d5f005a0da3 changeset: 90082:8d5f005a0da3 branch: 3.4 parent: 90080:de476c612548 user: Andrew Svetlov date: Tue Apr 01 01:13:30 2014 +0300 summary: Get rid of deprecated IOError in the doc files: Doc/c-api/exceptions.rst | 4 ++-- Doc/howto/unicode.rst | 2 +- Doc/library/importlib.rst | 12 ++++++------ Doc/tutorial/controlflow.rst | 2 +- Doc/tutorial/errors.rst | 4 ++-- 5 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -236,8 +236,8 @@ Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if *filenameObject* is not *NULL*, it is passed to the constructor of *type* as - a third parameter. In the case of exceptions such as :exc:`IOError` and - :exc:`OSError`, this is used to define the :attr:`filename` attribute of the + a third parameter. In the case of :exc:`OSError` exception, + this is used to define the :attr:`filename` attribute of the exception instance. diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -246,7 +246,7 @@ try: with open('/tmp/input.txt', 'r') as f: ... - except IOError: + except OSError: # 'File not found' error message. print("Fichier non trouv?") diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -449,12 +449,12 @@ Loaders that have a file-like storage back-end that allows storing arbitrary data can implement this abstract method to give direct access - to the data stored. :exc:`IOError` is to be raised if the *path* cannot + to the data stored. :exc:`OSError` is to be raised if the *path* cannot be found. The *path* is expected to be constructed using a module's :attr:`__file__` attribute or an item from a package's :attr:`__path__`. .. versionchanged:: 3.4 - Raises :exc:`IOError` instead of :exc:`NotImplementedError`. + Raises :exc:`OSError` instead of :exc:`NotImplementedError`. .. class:: InspectLoader @@ -609,12 +609,12 @@ - ``'size'`` (optional): the size in bytes of the source code. Any other keys in the dictionary are ignored, to allow for future - extensions. If the path cannot be handled, :exc:`IOError` is raised. + extensions. If the path cannot be handled, :exc:`OSError` is raised. .. versionadded:: 3.3 .. versionchanged:: 3.4 - Raise :exc:`IOError` instead of :exc:`NotImplementedError`. + Raise :exc:`OSError` instead of :exc:`NotImplementedError`. .. method:: path_mtime(path) @@ -624,10 +624,10 @@ .. deprecated:: 3.3 This method is deprecated in favour of :meth:`path_stats`. You don't have to implement it, but it is still available for compatibility - purposes. Raise :exc:`IOError` if the path cannot be handled. + purposes. Raise :exc:`OSError` if the path cannot be handled. .. versionchanged:: 3.4 - Raise :exc:`IOError` instead of :exc:`NotImplementedError`. + Raise :exc:`OSError` instead of :exc:`NotImplementedError`. .. method:: set_data(path, data) diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -370,7 +370,7 @@ return False retries = retries - 1 if retries < 0: - raise IOError('uncooperative user') + raise OSError('uncooperative user') print(complaint) This function can be called in several ways: diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -131,8 +131,8 @@ f = open('myfile.txt') s = f.readline() i = int(s.strip()) - except IOError as err: - print("I/O error: {0}".format(err)) + except OSError as err: + print("OS error: {0}".format(err)) except ValueError: print("Could not convert data to an integer.") except: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 01:08:55 2014 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 1 Apr 2014 01:08:55 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E4=3A_Get_rid_of_deprecated_IOError_in_the_doc?= Message-ID: <3fyTr73TWTz7LnV@mail.python.org> http://hg.python.org/cpython/rev/f5dda52a4ccd changeset: 90083:f5dda52a4ccd parent: 90081:c0c17718301f parent: 90082:8d5f005a0da3 user: Andrew Svetlov date: Tue Apr 01 01:14:27 2014 +0300 summary: Merge 3.4: Get rid of deprecated IOError in the doc files: Doc/c-api/exceptions.rst | 4 ++-- Doc/howto/unicode.rst | 2 +- Doc/library/importlib.rst | 12 ++++++------ Doc/tutorial/controlflow.rst | 2 +- Doc/tutorial/errors.rst | 4 ++-- 5 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -236,8 +236,8 @@ Similar to :c:func:`PyErr_SetFromErrno`, with the additional behavior that if *filenameObject* is not *NULL*, it is passed to the constructor of *type* as - a third parameter. In the case of exceptions such as :exc:`IOError` and - :exc:`OSError`, this is used to define the :attr:`filename` attribute of the + a third parameter. In the case of :exc:`OSError` exception, + this is used to define the :attr:`filename` attribute of the exception instance. diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -246,7 +246,7 @@ try: with open('/tmp/input.txt', 'r') as f: ... - except IOError: + except OSError: # 'File not found' error message. print("Fichier non trouv?") diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -449,12 +449,12 @@ Loaders that have a file-like storage back-end that allows storing arbitrary data can implement this abstract method to give direct access - to the data stored. :exc:`IOError` is to be raised if the *path* cannot + to the data stored. :exc:`OSError` is to be raised if the *path* cannot be found. The *path* is expected to be constructed using a module's :attr:`__file__` attribute or an item from a package's :attr:`__path__`. .. versionchanged:: 3.4 - Raises :exc:`IOError` instead of :exc:`NotImplementedError`. + Raises :exc:`OSError` instead of :exc:`NotImplementedError`. .. class:: InspectLoader @@ -609,12 +609,12 @@ - ``'size'`` (optional): the size in bytes of the source code. Any other keys in the dictionary are ignored, to allow for future - extensions. If the path cannot be handled, :exc:`IOError` is raised. + extensions. If the path cannot be handled, :exc:`OSError` is raised. .. versionadded:: 3.3 .. versionchanged:: 3.4 - Raise :exc:`IOError` instead of :exc:`NotImplementedError`. + Raise :exc:`OSError` instead of :exc:`NotImplementedError`. .. method:: path_mtime(path) @@ -624,10 +624,10 @@ .. deprecated:: 3.3 This method is deprecated in favour of :meth:`path_stats`. You don't have to implement it, but it is still available for compatibility - purposes. Raise :exc:`IOError` if the path cannot be handled. + purposes. Raise :exc:`OSError` if the path cannot be handled. .. versionchanged:: 3.4 - Raise :exc:`IOError` instead of :exc:`NotImplementedError`. + Raise :exc:`OSError` instead of :exc:`NotImplementedError`. .. method:: set_data(path, data) diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -370,7 +370,7 @@ return False retries = retries - 1 if retries < 0: - raise IOError('uncooperative user') + raise OSError('uncooperative user') print(complaint) This function can be called in several ways: diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -131,8 +131,8 @@ f = open('myfile.txt') s = f.readline() i = int(s.strip()) - except IOError as err: - print("I/O error: {0}".format(err)) + except OSError as err: + print("OS error: {0}".format(err)) except ValueError: print("Could not convert data to an integer.") except: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 01:08:56 2014 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 1 Apr 2014 01:08:56 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE2NzE2OiByZW1v?= =?utf-8?q?ve_deprecation_warning?= Message-ID: <3fyTr84rhlz7Lnk@mail.python.org> http://hg.python.org/cpython/rev/7b219429c404 changeset: 90084:7b219429c404 branch: 3.4 parent: 90082:8d5f005a0da3 user: Andrew Svetlov date: Tue Apr 01 02:08:25 2014 +0300 summary: #16716: remove deprecation warning files: Doc/library/select.rst | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/Doc/library/select.rst b/Doc/library/select.rst --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -380,10 +380,6 @@ that was never registered causes an :exc:`OSError` exception with errno :const:`ENOENT` to be raised. - .. deprecated:: 3.3 - - Instead of :exc:`IOError` the :exc:`OSError` exception is used now. - .. method:: poll.unregister(fd) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 01:08:57 2014 From: python-checkins at python.org (andrew.svetlov) Date: Tue, 1 Apr 2014 01:08:57 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_=2316716=3A_remove_deprecation_warning?= Message-ID: <3fyTr96gLFz7Lnl@mail.python.org> http://hg.python.org/cpython/rev/1c6c2ec8916a changeset: 90085:1c6c2ec8916a parent: 90083:f5dda52a4ccd parent: 90084:7b219429c404 user: Andrew Svetlov date: Tue Apr 01 02:08:41 2014 +0300 summary: #16716: remove deprecation warning files: Doc/library/select.rst | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/Doc/library/select.rst b/Doc/library/select.rst --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -380,10 +380,6 @@ that was never registered causes an :exc:`OSError` exception with errno :const:`ENOENT` to be raised. - .. deprecated:: 3.3 - - Instead of :exc:`IOError` the :exc:`OSError` exception is used now. - .. method:: poll.unregister(fd) -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Tue Apr 1 09:50:53 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 01 Apr 2014 09:50:53 +0200 Subject: [Python-checkins] Daily reference leaks (1c6c2ec8916a): sum=0 Message-ID: results for 1c6c2ec8916a on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogZyrB82', '-x'] From python-checkins at python.org Tue Apr 1 16:17:15 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 1 Apr 2014 16:17:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_inspect=3A_Remove_referenc?= =?utf-8?q?es_to_Python-3000?= Message-ID: <3fyt0C3P22z7LkW@mail.python.org> http://hg.python.org/cpython/rev/2fbba6d68b94 changeset: 90086:2fbba6d68b94 user: Yury Selivanov date: Tue Apr 01 10:17:08 2014 -0400 summary: inspect: Remove references to Python-3000 files: Lib/inspect.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -17,7 +17,7 @@ getclasstree() - arrange classes so as to represent their hierarchy getargspec(), getargvalues(), getcallargs() - get info about function arguments - getfullargspec() - same, with support for Python-3000 features + getfullargspec() - same, with support for Python 3 features formatargspec(), formatargvalues() - format an argument spec getouterframes(), getinnerframes() - get info about frames currentframe() - get the current stack frame @@ -920,7 +920,7 @@ 'varargs' and 'varkw' are the names of the * and ** arguments or None. 'defaults' is an n-tuple of the default values of the last n arguments. - Use the getfullargspec() API for Python-3000 code, as annotations + Use the getfullargspec() API for Python 3 code, as annotations and keyword arguments are supported. getargspec() will raise ValueError if the func has either annotations or keyword arguments. """ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 19:23:02 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 1 Apr 2014 19:23:02 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE1MDY3?= =?utf-8?q?=3A_Remove_reference_to_a_rejected_PEP=2E?= Message-ID: <3fyy6Z6nf9z7Ljn@mail.python.org> http://hg.python.org/cpython/rev/2e13f0d49b8e changeset: 90087:2e13f0d49b8e branch: 2.7 parent: 90069:8263d3c1cf34 user: Zachary Ware date: Tue Apr 01 12:18:58 2014 -0500 summary: Issue #15067: Remove reference to a rejected PEP. Further explanation of the concept stands alone without this reference, no need to potentially confuse people by bringing up a feature that doesn't exist. files: Doc/library/sqlite3.rst | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -727,9 +727,6 @@ sqlite3 module's supported types for SQLite: one of NoneType, int, long, float, str, unicode, buffer. -The :mod:`sqlite3` module uses Python object adaptation, as described in -:pep:`246` for this. The protocol to use is :class:`PrepareProtocol`. - There are two ways to enable the :mod:`sqlite3` module to adapt a custom Python type to one of the supported ones. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 19:23:04 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 1 Apr 2014 19:23:04 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE1MDY3?= =?utf-8?q?=3A_Port_2=2E7_sqlite3_docs_to_3=2E4?= Message-ID: <3fyy6c2jsmz7LjT@mail.python.org> http://hg.python.org/cpython/rev/4a2dabac976d changeset: 90088:4a2dabac976d branch: 3.4 parent: 90084:7b219429c404 user: Zachary Ware date: Tue Apr 01 12:21:56 2014 -0500 summary: Issue #15067: Port 2.7 sqlite3 docs to 3.4 files: Doc/library/sqlite3.rst | 97 +++++++++++++++------------- 1 files changed, 51 insertions(+), 46 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -13,8 +13,8 @@ application using SQLite and then port the code to a larger database such as PostgreSQL or Oracle. -sqlite3 was written by Gerhard H?ring and provides a SQL interface compliant -with the DB-API 2.0 specification described by :pep:`249`. +The sqlite3 module was written by Gerhard H?ring. It provides a SQL interface +compliant with the DB-API 2.0 specification described by :pep:`249`. To use the module, you must first create a :class:`Connection` object that represents the database. Here the data will be stored in the @@ -31,23 +31,29 @@ c = conn.cursor() # Create table - c.execute('''create table stocks - (date text, trans text, symbol text, - qty real, price real)''') + c.execute('''CREATE TABLE stocks + (date text, trans text, symbol text, qty real, price real)''') # Insert a row of data - c.execute("""insert into stocks - values ('2006-01-05','BUY','RHAT',100,35.14)""") + c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)") # Save (commit) the changes conn.commit() - # We can also close the cursor if we are done with it - c.close() + # We can also close the connection if we are done with it. + # Just be sure any changes have been committed or they will be lost. + conn.close() + +The data you've saved is persistent and is available in subsequent sessions:: + + import sqlite3 + conn = sqlite3.connect('example.db') + c = conn.cursor() Usually your SQL operations will need to use values from Python variables. You shouldn't assemble your query using Python's string operations because doing so -is insecure; it makes your program vulnerable to an SQL injection attack. +is insecure; it makes your program vulnerable to an SQL injection attack +(see http://xkcd.com/327/ for humorous example of what can go wrong). Instead, use the DB-API's parameter substitution. Put ``?`` as a placeholder wherever you want to use a value, and then provide a tuple of values as the @@ -56,19 +62,20 @@ example:: # Never do this -- insecure! - symbol = 'IBM' - c.execute("select * from stocks where symbol = '%s'" % symbol) + symbol = 'RHAT' + c.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol) # Do this instead - t = ('IBM',) - c.execute('select * from stocks where symbol=?', t) + t = ('RHAT',) + c.execute('SELECT * FROM stocks WHERE symbol=?', t) + print(c.fetchone()) - # Larger example - for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), - ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), - ('2006-04-06', 'SELL', 'IBM', 500, 53.00), - ]: - c.execute('insert into stocks values (?,?,?,?,?)', t) + # Larger example that inserts many records at a time + purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), + ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), + ('2006-04-06', 'SELL', 'IBM', 500, 53.00), + ] + c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases) To retrieve data after executing a SELECT statement, you can either treat the cursor as an :term:`iterator`, call the cursor's :meth:`~Cursor.fetchone` method to @@ -77,16 +84,13 @@ This example uses the iterator form:: - >>> c = conn.cursor() - >>> c.execute('select * from stocks order by price') - >>> for row in c: - ... print(row) - ... + >>> for row in c.execute('SELECT * FROM stocks ORDER BY price'): + print(row) + ('2006-01-05', 'BUY', 'RHAT', 100, 35.14) ('2006-03-28', 'BUY', 'IBM', 1000, 45.0) ('2006-04-06', 'SELL', 'IBM', 500, 53.0) - ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.0) - >>> + ('2006-04-05', 'BUY', 'MSFT', 1000, 72.0) .. seealso:: @@ -99,6 +103,9 @@ The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect. + http://www.w3schools.com/sql/ + Tutorial, reference and examples for learning SQL syntax. + :pep:`249` - Database API Specification 2.0 PEP written by Marc-Andr? Lemburg. @@ -517,7 +524,7 @@ .. method:: execute(sql, [parameters]) - Executes an SQL statement. The SQL statement may be parametrized (i. e. + Executes an SQL statement. The SQL statement may be parameterized (i. e. placeholders instead of SQL literals). The :mod:`sqlite3` module supports two kinds of placeholders: question marks (qmark style) and named placeholders (named style). @@ -714,19 +721,20 @@ This is how SQLite types are converted to Python types by default: -+-------------+---------------------------------------------+ -| SQLite type | Python type | -+=============+=============================================+ -| ``NULL`` | :const:`None` | -+-------------+---------------------------------------------+ -| ``INTEGER`` | :class:`int` | -+-------------+---------------------------------------------+ -| ``REAL`` | :class:`float` | -+-------------+---------------------------------------------+ -| ``TEXT`` | depends on text_factory, str by default | -+-------------+---------------------------------------------+ -| ``BLOB`` | :class:`bytes` | -+-------------+---------------------------------------------+ ++-------------+----------------------------------------------+ +| SQLite type | Python type | ++=============+==============================================+ +| ``NULL`` | :const:`None` | ++-------------+----------------------------------------------+ +| ``INTEGER`` | :class:`int` | ++-------------+----------------------------------------------+ +| ``REAL`` | :class:`float` | ++-------------+----------------------------------------------+ +| ``TEXT`` | depends on :attr:`~Connection.text_factory`, | +| | :class:`str` by default | ++-------------+----------------------------------------------+ +| ``BLOB`` | :class:`bytes` | ++-------------+----------------------------------------------+ The type system of the :mod:`sqlite3` module is extensible in two ways: you can store additional Python types in a SQLite database via object adaptation, and @@ -742,9 +750,6 @@ sqlite3 module's supported types for SQLite: one of NoneType, int, float, str, bytes. -The :mod:`sqlite3` module uses Python object adaptation, as described in -:pep:`246` for this. The protocol to use is :class:`PrepareProtocol`. - There are two ways to enable the :mod:`sqlite3` module to adapt a custom Python type to one of the supported ones. @@ -800,8 +805,8 @@ .. note:: - Converter functions **always** get called with a string, no matter under which - data type you sent the value to SQLite. + Converter functions **always** get called with a :class:`bytes` object, no + matter under which data type you sent the value to SQLite. :: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 19:23:05 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 1 Apr 2014 19:23:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2315067=3A_Merge_port_of_2=2E7_sqlite3_docs=2E?= Message-ID: <3fyy6d5ZMXz7LkW@mail.python.org> http://hg.python.org/cpython/rev/2c7ebb930b64 changeset: 90089:2c7ebb930b64 parent: 90086:2fbba6d68b94 parent: 90088:4a2dabac976d user: Zachary Ware date: Tue Apr 01 12:22:51 2014 -0500 summary: Closes #15067: Merge port of 2.7 sqlite3 docs. files: Doc/library/sqlite3.rst | 97 +++++++++++++++------------- 1 files changed, 51 insertions(+), 46 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -13,8 +13,8 @@ application using SQLite and then port the code to a larger database such as PostgreSQL or Oracle. -sqlite3 was written by Gerhard H?ring and provides a SQL interface compliant -with the DB-API 2.0 specification described by :pep:`249`. +The sqlite3 module was written by Gerhard H?ring. It provides a SQL interface +compliant with the DB-API 2.0 specification described by :pep:`249`. To use the module, you must first create a :class:`Connection` object that represents the database. Here the data will be stored in the @@ -31,23 +31,29 @@ c = conn.cursor() # Create table - c.execute('''create table stocks - (date text, trans text, symbol text, - qty real, price real)''') + c.execute('''CREATE TABLE stocks + (date text, trans text, symbol text, qty real, price real)''') # Insert a row of data - c.execute("""insert into stocks - values ('2006-01-05','BUY','RHAT',100,35.14)""") + c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)") # Save (commit) the changes conn.commit() - # We can also close the cursor if we are done with it - c.close() + # We can also close the connection if we are done with it. + # Just be sure any changes have been committed or they will be lost. + conn.close() + +The data you've saved is persistent and is available in subsequent sessions:: + + import sqlite3 + conn = sqlite3.connect('example.db') + c = conn.cursor() Usually your SQL operations will need to use values from Python variables. You shouldn't assemble your query using Python's string operations because doing so -is insecure; it makes your program vulnerable to an SQL injection attack. +is insecure; it makes your program vulnerable to an SQL injection attack +(see http://xkcd.com/327/ for humorous example of what can go wrong). Instead, use the DB-API's parameter substitution. Put ``?`` as a placeholder wherever you want to use a value, and then provide a tuple of values as the @@ -56,19 +62,20 @@ example:: # Never do this -- insecure! - symbol = 'IBM' - c.execute("select * from stocks where symbol = '%s'" % symbol) + symbol = 'RHAT' + c.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol) # Do this instead - t = ('IBM',) - c.execute('select * from stocks where symbol=?', t) + t = ('RHAT',) + c.execute('SELECT * FROM stocks WHERE symbol=?', t) + print(c.fetchone()) - # Larger example - for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), - ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), - ('2006-04-06', 'SELL', 'IBM', 500, 53.00), - ]: - c.execute('insert into stocks values (?,?,?,?,?)', t) + # Larger example that inserts many records at a time + purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00), + ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00), + ('2006-04-06', 'SELL', 'IBM', 500, 53.00), + ] + c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases) To retrieve data after executing a SELECT statement, you can either treat the cursor as an :term:`iterator`, call the cursor's :meth:`~Cursor.fetchone` method to @@ -77,16 +84,13 @@ This example uses the iterator form:: - >>> c = conn.cursor() - >>> c.execute('select * from stocks order by price') - >>> for row in c: - ... print(row) - ... + >>> for row in c.execute('SELECT * FROM stocks ORDER BY price'): + print(row) + ('2006-01-05', 'BUY', 'RHAT', 100, 35.14) ('2006-03-28', 'BUY', 'IBM', 1000, 45.0) ('2006-04-06', 'SELL', 'IBM', 500, 53.0) - ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.0) - >>> + ('2006-04-05', 'BUY', 'MSFT', 1000, 72.0) .. seealso:: @@ -99,6 +103,9 @@ The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect. + http://www.w3schools.com/sql/ + Tutorial, reference and examples for learning SQL syntax. + :pep:`249` - Database API Specification 2.0 PEP written by Marc-Andr? Lemburg. @@ -517,7 +524,7 @@ .. method:: execute(sql, [parameters]) - Executes an SQL statement. The SQL statement may be parametrized (i. e. + Executes an SQL statement. The SQL statement may be parameterized (i. e. placeholders instead of SQL literals). The :mod:`sqlite3` module supports two kinds of placeholders: question marks (qmark style) and named placeholders (named style). @@ -714,19 +721,20 @@ This is how SQLite types are converted to Python types by default: -+-------------+---------------------------------------------+ -| SQLite type | Python type | -+=============+=============================================+ -| ``NULL`` | :const:`None` | -+-------------+---------------------------------------------+ -| ``INTEGER`` | :class:`int` | -+-------------+---------------------------------------------+ -| ``REAL`` | :class:`float` | -+-------------+---------------------------------------------+ -| ``TEXT`` | depends on text_factory, str by default | -+-------------+---------------------------------------------+ -| ``BLOB`` | :class:`bytes` | -+-------------+---------------------------------------------+ ++-------------+----------------------------------------------+ +| SQLite type | Python type | ++=============+==============================================+ +| ``NULL`` | :const:`None` | ++-------------+----------------------------------------------+ +| ``INTEGER`` | :class:`int` | ++-------------+----------------------------------------------+ +| ``REAL`` | :class:`float` | ++-------------+----------------------------------------------+ +| ``TEXT`` | depends on :attr:`~Connection.text_factory`, | +| | :class:`str` by default | ++-------------+----------------------------------------------+ +| ``BLOB`` | :class:`bytes` | ++-------------+----------------------------------------------+ The type system of the :mod:`sqlite3` module is extensible in two ways: you can store additional Python types in a SQLite database via object adaptation, and @@ -742,9 +750,6 @@ sqlite3 module's supported types for SQLite: one of NoneType, int, float, str, bytes. -The :mod:`sqlite3` module uses Python object adaptation, as described in -:pep:`246` for this. The protocol to use is :class:`PrepareProtocol`. - There are two ways to enable the :mod:`sqlite3` module to adapt a custom Python type to one of the supported ones. @@ -800,8 +805,8 @@ .. note:: - Converter functions **always** get called with a string, no matter under which - data type you sent the value to SQLite. + Converter functions **always** get called with a :class:`bytes` object, no + matter under which data type you sent the value to SQLite. :: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 20:21:10 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 1 Apr 2014 20:21:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_simplify_check?= =?utf-8?q?=2C_since_now_there_are_only_new-style_classes?= Message-ID: <3fyzPf5WxCz7LjT@mail.python.org> http://hg.python.org/cpython/rev/abb85902ce79 changeset: 90090:abb85902ce79 branch: 3.4 parent: 90088:4a2dabac976d user: Benjamin Peterson date: Tue Apr 01 14:20:56 2014 -0400 summary: simplify check, since now there are only new-style classes files: Lib/urllib/request.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -511,9 +511,6 @@ If any of the handlers passed as arguments are subclasses of the default handlers, the default handlers will not be used. """ - def isclass(obj): - return isinstance(obj, type) or hasattr(obj, "__bases__") - opener = OpenerDirector() default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, HTTPDefaultErrorHandler, HTTPRedirectHandler, @@ -524,7 +521,7 @@ skip = set() for klass in default_classes: for check in handlers: - if isclass(check): + if instance(check, type): if issubclass(check, klass): skip.add(klass) elif isinstance(check, klass): @@ -536,7 +533,7 @@ opener.add_handler(klass()) for h in handlers: - if isclass(h): + if isinstance(h, type): h = h() opener.add_handler(h) return opener -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 20:21:12 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 1 Apr 2014 20:21:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3fyzPh078Tz7LjT@mail.python.org> http://hg.python.org/cpython/rev/72da7e4af99e changeset: 90091:72da7e4af99e parent: 90089:2c7ebb930b64 parent: 90090:abb85902ce79 user: Benjamin Peterson date: Tue Apr 01 14:21:03 2014 -0400 summary: merge 3.4 files: Lib/urllib/request.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -511,9 +511,6 @@ If any of the handlers passed as arguments are subclasses of the default handlers, the default handlers will not be used. """ - def isclass(obj): - return isinstance(obj, type) or hasattr(obj, "__bases__") - opener = OpenerDirector() default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, HTTPDefaultErrorHandler, HTTPRedirectHandler, @@ -524,7 +521,7 @@ skip = set() for klass in default_classes: for check in handlers: - if isclass(check): + if instance(check, type): if issubclass(check, klass): skip.add(klass) elif isinstance(check, klass): @@ -536,7 +533,7 @@ opener.add_handler(klass()) for h in handlers: - if isclass(h): + if isinstance(h, type): h = h() opener.add_handler(h) return opener -- Repository URL: http://hg.python.org/cpython From zachary.ware+pydev at gmail.com Tue Apr 1 20:28:53 2014 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Tue, 1 Apr 2014 13:28:53 -0500 Subject: [Python-checkins] cpython (3.4): simplify check, since now there are only new-style classes In-Reply-To: <3fyzPf5WxCz7LjT@mail.python.org> References: <3fyzPf5WxCz7LjT@mail.python.org> Message-ID: On Tue, Apr 1, 2014 at 1:21 PM, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/abb85902ce79 > changeset: 90090:abb85902ce79 > branch: 3.4 > parent: 90088:4a2dabac976d > user: Benjamin Peterson > date: Tue Apr 01 14:20:56 2014 -0400 > summary: > simplify check, since now there are only new-style classes > > files: > Lib/urllib/request.py | 7 ++----- > 1 files changed, 2 insertions(+), 5 deletions(-) > > > diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py > --- a/Lib/urllib/request.py > +++ b/Lib/urllib/request.py > @@ -511,9 +511,6 @@ > If any of the handlers passed as arguments are subclasses of the > default handlers, the default handlers will not be used. > """ > - def isclass(obj): > - return isinstance(obj, type) or hasattr(obj, "__bases__") > - > opener = OpenerDirector() > default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, > HTTPDefaultErrorHandler, HTTPRedirectHandler, > @@ -524,7 +521,7 @@ > skip = set() > for klass in default_classes: > for check in handlers: > - if isclass(check): > + if instance(check, type): Should be isinstance, should it not? -- Zach From solipsis at pitrou.net Tue Apr 1 20:36:24 2014 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 1 Apr 2014 20:36:24 +0200 Subject: [Python-checkins] cpython (3.4): simplify check, since now there are only new-style classes References: <3fyzPf5WxCz7LjT@mail.python.org> Message-ID: <20140401203624.0860bb40@fsol> On Tue, 1 Apr 2014 13:28:53 -0500 Zachary Ware wrote: > > @@ -524,7 +521,7 @@ > > skip = set() > > for klass in default_classes: > > for check in handlers: > > - if isclass(check): > > + if instance(check, type): > > Should be isinstance, should it not? Sounds like a well-tested code path :-) Regards Antoine. From python-checkins at python.org Tue Apr 1 22:27:41 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 1 Apr 2014 22:27:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_typo?= Message-ID: <3fz2Cd6lSBz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/1039d4ebc4bb changeset: 90092:1039d4ebc4bb branch: 3.4 parent: 90090:abb85902ce79 user: Benjamin Peterson date: Tue Apr 01 16:27:30 2014 -0400 summary: fix typo files: Lib/urllib/request.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -521,7 +521,7 @@ skip = set() for klass in default_classes: for check in handlers: - if instance(check, type): + if isinstance(check, type): if issubclass(check, klass): skip.add(klass) elif isinstance(check, klass): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 1 22:27:43 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 1 Apr 2014 22:27:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3fz2Cg1Pjhz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/ab34f1425c1d changeset: 90093:ab34f1425c1d parent: 90091:72da7e4af99e parent: 90092:1039d4ebc4bb user: Benjamin Peterson date: Tue Apr 01 16:27:35 2014 -0400 summary: merge 3.4 files: Lib/urllib/request.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -521,7 +521,7 @@ skip = set() for klass in default_classes: for check in handlers: - if instance(check, type): + if isinstance(check, type): if issubclass(check, klass): skip.add(klass) elif isinstance(check, klass): -- Repository URL: http://hg.python.org/cpython From benjamin at python.org Tue Apr 1 22:28:07 2014 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 01 Apr 2014 13:28:07 -0700 Subject: [Python-checkins] cpython (3.4): simplify check, since now there are only new-style classes In-Reply-To: References: <3fyzPf5WxCz7LjT@mail.python.org> Message-ID: <1396384087.19957.101588241.68209611@webmail.messagingengine.com> On Tue, Apr 1, 2014, at 11:28, Zachary Ware wrote: > On Tue, Apr 1, 2014 at 1:21 PM, benjamin.peterson > wrote: > > http://hg.python.org/cpython/rev/abb85902ce79 > > changeset: 90090:abb85902ce79 > > branch: 3.4 > > parent: 90088:4a2dabac976d > > user: Benjamin Peterson > > date: Tue Apr 01 14:20:56 2014 -0400 > > summary: > > simplify check, since now there are only new-style classes > > > > files: > > Lib/urllib/request.py | 7 ++----- > > 1 files changed, 2 insertions(+), 5 deletions(-) > > > > > > diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py > > --- a/Lib/urllib/request.py > > +++ b/Lib/urllib/request.py > > @@ -511,9 +511,6 @@ > > If any of the handlers passed as arguments are subclasses of the > > default handlers, the default handlers will not be used. > > """ > > - def isclass(obj): > > - return isinstance(obj, type) or hasattr(obj, "__bases__") > > - > > opener = OpenerDirector() > > default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, > > HTTPDefaultErrorHandler, HTTPRedirectHandler, > > @@ -524,7 +521,7 @@ > > skip = set() > > for klass in default_classes: > > for check in handlers: > > - if isclass(check): > > + if instance(check, type): > > Should be isinstance, should it not? Absolutely. From python-checkins at python.org Wed Apr 2 01:22:27 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 01:22:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E2=29=3A_remove_directo?= =?utf-8?q?ry_mode_check_from_makedirs_=28closes_=2321082=29?= Message-ID: <3fz65H6vPBz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/9186f4a18584 changeset: 90094:9186f4a18584 branch: 3.2 parent: 90070:fc28ce0983de user: Benjamin Peterson date: Tue Apr 01 19:13:18 2014 -0400 summary: remove directory mode check from makedirs (closes #21082) files: Doc/library/os.rst | 14 +++++++++----- Lib/os.py | 28 +++++----------------------- Lib/test/test_os.py | 7 +++---- Misc/NEWS | 3 +++ 4 files changed, 20 insertions(+), 32 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1188,11 +1188,8 @@ The default *mode* is ``0o777`` (octal). On some systems, *mode* is ignored. Where it is used, the current umask value is first masked out. - If *exists_ok* is ``False`` (the default), an :exc:`OSError` is raised if - the target directory already exists. If *exists_ok* is ``True`` an - :exc:`OSError` is still raised if the umask-masked *mode* is different from - the existing mode, on systems where the mode is used. :exc:`OSError` will - also be raised if the directory creation fails. + If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if the + target directory already exists. .. note:: @@ -1204,6 +1201,13 @@ .. versionadded:: 3.2 The *exist_ok* parameter. + .. versionchanged:: 3.2.5 + + Before Python 3.2.5, if *exist_ok* was ``True`` and the directory existed, + :func:`makedirs` would still raise an error if *mode* did not match the + mode of the existing directory. Since this behavior was impossible to + implement safely, it was removed in Python 3.2.6. See :issue:`21082`. + .. function:: pathconf(path, name) diff --git a/Lib/os.py b/Lib/os.py --- a/Lib/os.py +++ b/Lib/os.py @@ -114,12 +114,6 @@ SEEK_CUR = 1 SEEK_END = 2 - -def _get_masked_mode(mode): - mask = umask(0) - umask(mask) - return mode & ~mask - #' # Super directory utilities. @@ -128,11 +122,10 @@ def makedirs(name, mode=0o777, exist_ok=False): """makedirs(path [, mode=0o777][, exist_ok=False]) - Super-mkdir; create a leaf directory and all intermediate ones. - Works like mkdir, except that any intermediate path segment (not - just the rightmost) will be created if it does not exist. If the - target directory with the same mode as we specified already exists, - raises an OSError if exist_ok is False, otherwise no exception is + Super-mkdir; create a leaf directory and all intermediate ones. Works like + mkdir, except that any intermediate path segment (not just the rightmost) + will be created if it does not exist. If the target directory already + exists, raise an OSError if exist_ok is False. Otherwise no exception is raised. This is recursive. """ @@ -154,18 +147,7 @@ try: mkdir(name, mode) except OSError as e: - import stat as st - dir_exists = path.isdir(name) - expected_mode = _get_masked_mode(mode) - if dir_exists: - # S_ISGID is automatically copied by the OS from parent to child - # directories on mkdir. Don't consider it being set to be a mode - # mismatch as mkdir does not unset it when not specified in mode. - actual_mode = st.S_IMODE(lstat(name).st_mode) & ~st.S_ISGID - else: - actual_mode = -1 - if not (e.errno == errno.EEXIST and exist_ok and dir_exists and - actual_mode == expected_mode): + if not exist_ok or e.errno != errno.EEXIST or not path.isdir(name): raise def removedirs(name): diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -579,7 +579,7 @@ os.makedirs(path, mode) self.assertRaises(OSError, os.makedirs, path, mode) self.assertRaises(OSError, os.makedirs, path, mode, exist_ok=False) - self.assertRaises(OSError, os.makedirs, path, 0o776, exist_ok=True) + os.makedirs(path, 0o776, exist_ok=True) os.makedirs(path, mode=mode, exist_ok=True) finally: os.umask(old_mask) @@ -606,9 +606,8 @@ os.makedirs(path, mode, exist_ok=True) # remove the bit. os.chmod(path, stat.S_IMODE(os.lstat(path).st_mode) & ~S_ISGID) - with self.assertRaises(OSError): - # Should fail when the bit is not already set when demanded. - os.makedirs(path, mode | S_ISGID, exist_ok=True) + # May work even when the bit is not already set when demanded. + os.makedirs(path, mode | S_ISGID, exist_ok=True) finally: os.umask(old_mask) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Library ------- +- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this + changes behavior of makedirs when exist_ok=True. + - Issue #20246: Fix buffer overflow in socket.recvfrom_into. - Issue #12226: HTTPS is now used by default when connecting to PyPI. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 01:22:29 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 01:22:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2_=28=2321082=29?= Message-ID: <3fz65K2hLjz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/6370d44013f7 changeset: 90095:6370d44013f7 branch: 3.3 parent: 90071:cb3a8abc0870 parent: 90094:9186f4a18584 user: Benjamin Peterson date: Tue Apr 01 19:17:57 2014 -0400 summary: merge 3.2 (#21082) files: Doc/library/os.rst | 14 +++++++++----- Lib/os.py | 30 +++++------------------------- Lib/test/test_os.py | 7 +++---- Misc/NEWS | 3 +++ 4 files changed, 20 insertions(+), 34 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1563,11 +1563,8 @@ The default *mode* is ``0o777`` (octal). On some systems, *mode* is ignored. Where it is used, the current umask value is first masked out. - If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if - the target directory already exists. If *exist_ok* is ``True`` an - :exc:`OSError` is still raised if the umask-masked *mode* is different from - the existing mode, on systems where the mode is used. :exc:`OSError` will - also be raised if the directory creation fails. + If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if the + target directory already exists. .. note:: @@ -1579,6 +1576,13 @@ .. versionadded:: 3.2 The *exist_ok* parameter. + .. versionchanged:: 3.3.6 + + Before Python 3.3.6, if *exist_ok* was ``True`` and the directory existed, + :func:`makedirs` would still raise an error if *mode* did not match the + mode of the existing directory. Since this behavior was impossible to + implement safely, it was removed in Python 3.3.6. See :issue:`21082`. + .. function:: mkfifo(path, mode=0o666, *, dir_fd=None) diff --git a/Lib/os.py b/Lib/os.py --- a/Lib/os.py +++ b/Lib/os.py @@ -230,23 +230,16 @@ SEEK_CUR = 1 SEEK_END = 2 - -def _get_masked_mode(mode): - mask = umask(0) - umask(mask) - return mode & ~mask - # Super directory utilities. # (Inspired by Eric Raymond; the doc strings are mostly his) def makedirs(name, mode=0o777, exist_ok=False): """makedirs(path [, mode=0o777][, exist_ok=False]) - Super-mkdir; create a leaf directory and all intermediate ones. - Works like mkdir, except that any intermediate path segment (not - just the rightmost) will be created if it does not exist. If the - target directory with the same mode as we specified already exists, - raises an OSError if exist_ok is False, otherwise no exception is + Super-mkdir; create a leaf directory and all intermediate ones. Works like + mkdir, except that any intermediate path segment (not just the rightmost) + will be created if it does not exist. If the target directory already + exists, raise an OSError if exist_ok is False. Otherwise no exception is raised. This is recursive. """ @@ -268,20 +261,7 @@ try: mkdir(name, mode) except OSError as e: - dir_exists = path.isdir(name) - expected_mode = _get_masked_mode(mode) - if dir_exists: - # S_ISGID is automatically copied by the OS from parent to child - # directories on mkdir. Don't consider it being set to be a mode - # mismatch as mkdir does not unset it when not specified in mode. - actual_mode = st.S_IMODE(lstat(name).st_mode) & ~st.S_ISGID - else: - actual_mode = -1 - if not (e.errno == errno.EEXIST and exist_ok and dir_exists and - actual_mode == expected_mode): - if dir_exists and actual_mode != expected_mode: - e.strerror += ' (mode %o != expected mode %o)' % ( - actual_mode, expected_mode) + if not exist_ok or e.errno != errno.EEXIST or not path.isdir(name): raise def removedirs(name): diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -872,7 +872,7 @@ os.makedirs(path, mode) self.assertRaises(OSError, os.makedirs, path, mode) self.assertRaises(OSError, os.makedirs, path, mode, exist_ok=False) - self.assertRaises(OSError, os.makedirs, path, 0o776, exist_ok=True) + os.makedirs(path, 0o776, exist_ok=True) os.makedirs(path, mode=mode, exist_ok=True) os.umask(old_mask) @@ -898,9 +898,8 @@ os.makedirs(path, mode, exist_ok=True) # remove the bit. os.chmod(path, stat.S_IMODE(os.lstat(path).st_mode) & ~S_ISGID) - with self.assertRaises(OSError): - # Should fail when the bit is not already set when demanded. - os.makedirs(path, mode | S_ISGID, exist_ok=True) + # May work even when the bit is not already set when demanded. + os.makedirs(path, mode | S_ISGID, exist_ok=True) finally: os.umask(old_mask) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,6 +15,9 @@ - Issue #20633: Replace relative import by absolute import. +- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this + changes behavior of makedirs when exist_ok=True. + - Issue #20875: Prevent possible gzip "'read' is not defined" NameError. Patch by Claudiu Popa. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 01:22:30 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 01:22:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3_=28=2321082=29?= Message-ID: <3fz65L5LK4z7LlX@mail.python.org> http://hg.python.org/cpython/rev/c24dd53ab4b9 changeset: 90096:c24dd53ab4b9 branch: 3.4 parent: 90092:1039d4ebc4bb parent: 90095:6370d44013f7 user: Benjamin Peterson date: Tue Apr 01 19:18:48 2014 -0400 summary: merge 3.3 (#21082) files: Doc/library/os.rst | 14 +++++++++----- Lib/os.py | 30 +++++------------------------- Lib/test/test_os.py | 7 +++---- Misc/NEWS | 3 +++ 4 files changed, 20 insertions(+), 34 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1619,11 +1619,8 @@ The default *mode* is ``0o777`` (octal). On some systems, *mode* is ignored. Where it is used, the current umask value is first masked out. - If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if - the target directory already exists. If *exist_ok* is ``True`` an - :exc:`OSError` is still raised if the umask-masked *mode* is different from - the existing mode, on systems where the mode is used. :exc:`OSError` will - also be raised if the directory creation fails. + If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if the + target directory already exists. .. note:: @@ -1635,6 +1632,13 @@ .. versionadded:: 3.2 The *exist_ok* parameter. + .. versionchanged:: 3.3.6 + + Before Python 3.3.6, if *exist_ok* was ``True`` and the directory existed, + :func:`makedirs` would still raise an error if *mode* did not match the + mode of the existing directory. Since this behavior was impossible to + implement safely, it was removed in Python 3.3.6. See :issue:`21082`. + .. function:: mkfifo(path, mode=0o666, *, dir_fd=None) diff --git a/Lib/os.py b/Lib/os.py --- a/Lib/os.py +++ b/Lib/os.py @@ -206,23 +206,16 @@ SEEK_CUR = 1 SEEK_END = 2 - -def _get_masked_mode(mode): - mask = umask(0) - umask(mask) - return mode & ~mask - # Super directory utilities. # (Inspired by Eric Raymond; the doc strings are mostly his) def makedirs(name, mode=0o777, exist_ok=False): """makedirs(name [, mode=0o777][, exist_ok=False]) - Super-mkdir; create a leaf directory and all intermediate ones. - Works like mkdir, except that any intermediate path segment (not - just the rightmost) will be created if it does not exist. If the - target directory with the same mode as we specified already exists, - raises an OSError if exist_ok is False, otherwise no exception is + Super-mkdir; create a leaf directory and all intermediate ones. Works like + mkdir, except that any intermediate path segment (not just the rightmost) + will be created if it does not exist. If the target directory already + exists, raise an OSError if exist_ok is False. Otherwise no exception is raised. This is recursive. """ @@ -243,20 +236,7 @@ try: mkdir(name, mode) except OSError as e: - dir_exists = path.isdir(name) - expected_mode = _get_masked_mode(mode) - if dir_exists: - # S_ISGID is automatically copied by the OS from parent to child - # directories on mkdir. Don't consider it being set to be a mode - # mismatch as mkdir does not unset it when not specified in mode. - actual_mode = st.S_IMODE(lstat(name).st_mode) & ~st.S_ISGID - else: - actual_mode = -1 - if not (e.errno == errno.EEXIST and exist_ok and dir_exists and - actual_mode == expected_mode): - if dir_exists and actual_mode != expected_mode: - e.strerror += ' (mode %o != expected mode %o)' % ( - actual_mode, expected_mode) + if not exist_ok or e.errno != errno.EEXIST or not path.isdir(name): raise def removedirs(name): diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -901,7 +901,7 @@ os.makedirs(path, mode) self.assertRaises(OSError, os.makedirs, path, mode) self.assertRaises(OSError, os.makedirs, path, mode, exist_ok=False) - self.assertRaises(OSError, os.makedirs, path, 0o776, exist_ok=True) + os.makedirs(path, 0o776, exist_ok=True) os.makedirs(path, mode=mode, exist_ok=True) os.umask(old_mask) @@ -938,9 +938,8 @@ os.makedirs(path, mode, exist_ok=True) # remove the bit. os.chmod(path, stat.S_IMODE(os.lstat(path).st_mode) & ~S_ISGID) - with self.assertRaises(OSError): - # Should fail when the bit is not already set when demanded. - os.makedirs(path, mode | S_ISGID, exist_ok=True) + # May work even when the bit is not already set when demanded. + os.makedirs(path, mode | S_ISGID, exist_ok=True) finally: os.umask(old_mask) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -38,6 +38,9 @@ - Issue #20980: Stop wrapping exception when using ThreadPool. +- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this + changes behavior of makedirs when exist_ok=True. + - Issue #20990: Fix issues found by pyflakes for multiprocessing. - Issue #21015: SSL contexts will now automatically select an elliptic -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 01:22:32 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 01:22:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjEwODIp?= Message-ID: <3fz65N1HsCz7Lmh@mail.python.org> http://hg.python.org/cpython/rev/adfcdc831e98 changeset: 90097:adfcdc831e98 parent: 90093:ab34f1425c1d parent: 90096:c24dd53ab4b9 user: Benjamin Peterson date: Tue Apr 01 19:21:57 2014 -0400 summary: merge 3.4 (#21082) files: Doc/library/os.rst | 14 +++++++++----- Lib/os.py | 30 +++++------------------------- Lib/test/test_os.py | 7 +++---- Misc/NEWS | 3 +++ 4 files changed, 20 insertions(+), 34 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1619,11 +1619,8 @@ The default *mode* is ``0o777`` (octal). On some systems, *mode* is ignored. Where it is used, the current umask value is first masked out. - If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if - the target directory already exists. If *exist_ok* is ``True`` an - :exc:`OSError` is still raised if the umask-masked *mode* is different from - the existing mode, on systems where the mode is used. :exc:`OSError` will - also be raised if the directory creation fails. + If *exist_ok* is ``False`` (the default), an :exc:`OSError` is raised if the + target directory already exists. .. note:: @@ -1635,6 +1632,13 @@ .. versionadded:: 3.2 The *exist_ok* parameter. + .. versionchanged:: 3.3.6 + + Before Python 3.3.6, if *exist_ok* was ``True`` and the directory existed, + :func:`makedirs` would still raise an error if *mode* did not match the + mode of the existing directory. Since this behavior was impossible to + implement safely, it was removed in Python 3.3.6. See :issue:`21082`. + .. function:: mkfifo(path, mode=0o666, *, dir_fd=None) diff --git a/Lib/os.py b/Lib/os.py --- a/Lib/os.py +++ b/Lib/os.py @@ -206,23 +206,16 @@ SEEK_CUR = 1 SEEK_END = 2 - -def _get_masked_mode(mode): - mask = umask(0) - umask(mask) - return mode & ~mask - # Super directory utilities. # (Inspired by Eric Raymond; the doc strings are mostly his) def makedirs(name, mode=0o777, exist_ok=False): """makedirs(name [, mode=0o777][, exist_ok=False]) - Super-mkdir; create a leaf directory and all intermediate ones. - Works like mkdir, except that any intermediate path segment (not - just the rightmost) will be created if it does not exist. If the - target directory with the same mode as we specified already exists, - raises an OSError if exist_ok is False, otherwise no exception is + Super-mkdir; create a leaf directory and all intermediate ones. Works like + mkdir, except that any intermediate path segment (not just the rightmost) + will be created if it does not exist. If the target directory already + exists, raise an OSError if exist_ok is False. Otherwise no exception is raised. This is recursive. """ @@ -243,20 +236,7 @@ try: mkdir(name, mode) except OSError as e: - dir_exists = path.isdir(name) - expected_mode = _get_masked_mode(mode) - if dir_exists: - # S_ISGID is automatically copied by the OS from parent to child - # directories on mkdir. Don't consider it being set to be a mode - # mismatch as mkdir does not unset it when not specified in mode. - actual_mode = st.S_IMODE(lstat(name).st_mode) & ~st.S_ISGID - else: - actual_mode = -1 - if not (e.errno == errno.EEXIST and exist_ok and dir_exists and - actual_mode == expected_mode): - if dir_exists and actual_mode != expected_mode: - e.strerror += ' (mode %o != expected mode %o)' % ( - actual_mode, expected_mode) + if not exist_ok or e.errno != errno.EEXIST or not path.isdir(name): raise def removedirs(name): diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -901,7 +901,7 @@ os.makedirs(path, mode) self.assertRaises(OSError, os.makedirs, path, mode) self.assertRaises(OSError, os.makedirs, path, mode, exist_ok=False) - self.assertRaises(OSError, os.makedirs, path, 0o776, exist_ok=True) + os.makedirs(path, 0o776, exist_ok=True) os.makedirs(path, mode=mode, exist_ok=True) os.umask(old_mask) @@ -938,9 +938,8 @@ os.makedirs(path, mode, exist_ok=True) # remove the bit. os.chmod(path, stat.S_IMODE(os.lstat(path).st_mode) & ~S_ISGID) - with self.assertRaises(OSError): - # Should fail when the bit is not already set when demanded. - os.makedirs(path, mode | S_ISGID, exist_ok=True) + # May work even when the bit is not already set when demanded. + os.makedirs(path, mode | S_ISGID, exist_ok=True) finally: os.umask(old_mask) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,9 @@ - Issue #20980: Stop wrapping exception when using ThreadPool. +- Issue #21082: In os.makedirs, do not set the process-wide umask. Note this + changes behavior of makedirs when exist_ok=True. + - Issue #20990: Fix issues found by pyflakes for multiprocessing. - Issue #21015: SSL contexts will now automatically select an elliptic -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 01:22:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 01:22:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_update_version?= =?utf-8?q?_changed?= Message-ID: <3fz65P38ZKz7LrX@mail.python.org> http://hg.python.org/cpython/rev/2099da2fd317 changeset: 90098:2099da2fd317 branch: 3.4 parent: 90096:c24dd53ab4b9 user: Benjamin Peterson date: Tue Apr 01 19:22:06 2014 -0400 summary: update version changed files: Doc/library/os.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1632,12 +1632,12 @@ .. versionadded:: 3.2 The *exist_ok* parameter. - .. versionchanged:: 3.3.6 - - Before Python 3.3.6, if *exist_ok* was ``True`` and the directory existed, + .. versionchanged:: 3.4.1 + + Before Python 3.4.1, if *exist_ok* was ``True`` and the directory existed, :func:`makedirs` would still raise an error if *mode* did not match the mode of the existing directory. Since this behavior was impossible to - implement safely, it was removed in Python 3.3.6. See :issue:`21082`. + implement safely, it was removed in Python 3.4.1. See :issue:`21082`. .. function:: mkfifo(path, mode=0o666, *, dir_fd=None) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 01:22:34 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 01:22:34 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3fz65Q4f8dz7Lkb@mail.python.org> http://hg.python.org/cpython/rev/42433f4fa6d5 changeset: 90099:42433f4fa6d5 parent: 90097:adfcdc831e98 parent: 90098:2099da2fd317 user: Benjamin Peterson date: Tue Apr 01 19:22:14 2014 -0400 summary: merge 3.4 files: Doc/library/os.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1632,12 +1632,12 @@ .. versionadded:: 3.2 The *exist_ok* parameter. - .. versionchanged:: 3.3.6 - - Before Python 3.3.6, if *exist_ok* was ``True`` and the directory existed, + .. versionchanged:: 3.4.1 + + Before Python 3.4.1, if *exist_ok* was ``True`` and the directory existed, :func:`makedirs` would still raise an error if *mode* did not match the mode of the existing directory. Since this behavior was impossible to - implement safely, it was removed in Python 3.3.6. See :issue:`21082`. + implement safely, it was removed in Python 3.4.1. See :issue:`21082`. .. function:: mkfifo(path, mode=0o666, *, dir_fd=None) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 05:57:00 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 05:57:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E6=29=3A_Added_tag_v2?= =?utf-8?q?=2E6=2E7_for_changeset_a39a30ac7f2e?= Message-ID: <3fzDB47517z7Ljk@mail.python.org> http://hg.python.org/cpython/rev/23a60d89dbd4 changeset: 90100:23a60d89dbd4 branch: 2.6 parent: 86774:3edf679917ba user: Benjamin Peterson date: Tue Apr 01 23:55:11 2014 -0400 summary: Added tag v2.6.7 for changeset a39a30ac7f2e files: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -142,3 +142,4 @@ c9910fd022fc842e5578e1bf5a30ba55a37239fc v2.6.8 a0025037f11a73df5a7dd03e5a4027adad4cb94e v2.6.9rc1 fcb3ec2842f99454322492dd0ec2cf01322df093 v2.6.9 +a39a30ac7f2e6ba9c932249973197a984de25396 v2.6.7 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 07:11:45 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 07:11:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMjExMjU6?= =?utf-8?q?_minor_wording_tweak=2E?= Message-ID: <3fzFrK0Scsz7Lk7@mail.python.org> http://hg.python.org/cpython/rev/1c112eb4eb56 changeset: 90101:1c112eb4eb56 branch: 2.7 parent: 90087:2e13f0d49b8e user: Raymond Hettinger date: Tue Apr 01 22:11:34 2014 -0700 summary: Issue 21125: minor wording tweak. files: Doc/library/traceback.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -75,7 +75,7 @@ Return a list of up to *limit* "pre-processed" stack trace entries extracted from the traceback object *traceback*. It is useful for alternate formatting of stack traces. If *limit* is omitted or ``None``, all entries are extracted. A - "pre-processed" stack trace entry is a quadruple (*filename*, *line number*, + "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, *function name*, *text*) representing the information that is usually printed for a stack trace. The *text* is a string with leading and trailing whitespace stripped; if the source is not available it is ``None``. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 07:18:14 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 07:18:14 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjExMjU6?= =?utf-8?q?_minor_documentation_tweak=2E?= Message-ID: <3fzFzp29cPz7LlK@mail.python.org> http://hg.python.org/cpython/rev/dfa399e74fcf changeset: 90102:dfa399e74fcf branch: 3.4 parent: 90098:2099da2fd317 user: Raymond Hettinger date: Tue Apr 01 22:17:33 2014 -0700 summary: Issue 21125: minor documentation tweak. files: Doc/library/traceback.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -72,7 +72,7 @@ Return a list of up to *limit* "pre-processed" stack trace entries extracted from the traceback object *traceback*. It is useful for alternate formatting of stack traces. If *limit* is omitted or ``None``, all entries are extracted. A - "pre-processed" stack trace entry is a quadruple (*filename*, *line number*, + "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, *function name*, *text*) representing the information that is usually printed for a stack trace. The *text* is a string with leading and trailing whitespace stripped; if the source is not available it is ``None``. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 07:18:15 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 07:18:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <3fzFzq4Rp4z7Ll6@mail.python.org> http://hg.python.org/cpython/rev/1effaabe739b changeset: 90103:1effaabe739b parent: 90099:42433f4fa6d5 parent: 90102:dfa399e74fcf user: Raymond Hettinger date: Tue Apr 01 22:17:57 2014 -0700 summary: merge files: Doc/library/traceback.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst --- a/Doc/library/traceback.rst +++ b/Doc/library/traceback.rst @@ -72,7 +72,7 @@ Return a list of up to *limit* "pre-processed" stack trace entries extracted from the traceback object *traceback*. It is useful for alternate formatting of stack traces. If *limit* is omitted or ``None``, all entries are extracted. A - "pre-processed" stack trace entry is a quadruple (*filename*, *line number*, + "pre-processed" stack trace entry is a 4-tuple (*filename*, *line number*, *function name*, *text*) representing the information that is usually printed for a stack trace. The *text* is a string with leading and trailing whitespace stripped; if the source is not available it is ``None``. -- Repository URL: http://hg.python.org/cpython From root at python.org Wed Apr 2 08:21:34 2014 From: root at python.org (Cron Daemon) Date: Wed, 02 Apr 2014 08:21:34 +0200 Subject: [Python-checkins] Cron /home/docs/build-devguide Message-ID: abort: error: Connection reset by peer From python-checkins at python.org Wed Apr 2 09:59:19 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 09:59:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Documentation_?= =?utf-8?q?nit=2E__List_the_traditional_keyword-only_argument_first_and_th?= =?utf-8?q?e?= Message-ID: <3fzKYg2yZFz7Lls@mail.python.org> http://hg.python.org/cpython/rev/8e7fe184acfb changeset: 90104:8e7fe184acfb branch: 3.4 parent: 90102:dfa399e74fcf user: Raymond Hettinger date: Wed Apr 02 00:58:47 2014 -0700 summary: Documentation nit. List the traditional keyword-only argument first and the default last. files: Doc/library/functions.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -773,7 +773,7 @@ already arranged into argument tuples, see :func:`itertools.starmap`\. -.. function:: max(iterable, *[, default, key]) +.. function:: max(iterable, *[, key, default]) max(arg1, arg2, *args[, key]) Return the largest item in an iterable or the largest of two or more @@ -807,7 +807,7 @@ :ref:`typememoryview` for more information. -.. function:: min(iterable, *[, default, key]) +.. function:: min(iterable, *[, key, default]) min(arg1, arg2, *args[, key]) Return the smallest item in an iterable or the smallest of two or more -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 09:59:20 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 09:59:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <3fzKYh4kHRz7Lsc@mail.python.org> http://hg.python.org/cpython/rev/2a41d3f81b46 changeset: 90105:2a41d3f81b46 parent: 90103:1effaabe739b parent: 90104:8e7fe184acfb user: Raymond Hettinger date: Wed Apr 02 00:59:11 2014 -0700 summary: merge files: Doc/library/functions.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -773,7 +773,7 @@ already arranged into argument tuples, see :func:`itertools.starmap`\. -.. function:: max(iterable, *[, default, key]) +.. function:: max(iterable, *[, key, default]) max(arg1, arg2, *args[, key]) Return the largest item in an iterable or the largest of two or more @@ -807,7 +807,7 @@ :ref:`typememoryview` for more information. -.. function:: min(iterable, *[, default, key]) +.. function:: min(iterable, *[, key, default]) min(arg1, arg2, *args[, key]) Return the smallest item in an iterable or the smallest of two or more -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Wed Apr 2 10:01:32 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 02 Apr 2014 10:01:32 +0200 Subject: [Python-checkins] Daily reference leaks (42433f4fa6d5): sum=4 Message-ID: results for 42433f4fa6d5 on branch "default" -------------------------------------------- test_asyncio leaked [4, 0, 0] memory blocks, sum=4 test_site leaked [0, -2, 2] references, sum=0 test_site leaked [0, -2, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogVKevLr', '-x'] From python-checkins at python.org Wed Apr 2 12:17:42 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 12:17:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE4NjUy?= =?utf-8?q?=3A__Add_an_itertools_recipe_for_first=5Ftrue=28=29?= Message-ID: <3fzNdL0K2pz7Ljx@mail.python.org> http://hg.python.org/cpython/rev/3e2354dde892 changeset: 90106:3e2354dde892 branch: 3.4 parent: 90104:8e7fe184acfb user: Raymond Hettinger date: Wed Apr 02 03:16:42 2014 -0700 summary: Issue #18652: Add an itertools recipe for first_true() files: Doc/library/itertools.rst | 13 +++++++++++++ Lib/test/test_itertools.py | 16 ++++++++++++++++ 2 files changed, 29 insertions(+), 0 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -784,6 +784,19 @@ except exception: pass + def first_true(iterable, default=False, pred=None): + """Returns the first true value in the iterable. + + If no true value is found, returns *default* + + If *pred* is not None, returns the first item + for which pred(item) is true. + + """ + # first_true([a,b,c], x) --> a or b or c or x + # first_true([a,b], x, f) --> a if f(a) else b if f(b) else x + return next(filter(pred, iterable), default) + def random_product(*args, repeat=1): "Random selection from itertools.product(*args, **kwds)" pools = [tuple(pool) for pool in args] * repeat diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1998,6 +1998,19 @@ ... # unique_justseen('ABBCcAD', str.lower) --> A B C A D ... return map(next, map(itemgetter(1), groupby(iterable, key))) +>>> def first_true(iterable, default=False, pred=None): +... '''Returns the first true value in the iterable. +... +... If no true value is found, returns *default* +... +... If *pred* is not None, returns the first item +... for which pred(item) is true. +... +... ''' +... # first_true([a,b,c], x) --> a or b or c or x +... # first_true([a,b], x, f) --> a if f(a) else b if f(b) else x +... return next(filter(pred, iterable), default) + This is not part of the examples but it tests to make sure the definitions perform as purported. @@ -2075,6 +2088,9 @@ >>> list(unique_justseen('ABBCcAD', str.lower)) ['A', 'B', 'C', 'A', 'D'] +>>> first_true('ABC0DEF1', '9', str.isdigit) +'0' + """ __test__ = {'libreftest' : libreftest} -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 12:17:43 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 2 Apr 2014 12:17:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <3fzNdM23L5z7Llr@mail.python.org> http://hg.python.org/cpython/rev/6154c7f2bf75 changeset: 90107:6154c7f2bf75 parent: 90105:2a41d3f81b46 parent: 90106:3e2354dde892 user: Raymond Hettinger date: Wed Apr 02 03:17:33 2014 -0700 summary: merge files: Doc/library/itertools.rst | 13 +++++++++++++ Lib/test/test_itertools.py | 16 ++++++++++++++++ 2 files changed, 29 insertions(+), 0 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -784,6 +784,19 @@ except exception: pass + def first_true(iterable, default=False, pred=None): + """Returns the first true value in the iterable. + + If no true value is found, returns *default* + + If *pred* is not None, returns the first item + for which pred(item) is true. + + """ + # first_true([a,b,c], x) --> a or b or c or x + # first_true([a,b], x, f) --> a if f(a) else b if f(b) else x + return next(filter(pred, iterable), default) + def random_product(*args, repeat=1): "Random selection from itertools.product(*args, **kwds)" pools = [tuple(pool) for pool in args] * repeat diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1998,6 +1998,19 @@ ... # unique_justseen('ABBCcAD', str.lower) --> A B C A D ... return map(next, map(itemgetter(1), groupby(iterable, key))) +>>> def first_true(iterable, default=False, pred=None): +... '''Returns the first true value in the iterable. +... +... If no true value is found, returns *default* +... +... If *pred* is not None, returns the first item +... for which pred(item) is true. +... +... ''' +... # first_true([a,b,c], x) --> a or b or c or x +... # first_true([a,b], x, f) --> a if f(a) else b if f(b) else x +... return next(filter(pred, iterable), default) + This is not part of the examples but it tests to make sure the definitions perform as purported. @@ -2075,6 +2088,9 @@ >>> list(unique_justseen('ABBCcAD', str.lower)) ['A', 'B', 'C', 'A', 'D'] +>>> first_true('ABC0DEF1', '9', str.isdigit) +'0' + """ __test__ = {'libreftest' : libreftest} -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 18:05:47 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 18:05:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_make_test_name?= =?utf-8?q?_consistent_with_the_rest_of_the_file?= Message-ID: <3fzXLz1B3Kz7Lm1@mail.python.org> http://hg.python.org/cpython/rev/6185b71bdc1c changeset: 90108:6185b71bdc1c branch: 3.4 parent: 90106:3e2354dde892 user: Benjamin Peterson date: Wed Apr 02 12:05:35 2014 -0400 summary: make test name consistent with the rest of the file files: Lib/test/test_exceptions.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -763,7 +763,7 @@ pass self.assertEqual(e, (None, None, None)) - def testUnicodeChangeAttributes(self): + def test_unicode_change_attributes(self): # See issue 7309. This was a crasher. u = UnicodeEncodeError('baz', 'xxxxx', 1, 5, 'foo') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 18:05:48 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 18:05:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3fzXM02fJ6z7Lsd@mail.python.org> http://hg.python.org/cpython/rev/2c21a901717a changeset: 90109:2c21a901717a parent: 90107:6154c7f2bf75 parent: 90108:6185b71bdc1c user: Benjamin Peterson date: Wed Apr 02 12:05:41 2014 -0400 summary: merge 3.4 files: Lib/test/test_exceptions.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -763,7 +763,7 @@ pass self.assertEqual(e, (None, None, None)) - def testUnicodeChangeAttributes(self): + def test_unicode_change_attributes(self): # See issue 7309. This was a crasher. u = UnicodeEncodeError('baz', 'xxxxx', 1, 5, 'foo') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 18:17:08 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 18:17:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_bail_in_unicod?= =?utf-8?q?e_error=27s_=5F=5Fstr=5F=5F_methods_if_the_objects_are_not_prop?= =?utf-8?q?erly?= Message-ID: <3fzXc43BYdz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/140c5da3dc82 changeset: 90110:140c5da3dc82 branch: 3.4 parent: 90108:6185b71bdc1c user: Benjamin Peterson date: Wed Apr 02 12:15:06 2014 -0400 summary: bail in unicode error's __str__ methods if the objects are not properly initialized (closes #21134) files: Lib/test/test_exceptions.py | 6 ++++++ Misc/NEWS | 3 +++ Objects/exceptions.c | 12 ++++++++++++ 3 files changed, 21 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -800,6 +800,12 @@ u.start = 1000 self.assertEqual(str(u), "can't translate characters in position 1000-4: 965230951443685724997") + def test_unicode_errors_no_object(self): + # See issue #21134. + klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError + for klass in klasses: + self.assertEqual(str(klass.__new__(klass)), "") + @no_tracing def test_badisinstance(self): # Bug #2542: if issubclass(e, MyException) raises an exception, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #21134: Fix segfault when str is called on an uninitialized + UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. + - Issue #19537: Fix PyUnicode_DATA() alignment under m68k. Patch by Andreas Schwab. diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -1837,6 +1837,10 @@ PyObject *reason_str = NULL; PyObject *encoding_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason and encoding as strings, which they might not be if they've been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); @@ -1955,6 +1959,10 @@ PyObject *reason_str = NULL; PyObject *encoding_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason and encoding as strings, which they might not be if they've been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); @@ -2049,6 +2057,10 @@ PyObject *result = NULL; PyObject *reason_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason as a string, which it might not be if it's been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 18:17:09 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 18:17:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_bail_in_unicod?= =?utf-8?q?e_error=27s_=5F=5Fstr=5F=5F_methods_if_the_objects_are_not_prop?= =?utf-8?q?erly?= Message-ID: <3fzXc557MDz7LtP@mail.python.org> http://hg.python.org/cpython/rev/afa7fb2cbe3b changeset: 90111:afa7fb2cbe3b branch: 2.7 parent: 90101:1c112eb4eb56 user: Benjamin Peterson date: Wed Apr 02 12:15:06 2014 -0400 summary: bail in unicode error's __str__ methods if the objects are not properly initialized (closes #21134) files: Lib/test/test_exceptions.py | 6 ++++++ Misc/NEWS | 3 +++ Objects/exceptions.c | 12 ++++++++++++ 3 files changed, 21 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -431,6 +431,12 @@ u.start = 1000 self.assertEqual(str(u), "can't translate characters in position 1000-4: 965230951443685724997") + def test_unicode_errors_no_object(self): + # See issue #21134. + klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError + for klass in klasses: + self.assertEqual(str(klass.__new__(klass)), "") + def test_badisinstance(self): # Bug #2542: if issubclass(e, MyException) raises an exception, # it should be ignored diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -12,6 +12,9 @@ - Issue #20437: Fixed 43 potential bugs when deleting objects references. +- Issue #21134: Fix segfault when str is called on an uninitialized + UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. + - Issue #20494: Ensure that free()d memory arenas are really released on POSIX systems supporting anonymous memory mappings. Patch by Charles-Fran?ois Natali. diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -1648,6 +1648,10 @@ PyObject *reason_str = NULL; PyObject *encoding_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason and encoding as strings, which they might not be if they've been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); @@ -1733,6 +1737,10 @@ PyObject *reason_str = NULL; PyObject *encoding_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason and encoding as strings, which they might not be if they've been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); @@ -1830,6 +1838,10 @@ PyObject *result = NULL; PyObject *reason_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason as a string, which it might not be if it's been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 18:17:10 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 18:17:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjExMzQp?= Message-ID: <3fzXc672RWz7Lt3@mail.python.org> http://hg.python.org/cpython/rev/0aeaea247d7d changeset: 90112:0aeaea247d7d parent: 90109:2c21a901717a parent: 90110:140c5da3dc82 user: Benjamin Peterson date: Wed Apr 02 12:16:55 2014 -0400 summary: merge 3.4 (#21134) files: Lib/test/test_exceptions.py | 6 ++++++ Misc/NEWS | 3 +++ Objects/exceptions.c | 12 ++++++++++++ 3 files changed, 21 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -800,6 +800,12 @@ u.start = 1000 self.assertEqual(str(u), "can't translate characters in position 1000-4: 965230951443685724997") + def test_unicode_errors_no_object(self): + # See issue #21134. + klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError + for klass in klasses: + self.assertEqual(str(klass.__new__(klass)), "") + @no_tracing def test_badisinstance(self): # Bug #2542: if issubclass(e, MyException) raises an exception, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #21134: Fix segfault when str is called on an uninitialized + UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. + - Issue #19537: Fix PyUnicode_DATA() alignment under m68k. Patch by Andreas Schwab. diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -1837,6 +1837,10 @@ PyObject *reason_str = NULL; PyObject *encoding_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason and encoding as strings, which they might not be if they've been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); @@ -1955,6 +1959,10 @@ PyObject *reason_str = NULL; PyObject *encoding_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason and encoding as strings, which they might not be if they've been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); @@ -2049,6 +2057,10 @@ PyObject *result = NULL; PyObject *reason_str = NULL; + if (!uself->object) + /* Not properly initialized. */ + return PyUnicode_FromString(""); + /* Get reason as a string, which it might not be if it's been modified after we were contructed. */ reason_str = PyObject_Str(uself->reason); -- Repository URL: http://hg.python.org/cpython From tjreedy at udel.edu Wed Apr 2 20:49:53 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Apr 2014 14:49:53 -0400 Subject: [Python-checkins] cpython (3.4): bail in unicode error's __str__ methods if the objects are not properly In-Reply-To: <3fzXc43BYdz7LjZ@mail.python.org> References: <3fzXc43BYdz7LjZ@mail.python.org> Message-ID: <533C5BD1.6050405@udel.edu> On 4/2/2014 12:17 PM, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/140c5da3dc82 > changeset: 90110:140c5da3dc82 > branch: 3.4 > parent: 90108:6185b71bdc1c > user: Benjamin Peterson > date: Wed Apr 02 12:15:06 2014 -0400 > summary: > bail in unicode error's __str__ methods if the objects are not properly initialized (closes #21134) > + klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError I suspect the first should the first be Unicode*En*codeError. --------------- > > +- Issue #21134: Fix segfault when str is called on an uninitialized > + UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. From python-checkins at python.org Wed Apr 2 21:52:10 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 21:52:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_make_sure_to_t?= =?utf-8?q?est_UnicodeEncodeError=2C_too?= Message-ID: <3fzdNB59wjz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/095fbf03cad7 changeset: 90113:095fbf03cad7 branch: 3.4 parent: 90110:140c5da3dc82 user: Benjamin Peterson date: Wed Apr 02 15:51:38 2014 -0400 summary: make sure to test UnicodeEncodeError, too files: Lib/test/test_exceptions.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -802,7 +802,7 @@ def test_unicode_errors_no_object(self): # See issue #21134. - klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError + klasses = UnicodeEncodeError, UnicodeDecodeError, UnicodeTranslateError for klass in klasses: self.assertEqual(str(klass.__new__(klass)), "") -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 21:52:12 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 21:52:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_make_sure_to_t?= =?utf-8?q?est_UnicodeEncodeError=2C_too?= Message-ID: <3fzdND0Mtbz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/c03e8b257cf7 changeset: 90114:c03e8b257cf7 branch: 2.7 parent: 90111:afa7fb2cbe3b user: Benjamin Peterson date: Wed Apr 02 15:51:38 2014 -0400 summary: make sure to test UnicodeEncodeError, too files: Lib/test/test_exceptions.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -433,7 +433,7 @@ def test_unicode_errors_no_object(self): # See issue #21134. - klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError + klasses = UnicodeEncodeError, UnicodeDecodeError, UnicodeTranslateError for klass in klasses: self.assertEqual(str(klass.__new__(klass)), "") -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 2 21:52:13 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 2 Apr 2014 21:52:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3fzdNF2Rwsz7Ltx@mail.python.org> http://hg.python.org/cpython/rev/c9cb931b20f4 changeset: 90115:c9cb931b20f4 parent: 90112:0aeaea247d7d parent: 90113:095fbf03cad7 user: Benjamin Peterson date: Wed Apr 02 15:52:04 2014 -0400 summary: merge 3.4 files: Lib/test/test_exceptions.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -802,7 +802,7 @@ def test_unicode_errors_no_object(self): # See issue #21134. - klasses = UnicodeDecodeError, UnicodeDecodeError, UnicodeTranslateError + klasses = UnicodeEncodeError, UnicodeDecodeError, UnicodeTranslateError for klass in klasses: self.assertEqual(str(klass.__new__(klass)), "") -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Thu Apr 3 09:47:11 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 03 Apr 2014 09:47:11 +0200 Subject: [Python-checkins] Daily reference leaks (c9cb931b20f4): sum=8 Message-ID: results for c9cb931b20f4 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 4] memory blocks, sum=4 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogm9gwE_', '-x'] From python-checkins at python.org Thu Apr 3 14:33:47 2014 From: python-checkins at python.org (nick.coghlan) Date: Thu, 3 Apr 2014 14:33:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_467=3A_descope_dramatical?= =?utf-8?q?ly_based_on_Guido=27s_feedback?= Message-ID: <3g03bv41vjz7LjS@mail.python.org> http://hg.python.org/peps/rev/435fa0278b73 changeset: 5452:435fa0278b73 user: Nick Coghlan date: Thu Apr 03 22:33:36 2014 +1000 summary: PEP 467: descope dramatically based on Guido's feedback files: pep-0467.txt | 303 ++++++++++++-------------------------- 1 files changed, 95 insertions(+), 208 deletions(-) diff --git a/pep-0467.txt b/pep-0467.txt --- a/pep-0467.txt +++ b/pep-0467.txt @@ -22,28 +22,35 @@ This PEP proposes a number of small adjustments to the APIs of the ``bytes`` and ``bytearray`` types to make their behaviour more internally consistent -and to make it easier to operate entirely in the binary domain. +and to make it easier to operate entirely in the binary domain, as well as +changes to their documentation to make it easier to grasp their dual roles +as containers of "arbitrary binary data" and "binary data with ASCII +compatible segments". Background ========== -Over the course of Python 3's evolution, a number of adjustments have been -made to the core ``bytes`` and ``bytearray`` types as additional practical -experience was gained with using them in code beyond the Python 3 standard -library and test suite. However, to date, these changes have been made -on a relatively ad hoc tactical basis as specific issues were identified, -rather than as part of a systematic review of the APIs of these types. This -approach has allowed inconsistencies to creep into the API design as to which -input types are accepted by different methods. Additional inconsistencies -linger from an earlier pre-release design where there was *no* separate +To simplify the task of writing the Python 3 documentation, the ``bytes`` +and ``bytearray`` types were documented primarily in terms of the way they +differed from the Unicode based Python 3 ``str`` type. Even when I +`heavily revised the sequence documentation +`__ in 2012, I retained that +simplifying shortcut. + +However, it turns out that this approach to the documentation of these types +has a problem: it doesn't adequately introduce users to their hybrid nature, +where they can be manipulated *either* as a "sequence of integers" type, +*or* as ``str``-like types that assume ASCII compatible data. + +In addition to the documentation issues, there are some lingering design +quirks from an earlier pre-release design where there was *no* separate ``bytearray`` type, and instead the core ``bytes`` type was mutable (with -no immutable counterpart), as well as from the origins of these types in -the text-like behaviour of the Python 2 ``str`` type. +no immutable counterpart). -This PEP aims to provide the missing systematic review, with the goal of -ensuring that wherever feasible (given backwards compatibility constraints) -these current inconsistencies are addressed for the Python 3.5 release. +Finally, additional experience with using the existing Python 3 binary +sequence types in real world applications has suggested it would be +beneficial to make it easier to convert integers to length 1 bytes objects. Proposals @@ -55,10 +62,13 @@ factors: * removing remnants of the original design of ``bytes`` as a mutable type -* more consistently accepting length 1 ``bytes`` objects as input where an - integer between ``0`` and ``255`` inclusive is expected, and vice-versa -* allowing users to easily convert integer output to a length 1 ``bytes`` +* allowing users to easily convert integer values to a length 1 ``bytes`` object +* consistently applying the following analogies to the type API designs + and documentation: + + * ``bytes``: tuple of integers, with additional str-like methods + * ``bytearray``: list of integers, with additional str-like methods Alternate Constructors @@ -83,95 +93,69 @@ b'\x00\x00\x00' This PEP proposes that the current handling of integers in the bytes and -bytearray constructors by deprecated in Python 3.5 and removed in Python -3.6, being replaced by two more type appropriate alternate constructors -provided as class methods. The initial python-ideas thread [ideas-thread1]_ -that spawned this PEP was specifically aimed at deprecating this constructor -behaviour. +bytearray constructors by deprecated in Python 3.5 and targeted for +removal in Python 3.7, being replaced by two more explicit alternate +constructors provided as class methods. The initial python-ideas thread +[ideas-thread1]_ that spawned this PEP was specifically aimed at deprecating +this constructor behaviour. -For ``bytes``, a ``byte`` constructor is proposed that converts integers -(as indicated by ``operator.index``) in the appropriate range to a ``bytes`` -object, converts objects that support the buffer API to bytes, and also -passes through length 1 byte strings unchanged:: +Firstly, a ``byte`` constructor is proposed that converts integers +in the range 0 to 255 (inclusive) to a ``bytes`` object:: >>> bytes.byte(3) b'\x03' - >>> bytes.byte(bytearray(bytes([3]))) - b'\x03' - >>> bytes.byte(memoryview(bytes([3]))) - b'\x03' - >>> bytes.byte(bytes([3])) - b'\x03' + >>> bytearray.byte(3) + bytearray(b'\x03') >>> bytes.byte(512) Traceback (most recent call last): File "", line 1, in ValueError: bytes must be in range(0, 256) - >>> bytes.byte(b"ab") - Traceback (most recent call last): - File "", line 1, in - TypeError: bytes.byte() expected a byte, but buffer of length 2 found One specific use case for this alternate constructor is to easily convert the result of indexing operations on ``bytes`` and other binary sequences from an integer to a ``bytes`` object. The documentation for this API should note that its counterpart for the reverse conversion is ``ord()``. +The ``ord()`` documentation will also be updated to note that while +``chr()`` is the counterpart for ``str`` input, ``bytes.byte`` and +``bytearray.byte`` are the counterparts for binary input. -For ``bytearray``, a ``from_len`` constructor is proposed that preallocates -the buffer filled with a particular value (default to ``0``) as a direct +Secondly, a ``zeros`` constructor is proposed that serves as a direct replacement for the current constructor behaviour, rather than having to use sequence repetition to achieve the same effect in a less intuitive way:: - >>> bytearray.from_len(3) + >>> bytes.zeros(3) + b'\x00\x00\x00' + >>> bytearray.zeros(3) bytearray(b'\x00\x00\x00') - >>> bytearray.from_len(3, 6) - bytearray(b'\x06\x06\x06') -This part of the proposal was covered by an existing issue -[empty-buffer-issue]_ and a variety of names have been proposed -(``empty_buffer``, ``zeros``, ``zeroes``, ``allnull``, ``fill``). The -specific name currently proposed was chosen by analogy with -``dict.fromkeys()`` and ``itertools.chain.from_iter()`` to be completely -explicit that it is an alternate constructor rather than an in-place -mutation, as well as how it differs from the standard constructor. +The chosen name here is taken from the corresponding initialisation function +in NumPy (although, as these are sequence types rather than N-dimensional +matrices, the constructors take a length as input rather than a shape tuple) - -Open questions -^^^^^^^^^^^^^^ - -* Should ``bytearray.byte()`` also be added? Or is - ``bytearray(bytes.byte(x))`` sufficient for that case? -* Should ``bytes.from_len()`` also be added? Or is sequence repetition - sufficient for that case? -* Should ``bytearray.from_len()`` use a different name? -* Should ``bytes.byte()`` raise ``TypeError`` or ``ValueError`` for binary - sequences with more than one element? The ``TypeError`` currently proposed - is copied (with slightly improved wording) from the behaviour of ``ord()`` - with sequences containing more than one code point, while ``ValueError`` - would be more consistent with the existing handling of out-of-range - integer values. -* ``bytes.byte()`` is defined above as accepting length 1 binary sequences - as individual bytes, but this is currently inconsistent with the main - ``bytes`` constructor:: - - >>> bytes([b"a", b"b", b"c"]) - Traceback (most recent call last): - File "", line 1, in - TypeError: 'bytes' object cannot be interpreted as an integer - - Should the ``bytes`` constructor be changed to accept iterables of length 1 - bytes objects in addition to iterables of integers? If so, should it - allow a mixture of the two in a single iterable? +While ``bytes.byte`` and ``bytearray.zeros`` are expected to be the more +useful duo amongst the new constructors, ``bytes.zeros`` and +`bytearray.byte`` are provided in order to maintain API consistency between +the two types. Iteration --------- -Iteration over ``bytes`` objects and other binary sequences produces -integers. Rather than proposing a new method that would need to be added -not only to ``bytes``, ``bytearray`` and ``memoryview``, but potentially -to third party types as well, this PEP proposes that iteration to produce -length 1 ``bytes`` objects instead be handled by combining ``map`` with -the new ``bytes.byte()`` alternate constructor proposed above:: +While iteration over ``bytes`` objects and other binary sequences produces +integers, it is sometimes desirable to iterate over length 1 bytes objects +instead. + +To handle this situation more obviously (and more efficiently) than would be +the case with the ``map(bytes.byte, data)`` construct enabled by the above +constructor changes, this PEP proposes the addition of a new ``iterbytes`` +method to ``bytes``, ``bytearray`` and ``memoryview``:: + + for x in data.iterbytes(): + # x is a length 1 ``bytes`` object, rather than an integer + +Third party types and arbitrary containers of integers that lack the new +method can still be handled by combining ``map`` with the new +``bytes.byte()`` alternate constructor proposed above:: for x in map(bytes.byte, data): # x is a length 1 ``bytes`` object, rather than an integer @@ -179,139 +163,42 @@ # 0 to 255 inclusive -Consistent support for different input types --------------------------------------------- +Open questions +^^^^^^^^^^^^^^ -The ``bytes`` and ``bytearray`` methods inspired by the Python 2 ``str`` -type generally expect to operate on binary subsequences: other objects -implementing the buffer API. By contrast, the mutating APIs added to -the ``bytearray`` interface expect to operate on individual elements: -integer in the range 0 to 255 (inclusive). +* The fallback case above suggests that this could perhaps be better handled + as an ``iterbytes(data)`` *builtin*, that used ``data.__iterbytes__()`` + if defined, but otherwise fell back to ``map(bytes.byte, data)``:: -In Python 3.3, the binary search operations (``in``, ``count()``, -``find()``, ``index()``, ``rfind()`` and ``rindex()``) were updated to -accept integers in the range 0 to 255 (inclusive) as their first argument, -in addition to the existing support for binary subsequences. + for x in iterbytes(data): + # x is a length 1 ``bytes`` object, rather than an integer + # This works with *any* container of integers in the range + # 0 to 255 inclusive -This results in behaviour like the following in Python 3.3+:: - >>> data = bytes([1, 2, 3, 4]) - >>> 3 in data - True - >>> b"\x03" in data - True - >>> data.count(3) - 1 - >>> data.count(b"\x03") - 1 +Documentation clarifications +---------------------------- - >>> data.replace(3, 4) - Traceback (most recent call last): - File "", line 1, in - TypeError: expected bytes, bytearray or buffer compatible object - >>> data.replace(b"\x03", b"\x04") - b'\x01\x02\x04\x04' +In an attempt to clarify the `documentation +`__ +of the ``bytes`` and ``bytearray`` types, the following changes are +proposed: - >>> mutable = bytearray(data) - >>> mutable - bytearray(b'\x01\x02\x03\x04') - >>> mutable.append(b"\x05") - Traceback (most recent call last): - File "", line 1, in - TypeError: an integer is required - >>> mutable.append(5) - >>> mutable - bytearray(b'\x01\x02\x03\x04\x05') +* the documentation of the *sequence* behaviour of each type is moved to + section for that individual type. These sections will be updated to + explicitly make the ``tuple of integers`` and ``list of integers`` + analogies, as well as to make it clear that these parts of the API work + with arbitrary binary data +* the current "Bytes and bytearray operations" section will be updated to + "Handling binary data with ASCII compatible segments", and will explicitly + list *all* of the methods that are included. +* clarify that due to their origins in the API of the immutable ``str`` + type, even the ``bytearray`` versions of these methods do *not* operate + in place, but instead create a new object. - -This PEP proposes extending the behaviour of accepting integers as being -equivalent to the corresponding length 1 binary sequence to several other -``bytes`` and ``bytearray`` methods that currently expect a ``bytes`` -object for certain parameters. In essence, if a value is an acceptable -input to the new ``bytes.byte`` constructor defined above, then it would -be acceptable in the roles defined here (in addition to any other already -supported inputs): - -* ``startswith()`` prefix(es) -* ``endswith()`` suffix(es) - -* ``center()`` fill character -* ``ljust()`` fill character -* ``rjust()`` fill character - -* ``strip()`` character to strip -* ``lstrip()`` character to strip -* ``rstrip()`` character to strip - -* ``partition()`` separator argument -* ``rpartition()`` separator argument - -* ``split()`` separator argument -* ``rsplit()`` separator argument - -* ``replace()`` old value and new value - -In addition to the consistency motive, this approach also makes it easier -to work with the indexing behaviour , as the result of an indexing operation -can more easily be fed back in to other methods. - -For ``bytearray``, some additional changes are proposed to the current -integer based operations to ensure they remain consistent with the proposed -constructor changes:: - -* ``append()``: updated to be consistent with ``bytes.byte()`` -* ``remove()``: updated to be consistent with ``bytes.byte()`` -* ``+=``: updated to be consistent with ``bytes()`` changes (if any) -* ``extend()``: updated to be consistent with ``bytes()`` changes (if any) - -The general principle behind these changes is to restore the flexible -"element-or-subsequence" behaviour seen in the ``str`` API, even though -Python 3 actually represents subsequences and individual elements as -distinct types in the binary domain. - - -Acknowledgement of surprising behaviour of some ``bytearray`` methods ---------------------------------------------------------------------- - -Several of the ``bytes`` and ``bytearray`` methods have their origins in the -Python 2 ``str`` API. As ``str`` is an immutable type, all of these -operations are defined as returning a *new* instance, rather than operating -in place. This contrasts with methods on other mutable types like ``list``, -where ``list.sort()`` and ``list.reverse()`` operate in-place and return -``None``, rather than creating a new object. - -Backwards compatibility constraints make it impractical to change this -behaviour at this point, but it may be appropriate to explicitly call out -this quirk in the documentation for the ``bytearray`` type. It affects the -following methods that could reasonably be expected to operate in-place on -a mutable type: - -* ``center()`` -* ``ljust()`` -* ``rjust()`` -* ``strip()`` -* ``lstrip()`` -* ``rstrip()`` -* ``replace()`` -* ``lower()`` -* ``upper()`` -* ``swapcase()`` -* ``title()`` -* ``capitalize()`` -* ``translate()`` -* ``expandtabs()`` -* ``zfill()`` - -Note that the following ``bytearray`` operations *do* operate in place, as -they're part of the mutable sequence API in ``bytearray``, rather than being -inspired by the immutable Python 2 ``str`` API: - -* ``+=`` -* ``append()`` -* ``extend()`` -* ``reverse()`` -* ``remove()`` -* ``pop()`` +A patch for at least this part of the proposal will be prepared before +submitting the PEP for approval, as writing out these docs completely may +suggest additional opportunities for API consistency improvements. References -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Thu Apr 3 15:15:29 2014 From: python-checkins at python.org (eli.bendersky) Date: Thu, 3 Apr 2014 15:15:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwMzc1?= =?utf-8?q?=3A_Clarify_ET=27s_parsing_of_comments_and_processing_instructi?= =?utf-8?q?ons=2E?= Message-ID: <3g04X13N8tz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/871278b87c62 changeset: 90116:871278b87c62 branch: 3.4 parent: 90113:095fbf03cad7 user: Eli Bendersky date: Thu Apr 03 06:14:38 2014 -0700 summary: Issue #20375: Clarify ET's parsing of comments and processing instructions. Based on patch by Nikolaus Rath. files: Doc/library/xml.etree.elementtree.rst | 22 +++++++++++++++ 1 files changed, 22 insertions(+), 0 deletions(-) diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -106,6 +106,19 @@ '2008' +.. note:: + + Not all elements of the XML input will end up as elements of the + parsed tree. Currently, this module skips over any XML comments, + processing instructions, and document type declarations in the + input. Nevertheless, trees built using this module's API rather + than parsing from XML text can have comments and processing + instructions in them; they will be included when generating XML + output. A document type declaration may be accessed by passing a + custom :class:`TreeBuilder` instance to the :class:`XMLParser` + constructor. + + .. _elementtree-pull-parsing: Pull API for non-blocking parsing @@ -381,6 +394,10 @@ string containing the comment string. Returns an element instance representing a comment. + Note that :class:`XMLParser` skips over comments in the input + instead of creating comment objects for them. An :class:`ElementTree` will + only contain comment nodes if they have been inserted into to + the tree using one of the :class:`Element` methods. .. function:: dump(elem) @@ -461,6 +478,11 @@ containing the PI target. *text* is a string containing the PI contents, if given. Returns an element instance, representing a processing instruction. + Note that :class:`XMLParser` skips over processing instructions + in the input instead of creating comment objects for them. An + :class:`ElementTree` will only contain processing instruction nodes if + they have been inserted into to the tree using one of the + :class:`Element` methods. .. function:: register_namespace(prefix, uri) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 3 15:15:30 2014 From: python-checkins at python.org (eli.bendersky) Date: Thu, 3 Apr 2014 15:15:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320375=3A_Clarify_ET=27s_parsing_of_comments_and?= =?utf-8?q?_processing_instructions=2E?= Message-ID: <3g04X257LRz7LjW@mail.python.org> http://hg.python.org/cpython/rev/5c3166ec80e1 changeset: 90117:5c3166ec80e1 parent: 90115:c9cb931b20f4 parent: 90116:871278b87c62 user: Eli Bendersky date: Thu Apr 03 06:15:28 2014 -0700 summary: Issue #20375: Clarify ET's parsing of comments and processing instructions. Based on patch by Nikolaus Rath. files: Doc/library/xml.etree.elementtree.rst | 22 +++++++++++++++ 1 files changed, 22 insertions(+), 0 deletions(-) diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -106,6 +106,19 @@ '2008' +.. note:: + + Not all elements of the XML input will end up as elements of the + parsed tree. Currently, this module skips over any XML comments, + processing instructions, and document type declarations in the + input. Nevertheless, trees built using this module's API rather + than parsing from XML text can have comments and processing + instructions in them; they will be included when generating XML + output. A document type declaration may be accessed by passing a + custom :class:`TreeBuilder` instance to the :class:`XMLParser` + constructor. + + .. _elementtree-pull-parsing: Pull API for non-blocking parsing @@ -381,6 +394,10 @@ string containing the comment string. Returns an element instance representing a comment. + Note that :class:`XMLParser` skips over comments in the input + instead of creating comment objects for them. An :class:`ElementTree` will + only contain comment nodes if they have been inserted into to + the tree using one of the :class:`Element` methods. .. function:: dump(elem) @@ -461,6 +478,11 @@ containing the PI target. *text* is a string containing the PI contents, if given. Returns an element instance, representing a processing instruction. + Note that :class:`XMLParser` skips over processing instructions + in the input instead of creating comment objects for them. An + :class:`ElementTree` will only contain processing instruction nodes if + they have been inserted into to the tree using one of the + :class:`Element` methods. .. function:: register_namespace(prefix, uri) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 3 16:22:30 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 3 Apr 2014 16:22:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_remove_unused_?= =?utf-8?q?argument_=28closes_=2321135=29?= Message-ID: <3g061L2MhXz7LmV@mail.python.org> http://hg.python.org/cpython/rev/54bd06097619 changeset: 90118:54bd06097619 branch: 3.4 parent: 90116:871278b87c62 user: Benjamin Peterson date: Thu Apr 03 10:22:10 2014 -0400 summary: remove unused argument (closes #21135) files: Lib/cgi.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/cgi.py b/Lib/cgi.py --- a/Lib/cgi.py +++ b/Lib/cgi.py @@ -1045,7 +1045,7 @@ return s -def valid_boundary(s, _vb_pattern=None): +def valid_boundary(s): import re if isinstance(s, bytes): _vb_pattern = b"^[ -~]{0,200}[!-~]$" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 3 16:22:31 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 3 Apr 2014 16:22:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjExMzUp?= Message-ID: <3g061M41n6z7LqT@mail.python.org> http://hg.python.org/cpython/rev/2299cb5e8592 changeset: 90119:2299cb5e8592 parent: 90117:5c3166ec80e1 parent: 90118:54bd06097619 user: Benjamin Peterson date: Thu Apr 03 10:22:22 2014 -0400 summary: merge 3.4 (#21135) files: Lib/cgi.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/cgi.py b/Lib/cgi.py --- a/Lib/cgi.py +++ b/Lib/cgi.py @@ -1045,7 +1045,7 @@ return s -def valid_boundary(s, _vb_pattern=None): +def valid_boundary(s): import re if isinstance(s, bytes): _vb_pattern = b"^[ -~]{0,200}[!-~]$" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 3 16:31:54 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 3 Apr 2014 16:31:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_properly_close?= =?utf-8?q?_files_in_test=5Fzipfile_=28=2320887=29?= Message-ID: <3g06DB48LCz7LlJ@mail.python.org> http://hg.python.org/cpython/rev/03df2c1c6892 changeset: 90120:03df2c1c6892 branch: 2.7 parent: 90114:c03e8b257cf7 user: Benjamin Peterson date: Thu Apr 03 10:31:25 2014 -0400 summary: properly close files in test_zipfile (#20887) Patch by Matti Picus. files: Lib/test/test_zipfile.py | 74 +++++++++++++++++---------- 1 files changed, 46 insertions(+), 28 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -8,7 +8,6 @@ import io import sys import time -import shutil import struct import zipfile import unittest @@ -19,7 +18,7 @@ from unittest import skipUnless from test.test_support import TESTFN, TESTFN_UNICODE, TESTFN_ENCODING, \ - run_unittest, findfile, unlink, check_warnings + run_unittest, findfile, unlink, rmtree, check_warnings try: TESTFN_UNICODE.encode(TESTFN_ENCODING) except (UnicodeError, TypeError): @@ -367,7 +366,8 @@ produces the expected result.""" with zipfile.ZipFile(TESTFN2, "w") as zipfp: zipfp.write(TESTFN) - self.assertEqual(zipfp.read(TESTFN), open(TESTFN).read()) + with open(TESTFN,'r') as fid: + self.assertEqual(zipfp.read(TESTFN), fid.read()) @skipUnless(zlib, "requires zlib") def test_per_file_compression(self): @@ -406,11 +406,12 @@ self.assertEqual(writtenfile, correctfile) # make sure correct data is in correct file - self.assertEqual(fdata, open(writtenfile, "rb").read()) + with open(writtenfile, "rb") as fid: + self.assertEqual(fdata, fid.read()) os.remove(writtenfile) # remove the test file subdirectories - shutil.rmtree(os.path.join(os.getcwd(), 'ziptest2dir')) + rmtree(os.path.join(os.getcwd(), 'ziptest2dir')) def test_extract_all(self): with zipfile.ZipFile(TESTFN2, "w", zipfile.ZIP_STORED) as zipfp: @@ -422,11 +423,12 @@ for fpath, fdata in SMALL_TEST_DATA: outfile = os.path.join(os.getcwd(), fpath) - self.assertEqual(fdata, open(outfile, "rb").read()) + with open(outfile, "rb") as fid: + self.assertEqual(fdata, fid.read()) os.remove(outfile) # remove the test file subdirectories - shutil.rmtree(os.path.join(os.getcwd(), 'ziptest2dir')) + rmtree(os.path.join(os.getcwd(), 'ziptest2dir')) def check_file(self, filename, content): self.assertTrue(os.path.isfile(filename)) @@ -511,12 +513,12 @@ self.assertEqual(writtenfile, correctfile, msg="extract %r" % arcname) self.check_file(correctfile, content) - shutil.rmtree('target') + rmtree('target') with zipfile.ZipFile(TESTFN2, 'r') as zipfp: zipfp.extractall(targetpath) self.check_file(correctfile, content) - shutil.rmtree('target') + rmtree('target') correctfile = os.path.join(os.getcwd(), *fixedname.split('/')) @@ -525,12 +527,12 @@ self.assertEqual(writtenfile, correctfile, msg="extract %r" % arcname) self.check_file(correctfile, content) - shutil.rmtree(fixedname.split('/')[0]) + rmtree(fixedname.split('/')[0]) with zipfile.ZipFile(TESTFN2, 'r') as zipfp: zipfp.extractall() self.check_file(correctfile, content) - shutil.rmtree(fixedname.split('/')[0]) + rmtree(fixedname.split('/')[0]) os.remove(TESTFN2) @@ -775,11 +777,12 @@ self.assertNotIn('mod2.txt', names) finally: - shutil.rmtree(TESTFN2) + rmtree(TESTFN2) def test_write_non_pyfile(self): with zipfile.PyZipFile(TemporaryFile(), "w") as zipfp: - open(TESTFN, 'w').write('most definitely not a python file') + with open(TESTFN, 'w') as fid: + fid.write('most definitely not a python file') self.assertRaises(RuntimeError, zipfp.writepy, TESTFN) os.remove(TESTFN) @@ -942,8 +945,9 @@ self.assertRaises(RuntimeError, zipf.open, "foo.txt") self.assertRaises(RuntimeError, zipf.testzip) self.assertRaises(RuntimeError, zipf.writestr, "bogus.txt", "bogus") - open(TESTFN, 'w').write('zipfile test data') - self.assertRaises(RuntimeError, zipf.write, TESTFN) + with open(TESTFN, 'w') as fid: + fid.write('zipfile test data') + self.assertRaises(RuntimeError, zipf.write, TESTFN) def test_bad_constructor_mode(self): """Check that bad modes passed to ZipFile constructor are caught.""" @@ -1129,6 +1133,7 @@ pass try: zipf = zipfile.ZipFile(TESTFN, mode="r") + zipf.close() except zipfile.BadZipfile: self.fail("Unable to create empty ZIP file in 'w' mode") @@ -1136,6 +1141,7 @@ pass try: zipf = zipfile.ZipFile(TESTFN, mode="r") + zipf.close() except: self.fail("Unable to create empty ZIP file in 'a' mode") @@ -1332,12 +1338,11 @@ # Verify that (when the ZipFile is in control of creating file objects) # multiple open() calls can be made without interfering with each other. with zipfile.ZipFile(TESTFN2, mode="r") as zipf: - zopen1 = zipf.open('ones') - zopen2 = zipf.open('ones') - data1 = zopen1.read(500) - data2 = zopen2.read(500) - data1 += zopen1.read(500) - data2 += zopen2.read(500) + with zipf.open('ones') as zopen1, zipf.open('ones') as zopen2: + data1 = zopen1.read(500) + data2 = zopen2.read(500) + data1 += zopen1.read(500) + data2 += zopen2.read(500) self.assertEqual(data1, data2) def test_different_file(self): @@ -1364,6 +1369,17 @@ self.assertEqual(data1, '1'*FIXEDTEST_SIZE) self.assertEqual(data2, '2'*FIXEDTEST_SIZE) + def test_many_opens(self): + # Verify that read() and open() promptly close the file descriptor, + # and don't rely on the garbage collector to free resources. + with zipfile.ZipFile(TESTFN2, mode="r") as zipf: + for x in range(100): + zipf.read('ones') + with zipf.open('ones') as zopen1: + pass + with open(os.devnull) as f: + self.assertLess(f.fileno(), 100) + def tearDown(self): unlink(TESTFN2) @@ -1386,12 +1402,12 @@ def test_store_dir(self): os.mkdir(os.path.join(TESTFN2, "x")) - zipf = zipfile.ZipFile(TESTFN, "w") - zipf.write(os.path.join(TESTFN2, "x"), "x") - self.assertTrue(zipf.filelist[0].filename.endswith("x/")) + with zipfile.ZipFile(TESTFN, "w") as zipf: + zipf.write(os.path.join(TESTFN2, "x"), "x") + self.assertTrue(zipf.filelist[0].filename.endswith("x/")) def tearDown(self): - shutil.rmtree(TESTFN2) + rmtree(TESTFN2) if os.path.exists(TESTFN): unlink(TESTFN) @@ -1405,7 +1421,8 @@ for n, s in enumerate(self.seps): self.arcdata[s] = s.join(self.line_gen) + s self.arcfiles[s] = '%s-%d' % (TESTFN, n) - open(self.arcfiles[s], "wb").write(self.arcdata[s]) + with open(self.arcfiles[s], "wb") as fid: + fid.write(self.arcdata[s]) def make_test_archive(self, f, compression): # Create the ZIP archive @@ -1474,8 +1491,9 @@ # Read the ZIP archive with zipfile.ZipFile(f, "r") as zipfp: for sep, fn in self.arcfiles.items(): - for line, zipline in zip(self.line_gen, zipfp.open(fn, "rU")): - self.assertEqual(zipline, line + '\n') + with zipfp.open(fn, "rU") as fid: + for line, zipline in zip(self.line_gen, fid): + self.assertEqual(zipline, line + '\n') def test_read_stored(self): for f in (TESTFN2, TemporaryFile(), StringIO()): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 3 17:01:53 2014 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 3 Apr 2014 17:01:53 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjExNDM6?= =?utf-8?q?_Fix_typo_in_docs_for_max=28=29=2E?= Message-ID: <3g06tn1V1Bz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/8fe2bb0c5851 changeset: 90121:8fe2bb0c5851 branch: 3.4 parent: 90118:54bd06097619 user: Raymond Hettinger date: Thu Apr 03 08:01:22 2014 -0700 summary: Issue 21143: Fix typo in docs for max(). files: Doc/library/functions.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -781,7 +781,7 @@ If one positional argument is provided, it should be an :term:`iterable`. The largest item in the iterable is returned. If two or more positional - arguments are provided, the smallest of the positional arguments is + arguments are provided, the largest of the positional arguments is returned. There are two optional keyword-only arguments. The *key* argument specifies -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 3 17:01:54 2014 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 3 Apr 2014 17:01:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <3g06tp3BHgz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/d9337fe963cc changeset: 90122:d9337fe963cc parent: 90119:2299cb5e8592 parent: 90121:8fe2bb0c5851 user: Raymond Hettinger date: Thu Apr 03 08:01:44 2014 -0700 summary: merge files: Doc/library/functions.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -781,7 +781,7 @@ If one positional argument is provided, it should be an :term:`iterable`. The largest item in the iterable is returned. If two or more positional - arguments are provided, the smallest of the positional arguments is + arguments are provided, the largest of the positional arguments is returned. There are two optional keyword-only arguments. The *key* argument specifies -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Fri Apr 4 09:55:16 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 04 Apr 2014 09:55:16 +0200 Subject: [Python-checkins] Daily reference leaks (d9337fe963cc): sum=0 Message-ID: results for d9337fe963cc on branch "default" -------------------------------------------- test_site leaked [0, 2, -2] references, sum=0 test_site leaked [0, 2, -2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog4wL3wK', '-x'] From python-checkins at python.org Fri Apr 4 11:57:45 2014 From: python-checkins at python.org (vinay.sajip) Date: Fri, 4 Apr 2014 11:57:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTQ5?= =?utf-8?q?=3A_Improved_thread-safety_in_logging_cleanup_during_interprete?= =?utf-8?q?r?= Message-ID: <3g0c5P2mQbz7Lk9@mail.python.org> http://hg.python.org/cpython/rev/b6deab7204e6 changeset: 90123:b6deab7204e6 branch: 2.7 parent: 90120:03df2c1c6892 user: Vinay Sajip date: Fri Apr 04 10:47:53 2014 +0100 summary: Issue #21149: Improved thread-safety in logging cleanup during interpreter shutdown. files: Lib/logging/__init__.py | 22 ++++++++++++---------- Misc/NEWS | 3 +++ 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -1,4 +1,4 @@ -# Copyright 2001-2012 by Vinay Sajip. All Rights Reserved. +# Copyright 2001-2014 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, @@ -18,7 +18,7 @@ Logging package for Python. Based on PEP 282 and comments thereto in comp.lang.python. -Copyright (C) 2001-2012 Vinay Sajip. All Rights Reserved. +Copyright (C) 2001-2014 Vinay Sajip. All Rights Reserved. To use, simply 'import logging' and log away! """ @@ -46,6 +46,7 @@ __author__ = "Vinay Sajip " __status__ = "production" +# Note: the attributes below are no longer maintained. __version__ = "0.5.1.2" __date__ = "07 February 2010" @@ -622,16 +623,17 @@ Remove a handler reference from the internal cleanup list. """ # This function can be called during module teardown, when globals are - # set to None. If _acquireLock is None, assume this is the case and do - # nothing. - if (_acquireLock is not None and _handlerList is not None and - _releaseLock is not None): - _acquireLock() + # set to None. It can also be called from another thread. So we need to + # pre-emptively grab the necessary globals and check if they're None, + # to prevent race conditions and failures during interpreter shutdown. + acquire, release, handlers = _acquireLock, _releaseLock, _handlerList + if acquire and release and handlers: + acquire() try: - if wr in _handlerList: - _handlerList.remove(wr) + if wr in handlers: + handlers.remove(wr) finally: - _releaseLock() + release() def _addHandlerRef(handler): """ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,9 @@ Library ------- +- Issue #21149: Improved thread-safety in logging cleanup during interpreter + shutdown. Thanks to Devin Jeanpierre for the patch. + - Fix possible overflow bug in strop.expandtabs. You shouldn't be using this module! -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 11:57:46 2014 From: python-checkins at python.org (vinay.sajip) Date: Fri, 4 Apr 2014 11:57:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTQ5?= =?utf-8?q?=3A_Improved_thread-safety_in_logging_cleanup_during_interprete?= =?utf-8?q?r?= Message-ID: <3g0c5Q4H48z7Lkn@mail.python.org> http://hg.python.org/cpython/rev/b5c91b61991a changeset: 90124:b5c91b61991a branch: 3.4 parent: 90121:8fe2bb0c5851 user: Vinay Sajip date: Fri Apr 04 10:51:49 2014 +0100 summary: Issue #21149: Improved thread-safety in logging cleanup during interpreter shutdown. files: Lib/logging/__init__.py | 17 +++++++++-------- Misc/NEWS | 3 +++ 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -711,16 +711,17 @@ Remove a handler reference from the internal cleanup list. """ # This function can be called during module teardown, when globals are - # set to None. If _acquireLock is None, assume this is the case and do - # nothing. - if (_acquireLock is not None and _handlerList is not None and - _releaseLock is not None): - _acquireLock() + # set to None. It can also be called from another thread. So we need to + # pre-emptively grab the necessary globals and check if they're None, + # to prevent race conditions and failures during interpreter shutdown. + acquire, release, handlers = _acquireLock, _releaseLock, _handlerList + if acquire and release and handlers: + acquire() try: - if wr in _handlerList: - _handlerList.remove(wr) + if wr in handlers: + handlers.remove(wr) finally: - _releaseLock() + release() def _addHandlerRef(handler): """ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ Library ------- +- Issue #21149: Improved thread-safety in logging cleanup during interpreter + shutdown. Thanks to Devin Jeanpierre for the patch. + - Issue #20145: `assertRaisesRegex` and `assertWarnsRegex` now raise a TypeError if the second argument is not a string or compiled regex. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 11:57:47 2014 From: python-checkins at python.org (vinay.sajip) Date: Fri, 4 Apr 2014 11:57:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2321149=3A_Improved_thread-safety_in_logging_cle?= =?utf-8?q?anup_during_interpreter?= Message-ID: <3g0c5R62sdz7LnV@mail.python.org> http://hg.python.org/cpython/rev/76689a706900 changeset: 90125:76689a706900 parent: 90122:d9337fe963cc parent: 90124:b5c91b61991a user: Vinay Sajip date: Fri Apr 04 10:57:25 2014 +0100 summary: Closes #21149: Improved thread-safety in logging cleanup during interpreter shutdown. files: Lib/logging/__init__.py | 17 +++++++++-------- Misc/NEWS | 3 +++ 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -711,16 +711,17 @@ Remove a handler reference from the internal cleanup list. """ # This function can be called during module teardown, when globals are - # set to None. If _acquireLock is None, assume this is the case and do - # nothing. - if (_acquireLock is not None and _handlerList is not None and - _releaseLock is not None): - _acquireLock() + # set to None. It can also be called from another thread. So we need to + # pre-emptively grab the necessary globals and check if they're None, + # to prevent race conditions and failures during interpreter shutdown. + acquire, release, handlers = _acquireLock, _releaseLock, _handlerList + if acquire and release and handlers: + acquire() try: - if wr in _handlerList: - _handlerList.remove(wr) + if wr in handlers: + handlers.remove(wr) finally: - _releaseLock() + release() def _addHandlerRef(handler): """ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #21149: Improved thread-safety in logging cleanup during interpreter + shutdown. Thanks to Devin Jeanpierre for the patch. + - Issue #21058: Fix a leak of file descriptor in :func:`tempfile.NamedTemporaryFile`, close the file descriptor if :func:`io.open` fails -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 14:21:06 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 4 Apr 2014 14:21:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319505=3A_The_item?= =?utf-8?q?s=2C_keys=2C_and_values_views_of_OrderedDict_now_support?= Message-ID: <3g0gGp5dy5z7Lk0@mail.python.org> http://hg.python.org/cpython/rev/cee010fecdf5 changeset: 90126:cee010fecdf5 user: Serhiy Storchaka date: Fri Apr 04 15:19:36 2014 +0300 summary: Issue #19505: The items, keys, and values views of OrderedDict now support reverse iteration using reversed(). files: Doc/library/collections.rst | 3 ++ Lib/collections/__init__.py | 33 +++++++++++++++++++++-- Lib/test/test_collections.py | 5 +++ Misc/NEWS | 3 ++ 4 files changed, 41 insertions(+), 3 deletions(-) diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -978,6 +978,9 @@ keyword arguments, but their order is lost because Python's function call semantics pass-in keyword arguments using a regular unordered dictionary. +.. versionchanged:: 3.5 + The items, keys, and values :term:`views ` of :class:`OrderedDict` now + support reverse iteration using :func:`reversed`. :class:`OrderedDict` Examples and Recipes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -20,6 +20,23 @@ ### OrderedDict ################################################################################ +class _OrderedDictKeysView(KeysView): + + def __reversed__(self): + yield from reversed(self._mapping) + +class _OrderedDictItemsView(ItemsView): + + def __reversed__(self): + for key in reversed(self._mapping): + yield (key, self._mapping[key]) + +class _OrderedDictValuesView(ValuesView): + + def __reversed__(self): + for key in reversed(self._mapping): + yield self._mapping[key] + class _Link(object): __slots__ = 'prev', 'next', 'key', '__weakref__' @@ -162,9 +179,19 @@ return size update = __update = MutableMapping.update - keys = MutableMapping.keys - values = MutableMapping.values - items = MutableMapping.items + + def keys(self): + "D.keys() -> a set-like object providing a view on D's keys" + return _OrderedDictKeysView(self) + + def items(self): + "D.items() -> a set-like object providing a view on D's items" + return _OrderedDictItemsView(self) + + def values(self): + "D.values() -> an object providing a view on D's values" + return _OrderedDictValuesView(self) + __ne__ = MutableMapping.__ne__ __marker = object() diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -1187,6 +1187,11 @@ self.assertEqual(list(od.items()), pairs) self.assertEqual(list(reversed(od)), [t[0] for t in reversed(pairs)]) + self.assertEqual(list(reversed(od.keys())), + [t[0] for t in reversed(pairs)]) + self.assertEqual(list(reversed(od.values())), + [t[1] for t in reversed(pairs)]) + self.assertEqual(list(reversed(od.items())), list(reversed(pairs))) def test_popitem(self): pairs = [('c', 1), ('b', 2), ('a', 3), ('d', 4), ('e', 5), ('f', 6)] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #19505: The items, keys, and values views of OrderedDict now support + reverse iteration using reversed(). + - Issue #21149: Improved thread-safety in logging cleanup during interpreter shutdown. Thanks to Devin Jeanpierre for the patch. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 14:46:32 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 4 Apr 2014 14:46:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320636=3A_Improved?= =?utf-8?q?_the_repr_of_Tkinter_widgets=2E?= Message-ID: <3g0gr844Tzz7LkK@mail.python.org> http://hg.python.org/cpython/rev/66770f126c71 changeset: 90127:66770f126c71 user: Serhiy Storchaka date: Fri Apr 04 15:45:02 2014 +0300 summary: Issue #20636: Improved the repr of Tkinter widgets. files: Doc/library/turtle.rst | 2 +- Lib/tkinter/__init__.py | 5 +++++ Lib/tkinter/test/test_tkinter/test_misc.py | 5 +++++ Misc/NEWS | 2 ++ 4 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst --- a/Doc/library/turtle.rst +++ b/Doc/library/turtle.rst @@ -1879,7 +1879,7 @@ >>> cv = screen.getcanvas() >>> cv - + .. function:: getshapes() diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -1280,6 +1280,11 @@ def __str__(self): """Return the window path name of this widget.""" return self._w + + def __repr__(self): + return '<%s.%s object %s>' % ( + self.__class__.__module__, self.__class__.__qualname__, self._w) + # Pack methods that apply to the master _noarg_ = ['_noarg_'] def pack_propagate(self, flag=_noarg_): diff --git a/Lib/tkinter/test/test_tkinter/test_misc.py b/Lib/tkinter/test/test_tkinter/test_misc.py --- a/Lib/tkinter/test/test_tkinter/test_misc.py +++ b/Lib/tkinter/test/test_tkinter/test_misc.py @@ -10,6 +10,11 @@ def setUp(self): self.root = ttk.setup_master() + def test_repr(self): + t = tkinter.Toplevel(self.root, name='top') + f = tkinter.Frame(t, name='child') + self.assertEqual(repr(f), '') + def test_tk_setPalette(self): root = self.root root.tk_setPalette('black') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #20636: Improved the repr of Tkinter widgets. + - Issue #19505: The items, keys, and values views of OrderedDict now support reverse iteration using reversed(). -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 15:34:28 2014 From: python-checkins at python.org (giampaolo.rodola) Date: Fri, 4 Apr 2014 15:34:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_=2321076=3A_turn_signa?= =?utf-8?q?l_module_constants_into_enums?= Message-ID: <3g0hvS5h5Wz7LjM@mail.python.org> http://hg.python.org/cpython/rev/c9239171e429 changeset: 90128:c9239171e429 user: Giampaolo Rodola' date: Fri Apr 04 15:34:17 2014 +0200 summary: fix #21076: turn signal module constants into enums files: Doc/library/signal.rst | 10 +++ Doc/whatsnew/3.5.rst | 5 + Lib/signal.py | 84 ++++++++++++++++++++++++++++ Lib/test/test_doctest.py | 2 +- Lib/test/test_signal.py | 39 +++++++++++- Modules/signalmodule.c | 4 +- PC/config.c | 2 +- 7 files changed, 138 insertions(+), 8 deletions(-) diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -65,6 +65,16 @@ Module contents --------------- +.. versionchanged:: 3.5 + signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and sigmask + (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`) + related constants listed below were turned into + :class:`enums `. + :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and + :func:`sigwait` functions return human-readable + :class:`enums `. + + The variables defined in the :mod:`signal` module are: diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -134,6 +134,11 @@ Improved Modules ================ +* Different constants of :mod:`signal` module are now enumeration values using + the :mod:`enum` module. This allows meaningful names to be printed during + debugging, instead of integer ?magic numbers?. (contribute by Giampaolo + Rodola' in :issue:`21076`) + * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager` (contributed by Claudiu Popa in :issue:`20627`). diff --git a/Lib/signal.py b/Lib/signal.py new file mode 100644 --- /dev/null +++ b/Lib/signal.py @@ -0,0 +1,84 @@ +import _signal +from _signal import * +from functools import wraps as _wraps +from enum import IntEnum as _IntEnum + +_globals = globals() + +Signals = _IntEnum( + 'Signals', + {name: value for name, value in _globals.items() + if name.isupper() + and (name.startswith('SIG') and not name.startswith('SIG_')) + or name.startswith('CTRL_')}) + +class Handlers(_IntEnum): + SIG_DFL = _signal.SIG_DFL + SIG_IGN = _signal.SIG_IGN + +_globals.update(Signals.__members__) +_globals.update(Handlers.__members__) + +if 'pthread_sigmask' in _globals: + class Sigmasks(_IntEnum): + SIG_BLOCK = _signal.SIG_BLOCK + SIG_UNBLOCK = _signal.SIG_UNBLOCK + SIG_SETMASK = _signal.SIG_SETMASK + + _globals.update(Sigmasks.__members__) + + +def _int_to_enum(value, enum_klass): + """Convert a numeric value to an IntEnum member. + If it's not a known member, return the numeric value itself. + """ + try: + return enum_klass(value) + except ValueError: + return value + + +def _enum_to_int(value): + """Convert an IntEnum member to a numeric value. + If it's not a IntEnum member return the value itself. + """ + try: + return int(value) + except (ValueError, TypeError): + return value + + + at _wraps(_signal.signal) +def signal(signalnum, handler): + handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler)) + return _int_to_enum(handler, Handlers) + + + at _wraps(_signal.getsignal) +def getsignal(signalnum): + handler = _signal.getsignal(signalnum) + return _int_to_enum(handler, Handlers) + + +if 'pthread_sigmask' in _globals: + @_wraps(_signal.pthread_sigmask) + def pthread_sigmask(how, mask): + sigs_set = _signal.pthread_sigmask(how, mask) + return set(_int_to_enum(x, Signals) for x in sigs_set) + pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__ + + + at _wraps(_signal.sigpending) +def sigpending(): + sigs = _signal.sigpending() + return set(_int_to_enum(x, Signals) for x in sigs) + + +if 'sigwait' in _globals: + @_wraps(_signal.sigwait) + def sigwait(sigset): + retsig = _signal.sigwait(sigset) + return _int_to_enum(retsig, Signals) + sigwait.__doc__ = _signal.sigwait + +del _globals, _wraps diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -2897,7 +2897,7 @@ def test_main(): # Check the doctest cases in doctest itself: - support.run_doctest(doctest, verbosity=True) + ret = support.run_doctest(doctest, verbosity=True) # Check the doctest cases defined here: from test import test_doctest support.run_doctest(test_doctest, verbosity=True) diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py --- a/Lib/test/test_signal.py +++ b/Lib/test/test_signal.py @@ -1,6 +1,7 @@ import unittest from test import support from contextlib import closing +import enum import gc import pickle import select @@ -39,6 +40,22 @@ return None +class GenericTests(unittest.TestCase): + + def test_enums(self): + for name in dir(signal): + sig = getattr(signal, name) + if name in {'SIG_DFL', 'SIG_IGN'}: + self.assertIsInstance(sig, signal.Handlers) + elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}: + self.assertIsInstance(sig, signal.Sigmasks) + elif name.startswith('SIG') and not name.startswith('SIG_'): + self.assertIsInstance(sig, signal.Signals) + elif name.startswith('CTRL_'): + self.assertIsInstance(sig, signal.Signals) + self.assertEqual(sys.platform, "win32") + + @unittest.skipIf(sys.platform == "win32", "Not valid on Windows") class InterProcessSignalTests(unittest.TestCase): MAX_DURATION = 20 # Entire test should last at most 20 sec. @@ -195,6 +212,7 @@ def test_getsignal(self): hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler) + self.assertIsInstance(hup, signal.Handlers) self.assertEqual(signal.getsignal(signal.SIGHUP), self.trivial_signal_handler) signal.signal(signal.SIGHUP, hup) @@ -271,7 +289,7 @@ os.close(read) os.close(write) - """.format(signals, ordered, test_body) + """.format(tuple(map(int, signals)), ordered, test_body) assert_python_ok('-c', code) @@ -604,6 +622,8 @@ signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) os.kill(os.getpid(), signum) pending = signal.sigpending() + for sig in pending: + assert isinstance(sig, signal.Signals), repr(pending) if pending != {signum}: raise Exception('%s != {%s}' % (pending, signum)) try: @@ -660,6 +680,7 @@ code = '''if 1: import signal import sys + from signal import Signals def handler(signum, frame): 1/0 @@ -702,6 +723,7 @@ def test(signum): signal.alarm(1) received = signal.sigwait([signum]) + assert isinstance(received, signal.Signals), received if received != signum: raise Exception('received %s, not %s' % (received, signum)) ''') @@ -842,8 +864,14 @@ def kill(signum): os.kill(os.getpid(), signum) + def check_mask(mask): + for sig in mask: + assert isinstance(sig, signal.Signals), repr(sig) + def read_sigmask(): - return signal.pthread_sigmask(signal.SIG_BLOCK, []) + sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, []) + check_mask(sigmask) + return sigmask signum = signal.SIGUSR1 @@ -852,6 +880,7 @@ # Unblock SIGUSR1 (and copy the old mask) to test our signal handler old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum]) + check_mask(old_mask) try: kill(signum) except ZeroDivisionError: @@ -861,11 +890,13 @@ # Block and then raise SIGUSR1. The signal is blocked: the signal # handler is not called, and the signal is now pending - signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) + check_mask(mask) kill(signum) # Check the new mask blocked = read_sigmask() + check_mask(blocked) if signum not in blocked: raise Exception("%s not in %s" % (signum, blocked)) if old_mask ^ blocked != {signum}: @@ -928,7 +959,7 @@ def test_main(): try: - support.run_unittest(PosixTests, InterProcessSignalTests, + support.run_unittest(GenericTests, PosixTests, InterProcessSignalTests, WakeupFDTests, WakeupSignalTests, SiginterruptTest, ItimerTest, WindowsSignalTests, PendingSignalsTests) diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -967,7 +967,7 @@ }; PyMODINIT_FUNC -PyInit_signal(void) +PyInit__signal(void) { PyObject *m, *d, *x; int i; @@ -1380,7 +1380,7 @@ void PyOS_InitInterrupts(void) { - PyObject *m = PyImport_ImportModule("signal"); + PyObject *m = PyImport_ImportModule("_signal"); if (m) { Py_DECREF(m); } diff --git a/PC/config.c b/PC/config.c --- a/PC/config.c +++ b/PC/config.c @@ -19,7 +19,7 @@ extern PyObject* PyInit__md5(void); extern PyObject* PyInit_nt(void); extern PyObject* PyInit__operator(void); -extern PyObject* PyInit_signal(void); +extern PyObject* PyInit__signal(void); extern PyObject* PyInit__sha1(void); extern PyObject* PyInit__sha256(void); extern PyObject* PyInit__sha512(void); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 15:52:25 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 15:52:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E2=29=3A_fix_versioncha?= =?utf-8?q?nged_version?= Message-ID: <3g0jJ91RTYz7LjM@mail.python.org> http://hg.python.org/cpython/rev/b9913eb96643 changeset: 90129:b9913eb96643 branch: 3.2 parent: 90094:9186f4a18584 user: Benjamin Peterson date: Fri Apr 04 09:50:35 2014 -0400 summary: fix versionchanged version files: Doc/library/os.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1201,9 +1201,9 @@ .. versionadded:: 3.2 The *exist_ok* parameter. - .. versionchanged:: 3.2.5 - - Before Python 3.2.5, if *exist_ok* was ``True`` and the directory existed, + .. versionchanged:: 3.2.6 + + Before Python 3.2.6, if *exist_ok* was ``True`` and the directory existed, :func:`makedirs` would still raise an error if *mode* did not match the mode of the existing directory. Since this behavior was impossible to implement safely, it was removed in Python 3.2.6. See :issue:`21082`. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 15:52:26 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 15:52:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2?= Message-ID: <3g0jJB2tNGz7LjM@mail.python.org> http://hg.python.org/cpython/rev/63e6afd3ff1f changeset: 90130:63e6afd3ff1f branch: 3.3 parent: 90095:6370d44013f7 parent: 90129:b9913eb96643 user: Benjamin Peterson date: Fri Apr 04 09:51:45 2014 -0400 summary: merge 3.2 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 15:52:27 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 15:52:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <3g0jJC4lCbz7LnJ@mail.python.org> http://hg.python.org/cpython/rev/bfc0f54e66a8 changeset: 90131:bfc0f54e66a8 branch: 3.4 parent: 90124:b5c91b61991a parent: 90130:63e6afd3ff1f user: Benjamin Peterson date: Fri Apr 04 09:52:05 2014 -0400 summary: merge 3.3 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 15:52:28 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 15:52:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g0jJD6Y5Mz7LnT@mail.python.org> http://hg.python.org/cpython/rev/4acff7c4bd02 changeset: 90132:4acff7c4bd02 parent: 90128:c9239171e429 parent: 90131:bfc0f54e66a8 user: Benjamin Peterson date: Fri Apr 04 09:52:18 2014 -0400 summary: merge 3.4 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 15:59:56 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 15:59:56 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_note_that_the_?= =?utf-8?q?hash_of_an_arbitrary_object_is_only_derived_from_its_address?= Message-ID: <3g0jSr64P3z7Ljw@mail.python.org> http://hg.python.org/cpython/rev/6809b434752a changeset: 90133:6809b434752a branch: 2.7 parent: 90123:b6deab7204e6 user: Benjamin Peterson date: Fri Apr 04 09:58:13 2014 -0400 summary: note that the hash of an arbitrary object is only derived from its address (closes #21154) Patch from Armin Rigo. files: Doc/reference/datamodel.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1422,7 +1422,7 @@ User-defined classes have :meth:`__cmp__` and :meth:`__hash__` methods by default; with them, all objects compare unequal (except with themselves) - and ``x.__hash__()`` returns ``id(x)``. + and ``x.__hash__()`` returns a result derived from ``id(x)``. Classes which inherit a :meth:`__hash__` method from a parent class but change the meaning of :meth:`__cmp__` or :meth:`__eq__` such that the hash -- Repository URL: http://hg.python.org/cpython From brett at python.org Fri Apr 4 16:12:23 2014 From: brett at python.org (Brett Cannon) Date: Fri, 4 Apr 2014 10:12:23 -0400 Subject: [Python-checkins] cpython: fix #21076: turn signal module constants into enums In-Reply-To: <3g0hvS5h5Wz7LjM@mail.python.org> References: <3g0hvS5h5Wz7LjM@mail.python.org> Message-ID: This broke compilation on at least OS X, but I'm willing to bet for all UNIX-based systems. I have a fix in the works. On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola wrote: > http://hg.python.org/cpython/rev/c9239171e429 > changeset: 90128:c9239171e429 > user: Giampaolo Rodola' > date: Fri Apr 04 15:34:17 2014 +0200 > summary: > fix #21076: turn signal module constants into enums > > files: > Doc/library/signal.rst | 10 +++ > Doc/whatsnew/3.5.rst | 5 + > Lib/signal.py | 84 ++++++++++++++++++++++++++++ > Lib/test/test_doctest.py | 2 +- > Lib/test/test_signal.py | 39 +++++++++++- > Modules/signalmodule.c | 4 +- > PC/config.c | 2 +- > 7 files changed, 138 insertions(+), 8 deletions(-) > > > diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst > --- a/Doc/library/signal.rst > +++ b/Doc/library/signal.rst > @@ -65,6 +65,16 @@ > Module contents > --------------- > > +.. versionchanged:: 3.5 > + signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and sigmask > + (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`) > + related constants listed below were turned into > + :class:`enums `. > + :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and > + :func:`sigwait` functions return human-readable > + :class:`enums `. > + > + > The variables defined in the :mod:`signal` module are: > > > diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst > --- a/Doc/whatsnew/3.5.rst > +++ b/Doc/whatsnew/3.5.rst > @@ -134,6 +134,11 @@ > Improved Modules > ================ > > +* Different constants of :mod:`signal` module are now enumeration values > using > + the :mod:`enum` module. This allows meaningful names to be printed > during > + debugging, instead of integer ?magic numbers?. (contribute by Giampaolo > + Rodola' in :issue:`21076`) > + > * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager` > (contributed by Claudiu Popa in :issue:`20627`). > > diff --git a/Lib/signal.py b/Lib/signal.py > new file mode 100644 > --- /dev/null > +++ b/Lib/signal.py > @@ -0,0 +1,84 @@ > +import _signal > +from _signal import * > +from functools import wraps as _wraps > +from enum import IntEnum as _IntEnum > + > +_globals = globals() > + > +Signals = _IntEnum( > + 'Signals', > + {name: value for name, value in _globals.items() > + if name.isupper() > + and (name.startswith('SIG') and not name.startswith('SIG_')) > + or name.startswith('CTRL_')}) > + > +class Handlers(_IntEnum): > + SIG_DFL = _signal.SIG_DFL > + SIG_IGN = _signal.SIG_IGN > + > +_globals.update(Signals.__members__) > +_globals.update(Handlers.__members__) > + > +if 'pthread_sigmask' in _globals: > + class Sigmasks(_IntEnum): > + SIG_BLOCK = _signal.SIG_BLOCK > + SIG_UNBLOCK = _signal.SIG_UNBLOCK > + SIG_SETMASK = _signal.SIG_SETMASK > + > + _globals.update(Sigmasks.__members__) > + > + > +def _int_to_enum(value, enum_klass): > + """Convert a numeric value to an IntEnum member. > + If it's not a known member, return the numeric value itself. > + """ > + try: > + return enum_klass(value) > + except ValueError: > + return value > + > + > +def _enum_to_int(value): > + """Convert an IntEnum member to a numeric value. > + If it's not a IntEnum member return the value itself. > + """ > + try: > + return int(value) > + except (ValueError, TypeError): > + return value > + > + > + at _wraps(_signal.signal) > +def signal(signalnum, handler): > + handler = _signal.signal(_enum_to_int(signalnum), > _enum_to_int(handler)) > + return _int_to_enum(handler, Handlers) > + > + > + at _wraps(_signal.getsignal) > +def getsignal(signalnum): > + handler = _signal.getsignal(signalnum) > + return _int_to_enum(handler, Handlers) > + > + > +if 'pthread_sigmask' in _globals: > + @_wraps(_signal.pthread_sigmask) > + def pthread_sigmask(how, mask): > + sigs_set = _signal.pthread_sigmask(how, mask) > + return set(_int_to_enum(x, Signals) for x in sigs_set) > + pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__ > + > + > + at _wraps(_signal.sigpending) > +def sigpending(): > + sigs = _signal.sigpending() > + return set(_int_to_enum(x, Signals) for x in sigs) > + > + > +if 'sigwait' in _globals: > + @_wraps(_signal.sigwait) > + def sigwait(sigset): > + retsig = _signal.sigwait(sigset) > + return _int_to_enum(retsig, Signals) > + sigwait.__doc__ = _signal.sigwait > + > +del _globals, _wraps > diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py > --- a/Lib/test/test_doctest.py > +++ b/Lib/test/test_doctest.py > @@ -2897,7 +2897,7 @@ > > def test_main(): > # Check the doctest cases in doctest itself: > - support.run_doctest(doctest, verbosity=True) > + ret = support.run_doctest(doctest, verbosity=True) > # Check the doctest cases defined here: > from test import test_doctest > support.run_doctest(test_doctest, verbosity=True) > diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py > --- a/Lib/test/test_signal.py > +++ b/Lib/test/test_signal.py > @@ -1,6 +1,7 @@ > import unittest > from test import support > from contextlib import closing > +import enum > import gc > import pickle > import select > @@ -39,6 +40,22 @@ > return None > > > +class GenericTests(unittest.TestCase): > + > + def test_enums(self): > + for name in dir(signal): > + sig = getattr(signal, name) > + if name in {'SIG_DFL', 'SIG_IGN'}: > + self.assertIsInstance(sig, signal.Handlers) > + elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}: > + self.assertIsInstance(sig, signal.Sigmasks) > + elif name.startswith('SIG') and not name.startswith('SIG_'): > + self.assertIsInstance(sig, signal.Signals) > + elif name.startswith('CTRL_'): > + self.assertIsInstance(sig, signal.Signals) > + self.assertEqual(sys.platform, "win32") > + > + > @unittest.skipIf(sys.platform == "win32", "Not valid on Windows") > class InterProcessSignalTests(unittest.TestCase): > MAX_DURATION = 20 # Entire test should last at most 20 sec. > @@ -195,6 +212,7 @@ > > def test_getsignal(self): > hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler) > + self.assertIsInstance(hup, signal.Handlers) > self.assertEqual(signal.getsignal(signal.SIGHUP), > self.trivial_signal_handler) > signal.signal(signal.SIGHUP, hup) > @@ -271,7 +289,7 @@ > > os.close(read) > os.close(write) > - """.format(signals, ordered, test_body) > + """.format(tuple(map(int, signals)), ordered, test_body) > > assert_python_ok('-c', code) > > @@ -604,6 +622,8 @@ > signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) > os.kill(os.getpid(), signum) > pending = signal.sigpending() > + for sig in pending: > + assert isinstance(sig, signal.Signals), repr(pending) > if pending != {signum}: > raise Exception('%s != {%s}' % (pending, signum)) > try: > @@ -660,6 +680,7 @@ > code = '''if 1: > import signal > import sys > + from signal import Signals > > def handler(signum, frame): > 1/0 > @@ -702,6 +723,7 @@ > def test(signum): > signal.alarm(1) > received = signal.sigwait([signum]) > + assert isinstance(received, signal.Signals), received > if received != signum: > raise Exception('received %s, not %s' % (received, > signum)) > ''') > @@ -842,8 +864,14 @@ > def kill(signum): > os.kill(os.getpid(), signum) > > + def check_mask(mask): > + for sig in mask: > + assert isinstance(sig, signal.Signals), repr(sig) > + > def read_sigmask(): > - return signal.pthread_sigmask(signal.SIG_BLOCK, []) > + sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, []) > + check_mask(sigmask) > + return sigmask > > signum = signal.SIGUSR1 > > @@ -852,6 +880,7 @@ > > # Unblock SIGUSR1 (and copy the old mask) to test our signal > handler > old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum]) > + check_mask(old_mask) > try: > kill(signum) > except ZeroDivisionError: > @@ -861,11 +890,13 @@ > > # Block and then raise SIGUSR1. The signal is blocked: the signal > # handler is not called, and the signal is now pending > - signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) > + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) > + check_mask(mask) > kill(signum) > > # Check the new mask > blocked = read_sigmask() > + check_mask(blocked) > if signum not in blocked: > raise Exception("%s not in %s" % (signum, blocked)) > if old_mask ^ blocked != {signum}: > @@ -928,7 +959,7 @@ > > def test_main(): > try: > - support.run_unittest(PosixTests, InterProcessSignalTests, > + support.run_unittest(GenericTests, PosixTests, > InterProcessSignalTests, > WakeupFDTests, WakeupSignalTests, > SiginterruptTest, ItimerTest, > WindowsSignalTests, > PendingSignalsTests) > diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c > --- a/Modules/signalmodule.c > +++ b/Modules/signalmodule.c > @@ -967,7 +967,7 @@ > }; > > PyMODINIT_FUNC > -PyInit_signal(void) > +PyInit__signal(void) > { > PyObject *m, *d, *x; > int i; > @@ -1380,7 +1380,7 @@ > void > PyOS_InitInterrupts(void) > { > - PyObject *m = PyImport_ImportModule("signal"); > + PyObject *m = PyImport_ImportModule("_signal"); > if (m) { > Py_DECREF(m); > } > diff --git a/PC/config.c b/PC/config.c > --- a/PC/config.c > +++ b/PC/config.c > @@ -19,7 +19,7 @@ > extern PyObject* PyInit__md5(void); > extern PyObject* PyInit_nt(void); > extern PyObject* PyInit__operator(void); > -extern PyObject* PyInit_signal(void); > +extern PyObject* PyInit__signal(void); > extern PyObject* PyInit__sha1(void); > extern PyObject* PyInit__sha256(void); > extern PyObject* PyInit__sha512(void); > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-checkins at python.org Fri Apr 4 16:20:38 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 4 Apr 2014 16:20:38 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwOTQy?= =?utf-8?q?=3A_PyImport=5FImportFrozenModuleObject=28=29_no_longer_sets?= Message-ID: <3g0jwk1z7hz7Lkk@mail.python.org> http://hg.python.org/cpython/rev/fef890bd60b1 changeset: 90134:fef890bd60b1 branch: 3.4 parent: 90131:bfc0f54e66a8 user: Brett Cannon date: Fri Apr 04 10:01:46 2014 -0400 summary: Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__. This causes _frozen_importlib to no longer have __file__ set as well as any frozen module imported using imp.init_frozen() (which is deprecated). files: Doc/c-api/import.rst | 3 + Misc/NEWS | 6 ++ Python/import.c | 86 ++++++++++++++++++++----------- 3 files changed, 63 insertions(+), 32 deletions(-) diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -245,6 +245,9 @@ .. versionadded:: 3.3 + .. versionchanged:: 3.4 + The ``__file__`` attribute is no longer set on the module. + .. c:function:: int PyImport_ImportFrozenModule(const char *name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -109,6 +109,12 @@ - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. +C API +----- +- Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__ to + match what importlib does; this affects _frozen_importlib as well as any + module loaded using imp.init_frozen(). + Documentation ------------- diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -837,12 +837,10 @@ return m; } -PyObject* -PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, - PyObject *cpathname) +static PyObject * +module_dict_for_exec(PyObject *name) { - PyObject *modules = PyImport_GetModuleDict(); - PyObject *m, *d, *v; + PyObject *m, *d = NULL; m = PyImport_AddModuleObject(name); if (m == NULL) @@ -852,9 +850,51 @@ d = PyModule_GetDict(m); if (PyDict_GetItemString(d, "__builtins__") == NULL) { if (PyDict_SetItemString(d, "__builtins__", - PyEval_GetBuiltins()) != 0) - goto error; + PyEval_GetBuiltins()) != 0) { + remove_module(name); + return NULL; + } } + + return d; +} + +static PyObject * +exec_code_in_module(PyObject *name, PyObject *module_dict, PyObject *code_object) +{ + PyObject *modules = PyImport_GetModuleDict(); + PyObject *v, *m; + + v = PyEval_EvalCode(code_object, module_dict, module_dict); + if (v == NULL) { + remove_module(name); + return NULL; + } + Py_DECREF(v); + + if ((m = PyDict_GetItem(modules, name)) == NULL) { + PyErr_Format(PyExc_ImportError, + "Loaded module %R not found in sys.modules", + name); + return NULL; + } + + Py_INCREF(m); + + return m; +} + +PyObject* +PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, + PyObject *cpathname) +{ + PyObject *d, *v; + + d = module_dict_for_exec(name); + if (d == NULL) { + return NULL; + } + if (pathname != NULL) { v = pathname; } @@ -874,25 +914,7 @@ if (PyDict_SetItemString(d, "__cached__", v) != 0) PyErr_Clear(); /* Not important enough to report */ - v = PyEval_EvalCode(co, d, d); - if (v == NULL) - goto error; - Py_DECREF(v); - - if ((m = PyDict_GetItem(modules, name)) == NULL) { - PyErr_Format(PyExc_ImportError, - "Loaded module %R not found in sys.modules", - name); - return NULL; - } - - Py_INCREF(m); - - return m; - - error: - remove_module(name); - return NULL; + return exec_code_in_module(name, d, co); } @@ -1206,7 +1228,7 @@ PyImport_ImportFrozenModuleObject(PyObject *name) { const struct _frozen *p; - PyObject *co, *m, *path; + PyObject *co, *m, *d; int ispackage; int size; @@ -1235,7 +1257,7 @@ } if (ispackage) { /* Set __path__ to the empty list */ - PyObject *d, *l; + PyObject *l; int err; m = PyImport_AddModuleObject(name); if (m == NULL) @@ -1250,11 +1272,11 @@ if (err != 0) goto err_return; } - path = PyUnicode_FromString(""); - if (path == NULL) + d = module_dict_for_exec(name); + if (d == NULL) { goto err_return; - m = PyImport_ExecCodeModuleObject(name, co, path, NULL); - Py_DECREF(path); + } + m = exec_code_in_module(name, d, co); if (m == NULL) goto err_return; Py_DECREF(co); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 16:20:39 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 4 Apr 2014 16:20:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_of_fix_for_issue_=2320942?= Message-ID: <3g0jwl4Z8Mz7Ln6@mail.python.org> http://hg.python.org/cpython/rev/a11ec7aaac10 changeset: 90135:a11ec7aaac10 parent: 90127:66770f126c71 parent: 90134:fef890bd60b1 user: Brett Cannon date: Fri Apr 04 10:03:42 2014 -0400 summary: merge of fix for issue #20942 files: Doc/c-api/import.rst | 3 + Misc/NEWS | 6 ++ Python/import.c | 86 ++++++++++++++++++++----------- 3 files changed, 63 insertions(+), 32 deletions(-) diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -245,6 +245,9 @@ .. versionadded:: 3.3 + .. versionchanged:: 3.4 + The ``__file__`` attribute is no longer set on the module. + .. c:function:: int PyImport_ImportFrozenModule(const char *name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -144,6 +144,12 @@ - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. +C API +----- +- Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__ to + match what importlib does; this affects _frozen_importlib as well as any + module loaded using imp.init_frozen(). + Documentation ------------- diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -837,12 +837,10 @@ return m; } -PyObject* -PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, - PyObject *cpathname) +static PyObject * +module_dict_for_exec(PyObject *name) { - PyObject *modules = PyImport_GetModuleDict(); - PyObject *m, *d, *v; + PyObject *m, *d = NULL; m = PyImport_AddModuleObject(name); if (m == NULL) @@ -852,9 +850,51 @@ d = PyModule_GetDict(m); if (PyDict_GetItemString(d, "__builtins__") == NULL) { if (PyDict_SetItemString(d, "__builtins__", - PyEval_GetBuiltins()) != 0) - goto error; + PyEval_GetBuiltins()) != 0) { + remove_module(name); + return NULL; + } } + + return d; +} + +static PyObject * +exec_code_in_module(PyObject *name, PyObject *module_dict, PyObject *code_object) +{ + PyObject *modules = PyImport_GetModuleDict(); + PyObject *v, *m; + + v = PyEval_EvalCode(code_object, module_dict, module_dict); + if (v == NULL) { + remove_module(name); + return NULL; + } + Py_DECREF(v); + + if ((m = PyDict_GetItem(modules, name)) == NULL) { + PyErr_Format(PyExc_ImportError, + "Loaded module %R not found in sys.modules", + name); + return NULL; + } + + Py_INCREF(m); + + return m; +} + +PyObject* +PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, + PyObject *cpathname) +{ + PyObject *d, *v; + + d = module_dict_for_exec(name); + if (d == NULL) { + return NULL; + } + if (pathname != NULL) { v = pathname; } @@ -874,25 +914,7 @@ if (PyDict_SetItemString(d, "__cached__", v) != 0) PyErr_Clear(); /* Not important enough to report */ - v = PyEval_EvalCode(co, d, d); - if (v == NULL) - goto error; - Py_DECREF(v); - - if ((m = PyDict_GetItem(modules, name)) == NULL) { - PyErr_Format(PyExc_ImportError, - "Loaded module %R not found in sys.modules", - name); - return NULL; - } - - Py_INCREF(m); - - return m; - - error: - remove_module(name); - return NULL; + return exec_code_in_module(name, d, co); } @@ -1206,7 +1228,7 @@ PyImport_ImportFrozenModuleObject(PyObject *name) { const struct _frozen *p; - PyObject *co, *m, *path; + PyObject *co, *m, *d; int ispackage; int size; @@ -1235,7 +1257,7 @@ } if (ispackage) { /* Set __path__ to the empty list */ - PyObject *d, *l; + PyObject *l; int err; m = PyImport_AddModuleObject(name); if (m == NULL) @@ -1250,11 +1272,11 @@ if (err != 0) goto err_return; } - path = PyUnicode_FromString(""); - if (path == NULL) + d = module_dict_for_exec(name); + if (d == NULL) { goto err_return; - m = PyImport_ExecCodeModuleObject(name, co, path, NULL); - Py_DECREF(path); + } + m = exec_code_in_module(name, d, co); if (m == NULL) goto err_return; Py_DECREF(co); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 16:20:41 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 4 Apr 2014 16:20:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_along_w/_fix_for_issue_=232107_=28commit_c9239171e?= =?utf-8?q?429=29?= Message-ID: <3g0jwn0LRYz7Lp4@mail.python.org> http://hg.python.org/cpython/rev/c6e63bb132fb changeset: 90136:c6e63bb132fb parent: 90135:a11ec7aaac10 parent: 90132:4acff7c4bd02 user: Brett Cannon date: Fri Apr 04 10:20:28 2014 -0400 summary: merge along w/ fix for issue #2107 (commit c9239171e429) files: Doc/library/signal.rst | 10 +++ Doc/whatsnew/3.5.rst | 5 + Lib/signal.py | 84 ++++++++++++++++++++++++++++ Lib/test/test_doctest.py | 2 +- Lib/test/test_signal.py | 39 +++++++++++- Modules/signalmodule.c | 6 +- PC/config.c | 2 +- 7 files changed, 139 insertions(+), 9 deletions(-) diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -65,6 +65,16 @@ Module contents --------------- +.. versionchanged:: 3.5 + signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and sigmask + (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`) + related constants listed below were turned into + :class:`enums `. + :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and + :func:`sigwait` functions return human-readable + :class:`enums `. + + The variables defined in the :mod:`signal` module are: diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -134,6 +134,11 @@ Improved Modules ================ +* Different constants of :mod:`signal` module are now enumeration values using + the :mod:`enum` module. This allows meaningful names to be printed during + debugging, instead of integer ?magic numbers?. (contribute by Giampaolo + Rodola' in :issue:`21076`) + * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager` (contributed by Claudiu Popa in :issue:`20627`). diff --git a/Lib/signal.py b/Lib/signal.py new file mode 100644 --- /dev/null +++ b/Lib/signal.py @@ -0,0 +1,84 @@ +import _signal +from _signal import * +from functools import wraps as _wraps +from enum import IntEnum as _IntEnum + +_globals = globals() + +Signals = _IntEnum( + 'Signals', + {name: value for name, value in _globals.items() + if name.isupper() + and (name.startswith('SIG') and not name.startswith('SIG_')) + or name.startswith('CTRL_')}) + +class Handlers(_IntEnum): + SIG_DFL = _signal.SIG_DFL + SIG_IGN = _signal.SIG_IGN + +_globals.update(Signals.__members__) +_globals.update(Handlers.__members__) + +if 'pthread_sigmask' in _globals: + class Sigmasks(_IntEnum): + SIG_BLOCK = _signal.SIG_BLOCK + SIG_UNBLOCK = _signal.SIG_UNBLOCK + SIG_SETMASK = _signal.SIG_SETMASK + + _globals.update(Sigmasks.__members__) + + +def _int_to_enum(value, enum_klass): + """Convert a numeric value to an IntEnum member. + If it's not a known member, return the numeric value itself. + """ + try: + return enum_klass(value) + except ValueError: + return value + + +def _enum_to_int(value): + """Convert an IntEnum member to a numeric value. + If it's not a IntEnum member return the value itself. + """ + try: + return int(value) + except (ValueError, TypeError): + return value + + + at _wraps(_signal.signal) +def signal(signalnum, handler): + handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler)) + return _int_to_enum(handler, Handlers) + + + at _wraps(_signal.getsignal) +def getsignal(signalnum): + handler = _signal.getsignal(signalnum) + return _int_to_enum(handler, Handlers) + + +if 'pthread_sigmask' in _globals: + @_wraps(_signal.pthread_sigmask) + def pthread_sigmask(how, mask): + sigs_set = _signal.pthread_sigmask(how, mask) + return set(_int_to_enum(x, Signals) for x in sigs_set) + pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__ + + + at _wraps(_signal.sigpending) +def sigpending(): + sigs = _signal.sigpending() + return set(_int_to_enum(x, Signals) for x in sigs) + + +if 'sigwait' in _globals: + @_wraps(_signal.sigwait) + def sigwait(sigset): + retsig = _signal.sigwait(sigset) + return _int_to_enum(retsig, Signals) + sigwait.__doc__ = _signal.sigwait + +del _globals, _wraps diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -2897,7 +2897,7 @@ def test_main(): # Check the doctest cases in doctest itself: - support.run_doctest(doctest, verbosity=True) + ret = support.run_doctest(doctest, verbosity=True) # Check the doctest cases defined here: from test import test_doctest support.run_doctest(test_doctest, verbosity=True) diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py --- a/Lib/test/test_signal.py +++ b/Lib/test/test_signal.py @@ -1,6 +1,7 @@ import unittest from test import support from contextlib import closing +import enum import gc import pickle import select @@ -39,6 +40,22 @@ return None +class GenericTests(unittest.TestCase): + + def test_enums(self): + for name in dir(signal): + sig = getattr(signal, name) + if name in {'SIG_DFL', 'SIG_IGN'}: + self.assertIsInstance(sig, signal.Handlers) + elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}: + self.assertIsInstance(sig, signal.Sigmasks) + elif name.startswith('SIG') and not name.startswith('SIG_'): + self.assertIsInstance(sig, signal.Signals) + elif name.startswith('CTRL_'): + self.assertIsInstance(sig, signal.Signals) + self.assertEqual(sys.platform, "win32") + + @unittest.skipIf(sys.platform == "win32", "Not valid on Windows") class InterProcessSignalTests(unittest.TestCase): MAX_DURATION = 20 # Entire test should last at most 20 sec. @@ -195,6 +212,7 @@ def test_getsignal(self): hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler) + self.assertIsInstance(hup, signal.Handlers) self.assertEqual(signal.getsignal(signal.SIGHUP), self.trivial_signal_handler) signal.signal(signal.SIGHUP, hup) @@ -271,7 +289,7 @@ os.close(read) os.close(write) - """.format(signals, ordered, test_body) + """.format(tuple(map(int, signals)), ordered, test_body) assert_python_ok('-c', code) @@ -604,6 +622,8 @@ signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) os.kill(os.getpid(), signum) pending = signal.sigpending() + for sig in pending: + assert isinstance(sig, signal.Signals), repr(pending) if pending != {signum}: raise Exception('%s != {%s}' % (pending, signum)) try: @@ -660,6 +680,7 @@ code = '''if 1: import signal import sys + from signal import Signals def handler(signum, frame): 1/0 @@ -702,6 +723,7 @@ def test(signum): signal.alarm(1) received = signal.sigwait([signum]) + assert isinstance(received, signal.Signals), received if received != signum: raise Exception('received %s, not %s' % (received, signum)) ''') @@ -842,8 +864,14 @@ def kill(signum): os.kill(os.getpid(), signum) + def check_mask(mask): + for sig in mask: + assert isinstance(sig, signal.Signals), repr(sig) + def read_sigmask(): - return signal.pthread_sigmask(signal.SIG_BLOCK, []) + sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, []) + check_mask(sigmask) + return sigmask signum = signal.SIGUSR1 @@ -852,6 +880,7 @@ # Unblock SIGUSR1 (and copy the old mask) to test our signal handler old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum]) + check_mask(old_mask) try: kill(signum) except ZeroDivisionError: @@ -861,11 +890,13 @@ # Block and then raise SIGUSR1. The signal is blocked: the signal # handler is not called, and the signal is now pending - signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) + check_mask(mask) kill(signum) # Check the new mask blocked = read_sigmask() + check_mask(blocked) if signum not in blocked: raise Exception("%s not in %s" % (signum, blocked)) if old_mask ^ blocked != {signum}: @@ -928,7 +959,7 @@ def test_main(): try: - support.run_unittest(PosixTests, InterProcessSignalTests, + support.run_unittest(GenericTests, PosixTests, InterProcessSignalTests, WakeupFDTests, WakeupSignalTests, SiginterruptTest, ItimerTest, WindowsSignalTests, PendingSignalsTests) diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -956,7 +956,7 @@ static struct PyModuleDef signalmodule = { PyModuleDef_HEAD_INIT, - "signal", + "_signal", module_doc, -1, signal_methods, @@ -967,7 +967,7 @@ }; PyMODINIT_FUNC -PyInit_signal(void) +PyInit__signal(void) { PyObject *m, *d, *x; int i; @@ -1380,7 +1380,7 @@ void PyOS_InitInterrupts(void) { - PyObject *m = PyImport_ImportModule("signal"); + PyObject *m = PyImport_ImportModule("_signal"); if (m) { Py_DECREF(m); } diff --git a/PC/config.c b/PC/config.c --- a/PC/config.c +++ b/PC/config.c @@ -19,7 +19,7 @@ extern PyObject* PyInit__md5(void); extern PyObject* PyInit_nt(void); extern PyObject* PyInit__operator(void); -extern PyObject* PyInit_signal(void); +extern PyObject* PyInit__signal(void); extern PyObject* PyInit__sha1(void); extern PyObject* PyInit__sha256(void); extern PyObject* PyInit__sha512(void); -- Repository URL: http://hg.python.org/cpython From brett at python.org Fri Apr 4 16:21:39 2014 From: brett at python.org (Brett Cannon) Date: Fri, 4 Apr 2014 10:21:39 -0400 Subject: [Python-checkins] cpython: fix #21076: turn signal module constants into enums In-Reply-To: References: <3g0hvS5h5Wz7LjM@mail.python.org> Message-ID: On Fri, Apr 4, 2014 at 10:12 AM, Brett Cannon wrote: > This broke compilation on at least OS X, but I'm willing to bet for all > UNIX-based systems. I have a fix in the works. > Fix is in rev c6e63bb132fb . > > > On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola < > python-checkins at python.org> wrote: > >> http://hg.python.org/cpython/rev/c9239171e429 >> changeset: 90128:c9239171e429 >> user: Giampaolo Rodola' >> date: Fri Apr 04 15:34:17 2014 +0200 >> summary: >> fix #21076: turn signal module constants into enums >> >> files: >> Doc/library/signal.rst | 10 +++ >> Doc/whatsnew/3.5.rst | 5 + >> Lib/signal.py | 84 ++++++++++++++++++++++++++++ >> Lib/test/test_doctest.py | 2 +- >> Lib/test/test_signal.py | 39 +++++++++++- >> Modules/signalmodule.c | 4 +- >> PC/config.c | 2 +- >> 7 files changed, 138 insertions(+), 8 deletions(-) >> >> >> diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst >> --- a/Doc/library/signal.rst >> +++ b/Doc/library/signal.rst >> @@ -65,6 +65,16 @@ >> Module contents >> --------------- >> >> +.. versionchanged:: 3.5 >> + signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and >> sigmask >> + (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`) >> + related constants listed below were turned into >> + :class:`enums `. >> + :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and >> + :func:`sigwait` functions return human-readable >> + :class:`enums `. >> + >> + >> The variables defined in the :mod:`signal` module are: >> >> >> diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst >> --- a/Doc/whatsnew/3.5.rst >> +++ b/Doc/whatsnew/3.5.rst >> @@ -134,6 +134,11 @@ >> Improved Modules >> ================ >> >> +* Different constants of :mod:`signal` module are now enumeration values >> using >> + the :mod:`enum` module. This allows meaningful names to be printed >> during >> + debugging, instead of integer ?magic numbers?. (contribute by Giampaolo >> + Rodola' in :issue:`21076`) >> + >> * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager` >> (contributed by Claudiu Popa in :issue:`20627`). >> >> diff --git a/Lib/signal.py b/Lib/signal.py >> new file mode 100644 >> --- /dev/null >> +++ b/Lib/signal.py >> @@ -0,0 +1,84 @@ >> +import _signal >> +from _signal import * >> +from functools import wraps as _wraps >> +from enum import IntEnum as _IntEnum >> + >> +_globals = globals() >> + >> +Signals = _IntEnum( >> + 'Signals', >> + {name: value for name, value in _globals.items() >> + if name.isupper() >> + and (name.startswith('SIG') and not name.startswith('SIG_')) >> + or name.startswith('CTRL_')}) >> + >> +class Handlers(_IntEnum): >> + SIG_DFL = _signal.SIG_DFL >> + SIG_IGN = _signal.SIG_IGN >> + >> +_globals.update(Signals.__members__) >> +_globals.update(Handlers.__members__) >> + >> +if 'pthread_sigmask' in _globals: >> + class Sigmasks(_IntEnum): >> + SIG_BLOCK = _signal.SIG_BLOCK >> + SIG_UNBLOCK = _signal.SIG_UNBLOCK >> + SIG_SETMASK = _signal.SIG_SETMASK >> + >> + _globals.update(Sigmasks.__members__) >> + >> + >> +def _int_to_enum(value, enum_klass): >> + """Convert a numeric value to an IntEnum member. >> + If it's not a known member, return the numeric value itself. >> + """ >> + try: >> + return enum_klass(value) >> + except ValueError: >> + return value >> + >> + >> +def _enum_to_int(value): >> + """Convert an IntEnum member to a numeric value. >> + If it's not a IntEnum member return the value itself. >> + """ >> + try: >> + return int(value) >> + except (ValueError, TypeError): >> + return value >> + >> + >> + at _wraps(_signal.signal) >> +def signal(signalnum, handler): >> + handler = _signal.signal(_enum_to_int(signalnum), >> _enum_to_int(handler)) >> + return _int_to_enum(handler, Handlers) >> + >> + >> + at _wraps(_signal.getsignal) >> +def getsignal(signalnum): >> + handler = _signal.getsignal(signalnum) >> + return _int_to_enum(handler, Handlers) >> + >> + >> +if 'pthread_sigmask' in _globals: >> + @_wraps(_signal.pthread_sigmask) >> + def pthread_sigmask(how, mask): >> + sigs_set = _signal.pthread_sigmask(how, mask) >> + return set(_int_to_enum(x, Signals) for x in sigs_set) >> + pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__ >> + >> + >> + at _wraps(_signal.sigpending) >> +def sigpending(): >> + sigs = _signal.sigpending() >> + return set(_int_to_enum(x, Signals) for x in sigs) >> + >> + >> +if 'sigwait' in _globals: >> + @_wraps(_signal.sigwait) >> + def sigwait(sigset): >> + retsig = _signal.sigwait(sigset) >> + return _int_to_enum(retsig, Signals) >> + sigwait.__doc__ = _signal.sigwait >> + >> +del _globals, _wraps >> diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py >> --- a/Lib/test/test_doctest.py >> +++ b/Lib/test/test_doctest.py >> @@ -2897,7 +2897,7 @@ >> >> def test_main(): >> # Check the doctest cases in doctest itself: >> - support.run_doctest(doctest, verbosity=True) >> + ret = support.run_doctest(doctest, verbosity=True) >> # Check the doctest cases defined here: >> from test import test_doctest >> support.run_doctest(test_doctest, verbosity=True) >> diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py >> --- a/Lib/test/test_signal.py >> +++ b/Lib/test/test_signal.py >> @@ -1,6 +1,7 @@ >> import unittest >> from test import support >> from contextlib import closing >> +import enum >> import gc >> import pickle >> import select >> @@ -39,6 +40,22 @@ >> return None >> >> >> +class GenericTests(unittest.TestCase): >> + >> + def test_enums(self): >> + for name in dir(signal): >> + sig = getattr(signal, name) >> + if name in {'SIG_DFL', 'SIG_IGN'}: >> + self.assertIsInstance(sig, signal.Handlers) >> + elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}: >> + self.assertIsInstance(sig, signal.Sigmasks) >> + elif name.startswith('SIG') and not name.startswith('SIG_'): >> + self.assertIsInstance(sig, signal.Signals) >> + elif name.startswith('CTRL_'): >> + self.assertIsInstance(sig, signal.Signals) >> + self.assertEqual(sys.platform, "win32") >> + >> + >> @unittest.skipIf(sys.platform == "win32", "Not valid on Windows") >> class InterProcessSignalTests(unittest.TestCase): >> MAX_DURATION = 20 # Entire test should last at most 20 sec. >> @@ -195,6 +212,7 @@ >> >> def test_getsignal(self): >> hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler) >> + self.assertIsInstance(hup, signal.Handlers) >> self.assertEqual(signal.getsignal(signal.SIGHUP), >> self.trivial_signal_handler) >> signal.signal(signal.SIGHUP, hup) >> @@ -271,7 +289,7 @@ >> >> os.close(read) >> os.close(write) >> - """.format(signals, ordered, test_body) >> + """.format(tuple(map(int, signals)), ordered, test_body) >> >> assert_python_ok('-c', code) >> >> @@ -604,6 +622,8 @@ >> signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) >> os.kill(os.getpid(), signum) >> pending = signal.sigpending() >> + for sig in pending: >> + assert isinstance(sig, signal.Signals), repr(pending) >> if pending != {signum}: >> raise Exception('%s != {%s}' % (pending, signum)) >> try: >> @@ -660,6 +680,7 @@ >> code = '''if 1: >> import signal >> import sys >> + from signal import Signals >> >> def handler(signum, frame): >> 1/0 >> @@ -702,6 +723,7 @@ >> def test(signum): >> signal.alarm(1) >> received = signal.sigwait([signum]) >> + assert isinstance(received, signal.Signals), received >> if received != signum: >> raise Exception('received %s, not %s' % (received, >> signum)) >> ''') >> @@ -842,8 +864,14 @@ >> def kill(signum): >> os.kill(os.getpid(), signum) >> >> + def check_mask(mask): >> + for sig in mask: >> + assert isinstance(sig, signal.Signals), repr(sig) >> + >> def read_sigmask(): >> - return signal.pthread_sigmask(signal.SIG_BLOCK, []) >> + sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, []) >> + check_mask(sigmask) >> + return sigmask >> >> signum = signal.SIGUSR1 >> >> @@ -852,6 +880,7 @@ >> >> # Unblock SIGUSR1 (and copy the old mask) to test our signal >> handler >> old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum]) >> + check_mask(old_mask) >> try: >> kill(signum) >> except ZeroDivisionError: >> @@ -861,11 +890,13 @@ >> >> # Block and then raise SIGUSR1. The signal is blocked: the signal >> # handler is not called, and the signal is now pending >> - signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) >> + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) >> + check_mask(mask) >> kill(signum) >> >> # Check the new mask >> blocked = read_sigmask() >> + check_mask(blocked) >> if signum not in blocked: >> raise Exception("%s not in %s" % (signum, blocked)) >> if old_mask ^ blocked != {signum}: >> @@ -928,7 +959,7 @@ >> >> def test_main(): >> try: >> - support.run_unittest(PosixTests, InterProcessSignalTests, >> + support.run_unittest(GenericTests, PosixTests, >> InterProcessSignalTests, >> WakeupFDTests, WakeupSignalTests, >> SiginterruptTest, ItimerTest, >> WindowsSignalTests, >> PendingSignalsTests) >> diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c >> --- a/Modules/signalmodule.c >> +++ b/Modules/signalmodule.c >> @@ -967,7 +967,7 @@ >> }; >> >> PyMODINIT_FUNC >> -PyInit_signal(void) >> +PyInit__signal(void) >> { >> PyObject *m, *d, *x; >> int i; >> @@ -1380,7 +1380,7 @@ >> void >> PyOS_InitInterrupts(void) >> { >> - PyObject *m = PyImport_ImportModule("signal"); >> + PyObject *m = PyImport_ImportModule("_signal"); >> if (m) { >> Py_DECREF(m); >> } >> diff --git a/PC/config.c b/PC/config.c >> --- a/PC/config.c >> +++ b/PC/config.c >> @@ -19,7 +19,7 @@ >> extern PyObject* PyInit__md5(void); >> extern PyObject* PyInit_nt(void); >> extern PyObject* PyInit__operator(void); >> -extern PyObject* PyInit_signal(void); >> +extern PyObject* PyInit__signal(void); >> extern PyObject* PyInit__sha1(void); >> extern PyObject* PyInit__sha256(void); >> extern PyObject* PyInit__sha512(void); >> >> -- >> Repository URL: http://hg.python.org/cpython >> >> _______________________________________________ >> Python-checkins mailing list >> Python-checkins at python.org >> https://mail.python.org/mailman/listinfo/python-checkins >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-checkins at python.org Fri Apr 4 16:31:47 2014 From: python-checkins at python.org (victor.stinner) Date: Fri, 4 Apr 2014 16:31:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321076=3A_the_C_si?= =?utf-8?q?gnal_module_has_been_renamed_to_=5Fsignal?= Message-ID: <3g0k9b4kTxz7Ljw@mail.python.org> http://hg.python.org/cpython/rev/df5120efb86e changeset: 90137:df5120efb86e user: Victor Stinner date: Fri Apr 04 16:30:04 2014 +0200 summary: Issue #21076: the C signal module has been renamed to _signal files: Modules/Setup.config.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/Setup.config.in b/Modules/Setup.config.in --- a/Modules/Setup.config.in +++ b/Modules/Setup.config.in @@ -7,7 +7,7 @@ @USE_THREAD_MODULE at _thread _threadmodule.c # The signal module - at USE_SIGNAL_MODULE@signal signalmodule.c + at USE_SIGNAL_MODULE@_signal signalmodule.c # The rest of the modules previously listed in this file are built # by the setup.py script in Python 2.1 and later. -- Repository URL: http://hg.python.org/cpython From victor.stinner at gmail.com Fri Apr 4 16:33:37 2014 From: victor.stinner at gmail.com (Victor Stinner) Date: Fri, 4 Apr 2014 16:33:37 +0200 Subject: [Python-checkins] [Python-Dev] cpython: fix #21076: turn signal module constants into enums In-Reply-To: References: <3g0hvS5h5Wz7LjM@mail.python.org> Message-ID: 2014-04-04 16:21 GMT+02:00 Brett Cannon : > Fix is in rev c6e63bb132fb. Hum, this one was not enough for me. I also modified Modules/Setup.config.in: changeset: 90137:df5120efb86e tag: tip user: Victor Stinner date: Fri Apr 04 16:30:04 2014 +0200 files: Modules/Setup.config.in description: Issue #21076: the C signal module has been renamed to _signal diff -r c6e63bb132fb -r df5120efb86e Modules/Setup.config.in --- a/Modules/Setup.config.in Fri Apr 04 10:20:28 2014 -0400 +++ b/Modules/Setup.config.in Fri Apr 04 16:30:04 2014 +0200 @@ -7,7 +7,7 @@ @USE_THREAD_MODULE at _thread _threadmodule.c # The signal module - at USE_SIGNAL_MODULE@signal signalmodule.c + at USE_SIGNAL_MODULE@_signal signalmodule.c # The rest of the modules previously listed in this file are built # by the setup.py script in Python 2.1 and later. Victor From python-checkins at python.org Fri Apr 4 16:36:52 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 4 Apr 2014 16:36:52 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Finish_up_signal_-=3E_=5Fs?= =?utf-8?q?ignal_change?= Message-ID: <3g0kHS0481z7LpC@mail.python.org> http://hg.python.org/cpython/rev/4a57ab2499e1 changeset: 90138:4a57ab2499e1 user: Brett Cannon date: Fri Apr 04 10:36:43 2014 -0400 summary: Finish up signal -> _signal change files: PC/config.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PC/config.c b/PC/config.c --- a/PC/config.c +++ b/PC/config.c @@ -91,7 +91,7 @@ {"math", PyInit_math}, {"nt", PyInit_nt}, /* Use the NT os functions, not posix */ {"_operator", PyInit__operator}, - {"signal", PyInit_signal}, + {"_signal", PyInit__signal}, {"_md5", PyInit__md5}, {"_sha1", PyInit__sha1}, {"_sha256", PyInit__sha256}, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 17:00:15 2014 From: python-checkins at python.org (victor.stinner) Date: Fri, 4 Apr 2014 17:00:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321076=3A_sigpendi?= =?utf-8?q?ng=28=29_is_not_available_on_Windows?= Message-ID: <3g0kpR1cxZz7LjM@mail.python.org> http://hg.python.org/cpython/rev/b1f5b5d7997f changeset: 90139:b1f5b5d7997f user: Victor Stinner date: Fri Apr 04 16:57:24 2014 +0200 summary: Issue #21076: sigpending() is not available on Windows files: Lib/signal.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Lib/signal.py b/Lib/signal.py --- a/Lib/signal.py +++ b/Lib/signal.py @@ -68,10 +68,11 @@ pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__ - at _wraps(_signal.sigpending) -def sigpending(): - sigs = _signal.sigpending() - return set(_int_to_enum(x, Signals) for x in sigs) +if 'sigpending' in _globals: + @_wraps(_signal.sigpending) + def sigpending(): + sigs = _signal.sigpending() + return set(_int_to_enum(x, Signals) for x in sigs) if 'sigwait' in _globals: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 17:17:45 2014 From: python-checkins at python.org (giampaolo.rodola) Date: Fri, 4 Apr 2014 17:17:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_update_Misc/NEWS?= Message-ID: <3g0lBd4hwbz7Lnl@mail.python.org> http://hg.python.org/cpython/rev/f9a73831f9e7 changeset: 90140:f9a73831f9e7 user: Giampaolo Rodola' date: Fri Apr 04 17:17:36 2014 +0200 summary: update Misc/NEWS files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #21076: signal module constants were turned into enums. + Patch by Giampaolo Rodola'. + - Issue #20636: Improved the repr of Tkinter widgets. - Issue #19505: The items, keys, and values views of OrderedDict now support -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 19:53:47 2014 From: python-checkins at python.org (brett.cannon) Date: Fri, 4 Apr 2014 19:53:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317621=3A_Introduc?= =?utf-8?q?e_importlib=2Eutil=2ELazyLoader=2E?= Message-ID: <3g0pfg0LLGz7Lkk@mail.python.org> http://hg.python.org/cpython/rev/52b58618199c changeset: 90141:52b58618199c user: Brett Cannon date: Fri Apr 04 13:53:38 2014 -0400 summary: Issue #17621: Introduce importlib.util.LazyLoader. files: Doc/library/importlib.rst | 35 +++ Doc/whatsnew/3.5.rst | 4 + Lib/importlib/util.py | 94 ++++++++++- Lib/test/test_importlib/test_lazy.py | 132 +++++++++++++++ Misc/NEWS | 2 + 5 files changed, 266 insertions(+), 1 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -1191,3 +1191,38 @@ module will be file-based. .. versionadded:: 3.4 + +.. class:: LazyLoader(loader) + + A class which postpones the execution of the loader of a module until the + module has an attribute accessed. + + This class **only** works with loaders that define + :meth:`importlib.abc.Loader.exec_module` as control over what module type + is used for the module is required. For the same reasons, the loader + **cannot** define :meth:`importlib.abc.Loader.create_module`. Finally, + modules which substitute the object placed into :attr:`sys.modules` will + not work as there is no way to properly replace the module references + throughout the interpreter safely; :exc:`ValueError` is raised if such a + substitution is detected. + + .. note:: + For projects where startup time is critical, this class allows for + potentially minimizing the cost of loading a module if it is never used. + For projects where startup time is not essential then use of this class is + **heavily** discouraged due to error messages created during loading being + postponed and thus occurring out of context. + + .. versionadded:: 3.5 + + .. classmethod:: factory(loader) + + A static method which returns a callable that creates a lazy loader. This + is meant to be used in situations where the loader is passed by class + instead of by instance. + :: + + suffixes = importlib.machinery.SOURCE_SUFFIXES + loader = importlib.machinery.SourceFileLoader + lazy_loader = importlib.util.LazyLoader.factory(loader) + finder = importlib.machinery.FileFinder(path, [(lazy_loader, suffixes)]) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -149,6 +149,10 @@ subclassing of :class:`~inspect.Signature` easier (contributed by Yury Selivanov and Eric Snow in :issue:`17373`). +* :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in + applications where startup time is paramount (contributed by Brett Cannon in + :issue:`17621`). + Optimizations ============= diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py --- a/Lib/importlib/util.py +++ b/Lib/importlib/util.py @@ -1,5 +1,5 @@ """Utility code for constructing importers, etc.""" - +from . import abc from ._bootstrap import MAGIC_NUMBER from ._bootstrap import cache_from_source from ._bootstrap import decode_source @@ -12,6 +12,7 @@ from contextlib import contextmanager import functools import sys +import types import warnings @@ -200,3 +201,94 @@ return fxn(self, module, *args, **kwargs) return module_for_loader_wrapper + + +class _Module(types.ModuleType): + + """A subclass of the module type to allow __class__ manipulation.""" + + +class _LazyModule(types.ModuleType): + + """A subclass of the module type which triggers loading upon attribute access.""" + + def __getattribute__(self, attr): + """Trigger the load of the module and return the attribute.""" + # All module metadata must be garnered from __spec__ in order to avoid + # using mutated values. + # Stop triggering this method. + self.__class__ = _Module + # Get the original name to make sure no object substitution occurred + # in sys.modules. + original_name = self.__spec__.name + # Figure out exactly what attributes were mutated between the creation + # of the module and now. + attrs_then = self.__spec__.loader_state + attrs_now = self.__dict__ + attrs_updated = {} + for key, value in attrs_now.items(): + # Code that set the attribute may have kept a reference to the + # assigned object, making identity more important than equality. + if key not in attrs_then: + attrs_updated[key] = value + elif id(attrs_now[key]) != id(attrs_then[key]): + attrs_updated[key] = value + self.__spec__.loader.exec_module(self) + # If exec_module() was used directly there is no guarantee the module + # object was put into sys.modules. + if original_name in sys.modules: + if id(self) != id(sys.modules[original_name]): + msg = ('module object for {!r} substituted in sys.modules ' + 'during a lazy load') + raise ValueError(msg.format(original_name)) + # Update after loading since that's what would happen in an eager + # loading situation. + self.__dict__.update(attrs_updated) + return getattr(self, attr) + + def __delattr__(self, attr): + """Trigger the load and then perform the deletion.""" + # To trigger the load and raise an exception if the attribute + # doesn't exist. + self.__getattribute__(attr) + delattr(self, attr) + + +class LazyLoader(abc.Loader): + + """A loader that creates a module which defers loading until attribute access.""" + + @staticmethod + def __check_eager_loader(loader): + if not hasattr(loader, 'exec_module'): + raise TypeError('loader must define exec_module()') + elif hasattr(loader.__class__, 'create_module'): + if abc.Loader.create_module != loader.__class__.create_module: + # Only care if create_module() is overridden in a subclass of + # importlib.abc.Loader. + raise TypeError('loader cannot define create_module()') + + @classmethod + def factory(cls, loader): + """Construct a callable which returns the eager loader made lazy.""" + cls.__check_eager_loader(loader) + return lambda *args, **kwargs: cls(loader(*args, **kwargs)) + + def __init__(self, loader): + self.__check_eager_loader(loader) + self.loader = loader + + def create_module(self, spec): + """Create a module which can have its __class__ manipulated.""" + return _Module(spec.name) + + def exec_module(self, module): + """Make the module load lazily.""" + module.__spec__.loader = self.loader + module.__loader__ = self.loader + # Don't need to worry about deep-copying as trying to set an attribute + # on an object would have triggered the load, + # e.g. ``module.__spec__.loader = None`` would trigger a load from + # trying to access module.__spec__. + module.__spec__.loader_state = module.__dict__.copy() + module.__class__ = _LazyModule diff --git a/Lib/test/test_importlib/test_lazy.py b/Lib/test/test_importlib/test_lazy.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_importlib/test_lazy.py @@ -0,0 +1,132 @@ +import importlib +from importlib import abc +from importlib import util +import unittest + +from . import util as test_util + + +class CollectInit: + + def __init__(self, *args, **kwargs): + self.args = args + self.kwargs = kwargs + + def exec_module(self, module): + return self + + +class LazyLoaderFactoryTests(unittest.TestCase): + + def test_init(self): + factory = util.LazyLoader.factory(CollectInit) + # E.g. what importlib.machinery.FileFinder instantiates loaders with + # plus keyword arguments. + lazy_loader = factory('module name', 'module path', kw='kw') + loader = lazy_loader.loader + self.assertEqual(('module name', 'module path'), loader.args) + self.assertEqual({'kw': 'kw'}, loader.kwargs) + + def test_validation(self): + # No exec_module(), no lazy loading. + with self.assertRaises(TypeError): + util.LazyLoader.factory(object) + + +class TestingImporter(abc.MetaPathFinder, abc.Loader): + + module_name = 'lazy_loader_test' + mutated_name = 'changed' + loaded = None + source_code = 'attr = 42; __name__ = {!r}'.format(mutated_name) + + def find_spec(self, name, path, target=None): + if name != self.module_name: + return None + return util.spec_from_loader(name, util.LazyLoader(self)) + + def exec_module(self, module): + exec(self.source_code, module.__dict__) + self.loaded = module + + +class LazyLoaderTests(unittest.TestCase): + + def test_init(self): + with self.assertRaises(TypeError): + util.LazyLoader(object) + + def new_module(self, source_code=None): + loader = TestingImporter() + if source_code is not None: + loader.source_code = source_code + spec = util.spec_from_loader(TestingImporter.module_name, + util.LazyLoader(loader)) + module = spec.loader.create_module(spec) + module.__spec__ = spec + module.__loader__ = spec.loader + spec.loader.exec_module(module) + # Module is now lazy. + self.assertIsNone(loader.loaded) + return module + + def test_e2e(self): + # End-to-end test to verify the load is in fact lazy. + importer = TestingImporter() + assert importer.loaded is None + with test_util.uncache(importer.module_name): + with test_util.import_state(meta_path=[importer]): + module = importlib.import_module(importer.module_name) + self.assertIsNone(importer.loaded) + # Trigger load. + self.assertEqual(module.__loader__, importer) + self.assertIsNotNone(importer.loaded) + self.assertEqual(module, importer.loaded) + + def test_attr_unchanged(self): + # An attribute only mutated as a side-effect of import should not be + # changed needlessly. + module = self.new_module() + self.assertEqual(TestingImporter.mutated_name, module.__name__) + + def test_new_attr(self): + # A new attribute should persist. + module = self.new_module() + module.new_attr = 42 + self.assertEqual(42, module.new_attr) + + def test_mutated_preexisting_attr(self): + # Changing an attribute that already existed on the module -- + # e.g. __name__ -- should persist. + module = self.new_module() + module.__name__ = 'bogus' + self.assertEqual('bogus', module.__name__) + + def test_mutated_attr(self): + # Changing an attribute that comes into existence after an import + # should persist. + module = self.new_module() + module.attr = 6 + self.assertEqual(6, module.attr) + + def test_delete_eventual_attr(self): + # Deleting an attribute should stay deleted. + module = self.new_module() + del module.attr + self.assertFalse(hasattr(module, 'attr')) + + def test_delete_preexisting_attr(self): + module = self.new_module() + del module.__name__ + self.assertFalse(hasattr(module, '__name__')) + + def test_module_substitution_error(self): + source_code = 'import sys; sys.modules[__name__] = 42' + module = self.new_module(source_code) + with test_util.uncache(TestingImporter.module_name): + with self.assertRaises(ValueError): + module.__name__ + + +if __name__ == '__main__': + unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #17621: Introduce importlib.util.LazyLoader. + - Issue #21076: signal module constants were turned into enums. Patch by Giampaolo Rodola'. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 20:00:06 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 20:00:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_make_temporary?= =?utf-8?q?_read-only_files_writable=2C_so_rmtree_can_remove_them_=28=2321?= =?utf-8?q?128=29?= Message-ID: <3g0pny5vLsz7Llg@mail.python.org> http://hg.python.org/cpython/rev/b56341a49eab changeset: 90142:b56341a49eab branch: 2.7 parent: 90133:6809b434752a user: Benjamin Peterson date: Fri Apr 04 13:55:56 2014 -0400 summary: make temporary read-only files writable, so rmtree can remove them (#21128) files: Lib/test/test_argparse.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -47,6 +47,9 @@ def tearDown(self): os.chdir(self.old_dir) + for root, dirs, files in os.walk(self.temp_dir, topdown=False): + for name in files: + os.chmod(os.path.join(self.temp_dir, name), stat.S_IWRITE) shutil.rmtree(self.temp_dir, True) def create_readonly_file(self, filename): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 20:00:08 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 20:00:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_properly_expli?= =?utf-8?q?citly_close_file_=28=2321128=29?= Message-ID: <3g0pp00Tvjz7Lnp@mail.python.org> http://hg.python.org/cpython/rev/6f1ac58207cc changeset: 90143:6f1ac58207cc branch: 2.7 user: Benjamin Peterson date: Fri Apr 04 13:56:26 2014 -0400 summary: properly explicitly close file (#21128) files: Lib/test/test_file.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_file.py b/Lib/test/test_file.py --- a/Lib/test/test_file.py +++ b/Lib/test/test_file.py @@ -300,6 +300,7 @@ self.fail("readlines() after next() with empty buffer " "failed. Got %r, expected %r" % (line, testline)) # Reading after iteration hit EOF shouldn't hurt either + f.close() f = self.open(TESTFN, 'rb') try: for line in f: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 20:00:09 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 20:00:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_explicitly_clo?= =?utf-8?q?se_file_object_=28=2321128=29?= Message-ID: <3g0pp15CjXz7Lnp@mail.python.org> http://hg.python.org/cpython/rev/d7a37a1f2ca9 changeset: 90144:d7a37a1f2ca9 branch: 2.7 user: Benjamin Peterson date: Fri Apr 04 13:56:44 2014 -0400 summary: explicitly close file object (#21128) files: Lib/test/test_httpservers.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py --- a/Lib/test/test_httpservers.py +++ b/Lib/test/test_httpservers.py @@ -335,6 +335,7 @@ response = self.request(self.tempdir_name + '/') self.check_status_and_reason(response, 404) os.chmod(self.tempdir, 0755) + f.close() def test_head(self): response = self.request( -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 20:00:10 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 20:00:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_make_temporary?= =?utf-8?q?_read-only_files_writable=2C_so_rmtree_can_remove_them_=28=2321?= =?utf-8?q?128=29?= Message-ID: <3g0pp26c02z7LpK@mail.python.org> http://hg.python.org/cpython/rev/9fd33a504b58 changeset: 90145:9fd33a504b58 branch: 3.4 parent: 90134:fef890bd60b1 user: Benjamin Peterson date: Fri Apr 04 13:55:56 2014 -0400 summary: make temporary read-only files writable, so rmtree can remove them (#21128) files: Lib/test/test_argparse.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -47,6 +47,9 @@ def tearDown(self): os.chdir(self.old_dir) + for root, dirs, files in os.walk(self.temp_dir, topdown=False): + for name in files: + os.chmod(os.path.join(self.temp_dir, name), stat.S_IWRITE) shutil.rmtree(self.temp_dir, True) def create_readonly_file(self, filename): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 20:00:12 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 20:00:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_use_with_state?= =?utf-8?q?ment?= Message-ID: <3g0pp41CRlz7LpF@mail.python.org> http://hg.python.org/cpython/rev/87c150c43ca8 changeset: 90146:87c150c43ca8 branch: 2.7 parent: 90144:d7a37a1f2ca9 user: Benjamin Peterson date: Fri Apr 04 13:59:33 2014 -0400 summary: use with statement files: Lib/test/test_httpservers.py | 22 ++++++++++------------ 1 files changed, 10 insertions(+), 12 deletions(-) diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py --- a/Lib/test/test_httpservers.py +++ b/Lib/test/test_httpservers.py @@ -324,18 +324,16 @@ self.check_status_and_reason(response, 404) response = self.request('/' + 'ThisDoesNotExist' + '/') self.check_status_and_reason(response, 404) - f = open(os.path.join(self.tempdir_name, 'index.html'), 'w') - response = self.request('/' + self.tempdir_name + '/') - self.check_status_and_reason(response, 200) - - # chmod() doesn't work as expected on Windows, and filesystem - # permissions are ignored by root on Unix. - if os.name == 'posix' and os.geteuid() != 0: - os.chmod(self.tempdir, 0) - response = self.request(self.tempdir_name + '/') - self.check_status_and_reason(response, 404) - os.chmod(self.tempdir, 0755) - f.close() + with open(os.path.join(self.tempdir_name, 'index.html'), 'w') as fp: + response = self.request('/' + self.tempdir_name + '/') + self.check_status_and_reason(response, 200) + # chmod() doesn't work as expected on Windows, and filesystem + # permissions are ignored by root on Unix. + if os.name == 'posix' and os.geteuid() != 0: + os.chmod(self.tempdir, 0) + response = self.request(self.tempdir_name + '/') + self.check_status_and_reason(response, 404) + os.chmod(self.tempdir, 0755) def test_head(self): response = self.request( -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 20:00:13 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 4 Apr 2014 20:00:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjExMjgp?= Message-ID: <3g0pp5350mz7LpL@mail.python.org> http://hg.python.org/cpython/rev/78e75181d87f changeset: 90147:78e75181d87f parent: 90141:52b58618199c parent: 90145:9fd33a504b58 user: Benjamin Peterson date: Fri Apr 04 13:59:46 2014 -0400 summary: merge 3.4 (#21128) files: Lib/test/test_argparse.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -47,6 +47,9 @@ def tearDown(self): os.chdir(self.old_dir) + for root, dirs, files in os.walk(self.temp_dir, topdown=False): + for name in files: + os.chmod(os.path.join(self.temp_dir, name), stat.S_IWRITE) shutil.rmtree(self.temp_dir, True) def create_readonly_file(self, filename): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 21:02:53 2014 From: python-checkins at python.org (victor.stinner) Date: Fri, 4 Apr 2014 21:02:53 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_Fix_=5FP?= =?utf-8?q?yUnicodeTranslateError=5FCreate=28=29=2C_add_missing_format?= Message-ID: <3g0rBP5c4Rz7LkW@mail.python.org> http://hg.python.org/cpython/rev/95d4e8124c6a changeset: 90148:95d4e8124c6a user: Victor Stinner date: Fri Apr 04 20:59:44 2014 +0200 summary: Issue #21118: Fix _PyUnicodeTranslateError_Create(), add missing format character for the "end" parameter files: Objects/exceptions.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2124,7 +2124,7 @@ PyObject *object, Py_ssize_t start, Py_ssize_t end, const char *reason) { - return PyObject_CallFunction(PyExc_UnicodeTranslateError, "Ons", + return PyObject_CallFunction(PyExc_UnicodeTranslateError, "Onns", object, start, end, reason); } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 21:02:55 2014 From: python-checkins at python.org (victor.stinner) Date: Fri, 4 Apr 2014 21:02:55 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_Use_=5FP?= =?utf-8?q?yUnicodeWriter_API_in_str=2Etranslate=28=29_to_simplify_and?= Message-ID: <3g0rBR1XSYz7Lmj@mail.python.org> http://hg.python.org/cpython/rev/03b1dd29b327 changeset: 90149:03b1dd29b327 user: Victor Stinner date: Fri Apr 04 19:37:40 2014 +0200 summary: Issue #21118: Use _PyUnicodeWriter API in str.translate() to simplify and factorize the code files: Objects/unicodeobject.c | 292 ++++++++++----------------- 1 files changed, 108 insertions(+), 184 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8495,76 +8495,54 @@ return -1; } } -/* ensure that *outobj is at least requiredsize characters long, - if not reallocate and adjust various state variables. - Return 0 on success, -1 on error */ + +/* lookup the character, write the result into the writer. + Return 1 if the result was written into the writer, return 0 if the mapping + was undefined, raise an exception return -1 on error. */ static int -charmaptranslate_makespace(Py_UCS4 **outobj, Py_ssize_t *psize, - Py_ssize_t requiredsize) -{ - Py_ssize_t oldsize = *psize; - Py_UCS4 *new_outobj; - if (requiredsize > oldsize) { - /* exponentially overallocate to minimize reallocations */ - if (requiredsize < 2 * oldsize) - requiredsize = 2 * oldsize; - new_outobj = PyMem_Realloc(*outobj, requiredsize * sizeof(Py_UCS4)); - if (new_outobj == 0) +charmaptranslate_output(Py_UCS4 ch, PyObject *mapping, + _PyUnicodeWriter *writer) +{ + PyObject *item; + + if (charmaptranslate_lookup(ch, mapping, &item)) + return -1; + + if (item == NULL) { + /* not found => default to 1:1 mapping */ + if (_PyUnicodeWriter_WriteCharInline(writer, ch) < 0) { return -1; - *outobj = new_outobj; - *psize = requiredsize; - } - return 0; -} -/* lookup the character, put the result in the output string and adjust - various state variables. Return a new reference to the object that - was put in the output buffer in *result, or Py_None, if the mapping was - undefined (in which case no character was written). - The called must decref result. - Return 0 on success, -1 on error. */ -static int -charmaptranslate_output(PyObject *input, Py_ssize_t ipos, - PyObject *mapping, Py_UCS4 **output, - Py_ssize_t *osize, Py_ssize_t *opos, - PyObject **res) -{ - Py_UCS4 curinp = PyUnicode_READ_CHAR(input, ipos); - if (charmaptranslate_lookup(curinp, mapping, res)) - return -1; - if (*res==NULL) { - /* not found => default to 1:1 mapping */ - (*output)[(*opos)++] = curinp; - } - else if (*res==Py_None) - ; - else if (PyLong_Check(*res)) { - /* no overflow check, because we know that the space is enough */ - (*output)[(*opos)++] = (Py_UCS4)PyLong_AS_LONG(*res); - } - else if (PyUnicode_Check(*res)) { - Py_ssize_t repsize; - if (PyUnicode_READY(*res) == -1) + } + return 1; + } + + if (item == Py_None) { + Py_DECREF(item); + return 0; + } + + if (PyLong_Check(item)) { + Py_UCS4 ch = (Py_UCS4)PyLong_AS_LONG(item); + if (_PyUnicodeWriter_WriteCharInline(writer, ch) < 0) { + Py_DECREF(item); return -1; - repsize = PyUnicode_GET_LENGTH(*res); - if (repsize==1) { - /* no overflow check, because we know that the space is enough */ - (*output)[(*opos)++] = PyUnicode_READ_CHAR(*res, 0); - } - else if (repsize!=0) { - /* more than one character */ - Py_ssize_t requiredsize = *opos + - (PyUnicode_GET_LENGTH(input) - ipos) + - repsize - 1; - Py_ssize_t i; - if (charmaptranslate_makespace(output, osize, requiredsize)) - return -1; - for(i = 0; i < repsize; i++) - (*output)[(*opos)++] = PyUnicode_READ_CHAR(*res, i); - } - } - else - return -1; - return 0; + } + Py_DECREF(item); + return 1; + } + + if (!PyUnicode_Check(item)) { + Py_DECREF(item); + return -1; + } + + if (_PyUnicodeWriter_WriteStr(writer, item) < 0) { + Py_DECREF(item); + return -1; + } + + Py_DECREF(item); + return 1; } PyObject * @@ -8573,22 +8551,16 @@ const char *errors) { /* input object */ - char *idata; + char *data; Py_ssize_t size, i; int kind; /* output buffer */ - Py_UCS4 *output = NULL; - Py_ssize_t osize; - PyObject *res; - /* current output position */ - Py_ssize_t opos; + _PyUnicodeWriter writer; + /* error handler */ char *reason = "character maps to "; PyObject *errorHandler = NULL; PyObject *exc = NULL; - /* the following variable is used for caching string comparisons - * -1=not initialized, 0=unknown, 1=strict, 2=replace, - * 3=ignore, 4=xmlcharrefreplace */ - int known_errorHandler = -1; + int ignore; if (mapping == NULL) { PyErr_BadArgument(); @@ -8597,10 +8569,9 @@ if (PyUnicode_READY(input) == -1) return NULL; - idata = (char*)PyUnicode_DATA(input); + data = (char*)PyUnicode_DATA(input); kind = PyUnicode_KIND(input); size = PyUnicode_GET_LENGTH(input); - i = 0; if (size == 0) { Py_INCREF(input); @@ -8609,121 +8580,74 @@ /* allocate enough for a simple 1:1 translation without replacements, if we need more, we'll resize */ - osize = size; - output = PyMem_Malloc(osize * sizeof(Py_UCS4)); - opos = 0; - if (output == NULL) { - PyErr_NoMemory(); + _PyUnicodeWriter_Init(&writer); + if (_PyUnicodeWriter_Prepare(&writer, size, 127) == -1) goto onError; - } - + + ignore = (errors != NULL && strcmp(errors, "ignore") == 0); + + i = 0; while (i adjust input pointer */ + ++i; + continue; + } + + /* untranslatable character */ + collstart = i; + collend = i+1; + + /* find all untranslatable characters */ + while (collend < size) { + PyObject *x; + ch = PyUnicode_READ(kind, data, collend); + if (charmaptranslate_lookup(ch, mapping, &x)) + goto onError; Py_XDECREF(x); - goto onError; - } - Py_XDECREF(x); - if (x!=Py_None) /* it worked => adjust input pointer */ - ++i; - else { /* untranslatable character */ - PyObject *repunicode = NULL; /* initialize to prevent gcc warning */ - Py_ssize_t repsize; - Py_ssize_t newpos; - Py_ssize_t uni2; - /* startpos for collecting untranslatable chars */ - Py_ssize_t collstart = i; - Py_ssize_t collend = i+1; - Py_ssize_t coll; - - /* find all untranslatable characters */ - while (collend < size) { - if (charmaptranslate_lookup(PyUnicode_READ(kind,idata, collend), mapping, &x)) - goto onError; - Py_XDECREF(x); - if (x!=Py_None) - break; - ++collend; - } - /* cache callback name lookup - * (if not done yet, i.e. it's the first error) */ - if (known_errorHandler==-1) { - if ((errors==NULL) || (!strcmp(errors, "strict"))) - known_errorHandler = 1; - else if (!strcmp(errors, "replace")) - known_errorHandler = 2; - else if (!strcmp(errors, "ignore")) - known_errorHandler = 3; - else if (!strcmp(errors, "xmlcharrefreplace")) - known_errorHandler = 4; - else - known_errorHandler = 0; - } - switch (known_errorHandler) { - case 1: /* strict */ - make_translate_exception(&exc, - input, collstart, collend, reason); - if (exc != NULL) - PyCodec_StrictErrors(exc); + if (x != Py_None) + break; + ++collend; + } + + if (ignore) { + i = collend; + } + else { + repunicode = unicode_translate_call_errorhandler(errors, &errorHandler, + reason, input, &exc, + collstart, collend, &newpos); + if (repunicode == NULL) goto onError; - case 2: /* replace */ - /* No need to check for space, this is a 1:1 replacement */ - for (coll = collstart; coll0; ++uni2) - output[opos++] = PyUnicode_READ_CHAR(repunicode, uni2); - i = newpos; + if (_PyUnicodeWriter_WriteStr(&writer, repunicode) < 0) { Py_DECREF(repunicode); - } - } - } - res = PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, output, opos); - if (!res) - goto onError; - PyMem_Free(output); + goto onError; + } + Py_DECREF(repunicode); + i = newpos; + } + } Py_XDECREF(exc); Py_XDECREF(errorHandler); - return res; + return _PyUnicodeWriter_Finish(&writer); onError: - PyMem_Free(output); + _PyUnicodeWriter_Dealloc(&writer); Py_XDECREF(exc); Py_XDECREF(errorHandler); return NULL; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 21:06:01 2014 From: python-checkins at python.org (victor.stinner) Date: Fri, 4 Apr 2014 21:06:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTE4?= =?utf-8?q?=3A_Fix_=5FPyUnicodeTranslateError=5FCreate=28=29=2C_add_missin?= =?utf-8?q?g_format?= Message-ID: <3g0rG126HPz7Llg@mail.python.org> http://hg.python.org/cpython/rev/70990e795657 changeset: 90150:70990e795657 branch: 3.4 parent: 90145:9fd33a504b58 user: Victor Stinner date: Fri Apr 04 20:59:44 2014 +0200 summary: Issue #21118: Fix _PyUnicodeTranslateError_Create(), add missing format character for the "end" parameter files: Objects/exceptions.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2124,7 +2124,7 @@ PyObject *object, Py_ssize_t start, Py_ssize_t end, const char *reason) { - return PyObject_CallFunction(PyExc_UnicodeTranslateError, "Ons", + return PyObject_CallFunction(PyExc_UnicodeTranslateError, "Onns", object, start, end, reason); } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 4 21:06:02 2014 From: python-checkins at python.org (victor.stinner) Date: Fri, 4 Apr 2014 21:06:02 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Null_merge_with_Python_3=2E4?= Message-ID: <3g0rG23rKPz7Ln9@mail.python.org> http://hg.python.org/cpython/rev/087cdbf49e80 changeset: 90151:087cdbf49e80 parent: 90149:03b1dd29b327 parent: 90150:70990e795657 user: Victor Stinner date: Fri Apr 04 21:05:46 2014 +0200 summary: Null merge with Python 3.4 files: -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Apr 5 10:05:57 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 05 Apr 2014 10:05:57 +0200 Subject: [Python-checkins] Daily reference leaks (087cdbf49e80): sum=7 Message-ID: results for 087cdbf49e80 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogv_NK1B', '-x'] From python-checkins at python.org Sat Apr 5 10:29:17 2014 From: python-checkins at python.org (mark.dickinson) Date: Sat, 5 Apr 2014 10:29:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321136=3A_Avoid_un?= =?utf-8?q?necessary_normalization_in_Fractions_resulting_from_power?= Message-ID: <3g1B4s1T4pz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/91d7fadac271 changeset: 90152:91d7fadac271 user: Mark Dickinson date: Sat Apr 05 09:29:00 2014 +0100 summary: Issue #21136: Avoid unnecessary normalization in Fractions resulting from power and other operations. files: Lib/fractions.py | 23 ++++++++++++++--------- Lib/test/test_fractions.py | 3 ++- Misc/NEWS | 3 +++ 3 files changed, 19 insertions(+), 10 deletions(-) diff --git a/Lib/fractions.py b/Lib/fractions.py --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -70,7 +70,7 @@ __slots__ = ('_numerator', '_denominator') # We're immutable, so use __new__ not __init__ - def __new__(cls, numerator=0, denominator=None): + def __new__(cls, numerator=0, denominator=None, _normalize=True): """Constructs a Rational. Takes a string like '3/2' or '1.5', another Rational instance, a @@ -165,9 +165,12 @@ if denominator == 0: raise ZeroDivisionError('Fraction(%s, 0)' % numerator) - g = gcd(numerator, denominator) - self._numerator = numerator // g - self._denominator = denominator // g + if _normalize: + g = gcd(numerator, denominator) + numerator //= g + denominator //= g + self._numerator = numerator + self._denominator = denominator return self @classmethod @@ -453,10 +456,12 @@ power = b.numerator if power >= 0: return Fraction(a._numerator ** power, - a._denominator ** power) + a._denominator ** power, + _normalize=False) else: return Fraction(a._denominator ** -power, - a._numerator ** -power) + a._numerator ** -power, + _normalize=False) else: # A fractional power will generally produce an # irrational number. @@ -480,15 +485,15 @@ def __pos__(a): """+a: Coerces a subclass instance to Fraction""" - return Fraction(a._numerator, a._denominator) + return Fraction(a._numerator, a._denominator, _normalize=False) def __neg__(a): """-a""" - return Fraction(-a._numerator, a._denominator) + return Fraction(-a._numerator, a._denominator, _normalize=False) def __abs__(a): """abs(a)""" - return Fraction(abs(a._numerator), a._denominator) + return Fraction(abs(a._numerator), a._denominator, _normalize=False) def __trunc__(a): """trunc(a)""" diff --git a/Lib/test/test_fractions.py b/Lib/test/test_fractions.py --- a/Lib/test/test_fractions.py +++ b/Lib/test/test_fractions.py @@ -330,7 +330,6 @@ self.assertTypedEquals(F(-2, 10), round(F(-15, 100), 1)) self.assertTypedEquals(F(-2, 10), round(F(-25, 100), 1)) - def testArithmetic(self): self.assertEqual(F(1, 2), F(1, 10) + F(2, 5)) self.assertEqual(F(-3, 10), F(1, 10) - F(2, 5)) @@ -402,6 +401,8 @@ self.assertTypedEquals(2.0 , 4 ** F(1, 2)) self.assertTypedEquals(0.25, 2.0 ** F(-2, 1)) self.assertTypedEquals(1.0 + 0j, (1.0 + 0j) ** F(1, 10)) + self.assertRaises(ZeroDivisionError, operator.pow, + F(0, 1), -2) def testMixingWithDecimal(self): # Decimal refuses mixed arithmetic (but not mixed comparisons) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #21136: Avoid unnecessary normalization of Fractions resulting from + power and other operations. Patch by Raymond Hettinger. + - Issue #17621: Introduce importlib.util.LazyLoader. - Issue #21076: signal module constants were turned into enums. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 5 11:42:33 2014 From: python-checkins at python.org (victor.stinner) Date: Sat, 5 Apr 2014 11:42:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_Remove_u?= =?utf-8?q?nused_variable?= Message-ID: <3g1CjP0kMXz7Llq@mail.python.org> http://hg.python.org/cpython/rev/9acc8196a82c changeset: 90153:9acc8196a82c user: Victor Stinner date: Sat Apr 05 00:15:52 2014 +0200 summary: Issue #21118: Remove unused variable files: Objects/unicodeobject.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8595,7 +8595,6 @@ /* startpos for collecting untranslatable chars */ Py_ssize_t collstart; Py_ssize_t collend; - Py_ssize_t coll; Py_UCS4 ch; ch = PyUnicode_READ(kind, data, i); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 5 11:42:34 2014 From: python-checkins at python.org (victor.stinner) Date: Sat, 5 Apr 2014 11:42:34 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_Add_more?= =?utf-8?q?_unit_tests_on_str=2Etranslate=28=29?= Message-ID: <3g1CjQ24ltz7Ll6@mail.python.org> http://hg.python.org/cpython/rev/bd594dd71d46 changeset: 90154:bd594dd71d46 user: Victor Stinner date: Sat Apr 05 00:17:51 2014 +0200 summary: Issue #21118: Add more unit tests on str.translate() files: Lib/test/test_unicode.py | 20 ++++++++++++++++++++ 1 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -251,6 +251,7 @@ {ord('a'): None, ord('b'): ''}) self.checkequalnofix('xyyx', 'xzx', 'translate', {ord('z'): 'yy'}) + # this needs maketrans() self.checkequalnofix('abababc', 'abababc', 'translate', {'b': ''}) @@ -260,6 +261,25 @@ tbl = self.type2test.maketrans('abc', 'xyz', 'd') self.checkequalnofix('xyzzy', 'abdcdcbdddd', 'translate', tbl) + # various tests switching from ASCII to latin1 or the opposite; + # same length, remove a letter, or replace with a longer string. + self.assertEqual("[a]".translate(str.maketrans('a', 'X')), + "[X]") + self.assertEqual("[a]".translate(str.maketrans({'a': 'X'})), + "[X]") + self.assertEqual("[a]".translate(str.maketrans({'a': None})), + "[]") + self.assertEqual("[a]".translate(str.maketrans({'a': 'XXX'})), + "[XXX]") + self.assertEqual("[a]".translate(str.maketrans({'a': '\xe9'})), + "[\xe9]") + self.assertEqual("[a]".translate(str.maketrans({'a': '<\xe9>'})), + "[<\xe9>]") + self.assertEqual("[\xe9]".translate(str.maketrans({'\xe9': 'a'})), + "[a]") + self.assertEqual("[\xe9]".translate(str.maketrans({'\xe9': None})), + "[]") + self.assertRaises(TypeError, self.type2test.maketrans) self.assertRaises(ValueError, self.type2test.maketrans, 'abc', 'defg') self.assertRaises(TypeError, self.type2test.maketrans, 2, 'def') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 5 11:56:52 2014 From: python-checkins at python.org (victor.stinner) Date: Sat, 5 Apr 2014 11:56:52 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_Optimize?= =?utf-8?q?_str=2Etranslate=28=29_for_ASCII_=3D=3E_ASCII_translation?= Message-ID: <3g1D1w3Rwrz7LjM@mail.python.org> http://hg.python.org/cpython/rev/cca6b056236a changeset: 90155:cca6b056236a user: Victor Stinner date: Sat Apr 05 11:44:04 2014 +0200 summary: Issue #21118: Optimize str.translate() for ASCII => ASCII translation files: Objects/unicodeobject.c | 121 +++++++++++++++++++++++++++- 1 files changed, 120 insertions(+), 1 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8545,6 +8545,116 @@ return 1; } +static int +unicode_fast_translate_lookup(PyObject *mapping, Py_UCS1 ch, + Py_UCS1 *translate) +{ + PyObject *item; + int ret = 0; + + item = NULL; + if (charmaptranslate_lookup(ch, mapping, &item)) { + return -1; + } + + if (item == Py_None) { + /* deletion: skip fast translate */ + goto exit; + } + + if (item == NULL) { + /* not found => default to 1:1 mapping */ + translate[ch] = ch; + return 1; + } + + if (PyLong_Check(item)) { + long replace = (Py_UCS4)PyLong_AS_LONG(item); + if (replace == -1) { + Py_DECREF(item); + return -1; + } + if (replace < 0 || 127 < replace) { + /* invalid character or character outside ASCII: + skip the fast translate */ + goto exit; + } + translate[ch] = (Py_UCS1)replace; + } + else if (PyUnicode_Check(item)) { + Py_UCS4 replace; + + if (PyUnicode_READY(item) == -1) { + Py_DECREF(item); + return -1; + } + if (PyUnicode_GET_LENGTH(item) != 1) + goto exit; + + replace = PyUnicode_READ_CHAR(item, 0); + if (replace > 127) + goto exit; + translate[ch] = (Py_UCS1)replace; + } + else { + /* not a long or unicode */ + goto exit; + } + Py_DECREF(item); + item = NULL; + ret = 1; + +exit: + Py_XDECREF(item); + return ret; +} + +/* Fast path for ascii => ascii translation. Return 1 if the whole string + was translated into writer, return 0 if the input string was partially + translated into writer, raise an exception and return -1 on error. */ +static int +unicode_fast_translate(PyObject *input, PyObject *mapping, + _PyUnicodeWriter *writer) +{ + Py_UCS1 translate[128], ch, ch2; + Py_ssize_t len; + Py_UCS1 *in, *end, *out; + int res; + + if (PyUnicode_READY(input) == -1) + return -1; + if (!PyUnicode_IS_ASCII(input)) + return 0; + len = PyUnicode_GET_LENGTH(input); + + memset(translate, 0xff, 128); + + in = PyUnicode_1BYTE_DATA(input); + end = in + len; + + assert(PyUnicode_IS_ASCII(writer->buffer)); + assert(PyUnicode_GET_LENGTH(writer->buffer) == len); + out = PyUnicode_1BYTE_DATA(writer->buffer); + + for (; in < end; in++, out++) { + ch = *in; + ch2 = translate[ch]; + if (ch2 == 0xff) { + res = unicode_fast_translate_lookup(mapping, ch, translate); + if (res < 0) + return -1; + if (res == 0) { + writer->pos = in - PyUnicode_1BYTE_DATA(input); + return 0; + } + ch2 = translate[ch]; + } + *out = ch2; + } + writer->pos = len; + return 1; +} + PyObject * _PyUnicode_TranslateCharmap(PyObject *input, PyObject *mapping, @@ -8561,6 +8671,7 @@ PyObject *errorHandler = NULL; PyObject *exc = NULL; int ignore; + int res; if (mapping == NULL) { PyErr_BadArgument(); @@ -8584,9 +8695,17 @@ if (_PyUnicodeWriter_Prepare(&writer, size, 127) == -1) goto onError; + res = unicode_fast_translate(input, mapping, &writer); + if (res < 0) { + _PyUnicodeWriter_Dealloc(&writer); + return NULL; + } + if (res == 1) + return _PyUnicodeWriter_Finish(&writer); + ignore = (errors != NULL && strcmp(errors, "ignore") == 0); - i = 0; + i = writer.pos; while (i http://hg.python.org/cpython/rev/6a347c0ffbfc changeset: 90156:6a347c0ffbfc user: Victor Stinner date: Sat Apr 05 11:56:37 2014 +0200 summary: Issue #21118: Add unit test for invalid character replacement (code point higher than U+10ffff) files: Lib/test/test_unicode.py | 8 ++++++++ Objects/unicodeobject.c | 20 ++++++++++---------- 2 files changed, 18 insertions(+), 10 deletions(-) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -280,6 +280,14 @@ self.assertEqual("[\xe9]".translate(str.maketrans({'\xe9': None})), "[]") + # invalid Unicode characters + invalid_char = 0x10ffff+1 + for before in "a\xe9\u20ac\U0010ffff": + mapping = str.maketrans({before: invalid_char}) + text = "[%s]" % before + self.assertRaises(ValueError, text.translate, mapping) + + # errors self.assertRaises(TypeError, self.type2test.maketrans) self.assertRaises(ValueError, self.type2test.maketrans, 'abc', 'defg') self.assertRaises(TypeError, self.type2test.maketrans, 2, 'def') diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8473,10 +8473,10 @@ } else if (PyLong_Check(x)) { long value = PyLong_AS_LONG(x); - long max = PyUnicode_GetMax(); - if (value < 0 || value > max) { - PyErr_Format(PyExc_TypeError, - "character mapping must be in range(0x%x)", max+1); + if (value < 0 || value > MAX_UNICODE) { + PyErr_Format(PyExc_ValueError, + "character mapping must be in range(0x%x)", + MAX_UNICODE+1); Py_DECREF(x); return -1; } @@ -8522,7 +8522,9 @@ } if (PyLong_Check(item)) { - Py_UCS4 ch = (Py_UCS4)PyLong_AS_LONG(item); + long ch = (Py_UCS4)PyLong_AS_LONG(item); + /* PyLong_AS_LONG() cannot fail, charmaptranslate_lookup() already + used it */ if (_PyUnicodeWriter_WriteCharInline(writer, ch) < 0) { Py_DECREF(item); return -1; @@ -8570,11 +8572,9 @@ if (PyLong_Check(item)) { long replace = (Py_UCS4)PyLong_AS_LONG(item); - if (replace == -1) { - Py_DECREF(item); - return -1; - } - if (replace < 0 || 127 < replace) { + /* PyLong_AS_LONG() cannot fail, charmaptranslate_lookup() already + used it */ + if (127 < replace) { /* invalid character or character outside ASCII: skip the fast translate */ goto exit; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 5 14:27:30 2014 From: python-checkins at python.org (victor.stinner) Date: Sat, 5 Apr 2014 14:27:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_Optimize?= =?utf-8?q?_also_str=2Etranslate=28=29_for_ASCII_=3D=3E_ASCII_deletion?= Message-ID: <3g1HMk0yH9z7LkP@mail.python.org> http://hg.python.org/cpython/rev/47b0c076e17d changeset: 90157:47b0c076e17d user: Victor Stinner date: Sat Apr 05 14:27:07 2014 +0200 summary: Issue #21118: Optimize also str.translate() for ASCII => ASCII deletion files: Objects/unicodeobject.c | 54 +++++++++++++++++----------- 1 files changed, 32 insertions(+), 22 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8561,7 +8561,8 @@ if (item == Py_None) { /* deletion: skip fast translate */ - goto exit; + translate[ch] = 0xfe; + return 1; } if (item == NULL) { @@ -8614,12 +8615,12 @@ translated into writer, raise an exception and return -1 on error. */ static int unicode_fast_translate(PyObject *input, PyObject *mapping, - _PyUnicodeWriter *writer) -{ - Py_UCS1 translate[128], ch, ch2; + _PyUnicodeWriter *writer, int ignore) +{ + Py_UCS1 ascii_table[128], ch, ch2; Py_ssize_t len; Py_UCS1 *in, *end, *out; - int res; + int res = 0; if (PyUnicode_READY(input) == -1) return -1; @@ -8627,7 +8628,7 @@ return 0; len = PyUnicode_GET_LENGTH(input); - memset(translate, 0xff, 128); + memset(ascii_table, 0xff, 128); in = PyUnicode_1BYTE_DATA(input); end = in + len; @@ -8636,23 +8637,32 @@ assert(PyUnicode_GET_LENGTH(writer->buffer) == len); out = PyUnicode_1BYTE_DATA(writer->buffer); - for (; in < end; in++, out++) { + for (; in < end; in++) { ch = *in; - ch2 = translate[ch]; + ch2 = ascii_table[ch]; if (ch2 == 0xff) { - res = unicode_fast_translate_lookup(mapping, ch, translate); - if (res < 0) + int translate = unicode_fast_translate_lookup(mapping, ch, + ascii_table); + if (translate < 0) return -1; - if (res == 0) { - writer->pos = in - PyUnicode_1BYTE_DATA(input); - return 0; - } - ch2 = translate[ch]; - } + if (translate == 0) + goto exit; + ch2 = ascii_table[ch]; + } + if (ch2 == 0xfe) { + if (ignore) + continue; + goto exit; + } + assert(ch2 < 128); *out = ch2; - } - writer->pos = len; - return 1; + out++; + } + res = 1; + +exit: + writer->pos = out - PyUnicode_1BYTE_DATA(writer->buffer); + return res; } PyObject * @@ -8695,7 +8705,9 @@ if (_PyUnicodeWriter_Prepare(&writer, size, 127) == -1) goto onError; - res = unicode_fast_translate(input, mapping, &writer); + ignore = (errors != NULL && strcmp(errors, "ignore") == 0); + + res = unicode_fast_translate(input, mapping, &writer, ignore); if (res < 0) { _PyUnicodeWriter_Dealloc(&writer); return NULL; @@ -8703,8 +8715,6 @@ if (res == 1) return _PyUnicodeWriter_Finish(&writer); - ignore = (errors != NULL && strcmp(errors, "ignore") == 0); - i = writer.pos; while (i http://hg.python.org/cpython/rev/e56c71c6b45e changeset: 90158:e56c71c6b45e user: Victor Stinner date: Sat Apr 05 15:35:01 2014 +0200 summary: Issue #21118: str.translate() now raises a ValueError, not a TypeError, if the replacement character is bigger than U+10ffff code point. files: Lib/test/test_codeccallbacks.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_codeccallbacks.py b/Lib/test/test_codeccallbacks.py --- a/Lib/test/test_codeccallbacks.py +++ b/Lib/test/test_codeccallbacks.py @@ -819,7 +819,7 @@ def __getitem__(self, key): raise ValueError #self.assertRaises(ValueError, "\xff".translate, D()) - self.assertRaises(TypeError, "\xff".translate, {0xff: sys.maxunicode+1}) + self.assertRaises(ValueError, "\xff".translate, {0xff: sys.maxunicode+1}) self.assertRaises(TypeError, "\xff".translate, {0xff: ()}) def test_bug828737(self): -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sun Apr 6 09:23:29 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 06 Apr 2014 09:23:29 +0200 Subject: [Python-checkins] Daily reference leaks (e56c71c6b45e): sum=20 Message-ID: results for e56c71c6b45e on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 test_unicode leaked [7, 7, 7] references, sum=21 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogmlGgsa', '-x'] From python-checkins at python.org Sun Apr 6 20:01:05 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 6 Apr 2014 20:01:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_minor_docfix_?= =?utf-8?q?=28reported_at_docs=40python=2Eorg=29_trace=2Epy_argument_is_--?= =?utf-8?q?count_not?= Message-ID: <3g22k92PL4z7LjT@mail.python.org> http://hg.python.org/cpython/rev/c5aa95c7b5e5 changeset: 90159:c5aa95c7b5e5 branch: 2.7 parent: 90146:87c150c43ca8 user: Senthil Kumaran date: Sun Apr 06 10:58:51 2014 -0700 summary: minor docfix (reported at docs at python.org) trace.py argument is --count not --counts files: Doc/library/trace.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/trace.rst b/Doc/library/trace.rst --- a/Doc/library/trace.rst +++ b/Doc/library/trace.rst @@ -41,8 +41,8 @@ At least one of the following options must be specified when invoking :mod:`trace`. The :option:`--listfuncs <-l>` option is mutually exclusive with -the :option:`--trace <-t>` and :option:`--counts <-c>` options. When -:option:`--listfuncs <-l>` is provided, neither :option:`--counts <-c>` nor +the :option:`--trace <-t>` and :option:`--count <-c>` options. When +:option:`--listfuncs <-l>` is provided, neither :option:`--count <-c>` nor :option:`--trace <-t>` are accepted, and vice versa. .. program:: trace -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 6 20:01:07 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 6 Apr 2014 20:01:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_minor_docfix_?= =?utf-8?q?=28reported_at_docs=40python=2Eorg=29_trace=2Epy_argument_is_--?= =?utf-8?q?count_not?= Message-ID: <3g22kC1YG5z7LjT@mail.python.org> http://hg.python.org/cpython/rev/b49d990aaa9d changeset: 90160:b49d990aaa9d branch: 3.3 parent: 90130:63e6afd3ff1f user: Senthil Kumaran date: Sun Apr 06 10:59:47 2014 -0700 summary: minor docfix (reported at docs at python.org) trace.py argument is --count not --counts files: Doc/library/trace.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/trace.rst b/Doc/library/trace.rst --- a/Doc/library/trace.rst +++ b/Doc/library/trace.rst @@ -41,8 +41,8 @@ At least one of the following options must be specified when invoking :mod:`trace`. The :option:`--listfuncs <-l>` option is mutually exclusive with -the :option:`--trace <-t>` and :option:`--counts <-c>` options. When -:option:`--listfuncs <-l>` is provided, neither :option:`--counts <-c>` nor +the :option:`--trace <-t>` and :option:`--count <-c>` options. When +:option:`--listfuncs <-l>` is provided, neither :option:`--count <-c>` nor :option:`--trace <-t>` are accepted, and vice versa. .. program:: trace -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 6 20:01:08 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 6 Apr 2014 20:01:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_from_3=2E3?= Message-ID: <3g22kD4y8Dz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/d1d4bcf87e59 changeset: 90161:d1d4bcf87e59 branch: 3.4 parent: 90150:70990e795657 parent: 90160:b49d990aaa9d user: Senthil Kumaran date: Sun Apr 06 11:00:15 2014 -0700 summary: merge from 3.3 files: Doc/library/trace.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/trace.rst b/Doc/library/trace.rst --- a/Doc/library/trace.rst +++ b/Doc/library/trace.rst @@ -41,8 +41,8 @@ At least one of the following options must be specified when invoking :mod:`trace`. The :option:`--listfuncs <-l>` option is mutually exclusive with -the :option:`--trace <-t>` and :option:`--counts <-c>` options. When -:option:`--listfuncs <-l>` is provided, neither :option:`--counts <-c>` nor +the :option:`--trace <-t>` and :option:`--count <-c>` options. When +:option:`--listfuncs <-l>` is provided, neither :option:`--count <-c>` nor :option:`--trace <-t>` are accepted, and vice versa. .. program:: trace -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 6 20:01:10 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 6 Apr 2014 20:01:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E4?= Message-ID: <3g22kG0CjWz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/b466fd273625 changeset: 90162:b466fd273625 parent: 90158:e56c71c6b45e parent: 90161:d1d4bcf87e59 user: Senthil Kumaran date: Sun Apr 06 11:00:49 2014 -0700 summary: merge from 3.4 files: Doc/library/trace.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/trace.rst b/Doc/library/trace.rst --- a/Doc/library/trace.rst +++ b/Doc/library/trace.rst @@ -41,8 +41,8 @@ At least one of the following options must be specified when invoking :mod:`trace`. The :option:`--listfuncs <-l>` option is mutually exclusive with -the :option:`--trace <-t>` and :option:`--counts <-c>` options. When -:option:`--listfuncs <-l>` is provided, neither :option:`--counts <-c>` nor +the :option:`--trace <-t>` and :option:`--count <-c>` options. When +:option:`--listfuncs <-l>` is provided, neither :option:`--count <-c>` nor :option:`--trace <-t>` are accepted, and vice versa. .. program:: trace -- Repository URL: http://hg.python.org/cpython From benjamin at python.org Sun Apr 6 20:35:35 2014 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 06 Apr 2014 11:35:35 -0700 Subject: [Python-checkins] cpython (3.3): minor docfix (reported at docs@python.org) trace.py argument is --count not In-Reply-To: <3g22kC1YG5z7LjT@mail.python.org> References: <3g22kC1YG5z7LjT@mail.python.org> Message-ID: <1396809335.22219.103363933.6CF1B945@webmail.messagingengine.com> On Sun, Apr 6, 2014, at 11:01, senthil.kumaran wrote: > http://hg.python.org/cpython/rev/b49d990aaa9d > changeset: 90160:b49d990aaa9d > branch: 3.3 > parent: 90130:63e6afd3ff1f > user: Senthil Kumaran > date: Sun Apr 06 10:59:47 2014 -0700 > summary: > minor docfix (reported at docs at python.org) trace.py argument is --count > not --counts This change is of course not harmful, but remember that 3.3 is in security-fix only mode. From python-checkins at python.org Mon Apr 7 01:05:18 2014 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 7 Apr 2014 01:05:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_to_PEP_465_--_propose_?= =?utf-8?q?to_keep_=40_at_the_same_level_as_*=2E?= Message-ID: <3g29TB5pKwz7Ljl@mail.python.org> http://hg.python.org/peps/rev/e948cc3f7c8a changeset: 5453:e948cc3f7c8a user: Guido van Rossum date: Sun Apr 06 16:05:38 2014 -0700 summary: Update to PEP 465 -- propose to keep @ at the same level as *. files: pep-0465.txt | 151 +++++++++++++++++++++++++++----------- 1 files changed, 106 insertions(+), 45 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt --- a/pep-0465.txt +++ b/pep-0465.txt @@ -27,7 +27,7 @@ ======= ========================= =============================== Op Precedence/associativity Methods ======= ========================= =============================== -``@`` *To be determined* ``__matmul__``, ``__rmatmul__`` +``@`` Same as ``*`` ``__matmul__``, ``__rmatmul__`` ``@=`` n/a ``__imatmul__`` ======= ========================= =============================== @@ -761,11 +761,6 @@ * sympy * sage -If you know of any actively maintained Python libraries which provide -an interface for working with numerical arrays or matrices, and which -are not listed above, then please let the PEP author know: -njs at pobox.com - Implementation details ====================== @@ -833,12 +828,12 @@ Finally, there is the option of using multi-character tokens. Some options: -* Matlab uses a ``.*`` operator. Aside from being visually confusable - with ``*``, this would be a terrible choice for us because in - Matlab, ``*`` means matrix multiplication and ``.*`` means - elementwise multiplication, so using ``.*`` for matrix - multiplication would make us exactly backwards from what Matlab - users expect. +* Matlab and Julia use a ``.*`` operator. Aside from being visually + confusable with ``*``, this would be a terrible choice for us + because in Matlab and Julia, ``*`` means matrix multiplication and + ``.*`` means elementwise multiplication, so using ``.*`` for matrix + multiplication would make us exactly backwards from what Matlab and + Julia users expect. * APL apparently used ``+.?``, which by combining a multi-character token, confusing attribute-access-like . syntax, and a unicode @@ -863,7 +858,7 @@ ``[*]`` or ``|*|`` (but when used in context, the use of vertical grouping symbols tends to recreate the nested parentheses visual clutter that was noted as one of the major downsides of the function - syntax we're trying to get away from); ``^*`` and ``^^``. + syntax we're trying to get away from); ``^*``. So, it doesn't matter much, but ``@`` seems as good or better than any of the alternatives: @@ -889,6 +884,81 @@ * Whatever, we have to pick something. +Precedence and associativity +---------------------------- + +There was a long discussion [#associativity-discussions]_ about +whether ``@`` should be right- or left-associative (or even something +more exotic [#group-associativity]_). Almost all Python operators are +left-associative, so following this convention would be the simplest +approach, but there were two arguments that suggested matrix +multiplication might be worth making right-associative as a special +case: + +First, matrix multiplication has a tight conceptual association with +function application/composition, so many mathematically sophisticated +users have an intuition that an expression like :math:`R S x` proceeds +from right-to-left, with first :math:`S` transforming the vector +:math:`x`, and then :math:`R` transforming the result. This isn't +universally agreed (and not all number-crunchers are steeped in the +pure-math conceptual framework that motivates this intuition +[#oil-industry-versus-right-associativity]_), but at the least this +intuition is more common than for other operations like :math:`2 \cdot +3 \cdot 4` which everyone reads as going from left-to-right. + +Second, if expressions like ``Mat @ Mat @ vec`` appear often in code, +then programs will run faster (and efficiency-minded programmers will +be able to use fewer parentheses) if this is evaluated as ``Mat @ (Mat +@ vec)`` then if it is evaluated like ``(Mat @ Mat) @ vec``. + +However, weighing against these arguments are the following: + +Regarding the efficiency argument, empirically, we were unable to find +any evidence that ``Mat @ Mat @ vec`` type expressions actually +dominate in real-life code. Parsing a number of large projects that +use numpy, we found that when forced by numpy's current funcall syntax +to choose an order of operations for nested calls to ``dot``, people +actually use left-associative nesting slightly *more* often than +right-associative nesting [#numpy-associativity-counts]_. And anyway, +writing parentheses isn't so bad -- if an efficiency-minded programmer +is going to take the trouble to think through the best way to evaluate +some expression, they probably *should* write down the parentheses +regardless of whether they're needed, just to make it obvious to the +next reader that they order of operations matter. + +In addition, it turns out that other languages, including those with +much more of a focus on linear algebra, overwhelmingly make their +matmul operators left-associative. Specifically, the ``@`` equivalent +is left-associative in R, Matlab, Julia, IDL, and Gauss. The only +exceptions we found are Mathematica, in which ``a @ b @ c`` would be +parsed non-associatively as ``dot(a, b, c)``, and APL, in which all +operators are right-associative. There do not seem to exist any +languages that make ``@`` right-associative and ``*`` +left-associative. And these decisions don't seem to be controversial +-- I've never seen anyone complaining about this particular aspect of +any of these other languages, and the left-associativity of ``*`` +doesn't seem to bother users of the existing Python libraries that use +``*`` for matrix multiplication. So, at the least we can conclude from +this that making ``@`` left-associative will certainly not cause any +disasters. Making ``@`` right-associative, OTOH, would be exploring +new and uncertain ground. + +And another advantage of left-associativity is that it is much easier +to learn and remember that ``@`` acts like ``*``, than it is to +remember first that ``@`` is unlike other Python operators by being +right-associative, and then on top of this, also have to remember +whether it is more tightly or more loosely binding than +``*``. (Right-associativity forces us to choose a precedence, and +intuitions were about equally split on which precedence made more +sense. So this suggests that no matter which choice we made, no-one +would be able to guess or remember it.) + +On net, therefore, the general consensus of the numerical community is +that while matrix multiplication is something of a special case, it's +not special enough to break the rules, and ``@`` should parse like +``*`` does. + + (Non)-Definitions for built-in types ------------------------------------ @@ -930,32 +1000,7 @@ decided that the utility of this was sufficiently unclear that it would be better to leave it out for now, and only revisit the issue if -- once we have more experience with ``@`` -- it turns out that ``@@`` -is truly missed. [#atat-discussion] - - -Unresolved issues ------------------ - -Associativity of ``@`` -'''''''''''''''''''''' - -It's been suggested that ``@`` should be right-associative, on the -grounds that for expressions like ``Mat @ Mat @ vec``, the two -different evaluation orders produce the same result, but the -right-associative order ``Mat @ (Mat @ vec)`` will be faster and use -less memory than the left-associative order ``(Mat @ Mat) @ vec``. -(Matrix-vector multiplication is much cheaper than matrix-matrix -multiplication). It would be a shame if users found themselves -required to use an overabundance of parentheses to achieve acceptable -speed/memory usage in common situations, but, it's not currently clear -whether such cases actually are common enough to override Python's -general rule of left-associativity, or even whether they're more -common than the symmetric cases where left-associativity would be -faster (though this does seem intuitively plausible). The only way to -answer this is probably to do an audit of some real-world uses and -check how often the associativity matters in practice; if this PEP is -accepted in principle, then we should probably do this check before -finalizing it. +is truly missed. [#atat-discussion]_ Rejected alternatives to adding a new operator @@ -1130,8 +1175,9 @@ * numpy-discussion thread on whether to keep ``@@``: http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069448.html -* numpy-discussion thread on precedence/associativity of ``@``: - http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069444.html +* numpy-discussion threads on precedence/associativity of ``@``: + * http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069444.html + * http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069605.html References @@ -1207,10 +1253,10 @@ Matrix multiply counts were estimated by counting how often certain tokens which are used as matrix multiply function names occurred in - each package. In principle this could create false positives, but - as far as I know the counts are exact; it's unlikely that anyone is - using ``dot`` as a variable name when it's also the name of one of - the most widely-used numpy functions. + each package. This creates a small number of false positives for + scikit-learn, because we also count instances of the wrappers + around ``dot`` that this package uses, and so there are a few dozen + tokens which actually occur in ``import`` or ``def`` statements. All counts were made using the latest development version of each project as of 21 Feb 2014. @@ -1312,6 +1358,21 @@ elementwise multiplication, and ``%`` for matrix multiplication: https://mail.python.org/pipermail/matrix-sig/1995-August/000002.html +.. [#atat-discussion] http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069502.html + +.. [#associativity-discussions] + http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069444.html + http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069605.html + +.. [#oil-industry-versus-right-associativity] + http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069610.html + +.. [#numpy-associativity-counts] + http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069578.html + +.. [#group-associativity] + http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069530.html + Copyright ========= -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Apr 7 02:58:39 2014 From: python-checkins at python.org (eric.snow) Date: Mon, 7 Apr 2014 02:58:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_PEP_468=2E?= Message-ID: <3g2Czz5ztXzRbQ@mail.python.org> http://hg.python.org/peps/rev/0c10a68b04c2 changeset: 5454:0c10a68b04c2 user: Eric Snow date: Sun Apr 06 18:54:41 2014 -0600 summary: Add PEP 468. files: pep-0468.txt | 455 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 455 insertions(+), 0 deletions(-) diff --git a/pep-0468.txt b/pep-0468.txt new file mode 100644 --- /dev/null +++ b/pep-0468.txt @@ -0,0 +1,455 @@ +PEP: 468 +Title: Preserving the order of \*\*kwargs in a function. +Version: $Revision$ +Last-Modified: $Date$ +Author: Eric Snow +Discussions-To: python-ideas at python.org +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 5-Apr-2014 +Python-Version: 3.5 +Post-History: 5-Apr-2014 +Resolution: + + +Abstract +======== + +The \*\*kwargs syntax in a function definition indicates that the +interpreter should collect all keyword arguments that do not correspond +to other named parameters. However, +Python does not preserved the order in which those collected keyword +arguments were passed to the function. In some contexts the order +matters. This PEP introduces a mechanism by which the passed order of +collected keyword arguments will now be preserved. + + +Motivation +========== + +Python's \*\*kwargs syntax in function definitions provides a powerful +means of dynamically handling keyword arguments. In some applications +of the syntax (see _`Use Cases`), the semantics applied to the +collected keyword arguments requires that order be preserved. +Unsurprisingly, this is similar to how OrderedDict is related to dict. + +Currently to preserved the order you have to do so manually and +separately from the actual function call. This involves building an +ordered mapping, whether an OrderedDict or an iterable of 2-tuples, +which is then passed as a single argument to the function. +[#arg_unpacking]_ + +With the capability described in this PEP, that boilerplate would no +longer be required. + +For comparision, currently:: + + kwargs = OrderedDict() + kwargs['eggs'] = ... + ... + def spam(a, kwargs): + ... + +and with this proposal:: + + def spam(a, **kwargs): + ... + +Nick Coglan, speaking of some of the uses cases, summed it up well +[#nick_obvious]_:: + + These *can* all be done today, but *not* by using keyword arguments. + In my view, the problem to be addressed is that keyword arguments + *look* like they should work for these cases, because they have a + definite order in the source code. The only reason they don't work + is because the interpreter throws that ordering information away. + + It's a textbook case of a language feature becoming an attractive + nuisance in some circumstances: the simple and obvious solution for + the above use cases *doesn't actually work* for reasons that aren't + obviously clear if you don't have a firm grasp of Python's admittedly + complicated argument handling. + +This observation is supported by the appearance of this proposal over +the years and the numerous times that people have been confused by the +constructor for OrderedDict. [#past_threads]_ [#loss_of_order]_ +[#compact_dict]_ + + +Use Cases +========= + +As Nick noted, the current behavior of \*\*kwargs is unintuitive in +cases where one would expect order to matter. Aside from more specific +cases outlined below, in general "anything else where you want to +control the iteration order *and* set field names and values in a single +call will potentially benefit." [#nick_general]_ That matters in the +case of factories (e.g. __init__()) for ordered types. + +Serialization +------------- + +Obviously OrderedDict would benefit (both __init__() and update()) from +ordered kwargs. However, the benefit also extends to serialization +APIs [#nick_obvious]_:: + + In the context of serialisation, one key lesson we have learned is + that arbitrary ordering is a problem when you want to minimise + spurious diffs, and sorting isn't a simple solution. + + Tools like doctest don't tolerate spurious diffs at all, but are + often amenable to a sorting based answer. + + The cases where it would be highly desirable to be able use keyword + arguments to control the order of display of a collection of key + value pairs are ones like: + + * printing out key:value pairs in CLI output + * mapping semantic names to column order in a CSV + * serialising attributes and elements in particular orders in XML + * serialising map keys in particular orders in human readable formats + like JSON and YAML (particularly when they're going to be placed + under source control) + +Debugging +--------- + +In the words of Raymond Hettinger [#raymond_debug]_:: + + It makes it easier to debug if the arguments show-up in the order + they were created. AFAICT, no purpose is served by scrambling them. + +Other Use Cases +--------------- + +* Mock objects. [#mock]_ +* Controlling object presentation. +* Alternate namedtuple() where defaults can be specified. +* Specifying argument priority by order. + + +Concerns +======== + +Performance +----------- + +As already noted, the idea of ordered keyword arguments has come up on +a number of occasions. Each time it has been met with the same +response, namely that preserving keyword arg order would have a +sufficiently adverse effect on function call performance that it's not +worth doing. However, Guido noted the following [#guido_open]_:: + + Making **kwds ordered is still open, but requires careful design and + implementation to avoid slowing down function calls that don't benefit. + +As will be noted below, there are ways to work around this at the +expense of increased complication. Ultimately the simplest approach is +the one that makes the most sense: pack collected key word arguments +into an OrderedDict. However, without a C implementation of OrderedDict +there isn't much to discuss. That should change in Python 3.5. +[#c_ordereddict]_ + +In some cases the difference of performance between dict and OrderedDict +*may* be of significance. For instance: when the collected +kwargs has an extended lifetime outside the originating function or the +number of collected kwargs is massive. However, the difference in +performance (both CPU and memory) in those cases should not be +significant. Furthermore, the performance of the C OrderedDict +implementation is essentially identical with dict for the non-mutating +API. A concrete representation of the difference in performance will be +a part of this proposal before its resolution. + +Other Python Implementations +---------------------------- + +Another important issue to consider is that new features must be +cognizant of the multiple Python implementations. At some point each of +them would be expected to have implemented ordered kwargs. In this +regard there doesn't seem to be an issue with the idea. [#ironpython]_ +Each of the major Python implementations will be consulted regarding +this proposal before its resolution. + + +Specification +============= + +Starting in version 3.5 Python will preserve the order of keyword +arguments as passed to a function. To accomplish this the collected +kwargs will now be an OrderedDict rather than a dict. + +This will apply only to functions for which the definition uses the +\*\*kwargs syntax for collecting otherwise unspecified keyword +arguments. Only the order of those keyword arguments will be +preserved. + +Relationship to \*\*-unpacking syntax +------------------------------------- + +The ** unpacking syntax in function calls has no special connection with +this proposal. Keyword arguments provided by unpacking will be treated +in exactly the same way as they are now: ones that match defined +parameters are gather there and the remainder will be collected into the +ordered kwargs (just like any other unmatched keyword argument). + +Note that unpacking a mapping with undefined order, such as dict, will +preserve its iteration order like normal. It's just that the order will +remain undefined. The OrderedDict into which the unpacked key-value +pairs will then be packed will not be able to provide any alternate +ordering. This should not be surprising. + +There have been brief discussions of simply passing these mappings +through to the functions kwargs without unpacking and repacking them, +but that is both outside the scope of this proposal and probably a bad +idea regardless. (There is a reason those discussions were brief.) + +Relationship to inspect.Signature +--------------------------------- + +Signature objects should need no changes. The `kwargs` parameter of +inspect.BoundArguments (returned by Signature.bind() and +Signature.bind_partial()) will change from a dict to an OrderedDict. + +C-API +----- + +TBD + +Syntax +------ + +No syntax is added or changed by this proposal. + +Backward-Compatibility +---------------------- + +The following will change: + +* type(kwargs) +* iteration order of kwargs will now be consistent (except of course in + the case described above) +* as already noted, performance will be marginally different + +None of these should be an issue. However, each will be carefully +considered while this proposal is under discussion. + + +Reference Implementation +======================== + +TBD + +Implementation Notes +-------------------- + +TBD + + +Alternate Approaches +==================== + +Opt-out Decorator +----------------- + +This is identical to the current proposal with the exception that Python +would also provide a decorator in functools that would cause collected +keyword arguments to be packed into a normal dict instead of an +OrderedDict. + +Prognosis: + +This would only be necessary if performance is determined to be +significantly different in some uncommon cases or that there are other +backward-compatibility concerns that cannot be resolved otherwise. + +Opt-in Decorator +---------------- + +The status quo would be unchanged. Instead Python would provide a +decorator in functools that would register or mark the decorated +function as one that should get ordered keyword arguments. The +performance overhead to check the function at call time would be +marginal. + +Prognosis: + +The only real down-side is in the case of function wrappers factories +(e.g. functools.partial and many decorators) that aim to perfectly +preserve keyword arguments by using kwargs in the wrapper definition +and kwargs unpacking in the call to the wrapped function. Each wrapper +would have to be updated separately, though having functools.wraps() do +this automaticallywould help. + +__kworder__ +----------- + +The order of keyword arguments would be stored separately in a list at +call time. The list would be bound to __kworder__ in the function +locals. + +Prognosis: + +This likewise complicates the wrapper case. + +Compact dict with faster iteration +---------------------------------- + +Raymond Hettinger has introduced the idea of a dict implementation that +would result in preserving insertion order on dicts (until the first +deletion). This would be a perfect fit for kwargs. [#compact_dict]_ + +Prognosis: + +The idea is still uncertain in both viability and timeframe. + +\*\*\*kwargs +------------ + +This would add a new form to a function's signature as a mutually +exclusive parallel to \*\*kwargs. The new syntax, \*\*\*kwargs (note +that there are three asterisks), would indicate that kwargs should +preserve the order of keyword arguments. + +Prognosis: + +New syntax is only added to Python under the most *dire* circumstances. +With other available solutions, new syntax is not justifiable. +Furthermore, like all opt-in solutions, the new syntax would complicate +the pass-through case. + +annotations +----------- + +This is a variation on the decorator approach. Instead of using a +decorator to mark the function, you would use a function annotation on +\*\*kwargs. + +Prognosis: + +In addition to the pass-through complication, annotations have been +actively discouraged in Python core development. Use of annotations to +opt-in to order preservation runs the risk of interfering with other +application-level use of annotations. + +dict.__order__ +-------------- + +dict objects would have a new attribute, `__order__` that would default +to None and that in the kwargs case the interpreter would use in the +same way as described above for __kworder__. + +Prognosis: + +It would mean zero impact on kwargs performance but the change would be +pretty intrusive (Python uses dict a lot). Also, for the wrapper case +the interpreter would have to be careful to preserve `__order__`. + +KWArgsDict.__order__ +-------------------- + +This is the same as the `dict.__order__` idea, but kwargs would be an +instance of a new minimal dict subclass that provides the `__order__` +attribute. dict would instead be unchanged. + +Prognosis: + +Simply switching to OrderedDict is a less complicated and more intuitive +change. + + +Acknowledgements +================ + +Thanks to Andrew Barnert for helpful feedback and to the participants of +all the past email threads. + + +Footnotes +========= + +.. [#arg_unpacking] + + Alternately, you could also replace ** in your function definition + with * and then pass in key/value 2-tuples. This has the advantage + of not requiring the keys to be valid identifier strings. See + https://mail.python.org/pipermail/python-ideas/2014-April/027491.html. + + +References +========== + +.. [#nick_obvious] + https://mail.python.org/pipermail/python-ideas/2014-April/027512.html + +.. [#past_threads] + https://mail.python.org/pipermail/python-ideas/2009-April/004163.html + + https://mail.python.org/pipermail/python-ideas/2010-October/008445.html + + https://mail.python.org/pipermail/python-ideas/2011-January/009037.html + + https://mail.python.org/pipermail/python-ideas/2013-February/019690.html + + https://mail.python.org/pipermail/python-ideas/2013-May/020727.html + + https://mail.python.org/pipermail/python-ideas/2014-March/027225.html + + http://bugs.python.org/issue16276 + + http://bugs.python.org/issue16553 + + http://bugs.python.org/issue19026 + + http://bugs.python.org/issue5397#msg82972 + +.. [#loss_of_order] + https://mail.python.org/pipermail/python-dev/2007-February/071310.html + +.. [#compact_dict] + https://mail.python.org/pipermail/python-dev/2012-December/123028.html + + https://mail.python.org/pipermail/python-dev/2012-December/123105.html + + https://mail.python.org/pipermail/python-dev/2013-May/126327.html + + https://mail.python.org/pipermail/python-dev/2013-May/126328.html + +.. [#nick_general] + https://mail.python.org/pipermail/python-dev/2012-December/123105.html + +.. [#raymond_debug] + https://mail.python.org/pipermail/python-dev/2013-May/126327.html + +.. [#mock] + https://mail.python.org/pipermail/python-ideas/2009-April/004163.html + + https://mail.python.org/pipermail/python-ideas/2009-April/004165.html + + https://mail.python.org/pipermail/python-ideas/2009-April/004175.html + +.. [#guido_open] + https://mail.python.org/pipermail/python-dev/2013-May/126404.html + +.. [#c_ordereddict] + http://bugs.python.org/issue16991 + +.. [#ironpython] + https://mail.python.org/pipermail/python-dev/2012-December/123100.html + + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: + -- Repository URL: http://hg.python.org/peps From solipsis at pitrou.net Mon Apr 7 09:52:22 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 07 Apr 2014 09:52:22 +0200 Subject: [Python-checkins] Daily reference leaks (b466fd273625): sum=24 Message-ID: results for b466fd273625 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [2, 0, -2] references, sum=0 test_site leaked [2, 0, -2] memory blocks, sum=0 test_unicode leaked [7, 7, 7] references, sum=21 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogVrmREd', '-x'] From python-checkins at python.org Mon Apr 7 11:22:29 2014 From: python-checkins at python.org (victor.stinner) Date: Mon, 7 Apr 2014 11:22:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbzogRG9j?= =?utf-8?q?ument_Task=2Ecancel=28=29_properly=2E?= Message-ID: <3g2R9K30xfz7LjN@mail.python.org> http://hg.python.org/cpython/rev/027568b52a06 changeset: 90163:027568b52a06 branch: 3.4 parent: 90161:d1d4bcf87e59 user: Victor Stinner date: Mon Apr 07 11:18:06 2014 +0200 summary: asyncio: Document Task.cancel() properly. files: Lib/asyncio/tasks.py | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -250,6 +250,25 @@ print(line, file=file, end='') def cancel(self): + """Request that a task to cancel itself. + + This arranges for a CancellationError to be thrown into the + wrapped coroutine on the next cycle through the event loop. + The coroutine then has a chance to clean up or even deny + the request using try/except/finally. + + Contrary to Future.cancel(), this does not guarantee that the + task will be cancelled: the exception might be caught and + acted upon, delaying cancellation of the task or preventing it + completely. The task may also return a value or raise a + different exception. + + Immediately after this method is called, Task.cancelled() will + not return True (unless the task was already cancelled). A + task will be marked as cancelled when the wrapped coroutine + terminates with a CancelledError exception (even if cancel() + was not called). + """ if self.done(): return False if self._fut_waiter is not None: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 7 11:22:31 2014 From: python-checkins at python.org (victor.stinner) Date: Mon, 7 Apr 2014 11:22:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTU1?= =?utf-8?q?=3A_asyncio=2EEventLoop=2Ecreate=5Funix=5Fserver=28=29_now_rais?= =?utf-8?q?es_a_ValueError_if?= Message-ID: <3g2R9M2bXyz7Lk3@mail.python.org> http://hg.python.org/cpython/rev/a6b764848b20 changeset: 90164:a6b764848b20 branch: 3.4 user: Victor Stinner date: Mon Apr 07 11:18:54 2014 +0200 summary: Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError if path and sock are specified at the same time. files: Lib/asyncio/unix_events.py | 4 ++++ Lib/test/test_asyncio/test_events.py | 11 +++++++++++ Misc/NEWS | 3 +++ 3 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -206,6 +206,10 @@ raise TypeError('ssl argument must be an SSLContext or None') if path is not None: + if sock is not None: + raise ValueError( + 'path and sock can not be specified at the same time') + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) try: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -703,6 +703,17 @@ # close server server.close() + @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'No UNIX Sockets') + def test_create_unix_server_path_socket_error(self): + proto = MyProto(loop=self.loop) + sock = socket.socket() + with sock: + f = self.loop.create_unix_server(lambda: proto, '/test', sock=sock) + with self.assertRaisesRegex(ValueError, + 'path and sock can not be specified ' + 'at the same time'): + server = self.loop.run_until_complete(f) + def _create_ssl_context(self, certfile, keyfile=None): sslcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23) sslcontext.options |= ssl.OP_NO_SSLv2 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ Library ------- +- Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError + if path and sock are specified at the same time. + - Issue #21149: Improved thread-safety in logging cleanup during interpreter shutdown. Thanks to Devin Jeanpierre for the patch. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 7 11:22:32 2014 From: python-checkins at python.org (victor.stinner) Date: Mon, 7 Apr 2014 11:22:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_=28Merge_3=2E4=29_Issue_=2321155=3A_asyncio=2EEventLoop?= =?utf-8?q?=2Ecreate=5Funix=5Fserver=28=29_now_raises_a?= Message-ID: <3g2R9N4SH5z7LkL@mail.python.org> http://hg.python.org/cpython/rev/34ace7eb67e9 changeset: 90165:34ace7eb67e9 parent: 90162:b466fd273625 parent: 90164:a6b764848b20 user: Victor Stinner date: Mon Apr 07 11:20:22 2014 +0200 summary: (Merge 3.4) Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError if path and sock are specified at the same time. asyncio: Document Task.cancel() properly. files: Lib/asyncio/tasks.py | 19 ++++++++++++++++ Lib/asyncio/unix_events.py | 4 +++ Lib/test/test_asyncio/test_events.py | 11 +++++++++ Misc/NEWS | 3 ++ 4 files changed, 37 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -250,6 +250,25 @@ print(line, file=file, end='') def cancel(self): + """Request that a task to cancel itself. + + This arranges for a CancellationError to be thrown into the + wrapped coroutine on the next cycle through the event loop. + The coroutine then has a chance to clean up or even deny + the request using try/except/finally. + + Contrary to Future.cancel(), this does not guarantee that the + task will be cancelled: the exception might be caught and + acted upon, delaying cancellation of the task or preventing it + completely. The task may also return a value or raise a + different exception. + + Immediately after this method is called, Task.cancelled() will + not return True (unless the task was already cancelled). A + task will be marked as cancelled when the wrapped coroutine + terminates with a CancelledError exception (even if cancel() + was not called). + """ if self.done(): return False if self._fut_waiter is not None: diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -206,6 +206,10 @@ raise TypeError('ssl argument must be an SSLContext or None') if path is not None: + if sock is not None: + raise ValueError( + 'path and sock can not be specified at the same time') + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) try: diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -703,6 +703,17 @@ # close server server.close() + @unittest.skipUnless(hasattr(socket, 'AF_UNIX'), 'No UNIX Sockets') + def test_create_unix_server_path_socket_error(self): + proto = MyProto(loop=self.loop) + sock = socket.socket() + with sock: + f = self.loop.create_unix_server(lambda: proto, '/test', sock=sock) + with self.assertRaisesRegex(ValueError, + 'path and sock can not be specified ' + 'at the same time'): + server = self.loop.run_until_complete(f) + def _create_ssl_context(self, certfile, keyfile=None): sslcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23) sslcontext.options |= ssl.OP_NO_SSLv2 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ Library ------- +- Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError + if path and sock are specified at the same time. + - Issue #21136: Avoid unnecessary normalization of Fractions resulting from power and other operations. Patch by Raymond Hettinger. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 00:41:49 2014 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 8 Apr 2014 00:41:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Accept_PEP_465=2E?= Message-ID: <3g2mvd0z6Gz7LlW@mail.python.org> http://hg.python.org/peps/rev/75c84dc81ece changeset: 5455:75c84dc81ece user: Guido van Rossum date: Mon Apr 07 15:41:42 2014 -0700 summary: Accept PEP 465. files: pep-0465.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt --- a/pep-0465.txt +++ b/pep-0465.txt @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Nathaniel J. Smith -Status: Draft +Status: Accepted Type: Standards Track Content-Type: text/x-rst Created: 20-Feb-2014 @@ -769,7 +769,7 @@ added to the standard library, with the usual semantics. A corresponding function ``PyObject* PyObject_MatrixMultiply(PyObject -*o1, PyObject o2)`` is added to the C API. +*o1, PyObject *o2)`` is added to the C API. A new AST node is added named ``MatMult``, along with a new token ``ATEQUAL`` and new bytecode opcodes ``BINARY_MATRIX_MULTIPLY`` and -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Apr 8 01:35:44 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 01:35:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogZml4IHZlcmIgKGNs?= =?utf-8?q?oses_=2321174=29?= Message-ID: <3g2p5r32Mvz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/aff368b58a98 changeset: 90166:aff368b58a98 branch: 2.7 parent: 90159:c5aa95c7b5e5 user: Benjamin Peterson date: Mon Apr 07 19:34:33 2014 -0400 summary: fix verb (closes #21174) files: Doc/library/exceptions.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -158,9 +158,9 @@ .. exception:: GeneratorExit - Raise when a :term:`generator`\'s :meth:`close` method is called. It - directly inherits from :exc:`BaseException` instead of :exc:`StandardError` since - it is technically not an error. + Raised when a :term:`generator`\'s :meth:`close` method is called. It + directly inherits from :exc:`BaseException` instead of :exc:`StandardError` + since it is technically not an error. .. versionadded:: 2.5 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 01:35:45 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 01:35:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogZml4IHZlcmIgKGNs?= =?utf-8?q?oses_=2321174=29?= Message-ID: <3g2p5s4QJfz7LkM@mail.python.org> http://hg.python.org/cpython/rev/33528b9520e6 changeset: 90167:33528b9520e6 branch: 3.4 parent: 90164:a6b764848b20 user: Benjamin Peterson date: Mon Apr 07 19:34:33 2014 -0400 summary: fix verb (closes #21174) files: Doc/library/exceptions.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -161,7 +161,7 @@ .. exception:: GeneratorExit - Raise when a :term:`generator`\'s :meth:`close` method is called. It + Raised when a :term:`generator`\'s :meth:`close` method is called. It directly inherits from :exc:`BaseException` instead of :exc:`Exception` since it is technically not an error. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 01:35:46 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 01:35:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjExNzQp?= Message-ID: <3g2p5t6DqZz7LkJ@mail.python.org> http://hg.python.org/cpython/rev/c48164710ed5 changeset: 90168:c48164710ed5 parent: 90165:34ace7eb67e9 parent: 90167:33528b9520e6 user: Benjamin Peterson date: Mon Apr 07 19:35:36 2014 -0400 summary: merge 3.4 (#21174) files: Doc/library/exceptions.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -161,7 +161,7 @@ .. exception:: GeneratorExit - Raise when a :term:`generator`\'s :meth:`close` method is called. It + Raised when a :term:`generator`\'s :meth:`close` method is called. It directly inherits from :exc:`BaseException` instead of :exc:`Exception` since it is technically not an error. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 02:16:17 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 02:16:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_reference_leaks_in_the?= =?utf-8?q?_translate_fast_path_=28closes_=2321175=29?= Message-ID: <3g2q0d5FSmz7LkJ@mail.python.org> http://hg.python.org/cpython/rev/fa6debebfe8b changeset: 90169:fa6debebfe8b user: Benjamin Peterson date: Mon Apr 07 20:15:41 2014 -0400 summary: fix reference leaks in the translate fast path (closes #21175) Patch by Josh Rosenberg. files: Objects/unicodeobject.c | 26 ++++++++++---------------- 1 files changed, 10 insertions(+), 16 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8551,28 +8551,24 @@ unicode_fast_translate_lookup(PyObject *mapping, Py_UCS1 ch, Py_UCS1 *translate) { - PyObject *item; + PyObject *item = NULL; int ret = 0; - item = NULL; if (charmaptranslate_lookup(ch, mapping, &item)) { return -1; } if (item == Py_None) { - /* deletion: skip fast translate */ + /* deletion */ translate[ch] = 0xfe; - return 1; - } - - if (item == NULL) { + } + else if (item == NULL) { /* not found => default to 1:1 mapping */ translate[ch] = ch; return 1; } - - if (PyLong_Check(item)) { - long replace = (Py_UCS4)PyLong_AS_LONG(item); + else if (PyLong_Check(item)) { + Py_UCS4 replace = (Py_UCS4)PyLong_AS_LONG(item); /* PyLong_AS_LONG() cannot fail, charmaptranslate_lookup() already used it */ if (127 < replace) { @@ -8598,15 +8594,13 @@ translate[ch] = (Py_UCS1)replace; } else { - /* not a long or unicode */ + /* not None, NULL, long or unicode */ goto exit; } + ret = 1; + + exit: Py_DECREF(item); - item = NULL; - ret = 1; - -exit: - Py_XDECREF(item); return ret; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 06:33:43 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 8 Apr 2014 06:33:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321059=3A_Temporar?= =?utf-8?q?y_measure_to_make_the_Windows_buildbots_useful_again=2E?= Message-ID: <3g2wjg48pBz7LkR@mail.python.org> http://hg.python.org/cpython/rev/c507da0b573f changeset: 90170:c507da0b573f user: Zachary Ware date: Mon Apr 07 23:33:29 2014 -0500 summary: Issue #21059: Temporary measure to make the Windows buildbots useful again. Hopefully. files: Tools/buildbot/test-amd64.bat | 3 +++ Tools/buildbot/test.bat | 3 +++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Tools/buildbot/test-amd64.bat b/Tools/buildbot/test-amd64.bat --- a/Tools/buildbot/test-amd64.bat +++ b/Tools/buildbot/test-amd64.bat @@ -1,3 +1,6 @@ @rem Used by the buildbot "test" step. + +rem The following line should be removed before #20035 is closed +set TCL_LIBRARY=%CD%\..\tcltk64\lib\tcl8.6 cd PCbuild call rt.bat -d -q -x64 -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,3 +1,6 @@ @rem Used by the buildbot "test" step. + +rem The following line should be removed before #20035 is closed +set TCL_LIBRARY=%CD%\..\tcltk\lib\tcl8.6 cd PCbuild call rt.bat -d -q -uall -rwW -n --timeout=3600 %1 %2 %3 %4 %5 %6 %7 %8 %9 -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Tue Apr 8 09:05:41 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 08 Apr 2014 09:05:41 +0200 Subject: [Python-checkins] Daily reference leaks (fa6debebfe8b): sum=3 Message-ID: results for fa6debebfe8b on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloghzGIhA', '-x'] From python-checkins at python.org Tue Apr 8 09:14:31 2014 From: python-checkins at python.org (victor.stinner) Date: Tue, 8 Apr 2014 09:14:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321118=3A_PyLong?= =?utf-8?q?=5FAS=5FLONG=28=29_result_type_is_long?= Message-ID: <3g30HC6K2Tz7LjP@mail.python.org> http://hg.python.org/cpython/rev/cb632988bc09 changeset: 90171:cb632988bc09 user: Victor Stinner date: Tue Apr 08 09:14:21 2014 +0200 summary: Issue #21118: PyLong_AS_LONG() result type is long Even if PyLong_AS_LONG() cannot fail, I prefer to use the right type. files: Objects/unicodeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -8568,7 +8568,7 @@ return 1; } else if (PyLong_Check(item)) { - Py_UCS4 replace = (Py_UCS4)PyLong_AS_LONG(item); + long replace = PyLong_AS_LONG(item); /* PyLong_AS_LONG() cannot fail, charmaptranslate_lookup() already used it */ if (127 < replace) { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:45:06 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:45:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_the_name_of_th?= =?utf-8?q?e_slot_of_nb=5Fmultiply_not_nb=5Fmul?= Message-ID: <3g3BH65Dtdz7LkW@mail.python.org> http://hg.python.org/cpython/rev/96a3c9a4287f changeset: 90172:96a3c9a4287f branch: 3.4 parent: 90167:33528b9520e6 user: Benjamin Peterson date: Tue Apr 08 10:44:30 2014 -0400 summary: the name of the slot of nb_multiply not nb_mul files: Doc/c-api/typeobj.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1199,7 +1199,8 @@ This function is used by :c:func:`PySequence_Repeat` and has the same signature. It is also used by the ``*`` operator, after trying numeric - multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_mul` slot. + multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_multiply` + slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_item -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:45:08 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:45:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_the_name_of_th?= =?utf-8?q?e_slot_of_nb=5Fmultiply_not_nb=5Fmul?= Message-ID: <3g3BH80WMmz7LkW@mail.python.org> http://hg.python.org/cpython/rev/41dbabb7d383 changeset: 90173:41dbabb7d383 branch: 2.7 parent: 90166:aff368b58a98 user: Benjamin Peterson date: Tue Apr 08 10:44:30 2014 -0400 summary: the name of the slot of nb_multiply not nb_mul files: Doc/c-api/typeobj.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1295,7 +1295,8 @@ This function is used by :c:func:`PySequence_Repeat` and has the same signature. It is also used by the ``*`` operator, after trying numeric - multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_mul` slot. + multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_multiply` + slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_item -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:45:09 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:45:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g3BH92ztxz7LlR@mail.python.org> http://hg.python.org/cpython/rev/b1ac067c984f changeset: 90174:b1ac067c984f parent: 90171:cb632988bc09 parent: 90172:96a3c9a4287f user: Benjamin Peterson date: Tue Apr 08 10:44:50 2014 -0400 summary: merge 3.4 files: Doc/c-api/typeobj.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1199,7 +1199,8 @@ This function is used by :c:func:`PySequence_Repeat` and has the same signature. It is also used by the ``*`` operator, after trying numeric - multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_mul` slot. + multiplication via the :c:member:`~PyTypeObject.tp_as_number.nb_multiply` + slot. .. c:member:: ssizeargfunc PySequenceMethods.sq_item -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:49:18 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:49:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogUHlTZXF1ZW5jZV9G?= =?utf-8?q?ast_generally_returns_a_list_not_a_tuple_=28closes_=2316395=29?= Message-ID: <3g3BMy1k91z7Ljj@mail.python.org> http://hg.python.org/cpython/rev/b2187b82a658 changeset: 90175:b2187b82a658 branch: 3.4 parent: 90172:96a3c9a4287f user: Benjamin Peterson date: Tue Apr 08 10:48:36 2014 -0400 summary: PySequence_Fast generally returns a list not a tuple (closes #16395) files: Doc/c-api/sequence.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -123,10 +123,10 @@ .. c:function:: PyObject* PySequence_Fast(PyObject *o, const char *m) - Returns the sequence *o* as a tuple, unless it is already a tuple or list, in - which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access the - members of the result. Returns *NULL* on failure. If the object is not a - sequence, raises :exc:`TypeError` with *m* as the message text. + Return the sequence *o* as a list, unless it is already a tuple or list, in + which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access + the members of the result. Returns *NULL* on failure. If the object is not + a sequence, raises :exc:`TypeError` with *m* as the message text. .. c:function:: PyObject* PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:49:19 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:49:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogUHlTZXF1ZW5jZV9G?= =?utf-8?q?ast_generally_returns_a_list_not_a_tuple_=28closes_=2316395=29?= Message-ID: <3g3BMz4TYRz7Lkj@mail.python.org> http://hg.python.org/cpython/rev/b235db467cd5 changeset: 90176:b235db467cd5 branch: 2.7 parent: 90173:41dbabb7d383 user: Benjamin Peterson date: Tue Apr 08 10:48:36 2014 -0400 summary: PySequence_Fast generally returns a list not a tuple (closes #16395) files: Doc/c-api/sequence.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -167,10 +167,10 @@ .. c:function:: PyObject* PySequence_Fast(PyObject *o, const char *m) - Returns the sequence *o* as a tuple, unless it is already a tuple or list, in - which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access the - members of the result. Returns *NULL* on failure. If the object is not a - sequence, raises :exc:`TypeError` with *m* as the message text. + Return the sequence *o* as a list, unless it is already a tuple or list, in + which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access + the members of the result. Returns *NULL* on failure. If the object is not + a sequence, raises :exc:`TypeError` with *m* as the message text. .. c:function:: PyObject* PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:49:20 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:49:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMTYzOTUp?= Message-ID: <3g3BN075czz7Llb@mail.python.org> http://hg.python.org/cpython/rev/c833c35aa13a changeset: 90177:c833c35aa13a parent: 90174:b1ac067c984f parent: 90175:b2187b82a658 user: Benjamin Peterson date: Tue Apr 08 10:49:09 2014 -0400 summary: merge 3.4 (#16395) files: Doc/c-api/sequence.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/sequence.rst b/Doc/c-api/sequence.rst --- a/Doc/c-api/sequence.rst +++ b/Doc/c-api/sequence.rst @@ -123,10 +123,10 @@ .. c:function:: PyObject* PySequence_Fast(PyObject *o, const char *m) - Returns the sequence *o* as a tuple, unless it is already a tuple or list, in - which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access the - members of the result. Returns *NULL* on failure. If the object is not a - sequence, raises :exc:`TypeError` with *m* as the message text. + Return the sequence *o* as a list, unless it is already a tuple or list, in + which case *o* is returned. Use :c:func:`PySequence_Fast_GET_ITEM` to access + the members of the result. Returns *NULL* on failure. If the object is not + a sequence, raises :exc:`TypeError` with *m* as the message text. .. c:function:: PyObject* PySequence_Fast_GET_ITEM(PyObject *o, Py_ssize_t i) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:51:03 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:51:03 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogUHlTZXF1ZW5jZV9G?= =?utf-8?q?ast_generally_returns_a_list_=28=2316305=29?= Message-ID: <3g3BPz2prnz7LkP@mail.python.org> http://hg.python.org/cpython/rev/3016a9e8061a changeset: 90178:3016a9e8061a branch: 2.7 parent: 90176:b235db467cd5 user: Benjamin Peterson date: Tue Apr 08 10:50:54 2014 -0400 summary: PySequence_Fast generally returns a list (#16305) files: Include/abstract.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -1149,7 +1149,7 @@ PyAPI_FUNC(PyObject *) PySequence_Fast(PyObject *o, const char* m); /* - Returns the sequence, o, as a tuple, unless it's already a + Returns the sequence, o, as a list, unless it's already a tuple or list. Use PySequence_Fast_GET_ITEM to access the members of this list, and PySequence_Fast_GET_SIZE to get its length. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:51:44 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:51:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_use_imperative?= Message-ID: <3g3BQm6KkDz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/9f0e10dc7722 changeset: 90179:9f0e10dc7722 branch: 2.7 user: Benjamin Peterson date: Tue Apr 08 10:51:20 2014 -0400 summary: use imperative files: Include/abstract.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -1149,7 +1149,7 @@ PyAPI_FUNC(PyObject *) PySequence_Fast(PyObject *o, const char* m); /* - Returns the sequence, o, as a list, unless it's already a + Return the sequence, o, as a list, unless it's already a tuple or list. Use PySequence_Fast_GET_ITEM to access the members of this list, and PySequence_Fast_GET_SIZE to get its length. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:51:46 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:51:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_use_imperative?= Message-ID: <3g3BQp1vxxz7LkP@mail.python.org> http://hg.python.org/cpython/rev/e88e82751666 changeset: 90180:e88e82751666 branch: 3.4 parent: 90175:b2187b82a658 user: Benjamin Peterson date: Tue Apr 08 10:51:20 2014 -0400 summary: use imperative files: Include/abstract.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -1021,7 +1021,7 @@ PyAPI_FUNC(PyObject *) PySequence_Fast(PyObject *o, const char* m); /* - Returns the sequence, o, as a list, unless it's already a + Return the sequence, o, as a list, unless it's already a tuple or list. Use PySequence_Fast_GET_ITEM to access the members of this list, and PySequence_Fast_GET_SIZE to get its length. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 16:51:47 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 8 Apr 2014 16:51:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g3BQq4694z7LlR@mail.python.org> http://hg.python.org/cpython/rev/66751c26ae74 changeset: 90181:66751c26ae74 parent: 90177:c833c35aa13a parent: 90180:e88e82751666 user: Benjamin Peterson date: Tue Apr 08 10:51:37 2014 -0400 summary: merge 3.4 files: Include/abstract.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -1021,7 +1021,7 @@ PyAPI_FUNC(PyObject *) PySequence_Fast(PyObject *o, const char* m); /* - Returns the sequence, o, as a list, unless it's already a + Return the sequence, o, as a list, unless it's already a tuple or list. Use PySequence_Fast_GET_ITEM to access the members of this list, and PySequence_Fast_GET_SIZE to get its length. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 17:31:11 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 17:31:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogaW5zcGVjdC5zaWdu?= =?utf-8?q?autre=3A_Fix_functools=2Epartial_support=2E_Issue_=2321117?= Message-ID: <3g3CJH4Mfcz7LpT@mail.python.org> http://hg.python.org/cpython/rev/acbdbf2b06e0 changeset: 90182:acbdbf2b06e0 branch: 3.4 parent: 90180:e88e82751666 user: Yury Selivanov date: Tue Apr 08 11:28:02 2014 -0400 summary: inspect.signautre: Fix functools.partial support. Issue #21117 files: Lib/inspect.py | 135 ++++++++++++-------------- Lib/test/test_inspect.py | 99 +++++++++++++++---- Misc/NEWS | 5 + 3 files changed, 146 insertions(+), 93 deletions(-) diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -1511,7 +1511,8 @@ # look like after applying a 'functools.partial' object (or alike) # on it. - new_params = OrderedDict(wrapped_sig.parameters.items()) + old_params = wrapped_sig.parameters + new_params = OrderedDict(old_params.items()) partial_args = partial.args or () partial_keywords = partial.keywords or {} @@ -1525,32 +1526,57 @@ msg = 'partial object {!r} has incorrect arguments'.format(partial) raise ValueError(msg) from ex - for arg_name, arg_value in ba.arguments.items(): - param = new_params[arg_name] - if arg_name in partial_keywords: - # We set a new default value, because the following code - # is correct: - # - # >>> def foo(a): print(a) - # >>> print(partial(partial(foo, a=10), a=20)()) - # 20 - # >>> print(partial(partial(foo, a=10), a=20)(a=30)) - # 30 - # - # So, with 'partial' objects, passing a keyword argument is - # like setting a new default value for the corresponding - # parameter - # - # We also mark this parameter with '_partial_kwarg' - # flag. Later, in '_bind', the 'default' value of this - # parameter will be added to 'kwargs', to simulate - # the 'functools.partial' real call. - new_params[arg_name] = param.replace(default=arg_value, - _partial_kwarg=True) - - elif (param.kind not in (_VAR_KEYWORD, _VAR_POSITIONAL) and - not param._partial_kwarg): - new_params.pop(arg_name) + + transform_to_kwonly = False + for param_name, param in old_params.items(): + try: + arg_value = ba.arguments[param_name] + except KeyError: + pass + else: + if param.kind is _POSITIONAL_ONLY: + # If positional-only parameter is bound by partial, + # it effectively disappears from the signature + new_params.pop(param_name) + continue + + if param.kind is _POSITIONAL_OR_KEYWORD: + if param_name in partial_keywords: + # This means that this parameter, and all parameters + # after it should be keyword-only (and var-positional + # should be removed). Here's why. Consider the following + # function: + # foo(a, b, *args, c): + # pass + # + # "partial(foo, a='spam')" will have the following + # signature: "(*, a='spam', b, c)". Because attempting + # to call that partial with "(10, 20)" arguments will + # raise a TypeError, saying that "a" argument received + # multiple values. + transform_to_kwonly = True + # Set the new default value + new_params[param_name] = param.replace(default=arg_value) + else: + # was passed as a positional argument + new_params.pop(param.name) + continue + + if param.kind is _KEYWORD_ONLY: + # Set the new default value + new_params[param_name] = param.replace(default=arg_value) + + if transform_to_kwonly: + assert param.kind is not _POSITIONAL_ONLY + + if param.kind is _POSITIONAL_OR_KEYWORD: + new_param = new_params[param_name].replace(kind=_KEYWORD_ONLY) + new_params[param_name] = new_param + new_params.move_to_end(param_name) + elif param.kind in (_KEYWORD_ONLY, _VAR_KEYWORD): + new_params.move_to_end(param_name) + elif param.kind is _VAR_POSITIONAL: + new_params.pop(param.name) return wrapped_sig.replace(parameters=new_params.values()) @@ -2069,7 +2095,7 @@ `Parameter.KEYWORD_ONLY`, `Parameter.VAR_KEYWORD`. ''' - __slots__ = ('_name', '_kind', '_default', '_annotation', '_partial_kwarg') + __slots__ = ('_name', '_kind', '_default', '_annotation') POSITIONAL_ONLY = _POSITIONAL_ONLY POSITIONAL_OR_KEYWORD = _POSITIONAL_OR_KEYWORD @@ -2079,8 +2105,7 @@ empty = _empty - def __init__(self, name, kind, *, default=_empty, annotation=_empty, - _partial_kwarg=False): + def __init__(self, name, kind, *, default=_empty, annotation=_empty): if kind not in (_POSITIONAL_ONLY, _POSITIONAL_OR_KEYWORD, _VAR_POSITIONAL, _KEYWORD_ONLY, _VAR_KEYWORD): @@ -2105,8 +2130,6 @@ self._name = name - self._partial_kwarg = _partial_kwarg - @property def name(self): return self._name @@ -2123,8 +2146,8 @@ def kind(self): return self._kind - def replace(self, *, name=_void, kind=_void, annotation=_void, - default=_void, _partial_kwarg=_void): + def replace(self, *, name=_void, kind=_void, + annotation=_void, default=_void): '''Creates a customized copy of the Parameter.''' if name is _void: @@ -2139,11 +2162,7 @@ if default is _void: default = self._default - if _partial_kwarg is _void: - _partial_kwarg = self._partial_kwarg - - return type(self)(name, kind, default=default, annotation=annotation, - _partial_kwarg=_partial_kwarg) + return type(self)(name, kind, default=default, annotation=annotation) def __str__(self): kind = self.kind @@ -2169,17 +2188,6 @@ id(self), self.name) def __eq__(self, other): - # NB: We deliberately do not compare '_partial_kwarg' attributes - # here. Imagine we have a following situation: - # - # def foo(a, b=1): pass - # def bar(a, b): pass - # bar2 = functools.partial(bar, b=1) - # - # For the above scenario, signatures for `foo` and `bar2` should - # be equal. '_partial_kwarg' attribute is an internal flag, to - # distinguish between keyword parameters with defaults and - # keyword parameters which got their defaults from functools.partial return (issubclass(other.__class__, Parameter) and self._name == other._name and self._kind == other._kind and @@ -2219,12 +2227,7 @@ def args(self): args = [] for param_name, param in self._signature.parameters.items(): - if (param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY) or - param._partial_kwarg): - # Keyword arguments mapped by 'functools.partial' - # (Parameter._partial_kwarg is True) are mapped - # in 'BoundArguments.kwargs', along with VAR_KEYWORD & - # KEYWORD_ONLY + if param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY): break try: @@ -2249,8 +2252,7 @@ kwargs_started = False for param_name, param in self._signature.parameters.items(): if not kwargs_started: - if (param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY) or - param._partial_kwarg): + if param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY): kwargs_started = True else: if param_name not in self.arguments: @@ -2332,18 +2334,14 @@ name = param.name if kind < top_kind: - msg = 'wrong parameter order: {} before {}' + msg = 'wrong parameter order: {!r} before {!r}' msg = msg.format(top_kind, kind) raise ValueError(msg) elif kind > top_kind: kind_defaults = False top_kind = kind - if (kind in (_POSITIONAL_ONLY, _POSITIONAL_OR_KEYWORD) and - not param._partial_kwarg): - # If we have a positional-only or positional-or-keyword - # parameter, that does not have its default value set - # by 'functools.partial' or other "partial" signature: + if kind in (_POSITIONAL_ONLY, _POSITIONAL_OR_KEYWORD): if param.default is _empty: if kind_defaults: # No default for this parameter, but the @@ -2518,15 +2516,6 @@ parameters_ex = () arg_vals = iter(args) - if partial: - # Support for binding arguments to 'functools.partial' objects. - # See 'functools.partial' case in 'signature()' implementation - # for details. - for param_name, param in self.parameters.items(): - if (param._partial_kwarg and param_name not in kwargs): - # Simulating 'functools.partial' behavior - kwargs[param_name] = param.default - while True: # Let's iterate through the positional arguments and corresponding # parameters diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -1959,6 +1959,8 @@ def test_signature_on_partial(self): from functools import partial + Parameter = inspect.Parameter + def test(): pass @@ -1994,15 +1996,22 @@ self.assertEqual(self.signature(partial(test, b=1, c=2)), ((('a', ..., ..., "positional_or_keyword"), - ('b', 1, ..., "positional_or_keyword"), + ('b', 1, ..., "keyword_only"), ('c', 2, ..., "keyword_only"), ('d', ..., ..., "keyword_only")), ...)) self.assertEqual(self.signature(partial(test, 0, b=1, c=2)), - ((('b', 1, ..., "positional_or_keyword"), + ((('b', 1, ..., "keyword_only"), ('c', 2, ..., "keyword_only"), - ('d', ..., ..., "keyword_only"),), + ('d', ..., ..., "keyword_only")), + ...)) + + self.assertEqual(self.signature(partial(test, a=1)), + ((('a', 1, ..., "keyword_only"), + ('b', ..., ..., "keyword_only"), + ('c', ..., ..., "keyword_only"), + ('d', ..., ..., "keyword_only")), ...)) def test(a, *args, b, **kwargs): @@ -2014,13 +2023,18 @@ ('kwargs', ..., ..., "var_keyword")), ...)) + self.assertEqual(self.signature(partial(test, a=1)), + ((('a', 1, ..., "keyword_only"), + ('b', ..., ..., "keyword_only"), + ('kwargs', ..., ..., "var_keyword")), + ...)) + self.assertEqual(self.signature(partial(test, 1, 2, 3)), ((('args', ..., ..., "var_positional"), ('b', ..., ..., "keyword_only"), ('kwargs', ..., ..., "var_keyword")), ...)) - self.assertEqual(self.signature(partial(test, 1, 2, 3, test=True)), ((('args', ..., ..., "var_positional"), ('b', ..., ..., "keyword_only"), @@ -2067,7 +2081,7 @@ return a _foo = partial(partial(foo, a=10), a=20) self.assertEqual(self.signature(_foo), - ((('a', 20, ..., "positional_or_keyword"),), + ((('a', 20, ..., "keyword_only"),), ...)) # check that we don't have any side-effects in signature(), # and the partial object is still functioning @@ -2076,42 +2090,87 @@ def foo(a, b, c): return a, b, c _foo = partial(partial(foo, 1, b=20), b=30) + self.assertEqual(self.signature(_foo), - ((('b', 30, ..., "positional_or_keyword"), - ('c', ..., ..., "positional_or_keyword")), + ((('b', 30, ..., "keyword_only"), + ('c', ..., ..., "keyword_only")), ...)) self.assertEqual(_foo(c=10), (1, 30, 10)) - _foo = partial(_foo, 2) # now 'b' has two values - - # positional and keyword - with self.assertRaisesRegex(ValueError, "has incorrect arguments"): - inspect.signature(_foo) def foo(a, b, c, *, d): return a, b, c, d _foo = partial(partial(foo, d=20, c=20), b=10, d=30) self.assertEqual(self.signature(_foo), ((('a', ..., ..., "positional_or_keyword"), - ('b', 10, ..., "positional_or_keyword"), - ('c', 20, ..., "positional_or_keyword"), - ('d', 30, ..., "keyword_only")), + ('b', 10, ..., "keyword_only"), + ('c', 20, ..., "keyword_only"), + ('d', 30, ..., "keyword_only"), + ), ...)) ba = inspect.signature(_foo).bind(a=200, b=11) self.assertEqual(_foo(*ba.args, **ba.kwargs), (200, 11, 20, 30)) def foo(a=1, b=2, c=3): return a, b, c - _foo = partial(foo, a=10, c=13) - ba = inspect.signature(_foo).bind(11) + _foo = partial(foo, c=13) # (a=1, b=2, *, c=13) + + ba = inspect.signature(_foo).bind(a=11) self.assertEqual(_foo(*ba.args, **ba.kwargs), (11, 2, 13)) + ba = inspect.signature(_foo).bind(11, 12) self.assertEqual(_foo(*ba.args, **ba.kwargs), (11, 12, 13)) + ba = inspect.signature(_foo).bind(11, b=12) self.assertEqual(_foo(*ba.args, **ba.kwargs), (11, 12, 13)) + ba = inspect.signature(_foo).bind(b=12) - self.assertEqual(_foo(*ba.args, **ba.kwargs), (10, 12, 13)) - _foo = partial(_foo, b=10) - ba = inspect.signature(_foo).bind(12, 14) - self.assertEqual(_foo(*ba.args, **ba.kwargs), (12, 14, 13)) + self.assertEqual(_foo(*ba.args, **ba.kwargs), (1, 12, 13)) + + _foo = partial(_foo, b=10, c=20) + ba = inspect.signature(_foo).bind(12) + self.assertEqual(_foo(*ba.args, **ba.kwargs), (12, 10, 20)) + + + def foo(a, b, c, d, **kwargs): + pass + sig = inspect.signature(foo) + params = sig.parameters.copy() + params['a'] = params['a'].replace(kind=Parameter.POSITIONAL_ONLY) + params['b'] = params['b'].replace(kind=Parameter.POSITIONAL_ONLY) + foo.__signature__ = inspect.Signature(params.values()) + sig = inspect.signature(foo) + self.assertEqual(str(sig), '(a, b, /, c, d, **kwargs)') + + self.assertEqual(self.signature(partial(foo, 1)), + ((('b', ..., ..., 'positional_only'), + ('c', ..., ..., 'positional_or_keyword'), + ('d', ..., ..., 'positional_or_keyword'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, 2)), + ((('c', ..., ..., 'positional_or_keyword'), + ('d', ..., ..., 'positional_or_keyword'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, 2, 3)), + ((('d', ..., ..., 'positional_or_keyword'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, 2, c=3)), + ((('c', 3, ..., 'keyword_only'), + ('d', ..., ..., 'keyword_only'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, c=3)), + ((('b', ..., ..., 'positional_only'), + ('c', 3, ..., 'keyword_only'), + ('d', ..., ..., 'keyword_only'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) def test_signature_on_partialmethod(self): from functools import partialmethod diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -106,6 +106,11 @@ (Original patches by Hirokazu Yamamoto and Amaury Forgeot d'Arc, with suggested wording by David Gutteridge) +- Issue #21117: Fix inspect.signature to better support functools.partial. + Due to the specifics of functools.partial implementation, + positional-or-keyword arguments passed as keyword arguments become + keyword-only. + IDLE ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 17:31:13 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 17:31:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_inspect=2Esignautre=3A_Fix?= =?utf-8?q?_functools=2Epartial_support=2E_Issue_=2321117?= Message-ID: <3g3CJK0tmRz7Ln5@mail.python.org> http://hg.python.org/cpython/rev/21709cb4a8f5 changeset: 90183:21709cb4a8f5 parent: 90181:66751c26ae74 user: Yury Selivanov date: Tue Apr 08 11:30:45 2014 -0400 summary: inspect.signautre: Fix functools.partial support. Issue #21117 files: Lib/inspect.py | 139 ++++++++++++-------------- Lib/test/test_inspect.py | 101 +++++++++++++++---- Misc/NEWS | 5 + 3 files changed, 147 insertions(+), 98 deletions(-) diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -1518,7 +1518,8 @@ on it. """ - new_params = OrderedDict(wrapped_sig.parameters.items()) + old_params = wrapped_sig.parameters + new_params = OrderedDict(old_params.items()) partial_args = partial.args or () partial_keywords = partial.keywords or {} @@ -1532,32 +1533,57 @@ msg = 'partial object {!r} has incorrect arguments'.format(partial) raise ValueError(msg) from ex - for arg_name, arg_value in ba.arguments.items(): - param = new_params[arg_name] - if arg_name in partial_keywords: - # We set a new default value, because the following code - # is correct: - # - # >>> def foo(a): print(a) - # >>> print(partial(partial(foo, a=10), a=20)()) - # 20 - # >>> print(partial(partial(foo, a=10), a=20)(a=30)) - # 30 - # - # So, with 'partial' objects, passing a keyword argument is - # like setting a new default value for the corresponding - # parameter - # - # We also mark this parameter with '_partial_kwarg' - # flag. Later, in '_bind', the 'default' value of this - # parameter will be added to 'kwargs', to simulate - # the 'functools.partial' real call. - new_params[arg_name] = param.replace(default=arg_value, - _partial_kwarg=True) - - elif (param.kind not in (_VAR_KEYWORD, _VAR_POSITIONAL) and - not param._partial_kwarg): - new_params.pop(arg_name) + + transform_to_kwonly = False + for param_name, param in old_params.items(): + try: + arg_value = ba.arguments[param_name] + except KeyError: + pass + else: + if param.kind is _POSITIONAL_ONLY: + # If positional-only parameter is bound by partial, + # it effectively disappears from the signature + new_params.pop(param_name) + continue + + if param.kind is _POSITIONAL_OR_KEYWORD: + if param_name in partial_keywords: + # This means that this parameter, and all parameters + # after it should be keyword-only (and var-positional + # should be removed). Here's why. Consider the following + # function: + # foo(a, b, *args, c): + # pass + # + # "partial(foo, a='spam')" will have the following + # signature: "(*, a='spam', b, c)". Because attempting + # to call that partial with "(10, 20)" arguments will + # raise a TypeError, saying that "a" argument received + # multiple values. + transform_to_kwonly = True + # Set the new default value + new_params[param_name] = param.replace(default=arg_value) + else: + # was passed as a positional argument + new_params.pop(param.name) + continue + + if param.kind is _KEYWORD_ONLY: + # Set the new default value + new_params[param_name] = param.replace(default=arg_value) + + if transform_to_kwonly: + assert param.kind is not _POSITIONAL_ONLY + + if param.kind is _POSITIONAL_OR_KEYWORD: + new_param = new_params[param_name].replace(kind=_KEYWORD_ONLY) + new_params[param_name] = new_param + new_params.move_to_end(param_name) + elif param.kind in (_KEYWORD_ONLY, _VAR_KEYWORD): + new_params.move_to_end(param_name) + elif param.kind is _VAR_POSITIONAL: + new_params.pop(param.name) return wrapped_sig.replace(parameters=new_params.values()) @@ -2103,7 +2129,7 @@ `Parameter.KEYWORD_ONLY`, `Parameter.VAR_KEYWORD`. """ - __slots__ = ('_name', '_kind', '_default', '_annotation', '_partial_kwarg') + __slots__ = ('_name', '_kind', '_default', '_annotation') POSITIONAL_ONLY = _POSITIONAL_ONLY POSITIONAL_OR_KEYWORD = _POSITIONAL_OR_KEYWORD @@ -2113,8 +2139,7 @@ empty = _empty - def __init__(self, name, kind, *, default=_empty, annotation=_empty, - _partial_kwarg=False): + def __init__(self, name, kind, *, default=_empty, annotation=_empty): if kind not in (_POSITIONAL_ONLY, _POSITIONAL_OR_KEYWORD, _VAR_POSITIONAL, _KEYWORD_ONLY, _VAR_KEYWORD): @@ -2139,17 +2164,13 @@ self._name = name - self._partial_kwarg = _partial_kwarg - def __reduce__(self): return (type(self), (self._name, self._kind), - {'_partial_kwarg': self._partial_kwarg, - '_default': self._default, + {'_default': self._default, '_annotation': self._annotation}) def __setstate__(self, state): - self._partial_kwarg = state['_partial_kwarg'] self._default = state['_default'] self._annotation = state['_annotation'] @@ -2169,8 +2190,8 @@ def kind(self): return self._kind - def replace(self, *, name=_void, kind=_void, annotation=_void, - default=_void, _partial_kwarg=_void): + def replace(self, *, name=_void, kind=_void, + annotation=_void, default=_void): """Creates a customized copy of the Parameter.""" if name is _void: @@ -2185,11 +2206,7 @@ if default is _void: default = self._default - if _partial_kwarg is _void: - _partial_kwarg = self._partial_kwarg - - return type(self)(name, kind, default=default, annotation=annotation, - _partial_kwarg=_partial_kwarg) + return type(self)(name, kind, default=default, annotation=annotation) def __str__(self): kind = self.kind @@ -2215,17 +2232,6 @@ id(self), self) def __eq__(self, other): - # NB: We deliberately do not compare '_partial_kwarg' attributes - # here. Imagine we have a following situation: - # - # def foo(a, b=1): pass - # def bar(a, b): pass - # bar2 = functools.partial(bar, b=1) - # - # For the above scenario, signatures for `foo` and `bar2` should - # be equal. '_partial_kwarg' attribute is an internal flag, to - # distinguish between keyword parameters with defaults and - # keyword parameters which got their defaults from functools.partial return (issubclass(other.__class__, Parameter) and self._name == other._name and self._kind == other._kind and @@ -2265,12 +2271,7 @@ def args(self): args = [] for param_name, param in self._signature.parameters.items(): - if (param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY) or - param._partial_kwarg): - # Keyword arguments mapped by 'functools.partial' - # (Parameter._partial_kwarg is True) are mapped - # in 'BoundArguments.kwargs', along with VAR_KEYWORD & - # KEYWORD_ONLY + if param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY): break try: @@ -2295,8 +2296,7 @@ kwargs_started = False for param_name, param in self._signature.parameters.items(): if not kwargs_started: - if (param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY) or - param._partial_kwarg): + if param.kind in (_VAR_KEYWORD, _KEYWORD_ONLY): kwargs_started = True else: if param_name not in self.arguments: @@ -2378,18 +2378,14 @@ name = param.name if kind < top_kind: - msg = 'wrong parameter order: {} before {}' + msg = 'wrong parameter order: {!r} before {!r}' msg = msg.format(top_kind, kind) raise ValueError(msg) elif kind > top_kind: kind_defaults = False top_kind = kind - if (kind in (_POSITIONAL_ONLY, _POSITIONAL_OR_KEYWORD) and - not param._partial_kwarg): - # If we have a positional-only or positional-or-keyword - # parameter, that does not have its default value set - # by 'functools.partial' or other "partial" signature: + if kind in (_POSITIONAL_ONLY, _POSITIONAL_OR_KEYWORD): if param.default is _empty: if kind_defaults: # No default for this parameter, but the @@ -2570,15 +2566,6 @@ parameters_ex = () arg_vals = iter(args) - if partial: - # Support for binding arguments to 'functools.partial' objects. - # See 'functools.partial' case in 'signature()' implementation - # for details. - for param_name, param in self.parameters.items(): - if (param._partial_kwarg and param_name not in kwargs): - # Simulating 'functools.partial' behavior - kwargs[param_name] = param.default - while True: # Let's iterate through the positional arguments and corresponding # parameters diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -1689,13 +1689,11 @@ foo_partial = functools.partial(foo, a=1) sig = inspect.signature(foo_partial) - self.assertTrue(sig.parameters['a']._partial_kwarg) for ver in range(pickle.HIGHEST_PROTOCOL + 1): with self.subTest(pickle_ver=ver, subclass=False): sig_pickled = pickle.loads(pickle.dumps(sig, ver)) self.assertEqual(sig, sig_pickled) - self.assertTrue(sig_pickled.parameters['a']._partial_kwarg) # Test that basic sub-classing works sig = inspect.signature(foo) @@ -2005,6 +2003,8 @@ def test_signature_on_partial(self): from functools import partial + Parameter = inspect.Parameter + def test(): pass @@ -2040,15 +2040,22 @@ self.assertEqual(self.signature(partial(test, b=1, c=2)), ((('a', ..., ..., "positional_or_keyword"), - ('b', 1, ..., "positional_or_keyword"), + ('b', 1, ..., "keyword_only"), ('c', 2, ..., "keyword_only"), ('d', ..., ..., "keyword_only")), ...)) self.assertEqual(self.signature(partial(test, 0, b=1, c=2)), - ((('b', 1, ..., "positional_or_keyword"), + ((('b', 1, ..., "keyword_only"), ('c', 2, ..., "keyword_only"), - ('d', ..., ..., "keyword_only"),), + ('d', ..., ..., "keyword_only")), + ...)) + + self.assertEqual(self.signature(partial(test, a=1)), + ((('a', 1, ..., "keyword_only"), + ('b', ..., ..., "keyword_only"), + ('c', ..., ..., "keyword_only"), + ('d', ..., ..., "keyword_only")), ...)) def test(a, *args, b, **kwargs): @@ -2060,13 +2067,18 @@ ('kwargs', ..., ..., "var_keyword")), ...)) + self.assertEqual(self.signature(partial(test, a=1)), + ((('a', 1, ..., "keyword_only"), + ('b', ..., ..., "keyword_only"), + ('kwargs', ..., ..., "var_keyword")), + ...)) + self.assertEqual(self.signature(partial(test, 1, 2, 3)), ((('args', ..., ..., "var_positional"), ('b', ..., ..., "keyword_only"), ('kwargs', ..., ..., "var_keyword")), ...)) - self.assertEqual(self.signature(partial(test, 1, 2, 3, test=True)), ((('args', ..., ..., "var_positional"), ('b', ..., ..., "keyword_only"), @@ -2113,7 +2125,7 @@ return a _foo = partial(partial(foo, a=10), a=20) self.assertEqual(self.signature(_foo), - ((('a', 20, ..., "positional_or_keyword"),), + ((('a', 20, ..., "keyword_only"),), ...)) # check that we don't have any side-effects in signature(), # and the partial object is still functioning @@ -2122,42 +2134,87 @@ def foo(a, b, c): return a, b, c _foo = partial(partial(foo, 1, b=20), b=30) + self.assertEqual(self.signature(_foo), - ((('b', 30, ..., "positional_or_keyword"), - ('c', ..., ..., "positional_or_keyword")), + ((('b', 30, ..., "keyword_only"), + ('c', ..., ..., "keyword_only")), ...)) self.assertEqual(_foo(c=10), (1, 30, 10)) - _foo = partial(_foo, 2) # now 'b' has two values - - # positional and keyword - with self.assertRaisesRegex(ValueError, "has incorrect arguments"): - inspect.signature(_foo) def foo(a, b, c, *, d): return a, b, c, d _foo = partial(partial(foo, d=20, c=20), b=10, d=30) self.assertEqual(self.signature(_foo), ((('a', ..., ..., "positional_or_keyword"), - ('b', 10, ..., "positional_or_keyword"), - ('c', 20, ..., "positional_or_keyword"), - ('d', 30, ..., "keyword_only")), + ('b', 10, ..., "keyword_only"), + ('c', 20, ..., "keyword_only"), + ('d', 30, ..., "keyword_only"), + ), ...)) ba = inspect.signature(_foo).bind(a=200, b=11) self.assertEqual(_foo(*ba.args, **ba.kwargs), (200, 11, 20, 30)) def foo(a=1, b=2, c=3): return a, b, c - _foo = partial(foo, a=10, c=13) - ba = inspect.signature(_foo).bind(11) + _foo = partial(foo, c=13) # (a=1, b=2, *, c=13) + + ba = inspect.signature(_foo).bind(a=11) self.assertEqual(_foo(*ba.args, **ba.kwargs), (11, 2, 13)) + ba = inspect.signature(_foo).bind(11, 12) self.assertEqual(_foo(*ba.args, **ba.kwargs), (11, 12, 13)) + ba = inspect.signature(_foo).bind(11, b=12) self.assertEqual(_foo(*ba.args, **ba.kwargs), (11, 12, 13)) + ba = inspect.signature(_foo).bind(b=12) - self.assertEqual(_foo(*ba.args, **ba.kwargs), (10, 12, 13)) - _foo = partial(_foo, b=10) - ba = inspect.signature(_foo).bind(12, 14) - self.assertEqual(_foo(*ba.args, **ba.kwargs), (12, 14, 13)) + self.assertEqual(_foo(*ba.args, **ba.kwargs), (1, 12, 13)) + + _foo = partial(_foo, b=10, c=20) + ba = inspect.signature(_foo).bind(12) + self.assertEqual(_foo(*ba.args, **ba.kwargs), (12, 10, 20)) + + + def foo(a, b, c, d, **kwargs): + pass + sig = inspect.signature(foo) + params = sig.parameters.copy() + params['a'] = params['a'].replace(kind=Parameter.POSITIONAL_ONLY) + params['b'] = params['b'].replace(kind=Parameter.POSITIONAL_ONLY) + foo.__signature__ = inspect.Signature(params.values()) + sig = inspect.signature(foo) + self.assertEqual(str(sig), '(a, b, /, c, d, **kwargs)') + + self.assertEqual(self.signature(partial(foo, 1)), + ((('b', ..., ..., 'positional_only'), + ('c', ..., ..., 'positional_or_keyword'), + ('d', ..., ..., 'positional_or_keyword'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, 2)), + ((('c', ..., ..., 'positional_or_keyword'), + ('d', ..., ..., 'positional_or_keyword'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, 2, 3)), + ((('d', ..., ..., 'positional_or_keyword'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, 2, c=3)), + ((('c', 3, ..., 'keyword_only'), + ('d', ..., ..., 'keyword_only'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) + + self.assertEqual(self.signature(partial(foo, 1, c=3)), + ((('b', ..., ..., 'positional_only'), + ('c', 3, ..., 'keyword_only'), + ('d', ..., ..., 'keyword_only'), + ('kwargs', ..., ..., 'var_keyword')), + ...)) def test_signature_on_partialmethod(self): from functools import partialmethod diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -149,6 +149,11 @@ (Original patches by Hirokazu Yamamoto and Amaury Forgeot d'Arc, with suggested wording by David Gutteridge) +- Issue #21117: Fix inspect.signature to better support functools.partial. + Due to the specifics of functools.partial implementation, + positional-or-keyword arguments passed as keyword arguments become + keyword-only. + IDLE ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 17:47:12 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 17:47:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_inspect=3A_Make_Signature_?= =?utf-8?q?and_Parameter_hashable=2E_Issue_=2320334=2E?= Message-ID: <3g3Cfm0975z7LpW@mail.python.org> http://hg.python.org/cpython/rev/932d69ef0c63 changeset: 90184:932d69ef0c63 user: Yury Selivanov date: Tue Apr 08 11:46:50 2014 -0400 summary: inspect: Make Signature and Parameter hashable. Issue #20334. files: Doc/library/inspect.rst | 4 +- Doc/whatsnew/3.5.rst | 3 +- Lib/inspect.py | 16 +++++++++++ Lib/test/test_inspect.py | 40 +++++++++++++++++++++------- Misc/NEWS | 2 + 5 files changed, 52 insertions(+), 13 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -463,7 +463,7 @@ modified copy. .. versionchanged:: 3.5 - Signature objects are picklable. + Signature objects are picklable and hashable. .. attribute:: Signature.empty @@ -530,7 +530,7 @@ you can use :meth:`Parameter.replace` to create a modified copy. .. versionchanged:: 3.5 - Parameter objects are picklable. + Parameter objects are picklable and hashable. .. attribute:: Parameter.empty diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -143,7 +143,8 @@ (contributed by Claudiu Popa in :issue:`20627`). * :class:`inspect.Signature` and :class:`inspect.Parameter` are now - picklable (contributed by Yury Selivanov in :issue:`20726`). + picklable and hashable (contributed by Yury Selivanov in :issue:`20726` + and :issue:`20334`). * New class method :meth:`inspect.Signature.from_callable`, which makes subclassing of :class:`~inspect.Signature` easier (contributed diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -2231,6 +2231,16 @@ return '<{} at {:#x} "{}">'.format(self.__class__.__name__, id(self), self) + def __hash__(self): + hash_tuple = (self.name, int(self.kind)) + + if self._annotation is not _empty: + hash_tuple += (self._annotation,) + if self._default is not _empty: + hash_tuple += (self._default,) + + return hash(hash_tuple) + def __eq__(self, other): return (issubclass(other.__class__, Parameter) and self._name == other._name and @@ -2524,6 +2534,12 @@ return type(self)(parameters, return_annotation=return_annotation) + def __hash__(self): + hash_tuple = tuple(self.parameters.values()) + if self._return_annotation is not _empty: + hash_tuple += (self._return_annotation,) + return hash(hash_tuple) + def __eq__(self, other): if (not issubclass(type(other), Signature) or self.return_annotation != other.return_annotation or diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -2513,11 +2513,29 @@ def bar(pos, *args, c, b, a=42, **kwargs:int): pass self.assertEqual(inspect.signature(foo), inspect.signature(bar)) - def test_signature_unhashable(self): + def test_signature_hashable(self): + S = inspect.Signature + P = inspect.Parameter + def foo(a): pass - sig = inspect.signature(foo) + foo_sig = inspect.signature(foo) + + manual_sig = S(parameters=[P('a', P.POSITIONAL_OR_KEYWORD)]) + + self.assertEqual(hash(foo_sig), hash(manual_sig)) + self.assertNotEqual(hash(foo_sig), + hash(manual_sig.replace(return_annotation='spam'))) + + def bar(a) -> 1: pass + self.assertNotEqual(hash(foo_sig), hash(inspect.signature(bar))) + + def foo(a={}): pass with self.assertRaisesRegex(TypeError, 'unhashable type'): - hash(sig) + hash(inspect.signature(foo)) + + def foo(a) -> {}: pass + with self.assertRaisesRegex(TypeError, 'unhashable type'): + hash(inspect.signature(foo)) def test_signature_str(self): def foo(a:int=1, *, b, c=None, **kwargs) -> 42: @@ -2651,6 +2669,15 @@ self.assertTrue(repr(p).startswith(' http://hg.python.org/cpython/rev/0973d45197cc changeset: 90185:0973d45197cc branch: 3.4 parent: 90182:acbdbf2b06e0 user: Yury Selivanov date: Tue Apr 08 12:03:07 2014 -0400 summary: docs: Document __objclass__. Closes #19281. Initial patch by Nick Coghlan files: Doc/reference/datamodel.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1467,6 +1467,14 @@ Called to delete the attribute on an instance *instance* of the owner class. +The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as +specifying the class where this object was defined (setting this appropriately +can assist in runtime introspection of dynamic class attributes). For callables, +it may indicate that an instance of the given type (or a subclass) is expected +or required as the first positional argument (for example, CPython sets this +attribute for unbound methods that are implemented in C). + + .. _descriptor-invocation: Invoking Descriptors -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 18:07:32 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 18:07:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_docs=3A_Document_=5F=5Fobj?= =?utf-8?b?Y2xhc3NfXy4gQ2xvc2VzICMxOTI4MS4=?= Message-ID: <3g3D6D3mDCz7Ln2@mail.python.org> http://hg.python.org/cpython/rev/2a953cb5642d changeset: 90186:2a953cb5642d parent: 90184:932d69ef0c63 user: Yury Selivanov date: Tue Apr 08 12:04:04 2014 -0400 summary: docs: Document __objclass__. Closes #19281. Initial patch by Nick Coghlan files: Doc/reference/datamodel.rst | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1467,6 +1467,14 @@ Called to delete the attribute on an instance *instance* of the owner class. +The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as +specifying the class where this object was defined (setting this appropriately +can assist in runtime introspection of dynamic class attributes). For callables, +it may indicate that an instance of the given type (or a subclass) is expected +or required as the first positional argument (for example, CPython sets this +attribute for unbound methods that are implemented in C). + + .. _descriptor-invocation: Invoking Descriptors -- Repository URL: http://hg.python.org/cpython From ethan at stoneleaf.us Tue Apr 8 18:33:30 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 08 Apr 2014 09:33:30 -0700 Subject: [Python-checkins] cpython (3.4): docs: Document __objclass__. Closes #19281. In-Reply-To: <3g3D6C20xQz7Lnb@mail.python.org> References: <3g3D6C20xQz7Lnb@mail.python.org> Message-ID: <534424DA.5080702@stoneleaf.us> On 04/08/2014 09:07 AM, yury.selivanov wrote: > http://hg.python.org/cpython/rev/0973d45197cc > > +The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as > +specifying the class where this object was defined (setting this appropriately > +can assist in runtime introspection of dynamic class attributes). For callables, > +it may indicate that an instance of the given type (or a subclass) is expected > +or required as the first positional argument (for example, CPython sets this > +attribute for unbound methods that are implemented in C). This would read better as The attribute :attr:`__objclass__` . . . -- ~Ethan~ From python-checkins at python.org Tue Apr 8 20:01:20 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 20:01:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogZG9jczogQmV0dGVy?= =?utf-8?q?_wording_for_=5F=5Fobjclass=5F=5F_docs=2E_Issue_=2319281?= Message-ID: <3g3GdX2Yt9z7Lnv@mail.python.org> http://hg.python.org/cpython/rev/372b19005011 changeset: 90187:372b19005011 branch: 3.4 parent: 90185:0973d45197cc user: Yury Selivanov date: Tue Apr 08 14:00:35 2014 -0400 summary: docs: Better wording for __objclass__ docs. Issue #19281 files: Doc/reference/datamodel.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1467,12 +1467,12 @@ Called to delete the attribute on an instance *instance* of the owner class. -The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as -specifying the class where this object was defined (setting this appropriately -can assist in runtime introspection of dynamic class attributes). For callables, -it may indicate that an instance of the given type (or a subclass) is expected -or required as the first positional argument (for example, CPython sets this -attribute for unbound methods that are implemented in C). +The attribute :attr:`__objclass__` is interpreted by the :mod:`inspect` module +as specifying the class where this object was defined (setting this +appropriately can assist in runtime introspection of dynamic class attributes). +For callables, it may indicate that an instance of the given type (or a +subclass) is expected or required as the first positional argument (for example, +CPython sets this attribute for unbound methods that are implemented in C). .. _descriptor-invocation: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 20:01:21 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 20:01:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_docs=3A_Better_wording_for?= =?utf-8?q?_=5F=5Fobjclass=5F=5F_docs=2E_Issue_=2319281?= Message-ID: <3g3GdY4JKcz7Lnr@mail.python.org> http://hg.python.org/cpython/rev/febd63a7e927 changeset: 90188:febd63a7e927 parent: 90186:2a953cb5642d user: Yury Selivanov date: Tue Apr 08 14:01:12 2014 -0400 summary: docs: Better wording for __objclass__ docs. Issue #19281 files: Doc/reference/datamodel.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1467,12 +1467,12 @@ Called to delete the attribute on an instance *instance* of the owner class. -The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as -specifying the class where this object was defined (setting this appropriately -can assist in runtime introspection of dynamic class attributes). For callables, -it may indicate that an instance of the given type (or a subclass) is expected -or required as the first positional argument (for example, CPython sets this -attribute for unbound methods that are implemented in C). +The attribute :attr:`__objclass__` is interpreted by the :mod:`inspect` module +as specifying the class where this object was defined (setting this +appropriately can assist in runtime introspection of dynamic class attributes). +For callables, it may indicate that an instance of the given type (or a +subclass) is expected or required as the first positional argument (for example, +CPython sets this attribute for unbound methods that are implemented in C). .. _descriptor-invocation: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 8 21:00:26 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 8 Apr 2014 21:00:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy40?= Message-ID: <3g3Hxk0l5JzRSm@mail.python.org> http://hg.python.org/cpython/rev/ebe18b90618c changeset: 90189:ebe18b90618c parent: 90188:febd63a7e927 parent: 90187:372b19005011 user: Yury Selivanov date: Tue Apr 08 14:58:35 2014 -0400 summary: Merge 3.4 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 02:48:15 2014 From: python-checkins at python.org (ned.deily) Date: Wed, 9 Apr 2014 02:48:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNjQ0?= =?utf-8?q?=3A_OS_X_installer_build_support_for_documentation_build_change?= =?utf-8?q?s?= Message-ID: <3g3Rg32VVmz7LkL@mail.python.org> http://hg.python.org/cpython/rev/01f1e14cad23 changeset: 90190:01f1e14cad23 branch: 3.4 parent: 90187:372b19005011 user: Ned Deily date: Mon Apr 07 12:10:21 2014 -0700 summary: Issue #20644: OS X installer build support for documentation build changes in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. files: Mac/BuildScript/README.txt | 44 ++++++++++------- Mac/BuildScript/build-installer.py | 11 +++- Misc/NEWS | 6 ++ 3 files changed, 40 insertions(+), 21 deletions(-) diff --git a/Mac/BuildScript/README.txt b/Mac/BuildScript/README.txt --- a/Mac/BuildScript/README.txt +++ b/Mac/BuildScript/README.txt @@ -12,9 +12,9 @@ for each release. 1. 32-bit-only, i386 and PPC universal, capable on running on all machines - supported by Mac OS X 10.5 through (at least) 10.8:: + supported by Mac OS X 10.5 through (at least) 10.9:: - /usr/bin/python build-installer.py \ + /path/to/bootstrap/python2.7 build-installer.py \ --sdk-path=/Developer/SDKs/MacOSX10.5.sdk \ --universal-archs=32-bit \ --dep-target=10.5 @@ -22,14 +22,14 @@ - builds the following third-party libraries * NCurses 5.9 (http://bugs.python.org/issue15037) - * SQLite 3.8.1 + * SQLite 3.8.3.1 * XZ 5.0.5 - uses system-supplied versions of third-party libraries * readline module links with Apple BSD editline (libedit) - - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.19) to be installed for building + - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.20) to be installed for building - recommended build environment: @@ -38,7 +38,8 @@ * ``MacOSX10.5`` SDK * ``MACOSX_DEPLOYMENT_TARGET=10.5`` * Apple ``gcc-4.2`` - * system Python 2.5 for documentation build with Sphinx + * bootstrap non-framework Python 2.7 for documentation build with + Sphinx (as of 3.4.1) - alternate build environments: @@ -48,7 +49,7 @@ 2. 64-bit / 32-bit, x86_64 and i386 universal, for OS X 10.6 (and later):: - /usr/bin/python build-installer.py \ + /path/to/bootstrap/python2.7 build-installer.py \ --sdk-path=/Developer/SDKs/MacOSX10.6.sdk \ --universal-archs=intel \ --dep-target=10.6 @@ -56,14 +57,14 @@ - builds the following third-party libraries * NCurses 5.9 (http://bugs.python.org/issue15037) - * SQLite 3.8.1 + * SQLite 3.8.3.1 * XZ 5.0.5 - uses system-supplied versions of third-party libraries * readline module links with Apple BSD editline (libedit) - - requires ActiveState Tcl/Tk 8.5.15 (or later) to be installed for building + - requires ActiveState Tcl/Tk 8.5.15.1 (or later) to be installed for building - recommended build environment: @@ -72,7 +73,8 @@ * ``MacOSX10.6`` SDK * ``MACOSX_DEPLOYMENT_TARGET=10.6`` * Apple ``gcc-4.2`` - * system Python 2.6 for documentation build with Sphinx + * bootstrap non-framework Python 2.7 for documentation build with + Sphinx (as of 3.4.1) - alternate build environments: @@ -82,10 +84,10 @@ considered a migration aid by Apple and is not likely to be fixed, its use should be avoided. The other compiler, ``clang``, has been undergoing rapid development. While it appears to have become - production-ready in the most recent Xcode 4 releases (Xcode 4.6.3 - as of this writing), there are still some open issues when - building Python and there has not yet been the level of exposure in - production environments that the Xcode 3 gcc-4.2 compiler has had. + production-ready in the most recent Xcode 5 releases, the versions + available on the deprecated Xcode 4.x for 10.6 were early releases + and did not receive the level of exposure in production environments + that the Xcode 3 gcc-4.2 compiler has had. * For Python 2.7.x and 3.2.x, the 32-bit-only installer was configured to @@ -111,7 +113,7 @@ * Zlib 1.2.3 * Oracle Sleepycat DB 4.8 (Python 2.x only) - - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.19) to be installed for building + - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.20) to be installed for building - recommended build environment: @@ -137,7 +139,13 @@ interfere with the build. * The documentation for the release is built using Sphinx - because it is included in the installer. + because it is included in the installer. For 2.7.x and 3.x.x up to and + including 3.4.0, the ``Doc/Makefile`` uses ``svn`` to download repos of + ``Sphinx`` and its dependencies. Beginning with 3.4.1, the ``Doc/Makefile`` + assumes there is an externally-provided ``sphinx-build`` and requires at + least Python 2.6 to run. Because of this, it is no longer possible to + build a 3.4.1 or later installer on OS X 10.5 using the Apple-supplied + Python 2.5. * It is safest to start each variant build with an empty source directory populated with a fresh copy of the untarred source. @@ -195,8 +203,8 @@ on all supported OS X releases and architectures. As a practical matter, that is generally not possible. At a minimum, variant 1 should be run on a PPC G4 system with OS X 10.5 and at least one Intel system running OS X -10.8, 10.7, 10.6, or 10.5. Variant 2 should be run on 10.8, 10.7, and 10.6 -systems in both 32-bit and 64-bit modes.:: +10.9, 10.8, 10.7, 10.6, or 10.5. Variant 2 should be run on 10.9, 10.8, +10.7, and 10.6 systems in both 32-bit and 64-bit modes.:: /usr/local/bin/pythonn.n -m test -w -u all,-largefile /usr/local/bin/pythonn.n-32 -m test -w -u all @@ -207,7 +215,7 @@ during testing and can be ignored. It is also recommend to launch IDLE and verify that it is at least -functional. Double-click on the IDLE app icon in ``/Applications/Pythonn.n``. +functional. Double-click on the IDLE app icon in ``/Applications/Python n.n``. It should also be tested from the command line:: /usr/local/bin/idlen.n diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -194,7 +194,7 @@ LT_10_5 = bool(DEPTARGET < '10.5') # Disable for now - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 5)): + if False: # if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 5)): result.extend([ dict( name="Tcl 8.5.15", @@ -607,6 +607,10 @@ base_path = base_path + ':' + OLD_DEVELOPER_TOOLS os.environ['PATH'] = base_path print("Setting default PATH: %s"%(os.environ['PATH'])) + # Ensure ws have access to hg and to sphinx-build. + # You may have to create links in /usr/bin for them. + runCommand('hg --version') + runCommand('sphinx-build --version') def parseOptions(args=None): @@ -920,8 +924,9 @@ docdir = os.path.join(rootDir, 'pydocs') curDir = os.getcwd() os.chdir(buildDir) - runCommand('make update') - runCommand("make html PYTHON='%s'" % os.path.abspath(sys.executable)) + runCommand('make clean') + # Assume sphinx-build is on our PATH, checked in checkEnvironment + runCommand('make html') os.chdir(curDir) if not os.path.exists(docdir): os.mkdir(docdir) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -117,6 +117,12 @@ - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. +Build +----- + +- Issue #20644: OS X installer build support for documentation build changes + in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. + C API ----- - Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__ to -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 02:48:16 2014 From: python-checkins at python.org (ned.deily) Date: Wed, 9 Apr 2014 02:48:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320644=3A_merge_from_3=2E4?= Message-ID: <3g3Rg450RZz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/7d004ad09bf5 changeset: 90191:7d004ad09bf5 parent: 90189:ebe18b90618c parent: 90190:01f1e14cad23 user: Ned Deily date: Tue Apr 08 17:47:48 2014 -0700 summary: Issue #20644: merge from 3.4 files: Mac/BuildScript/README.txt | 44 ++++++++++------- Mac/BuildScript/build-installer.py | 11 +++- Misc/NEWS | 6 ++ 3 files changed, 40 insertions(+), 21 deletions(-) diff --git a/Mac/BuildScript/README.txt b/Mac/BuildScript/README.txt --- a/Mac/BuildScript/README.txt +++ b/Mac/BuildScript/README.txt @@ -12,9 +12,9 @@ for each release. 1. 32-bit-only, i386 and PPC universal, capable on running on all machines - supported by Mac OS X 10.5 through (at least) 10.8:: + supported by Mac OS X 10.5 through (at least) 10.9:: - /usr/bin/python build-installer.py \ + /path/to/bootstrap/python2.7 build-installer.py \ --sdk-path=/Developer/SDKs/MacOSX10.5.sdk \ --universal-archs=32-bit \ --dep-target=10.5 @@ -22,14 +22,14 @@ - builds the following third-party libraries * NCurses 5.9 (http://bugs.python.org/issue15037) - * SQLite 3.8.1 + * SQLite 3.8.3.1 * XZ 5.0.5 - uses system-supplied versions of third-party libraries * readline module links with Apple BSD editline (libedit) - - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.19) to be installed for building + - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.20) to be installed for building - recommended build environment: @@ -38,7 +38,8 @@ * ``MacOSX10.5`` SDK * ``MACOSX_DEPLOYMENT_TARGET=10.5`` * Apple ``gcc-4.2`` - * system Python 2.5 for documentation build with Sphinx + * bootstrap non-framework Python 2.7 for documentation build with + Sphinx (as of 3.4.1) - alternate build environments: @@ -48,7 +49,7 @@ 2. 64-bit / 32-bit, x86_64 and i386 universal, for OS X 10.6 (and later):: - /usr/bin/python build-installer.py \ + /path/to/bootstrap/python2.7 build-installer.py \ --sdk-path=/Developer/SDKs/MacOSX10.6.sdk \ --universal-archs=intel \ --dep-target=10.6 @@ -56,14 +57,14 @@ - builds the following third-party libraries * NCurses 5.9 (http://bugs.python.org/issue15037) - * SQLite 3.8.1 + * SQLite 3.8.3.1 * XZ 5.0.5 - uses system-supplied versions of third-party libraries * readline module links with Apple BSD editline (libedit) - - requires ActiveState Tcl/Tk 8.5.15 (or later) to be installed for building + - requires ActiveState Tcl/Tk 8.5.15.1 (or later) to be installed for building - recommended build environment: @@ -72,7 +73,8 @@ * ``MacOSX10.6`` SDK * ``MACOSX_DEPLOYMENT_TARGET=10.6`` * Apple ``gcc-4.2`` - * system Python 2.6 for documentation build with Sphinx + * bootstrap non-framework Python 2.7 for documentation build with + Sphinx (as of 3.4.1) - alternate build environments: @@ -82,10 +84,10 @@ considered a migration aid by Apple and is not likely to be fixed, its use should be avoided. The other compiler, ``clang``, has been undergoing rapid development. While it appears to have become - production-ready in the most recent Xcode 4 releases (Xcode 4.6.3 - as of this writing), there are still some open issues when - building Python and there has not yet been the level of exposure in - production environments that the Xcode 3 gcc-4.2 compiler has had. + production-ready in the most recent Xcode 5 releases, the versions + available on the deprecated Xcode 4.x for 10.6 were early releases + and did not receive the level of exposure in production environments + that the Xcode 3 gcc-4.2 compiler has had. * For Python 2.7.x and 3.2.x, the 32-bit-only installer was configured to @@ -111,7 +113,7 @@ * Zlib 1.2.3 * Oracle Sleepycat DB 4.8 (Python 2.x only) - - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.19) to be installed for building + - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.20) to be installed for building - recommended build environment: @@ -137,7 +139,13 @@ interfere with the build. * The documentation for the release is built using Sphinx - because it is included in the installer. + because it is included in the installer. For 2.7.x and 3.x.x up to and + including 3.4.0, the ``Doc/Makefile`` uses ``svn`` to download repos of + ``Sphinx`` and its dependencies. Beginning with 3.4.1, the ``Doc/Makefile`` + assumes there is an externally-provided ``sphinx-build`` and requires at + least Python 2.6 to run. Because of this, it is no longer possible to + build a 3.4.1 or later installer on OS X 10.5 using the Apple-supplied + Python 2.5. * It is safest to start each variant build with an empty source directory populated with a fresh copy of the untarred source. @@ -195,8 +203,8 @@ on all supported OS X releases and architectures. As a practical matter, that is generally not possible. At a minimum, variant 1 should be run on a PPC G4 system with OS X 10.5 and at least one Intel system running OS X -10.8, 10.7, 10.6, or 10.5. Variant 2 should be run on 10.8, 10.7, and 10.6 -systems in both 32-bit and 64-bit modes.:: +10.9, 10.8, 10.7, 10.6, or 10.5. Variant 2 should be run on 10.9, 10.8, +10.7, and 10.6 systems in both 32-bit and 64-bit modes.:: /usr/local/bin/pythonn.n -m test -w -u all,-largefile /usr/local/bin/pythonn.n-32 -m test -w -u all @@ -207,7 +215,7 @@ during testing and can be ignored. It is also recommend to launch IDLE and verify that it is at least -functional. Double-click on the IDLE app icon in ``/Applications/Pythonn.n``. +functional. Double-click on the IDLE app icon in ``/Applications/Python n.n``. It should also be tested from the command line:: /usr/local/bin/idlen.n diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -194,7 +194,7 @@ LT_10_5 = bool(DEPTARGET < '10.5') # Disable for now - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 5)): + if False: # if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 5)): result.extend([ dict( name="Tcl 8.5.15", @@ -607,6 +607,10 @@ base_path = base_path + ':' + OLD_DEVELOPER_TOOLS os.environ['PATH'] = base_path print("Setting default PATH: %s"%(os.environ['PATH'])) + # Ensure ws have access to hg and to sphinx-build. + # You may have to create links in /usr/bin for them. + runCommand('hg --version') + runCommand('sphinx-build --version') def parseOptions(args=None): @@ -920,8 +924,9 @@ docdir = os.path.join(rootDir, 'pydocs') curDir = os.getcwd() os.chdir(buildDir) - runCommand('make update') - runCommand("make html PYTHON='%s'" % os.path.abspath(sys.executable)) + runCommand('make clean') + # Assume sphinx-build is on our PATH, checked in checkEnvironment + runCommand('make html') os.chdir(curDir) if not os.path.exists(docdir): os.mkdir(docdir) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -162,6 +162,12 @@ - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. +Build +----- + +- Issue #20644: OS X installer build support for documentation build changes + in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. + C API ----- - Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__ to -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 03:57:36 2014 From: python-checkins at python.org (ned.deily) Date: Wed, 9 Apr 2014 03:57:36 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMDk3?= =?utf-8?q?=3A_Update_Makefile_with_changed_install_locations_of_test?= Message-ID: <3g3TC43s7xz7LjN@mail.python.org> http://hg.python.org/cpython/rev/99265d30fa38 changeset: 90192:99265d30fa38 branch: 3.4 parent: 90190:01f1e14cad23 user: Ned Deily date: Tue Apr 08 18:54:49 2014 -0700 summary: Issue #21097: Update Makefile with changed install locations of test directories. files: Makefile.pre.in | 40 ++++++++++++++++++------------------ 1 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1119,26 +1119,26 @@ test/imghdrdata \ test/subprocessdata test/sndhdrdata test/support \ test/tracedmodules test/encoded_modules \ - test/namespace_pkgs \ - test/namespace_pkgs/both_portions \ - test/namespace_pkgs/both_portions/foo \ - test/namespace_pkgs/not_a_namespace_pkg \ - test/namespace_pkgs/not_a_namespace_pkg/foo \ - test/namespace_pkgs/portion1 \ - test/namespace_pkgs/portion1/foo \ - test/namespace_pkgs/portion2 \ - test/namespace_pkgs/portion2/foo \ - test/namespace_pkgs/project1 \ - test/namespace_pkgs/project1/parent \ - test/namespace_pkgs/project1/parent/child \ - test/namespace_pkgs/project2 \ - test/namespace_pkgs/project2/parent \ - test/namespace_pkgs/project2/parent/child \ - test/namespace_pkgs/project3 \ - test/namespace_pkgs/project3/parent \ - test/namespace_pkgs/project3/parent/child \ - test/namespace_pkgs/module_and_namespace_package \ - test/namespace_pkgs/module_and_namespace_package/a_test \ + test/test_importlib/namespace_pkgs \ + test/test_importlib/namespace_pkgs/both_portions \ + test/test_importlib/namespace_pkgs/both_portions/foo \ + test/test_importlib/namespace_pkgs/not_a_namespace_pkg \ + test/test_importlib/namespace_pkgs/not_a_namespace_pkg/foo \ + test/test_importlib/namespace_pkgs/portion1 \ + test/test_importlib/namespace_pkgs/portion1/foo \ + test/test_importlib/namespace_pkgs/portion2 \ + test/test_importlib/namespace_pkgs/portion2/foo \ + test/test_importlib/namespace_pkgs/project1 \ + test/test_importlib/namespace_pkgs/project1/parent \ + test/test_importlib/namespace_pkgs/project1/parent/child \ + test/test_importlib/namespace_pkgs/project2 \ + test/test_importlib/namespace_pkgs/project2/parent \ + test/test_importlib/namespace_pkgs/project2/parent/child \ + test/test_importlib/namespace_pkgs/project3 \ + test/test_importlib/namespace_pkgs/project3/parent \ + test/test_importlib/namespace_pkgs/project3/parent/child \ + test/test_importlib/namespace_pkgs/module_and_namespace_package \ + test/test_importlib/namespace_pkgs/module_and_namespace_package/a_test \ asyncio \ test/test_asyncio \ collections concurrent concurrent/futures encodings \ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 03:57:37 2014 From: python-checkins at python.org (ned.deily) Date: Wed, 9 Apr 2014 03:57:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321097=3A_merge_from_3=2E4?= Message-ID: <3g3TC55htbz7LjY@mail.python.org> http://hg.python.org/cpython/rev/7aae2b9fcfad changeset: 90193:7aae2b9fcfad parent: 90191:7d004ad09bf5 parent: 90192:99265d30fa38 user: Ned Deily date: Tue Apr 08 18:57:10 2014 -0700 summary: Issue #21097: merge from 3.4 files: Makefile.pre.in | 40 ++++++++++++++++++------------------ 1 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1119,26 +1119,26 @@ test/imghdrdata \ test/subprocessdata test/sndhdrdata test/support \ test/tracedmodules test/encoded_modules \ - test/namespace_pkgs \ - test/namespace_pkgs/both_portions \ - test/namespace_pkgs/both_portions/foo \ - test/namespace_pkgs/not_a_namespace_pkg \ - test/namespace_pkgs/not_a_namespace_pkg/foo \ - test/namespace_pkgs/portion1 \ - test/namespace_pkgs/portion1/foo \ - test/namespace_pkgs/portion2 \ - test/namespace_pkgs/portion2/foo \ - test/namespace_pkgs/project1 \ - test/namespace_pkgs/project1/parent \ - test/namespace_pkgs/project1/parent/child \ - test/namespace_pkgs/project2 \ - test/namespace_pkgs/project2/parent \ - test/namespace_pkgs/project2/parent/child \ - test/namespace_pkgs/project3 \ - test/namespace_pkgs/project3/parent \ - test/namespace_pkgs/project3/parent/child \ - test/namespace_pkgs/module_and_namespace_package \ - test/namespace_pkgs/module_and_namespace_package/a_test \ + test/test_importlib/namespace_pkgs \ + test/test_importlib/namespace_pkgs/both_portions \ + test/test_importlib/namespace_pkgs/both_portions/foo \ + test/test_importlib/namespace_pkgs/not_a_namespace_pkg \ + test/test_importlib/namespace_pkgs/not_a_namespace_pkg/foo \ + test/test_importlib/namespace_pkgs/portion1 \ + test/test_importlib/namespace_pkgs/portion1/foo \ + test/test_importlib/namespace_pkgs/portion2 \ + test/test_importlib/namespace_pkgs/portion2/foo \ + test/test_importlib/namespace_pkgs/project1 \ + test/test_importlib/namespace_pkgs/project1/parent \ + test/test_importlib/namespace_pkgs/project1/parent/child \ + test/test_importlib/namespace_pkgs/project2 \ + test/test_importlib/namespace_pkgs/project2/parent \ + test/test_importlib/namespace_pkgs/project2/parent/child \ + test/test_importlib/namespace_pkgs/project3 \ + test/test_importlib/namespace_pkgs/project3/parent \ + test/test_importlib/namespace_pkgs/project3/parent/child \ + test/test_importlib/namespace_pkgs/module_and_namespace_package \ + test/test_importlib/namespace_pkgs/module_and_namespace_package/a_test \ asyncio \ test/test_asyncio \ collections concurrent concurrent/futures encodings \ -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Wed Apr 9 09:17:46 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 09 Apr 2014 09:17:46 +0200 Subject: [Python-checkins] Daily reference leaks (7d004ad09bf5): sum=3 Message-ID: results for 7d004ad09bf5 on branch "default" -------------------------------------------- test_asyncio leaked [0, 4, 0] memory blocks, sum=4 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogJYo9dD', '-x'] From python-checkins at python.org Wed Apr 9 13:04:14 2014 From: python-checkins at python.org (mark.dickinson) Date: Wed, 9 Apr 2014 13:04:14 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTc5?= =?utf-8?q?=3A_Fix_description_of_=27round=27_function_for_numbers=2EReal?= =?utf-8?q?=2E?= Message-ID: <3g3jKp1SJBz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/7ec8c4555772 changeset: 90194:7ec8c4555772 branch: 2.7 parent: 90179:9f0e10dc7722 user: Mark Dickinson date: Wed Apr 09 07:03:27 2014 -0400 summary: Issue #21179: Fix description of 'round' function for numbers.Real. files: Doc/library/stdtypes.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -399,8 +399,8 @@ | ``math.trunc(x)`` | *x* truncated to Integral | | +--------------------+------------------------------------+--------+ | ``round(x[, n])`` | *x* rounded to n digits, | | -| | rounding half to even. If n is | | -| | omitted, it defaults to 0. | | +| | rounding ties away from zero. If n | | +| | is omitted, it defaults to 0. | | +--------------------+------------------------------------+--------+ | ``math.floor(x)`` | the greatest integral float <= *x* | | +--------------------+------------------------------------+--------+ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 17:45:27 2014 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 9 Apr 2014 17:45:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogaXNzdWUgIzIxMTkw?= =?utf-8?q?=3A_Fix_the_docs_README_link?= Message-ID: <3g3qZH0NZGz7Lk0@mail.python.org> http://hg.python.org/cpython/rev/aec35c0d308b changeset: 90195:aec35c0d308b branch: 2.7 user: Senthil Kumaran date: Wed Apr 09 11:41:02 2014 -0400 summary: issue #21190: Fix the docs README link files: Doc/README.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -3,7 +3,7 @@ This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, prebuilt versions are -available at http://docs.python.org/download/. +available at https://docs.python.org/2/download.html Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 17:45:28 2014 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 9 Apr 2014 17:45:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogaXNzdWUgIzIxMTkw?= =?utf-8?q?=3A_Fix_the_broken_docs_download_link?= Message-ID: <3g3qZJ1mYnz7LkT@mail.python.org> http://hg.python.org/cpython/rev/df8f49f8cdd2 changeset: 90196:df8f49f8cdd2 branch: 3.4 parent: 90192:99265d30fa38 user: Senthil Kumaran date: Wed Apr 09 11:44:34 2014 -0400 summary: issue #21190: Fix the broken docs download link files: Doc/README.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -3,7 +3,7 @@ This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, prebuilt versions are -available at . +available at . Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 17:45:29 2014 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 9 Apr 2014 17:45:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E4?= Message-ID: <3g3qZK3Nvyz7Ljk@mail.python.org> http://hg.python.org/cpython/rev/55007370d46c changeset: 90197:55007370d46c parent: 90193:7aae2b9fcfad parent: 90196:df8f49f8cdd2 user: Senthil Kumaran date: Wed Apr 09 11:45:14 2014 -0400 summary: merge from 3.4 files: Doc/README.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -3,7 +3,7 @@ This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, prebuilt versions are -available at . +available at . Documentation on the authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 9 21:40:37 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 9 Apr 2014 21:40:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_make_sure_fdop?= =?utf-8?q?en_always_closes_the_fd_in_error_cases_=28closes_=2321191=29?= Message-ID: <3g3wnd2Tmjz7LjW@mail.python.org> http://hg.python.org/cpython/rev/4a3b455abf76 changeset: 90198:4a3b455abf76 branch: 2.7 parent: 90195:aec35c0d308b user: Benjamin Peterson date: Wed Apr 09 15:40:18 2014 -0400 summary: make sure fdopen always closes the fd in error cases (closes #21191) files: Doc/library/os.rst | 5 +++-- Lib/test/test_posix.py | 4 ++++ Misc/NEWS | 3 +++ Modules/posixmodule.c | 16 ++++++++++++---- 4 files changed, 22 insertions(+), 6 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -463,8 +463,9 @@ .. index:: single: I/O control; buffering Return an open file object connected to the file descriptor *fd*. The *mode* - and *bufsize* arguments have the same meaning as the corresponding arguments to - the built-in :func:`open` function. + and *bufsize* arguments have the same meaning as the corresponding arguments + to the built-in :func:`open` function. If :func:`fdopen` raises an + exception, it closes *fd*. Availability: Unix, Windows. diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -194,6 +194,10 @@ self.fdopen_helper('r') self.fdopen_helper('r', 100) + fd = os.open(test_support.TESTFN, os.O_RDONLY) + self.assertRaises(OSError, posix.fdopen, fd, 'w') + self.assertRaises(OSError, os.close, fd) # fd should be closed. + @unittest.skipUnless(hasattr(posix, 'O_EXLOCK'), 'test needs posix.O_EXLOCK') def test_osexlock(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,9 @@ Library ------- +- Issue #21191: In os.fdopen, alwyas close the file descriptor when an exception + happens. + - Issue #21149: Improved thread-safety in logging cleanup during interpreter shutdown. Thanks to Devin Jeanpierre for the patch. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -6841,16 +6841,21 @@ /* Sanitize mode. See fileobject.c */ mode = PyMem_MALLOC(strlen(orgmode)+3); if (!mode) { + close(fd); PyErr_NoMemory(); return NULL; } strcpy(mode, orgmode); if (_PyFile_SanitizeMode(mode)) { + close(fd); PyMem_FREE(mode); return NULL; } - if (!_PyVerify_fd(fd)) - return posix_error(); + if (!_PyVerify_fd(fd)) { + posix_error(); + close(fd); + return NULL; + } Py_BEGIN_ALLOW_THREADS #if !defined(MS_WINDOWS) && defined(HAVE_FCNTL_H) if (mode[0] == 'a') { @@ -6871,8 +6876,11 @@ #endif Py_END_ALLOW_THREADS PyMem_FREE(mode); - if (fp == NULL) - return posix_error(); + if (fp == NULL) { + posix_error(); + close(fd); + return NULL; + } /* The dummy filename used here must be kept in sync with the value tested against in gzip.GzipFile.__init__() - see issue #13781. */ f = PyFile_FromFile(fp, "", orgmode, fclose); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 01:23:57 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 10 Apr 2014 01:23:57 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwNjQ0?= =?utf-8?q?=3A_Keep_build-installer=2Epy_in_sync_across_active_versions=2E?= Message-ID: <3g41lK3vLGz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/88572ccb8ebe changeset: 90199:88572ccb8ebe branch: 2.7 user: Ned Deily date: Wed Apr 09 16:12:11 2014 -0700 summary: Issue #20644: Keep build-installer.py in sync across active versions. files: Mac/BuildScript/build-installer.py | 127 ++++++---------- 1 files changed, 50 insertions(+), 77 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -8,7 +8,9 @@ Please ensure that this script keeps working with Python 2.5, to avoid bootstrap issues (/usr/bin/python is Python 2.5 on OSX 10.5). Sphinx, which is used to build the documentation, currently requires at least -Python 2.4. +Python 2.4. However, as of Python 3.4.1, Doc builds require an external +sphinx-build and the current versions of Sphinx now require at least +Python 2.6. In addition to what is supplied with OS X 10.5+ and Xcode 3+, the script requires an installed version of hg and a third-party version of @@ -21,8 +23,8 @@ 32-bit-only installer builds are still possible on OS X 10.4 with Xcode 2.5 and the installation of additional components, such as a newer Python -(2.5 is needed for Python parser updates), hg, and svn (for the documentation -build). +(2.5 is needed for Python parser updates), hg, and for the documentation +build either svn (pre-3.4.1) or sphinx-build (3.4.1 and later). Usage: see USAGE variable in the script. """ @@ -193,7 +195,8 @@ LT_10_5 = bool(DEPTARGET < '10.5') - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 4)): +# Disable for now + if False: # if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 5)): result.extend([ dict( name="Tcl 8.5.15", @@ -237,9 +240,9 @@ if getVersionTuple() >= (3, 3): result.extend([ dict( - name="XZ 5.0.3", - url="http://tukaani.org/xz/xz-5.0.3.tar.gz", - checksum='fefe52f9ecd521de2a8ce38c21a27574', + name="XZ 5.0.5", + url="http://tukaani.org/xz/xz-5.0.5.tar.gz", + checksum='19d924e066b6fff0bc9d1981b4e53196', configure_pre=[ '--disable-dependency-tracking', ] @@ -282,9 +285,9 @@ ), ), dict( - name="SQLite 3.7.13", - url="http://www.sqlite.org/sqlite-autoconf-3071300.tar.gz", - checksum='c97df403e8a3d5b67bb408fcd6aabd8e', + name="SQLite 3.8.3.1", + url="http://www.sqlite.org/2014/sqlite-autoconf-3080301.tar.gz", + checksum='509ff98d8dc9729b618b7e96612079c6', extra_cflags=('-Os ' '-DSQLITE_ENABLE_FTS4 ' '-DSQLITE_ENABLE_FTS3_PARENTHESIS ' @@ -364,6 +367,8 @@ # Instructions for building packages inside the .mpkg. def pkg_recipes(): unselected_for_python3 = ('selected', 'unselected')[PYTHON_3] + # unselected if 3.0 through 3.3, selected otherwise (2.x or >= 3.4) + unselected_for_lt_python34 = ('selected', 'unselected')[(3, 0) <= getVersionTuple() < (3, 4)] result = [ dict( name="PythonFramework", @@ -432,10 +437,27 @@ topdir="/Library/Frameworks/Python.framework", source="/empty-dir", required=False, - selected=unselected_for_python3, + selected=unselected_for_lt_python34, ), ] + if getVersionTuple() >= (3, 4): + result.append( + dict( + name="PythonInstallPip", + long_name="Install or upgrade pip", + readme="""\ + This package installs (or upgrades from an earlier version) + pip, a tool for installing and managing Python packages. + """, + postflight="scripts/postflight.ensurepip", + topdir="/Library/Frameworks/Python.framework", + source="/empty-dir", + required=False, + selected='selected', + ) + ) + if DEPTARGET < '10.4' and not PYTHON_3: result.append( dict( @@ -453,6 +475,7 @@ selected=unselected_for_python3, ) ) + return result def fatal(msg): @@ -567,20 +590,6 @@ % frameworks['Tk'], ] - # For 10.6+ builds, we build two versions of _tkinter: - # - the traditional version (renamed to _tkinter_library.so) linked - # with /Library/Frameworks/{Tcl,Tk}.framework - # - the default version linked with our builtin copies of Tcl and Tk - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 4)): - EXPECTED_SHARED_LIBS['_tkinter_library.so'] = \ - EXPECTED_SHARED_LIBS['_tkinter.so'] - EXPECTED_SHARED_LIBS['_tkinter.so'] = [ - "/Library/Frameworks/Python.framework/Versions/%s/lib/libtcl%s.dylib" - % (getVersion(), frameworks['Tcl']), - "/Library/Frameworks/Python.framework/Versions/%s/lib/libtk%s.dylib" - % (getVersion(), frameworks['Tk']), - ] - # Remove inherited environment variables which might influence build environ_var_prefixes = ['CPATH', 'C_INCLUDE_', 'DYLD_', 'LANG', 'LC_', 'LD_', 'LIBRARY_', 'PATH', 'PYTHON'] @@ -601,7 +610,11 @@ base_path = base_path + ':' + OLD_DEVELOPER_TOOLS os.environ['PATH'] = base_path print("Setting default PATH: %s"%(os.environ['PATH'])) - + # Ensure ws have access to hg and to sphinx-build. + # You may have to create links in /usr/bin for them. + runCommand('hg --version') + if getVersionTuple() >= (3, 4): + runCommand('sphinx-build --version') def parseOptions(args=None): """ @@ -914,8 +927,15 @@ docdir = os.path.join(rootDir, 'pydocs') curDir = os.getcwd() os.chdir(buildDir) - runCommand('make update') - runCommand("make html PYTHON='%s'" % os.path.abspath(sys.executable)) + # The Doc build changed for 3.4 (technically, for 3.4.1) + if getVersionTuple() < (3, 4): + # This step does an svn checkout of sphinx and its dependencies + runCommand('make update') + runCommand("make html PYTHON='%s'" % os.path.abspath(sys.executable)) + else: + runCommand('make clean') + # Assume sphinx-build is on our PATH, checked in checkEnvironment + runCommand('make html') os.chdir(curDir) if not os.path.exists(docdir): os.mkdir(docdir) @@ -955,34 +975,19 @@ runCommand("%s -C --enable-framework --enable-universalsdk=%s " "--with-universal-archs=%s " "%s " + "%s " "LDFLAGS='-g -L%s/libraries/usr/local/lib' " "CFLAGS='-g -I%s/libraries/usr/local/include' 2>&1"%( shellQuote(os.path.join(SRCDIR, 'configure')), shellQuote(SDKPATH), UNIVERSALARCHS, (' ', '--with-computed-gotos ')[PYTHON_3], + (' ', '--without-ensurepip ')[getVersionTuple() >= (3, 4)], shellQuote(WORKDIR)[1:-1], shellQuote(WORKDIR)[1:-1])) print("Running make") runCommand("make") - # For deployment targets of 10.6 and higher, we build our own version - # of Tcl and Cocoa Aqua Tk libs because the Apple-supplied Tk 8.5 is - # out-of-date and has critical bugs. Save the _tkinter.so that was - # linked with /Library/Frameworks/{Tck,Tk}.framework and build - # another _tkinter.so linked with our builtin Tcl and Tk libs. - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 4)): - runCommand("find build -name '_tkinter.so' " - " -execdir mv '{}' _tkinter_library.so \;") - print("Running make to build builtin _tkinter") - runCommand("make TCLTK_INCLUDES='-I%s/libraries/usr/local/include' " - "TCLTK_LIBS='-L%s/libraries/usr/local/lib -ltcl8.5 -ltk8.5'"%( - shellQuote(WORKDIR)[1:-1], - shellQuote(WORKDIR)[1:-1])) - # make a copy which will be moved to lib-tkinter later - runCommand("find build -name '_tkinter.so' " - " -execdir cp -p '{}' _tkinter_builtin.so \;") - print("Running make install") runCommand("make install DESTDIR=%s"%( shellQuote(rootDir))) @@ -1007,27 +1012,11 @@ 'Python.framework', 'Versions', version, 'lib', 'python%s'%(version,)) - # If we made multiple versions of _tkinter, move them to - # their own directories under python lib. This allows - # users to select which to import by manipulating sys.path - # directly or with PYTHONPATH. - - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 4)): - TKINTERS = ['builtin', 'library'] - tkinter_moves = [('_tkinter_' + tkn + '.so', - os.path.join(path_to_lib, 'lib-tkinter', tkn)) - for tkn in TKINTERS] - # Create the destination directories under lib-tkinter. - # The permissions and uid/gid will be fixed up next. - for tkm in tkinter_moves: - os.makedirs(tkm[1]) - print("Fix file modes") frmDir = os.path.join(rootDir, 'Library', 'Frameworks', 'Python.framework') gid = grp.getgrnam('admin').gr_gid shared_lib_error = False - moves_list = [] for dirpath, dirnames, filenames in os.walk(frmDir): for dn in dirnames: os.chmod(os.path.join(dirpath, dn), STAT_0o775) @@ -1053,25 +1042,9 @@ % (sl, p)) shared_lib_error = True - # If this is a _tkinter variant, move it to its own directory - # now that we have fixed its permissions and checked that it - # was linked properly. The directory was created earlier. - # The files are moved after the entire tree has been walked - # since the shared library checking depends on the files - # having unique names. - if (DEPTARGET > '10.5') and (getVersionTuple() >= (3, 4)): - for tkm in tkinter_moves: - if fn == tkm[0]: - moves_list.append( - (p, os.path.join(tkm[1], '_tkinter.so'))) - if shared_lib_error: fatal("Unexpected shared library errors.") - # Now do the moves. - for ml in moves_list: - shutil.move(ml[0], ml[1]) - if PYTHON_3: LDVERSION=None VERSION=None -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 01:23:58 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 10 Apr 2014 01:23:58 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTg3?= =?utf-8?q?=3A_Fix_OS_X_installer_fail-to-build_with_Xcode_5=2E1=2E?= Message-ID: <3g41lL5MGpz7Ljt@mail.python.org> http://hg.python.org/cpython/rev/63a55ed6622b changeset: 90200:63a55ed6622b branch: 2.7 user: Ned Deily date: Wed Apr 09 16:13:46 2014 -0700 summary: Issue #21187: Fix OS X installer fail-to-build with Xcode 5.1. files: Mac/BuildScript/build-installer.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -867,7 +867,7 @@ ' -arch '.join(archList), shellQuote(SDKPATH)[1:-1], shellQuote(basedir)[1:-1],), - "LDFLAGS=-mmacosx-version-min=%s -syslibroot,%s -L%s/usr/local/lib -arch %s"%( + "LDFLAGS=-mmacosx-version-min=%s -isysroot %s -L%s/usr/local/lib -arch %s"%( DEPTARGET, shellQuote(SDKPATH)[1:-1], shellQuote(basedir)[1:-1], -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 01:24:00 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 10 Apr 2014 01:24:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNjQ0?= =?utf-8?q?=3A_Keep_build-installer=2Epy_in_sync_across_active_versions=2E?= Message-ID: <3g41lN0Gtfz7LkD@mail.python.org> http://hg.python.org/cpython/rev/e0722b5b9412 changeset: 90201:e0722b5b9412 branch: 3.4 parent: 90196:df8f49f8cdd2 user: Ned Deily date: Wed Apr 09 16:15:20 2014 -0700 summary: Issue #20644: Keep build-installer.py in sync across active versions. files: Mac/BuildScript/build-installer.py | 27 ++++++++++++------ 1 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -8,7 +8,9 @@ Please ensure that this script keeps working with Python 2.5, to avoid bootstrap issues (/usr/bin/python is Python 2.5 on OSX 10.5). Sphinx, which is used to build the documentation, currently requires at least -Python 2.4. +Python 2.4. However, as of Python 3.4.1, Doc builds require an external +sphinx-build and the current versions of Sphinx now require at least +Python 2.6. In addition to what is supplied with OS X 10.5+ and Xcode 3+, the script requires an installed version of hg and a third-party version of @@ -21,8 +23,8 @@ 32-bit-only installer builds are still possible on OS X 10.4 with Xcode 2.5 and the installation of additional components, such as a newer Python -(2.5 is needed for Python parser updates), hg, and svn (for the documentation -build). +(2.5 is needed for Python parser updates), hg, and for the documentation +build either svn (pre-3.4.1) or sphinx-build (3.4.1 and later). Usage: see USAGE variable in the script. """ @@ -365,7 +367,8 @@ # Instructions for building packages inside the .mpkg. def pkg_recipes(): unselected_for_python3 = ('selected', 'unselected')[PYTHON_3] - unselected_for_lt_python34 = ('selected', 'unselected')[getVersionTuple() < (3, 4)] + # unselected if 3.0 through 3.3, selected otherwise (2.x or >= 3.4) + unselected_for_lt_python34 = ('selected', 'unselected')[(3, 0) <= getVersionTuple() < (3, 4)] result = [ dict( name="PythonFramework", @@ -610,8 +613,8 @@ # Ensure ws have access to hg and to sphinx-build. # You may have to create links in /usr/bin for them. runCommand('hg --version') - runCommand('sphinx-build --version') - + if getVersionTuple() >= (3, 4): + runCommand('sphinx-build --version') def parseOptions(args=None): """ @@ -924,9 +927,15 @@ docdir = os.path.join(rootDir, 'pydocs') curDir = os.getcwd() os.chdir(buildDir) - runCommand('make clean') - # Assume sphinx-build is on our PATH, checked in checkEnvironment - runCommand('make html') + # The Doc build changed for 3.4 (technically, for 3.4.1) + if getVersionTuple() < (3, 4): + # This step does an svn checkout of sphinx and its dependencies + runCommand('make update') + runCommand("make html PYTHON='%s'" % os.path.abspath(sys.executable)) + else: + runCommand('make clean') + # Assume sphinx-build is on our PATH, checked in checkEnvironment + runCommand('make html') os.chdir(curDir) if not os.path.exists(docdir): os.mkdir(docdir) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 01:24:01 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 10 Apr 2014 01:24:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320644=3A_merge_with_3=2E4?= Message-ID: <3g41lP1zLvz7LjT@mail.python.org> http://hg.python.org/cpython/rev/63bd9474be49 changeset: 90202:63bd9474be49 parent: 90197:55007370d46c parent: 90201:e0722b5b9412 user: Ned Deily date: Wed Apr 09 16:20:08 2014 -0700 summary: Issue #20644: merge with 3.4 files: Mac/BuildScript/build-installer.py | 27 ++++++++++++------ 1 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -8,7 +8,9 @@ Please ensure that this script keeps working with Python 2.5, to avoid bootstrap issues (/usr/bin/python is Python 2.5 on OSX 10.5). Sphinx, which is used to build the documentation, currently requires at least -Python 2.4. +Python 2.4. However, as of Python 3.4.1, Doc builds require an external +sphinx-build and the current versions of Sphinx now require at least +Python 2.6. In addition to what is supplied with OS X 10.5+ and Xcode 3+, the script requires an installed version of hg and a third-party version of @@ -21,8 +23,8 @@ 32-bit-only installer builds are still possible on OS X 10.4 with Xcode 2.5 and the installation of additional components, such as a newer Python -(2.5 is needed for Python parser updates), hg, and svn (for the documentation -build). +(2.5 is needed for Python parser updates), hg, and for the documentation +build either svn (pre-3.4.1) or sphinx-build (3.4.1 and later). Usage: see USAGE variable in the script. """ @@ -365,7 +367,8 @@ # Instructions for building packages inside the .mpkg. def pkg_recipes(): unselected_for_python3 = ('selected', 'unselected')[PYTHON_3] - unselected_for_lt_python34 = ('selected', 'unselected')[getVersionTuple() < (3, 4)] + # unselected if 3.0 through 3.3, selected otherwise (2.x or >= 3.4) + unselected_for_lt_python34 = ('selected', 'unselected')[(3, 0) <= getVersionTuple() < (3, 4)] result = [ dict( name="PythonFramework", @@ -610,8 +613,8 @@ # Ensure ws have access to hg and to sphinx-build. # You may have to create links in /usr/bin for them. runCommand('hg --version') - runCommand('sphinx-build --version') - + if getVersionTuple() >= (3, 4): + runCommand('sphinx-build --version') def parseOptions(args=None): """ @@ -924,9 +927,15 @@ docdir = os.path.join(rootDir, 'pydocs') curDir = os.getcwd() os.chdir(buildDir) - runCommand('make clean') - # Assume sphinx-build is on our PATH, checked in checkEnvironment - runCommand('make html') + # The Doc build changed for 3.4 (technically, for 3.4.1) + if getVersionTuple() < (3, 4): + # This step does an svn checkout of sphinx and its dependencies + runCommand('make update') + runCommand("make html PYTHON='%s'" % os.path.abspath(sys.executable)) + else: + runCommand('make clean') + # Assume sphinx-build is on our PATH, checked in checkEnvironment + runCommand('make html') os.chdir(curDir) if not os.path.exists(docdir): os.mkdir(docdir) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 01:24:02 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 10 Apr 2014 01:24:02 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTg3?= =?utf-8?q?=3A_Fix_OS_X_installer_fail-to-build_with_Xcode_5=2E1=2E?= Message-ID: <3g41lQ3m5sz7LkC@mail.python.org> http://hg.python.org/cpython/rev/a3299de5fc93 changeset: 90203:a3299de5fc93 branch: 3.4 parent: 90201:e0722b5b9412 user: Ned Deily date: Wed Apr 09 16:16:08 2014 -0700 summary: Issue #21187: Fix OS X installer fail-to-build with Xcode 5.1. files: Mac/BuildScript/build-installer.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -867,7 +867,7 @@ ' -arch '.join(archList), shellQuote(SDKPATH)[1:-1], shellQuote(basedir)[1:-1],), - "LDFLAGS=-mmacosx-version-min=%s -syslibroot,%s -L%s/usr/local/lib -arch %s"%( + "LDFLAGS=-mmacosx-version-min=%s -isysroot %s -L%s/usr/local/lib -arch %s"%( DEPTARGET, shellQuote(SDKPATH)[1:-1], shellQuote(basedir)[1:-1], -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 01:24:03 2014 From: python-checkins at python.org (ned.deily) Date: Thu, 10 Apr 2014 01:24:03 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321187=3A_merge_with_3=2E4?= Message-ID: <3g41lR59wDz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/b402e5e06f85 changeset: 90204:b402e5e06f85 parent: 90202:63bd9474be49 parent: 90203:a3299de5fc93 user: Ned Deily date: Wed Apr 09 16:22:19 2014 -0700 summary: Issue #21187: merge with 3.4 files: Mac/BuildScript/build-installer.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -867,7 +867,7 @@ ' -arch '.join(archList), shellQuote(SDKPATH)[1:-1], shellQuote(basedir)[1:-1],), - "LDFLAGS=-mmacosx-version-min=%s -syslibroot,%s -L%s/usr/local/lib -arch %s"%( + "LDFLAGS=-mmacosx-version-min=%s -isysroot %s -L%s/usr/local/lib -arch %s"%( DEPTARGET, shellQuote(SDKPATH)[1:-1], shellQuote(basedir)[1:-1], -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 03:53:51 2014 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 10 Apr 2014 03:53:51 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_algorithmic_notes_for_?= =?utf-8?b?bnNtYWxsZXN0KCkgYW5kIG5sYXJnZXN0KCku?= Message-ID: <3g454H2pN1z7LjS@mail.python.org> http://hg.python.org/cpython/rev/b651d519b9f2 changeset: 90205:b651d519b9f2 user: Raymond Hettinger date: Wed Apr 09 19:53:45 2014 -0600 summary: Add algorithmic notes for nsmallest() and nlargest(). files: Lib/heapq.py | 56 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 56 insertions(+), 0 deletions(-) diff --git a/Lib/heapq.py b/Lib/heapq.py --- a/Lib/heapq.py +++ b/Lib/heapq.py @@ -192,6 +192,62 @@ for i in reversed(range(n//2)): _siftup_max(x, i) + +# Algorithm notes for nlargest() and nsmallest() +# ============================================== +# +# Makes just one pass over the data while keeping the n most extreme values +# in a heap. Memory consumption is limited to keeping n values in a list. +# +# Number of comparisons for n random inputs, keeping the k smallest values: +# ----------------------------------------------------------- +# Step Comparisons Action +# 1 2*k heapify the first k-inputs +# 2 n-k compare new input elements to top of heap +# 3 k*lg2(k)*(ln(n)-lg(k)) add new extreme values to the heap +# 4 k*lg2(k) final sort of the k most extreme values +# +# n-random inputs k-extreme values number of comparisons % more than min() +# --------------- ---------------- ------------------- ----------------- +# 10,000 100 13,634 36.3% +# 100,000 100 105,163 5.2% +# 1,000,000 100 1,006,694 0.7% +# +# Computing the number of comparisons for step 3: +# ----------------------------------------------- +# * For the i-th new value from the iterable, the probability of being in the +# k most extreme values is k/i. For example, the probability of the 101st +# value seen being in the 100 most extreme values is 100/101. +# * If the value is a new extreme value, the cost of inserting it into the +# heap is log(k, 2). +# * The probabilty times the cost gives: +# (k/i) * log(k, 2) +# * Summing across the remaining n-k elements gives: +# sum((k/i) * log(k, 2) for xrange(k+1, n+1)) +# * This reduces to: +# (H(n) - H(k)) * k * log(k, 2) +# * Where H(n) is the n-th harmonic number estimated by: +# H(n) = log(n, e) + gamma + 1.0 / (2.0 * n) +# gamma = 0.5772156649 +# http://en.wikipedia.org/wiki/Harmonic_series_(mathematics)#Rate_of_divergence +# * Substituting the H(n) formula and ignoring the (1/2*n) fraction gives: +# comparisons = k * log(k, 2) * (log(n,e) - log(k, e)) +# +# Worst-case for step 3: +# --------------------- +# In the worst case, the input data is reversed sorted so that every new element +# must be inserted in the heap: +# comparisons = log(k, 2) * (n - k) +# +# Alternative Algorithms +# ---------------------- +# Other algorithms were not used because they: +# 1) Took much more auxiliary memory, +# 2) Made multiple passes over the data. +# 3) Made more comparisons in common cases (small k, large n, semi-random input). +# See detailed comparisons at: +# http://code.activestate.com/recipes/577573-compare-algorithms-for-heapqsmallest + def nlargest(n, iterable): """Find the n largest elements in a dataset. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 05:56:07 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 05:56:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_PEP_465=3A_a_dedicated_inf?= =?utf-8?q?ix_operator_for_matrix_multiplication_=28closes_=2321176=29?= Message-ID: <3g47nM1hCMz7Lkc@mail.python.org> http://hg.python.org/cpython/rev/c553d8f72d65 changeset: 90206:c553d8f72d65 user: Benjamin Peterson date: Wed Apr 09 23:55:56 2014 -0400 summary: PEP 465: a dedicated infix operator for matrix multiplication (closes #21176) files: Doc/c-api/number.rst | 17 + Doc/c-api/typeobj.rst | 3 + Doc/library/dis.rst | 10 + Doc/library/operator.rst | 18 + Doc/library/token.rst | 1 + Doc/reference/datamodel.rst | 60 +- Doc/reference/expressions.rst | 17 +- Doc/reference/simple_stmts.rst | 2 +- Grammar/Grammar | 4 +- Include/Python-ast.h | 6 +- Include/abstract.h | 12 + Include/object.h | 3 + Include/opcode.h | 3 + Include/token.h | 13 +- Include/typeslots.h | 2 + Lib/importlib/_bootstrap.py | 3 +- Lib/opcode.py | 3 + Lib/operator.py | 11 + Lib/test/test_augassign.py | 15 + Lib/test/test_capi.py | 17 + Lib/test/test_descr.py | 1 + Lib/test/test_grammar.py | 14 + Lib/test/test_operator.py | 11 + Lib/test/test_sys.py | 2 +- Lib/test/test_tokenize.py | 5 +- Lib/token.py | 11 +- Lib/tokenize.py | 5 +- Misc/NEWS | 2 + Modules/_operator.c | 4 + Modules/_testcapimodule.c | 107 +++++ Objects/abstract.c | 14 + Objects/typeobject.c | 10 + Objects/typeslots.inc | 2 + Parser/Python.asdl | 2 +- Parser/tokenizer.c | 8 +- Python/Python-ast.c | 24 +- Python/ast.c | 8 +- Python/ceval.c | 24 + Python/compile.c | 6 + Python/graminit.c | 435 ++++++++++---------- Python/importlib.h | 320 +++++++------- Python/opcode_targets.h | 4 +- 42 files changed, 800 insertions(+), 439 deletions(-) diff --git a/Doc/c-api/number.rst b/Doc/c-api/number.rst --- a/Doc/c-api/number.rst +++ b/Doc/c-api/number.rst @@ -30,6 +30,14 @@ the equivalent of the Python expression ``o1 * o2``. +.. c:function:: PyObject* PyNumber_MatrixMultiply(PyObject *o1, PyObject *o2) + + Returns the result of matrix multiplication on *o1* and *o2*, or *NULL* on + failure. This is the equivalent of the Python expression ``o1 @ o2``. + + .. versionadded:: 3.5 + + .. c:function:: PyObject* PyNumber_FloorDivide(PyObject *o1, PyObject *o2) Return the floor of *o1* divided by *o2*, or *NULL* on failure. This is @@ -146,6 +154,15 @@ the Python statement ``o1 *= o2``. +.. c:function:: PyObject* PyNumber_InPlaceMatrixMultiply(PyObject *o1, PyObject *o2) + + Returns the result of matrix multiplication on *o1* and *o2*, or *NULL* on + failure. The operation is done *in-place* when *o1* supports it. This is + the equivalent of the Python statement ``o1 @= o2``. + + .. versionadded:: 3.5 + + .. c:function:: PyObject* PyNumber_InPlaceFloorDivide(PyObject *o1, PyObject *o2) Returns the mathematical floor of dividing *o1* by *o2*, or *NULL* on failure. diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1121,6 +1121,9 @@ binaryfunc nb_inplace_true_divide; unaryfunc nb_index; + + binaryfunc nb_matrix_multiply; + binaryfunc nb_inplace_matrix_multiply; } PyNumberMethods; .. note:: diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -364,6 +364,11 @@ Implements ``TOS = TOS1 * TOS``. +.. opcode:: BINARY_MATRIX_MULTIPLY + + Implements ``TOS = TOS1 @ TOS``. + + .. opcode:: BINARY_FLOOR_DIVIDE Implements ``TOS = TOS1 // TOS``. @@ -436,6 +441,11 @@ Implements in-place ``TOS = TOS1 * TOS``. +.. opcode:: INPLACE_MATRIX_MULTIPLY + + Implements in-place ``TOS = TOS1 @ TOS``. + + .. opcode:: INPLACE_FLOOR_DIVIDE Implements in-place ``TOS = TOS1 // TOS``. diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -138,6 +138,14 @@ Return ``a * b``, for *a* and *b* numbers. +.. function:: matmul(a, b) + __matmul__(a, b) + + Return ``a @ b``. + + .. versionadded:: 3.5 + + .. function:: neg(obj) __neg__(obj) @@ -400,6 +408,8 @@ +-----------------------+-------------------------+---------------------------------------+ | Multiplication | ``a * b`` | ``mul(a, b)`` | +-----------------------+-------------------------+---------------------------------------+ +| Matrix Multiplication | ``a @ b`` | ``matmul(a, b)`` | ++-----------------------+-------------------------+---------------------------------------+ | Negation (Arithmetic) | ``- a`` | ``neg(a)`` | +-----------------------+-------------------------+---------------------------------------+ | Negation (Logical) | ``not a`` | ``not_(a)`` | @@ -508,6 +518,14 @@ ``a = imul(a, b)`` is equivalent to ``a *= b``. +.. function:: imatmul(a, b) + __imatmul__(a, b) + + ``a = imatmul(a, b)`` is equivalent to ``a @= b``. + + .. versionadded:: 3.5 + + .. function:: ior(a, b) __ior__(a, b) diff --git a/Doc/library/token.rst b/Doc/library/token.rst --- a/Doc/library/token.rst +++ b/Doc/library/token.rst @@ -93,6 +93,7 @@ DOUBLESLASH DOUBLESLASHEQUAL AT + ATEQUAL RARROW ELLIPSIS OP diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1970,6 +1970,7 @@ .. method:: object.__add__(self, other) object.__sub__(self, other) object.__mul__(self, other) + object.__matmul__(self, other) object.__truediv__(self, other) object.__floordiv__(self, other) object.__mod__(self, other) @@ -1986,15 +1987,16 @@ builtin: pow builtin: pow - These methods are called to implement the binary arithmetic operations (``+``, - ``-``, ``*``, ``/``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``, - ``>>``, ``&``, ``^``, ``|``). For instance, to evaluate the expression - ``x + y``, where *x* is an instance of a class that has an :meth:`__add__` - method, ``x.__add__(y)`` is called. The :meth:`__divmod__` method should be the - equivalent to using :meth:`__floordiv__` and :meth:`__mod__`; it should not be - related to :meth:`__truediv__`. Note that :meth:`__pow__` should be defined - to accept an optional third argument if the ternary version of the built-in - :func:`pow` function is to be supported. + These methods are called to implement the binary arithmetic operations + (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`, + :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``). For instance, to + evaluate the expression ``x + y``, where *x* is an instance of a class that + has an :meth:`__add__` method, ``x.__add__(y)`` is called. The + :meth:`__divmod__` method should be the equivalent to using + :meth:`__floordiv__` and :meth:`__mod__`; it should not be related to + :meth:`__truediv__`. Note that :meth:`__pow__` should be defined to accept + an optional third argument if the ternary version of the built-in :func:`pow` + function is to be supported. If one of those methods does not support the operation with the supplied arguments, it should return ``NotImplemented``. @@ -2003,6 +2005,7 @@ .. method:: object.__radd__(self, other) object.__rsub__(self, other) object.__rmul__(self, other) + object.__rmatmul__(self, other) object.__rtruediv__(self, other) object.__rfloordiv__(self, other) object.__rmod__(self, other) @@ -2018,14 +2021,14 @@ builtin: divmod builtin: pow - These methods are called to implement the binary arithmetic operations (``+``, - ``-``, ``*``, ``/``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, - ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected (swapped) operands. - These functions are only called if the left operand does not support the - corresponding operation and the operands are of different types. [#]_ For - instance, to evaluate the expression ``x - y``, where *y* is an instance of - a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)`` is called if - ``x.__sub__(y)`` returns *NotImplemented*. + These methods are called to implement the binary arithmetic operations + (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`, + :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected + (swapped) operands. These functions are only called if the left operand does + not support the corresponding operation and the operands are of different + types. [#]_ For instance, to evaluate the expression ``x - y``, where *y* is + an instance of a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)`` + is called if ``x.__sub__(y)`` returns *NotImplemented*. .. index:: builtin: pow @@ -2043,6 +2046,7 @@ .. method:: object.__iadd__(self, other) object.__isub__(self, other) object.__imul__(self, other) + object.__imatmul__(self, other) object.__itruediv__(self, other) object.__ifloordiv__(self, other) object.__imod__(self, other) @@ -2054,17 +2058,17 @@ object.__ior__(self, other) These methods are called to implement the augmented arithmetic assignments - (``+=``, ``-=``, ``*=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``, ``>>=``, - ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation - in-place (modifying *self*) and return the result (which could be, but does - not have to be, *self*). If a specific method is not defined, the augmented - assignment falls back to the normal methods. For instance, if *x* is an - instance of a class with an :meth:`__iadd__` method, ``x += y`` is equivalent - to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and ``y.__radd__(x)`` - are considered, as with the evaluation of ``x + y``. In certain situations, - augmented assignment can result in unexpected errors (see - :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in - fact part of the data model. + (``+=``, ``-=``, ``*=``, ``@=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``, + ``>>=``, ``&=``, ``^=``, ``|=``). These methods should attempt to do the + operation in-place (modifying *self*) and return the result (which could be, + but does not have to be, *self*). If a specific method is not defined, the + augmented assignment falls back to the normal methods. For instance, if *x* + is an instance of a class with an :meth:`__iadd__` method, ``x += y`` is + equivalent to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and + ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``. In + certain situations, augmented assignment can result in unexpected errors (see + :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in fact + part of the data model. .. method:: object.__neg__(self) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -892,8 +892,9 @@ operators and one for additive operators: .. productionlist:: - m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "//" `u_expr` | `m_expr` "/" `u_expr` - : | `m_expr` "%" `u_expr` + m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "@" `m_expr` | + : `m_expr` "//" `u_expr`| `m_expr` "/" `u_expr` | + : `m_expr` "%" `u_expr` a_expr: `m_expr` | `a_expr` "+" `m_expr` | `a_expr` "-" `m_expr` .. index:: single: multiplication @@ -904,6 +905,13 @@ common type and then multiplied together. In the latter case, sequence repetition is performed; a negative repetition factor yields an empty sequence. +.. index:: single: matrix multiplication + +The ``@`` (at) operator is intended to be used for matrix multiplication. No +builtin Python types implement this operator. + +.. versionadded:: 3.5 + .. index:: exception: ZeroDivisionError single: division @@ -1346,8 +1354,9 @@ +-----------------------------------------------+-------------------------------------+ | ``+``, ``-`` | Addition and subtraction | +-----------------------------------------------+-------------------------------------+ -| ``*``, ``/``, ``//``, ``%`` | Multiplication, division, remainder | -| | [#]_ | +| ``*``, ``@``, ``/``, ``//``, ``%`` | Multiplication, matrix | +| | multiplication division, | +| | remainder [#]_ | +-----------------------------------------------+-------------------------------------+ | ``+x``, ``-x``, ``~x`` | Positive, negative, bitwise NOT | +-----------------------------------------------+-------------------------------------+ diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -267,7 +267,7 @@ .. productionlist:: augmented_assignment_stmt: `augtarget` `augop` (`expression_list` | `yield_expression`) augtarget: `identifier` | `attributeref` | `subscription` | `slicing` - augop: "+=" | "-=" | "*=" | "/=" | "//=" | "%=" | "**=" + augop: "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**=" : | ">>=" | "<<=" | "&=" | "^=" | "|=" (See section :ref:`primaries` for the syntax definitions for the last three diff --git a/Grammar/Grammar b/Grammar/Grammar --- a/Grammar/Grammar +++ b/Grammar/Grammar @@ -40,7 +40,7 @@ expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] -augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | +augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') # For normal assignments, additional restrictions enforced by the interpreter del_stmt: 'del' exprlist @@ -97,7 +97,7 @@ and_expr: shift_expr ('&' shift_expr)* shift_expr: arith_expr (('<<'|'>>') arith_expr)* arith_expr: term (('+'|'-') term)* -term: factor (('*'|'/'|'%'|'//') factor)* +term: factor (('*'|'@'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ['**' factor] atom: ('(' [yield_expr|testlist_comp] ')' | diff --git a/Include/Python-ast.h b/Include/Python-ast.h --- a/Include/Python-ast.h +++ b/Include/Python-ast.h @@ -15,9 +15,9 @@ typedef enum _boolop { And=1, Or=2 } boolop_ty; -typedef enum _operator { Add=1, Sub=2, Mult=3, Div=4, Mod=5, Pow=6, LShift=7, - RShift=8, BitOr=9, BitXor=10, BitAnd=11, FloorDiv=12 } - operator_ty; +typedef enum _operator { Add=1, Sub=2, Mult=3, MatMult=4, Div=5, Mod=6, Pow=7, + LShift=8, RShift=9, BitOr=10, BitXor=11, BitAnd=12, + FloorDiv=13 } operator_ty; typedef enum _unaryop { Invert=1, Not=2, UAdd=3, USub=4 } unaryop_ty; diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -658,6 +658,12 @@ o1*o2. */ + PyAPI_FUNC(PyObject *) PyNumber_MatrixMultiply(PyObject *o1, PyObject *o2); + + /* + This is the equivalent of the Python expression: o1 @ o2. + */ + PyAPI_FUNC(PyObject *) PyNumber_FloorDivide(PyObject *o1, PyObject *o2); /* @@ -832,6 +838,12 @@ o1 *= o2. */ + PyAPI_FUNC(PyObject *) PyNumber_InPlaceMatrixMultiply(PyObject *o1, PyObject *o2); + + /* + This is the equivalent of the Python expression: o1 @= o2. + */ + PyAPI_FUNC(PyObject *) PyNumber_InPlaceFloorDivide(PyObject *o1, PyObject *o2); diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -275,6 +275,9 @@ binaryfunc nb_inplace_true_divide; unaryfunc nb_index; + + binaryfunc nb_matrix_multiply; + binaryfunc nb_inplace_matrix_multiply; } PyNumberMethods; typedef struct { diff --git a/Include/opcode.h b/Include/opcode.h --- a/Include/opcode.h +++ b/Include/opcode.h @@ -20,6 +20,9 @@ #define UNARY_INVERT 15 +#define BINARY_MATRIX_MULTIPLY 16 +#define INPLACE_MATRIX_MULTIPLY 17 + #define BINARY_POWER 19 #define BINARY_MULTIPLY 20 diff --git a/Include/token.h b/Include/token.h --- a/Include/token.h +++ b/Include/token.h @@ -58,13 +58,14 @@ #define DOUBLESTAREQUAL 46 #define DOUBLESLASH 47 #define DOUBLESLASHEQUAL 48 -#define AT 49 -#define RARROW 50 -#define ELLIPSIS 51 +#define AT 49 +#define ATEQUAL 50 +#define RARROW 51 +#define ELLIPSIS 52 /* Don't forget to update the table _PyParser_TokenNames in tokenizer.c! */ -#define OP 52 -#define ERRORTOKEN 53 -#define N_TOKENS 54 +#define OP 53 +#define ERRORTOKEN 54 +#define N_TOKENS 55 /* Special definitions for cooperation with parser */ diff --git a/Include/typeslots.h b/Include/typeslots.h --- a/Include/typeslots.h +++ b/Include/typeslots.h @@ -74,3 +74,5 @@ #define Py_tp_members 72 #define Py_tp_getset 73 #define Py_tp_free 74 +#define Py_nb_matrix_multiply 75 +#define Py_nb_inplace_matrix_multiply 76 diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -419,12 +419,13 @@ # Python 3.4a4 3290 (changes to __qualname__ computation) # Python 3.4a4 3300 (more changes to __qualname__ computation) # Python 3.4rc2 3310 (alter __qualname__ computation) +# Python 3.5a0 3320 (matrix multiplication operator) # # MAGIC must change whenever the bytecode emitted by the compiler may no # longer be understood by older implementations of the eval loop (usually # due to the addition of new opcodes). -MAGIC_NUMBER = (3310).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3320).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/opcode.py b/Lib/opcode.py --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -70,6 +70,9 @@ def_op('UNARY_INVERT', 15) +def_op('BINARY_MATRIX_MULTIPLY', 16) +def_op('INPLACE_MATRIX_MULTIPLY', 17) + def_op('BINARY_POWER', 19) def_op('BINARY_MULTIPLY', 20) diff --git a/Lib/operator.py b/Lib/operator.py --- a/Lib/operator.py +++ b/Lib/operator.py @@ -105,6 +105,10 @@ "Same as a * b." return a * b +def matmul(a, b): + "Same as a @ b." + return a @ b + def neg(a): "Same as -a." return -a @@ -326,6 +330,11 @@ a *= b return a +def imatmul(a, b): + "Same as a @= b." + a @= b + return a + def ior(a, b): "Same as a |= b." a |= b @@ -383,6 +392,7 @@ __lshift__ = lshift __mod__ = mod __mul__ = mul +__matmul__ = matmul __neg__ = neg __or__ = or_ __pos__ = pos @@ -403,6 +413,7 @@ __ilshift__ = ilshift __imod__ = imod __imul__ = imul +__imatmul__ = imatmul __ior__ = ior __ipow__ = ipow __irshift__ = irshift diff --git a/Lib/test/test_augassign.py b/Lib/test/test_augassign.py --- a/Lib/test/test_augassign.py +++ b/Lib/test/test_augassign.py @@ -136,6 +136,14 @@ output.append("__imul__ called") return self + def __matmul__(self, val): + output.append("__matmul__ called") + def __rmatmul__(self, val): + output.append("__rmatmul__ called") + def __imatmul__(self, val): + output.append("__imatmul__ called") + return self + def __div__(self, val): output.append("__div__ called") def __rdiv__(self, val): @@ -233,6 +241,10 @@ 1 * x x *= 1 + x @ 1 + 1 @ x + x @= 1 + x / 1 1 / x x /= 1 @@ -279,6 +291,9 @@ __mul__ called __rmul__ called __imul__ called +__matmul__ called +__rmatmul__ called +__imatmul__ called __truediv__ called __rtruediv__ called __itruediv__ called diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -150,6 +150,23 @@ self.assertEqual(_testcapi.docstring_with_signature_and_extra_newlines.__text_signature__, "($module, /, parameter)") + def test_c_type_with_matrix_multiplication(self): + M = _testcapi.matmulType + m1 = M() + m2 = M() + self.assertEqual(m1 @ m2, ("matmul", m1, m2)) + self.assertEqual(m1 @ 42, ("matmul", m1, 42)) + self.assertEqual(42 @ m1, ("matmul", 42, m1)) + o = m1 + o @= m2 + self.assertEqual(o, ("imatmul", m1, m2)) + o = m1 + o @= 42 + self.assertEqual(o, ("imatmul", m1, 42)) + o = 42 + o @= m1 + self.assertEqual(o, ("matmul", 42, m1)) + @unittest.skipUnless(threading, 'Threading required for this test.') class TestPendingCalls(unittest.TestCase): diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -4160,6 +4160,7 @@ ('__add__', 'x + y', 'x += y'), ('__sub__', 'x - y', 'x -= y'), ('__mul__', 'x * y', 'x *= y'), + ('__matmul__', 'x @ y', 'x @= y'), ('__truediv__', 'operator.truediv(x, y)', None), ('__floordiv__', 'operator.floordiv(x, y)', None), ('__div__', 'x / y', 'x /= y'), diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -985,6 +985,20 @@ self.assertFalse((False is 2) is 3) self.assertFalse(False is 2 is 3) + def test_matrix_mul(self): + # This is not intended to be a comprehensive test, rather just to be few + # samples of the @ operator in test_grammar.py. + class M: + def __matmul__(self, o): + return 4 + def __imatmul__(self, o): + self.other = o + return self + m = M() + self.assertEqual(m @ m, 4) + m @= 42 + self.assertEqual(m.other, 42) + def test_main(): run_unittest(TokenTests, GrammarTests) diff --git a/Lib/test/test_operator.py b/Lib/test/test_operator.py --- a/Lib/test/test_operator.py +++ b/Lib/test/test_operator.py @@ -203,6 +203,15 @@ self.assertRaises(TypeError, operator.mul, None, None) self.assertTrue(operator.mul(5, 2) == 10) + def test_matmul(self): + operator = self.module + self.assertRaises(TypeError, operator.matmul) + self.assertRaises(TypeError, operator.matmul, 42, 42) + class M: + def __matmul__(self, other): + return other - 1 + self.assertEqual(M() @ 42, 41) + def test_neg(self): operator = self.module self.assertRaises(TypeError, operator.neg) @@ -416,6 +425,7 @@ def __ilshift__ (self, other): return "ilshift" def __imod__ (self, other): return "imod" def __imul__ (self, other): return "imul" + def __imatmul__ (self, other): return "imatmul" def __ior__ (self, other): return "ior" def __ipow__ (self, other): return "ipow" def __irshift__ (self, other): return "irshift" @@ -430,6 +440,7 @@ self.assertEqual(operator.ilshift (c, 5), "ilshift") self.assertEqual(operator.imod (c, 5), "imod") self.assertEqual(operator.imul (c, 5), "imul") + self.assertEqual(operator.imatmul (c, 5), "imatmul") self.assertEqual(operator.ior (c, 5), "ior") self.assertEqual(operator.ipow (c, 5), "ipow") self.assertEqual(operator.irshift (c, 5), "irshift") diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -952,7 +952,7 @@ check(int, s) # (PyTypeObject + PyNumberMethods + PyMappingMethods + # PySequenceMethods + PyBufferProcs + 4P) - s = vsize('P2n15Pl4Pn9Pn11PIP') + struct.calcsize('34P 3P 10P 2P 4P') + s = vsize('P2n17Pl4Pn9Pn11PIP') + struct.calcsize('34P 3P 10P 2P 4P') # Separate block for PyDictKeysObject with 4 entries s += struct.calcsize("2nPn") + 4*struct.calcsize("n2P") # class diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -464,7 +464,7 @@ Multiplicative - >>> dump_tokens("x = 1//1*1/5*12%0x12") + >>> dump_tokens("x = 1//1*1/5*12%0x12 at 42") ENCODING 'utf-8' (0, 0) (0, 0) NAME 'x' (1, 0) (1, 1) OP '=' (1, 2) (1, 3) @@ -479,6 +479,8 @@ NUMBER '12' (1, 13) (1, 15) OP '%' (1, 15) (1, 16) NUMBER '0x12' (1, 16) (1, 20) + OP '@' (1, 20) (1, 21) + NUMBER '42' (1, 21) (1, 23) Unary @@ -1154,6 +1156,7 @@ self.assertExactTypeEqual('//', token.DOUBLESLASH) self.assertExactTypeEqual('//=', token.DOUBLESLASHEQUAL) self.assertExactTypeEqual('@', token.AT) + self.assertExactTypeEqual('@=', token.ATEQUAL) self.assertExactTypeEqual('a**2+b**2==c**2', NAME, token.DOUBLESTAR, NUMBER, diff --git a/Lib/token.py b/Lib/token.py --- a/Lib/token.py +++ b/Lib/token.py @@ -60,11 +60,12 @@ DOUBLESLASH = 47 DOUBLESLASHEQUAL = 48 AT = 49 -RARROW = 50 -ELLIPSIS = 51 -OP = 52 -ERRORTOKEN = 53 -N_TOKENS = 54 +ATEQUAL = 50 +RARROW = 51 +ELLIPSIS = 52 +OP = 53 +ERRORTOKEN = 54 +N_TOKENS = 55 NT_OFFSET = 256 #--end constants-- diff --git a/Lib/tokenize.py b/Lib/tokenize.py --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -91,7 +91,8 @@ '**=': DOUBLESTAREQUAL, '//': DOUBLESLASH, '//=': DOUBLESLASHEQUAL, - '@': AT + '@': AT, + '@=': ATEQUAL, } class TokenInfo(collections.namedtuple('TokenInfo', 'type string start end line')): @@ -150,7 +151,7 @@ # recognized as two instances of =). Operator = group(r"\*\*=?", r">>=?", r"<<=?", r"!=", r"//=?", r"->", - r"[+\-*/%&|^=<>]=?", + r"[+\-*/%&@|^=<>]=?", r"~") Bracket = '[][(){}]' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- PEP 465 and Issue #21176: Add the '@' operator for matrix multiplication. + - Issue #21134: Fix segfault when str is called on an uninitialized UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. diff --git a/Modules/_operator.c b/Modules/_operator.c --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -69,6 +69,7 @@ spam2(op_add , PyNumber_Add) spam2(op_sub , PyNumber_Subtract) spam2(op_mul , PyNumber_Multiply) +spam2(op_matmul , PyNumber_MatrixMultiply) spam2(op_floordiv , PyNumber_FloorDivide) spam2(op_truediv , PyNumber_TrueDivide) spam2(op_mod , PyNumber_Remainder) @@ -86,6 +87,7 @@ spam2(op_iadd , PyNumber_InPlaceAdd) spam2(op_isub , PyNumber_InPlaceSubtract) spam2(op_imul , PyNumber_InPlaceMultiply) +spam2(op_imatmul , PyNumber_InPlaceMatrixMultiply) spam2(op_ifloordiv , PyNumber_InPlaceFloorDivide) spam2(op_itruediv , PyNumber_InPlaceTrueDivide) spam2(op_imod , PyNumber_InPlaceRemainder) @@ -343,6 +345,7 @@ spam2(add, "add(a, b) -- Same as a + b.") spam2(sub, "sub(a, b) -- Same as a - b.") spam2(mul, "mul(a, b) -- Same as a * b.") +spam2(matmul, "matmul(a, b) -- Same as a @ b.") spam2(floordiv, "floordiv(a, b) -- Same as a // b.") spam2(truediv, "truediv(a, b) -- Same as a / b.") spam2(mod, "mod(a, b) -- Same as a % b.") @@ -360,6 +363,7 @@ spam2(iadd, "a = iadd(a, b) -- Same as a += b.") spam2(isub, "a = isub(a, b) -- Same as a -= b.") spam2(imul, "a = imul(a, b) -- Same as a *= b.") +spam2(imatmul, "a = imatmul(a, b) -- Same as a @= b.") spam2(ifloordiv, "a = ifloordiv(a, b) -- Same as a //= b.") spam2(itruediv, "a = itruediv(a, b) -- Same as a /= b") spam2(imod, "a = imod(a, b) -- Same as a %= b.") diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3298,6 +3298,109 @@ }; +typedef struct { + PyObject_HEAD +} matmulObject; + +static PyObject * +matmulType_matmul(PyObject *self, PyObject *other) +{ + return Py_BuildValue("(sOO)", "matmul", self, other); +} + +static PyObject * +matmulType_imatmul(PyObject *self, PyObject *other) +{ + return Py_BuildValue("(sOO)", "imatmul", self, other); +} + +static void +matmulType_dealloc(PyObject *self) +{ + return Py_TYPE(self)->tp_free(self); +} + +static PyNumberMethods matmulType_as_number = { + 0, /* nb_add */ + 0, /* nb_subtract */ + 0, /* nb_multiply */ + 0, /* nb_remainde r*/ + 0, /* nb_divmod */ + 0, /* nb_power */ + 0, /* nb_negative */ + 0, /* tp_positive */ + 0, /* tp_absolute */ + 0, /* tp_bool */ + 0, /* nb_invert */ + 0, /* nb_lshift */ + 0, /* nb_rshift */ + 0, /* nb_and */ + 0, /* nb_xor */ + 0, /* nb_or */ + 0, /* nb_int */ + 0, /* nb_reserved */ + 0, /* nb_float */ + 0, /* nb_inplace_add */ + 0, /* nb_inplace_subtract */ + 0, /* nb_inplace_multiply */ + 0, /* nb_inplace_remainder */ + 0, /* nb_inplace_power */ + 0, /* nb_inplace_lshift */ + 0, /* nb_inplace_rshift */ + 0, /* nb_inplace_and */ + 0, /* nb_inplace_xor */ + 0, /* nb_inplace_or */ + 0, /* nb_floor_divide */ + 0, /* nb_true_divide */ + 0, /* nb_inplace_floor_divide */ + 0, /* nb_inplace_true_divide */ + 0, /* nb_index */ + matmulType_matmul, /* nb_matrix_multiply */ + matmulType_imatmul /* nb_matrix_inplace_multiply */ +}; + +static PyTypeObject matmulType = { + PyVarObject_HEAD_INIT(NULL, 0) + "matmulType", + sizeof(matmulObject), /* tp_basicsize */ + 0, /* tp_itemsize */ + matmulType_dealloc, /* destructor tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_reserved */ + 0, /* tp_repr */ + &matmulType_as_number, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + PyObject_GenericSetAttr, /* tp_setattro */ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + "C level type with matrix operations defined", + 0, /* traverseproc tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + 0, /* tp_members */ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + PyType_GenericNew, /* tp_new */ + PyObject_Del, /* tp_free */ +}; + static struct PyModuleDef _testcapimodule = { PyModuleDef_HEAD_INIT, @@ -3327,6 +3430,10 @@ /* don't use a name starting with "test", since we don't want test_capi to automatically call this */ PyModule_AddObject(m, "_test_structmembersType", (PyObject *)&test_structmembersType); + if (PyType_Ready(&matmulType) < 0) + return NULL; + Py_INCREF(&matmulType); + PyModule_AddObject(m, "matmulType", (PyObject *)&matmulType); PyModule_AddObject(m, "CHAR_MAX", PyLong_FromLong(CHAR_MAX)); PyModule_AddObject(m, "CHAR_MIN", PyLong_FromLong(CHAR_MIN)); diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -932,6 +932,12 @@ } PyObject * +PyNumber_MatrixMultiply(PyObject *v, PyObject *w) +{ + return binary_op(v, w, NB_SLOT(nb_matrix_multiply), "@"); +} + +PyObject * PyNumber_FloorDivide(PyObject *v, PyObject *w) { return binary_op(v, w, NB_SLOT(nb_floor_divide), "//"); @@ -1012,6 +1018,7 @@ INPLACE_BINOP(PyNumber_InPlaceLshift, nb_inplace_lshift, nb_lshift, "<<=") INPLACE_BINOP(PyNumber_InPlaceRshift, nb_inplace_rshift, nb_rshift, ">>=") INPLACE_BINOP(PyNumber_InPlaceSubtract, nb_inplace_subtract, nb_subtract, "-=") +INPLACE_BINOP(PyNumber_InMatrixMultiply, nb_inplace_matrix_multiply, nb_matrix_multiply, "@=") PyObject * PyNumber_InPlaceFloorDivide(PyObject *v, PyObject *w) @@ -1078,6 +1085,13 @@ } PyObject * +PyNumber_InPlaceMatrixMultiply(PyObject *v, PyObject *w) +{ + return binary_iop(v, w, NB_SLOT(nb_inplace_matrix_multiply), + NB_SLOT(nb_matrix_multiply), "@="); +} + +PyObject * PyNumber_InPlaceRemainder(PyObject *v, PyObject *w) { return binary_iop(v, w, NB_SLOT(nb_inplace_remainder), diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -4469,6 +4469,8 @@ COPYNUM(nb_inplace_true_divide); COPYNUM(nb_inplace_floor_divide); COPYNUM(nb_index); + COPYNUM(nb_matrix_multiply); + COPYNUM(nb_inplace_matrix_multiply); } if (type->tp_as_sequence != NULL && base->tp_as_sequence != NULL) { @@ -5605,6 +5607,7 @@ SLOT1BIN(slot_nb_add, nb_add, "__add__", "__radd__") SLOT1BIN(slot_nb_subtract, nb_subtract, "__sub__", "__rsub__") SLOT1BIN(slot_nb_multiply, nb_multiply, "__mul__", "__rmul__") +SLOT1BIN(slot_nb_matrix_multiply, nb_matrix_multiply, "__matmul__", "__rmatmul__") SLOT1BIN(slot_nb_remainder, nb_remainder, "__mod__", "__rmod__") SLOT1BIN(slot_nb_divmod, nb_divmod, "__divmod__", "__rdivmod__") @@ -5698,6 +5701,7 @@ SLOT1(slot_nb_inplace_add, "__iadd__", PyObject *, "O") SLOT1(slot_nb_inplace_subtract, "__isub__", PyObject *, "O") SLOT1(slot_nb_inplace_multiply, "__imul__", PyObject *, "O") +SLOT1(slot_nb_inplace_matrix_multiply, "__imatmul__", PyObject *, "O") SLOT1(slot_nb_inplace_remainder, "__imod__", PyObject *, "O") /* Can't use SLOT1 here, because nb_inplace_power is ternary */ static PyObject * @@ -6278,6 +6282,12 @@ "__index__($self, /)\n--\n\n" "Return self converted to an integer, if self is suitable" "for use as an index into a list."), + BINSLOT("__matmul__", nb_matrix_multiply, slot_nb_matrix_multiply, + "@"), + RBINSLOT("__rmatmul__", nb_matrix_multiply, slot_nb_matrix_multiply, + "@"), + IBSLOT("__imatmul__", nb_inplace_matrix_multiply, slot_nb_inplace_matrix_multiply, + wrap_binaryfunc, "@="), MPSLOT("__len__", mp_length, slot_mp_length, wrap_lenfunc, "__len__($self, /)\n--\n\nReturn len(self)."), MPSLOT("__getitem__", mp_subscript, slot_mp_subscript, diff --git a/Objects/typeslots.inc b/Objects/typeslots.inc --- a/Objects/typeslots.inc +++ b/Objects/typeslots.inc @@ -73,3 +73,5 @@ offsetof(PyHeapTypeObject, ht_type.tp_members), offsetof(PyHeapTypeObject, ht_type.tp_getset), offsetof(PyHeapTypeObject, ht_type.tp_free), +offsetof(PyHeapTypeObject, as_number.nb_matrix_multiply), +offsetof(PyHeapTypeObject, as_number.nb_inplace_matrix_multiply), diff --git a/Parser/Python.asdl b/Parser/Python.asdl --- a/Parser/Python.asdl +++ b/Parser/Python.asdl @@ -91,7 +91,7 @@ boolop = And | Or - operator = Add | Sub | Mult | Div | Mod | Pow | LShift + operator = Add | Sub | Mult | MatMult | Div | Mod | Pow | LShift | RShift | BitOr | BitXor | BitAnd | FloorDiv unaryop = Invert | Not | UAdd | USub diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -98,6 +98,7 @@ "DOUBLESLASH", "DOUBLESLASHEQUAL", "AT", + "ATEQUAL", "RARROW", "ELLIPSIS", /* This table must match the #defines in token.h! */ @@ -1131,7 +1132,7 @@ case '}': return RBRACE; case '^': return CIRCUMFLEX; case '~': return TILDE; - case '@': return AT; + case '@': return AT; default: return OP; } } @@ -1207,6 +1208,11 @@ case '=': return CIRCUMFLEXEQUAL; } break; + case '@': + switch (c2) { + case '=': return ATEQUAL; + } + break; } return OP; } diff --git a/Python/Python-ast.c b/Python/Python-ast.c --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -349,13 +349,14 @@ static PyTypeObject *Or_type; static PyTypeObject *operator_type; static PyObject *Add_singleton, *Sub_singleton, *Mult_singleton, -*Div_singleton, *Mod_singleton, *Pow_singleton, *LShift_singleton, -*RShift_singleton, *BitOr_singleton, *BitXor_singleton, *BitAnd_singleton, -*FloorDiv_singleton; +*MatMult_singleton, *Div_singleton, *Mod_singleton, *Pow_singleton, +*LShift_singleton, *RShift_singleton, *BitOr_singleton, *BitXor_singleton, +*BitAnd_singleton, *FloorDiv_singleton; static PyObject* ast2obj_operator(operator_ty); static PyTypeObject *Add_type; static PyTypeObject *Sub_type; static PyTypeObject *Mult_type; +static PyTypeObject *MatMult_type; static PyTypeObject *Div_type; static PyTypeObject *Mod_type; static PyTypeObject *Pow_type; @@ -970,6 +971,10 @@ if (!Mult_type) return 0; Mult_singleton = PyType_GenericNew(Mult_type, NULL, NULL); if (!Mult_singleton) return 0; + MatMult_type = make_type("MatMult", operator_type, NULL, 0); + if (!MatMult_type) return 0; + MatMult_singleton = PyType_GenericNew(MatMult_type, NULL, NULL); + if (!MatMult_singleton) return 0; Div_type = make_type("Div", operator_type, NULL, 0); if (!Div_type) return 0; Div_singleton = PyType_GenericNew(Div_type, NULL, NULL); @@ -3232,6 +3237,9 @@ case Mult: Py_INCREF(Mult_singleton); return Mult_singleton; + case MatMult: + Py_INCREF(MatMult_singleton); + return MatMult_singleton; case Div: Py_INCREF(Div_singleton); return Div_singleton; @@ -6175,6 +6183,14 @@ *out = Mult; return 0; } + isinstance = PyObject_IsInstance(obj, (PyObject *)MatMult_type); + if (isinstance == -1) { + return 1; + } + if (isinstance) { + *out = MatMult; + return 0; + } isinstance = PyObject_IsInstance(obj, (PyObject *)Div_type); if (isinstance == -1) { return 1; @@ -6956,6 +6972,8 @@ if (PyDict_SetItemString(d, "Add", (PyObject*)Add_type) < 0) return NULL; if (PyDict_SetItemString(d, "Sub", (PyObject*)Sub_type) < 0) return NULL; if (PyDict_SetItemString(d, "Mult", (PyObject*)Mult_type) < 0) return NULL; + if (PyDict_SetItemString(d, "MatMult", (PyObject*)MatMult_type) < 0) return + NULL; if (PyDict_SetItemString(d, "Div", (PyObject*)Div_type) < 0) return NULL; if (PyDict_SetItemString(d, "Mod", (PyObject*)Mod_type) < 0) return NULL; if (PyDict_SetItemString(d, "Pow", (PyObject*)Pow_type) < 0) return NULL; diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -825,6 +825,8 @@ return Sub; case STAR: return Mult; + case AT: + return MatMult; case SLASH: return Div; case DOUBLESLASH: @@ -1030,6 +1032,8 @@ return Pow; else return Mult; + case '@': + return MatMult; default: PyErr_Format(PyExc_SystemError, "invalid augassign: %s", STR(n)); return (operator_ty)0; @@ -2266,7 +2270,7 @@ and_expr: shift_expr ('&' shift_expr)* shift_expr: arith_expr (('<<'|'>>') arith_expr)* arith_expr: term (('+'|'-') term)* - term: factor (('*'|'/'|'%'|'//') factor)* + term: factor (('*'|'@'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ('**' factor)* */ @@ -2577,7 +2581,7 @@ /* expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist))*) testlist_star_expr: (test|star_expr) (',' test|star_expr)* [','] - augassign: '+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' + augassign: '+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=' test: ... here starts the operator precendence dance */ diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1495,6 +1495,18 @@ DISPATCH(); } + TARGET(BINARY_MATRIX_MULTIPLY) { + PyObject *right = POP(); + PyObject *left = TOP(); + PyObject *res = PyNumber_MatrixMultiply(left, right); + Py_DECREF(left); + Py_DECREF(right); + SET_TOP(res); + if (res == NULL) + goto error; + DISPATCH(); + } + TARGET(BINARY_TRUE_DIVIDE) { PyObject *divisor = POP(); PyObject *dividend = TOP(); @@ -1685,6 +1697,18 @@ DISPATCH(); } + TARGET(INPLACE_MATRIX_MULTIPLY) { + PyObject *right = POP(); + PyObject *left = TOP(); + PyObject *res = PyNumber_InPlaceMatrixMultiply(left, right); + Py_DECREF(left); + Py_DECREF(right); + SET_TOP(res); + if (res == NULL) + goto error; + DISPATCH(); + } + TARGET(INPLACE_TRUE_DIVIDE) { PyObject *divisor = POP(); PyObject *dividend = TOP(); diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -881,6 +881,7 @@ case BINARY_POWER: case BINARY_MULTIPLY: + case BINARY_MATRIX_MULTIPLY: case BINARY_MODULO: case BINARY_ADD: case BINARY_SUBTRACT: @@ -895,6 +896,7 @@ case INPLACE_ADD: case INPLACE_SUBTRACT: case INPLACE_MULTIPLY: + case INPLACE_MATRIX_MULTIPLY: case INPLACE_MODULO: return -1; case STORE_SUBSCR: @@ -2625,6 +2627,8 @@ return BINARY_SUBTRACT; case Mult: return BINARY_MULTIPLY; + case MatMult: + return BINARY_MATRIX_MULTIPLY; case Div: return BINARY_TRUE_DIVIDE; case Mod: @@ -2689,6 +2693,8 @@ return INPLACE_SUBTRACT; case Mult: return INPLACE_MULTIPLY; + case MatMult: + return INPLACE_MATRIX_MULTIPLY; case Div: return INPLACE_TRUE_DIVIDE; case Mod: diff --git a/Python/graminit.c b/Python/graminit.c --- a/Python/graminit.c +++ b/Python/graminit.c @@ -476,7 +476,7 @@ {2, arcs_16_1}, {3, arcs_16_2}, }; -static arc arcs_17_0[12] = { +static arc arcs_17_0[13] = { {49, 1}, {50, 1}, {51, 1}, @@ -489,19 +489,20 @@ {58, 1}, {59, 1}, {60, 1}, + {61, 1}, }; static arc arcs_17_1[1] = { {0, 1}, }; static state states_17[2] = { - {12, arcs_17_0}, + {13, arcs_17_0}, {1, arcs_17_1}, }; static arc arcs_18_0[1] = { - {61, 1}, + {62, 1}, }; static arc arcs_18_1[1] = { - {62, 2}, + {63, 2}, }; static arc arcs_18_2[1] = { {0, 2}, @@ -512,7 +513,7 @@ {1, arcs_18_2}, }; static arc arcs_19_0[1] = { - {63, 1}, + {64, 1}, }; static arc arcs_19_1[1] = { {0, 1}, @@ -522,11 +523,11 @@ {1, arcs_19_1}, }; static arc arcs_20_0[5] = { - {64, 1}, {65, 1}, {66, 1}, {67, 1}, {68, 1}, + {69, 1}, }; static arc arcs_20_1[1] = { {0, 1}, @@ -536,7 +537,7 @@ {1, arcs_20_1}, }; static arc arcs_21_0[1] = { - {69, 1}, + {70, 1}, }; static arc arcs_21_1[1] = { {0, 1}, @@ -546,7 +547,7 @@ {1, arcs_21_1}, }; static arc arcs_22_0[1] = { - {70, 1}, + {71, 1}, }; static arc arcs_22_1[1] = { {0, 1}, @@ -556,7 +557,7 @@ {1, arcs_22_1}, }; static arc arcs_23_0[1] = { - {71, 1}, + {72, 1}, }; static arc arcs_23_1[2] = { {9, 2}, @@ -581,14 +582,14 @@ {1, arcs_24_1}, }; static arc arcs_25_0[1] = { - {72, 1}, + {73, 1}, }; static arc arcs_25_1[2] = { {24, 2}, {0, 1}, }; static arc arcs_25_2[2] = { - {73, 3}, + {74, 3}, {0, 2}, }; static arc arcs_25_3[1] = { @@ -605,8 +606,8 @@ {1, arcs_25_4}, }; static arc arcs_26_0[2] = { - {74, 1}, {75, 1}, + {76, 1}, }; static arc arcs_26_1[1] = { {0, 1}, @@ -616,10 +617,10 @@ {1, arcs_26_1}, }; static arc arcs_27_0[1] = { - {76, 1}, + {77, 1}, }; static arc arcs_27_1[1] = { - {77, 2}, + {78, 2}, }; static arc arcs_27_2[1] = { {0, 2}, @@ -630,32 +631,32 @@ {1, arcs_27_2}, }; static arc arcs_28_0[1] = { - {73, 1}, + {74, 1}, }; static arc arcs_28_1[3] = { - {78, 2}, {79, 2}, + {80, 2}, {12, 3}, }; static arc arcs_28_2[4] = { - {78, 2}, {79, 2}, + {80, 2}, {12, 3}, - {76, 4}, + {77, 4}, }; static arc arcs_28_3[1] = { - {76, 4}, + {77, 4}, }; static arc arcs_28_4[3] = { {31, 5}, {13, 6}, - {80, 5}, + {81, 5}, }; static arc arcs_28_5[1] = { {0, 5}, }; static arc arcs_28_6[1] = { - {80, 7}, + {81, 7}, }; static arc arcs_28_7[1] = { {15, 5}, @@ -674,7 +675,7 @@ {21, 1}, }; static arc arcs_29_1[2] = { - {82, 2}, + {83, 2}, {0, 1}, }; static arc arcs_29_2[1] = { @@ -693,7 +694,7 @@ {12, 1}, }; static arc arcs_30_1[2] = { - {82, 2}, + {83, 2}, {0, 1}, }; static arc arcs_30_2[1] = { @@ -709,14 +710,14 @@ {1, arcs_30_3}, }; static arc arcs_31_0[1] = { - {81, 1}, + {82, 1}, }; static arc arcs_31_1[2] = { {30, 2}, {0, 1}, }; static arc arcs_31_2[2] = { - {81, 1}, + {82, 1}, {0, 2}, }; static state states_31[3] = { @@ -725,7 +726,7 @@ {2, arcs_31_2}, }; static arc arcs_32_0[1] = { - {83, 1}, + {84, 1}, }; static arc arcs_32_1[2] = { {30, 0}, @@ -739,7 +740,7 @@ {21, 1}, }; static arc arcs_33_1[2] = { - {78, 0}, + {79, 0}, {0, 1}, }; static state states_33[2] = { @@ -747,7 +748,7 @@ {2, arcs_33_1}, }; static arc arcs_34_0[1] = { - {84, 1}, + {85, 1}, }; static arc arcs_34_1[1] = { {21, 2}, @@ -762,7 +763,7 @@ {2, arcs_34_2}, }; static arc arcs_35_0[1] = { - {85, 1}, + {86, 1}, }; static arc arcs_35_1[1] = { {21, 2}, @@ -777,7 +778,7 @@ {2, arcs_35_2}, }; static arc arcs_36_0[1] = { - {86, 1}, + {87, 1}, }; static arc arcs_36_1[1] = { {24, 2}, @@ -800,11 +801,11 @@ {1, arcs_36_4}, }; static arc arcs_37_0[8] = { - {87, 1}, {88, 1}, {89, 1}, {90, 1}, {91, 1}, + {92, 1}, {19, 1}, {18, 1}, {17, 1}, @@ -817,7 +818,7 @@ {1, arcs_37_1}, }; static arc arcs_38_0[1] = { - {92, 1}, + {93, 1}, }; static arc arcs_38_1[1] = { {24, 2}, @@ -829,8 +830,8 @@ {26, 4}, }; static arc arcs_38_4[3] = { - {93, 1}, - {94, 5}, + {94, 1}, + {95, 5}, {0, 4}, }; static arc arcs_38_5[1] = { @@ -853,7 +854,7 @@ {1, arcs_38_7}, }; static arc arcs_39_0[1] = { - {95, 1}, + {96, 1}, }; static arc arcs_39_1[1] = { {24, 2}, @@ -865,7 +866,7 @@ {26, 4}, }; static arc arcs_39_4[2] = { - {94, 5}, + {95, 5}, {0, 4}, }; static arc arcs_39_5[1] = { @@ -888,13 +889,13 @@ {1, arcs_39_7}, }; static arc arcs_40_0[1] = { - {96, 1}, + {97, 1}, }; static arc arcs_40_1[1] = { - {62, 2}, + {63, 2}, }; static arc arcs_40_2[1] = { - {97, 3}, + {98, 3}, }; static arc arcs_40_3[1] = { {9, 4}, @@ -906,7 +907,7 @@ {26, 6}, }; static arc arcs_40_6[2] = { - {94, 7}, + {95, 7}, {0, 6}, }; static arc arcs_40_7[1] = { @@ -931,7 +932,7 @@ {1, arcs_40_9}, }; static arc arcs_41_0[1] = { - {98, 1}, + {99, 1}, }; static arc arcs_41_1[1] = { {25, 2}, @@ -940,8 +941,8 @@ {26, 3}, }; static arc arcs_41_3[2] = { - {99, 4}, - {100, 5}, + {100, 4}, + {101, 5}, }; static arc arcs_41_4[1] = { {25, 6}, @@ -956,9 +957,9 @@ {26, 9}, }; static arc arcs_41_8[4] = { - {99, 4}, - {94, 10}, - {100, 5}, + {100, 4}, + {95, 10}, + {101, 5}, {0, 8}, }; static arc arcs_41_9[1] = { @@ -971,7 +972,7 @@ {26, 12}, }; static arc arcs_41_12[2] = { - {100, 5}, + {101, 5}, {0, 12}, }; static state states_41[13] = { @@ -990,10 +991,10 @@ {2, arcs_41_12}, }; static arc arcs_42_0[1] = { - {101, 1}, + {102, 1}, }; static arc arcs_42_1[1] = { - {102, 2}, + {103, 2}, }; static arc arcs_42_2[2] = { {30, 1}, @@ -1016,11 +1017,11 @@ {24, 1}, }; static arc arcs_43_1[2] = { - {82, 2}, + {83, 2}, {0, 1}, }; static arc arcs_43_2[1] = { - {103, 3}, + {104, 3}, }; static arc arcs_43_3[1] = { {0, 3}, @@ -1032,14 +1033,14 @@ {1, arcs_43_3}, }; static arc arcs_44_0[1] = { - {104, 1}, + {105, 1}, }; static arc arcs_44_1[2] = { {24, 2}, {0, 1}, }; static arc arcs_44_2[2] = { - {82, 3}, + {83, 3}, {0, 2}, }; static arc arcs_44_3[1] = { @@ -1063,14 +1064,14 @@ {0, 1}, }; static arc arcs_45_2[1] = { - {105, 3}, + {106, 3}, }; static arc arcs_45_3[1] = { {6, 4}, }; static arc arcs_45_4[2] = { {6, 4}, - {106, 1}, + {107, 1}, }; static state states_45[5] = { {2, arcs_45_0}, @@ -1080,21 +1081,21 @@ {2, arcs_45_4}, }; static arc arcs_46_0[2] = { - {107, 1}, - {108, 2}, + {108, 1}, + {109, 2}, }; static arc arcs_46_1[2] = { - {92, 3}, + {93, 3}, {0, 1}, }; static arc arcs_46_2[1] = { {0, 2}, }; static arc arcs_46_3[1] = { - {107, 4}, + {108, 4}, }; static arc arcs_46_4[1] = { - {94, 5}, + {95, 5}, }; static arc arcs_46_5[1] = { {24, 2}, @@ -1108,8 +1109,8 @@ {1, arcs_46_5}, }; static arc arcs_47_0[2] = { - {107, 1}, - {110, 1}, + {108, 1}, + {111, 1}, }; static arc arcs_47_1[1] = { {0, 1}, @@ -1119,7 +1120,7 @@ {1, arcs_47_1}, }; static arc arcs_48_0[1] = { - {111, 1}, + {112, 1}, }; static arc arcs_48_1[2] = { {33, 2}, @@ -1142,7 +1143,7 @@ {1, arcs_48_4}, }; static arc arcs_49_0[1] = { - {111, 1}, + {112, 1}, }; static arc arcs_49_1[2] = { {33, 2}, @@ -1152,7 +1153,7 @@ {25, 3}, }; static arc arcs_49_3[1] = { - {109, 4}, + {110, 4}, }; static arc arcs_49_4[1] = { {0, 4}, @@ -1165,10 +1166,10 @@ {1, arcs_49_4}, }; static arc arcs_50_0[1] = { - {112, 1}, + {113, 1}, }; static arc arcs_50_1[2] = { - {113, 0}, + {114, 0}, {0, 1}, }; static state states_50[2] = { @@ -1176,10 +1177,10 @@ {2, arcs_50_1}, }; static arc arcs_51_0[1] = { - {114, 1}, + {115, 1}, }; static arc arcs_51_1[2] = { - {115, 0}, + {116, 0}, {0, 1}, }; static state states_51[2] = { @@ -1187,11 +1188,11 @@ {2, arcs_51_1}, }; static arc arcs_52_0[2] = { - {116, 1}, - {117, 2}, + {117, 1}, + {118, 2}, }; static arc arcs_52_1[1] = { - {114, 2}, + {115, 2}, }; static arc arcs_52_2[1] = { {0, 2}, @@ -1202,10 +1203,10 @@ {1, arcs_52_2}, }; static arc arcs_53_0[1] = { - {103, 1}, + {104, 1}, }; static arc arcs_53_1[2] = { - {118, 0}, + {119, 0}, {0, 1}, }; static state states_53[2] = { @@ -1213,25 +1214,25 @@ {2, arcs_53_1}, }; static arc arcs_54_0[10] = { - {119, 1}, {120, 1}, {121, 1}, {122, 1}, {123, 1}, {124, 1}, {125, 1}, - {97, 1}, - {116, 2}, - {126, 3}, + {126, 1}, + {98, 1}, + {117, 2}, + {127, 3}, }; static arc arcs_54_1[1] = { {0, 1}, }; static arc arcs_54_2[1] = { - {97, 1}, + {98, 1}, }; static arc arcs_54_3[2] = { - {116, 1}, + {117, 1}, {0, 3}, }; static state states_54[4] = { @@ -1244,7 +1245,7 @@ {31, 1}, }; static arc arcs_55_1[1] = { - {103, 2}, + {104, 2}, }; static arc arcs_55_2[1] = { {0, 2}, @@ -1255,10 +1256,10 @@ {1, arcs_55_2}, }; static arc arcs_56_0[1] = { - {127, 1}, + {128, 1}, }; static arc arcs_56_1[2] = { - {128, 0}, + {129, 0}, {0, 1}, }; static state states_56[2] = { @@ -1266,10 +1267,10 @@ {2, arcs_56_1}, }; static arc arcs_57_0[1] = { - {129, 1}, + {130, 1}, }; static arc arcs_57_1[2] = { - {130, 0}, + {131, 0}, {0, 1}, }; static state states_57[2] = { @@ -1277,10 +1278,10 @@ {2, arcs_57_1}, }; static arc arcs_58_0[1] = { - {131, 1}, + {132, 1}, }; static arc arcs_58_1[2] = { - {132, 0}, + {133, 0}, {0, 1}, }; static state states_58[2] = { @@ -1288,11 +1289,11 @@ {2, arcs_58_1}, }; static arc arcs_59_0[1] = { - {133, 1}, + {134, 1}, }; static arc arcs_59_1[3] = { - {134, 0}, {135, 0}, + {136, 0}, {0, 1}, }; static state states_59[2] = { @@ -1300,11 +1301,11 @@ {3, arcs_59_1}, }; static arc arcs_60_0[1] = { - {136, 1}, + {137, 1}, }; static arc arcs_60_1[3] = { - {137, 0}, {138, 0}, + {139, 0}, {0, 1}, }; static state states_60[2] = { @@ -1312,27 +1313,28 @@ {3, arcs_60_1}, }; static arc arcs_61_0[1] = { - {139, 1}, + {140, 1}, }; -static arc arcs_61_1[5] = { +static arc arcs_61_1[6] = { {31, 0}, - {140, 0}, + {11, 0}, {141, 0}, {142, 0}, + {143, 0}, {0, 1}, }; static state states_61[2] = { {1, arcs_61_0}, - {5, arcs_61_1}, + {6, arcs_61_1}, }; static arc arcs_62_0[4] = { - {137, 1}, {138, 1}, - {143, 1}, - {144, 2}, + {139, 1}, + {144, 1}, + {145, 2}, }; static arc arcs_62_1[1] = { - {139, 2}, + {140, 2}, }; static arc arcs_62_2[1] = { {0, 2}, @@ -1343,15 +1345,15 @@ {1, arcs_62_2}, }; static arc arcs_63_0[1] = { - {145, 1}, + {146, 1}, }; static arc arcs_63_1[3] = { - {146, 1}, + {147, 1}, {32, 2}, {0, 1}, }; static arc arcs_63_2[1] = { - {139, 3}, + {140, 3}, }; static arc arcs_63_3[1] = { {0, 3}, @@ -1364,44 +1366,44 @@ }; static arc arcs_64_0[10] = { {13, 1}, - {148, 2}, - {150, 3}, + {149, 2}, + {151, 3}, {21, 4}, - {153, 4}, - {154, 5}, - {79, 4}, - {155, 4}, + {154, 4}, + {155, 5}, + {80, 4}, {156, 4}, {157, 4}, + {158, 4}, }; static arc arcs_64_1[3] = { {47, 6}, - {147, 6}, + {148, 6}, {15, 4}, }; static arc arcs_64_2[2] = { - {147, 7}, - {149, 4}, + {148, 7}, + {150, 4}, }; static arc arcs_64_3[2] = { - {151, 8}, - {152, 4}, + {152, 8}, + {153, 4}, }; static arc arcs_64_4[1] = { {0, 4}, }; static arc arcs_64_5[2] = { - {154, 5}, + {155, 5}, {0, 5}, }; static arc arcs_64_6[1] = { {15, 4}, }; static arc arcs_64_7[1] = { - {149, 4}, + {150, 4}, }; static arc arcs_64_8[1] = { - {152, 4}, + {153, 4}, }; static state states_64[9] = { {10, arcs_64_0}, @@ -1419,7 +1421,7 @@ {48, 1}, }; static arc arcs_65_1[3] = { - {158, 2}, + {159, 2}, {30, 3}, {0, 1}, }; @@ -1444,15 +1446,15 @@ }; static arc arcs_66_0[3] = { {13, 1}, - {148, 2}, - {78, 3}, + {149, 2}, + {79, 3}, }; static arc arcs_66_1[2] = { {14, 4}, {15, 5}, }; static arc arcs_66_2[1] = { - {159, 6}, + {160, 6}, }; static arc arcs_66_3[1] = { {21, 5}, @@ -1464,7 +1466,7 @@ {0, 5}, }; static arc arcs_66_6[1] = { - {149, 5}, + {150, 5}, }; static state states_66[7] = { {3, arcs_66_0}, @@ -1476,14 +1478,14 @@ {1, arcs_66_6}, }; static arc arcs_67_0[1] = { - {160, 1}, + {161, 1}, }; static arc arcs_67_1[2] = { {30, 2}, {0, 1}, }; static arc arcs_67_2[2] = { - {160, 1}, + {161, 1}, {0, 2}, }; static state states_67[3] = { @@ -1501,11 +1503,11 @@ }; static arc arcs_68_2[3] = { {24, 3}, - {161, 4}, + {162, 4}, {0, 2}, }; static arc arcs_68_3[2] = { - {161, 4}, + {162, 4}, {0, 3}, }; static arc arcs_68_4[1] = { @@ -1534,7 +1536,7 @@ {1, arcs_69_2}, }; static arc arcs_70_0[2] = { - {103, 1}, + {104, 1}, {48, 1}, }; static arc arcs_70_1[2] = { @@ -1542,7 +1544,7 @@ {0, 1}, }; static arc arcs_70_2[3] = { - {103, 1}, + {104, 1}, {48, 1}, {0, 2}, }; @@ -1572,7 +1574,7 @@ }; static arc arcs_72_1[4] = { {25, 2}, - {158, 3}, + {159, 3}, {30, 4}, {0, 1}, }; @@ -1587,7 +1589,7 @@ {0, 4}, }; static arc arcs_72_5[3] = { - {158, 3}, + {159, 3}, {30, 7}, {0, 5}, }; @@ -1623,7 +1625,7 @@ {2, arcs_72_10}, }; static arc arcs_73_0[1] = { - {162, 1}, + {163, 1}, }; static arc arcs_73_1[1] = { {21, 2}, @@ -1659,7 +1661,7 @@ {1, arcs_73_7}, }; static arc arcs_74_0[3] = { - {163, 1}, + {164, 1}, {31, 2}, {32, 3}, }; @@ -1674,7 +1676,7 @@ {24, 6}, }; static arc arcs_74_4[4] = { - {163, 1}, + {164, 1}, {31, 2}, {32, 3}, {0, 4}, @@ -1687,7 +1689,7 @@ {0, 6}, }; static arc arcs_74_7[2] = { - {163, 5}, + {164, 5}, {32, 3}, }; static state states_74[8] = { @@ -1704,7 +1706,7 @@ {24, 1}, }; static arc arcs_75_1[3] = { - {158, 2}, + {159, 2}, {29, 3}, {0, 1}, }; @@ -1721,8 +1723,8 @@ {1, arcs_75_3}, }; static arc arcs_76_0[2] = { - {158, 1}, - {165, 1}, + {159, 1}, + {166, 1}, }; static arc arcs_76_1[1] = { {0, 1}, @@ -1732,19 +1734,19 @@ {1, arcs_76_1}, }; static arc arcs_77_0[1] = { - {96, 1}, + {97, 1}, }; static arc arcs_77_1[1] = { - {62, 2}, + {63, 2}, }; static arc arcs_77_2[1] = { - {97, 3}, + {98, 3}, }; static arc arcs_77_3[1] = { - {107, 4}, + {108, 4}, }; static arc arcs_77_4[2] = { - {164, 5}, + {165, 5}, {0, 4}, }; static arc arcs_77_5[1] = { @@ -1759,13 +1761,13 @@ {1, arcs_77_5}, }; static arc arcs_78_0[1] = { - {92, 1}, + {93, 1}, }; static arc arcs_78_1[1] = { - {109, 2}, + {110, 2}, }; static arc arcs_78_2[2] = { - {164, 3}, + {165, 3}, {0, 2}, }; static arc arcs_78_3[1] = { @@ -1788,10 +1790,10 @@ {1, arcs_79_1}, }; static arc arcs_80_0[1] = { - {167, 1}, + {168, 1}, }; static arc arcs_80_1[2] = { - {168, 2}, + {169, 2}, {0, 1}, }; static arc arcs_80_2[1] = { @@ -1803,7 +1805,7 @@ {1, arcs_80_2}, }; static arc arcs_81_0[2] = { - {73, 1}, + {74, 1}, {9, 2}, }; static arc arcs_81_1[1] = { @@ -1819,11 +1821,11 @@ }; static dfa dfas[82] = { {256, "single_input", 0, 3, states_0, - "\004\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"}, + "\004\050\060\200\000\000\000\100\301\047\341\040\113\000\041\000\000\014\241\174\010\001"}, {257, "file_input", 0, 2, states_1, - "\204\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"}, + "\204\050\060\200\000\000\000\100\301\047\341\040\113\000\041\000\000\014\241\174\010\001"}, {258, "eval_input", 0, 3, states_2, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {259, "decorator", 0, 7, states_3, "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {260, "decorators", 0, 2, states_4, @@ -1843,39 +1845,39 @@ {267, "vfpdef", 0, 2, states_11, "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {268, "stmt", 0, 2, states_12, - "\000\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"}, + "\000\050\060\200\000\000\000\100\301\047\341\040\113\000\041\000\000\014\241\174\010\001"}, {269, "simple_stmt", 0, 4, states_13, - "\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"}, + "\000\040\040\200\000\000\000\100\301\047\341\000\000\000\041\000\000\014\241\174\000\001"}, {270, "small_stmt", 0, 2, states_14, - "\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"}, + "\000\040\040\200\000\000\000\100\301\047\341\000\000\000\041\000\000\014\241\174\000\001"}, {271, "expr_stmt", 0, 6, states_15, - "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\200\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {272, "testlist_star_expr", 0, 3, states_16, - "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\200\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {273, "augassign", 0, 2, states_17, - "\000\000\000\000\000\000\376\037\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\376\077\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {274, "del_stmt", 0, 3, states_18, - "\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {275, "pass_stmt", 0, 2, states_19, - "\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {276, "flow_stmt", 0, 2, states_20, - "\000\000\000\000\000\000\000\000\340\001\000\000\000\000\000\000\000\000\000\000\200\000"}, + "\000\000\000\000\000\000\000\000\300\003\000\000\000\000\000\000\000\000\000\000\000\001"}, {277, "break_stmt", 0, 2, states_21, - "\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {278, "continue_stmt", 0, 2, states_22, - "\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {279, "return_stmt", 0, 3, states_23, - "\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000"}, {280, "yield_stmt", 0, 2, states_24, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001"}, {281, "raise_stmt", 0, 5, states_25, - "\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000"}, {282, "import_stmt", 0, 2, states_26, - "\000\000\000\000\000\000\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\044\000\000\000\000\000\000\000\000\000\000\000\000"}, {283, "import_name", 0, 3, states_27, - "\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000"}, {284, "import_from", 0, 8, states_28, - "\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000\000"}, {285, "import_as_name", 0, 4, states_29, "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {286, "dotted_as_name", 0, 4, states_30, @@ -1887,103 +1889,103 @@ {289, "dotted_name", 0, 2, states_33, "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {290, "global_stmt", 0, 3, states_34, - "\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000"}, {291, "nonlocal_stmt", 0, 3, states_35, - "\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000"}, {292, "assert_stmt", 0, 5, states_36, - "\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000"}, {293, "compound_stmt", 0, 2, states_37, - "\000\010\020\000\000\000\000\000\000\000\000\220\045\000\000\000\000\000\000\000\004\000"}, + "\000\010\020\000\000\000\000\000\000\000\000\040\113\000\000\000\000\000\000\000\010\000"}, {294, "if_stmt", 0, 8, states_38, - "\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000"}, {295, "while_stmt", 0, 8, states_39, - "\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"}, {296, "for_stmt", 0, 10, states_40, - "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000"}, {297, "try_stmt", 0, 13, states_41, - "\000\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\000\000"}, {298, "with_stmt", 0, 5, states_42, - "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000"}, {299, "with_item", 0, 4, states_43, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {300, "except_clause", 0, 5, states_44, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000"}, {301, "suite", 0, 5, states_45, - "\004\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"}, + "\004\040\040\200\000\000\000\100\301\047\341\000\000\000\041\000\000\014\241\174\000\001"}, {302, "test", 0, 6, states_46, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {303, "test_nocond", 0, 2, states_47, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {304, "lambdef", 0, 5, states_48, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000"}, {305, "lambdef_nocond", 0, 5, states_49, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000"}, {306, "or_test", 0, 2, states_50, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\040\000\000\014\241\174\000\000"}, {307, "and_test", 0, 2, states_51, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\040\000\000\014\241\174\000\000"}, {308, "not_test", 0, 3, states_52, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\040\000\000\014\241\174\000\000"}, {309, "comparison", 0, 2, states_53, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {310, "comp_op", 0, 4, states_54, - "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\220\177\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\004\000\040\377\000\000\000\000\000\000"}, {311, "star_expr", 0, 3, states_55, "\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {312, "expr", 0, 2, states_56, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {313, "xor_expr", 0, 2, states_57, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {314, "and_expr", 0, 2, states_58, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {315, "shift_expr", 0, 2, states_59, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {316, "arith_expr", 0, 2, states_60, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {317, "term", 0, 2, states_61, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {318, "factor", 0, 3, states_62, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {319, "power", 0, 4, states_63, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\240\174\000\000"}, {320, "atom", 0, 9, states_64, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\240\174\000\000"}, {321, "testlist_comp", 0, 5, states_65, - "\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\200\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {322, "trailer", 0, 7, states_66, - "\000\040\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\020\000\000\000"}, + "\000\040\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\040\000\000\000"}, {323, "subscriptlist", 0, 3, states_67, - "\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\002\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {324, "subscript", 0, 5, states_68, - "\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\002\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {325, "sliceop", 0, 3, states_69, "\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {326, "exprlist", 0, 3, states_70, - "\000\040\040\200\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"}, + "\000\040\040\200\000\000\000\000\000\000\001\000\000\000\000\000\000\014\241\174\000\000"}, {327, "testlist", 0, 3, states_71, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {328, "dictorsetmaker", 0, 11, states_72, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {329, "classdef", 0, 8, states_73, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\004\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\010\000"}, {330, "arglist", 0, 8, states_74, - "\000\040\040\200\001\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\200\001\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {331, "argument", 0, 4, states_75, - "\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\000\001\000\000\000\041\000\000\014\241\174\000\000"}, {332, "comp_iter", 0, 2, states_76, - "\000\000\000\000\000\000\000\000\000\000\000\020\001\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\040\002\000\000\000\000\000\000\000\000\000"}, {333, "comp_for", 0, 6, states_77, - "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000"}, {334, "comp_if", 0, 4, states_78, - "\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000"}, {335, "encoding_decl", 0, 2, states_79, "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {336, "yield_expr", 0, 3, states_80, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"}, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001"}, {337, "yield_arg", 0, 3, states_81, - "\000\040\040\000\000\000\000\000\000\202\000\000\000\200\020\000\000\206\120\076\000\000"}, + "\000\040\040\000\000\000\000\000\000\004\001\000\000\000\041\000\000\014\241\174\000\000"}, }; -static label labels[169] = { +static label labels[170] = { {0, "EMPTY"}, {256, 0}, {4, 0}, @@ -2007,7 +2009,7 @@ {1, "def"}, {1, 0}, {263, 0}, - {50, 0}, + {51, 0}, {302, 0}, {11, 0}, {301, 0}, @@ -2036,6 +2038,7 @@ {36, 0}, {37, 0}, {38, 0}, + {50, 0}, {39, 0}, {40, 0}, {41, 0}, @@ -2063,7 +2066,7 @@ {1, "import"}, {288, 0}, {23, 0}, - {51, 0}, + {52, 0}, {287, 0}, {285, 0}, {1, "as"}, @@ -2157,6 +2160,6 @@ grammar _PyParser_Grammar = { 82, dfas, - {169, labels}, + {170, labels}, 256 }; diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -15,8 +15,8 @@ &&_unknown_opcode, &&_unknown_opcode, &&TARGET_UNARY_INVERT, - &&_unknown_opcode, - &&_unknown_opcode, + &&TARGET_BINARY_MATRIX_MULTIPLY, + &&TARGET_INPLACE_MATRIX_MULTIPLY, &&_unknown_opcode, &&TARGET_BINARY_POWER, &&TARGET_BINARY_MULTIPLY, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 05:57:26 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 05:57:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_pep_465_implemented?= Message-ID: <3g47pt6tdZz7LjZ@mail.python.org> http://hg.python.org/peps/rev/56b554e51e5a changeset: 5456:56b554e51e5a user: Benjamin Peterson date: Wed Apr 09 23:57:23 2014 -0400 summary: pep 465 implemented files: pep-0465.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0465.txt b/pep-0465.txt --- a/pep-0465.txt +++ b/pep-0465.txt @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Nathaniel J. Smith -Status: Accepted +Status: Final Type: Standards Track Content-Type: text/x-rst Created: 20-Feb-2014 -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Thu Apr 10 06:15:42 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:15:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_add_matrix_mul?= =?utf-8?q?tiplication_operator_support_to_2to3?= Message-ID: <3g48Cy30GFz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/63dee6d79fd5 changeset: 90207:63dee6d79fd5 branch: 3.4 parent: 90203:a3299de5fc93 user: Benjamin Peterson date: Thu Apr 10 00:12:47 2014 -0400 summary: add matrix multiplication operator support to 2to3 files: Lib/lib2to3/Grammar.txt | 4 ++-- Lib/lib2to3/pgen2/grammar.py | 1 + Lib/lib2to3/pgen2/token.py | 13 +++++++------ Lib/lib2to3/pgen2/tokenize.py | 2 +- Lib/lib2to3/tests/test_parser.py | 6 ++++++ Misc/NEWS | 2 ++ 6 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -56,7 +56,7 @@ expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] -augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | +augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') # For normal assignments, additional restrictions enforced by the interpreter print_stmt: 'print' ( [ test (',' test)* [','] ] | @@ -119,7 +119,7 @@ and_expr: shift_expr ('&' shift_expr)* shift_expr: arith_expr (('<<'|'>>') arith_expr)* arith_expr: term (('+'|'-') term)* -term: factor (('*'|'/'|'%'|'//') factor)* +term: factor (('*'|'@'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ['**' factor] atom: ('(' [yield_expr|testlist_gexp] ')' | diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -149,6 +149,7 @@ { LBRACE } RBRACE @ AT +@= ATEQUAL == EQEQUAL != NOTEQUAL <> NOTEQUAL diff --git a/Lib/lib2to3/pgen2/token.py b/Lib/lib2to3/pgen2/token.py --- a/Lib/lib2to3/pgen2/token.py +++ b/Lib/lib2to3/pgen2/token.py @@ -57,12 +57,13 @@ DOUBLESLASH = 48 DOUBLESLASHEQUAL = 49 AT = 50 -OP = 51 -COMMENT = 52 -NL = 53 -RARROW = 54 -ERRORTOKEN = 55 -N_TOKENS = 56 +ATEQUAL = 51 +OP = 52 +COMMENT = 53 +NL = 54 +RARROW = 55 +ERRORTOKEN = 56 +N_TOKENS = 57 NT_OFFSET = 256 #--end constants-- diff --git a/Lib/lib2to3/pgen2/tokenize.py b/Lib/lib2to3/pgen2/tokenize.py --- a/Lib/lib2to3/pgen2/tokenize.py +++ b/Lib/lib2to3/pgen2/tokenize.py @@ -84,7 +84,7 @@ # recognized as two instances of =). Operator = group(r"\*\*=?", r">>=?", r"<<=?", r"<>", r"!=", r"//=?", r"->", - r"[+\-*/%&|^=<>]=?", + r"[+\-*/%&@|^=<>]=?", r"~") Bracket = '[][(){}]' diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -48,6 +48,12 @@ raise AssertionError("Syntax shouldn't have been valid") +class TestMatrixMultiplication(GrammarTest): + def test_matrix_multiplication_operator(self): + self.validate("a @ b") + self.validate("a @= b") + + class TestRaiseChanges(GrammarTest): def test_2x_style_1(self): self.validate("raise") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -168,6 +168,8 @@ Tools/Demos ----------- +- Add support for the PEP 465 matrix multiplication operator to 2to3. + - Issue #16047: Fix module exception list and __file__ handling in freeze. Patch by Meador Inge. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 06:15:43 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:15:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_add_matrix_mul?= =?utf-8?q?tiplication_operator_support_to_2to3?= Message-ID: <3g48Cz4YdSz7Ljt@mail.python.org> http://hg.python.org/cpython/rev/a12f306ee0eb changeset: 90208:a12f306ee0eb branch: 2.7 parent: 90200:63a55ed6622b user: Benjamin Peterson date: Thu Apr 10 00:12:47 2014 -0400 summary: add matrix multiplication operator support to 2to3 files: Lib/lib2to3/Grammar.txt | 4 ++-- Lib/lib2to3/pgen2/grammar.py | 1 + Lib/lib2to3/pgen2/token.py | 13 +++++++------ Lib/lib2to3/pgen2/tokenize.py | 2 +- Lib/lib2to3/tests/test_parser.py | 6 ++++++ Misc/NEWS | 2 ++ 6 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -56,7 +56,7 @@ expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] -augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | +augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') # For normal assignments, additional restrictions enforced by the interpreter print_stmt: 'print' ( [ test (',' test)* [','] ] | @@ -119,7 +119,7 @@ and_expr: shift_expr ('&' shift_expr)* shift_expr: arith_expr (('<<'|'>>') arith_expr)* arith_expr: term (('+'|'-') term)* -term: factor (('*'|'/'|'%'|'//') factor)* +term: factor (('*'|'@'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ['**' factor] atom: ('(' [yield_expr|testlist_gexp] ')' | diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -151,6 +151,7 @@ { LBRACE } RBRACE @ AT +@= ATEQUAL == EQEQUAL != NOTEQUAL <> NOTEQUAL diff --git a/Lib/lib2to3/pgen2/token.py b/Lib/lib2to3/pgen2/token.py --- a/Lib/lib2to3/pgen2/token.py +++ b/Lib/lib2to3/pgen2/token.py @@ -57,12 +57,13 @@ DOUBLESLASH = 48 DOUBLESLASHEQUAL = 49 AT = 50 -OP = 51 -COMMENT = 52 -NL = 53 -RARROW = 54 -ERRORTOKEN = 55 -N_TOKENS = 56 +ATEQUAL = 51 +OP = 52 +COMMENT = 53 +NL = 54 +RARROW = 55 +ERRORTOKEN = 56 +N_TOKENS = 57 NT_OFFSET = 256 #--end constants-- diff --git a/Lib/lib2to3/pgen2/tokenize.py b/Lib/lib2to3/pgen2/tokenize.py --- a/Lib/lib2to3/pgen2/tokenize.py +++ b/Lib/lib2to3/pgen2/tokenize.py @@ -84,7 +84,7 @@ # recognized as two instances of =). Operator = group(r"\*\*=?", r">>=?", r"<<=?", r"<>", r"!=", r"//=?", r"->", - r"[+\-*/%&|^=<>]=?", + r"[+\-*/%&@|^=<>]=?", r"~") Bracket = '[][(){}]' diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -44,6 +44,12 @@ raise AssertionError("Syntax shouldn't have been valid") +class TestMatrixMultiplication(GrammarTest): + def test_matrix_multiplication_operator(self): + self.validate("a @ b") + self.validate("a @= b") + + class TestRaiseChanges(GrammarTest): def test_2x_style_1(self): self.validate("raise") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -266,6 +266,8 @@ Tools/Demos ----------- +- Add support for the PEP 465 matrix multiplication operator to 2to3. + - Issue #19936: Added executable bits or shebang lines to Python scripts which requires them. Disable executable bits and shebang lines in test and benchmark files in order to prevent using a random system python, and in -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 06:15:44 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:15:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g48D06NNsz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/609a99ffa627 changeset: 90209:609a99ffa627 parent: 90206:c553d8f72d65 parent: 90207:63dee6d79fd5 user: Benjamin Peterson date: Thu Apr 10 00:15:34 2014 -0400 summary: merge 3.4 files: Lib/lib2to3/Grammar.txt | 4 ++-- Lib/lib2to3/pgen2/grammar.py | 1 + Lib/lib2to3/pgen2/token.py | 13 +++++++------ Lib/lib2to3/pgen2/tokenize.py | 2 +- Lib/lib2to3/tests/test_parser.py | 6 ++++++ Misc/NEWS | 2 ++ 6 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -56,7 +56,7 @@ expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] -augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | +augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') # For normal assignments, additional restrictions enforced by the interpreter print_stmt: 'print' ( [ test (',' test)* [','] ] | @@ -119,7 +119,7 @@ and_expr: shift_expr ('&' shift_expr)* shift_expr: arith_expr (('<<'|'>>') arith_expr)* arith_expr: term (('+'|'-') term)* -term: factor (('*'|'/'|'%'|'//') factor)* +term: factor (('*'|'@'|'/'|'%'|'//') factor)* factor: ('+'|'-'|'~') factor | power power: atom trailer* ['**' factor] atom: ('(' [yield_expr|testlist_gexp] ')' | diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -149,6 +149,7 @@ { LBRACE } RBRACE @ AT +@= ATEQUAL == EQEQUAL != NOTEQUAL <> NOTEQUAL diff --git a/Lib/lib2to3/pgen2/token.py b/Lib/lib2to3/pgen2/token.py --- a/Lib/lib2to3/pgen2/token.py +++ b/Lib/lib2to3/pgen2/token.py @@ -57,12 +57,13 @@ DOUBLESLASH = 48 DOUBLESLASHEQUAL = 49 AT = 50 -OP = 51 -COMMENT = 52 -NL = 53 -RARROW = 54 -ERRORTOKEN = 55 -N_TOKENS = 56 +ATEQUAL = 51 +OP = 52 +COMMENT = 53 +NL = 54 +RARROW = 55 +ERRORTOKEN = 56 +N_TOKENS = 57 NT_OFFSET = 256 #--end constants-- diff --git a/Lib/lib2to3/pgen2/tokenize.py b/Lib/lib2to3/pgen2/tokenize.py --- a/Lib/lib2to3/pgen2/tokenize.py +++ b/Lib/lib2to3/pgen2/tokenize.py @@ -84,7 +84,7 @@ # recognized as two instances of =). Operator = group(r"\*\*=?", r">>=?", r"<<=?", r"<>", r"!=", r"//=?", r"->", - r"[+\-*/%&|^=<>]=?", + r"[+\-*/%&@|^=<>]=?", r"~") Bracket = '[][(){}]' diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -48,6 +48,12 @@ raise AssertionError("Syntax shouldn't have been valid") +class TestMatrixMultiplication(GrammarTest): + def test_matrix_multiplication_operator(self): + self.validate("a @ b") + self.validate("a @= b") + + class TestRaiseChanges(GrammarTest): def test_2x_style_1(self): self.validate("raise") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -215,6 +215,8 @@ Tools/Demos ----------- +- Add support for the PEP 465 matrix multiplication operator to 2to3. + - Issue #16047: Fix module exception list and __file__ handling in freeze. Patch by Meador Inge. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 06:17:54 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:17:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_teach_unparse_about_matrix?= =?utf-8?q?_multiplication?= Message-ID: <3g48GV26cCz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/6a4ef8928e1b changeset: 90210:6a4ef8928e1b user: Benjamin Peterson date: Thu Apr 10 00:17:48 2014 -0400 summary: teach unparse about matrix multiplication files: Tools/parser/unparse.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -401,7 +401,7 @@ self.dispatch(t.operand) self.write(")") - binop = { "Add":"+", "Sub":"-", "Mult":"*", "Div":"/", "Mod":"%", + binop = { "Add":"+", "Sub":"-", "Mult":"*", "MatMult":"@", "Div":"/", "Mod":"%", "LShift":"<<", "RShift":">>", "BitOr":"|", "BitXor":"^", "BitAnd":"&", "FloorDiv":"//", "Pow": "**"} def _BinOp(self, t): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 06:25:09 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:25:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_teach_2to3_abo?= =?utf-8?q?ut_=27yield_from=27?= Message-ID: <3g48Qs5Rgkz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/094615256a54 changeset: 90211:094615256a54 branch: 3.4 parent: 90207:63dee6d79fd5 user: Benjamin Peterson date: Thu Apr 10 00:23:18 2014 -0400 summary: teach 2to3 about 'yield from' files: Lib/lib2to3/Grammar.txt | 3 ++- Lib/lib2to3/tests/test_parser.py | 7 +++++++ Misc/NEWS | 2 ++ 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -155,4 +155,5 @@ # not used in grammar, but may appear in "node" passed from Parser to Compiler encoding_decl: NAME -yield_expr: 'yield' [testlist] +yield_expr: 'yield' [yield_arg] +yield_arg: 'from' test | testlist diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -54,6 +54,13 @@ self.validate("a @= b") +class TestYieldFrom(GrammarTest): + def test_matrix_multiplication_operator(self): + self.validate("yield from x") + self.validate("(yield from x) + y") + self.invalid_syntax("yield from") + + class TestRaiseChanges(GrammarTest): def test_2x_style_1(self): self.validate("raise") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -168,6 +168,8 @@ Tools/Demos ----------- +- Add support for ``yield from`` to 2to3. + - Add support for the PEP 465 matrix multiplication operator to 2to3. - Issue #16047: Fix module exception list and __file__ handling in freeze. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 06:25:11 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:25:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_teach_2to3_abo?= =?utf-8?q?ut_=27yield_from=27?= Message-ID: <3g48Qv09RPz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/340a9d2f6bde changeset: 90212:340a9d2f6bde branch: 2.7 parent: 90208:a12f306ee0eb user: Benjamin Peterson date: Thu Apr 10 00:23:18 2014 -0400 summary: teach 2to3 about 'yield from' files: Lib/lib2to3/Grammar.txt | 3 ++- Lib/lib2to3/tests/test_parser.py | 7 +++++++ Misc/NEWS | 2 ++ 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -155,4 +155,5 @@ # not used in grammar, but may appear in "node" passed from Parser to Compiler encoding_decl: NAME -yield_expr: 'yield' [testlist] +yield_expr: 'yield' [yield_arg] +yield_arg: 'from' test | testlist diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -50,6 +50,13 @@ self.validate("a @= b") +class TestYieldFrom(GrammarTest): + def test_matrix_multiplication_operator(self): + self.validate("yield from x") + self.validate("(yield from x) + y") + self.invalid_syntax("yield from") + + class TestRaiseChanges(GrammarTest): def test_2x_style_1(self): self.validate("raise") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -266,6 +266,8 @@ Tools/Demos ----------- +- Add support for ``yield from`` to 2to3. + - Add support for the PEP 465 matrix multiplication operator to 2to3. - Issue #19936: Added executable bits or shebang lines to Python scripts which -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 06:25:12 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 10 Apr 2014 06:25:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g48Qw20zZz7LkK@mail.python.org> http://hg.python.org/cpython/rev/16f090d912d1 changeset: 90213:16f090d912d1 parent: 90210:6a4ef8928e1b parent: 90211:094615256a54 user: Benjamin Peterson date: Thu Apr 10 00:24:47 2014 -0400 summary: merge 3.4 files: Lib/lib2to3/Grammar.txt | 3 ++- Lib/lib2to3/tests/test_parser.py | 7 +++++++ Misc/NEWS | 2 ++ 3 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -155,4 +155,5 @@ # not used in grammar, but may appear in "node" passed from Parser to Compiler encoding_decl: NAME -yield_expr: 'yield' [testlist] +yield_expr: 'yield' [yield_arg] +yield_arg: 'from' test | testlist diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -54,6 +54,13 @@ self.validate("a @= b") +class TestYieldFrom(GrammarTest): + def test_matrix_multiplication_operator(self): + self.validate("yield from x") + self.validate("(yield from x) + y") + self.invalid_syntax("yield from") + + class TestRaiseChanges(GrammarTest): def test_2x_style_1(self): self.validate("raise") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -215,6 +215,8 @@ Tools/Demos ----------- +- Add support for ``yield from`` to 2to3. + - Add support for the PEP 465 matrix multiplication operator to 2to3. - Issue #16047: Fix module exception list and __file__ handling in freeze. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 08:14:15 2014 From: python-checkins at python.org (vinay.sajip) Date: Thu, 10 Apr 2014 08:14:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTcy?= =?utf-8?q?=3A_isinstance_check_relaxed_from_dict_to_collections=2EMapping?= =?utf-8?q?=2E?= Message-ID: <3g4Brl70X0z7LjS@mail.python.org> http://hg.python.org/cpython/rev/d08e3586dde3 changeset: 90214:d08e3586dde3 branch: 2.7 parent: 90212:340a9d2f6bde user: Vinay Sajip date: Thu Apr 10 07:07:59 2014 +0100 summary: Issue #21172: isinstance check relaxed from dict to collections.Mapping. files: Lib/logging/__init__.py | 10 ++++++++-- Misc/NEWS | 2 ++ 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -23,7 +23,7 @@ To use, simply 'import logging' and log away! """ -import sys, os, time, cStringIO, traceback, warnings, weakref +import sys, os, time, cStringIO, traceback, warnings, weakref, collections __all__ = ['BASIC_FORMAT', 'BufferingFormatter', 'CRITICAL', 'DEBUG', 'ERROR', 'FATAL', 'FileHandler', 'Filter', 'Formatter', 'Handler', 'INFO', @@ -261,7 +261,13 @@ # 'Value is %d' instead of 'Value is 0'. # For the use case of passing a dictionary, this should not be a # problem. - if args and len(args) == 1 and isinstance(args[0], dict) and args[0]: + # Issue #21172: a request was made to relax the isinstance check + # to hasattr(args[0], '__getitem__'). However, the docs on string + # formatting still seem to suggest a mapping object is required. + # Thus, while not removing the isinstance check, it does now look + # for collections.Mapping rather than, as before, dict. + if (args and len(args) == 1 and isinstance(args[0], collections.Mapping) + and args[0]): args = args[0] self.args = args self.levelname = getLevelName(level) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,8 @@ Library ------- +- Issue #21172: isinstance check relaxed from dict to collections.Mapping. + - Issue #21191: In os.fdopen, alwyas close the file descriptor when an exception happens. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 08:14:17 2014 From: python-checkins at python.org (vinay.sajip) Date: Thu, 10 Apr 2014 08:14:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTcy?= =?utf-8?q?=3A_isinstance_check_relaxed_from_dict_to_collections=2EMapping?= =?utf-8?q?=2E?= Message-ID: <3g4Brn1jZPz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/5e303360db14 changeset: 90215:5e303360db14 branch: 3.4 parent: 90211:094615256a54 user: Vinay Sajip date: Thu Apr 10 07:12:19 2014 +0100 summary: Issue #21172: isinstance check relaxed from dict to collections.Mapping. files: Lib/logging/__init__.py | 11 +++++++++-- Misc/NEWS | 2 ++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -23,7 +23,8 @@ To use, simply 'import logging' and log away! """ -import sys, os, time, io, traceback, warnings, weakref +import sys, os, time, io, traceback, warnings, weakref, collections + from string import Template __all__ = ['BASIC_FORMAT', 'BufferingFormatter', 'CRITICAL', 'DEBUG', 'ERROR', @@ -253,7 +254,13 @@ # 'Value is %d' instead of 'Value is 0'. # For the use case of passing a dictionary, this should not be a # problem. - if args and len(args) == 1 and isinstance(args[0], dict) and args[0]: + # Issue #21172: a request was made to relax the isinstance check + # to hasattr(args[0], '__getitem__'). However, the docs on string + # formatting still seem to suggest a mapping object is required. + # Thus, while not removing the isinstance check, it does now look + # for collections.Mapping rather than, as before, dict. + if (args and len(args) == 1 and isinstance(args[0], collections.Mapping) + and args[0]): args = args[0] self.args = args self.levelname = getLevelName(level) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,8 @@ Library ------- +- Issue #21172: isinstance check relaxed from dict to collections.Mapping. + - Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError if path and sock are specified at the same time. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 08:14:18 2014 From: python-checkins at python.org (vinay.sajip) Date: Thu, 10 Apr 2014 08:14:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2321172=3A_Merged_fix_from_3=2E4=2E?= Message-ID: <3g4Brp3BX6z7Ljs@mail.python.org> http://hg.python.org/cpython/rev/8e6b8cfeb172 changeset: 90216:8e6b8cfeb172 parent: 90213:16f090d912d1 parent: 90215:5e303360db14 user: Vinay Sajip date: Thu Apr 10 07:14:01 2014 +0100 summary: Closes #21172: Merged fix from 3.4. files: Lib/logging/__init__.py | 11 +++++++++-- Misc/NEWS | 2 ++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -23,7 +23,8 @@ To use, simply 'import logging' and log away! """ -import sys, os, time, io, traceback, warnings, weakref +import sys, os, time, io, traceback, warnings, weakref, collections + from string import Template __all__ = ['BASIC_FORMAT', 'BufferingFormatter', 'CRITICAL', 'DEBUG', 'ERROR', @@ -253,7 +254,13 @@ # 'Value is %d' instead of 'Value is 0'. # For the use case of passing a dictionary, this should not be a # problem. - if args and len(args) == 1 and isinstance(args[0], dict) and args[0]: + # Issue #21172: a request was made to relax the isinstance check + # to hasattr(args[0], '__getitem__'). However, the docs on string + # formatting still seem to suggest a mapping object is required. + # Thus, while not removing the isinstance check, it does now look + # for collections.Mapping rather than, as before, dict. + if (args and len(args) == 1 and isinstance(args[0], collections.Mapping) + and args[0]): args = args[0] self.args = args self.levelname = getLevelName(level) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -31,6 +31,8 @@ Library ------- +- Issue #21172: isinstance check relaxed from dict to collections.Mapping. + - Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError if path and sock are specified at the same time. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 10 09:18:32 2014 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 10 Apr 2014 09:18:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_comment_for_the_com?= =?utf-8?q?parison_table_to_use_measured_results_rather_than?= Message-ID: <3g4DGw3z9yz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/5caae834175c changeset: 90217:5caae834175c user: Raymond Hettinger date: Thu Apr 10 01:18:01 2014 -0600 summary: Update comment for the comparison table to use measured results rather than predicted. files: Lib/heapq.py | 19 ++++++++++--------- 1 files changed, 10 insertions(+), 9 deletions(-) diff --git a/Lib/heapq.py b/Lib/heapq.py --- a/Lib/heapq.py +++ b/Lib/heapq.py @@ -202,16 +202,17 @@ # Number of comparisons for n random inputs, keeping the k smallest values: # ----------------------------------------------------------- # Step Comparisons Action -# 1 2*k heapify the first k-inputs -# 2 n-k compare new input elements to top of heap -# 3 k*lg2(k)*(ln(n)-lg(k)) add new extreme values to the heap +# 1 1.66*k heapify the first k-inputs +# 2 n - k compare new input elements to top of heap +# 3 k*lg2(k)*(ln(n)-ln(k)) add new extreme values to the heap # 4 k*lg2(k) final sort of the k most extreme values # -# n-random inputs k-extreme values number of comparisons % more than min() -# --------------- ---------------- ------------------- ----------------- -# 10,000 100 13,634 36.3% -# 100,000 100 105,163 5.2% -# 1,000,000 100 1,006,694 0.7% +# number of comparisons +# n-random inputs k-extreme values average of 5 trials % more than min() +# --------------- ---------------- ------------------- ----------------- +# 10,000 100 14,046 40.5% +# 100,000 100 105,749 5.7% +# 1,000,000 100 1,007,751 0.8% # # Computing the number of comparisons for step 3: # ----------------------------------------------- @@ -234,7 +235,7 @@ # comparisons = k * log(k, 2) * (log(n,e) - log(k, e)) # # Worst-case for step 3: -# --------------------- +# ---------------------- # In the worst case, the input data is reversed sorted so that every new element # must be inserted in the heap: # comparisons = log(k, 2) * (n - k) -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Thu Apr 10 09:55:34 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 10 Apr 2014 09:55:34 +0200 Subject: [Python-checkins] Daily reference leaks (b402e5e06f85): sum=3 Message-ID: results for b402e5e06f85 on branch "default" -------------------------------------------- test_asyncio leaked [4, 0, 0] memory blocks, sum=4 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogHygoa9', '-x'] From python-checkins at python.org Thu Apr 10 14:55:24 2014 From: python-checkins at python.org (brett.cannon) Date: Thu, 10 Apr 2014 14:55:24 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Add_Steven_d=27Aprano_to_?= =?utf-8?q?the_log?= Message-ID: <3g4Mlc1yZjz7LjZ@mail.python.org> http://hg.python.org/devguide/rev/3ee9ee88bc7e changeset: 685:3ee9ee88bc7e user: Brett Cannon date: Thu Apr 10 08:55:19 2014 -0400 summary: Add Steven d'Aprano to the log files: developers.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/developers.rst b/developers.rst --- a/developers.rst +++ b/developers.rst @@ -25,6 +25,9 @@ Permissions History ------------------- +- Steven d'Aprano was given push privileges on Feb 08 2014 by BAC, for the + statistics module, on recommendation by Nick Coghlan. + - Yury Selivanov was given push privileges on Jan 23 2014 by GFB, for "inspect" module and general contributions, on recommendation by Nick Coghlan. -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Thu Apr 10 15:29:53 2014 From: python-checkins at python.org (mark.dickinson) Date: Thu, 10 Apr 2014 15:29:53 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320539=3A_Improve_?= =?utf-8?q?math=2Efactorial_error_messages_and_types_for_large_inputs=2E?= Message-ID: <3g4NWP4Vh1z7LkN@mail.python.org> http://hg.python.org/cpython/rev/273e17260d25 changeset: 90218:273e17260d25 user: Mark Dickinson date: Thu Apr 10 09:29:39 2014 -0400 summary: Issue #20539: Improve math.factorial error messages and types for large inputs. - Better message for the OverflowError in large positive inputs. - Changed exception type from OverflowError to ValueError for large negative inputs. files: Lib/test/test_math.py | 12 ++++++++++-- Misc/NEWS | 4 ++++ Modules/mathmodule.c | 16 ++++++++++++---- 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -422,9 +422,17 @@ self.assertEqual(math.factorial(i), py_factorial(i)) self.assertRaises(ValueError, math.factorial, -1) self.assertRaises(ValueError, math.factorial, -1.0) + self.assertRaises(ValueError, math.factorial, -10**100) + self.assertRaises(ValueError, math.factorial, -1e100) self.assertRaises(ValueError, math.factorial, math.pi) - self.assertRaises(OverflowError, math.factorial, sys.maxsize+1) - self.assertRaises(OverflowError, math.factorial, 10e100) + + # Other implementations may place different upper bounds. + @support.cpython_only + def testFactorialHugeInputs(self): + # Currently raises ValueError for inputs that are too large + # to fit into a C long. + self.assertRaises(OverflowError, math.factorial, 10**100) + self.assertRaises(OverflowError, math.factorial, 1e100) def testFloor(self): self.assertRaises(TypeError, math.floor) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -31,6 +31,10 @@ Library ------- +- Issue #20539: Improved math.factorial error message for large positive inputs + and changed exception type (OverflowError -> ValueError) for large negative + inputs. + - Issue #21172: isinstance check relaxed from dict to collections.Mapping. - Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -1408,6 +1408,7 @@ math_factorial(PyObject *self, PyObject *arg) { long x; + int overflow; PyObject *result, *odd_part, *two_valuation; if (PyFloat_Check(arg)) { @@ -1421,15 +1422,22 @@ lx = PyLong_FromDouble(dx); if (lx == NULL) return NULL; - x = PyLong_AsLong(lx); + x = PyLong_AsLongAndOverflow(lx, &overflow); Py_DECREF(lx); } else - x = PyLong_AsLong(arg); + x = PyLong_AsLongAndOverflow(arg, &overflow); - if (x == -1 && PyErr_Occurred()) + if (x == -1 && PyErr_Occurred()) { return NULL; - if (x < 0) { + } + else if (overflow == 1) { + PyErr_Format(PyExc_OverflowError, + "factorial() argument should not exceed %ld", + LONG_MAX); + return NULL; + } + else if (overflow == -1 || x < 0) { PyErr_SetString(PyExc_ValueError, "factorial() not defined for negative values"); return NULL; -- Repository URL: http://hg.python.org/cpython From root at python.org Thu Apr 10 16:55:57 2014 From: root at python.org (Cron Daemon) Date: Thu, 10 Apr 2014 16:55:57 +0200 Subject: [Python-checkins] Cron /home/docs/build-devguide Message-ID: abort: error: Name or service not known From solipsis at pitrou.net Fri Apr 11 09:54:56 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 11 Apr 2014 09:54:56 +0200 Subject: [Python-checkins] Daily reference leaks (273e17260d25): sum=3 Message-ID: results for 273e17260d25 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 2, 0] references, sum=0 test_site leaked [-2, 2, 0] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogT8_mDT', '-x'] From python-checkins at python.org Fri Apr 11 20:12:10 2014 From: python-checkins at python.org (terry.reedy) Date: Fri, 11 Apr 2014 20:12:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTcw?= =?utf-8?q?=3A_Removed_invalid_parameter_names_from_unittest_doc=2E?= Message-ID: <3g56kf1Bpvz7LjP@mail.python.org> http://hg.python.org/cpython/rev/c1ea2846a564 changeset: 90219:c1ea2846a564 branch: 2.7 parent: 90214:d08e3586dde3 user: Terry Jan Reedy date: Fri Apr 11 14:11:00 2014 -0400 summary: Issue #21170: Removed invalid parameter names from unittest doc. Patch by Kushal Das. files: Doc/library/unittest.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1687,14 +1687,14 @@ Called after the test case *test* has been executed, regardless of the outcome. - .. method:: startTestRun(test) + .. method:: startTestRun() Called once before any tests are executed. .. versionadded:: 2.7 - .. method:: stopTestRun(test) + .. method:: stopTestRun() Called once after all tests are executed. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 11 20:12:11 2014 From: python-checkins at python.org (terry.reedy) Date: Fri, 11 Apr 2014 20:12:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTcw?= =?utf-8?q?=3A_Removed_invalid_parameter_names_from_unittest_doc=2E?= Message-ID: <3g56kg31hyz7LjP@mail.python.org> http://hg.python.org/cpython/rev/5734175a87d1 changeset: 90220:5734175a87d1 branch: 3.4 parent: 90215:5e303360db14 user: Terry Jan Reedy date: Fri Apr 11 14:11:11 2014 -0400 summary: Issue #21170: Removed invalid parameter names from unittest doc. Patch by Kushal Das. files: Doc/library/unittest.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1807,14 +1807,14 @@ Called after the test case *test* has been executed, regardless of the outcome. - .. method:: startTestRun(test) + .. method:: startTestRun() Called once before any tests are executed. .. versionadded:: 3.1 - .. method:: stopTestRun(test) + .. method:: stopTestRun() Called once after all tests are executed. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 11 20:12:12 2014 From: python-checkins at python.org (terry.reedy) Date: Fri, 11 Apr 2014 20:12:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3g56kh4sl9z7Lk0@mail.python.org> http://hg.python.org/cpython/rev/a3f8afbd3973 changeset: 90221:a3f8afbd3973 parent: 90218:273e17260d25 parent: 90220:5734175a87d1 user: Terry Jan Reedy date: Fri Apr 11 14:11:34 2014 -0400 summary: Merge with 3.4 files: Doc/library/unittest.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1807,14 +1807,14 @@ Called after the test case *test* has been executed, regardless of the outcome. - .. method:: startTestRun(test) + .. method:: startTestRun() Called once before any tests are executed. .. versionadded:: 3.1 - .. method:: stopTestRun(test) + .. method:: stopTestRun() Called once after all tests are executed. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 11 20:35:15 2014 From: python-checkins at python.org (mark.dickinson) Date: Fri, 11 Apr 2014 20:35:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321193=3A_Make_=28?= =?utf-8?q?e=2Eg=2E=2C=29_pow=282=2C_-3=2C_5=29_raise_ValueError_rather_th?= =?utf-8?q?an?= Message-ID: <3g57FH2yqXz7LjM@mail.python.org> http://hg.python.org/cpython/rev/dc6c2ab7fec2 changeset: 90222:dc6c2ab7fec2 user: Mark Dickinson date: Fri Apr 11 14:34:40 2014 -0400 summary: Issue #21193: Make (e.g.,) pow(2, -3, 5) raise ValueError rather than TypeError. Patch by Josh Rosenberg. files: Lib/test/test_builtin.py | 2 +- Misc/ACKS | 1 + Misc/NEWS | 3 +++ Objects/longobject.c | 2 +- 4 files changed, 6 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -1085,7 +1085,7 @@ if isinstance(x, float) or \ isinstance(y, float) or \ isinstance(z, float): - self.assertRaises(TypeError, pow, x, y, z) + self.assertRaises(ValueError, pow, x, y, z) else: self.assertAlmostEqual(pow(x, y, z), 24.0) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1111,6 +1111,7 @@ Case Roole Timothy Roscoe Erik Rose +Josh Rosenberg Jim Roskind Brian Rosner Guido van Rossum diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #21193: pow(a, b, c) now raises ValueError rather than TypeError when b + is negative. Patch by Josh Rosenberg. + - PEP 465 and Issue #21176: Add the '@' operator for matrix multiplication. - Issue #21134: Fix segfault when str is called on an uninitialized diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -3841,7 +3841,7 @@ if (Py_SIZE(b) < 0) { /* if exponent is negative */ if (c) { - PyErr_SetString(PyExc_TypeError, "pow() 2nd argument " + PyErr_SetString(PyExc_ValueError, "pow() 2nd argument " "cannot be negative when 3rd argument specified"); goto Error; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 11 21:38:10 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 11 Apr 2014 21:38:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_test_the_change_of_=232119?= =?utf-8?q?3_correctly?= Message-ID: <3g58dt3rYcz7LkL@mail.python.org> http://hg.python.org/cpython/rev/a8f3ca72f703 changeset: 90223:a8f3ca72f703 user: Benjamin Peterson date: Fri Apr 11 15:37:18 2014 -0400 summary: test the change of #21193 correctly files: Lib/test/test_builtin.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -1085,14 +1085,14 @@ if isinstance(x, float) or \ isinstance(y, float) or \ isinstance(z, float): - self.assertRaises(ValueError, pow, x, y, z) + self.assertRaises(TypeError, pow, x, y, z) else: self.assertAlmostEqual(pow(x, y, z), 24.0) self.assertAlmostEqual(pow(-1, 0.5), 1j) self.assertAlmostEqual(pow(-1, 1/3), 0.5 + 0.8660254037844386j) - self.assertRaises(TypeError, pow, -1, -2, 3) + self.assertRaises(ValueError, pow, -1, -2, 3) self.assertRaises(ValueError, pow, 1, 2, 0) self.assertRaises(TypeError, pow) -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Apr 12 09:55:47 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 12 Apr 2014 09:55:47 +0200 Subject: [Python-checkins] Daily reference leaks (a8f3ca72f703): sum=7 Message-ID: results for a8f3ca72f703 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogXdgjek', '-x'] From python-checkins at python.org Sat Apr 12 18:33:42 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 12 Apr 2014 18:33:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Remove_referen?= =?utf-8?q?ces_to_the_obsolete_Mac_Carbon_modules_in_the_GUI?= Message-ID: <3g5hVZ1QvDz7LjP@mail.python.org> http://hg.python.org/cpython/rev/218e28a935ab changeset: 90224:218e28a935ab branch: 2.7 parent: 90219:c1ea2846a564 user: Ned Deily date: Sat Apr 12 09:31:28 2014 -0700 summary: Remove references to the obsolete Mac Carbon modules in the GUI section of the FAQ. files: Doc/faq/gui.rst | 14 +++++--------- 1 files changed, 5 insertions(+), 9 deletions(-) diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst --- a/Doc/faq/gui.rst +++ b/Doc/faq/gui.rst @@ -21,15 +21,15 @@ Standard builds of Python include an object-oriented interface to the Tcl/Tk widget set, called Tkinter. This is probably the easiest to install and use. For more info about Tk, including pointers to the source, see the Tcl/Tk home -page at http://www.tcl.tk. Tcl/Tk is fully portable to the MacOS, Windows, and -Unix platforms. +page at http://www.tcl.tk. Tcl/Tk is fully portable to the Mac OS X, Windows, +and Unix platforms. wxWidgets --------- wxWidgets (http://www.wxwidgets.org) is a free, portable GUI class library written in C++ that provides a native look and feel on a -number of platforms, with Windows, MacOS X, GTK, X11, all listed as +number of platforms, with Windows, Mac OS X, GTK, X11, all listed as current stable targets. Language bindings are available for a number of languages including Python, Perl, Ruby, etc. @@ -88,13 +88,9 @@ What platform-specific GUI toolkits exist for Python? ======================================================== -`The Mac port `_ by Jack Jansen has a rich and -ever-growing set of modules that support the native Mac toolbox calls. The port -supports MacOS X's Carbon libraries. - By installing the `PyObjc Objective-C bridge -`_, Python programs can use MacOS X's -Cocoa libraries. See the documentation that comes with the Mac port. +`_, Python programs can use Mac OS X's +Cocoa libraries. :ref:`Pythonwin ` by Mark Hammond includes an interface to the Microsoft Foundation Classes and a Python programming environment -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 12 18:33:43 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 12 Apr 2014 18:33:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Remove_referen?= =?utf-8?q?ces_to_the_obsolete_Mac_Carbon_modules_in_the_GUI?= Message-ID: <3g5hVb3Grwz7LjP@mail.python.org> http://hg.python.org/cpython/rev/b9a1e39c3240 changeset: 90225:b9a1e39c3240 branch: 3.4 parent: 90220:5734175a87d1 user: Ned Deily date: Sat Apr 12 09:32:04 2014 -0700 summary: Remove references to the obsolete Mac Carbon modules in the GUI section of the FAQ. files: Doc/faq/gui.rst | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst --- a/Doc/faq/gui.rst +++ b/Doc/faq/gui.rst @@ -32,14 +32,14 @@ `binary distributions `_ of Python) and use. For more info about Tk, including pointers to the source, see the `Tcl/Tk home page `_. Tcl/Tk is fully portable to the -MacOS, Windows, and Unix platforms. +Mac OS X, Windows, and Unix platforms. wxWidgets --------- wxWidgets (http://www.wxwidgets.org) is a free, portable GUI class library written in C++ that provides a native look and feel on a -number of platforms, with Windows, MacOS X, GTK, X11, all listed as +number of platforms, with Windows, Mac OS X, GTK, X11, all listed as current stable targets. Language bindings are available for a number of languages including Python, Perl, Ruby, etc. @@ -102,13 +102,9 @@ What platform-specific GUI toolkits exist for Python? ======================================================== -`The Mac port `_ by Jack Jansen has a rich and -ever-growing set of modules that support the native Mac toolbox calls. The port -supports MacOS X's Carbon libraries. - By installing the `PyObjc Objective-C bridge -`_, Python programs can use MacOS X's -Cocoa libraries. See the documentation that comes with the Mac port. +`_, Python programs can use Mac OS X's +Cocoa libraries. :ref:`Pythonwin ` by Mark Hammond includes an interface to the Microsoft Foundation Classes and a Python programming environment -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 12 18:33:44 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 12 Apr 2014 18:33:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Remove_references_to_the_obsolete_Mac_Carbon_modules_in_?= =?utf-8?q?the_GUI?= Message-ID: <3g5hVc5sTGz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/84bc6998f760 changeset: 90226:84bc6998f760 parent: 90223:a8f3ca72f703 parent: 90225:b9a1e39c3240 user: Ned Deily date: Sat Apr 12 09:33:05 2014 -0700 summary: Remove references to the obsolete Mac Carbon modules in the GUI section of the FAQ. files: Doc/faq/gui.rst | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Doc/faq/gui.rst b/Doc/faq/gui.rst --- a/Doc/faq/gui.rst +++ b/Doc/faq/gui.rst @@ -32,14 +32,14 @@ `binary distributions `_ of Python) and use. For more info about Tk, including pointers to the source, see the `Tcl/Tk home page `_. Tcl/Tk is fully portable to the -MacOS, Windows, and Unix platforms. +Mac OS X, Windows, and Unix platforms. wxWidgets --------- wxWidgets (http://www.wxwidgets.org) is a free, portable GUI class library written in C++ that provides a native look and feel on a -number of platforms, with Windows, MacOS X, GTK, X11, all listed as +number of platforms, with Windows, Mac OS X, GTK, X11, all listed as current stable targets. Language bindings are available for a number of languages including Python, Perl, Ruby, etc. @@ -102,13 +102,9 @@ What platform-specific GUI toolkits exist for Python? ======================================================== -`The Mac port `_ by Jack Jansen has a rich and -ever-growing set of modules that support the native Mac toolbox calls. The port -supports MacOS X's Carbon libraries. - By installing the `PyObjc Objective-C bridge -`_, Python programs can use MacOS X's -Cocoa libraries. See the documentation that comes with the Mac port. +`_, Python programs can use Mac OS X's +Cocoa libraries. :ref:`Pythonwin ` by Mark Hammond includes an interface to the Microsoft Foundation Classes and a Python programming environment -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sun Apr 13 09:55:23 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 13 Apr 2014 09:55:23 +0200 Subject: [Python-checkins] Daily reference leaks (84bc6998f760): sum=11 Message-ID: results for 84bc6998f760 on branch "default" -------------------------------------------- test_asyncio leaked [4, 0, 0] memory blocks, sum=4 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [2, 0, 0] references, sum=2 test_site leaked [2, 0, 0] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogMQwu59', '-x'] From python-checkins at python.org Sun Apr 13 16:09:05 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Apr 2014 16:09:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTcx?= =?utf-8?q?=3A_Fixed_undocumented_filter_API_of_the_rot13_codec=2E?= Message-ID: <3g6FFF2mQmzLq6@mail.python.org> http://hg.python.org/cpython/rev/a81f0caab279 changeset: 90227:a81f0caab279 branch: 3.4 parent: 90225:b9a1e39c3240 user: Serhiy Storchaka date: Sun Apr 13 17:07:04 2014 +0300 summary: Issue #21171: Fixed undocumented filter API of the rot13 codec. Patch by Berker Peksag. files: Lib/encodings/rot_13.py | 2 +- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/encodings/rot_13.py b/Lib/encodings/rot_13.py --- a/Lib/encodings/rot_13.py +++ b/Lib/encodings/rot_13.py @@ -106,7 +106,7 @@ ### Filter API def rot13(infile, outfile): - outfile.write(infile.read().encode('rot-13')) + outfile.write(codecs.encode(infile.read(), 'rot-13')) if __name__ == '__main__': import sys diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ Library ------- +- Issue #21171: Fixed undocumented filter API of the rot13 codec. + Patch by Berker Peksag. + - Issue #21172: isinstance check relaxed from dict to collections.Mapping. - Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 13 16:09:06 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Apr 2014 16:09:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321171=3A_Fixed_undocumented_filter_API_of_the_r?= =?utf-8?q?ot13_codec=2E?= Message-ID: <3g6FFG4TGlz7LjY@mail.python.org> http://hg.python.org/cpython/rev/f86504da2fcc changeset: 90228:f86504da2fcc parent: 90226:84bc6998f760 parent: 90227:a81f0caab279 user: Serhiy Storchaka date: Sun Apr 13 17:08:51 2014 +0300 summary: Issue #21171: Fixed undocumented filter API of the rot13 codec. Patch by Berker Peksag. files: Lib/encodings/rot_13.py | 2 +- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/encodings/rot_13.py b/Lib/encodings/rot_13.py --- a/Lib/encodings/rot_13.py +++ b/Lib/encodings/rot_13.py @@ -106,7 +106,7 @@ ### Filter API def rot13(infile, outfile): - outfile.write(infile.read().encode('rot-13')) + outfile.write(codecs.encode(infile.read(), 'rot-13')) if __name__ == '__main__': import sys diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,9 @@ Library ------- +- Issue #21171: Fixed undocumented filter API of the rot13 codec. + Patch by Berker Peksag. + - Issue #20539: Improved math.factorial error message for large positive inputs and changed exception type (OverflowError -> ValueError) for large negative inputs. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 13 16:38:59 2014 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 13 Apr 2014 16:38:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Extend_Python_2=2E7_life_till?= =?utf-8?q?_2020=2E?= Message-ID: <3g6Fvl3JTyz7LjT@mail.python.org> http://hg.python.org/peps/rev/76d43e52d978 changeset: 5457:76d43e52d978 user: Guido van Rossum date: Sun Apr 13 10:39:07 2014 -0400 summary: Extend Python 2.7 life till 2020. files: pep-0373.txt | 20 ++++++++++++++++++-- 1 files changed, 18 insertions(+), 2 deletions(-) diff --git a/pep-0373.txt b/pep-0373.txt --- a/pep-0373.txt +++ b/pep-0373.txt @@ -18,6 +18,21 @@ items. Small features may be added up to and including the first beta release. Bugs may be fixed until the final release. +Update +====== + +The End Of Life date (EOL, sunset date) for Python 2.7 has been moved +five years into the future, to 2020. This decision was made to +clarify the status of Python 2.7 and relieve worries for those users +who cannot yet migrate to Python 3. See also PEP 466. + +This declaration does not guarantee that bugfix releases will be made +on a regular basis, but it should enable volunteers who want to +contribute bugfixes for Python 2.7 and it should satisfy vendors who +still have to support Python 2 for years to come. + +There will be no Python 2.8. + Release Manager and Crew ======================== @@ -50,15 +65,16 @@ ==================== Being the last of the 2.x series, 2.7 will have an extended period of -maintenance. The current plan is to support it for at least 5 years +maintenance. The current plan is to support it for at least 10 years from the initial 2.7 release. This means there will be bugfix releases -until 2015. +until 2020. Planned future release dates: - 2.7.7 May 2014 - 2.7.8 November 2014 - 2.7.9 May 2015 +- beyond this date, releases as needed Dates of previous maintenance releases: -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Apr 13 18:55:27 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Apr 2014 18:55:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNjM1?= =?utf-8?q?=3A_Added_tests_for_Tk_geometry_managers=2E?= Message-ID: <3g6JxC5kpqz7LjY@mail.python.org> http://hg.python.org/cpython/rev/e8c184d8407d changeset: 90229:e8c184d8407d branch: 3.4 parent: 90227:a81f0caab279 user: Serhiy Storchaka date: Sun Apr 13 19:52:23 2014 +0300 summary: Issue #20635: Added tests for Tk geometry managers. files: Lib/tkinter/test/test_tkinter/test_geometry_managers.py | 902 ++++++++++ Lib/tkinter/test/widget_tests.py | 11 + Misc/NEWS | 2 + 3 files changed, 915 insertions(+), 0 deletions(-) diff --git a/Lib/tkinter/test/test_tkinter/test_geometry_managers.py b/Lib/tkinter/test/test_tkinter/test_geometry_managers.py new file mode 100644 --- /dev/null +++ b/Lib/tkinter/test/test_tkinter/test_geometry_managers.py @@ -0,0 +1,902 @@ +import unittest +import re +import tkinter +from tkinter import TclError +from test.support import requires + +from tkinter.test.support import pixels_conv, tcl_version, requires_tcl +from tkinter.test.widget_tests import AbstractWidgetTest + +requires('gui') + + +class PackTest(AbstractWidgetTest, unittest.TestCase): + + def create2(self): + pack = tkinter.Toplevel(self.root, name='pack') + pack.wm_geometry('300x200+0+0') + pack.wm_minsize(1, 1) + a = tkinter.Frame(pack, name='a', width=20, height=40, bg='red') + b = tkinter.Frame(pack, name='b', width=50, height=30, bg='blue') + c = tkinter.Frame(pack, name='c', width=80, height=80, bg='green') + d = tkinter.Frame(pack, name='d', width=40, height=30, bg='yellow') + return pack, a, b, c, d + + def test_pack_configure_after(self): + pack, a, b, c, d = self.create2() + with self.assertRaisesRegex(TclError, 'window "%s" isn\'t packed' % b): + a.pack_configure(after=b) + with self.assertRaisesRegex(TclError, 'bad window path name ".foo"'): + a.pack_configure(after='.foo') + a.pack_configure(side='top') + b.pack_configure(side='top') + c.pack_configure(side='top') + d.pack_configure(side='top') + self.assertEqual(pack.pack_slaves(), [a, b, c, d]) + a.pack_configure(after=b) + self.assertEqual(pack.pack_slaves(), [b, a, c, d]) + a.pack_configure(after=a) + self.assertEqual(pack.pack_slaves(), [b, a, c, d]) + + def test_pack_configure_anchor(self): + pack, a, b, c, d = self.create2() + def check(anchor, geom): + a.pack_configure(side='top', ipadx=5, padx=10, ipady=15, pady=20, + expand=True, anchor=anchor) + self.root.update() + self.assertEqual(a.winfo_geometry(), geom) + check('n', '30x70+135+20') + check('ne', '30x70+260+20') + check('e', '30x70+260+65') + check('se', '30x70+260+110') + check('s', '30x70+135+110') + check('sw', '30x70+10+110') + check('w', '30x70+10+65') + check('nw', '30x70+10+20') + check('center', '30x70+135+65') + + def test_pack_configure_before(self): + pack, a, b, c, d = self.create2() + with self.assertRaisesRegex(TclError, 'window "%s" isn\'t packed' % b): + a.pack_configure(before=b) + with self.assertRaisesRegex(TclError, 'bad window path name ".foo"'): + a.pack_configure(before='.foo') + a.pack_configure(side='top') + b.pack_configure(side='top') + c.pack_configure(side='top') + d.pack_configure(side='top') + self.assertEqual(pack.pack_slaves(), [a, b, c, d]) + a.pack_configure(before=d) + self.assertEqual(pack.pack_slaves(), [b, c, a, d]) + a.pack_configure(before=a) + self.assertEqual(pack.pack_slaves(), [b, c, a, d]) + + def test_pack_configure_expand(self): + pack, a, b, c, d = self.create2() + def check(*geoms): + self.root.update() + self.assertEqual(a.winfo_geometry(), geoms[0]) + self.assertEqual(b.winfo_geometry(), geoms[1]) + self.assertEqual(c.winfo_geometry(), geoms[2]) + self.assertEqual(d.winfo_geometry(), geoms[3]) + a.pack_configure(side='left') + b.pack_configure(side='top') + c.pack_configure(side='right') + d.pack_configure(side='bottom') + check('20x40+0+80', '50x30+135+0', '80x80+220+75', '40x30+100+170') + a.pack_configure(side='left', expand='yes') + b.pack_configure(side='top', expand='on') + c.pack_configure(side='right', expand=True) + d.pack_configure(side='bottom', expand=1) + check('20x40+40+80', '50x30+175+35', '80x80+180+110', '40x30+100+135') + a.pack_configure(side='left', expand='yes', fill='both') + b.pack_configure(side='top', expand='on', fill='both') + c.pack_configure(side='right', expand=True, fill='both') + d.pack_configure(side='bottom', expand=1, fill='both') + check('100x200+0+0', '200x100+100+0', '160x100+140+100', '40x100+100+100') + + def test_pack_configure_in(self): + pack, a, b, c, d = self.create2() + a.pack_configure(side='top') + b.pack_configure(side='top') + c.pack_configure(side='top') + d.pack_configure(side='top') + a.pack_configure(in_=pack) + self.assertEqual(pack.pack_slaves(), [b, c, d, a]) + a.pack_configure(in_=c) + self.assertEqual(pack.pack_slaves(), [b, c, d]) + self.assertEqual(c.pack_slaves(), [a]) + with self.assertRaisesRegex(TclError, + 'can\'t pack %s inside itself' % (a,)): + a.pack_configure(in_=a) + with self.assertRaisesRegex(TclError, 'bad window path name ".foo"'): + a.pack_configure(in_='.foo') + + def test_pack_configure_padx_ipadx_fill(self): + pack, a, b, c, d = self.create2() + def check(geom1, geom2, **kwargs): + a.pack_forget() + b.pack_forget() + a.pack_configure(**kwargs) + b.pack_configure(expand=True, fill='both') + self.root.update() + self.assertEqual(a.winfo_geometry(), geom1) + self.assertEqual(b.winfo_geometry(), geom2) + check('20x40+260+80', '240x200+0+0', side='right', padx=20) + check('20x40+250+80', '240x200+0+0', side='right', padx=(10, 30)) + check('60x40+240+80', '240x200+0+0', side='right', ipadx=20) + check('30x40+260+80', '250x200+0+0', side='right', ipadx=5, padx=10) + check('20x40+260+80', '240x200+0+0', side='right', padx=20, fill='x') + check('20x40+249+80', '240x200+0+0', + side='right', padx=(9, 31), fill='x') + check('60x40+240+80', '240x200+0+0', side='right', ipadx=20, fill='x') + check('30x40+260+80', '250x200+0+0', + side='right', ipadx=5, padx=10, fill='x') + check('30x40+255+80', '250x200+0+0', + side='right', ipadx=5, padx=(5, 15), fill='x') + check('20x40+140+0', '300x160+0+40', side='top', padx=20) + check('20x40+120+0', '300x160+0+40', side='top', padx=(0, 40)) + check('60x40+120+0', '300x160+0+40', side='top', ipadx=20) + check('30x40+135+0', '300x160+0+40', side='top', ipadx=5, padx=10) + check('30x40+130+0', '300x160+0+40', side='top', ipadx=5, padx=(5, 15)) + check('260x40+20+0', '300x160+0+40', side='top', padx=20, fill='x') + check('260x40+25+0', '300x160+0+40', + side='top', padx=(25, 15), fill='x') + check('300x40+0+0', '300x160+0+40', side='top', ipadx=20, fill='x') + check('280x40+10+0', '300x160+0+40', + side='top', ipadx=5, padx=10, fill='x') + check('280x40+5+0', '300x160+0+40', + side='top', ipadx=5, padx=(5, 15), fill='x') + a.pack_configure(padx='1c') + self.assertEqual(a.pack_info()['padx'], + self._str(pack.winfo_pixels('1c'))) + a.pack_configure(ipadx='1c') + self.assertEqual(a.pack_info()['ipadx'], + self._str(pack.winfo_pixels('1c'))) + + def test_pack_configure_pady_ipady_fill(self): + pack, a, b, c, d = self.create2() + def check(geom1, geom2, **kwargs): + a.pack_forget() + b.pack_forget() + a.pack_configure(**kwargs) + b.pack_configure(expand=True, fill='both') + self.root.update() + self.assertEqual(a.winfo_geometry(), geom1) + self.assertEqual(b.winfo_geometry(), geom2) + check('20x40+280+80', '280x200+0+0', side='right', pady=20) + check('20x40+280+70', '280x200+0+0', side='right', pady=(10, 30)) + check('20x80+280+60', '280x200+0+0', side='right', ipady=20) + check('20x50+280+75', '280x200+0+0', side='right', ipady=5, pady=10) + check('20x40+280+80', '280x200+0+0', side='right', pady=20, fill='x') + check('20x40+280+69', '280x200+0+0', + side='right', pady=(9, 31), fill='x') + check('20x80+280+60', '280x200+0+0', side='right', ipady=20, fill='x') + check('20x50+280+75', '280x200+0+0', + side='right', ipady=5, pady=10, fill='x') + check('20x50+280+70', '280x200+0+0', + side='right', ipady=5, pady=(5, 15), fill='x') + check('20x40+140+20', '300x120+0+80', side='top', pady=20) + check('20x40+140+0', '300x120+0+80', side='top', pady=(0, 40)) + check('20x80+140+0', '300x120+0+80', side='top', ipady=20) + check('20x50+140+10', '300x130+0+70', side='top', ipady=5, pady=10) + check('20x50+140+5', '300x130+0+70', side='top', ipady=5, pady=(5, 15)) + check('300x40+0+20', '300x120+0+80', side='top', pady=20, fill='x') + check('300x40+0+25', '300x120+0+80', + side='top', pady=(25, 15), fill='x') + check('300x80+0+0', '300x120+0+80', side='top', ipady=20, fill='x') + check('300x50+0+10', '300x130+0+70', + side='top', ipady=5, pady=10, fill='x') + check('300x50+0+5', '300x130+0+70', + side='top', ipady=5, pady=(5, 15), fill='x') + a.pack_configure(pady='1c') + self.assertEqual(a.pack_info()['pady'], + self._str(pack.winfo_pixels('1c'))) + a.pack_configure(ipady='1c') + self.assertEqual(a.pack_info()['ipady'], + self._str(pack.winfo_pixels('1c'))) + + def test_pack_configure_side(self): + pack, a, b, c, d = self.create2() + def check(side, geom1, geom2): + a.pack_configure(side=side) + self.assertEqual(a.pack_info()['side'], side) + b.pack_configure(expand=True, fill='both') + self.root.update() + self.assertEqual(a.winfo_geometry(), geom1) + self.assertEqual(b.winfo_geometry(), geom2) + check('top', '20x40+140+0', '300x160+0+40') + check('bottom', '20x40+140+160', '300x160+0+0') + check('left', '20x40+0+80', '280x200+20+0') + check('right', '20x40+280+80', '280x200+0+0') + + def test_pack_forget(self): + pack, a, b, c, d = self.create2() + a.pack_configure() + b.pack_configure() + c.pack_configure() + self.assertEqual(pack.pack_slaves(), [a, b, c]) + b.pack_forget() + self.assertEqual(pack.pack_slaves(), [a, c]) + b.pack_forget() + self.assertEqual(pack.pack_slaves(), [a, c]) + d.pack_forget() + + def test_pack_info(self): + pack, a, b, c, d = self.create2() + with self.assertRaisesRegex(TclError, 'window "%s" isn\'t packed' % a): + a.pack_info() + a.pack_configure() + b.pack_configure(side='right', in_=a, anchor='s', expand=True, fill='x', + ipadx=5, padx=10, ipady=2, pady=(5, 15)) + info = a.pack_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['anchor'], 'center') + self.assertEqual(info['expand'], self._str(0)) + self.assertEqual(info['fill'], 'none') + self.assertEqual(info['in'], pack) + self.assertEqual(info['ipadx'], self._str(0)) + self.assertEqual(info['ipady'], self._str(0)) + self.assertEqual(info['padx'], self._str(0)) + self.assertEqual(info['pady'], self._str(0)) + self.assertEqual(info['side'], 'top') + info = b.pack_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['anchor'], 's') + self.assertEqual(info['expand'], self._str(1)) + self.assertEqual(info['fill'], 'x') + self.assertEqual(info['in'], a) + self.assertEqual(info['ipadx'], self._str(5)) + self.assertEqual(info['ipady'], self._str(2)) + self.assertEqual(info['padx'], self._str(10)) + self.assertEqual(info['pady'], self._str((5, 15))) + self.assertEqual(info['side'], 'right') + + def test_pack_propagate(self): + pack, a, b, c, d = self.create2() + pack.configure(width=300, height=200) + a.pack_configure() + pack.pack_propagate(False) + self.root.update() + self.assertEqual(pack.winfo_reqwidth(), 300) + self.assertEqual(pack.winfo_reqheight(), 200) + pack.pack_propagate(True) + self.root.update() + self.assertEqual(pack.winfo_reqwidth(), 20) + self.assertEqual(pack.winfo_reqheight(), 40) + + def test_pack_slaves(self): + pack, a, b, c, d = self.create2() + self.assertEqual(pack.pack_slaves(), []) + a.pack_configure() + self.assertEqual(pack.pack_slaves(), [a]) + b.pack_configure() + self.assertEqual(pack.pack_slaves(), [a, b]) + + +class PlaceTest(AbstractWidgetTest, unittest.TestCase): + + def create2(self): + t = tkinter.Toplevel(self.root, width=300, height=200, bd=0) + t.wm_geometry('+0+0') + f = tkinter.Frame(t, width=154, height=84, bd=2, relief='raised') + f.place_configure(x=48, y=38) + f2 = tkinter.Frame(t, width=30, height=60, bd=2, relief='raised') + self.root.update() + return t, f, f2 + + def test_place_configure_in(self): + t, f, f2 = self.create2() + self.assertEqual(f2.winfo_manager(), '') + with self.assertRaisesRegex(TclError, "can't place %s relative to " + "itself" % re.escape(str(f2))): + f2.place_configure(in_=f2) + if tcl_version >= (8, 5): + self.assertEqual(f2.winfo_manager(), '') + with self.assertRaisesRegex(TclError, 'bad window path name'): + f2.place_configure(in_='spam') + f2.place_configure(in_=f) + self.assertEqual(f2.winfo_manager(), 'place') + + def test_place_configure_x(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['x'], '0') + self.root.update() + self.assertEqual(f2.winfo_x(), 50) + f2.place_configure(x=100) + self.assertEqual(f2.place_info()['x'], '100') + self.root.update() + self.assertEqual(f2.winfo_x(), 150) + f2.place_configure(x=-10, relx=1) + self.assertEqual(f2.place_info()['x'], '-10') + self.root.update() + self.assertEqual(f2.winfo_x(), 190) + with self.assertRaisesRegex(TclError, 'bad screen distance "spam"'): + f2.place_configure(in_=f, x='spam') + + def test_place_configure_y(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['y'], '0') + self.root.update() + self.assertEqual(f2.winfo_y(), 40) + f2.place_configure(y=50) + self.assertEqual(f2.place_info()['y'], '50') + self.root.update() + self.assertEqual(f2.winfo_y(), 90) + f2.place_configure(y=-10, rely=1) + self.assertEqual(f2.place_info()['y'], '-10') + self.root.update() + self.assertEqual(f2.winfo_y(), 110) + with self.assertRaisesRegex(TclError, 'bad screen distance "spam"'): + f2.place_configure(in_=f, y='spam') + + def test_place_configure_relx(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['relx'], '0') + self.root.update() + self.assertEqual(f2.winfo_x(), 50) + f2.place_configure(relx=0.5) + self.assertEqual(f2.place_info()['relx'], '0.5') + self.root.update() + self.assertEqual(f2.winfo_x(), 125) + f2.place_configure(relx=1) + self.assertEqual(f2.place_info()['relx'], '1') + self.root.update() + self.assertEqual(f2.winfo_x(), 200) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "spam"'): + f2.place_configure(in_=f, relx='spam') + + def test_place_configure_rely(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['rely'], '0') + self.root.update() + self.assertEqual(f2.winfo_y(), 40) + f2.place_configure(rely=0.5) + self.assertEqual(f2.place_info()['rely'], '0.5') + self.root.update() + self.assertEqual(f2.winfo_y(), 80) + f2.place_configure(rely=1) + self.assertEqual(f2.place_info()['rely'], '1') + self.root.update() + self.assertEqual(f2.winfo_y(), 120) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "spam"'): + f2.place_configure(in_=f, rely='spam') + + def test_place_configure_anchor(self): + f = tkinter.Frame(self.root) + with self.assertRaisesRegex(TclError, 'bad anchor "j"'): + f.place_configure(anchor='j') + with self.assertRaisesRegex(TclError, 'ambiguous anchor ""'): + f.place_configure(anchor='') + for value in 'n', 'ne', 'e', 'se', 's', 'sw', 'w', 'nw', 'center': + f.place_configure(anchor=value) + self.assertEqual(f.place_info()['anchor'], value) + + def test_place_configure_width(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, width=120) + self.root.update() + self.assertEqual(f2.winfo_width(), 120) + f2.place_configure(width='') + self.root.update() + self.assertEqual(f2.winfo_width(), 30) + with self.assertRaisesRegex(TclError, 'bad screen distance "abcd"'): + f2.place_configure(width='abcd') + + def test_place_configure_height(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, height=120) + self.root.update() + self.assertEqual(f2.winfo_height(), 120) + f2.place_configure(height='') + self.root.update() + self.assertEqual(f2.winfo_height(), 60) + with self.assertRaisesRegex(TclError, 'bad screen distance "abcd"'): + f2.place_configure(height='abcd') + + def test_place_configure_relwidth(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, relwidth=0.5) + self.root.update() + self.assertEqual(f2.winfo_width(), 75) + f2.place_configure(relwidth='') + self.root.update() + self.assertEqual(f2.winfo_width(), 30) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "abcd"'): + f2.place_configure(relwidth='abcd') + + def test_place_configure_relheight(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, relheight=0.5) + self.root.update() + self.assertEqual(f2.winfo_height(), 40) + f2.place_configure(relheight='') + self.root.update() + self.assertEqual(f2.winfo_height(), 60) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "abcd"'): + f2.place_configure(relheight='abcd') + + def test_place_configure_bordermode(self): + f = tkinter.Frame(self.root) + with self.assertRaisesRegex(TclError, 'bad bordermode "j"'): + f.place_configure(bordermode='j') + with self.assertRaisesRegex(TclError, 'ambiguous bordermode ""'): + f.place_configure(bordermode='') + for value in 'inside', 'outside', 'ignore': + f.place_configure(bordermode=value) + self.assertEqual(f.place_info()['bordermode'], value) + + def test_place_forget(self): + foo = tkinter.Frame(self.root) + foo.place_configure(width=50, height=50) + self.root.update() + foo.place_forget() + self.root.update() + self.assertFalse(foo.winfo_ismapped()) + with self.assertRaises(TypeError): + foo.place_forget(0) + + def test_place_info(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, x=1, y=2, width=3, height=4, + relx=0.1, rely=0.2, relwidth=0.3, relheight=0.4, + anchor='se', bordermode='outside') + info = f2.place_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['x'], '1') + self.assertEqual(info['y'], '2') + self.assertEqual(info['width'], '3') + self.assertEqual(info['height'], '4') + self.assertEqual(info['relx'], '0.1') + self.assertEqual(info['rely'], '0.2') + self.assertEqual(info['relwidth'], '0.3') + self.assertEqual(info['relheight'], '0.4') + self.assertEqual(info['anchor'], 'se') + self.assertEqual(info['bordermode'], 'outside') + self.assertEqual(info['x'], '1') + self.assertEqual(info['x'], '1') + with self.assertRaises(TypeError): + f2.place_info(0) + + def test_place_slaves(self): + foo = tkinter.Frame(self.root) + bar = tkinter.Frame(self.root) + self.assertEqual(foo.place_slaves(), []) + bar.place_configure(in_=foo) + self.assertEqual(foo.place_slaves(), [bar]) + with self.assertRaises(TypeError): + foo.place_slaves(0) + + +class GridTest(AbstractWidgetTest, unittest.TestCase): + + def tearDown(self): + super().tearDown() + cols, rows = self.root.grid_size() + for i in range(cols + 1): + self.root.grid_columnconfigure(i, weight=0, minsize=0, pad=0, uniform='') + for i in range(rows + 1): + self.root.grid_rowconfigure(i, weight=0, minsize=0, pad=0, uniform='') + self.root.grid_propagate(1) + if tcl_version >= (8, 5): + self.root.grid_anchor('nw') + + def test_grid_configure(self): + b = tkinter.Button(self.root) + self.addCleanup(b.destroy) + self.assertEqual(b.grid_info(), {}) + b.grid_configure() + self.assertEqual(b.grid_info()['in'], self.root) + self.assertEqual(b.grid_info()['column'], self._str(0)) + self.assertEqual(b.grid_info()['row'], self._str(0)) + b.grid_configure({'column': 1}, row=2) + self.assertEqual(b.grid_info()['column'], self._str(1)) + self.assertEqual(b.grid_info()['row'], self._str(2)) + + def test_grid_configure_column(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad column value "-1": ' + 'must be a non-negative integer'): + b.grid_configure(column=-1) + b.grid_configure(column=2) + self.assertEqual(b.grid_info()['column'], self._str(2)) + + def test_grid_configure_columnspan(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad columnspan value "0": ' + 'must be a positive integer'): + b.grid_configure(columnspan=0) + b.grid_configure(columnspan=2) + self.assertEqual(b.grid_info()['columnspan'], self._str(2)) + + def test_grid_configure_in(self): + f = tkinter.Frame(self.root) + b = tkinter.Button(self.root) + self.assertEqual(b.grid_info(), {}) + b.grid_configure() + self.assertEqual(b.grid_info()['in'], self.root) + b.grid_configure(in_=f) + self.assertEqual(b.grid_info()['in'], f) + b.grid_configure({'in': self.root}) + self.assertEqual(b.grid_info()['in'], self.root) + + def test_grid_configure_ipadx(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad ipadx value "-1": ' + 'must be positive screen distance'): + b.grid_configure(ipadx=-1) + b.grid_configure(ipadx=1) + self.assertEqual(b.grid_info()['ipadx'], self._str(1)) + b.grid_configure(ipadx='.5c') + self.assertEqual(b.grid_info()['ipadx'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_ipady(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad ipady value "-1": ' + 'must be positive screen distance'): + b.grid_configure(ipady=-1) + b.grid_configure(ipady=1) + self.assertEqual(b.grid_info()['ipady'], self._str(1)) + b.grid_configure(ipady='.5c') + self.assertEqual(b.grid_info()['ipady'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_padx(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad pad value "-1": ' + 'must be positive screen distance'): + b.grid_configure(padx=-1) + b.grid_configure(padx=1) + self.assertEqual(b.grid_info()['padx'], self._str(1)) + b.grid_configure(padx=(10, 5)) + self.assertEqual(b.grid_info()['padx'], self._str((10, 5))) + b.grid_configure(padx='.5c') + self.assertEqual(b.grid_info()['padx'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_pady(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad pad value "-1": ' + 'must be positive screen distance'): + b.grid_configure(pady=-1) + b.grid_configure(pady=1) + self.assertEqual(b.grid_info()['pady'], self._str(1)) + b.grid_configure(pady=(10, 5)) + self.assertEqual(b.grid_info()['pady'], self._str((10, 5))) + b.grid_configure(pady='.5c') + self.assertEqual(b.grid_info()['pady'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_row(self): + b = tkinter.Button(self.root) + self.addCleanup(b.destroy) + with self.assertRaisesRegex(TclError, 'bad (row|grid) value "-1": ' + 'must be a non-negative integer'): + b.grid_configure(row=-1) + b.grid_configure(row=2) + self.assertEqual(b.grid_info()['row'], self._str(2)) + + def test_grid_configure_rownspan(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad rowspan value "0": ' + 'must be a positive integer'): + b.grid_configure(rowspan=0) + b.grid_configure(rowspan=2) + self.assertEqual(b.grid_info()['rowspan'], self._str(2)) + + def test_grid_configure_sticky(self): + f = tkinter.Frame(self.root, bg='red') + with self.assertRaisesRegex(TclError, 'bad stickyness value "glue"'): + f.grid_configure(sticky='glue') + f.grid_configure(sticky='ne') + self.assertEqual(f.grid_info()['sticky'], 'ne') + f.grid_configure(sticky='n,s,e,w') + self.assertEqual(f.grid_info()['sticky'], 'nesw') + + def test_grid_columnconfigure(self): + with self.assertRaises(TypeError): + self.root.grid_columnconfigure() + self.assertEqual(self.root.grid_columnconfigure(0), + {'minsize': 0, 'pad': 0, 'uniform': None, 'weight': 0}) + with self.assertRaisesRegex(TclError, 'bad option "-foo"'): + self.root.grid_columnconfigure(0, 'foo') + self.root.grid_columnconfigure((0, 3), weight=2) + with self.assertRaisesRegex(TclError, + 'must specify a single element on retrieval'): + self.root.grid_columnconfigure((0, 3)) + b = tkinter.Button(self.root) + b.grid_configure(column=0, row=0) + if tcl_version >= (8, 5): + self.root.grid_columnconfigure('all', weight=3) + with self.assertRaisesRegex(TclError, 'expected integer but got "all"'): + self.root.grid_columnconfigure('all') + self.assertEqual(self.root.grid_columnconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_columnconfigure(3, 'weight'), 2) + self.assertEqual(self.root.grid_columnconfigure(265, 'weight'), 0) + if tcl_version >= (8, 5): + self.root.grid_columnconfigure(b, weight=4) + self.assertEqual(self.root.grid_columnconfigure(0, 'weight'), 4) + + def test_grid_columnconfigure_minsize(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_columnconfigure(0, minsize='foo') + self.root.grid_columnconfigure(0, minsize=10) + self.assertEqual(self.root.grid_columnconfigure(0, 'minsize'), 10) + self.assertEqual(self.root.grid_columnconfigure(0)['minsize'], 10) + + def test_grid_columnconfigure_weight(self): + with self.assertRaisesRegex(TclError, 'expected integer but got "bad"'): + self.root.grid_columnconfigure(0, weight='bad') + with self.assertRaisesRegex(TclError, 'invalid arg "-weight": ' + 'should be non-negative'): + self.root.grid_columnconfigure(0, weight=-3) + self.root.grid_columnconfigure(0, weight=3) + self.assertEqual(self.root.grid_columnconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_columnconfigure(0)['weight'], 3) + + def test_grid_columnconfigure_pad(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_columnconfigure(0, pad='foo') + with self.assertRaisesRegex(TclError, 'invalid arg "-pad": ' + 'should be non-negative'): + self.root.grid_columnconfigure(0, pad=-3) + self.root.grid_columnconfigure(0, pad=3) + self.assertEqual(self.root.grid_columnconfigure(0, 'pad'), 3) + self.assertEqual(self.root.grid_columnconfigure(0)['pad'], 3) + + def test_grid_columnconfigure_uniform(self): + self.root.grid_columnconfigure(0, uniform='foo') + self.assertEqual(self.root.grid_columnconfigure(0, 'uniform'), 'foo') + self.assertEqual(self.root.grid_columnconfigure(0)['uniform'], 'foo') + + def test_grid_rowconfigure(self): + with self.assertRaises(TypeError): + self.root.grid_rowconfigure() + self.assertEqual(self.root.grid_rowconfigure(0), + {'minsize': 0, 'pad': 0, 'uniform': None, 'weight': 0}) + with self.assertRaisesRegex(TclError, 'bad option "-foo"'): + self.root.grid_rowconfigure(0, 'foo') + self.root.grid_rowconfigure((0, 3), weight=2) + with self.assertRaisesRegex(TclError, + 'must specify a single element on retrieval'): + self.root.grid_rowconfigure((0, 3)) + b = tkinter.Button(self.root) + b.grid_configure(column=0, row=0) + if tcl_version >= (8, 5): + self.root.grid_rowconfigure('all', weight=3) + with self.assertRaisesRegex(TclError, 'expected integer but got "all"'): + self.root.grid_rowconfigure('all') + self.assertEqual(self.root.grid_rowconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_rowconfigure(3, 'weight'), 2) + self.assertEqual(self.root.grid_rowconfigure(265, 'weight'), 0) + if tcl_version >= (8, 5): + self.root.grid_rowconfigure(b, weight=4) + self.assertEqual(self.root.grid_rowconfigure(0, 'weight'), 4) + + def test_grid_rowconfigure_minsize(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_rowconfigure(0, minsize='foo') + self.root.grid_rowconfigure(0, minsize=10) + self.assertEqual(self.root.grid_rowconfigure(0, 'minsize'), 10) + self.assertEqual(self.root.grid_rowconfigure(0)['minsize'], 10) + + def test_grid_rowconfigure_weight(self): + with self.assertRaisesRegex(TclError, 'expected integer but got "bad"'): + self.root.grid_rowconfigure(0, weight='bad') + with self.assertRaisesRegex(TclError, 'invalid arg "-weight": ' + 'should be non-negative'): + self.root.grid_rowconfigure(0, weight=-3) + self.root.grid_rowconfigure(0, weight=3) + self.assertEqual(self.root.grid_rowconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_rowconfigure(0)['weight'], 3) + + def test_grid_rowconfigure_pad(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_rowconfigure(0, pad='foo') + with self.assertRaisesRegex(TclError, 'invalid arg "-pad": ' + 'should be non-negative'): + self.root.grid_rowconfigure(0, pad=-3) + self.root.grid_rowconfigure(0, pad=3) + self.assertEqual(self.root.grid_rowconfigure(0, 'pad'), 3) + self.assertEqual(self.root.grid_rowconfigure(0)['pad'], 3) + + def test_grid_rowconfigure_uniform(self): + self.root.grid_rowconfigure(0, uniform='foo') + self.assertEqual(self.root.grid_rowconfigure(0, 'uniform'), 'foo') + self.assertEqual(self.root.grid_rowconfigure(0)['uniform'], 'foo') + + def test_grid_forget(self): + b = tkinter.Button(self.root) + c = tkinter.Button(self.root) + b.grid_configure(row=2, column=2, rowspan=2, columnspan=2, + padx=3, pady=4, sticky='ns') + self.assertEqual(self.root.grid_slaves(), [b]) + b.grid_forget() + c.grid_forget() + self.assertEqual(self.root.grid_slaves(), []) + self.assertEqual(b.grid_info(), {}) + b.grid_configure(row=0, column=0) + info = b.grid_info() + self.assertEqual(info['row'], self._str(0)) + self.assertEqual(info['column'], self._str(0)) + self.assertEqual(info['rowspan'], self._str(1)) + self.assertEqual(info['columnspan'], self._str(1)) + self.assertEqual(info['padx'], self._str(0)) + self.assertEqual(info['pady'], self._str(0)) + self.assertEqual(info['sticky'], '') + + def test_grid_remove(self): + b = tkinter.Button(self.root) + c = tkinter.Button(self.root) + b.grid_configure(row=2, column=2, rowspan=2, columnspan=2, + padx=3, pady=4, sticky='ns') + self.assertEqual(self.root.grid_slaves(), [b]) + b.grid_remove() + c.grid_remove() + self.assertEqual(self.root.grid_slaves(), []) + self.assertEqual(b.grid_info(), {}) + b.grid_configure(row=0, column=0) + info = b.grid_info() + self.assertEqual(info['row'], self._str(0)) + self.assertEqual(info['column'], self._str(0)) + self.assertEqual(info['rowspan'], self._str(2)) + self.assertEqual(info['columnspan'], self._str(2)) + self.assertEqual(info['padx'], self._str(3)) + self.assertEqual(info['pady'], self._str(4)) + self.assertEqual(info['sticky'], 'ns') + + def test_grid_info(self): + b = tkinter.Button(self.root) + self.assertEqual(b.grid_info(), {}) + b.grid_configure(row=2, column=2, rowspan=2, columnspan=2, + padx=3, pady=4, sticky='ns') + info = b.grid_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['in'], self.root) + self.assertEqual(info['row'], self._str(2)) + self.assertEqual(info['column'], self._str(2)) + self.assertEqual(info['rowspan'], self._str(2)) + self.assertEqual(info['columnspan'], self._str(2)) + self.assertEqual(info['padx'], self._str(3)) + self.assertEqual(info['pady'], self._str(4)) + self.assertEqual(info['sticky'], 'ns') + + @requires_tcl(8, 5) + def test_grid_anchor(self): + with self.assertRaisesRegex(TclError, 'bad anchor "x"'): + self.root.grid_anchor('x') + with self.assertRaisesRegex(TclError, 'ambiguous anchor ""'): + self.root.grid_anchor('') + with self.assertRaises(TypeError): + self.root.grid_anchor('se', 'nw') + self.root.grid_anchor('se') + self.assertEqual(self.root.tk.call('grid', 'anchor', self.root), 'se') + + def test_grid_bbox(self): + self.assertEqual(self.root.grid_bbox(), (0, 0, 0, 0)) + self.assertEqual(self.root.grid_bbox(0, 0), (0, 0, 0, 0)) + self.assertEqual(self.root.grid_bbox(0, 0, 1, 1), (0, 0, 0, 0)) + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox('x', 0) + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox(0, 'x') + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox(0, 0, 'x', 0) + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox(0, 0, 0, 'x') + with self.assertRaises(TypeError): + self.root.grid_bbox(0, 0, 0, 0, 0) + t = tkinter.Toplevel(self.root) + # de-maximize + t.wm_geometry('1x1+0+0') + t.wm_geometry('') + f1 = tkinter.Frame(t, width=75, height=75, bg='red') + f2 = tkinter.Frame(t, width=90, height=90, bg='blue') + f1.grid_configure(row=0, column=0) + f2.grid_configure(row=1, column=1) + self.root.update() + self.assertEqual(t.grid_bbox(), (0, 0, 165, 165)) + self.assertEqual(t.grid_bbox(0, 0), (0, 0, 75, 75)) + self.assertEqual(t.grid_bbox(0, 0, 1, 1), (0, 0, 165, 165)) + self.assertEqual(t.grid_bbox(1, 1), (75, 75, 90, 90)) + self.assertEqual(t.grid_bbox(10, 10, 0, 0), (0, 0, 165, 165)) + self.assertEqual(t.grid_bbox(-2, -2, -1, -1), (0, 0, 0, 0)) + self.assertEqual(t.grid_bbox(10, 10, 12, 12), (165, 165, 0, 0)) + + def test_grid_location(self): + with self.assertRaises(TypeError): + self.root.grid_location() + with self.assertRaises(TypeError): + self.root.grid_location(0) + with self.assertRaises(TypeError): + self.root.grid_location(0, 0, 0) + with self.assertRaisesRegex(TclError, 'bad screen distance "x"'): + self.root.grid_location('x', 'y') + with self.assertRaisesRegex(TclError, 'bad screen distance "y"'): + self.root.grid_location('1c', 'y') + t = tkinter.Toplevel(self.root) + # de-maximize + t.wm_geometry('1x1+0+0') + t.wm_geometry('') + f = tkinter.Frame(t, width=200, height=100, + highlightthickness=0, bg='red') + self.assertEqual(f.grid_location(10, 10), (-1, -1)) + f.grid_configure() + self.root.update() + self.assertEqual(t.grid_location(-10, -10), (-1, -1)) + self.assertEqual(t.grid_location(-10, 0), (-1, 0)) + self.assertEqual(t.grid_location(-1, 0), (-1, 0)) + self.assertEqual(t.grid_location(0, -10), (0, -1)) + self.assertEqual(t.grid_location(0, -1), (0, -1)) + self.assertEqual(t.grid_location(0, 0), (0, 0)) + self.assertEqual(t.grid_location(200, 0), (0, 0)) + self.assertEqual(t.grid_location(201, 0), (1, 0)) + self.assertEqual(t.grid_location(0, 100), (0, 0)) + self.assertEqual(t.grid_location(0, 101), (0, 1)) + self.assertEqual(t.grid_location(201, 101), (1, 1)) + + def test_grid_propagate(self): + self.assertEqual(self.root.grid_propagate(), True) + with self.assertRaises(TypeError): + self.root.grid_propagate(False, False) + self.root.grid_propagate(False) + self.assertFalse(self.root.grid_propagate()) + f = tkinter.Frame(self.root, width=100, height=100, bg='red') + f.grid_configure(row=0, column=0) + self.root.update() + self.assertEqual(f.winfo_width(), 100) + self.assertEqual(f.winfo_height(), 100) + f.grid_propagate(False) + g = tkinter.Frame(self.root, width=75, height=85, bg='green') + g.grid_configure(in_=f, row=0, column=0) + self.root.update() + self.assertEqual(f.winfo_width(), 100) + self.assertEqual(f.winfo_height(), 100) + f.grid_propagate(True) + self.root.update() + self.assertEqual(f.winfo_width(), 75) + self.assertEqual(f.winfo_height(), 85) + + def test_grid_size(self): + with self.assertRaises(TypeError): + self.root.grid_size(0) + self.assertEqual(self.root.grid_size(), (0, 0)) + f = tkinter.Scale(self.root) + f.grid_configure(row=0, column=0) + self.assertEqual(self.root.grid_size(), (1, 1)) + f.grid_configure(row=4, column=5) + self.assertEqual(self.root.grid_size(), (6, 5)) + + def test_grid_slaves(self): + self.assertEqual(self.root.grid_slaves(), []) + a = tkinter.Label(self.root) + a.grid_configure(row=0, column=1) + b = tkinter.Label(self.root) + b.grid_configure(row=1, column=0) + c = tkinter.Label(self.root) + c.grid_configure(row=1, column=1) + d = tkinter.Label(self.root) + d.grid_configure(row=1, column=1) + self.assertEqual(self.root.grid_slaves(), [d, c, b, a]) + self.assertEqual(self.root.grid_slaves(row=0), [a]) + self.assertEqual(self.root.grid_slaves(row=1), [d, c, b]) + self.assertEqual(self.root.grid_slaves(column=0), [b]) + self.assertEqual(self.root.grid_slaves(column=1), [d, c, a]) + self.assertEqual(self.root.grid_slaves(row=1, column=1), [d, c]) + + +tests_gui = ( + PackTest, PlaceTest, GridTest, +) + +if __name__ == '__main__': + unittest.main() diff --git a/Lib/tkinter/test/widget_tests.py b/Lib/tkinter/test/widget_tests.py --- a/Lib/tkinter/test/widget_tests.py +++ b/Lib/tkinter/test/widget_tests.py @@ -33,6 +33,17 @@ if not self.root.wantobjects(): self.wantobjects = False + def tearDown(self): + for w in self.root.winfo_children(): + w.destroy() + + def _str(self, value): + if self.wantobjects and tcl_version >= (8, 6): + return value + if isinstance(value, tuple): + return ' '.join(map(self._str, value)) + return str(value) + def create(self, **kwargs): widget = self._create(**kwargs) self.addCleanup(widget.destroy) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -154,6 +154,8 @@ Tests ----- +- Issue #20635: Added tests for Tk geometry managers. + - Add test case for freeze. - Issue #20743: Fix a reference leak in test_tcl. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 13 18:55:29 2014 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 13 Apr 2014 18:55:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320635=3A_Added_tests_for_Tk_geometry_managers?= =?utf-8?q?=2E?= Message-ID: <3g6JxF3wBBz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/e8acef4f8567 changeset: 90230:e8acef4f8567 parent: 90228:f86504da2fcc parent: 90229:e8c184d8407d user: Serhiy Storchaka date: Sun Apr 13 19:55:08 2014 +0300 summary: Issue #20635: Added tests for Tk geometry managers. files: Lib/tkinter/test/test_tkinter/test_geometry_managers.py | 902 ++++++++++ Lib/tkinter/test/widget_tests.py | 11 + Misc/NEWS | 2 + 3 files changed, 915 insertions(+), 0 deletions(-) diff --git a/Lib/tkinter/test/test_tkinter/test_geometry_managers.py b/Lib/tkinter/test/test_tkinter/test_geometry_managers.py new file mode 100644 --- /dev/null +++ b/Lib/tkinter/test/test_tkinter/test_geometry_managers.py @@ -0,0 +1,902 @@ +import unittest +import re +import tkinter +from tkinter import TclError +from test.support import requires + +from tkinter.test.support import pixels_conv, tcl_version, requires_tcl +from tkinter.test.widget_tests import AbstractWidgetTest + +requires('gui') + + +class PackTest(AbstractWidgetTest, unittest.TestCase): + + def create2(self): + pack = tkinter.Toplevel(self.root, name='pack') + pack.wm_geometry('300x200+0+0') + pack.wm_minsize(1, 1) + a = tkinter.Frame(pack, name='a', width=20, height=40, bg='red') + b = tkinter.Frame(pack, name='b', width=50, height=30, bg='blue') + c = tkinter.Frame(pack, name='c', width=80, height=80, bg='green') + d = tkinter.Frame(pack, name='d', width=40, height=30, bg='yellow') + return pack, a, b, c, d + + def test_pack_configure_after(self): + pack, a, b, c, d = self.create2() + with self.assertRaisesRegex(TclError, 'window "%s" isn\'t packed' % b): + a.pack_configure(after=b) + with self.assertRaisesRegex(TclError, 'bad window path name ".foo"'): + a.pack_configure(after='.foo') + a.pack_configure(side='top') + b.pack_configure(side='top') + c.pack_configure(side='top') + d.pack_configure(side='top') + self.assertEqual(pack.pack_slaves(), [a, b, c, d]) + a.pack_configure(after=b) + self.assertEqual(pack.pack_slaves(), [b, a, c, d]) + a.pack_configure(after=a) + self.assertEqual(pack.pack_slaves(), [b, a, c, d]) + + def test_pack_configure_anchor(self): + pack, a, b, c, d = self.create2() + def check(anchor, geom): + a.pack_configure(side='top', ipadx=5, padx=10, ipady=15, pady=20, + expand=True, anchor=anchor) + self.root.update() + self.assertEqual(a.winfo_geometry(), geom) + check('n', '30x70+135+20') + check('ne', '30x70+260+20') + check('e', '30x70+260+65') + check('se', '30x70+260+110') + check('s', '30x70+135+110') + check('sw', '30x70+10+110') + check('w', '30x70+10+65') + check('nw', '30x70+10+20') + check('center', '30x70+135+65') + + def test_pack_configure_before(self): + pack, a, b, c, d = self.create2() + with self.assertRaisesRegex(TclError, 'window "%s" isn\'t packed' % b): + a.pack_configure(before=b) + with self.assertRaisesRegex(TclError, 'bad window path name ".foo"'): + a.pack_configure(before='.foo') + a.pack_configure(side='top') + b.pack_configure(side='top') + c.pack_configure(side='top') + d.pack_configure(side='top') + self.assertEqual(pack.pack_slaves(), [a, b, c, d]) + a.pack_configure(before=d) + self.assertEqual(pack.pack_slaves(), [b, c, a, d]) + a.pack_configure(before=a) + self.assertEqual(pack.pack_slaves(), [b, c, a, d]) + + def test_pack_configure_expand(self): + pack, a, b, c, d = self.create2() + def check(*geoms): + self.root.update() + self.assertEqual(a.winfo_geometry(), geoms[0]) + self.assertEqual(b.winfo_geometry(), geoms[1]) + self.assertEqual(c.winfo_geometry(), geoms[2]) + self.assertEqual(d.winfo_geometry(), geoms[3]) + a.pack_configure(side='left') + b.pack_configure(side='top') + c.pack_configure(side='right') + d.pack_configure(side='bottom') + check('20x40+0+80', '50x30+135+0', '80x80+220+75', '40x30+100+170') + a.pack_configure(side='left', expand='yes') + b.pack_configure(side='top', expand='on') + c.pack_configure(side='right', expand=True) + d.pack_configure(side='bottom', expand=1) + check('20x40+40+80', '50x30+175+35', '80x80+180+110', '40x30+100+135') + a.pack_configure(side='left', expand='yes', fill='both') + b.pack_configure(side='top', expand='on', fill='both') + c.pack_configure(side='right', expand=True, fill='both') + d.pack_configure(side='bottom', expand=1, fill='both') + check('100x200+0+0', '200x100+100+0', '160x100+140+100', '40x100+100+100') + + def test_pack_configure_in(self): + pack, a, b, c, d = self.create2() + a.pack_configure(side='top') + b.pack_configure(side='top') + c.pack_configure(side='top') + d.pack_configure(side='top') + a.pack_configure(in_=pack) + self.assertEqual(pack.pack_slaves(), [b, c, d, a]) + a.pack_configure(in_=c) + self.assertEqual(pack.pack_slaves(), [b, c, d]) + self.assertEqual(c.pack_slaves(), [a]) + with self.assertRaisesRegex(TclError, + 'can\'t pack %s inside itself' % (a,)): + a.pack_configure(in_=a) + with self.assertRaisesRegex(TclError, 'bad window path name ".foo"'): + a.pack_configure(in_='.foo') + + def test_pack_configure_padx_ipadx_fill(self): + pack, a, b, c, d = self.create2() + def check(geom1, geom2, **kwargs): + a.pack_forget() + b.pack_forget() + a.pack_configure(**kwargs) + b.pack_configure(expand=True, fill='both') + self.root.update() + self.assertEqual(a.winfo_geometry(), geom1) + self.assertEqual(b.winfo_geometry(), geom2) + check('20x40+260+80', '240x200+0+0', side='right', padx=20) + check('20x40+250+80', '240x200+0+0', side='right', padx=(10, 30)) + check('60x40+240+80', '240x200+0+0', side='right', ipadx=20) + check('30x40+260+80', '250x200+0+0', side='right', ipadx=5, padx=10) + check('20x40+260+80', '240x200+0+0', side='right', padx=20, fill='x') + check('20x40+249+80', '240x200+0+0', + side='right', padx=(9, 31), fill='x') + check('60x40+240+80', '240x200+0+0', side='right', ipadx=20, fill='x') + check('30x40+260+80', '250x200+0+0', + side='right', ipadx=5, padx=10, fill='x') + check('30x40+255+80', '250x200+0+0', + side='right', ipadx=5, padx=(5, 15), fill='x') + check('20x40+140+0', '300x160+0+40', side='top', padx=20) + check('20x40+120+0', '300x160+0+40', side='top', padx=(0, 40)) + check('60x40+120+0', '300x160+0+40', side='top', ipadx=20) + check('30x40+135+0', '300x160+0+40', side='top', ipadx=5, padx=10) + check('30x40+130+0', '300x160+0+40', side='top', ipadx=5, padx=(5, 15)) + check('260x40+20+0', '300x160+0+40', side='top', padx=20, fill='x') + check('260x40+25+0', '300x160+0+40', + side='top', padx=(25, 15), fill='x') + check('300x40+0+0', '300x160+0+40', side='top', ipadx=20, fill='x') + check('280x40+10+0', '300x160+0+40', + side='top', ipadx=5, padx=10, fill='x') + check('280x40+5+0', '300x160+0+40', + side='top', ipadx=5, padx=(5, 15), fill='x') + a.pack_configure(padx='1c') + self.assertEqual(a.pack_info()['padx'], + self._str(pack.winfo_pixels('1c'))) + a.pack_configure(ipadx='1c') + self.assertEqual(a.pack_info()['ipadx'], + self._str(pack.winfo_pixels('1c'))) + + def test_pack_configure_pady_ipady_fill(self): + pack, a, b, c, d = self.create2() + def check(geom1, geom2, **kwargs): + a.pack_forget() + b.pack_forget() + a.pack_configure(**kwargs) + b.pack_configure(expand=True, fill='both') + self.root.update() + self.assertEqual(a.winfo_geometry(), geom1) + self.assertEqual(b.winfo_geometry(), geom2) + check('20x40+280+80', '280x200+0+0', side='right', pady=20) + check('20x40+280+70', '280x200+0+0', side='right', pady=(10, 30)) + check('20x80+280+60', '280x200+0+0', side='right', ipady=20) + check('20x50+280+75', '280x200+0+0', side='right', ipady=5, pady=10) + check('20x40+280+80', '280x200+0+0', side='right', pady=20, fill='x') + check('20x40+280+69', '280x200+0+0', + side='right', pady=(9, 31), fill='x') + check('20x80+280+60', '280x200+0+0', side='right', ipady=20, fill='x') + check('20x50+280+75', '280x200+0+0', + side='right', ipady=5, pady=10, fill='x') + check('20x50+280+70', '280x200+0+0', + side='right', ipady=5, pady=(5, 15), fill='x') + check('20x40+140+20', '300x120+0+80', side='top', pady=20) + check('20x40+140+0', '300x120+0+80', side='top', pady=(0, 40)) + check('20x80+140+0', '300x120+0+80', side='top', ipady=20) + check('20x50+140+10', '300x130+0+70', side='top', ipady=5, pady=10) + check('20x50+140+5', '300x130+0+70', side='top', ipady=5, pady=(5, 15)) + check('300x40+0+20', '300x120+0+80', side='top', pady=20, fill='x') + check('300x40+0+25', '300x120+0+80', + side='top', pady=(25, 15), fill='x') + check('300x80+0+0', '300x120+0+80', side='top', ipady=20, fill='x') + check('300x50+0+10', '300x130+0+70', + side='top', ipady=5, pady=10, fill='x') + check('300x50+0+5', '300x130+0+70', + side='top', ipady=5, pady=(5, 15), fill='x') + a.pack_configure(pady='1c') + self.assertEqual(a.pack_info()['pady'], + self._str(pack.winfo_pixels('1c'))) + a.pack_configure(ipady='1c') + self.assertEqual(a.pack_info()['ipady'], + self._str(pack.winfo_pixels('1c'))) + + def test_pack_configure_side(self): + pack, a, b, c, d = self.create2() + def check(side, geom1, geom2): + a.pack_configure(side=side) + self.assertEqual(a.pack_info()['side'], side) + b.pack_configure(expand=True, fill='both') + self.root.update() + self.assertEqual(a.winfo_geometry(), geom1) + self.assertEqual(b.winfo_geometry(), geom2) + check('top', '20x40+140+0', '300x160+0+40') + check('bottom', '20x40+140+160', '300x160+0+0') + check('left', '20x40+0+80', '280x200+20+0') + check('right', '20x40+280+80', '280x200+0+0') + + def test_pack_forget(self): + pack, a, b, c, d = self.create2() + a.pack_configure() + b.pack_configure() + c.pack_configure() + self.assertEqual(pack.pack_slaves(), [a, b, c]) + b.pack_forget() + self.assertEqual(pack.pack_slaves(), [a, c]) + b.pack_forget() + self.assertEqual(pack.pack_slaves(), [a, c]) + d.pack_forget() + + def test_pack_info(self): + pack, a, b, c, d = self.create2() + with self.assertRaisesRegex(TclError, 'window "%s" isn\'t packed' % a): + a.pack_info() + a.pack_configure() + b.pack_configure(side='right', in_=a, anchor='s', expand=True, fill='x', + ipadx=5, padx=10, ipady=2, pady=(5, 15)) + info = a.pack_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['anchor'], 'center') + self.assertEqual(info['expand'], self._str(0)) + self.assertEqual(info['fill'], 'none') + self.assertEqual(info['in'], pack) + self.assertEqual(info['ipadx'], self._str(0)) + self.assertEqual(info['ipady'], self._str(0)) + self.assertEqual(info['padx'], self._str(0)) + self.assertEqual(info['pady'], self._str(0)) + self.assertEqual(info['side'], 'top') + info = b.pack_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['anchor'], 's') + self.assertEqual(info['expand'], self._str(1)) + self.assertEqual(info['fill'], 'x') + self.assertEqual(info['in'], a) + self.assertEqual(info['ipadx'], self._str(5)) + self.assertEqual(info['ipady'], self._str(2)) + self.assertEqual(info['padx'], self._str(10)) + self.assertEqual(info['pady'], self._str((5, 15))) + self.assertEqual(info['side'], 'right') + + def test_pack_propagate(self): + pack, a, b, c, d = self.create2() + pack.configure(width=300, height=200) + a.pack_configure() + pack.pack_propagate(False) + self.root.update() + self.assertEqual(pack.winfo_reqwidth(), 300) + self.assertEqual(pack.winfo_reqheight(), 200) + pack.pack_propagate(True) + self.root.update() + self.assertEqual(pack.winfo_reqwidth(), 20) + self.assertEqual(pack.winfo_reqheight(), 40) + + def test_pack_slaves(self): + pack, a, b, c, d = self.create2() + self.assertEqual(pack.pack_slaves(), []) + a.pack_configure() + self.assertEqual(pack.pack_slaves(), [a]) + b.pack_configure() + self.assertEqual(pack.pack_slaves(), [a, b]) + + +class PlaceTest(AbstractWidgetTest, unittest.TestCase): + + def create2(self): + t = tkinter.Toplevel(self.root, width=300, height=200, bd=0) + t.wm_geometry('+0+0') + f = tkinter.Frame(t, width=154, height=84, bd=2, relief='raised') + f.place_configure(x=48, y=38) + f2 = tkinter.Frame(t, width=30, height=60, bd=2, relief='raised') + self.root.update() + return t, f, f2 + + def test_place_configure_in(self): + t, f, f2 = self.create2() + self.assertEqual(f2.winfo_manager(), '') + with self.assertRaisesRegex(TclError, "can't place %s relative to " + "itself" % re.escape(str(f2))): + f2.place_configure(in_=f2) + if tcl_version >= (8, 5): + self.assertEqual(f2.winfo_manager(), '') + with self.assertRaisesRegex(TclError, 'bad window path name'): + f2.place_configure(in_='spam') + f2.place_configure(in_=f) + self.assertEqual(f2.winfo_manager(), 'place') + + def test_place_configure_x(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['x'], '0') + self.root.update() + self.assertEqual(f2.winfo_x(), 50) + f2.place_configure(x=100) + self.assertEqual(f2.place_info()['x'], '100') + self.root.update() + self.assertEqual(f2.winfo_x(), 150) + f2.place_configure(x=-10, relx=1) + self.assertEqual(f2.place_info()['x'], '-10') + self.root.update() + self.assertEqual(f2.winfo_x(), 190) + with self.assertRaisesRegex(TclError, 'bad screen distance "spam"'): + f2.place_configure(in_=f, x='spam') + + def test_place_configure_y(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['y'], '0') + self.root.update() + self.assertEqual(f2.winfo_y(), 40) + f2.place_configure(y=50) + self.assertEqual(f2.place_info()['y'], '50') + self.root.update() + self.assertEqual(f2.winfo_y(), 90) + f2.place_configure(y=-10, rely=1) + self.assertEqual(f2.place_info()['y'], '-10') + self.root.update() + self.assertEqual(f2.winfo_y(), 110) + with self.assertRaisesRegex(TclError, 'bad screen distance "spam"'): + f2.place_configure(in_=f, y='spam') + + def test_place_configure_relx(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['relx'], '0') + self.root.update() + self.assertEqual(f2.winfo_x(), 50) + f2.place_configure(relx=0.5) + self.assertEqual(f2.place_info()['relx'], '0.5') + self.root.update() + self.assertEqual(f2.winfo_x(), 125) + f2.place_configure(relx=1) + self.assertEqual(f2.place_info()['relx'], '1') + self.root.update() + self.assertEqual(f2.winfo_x(), 200) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "spam"'): + f2.place_configure(in_=f, relx='spam') + + def test_place_configure_rely(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f) + self.assertEqual(f2.place_info()['rely'], '0') + self.root.update() + self.assertEqual(f2.winfo_y(), 40) + f2.place_configure(rely=0.5) + self.assertEqual(f2.place_info()['rely'], '0.5') + self.root.update() + self.assertEqual(f2.winfo_y(), 80) + f2.place_configure(rely=1) + self.assertEqual(f2.place_info()['rely'], '1') + self.root.update() + self.assertEqual(f2.winfo_y(), 120) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "spam"'): + f2.place_configure(in_=f, rely='spam') + + def test_place_configure_anchor(self): + f = tkinter.Frame(self.root) + with self.assertRaisesRegex(TclError, 'bad anchor "j"'): + f.place_configure(anchor='j') + with self.assertRaisesRegex(TclError, 'ambiguous anchor ""'): + f.place_configure(anchor='') + for value in 'n', 'ne', 'e', 'se', 's', 'sw', 'w', 'nw', 'center': + f.place_configure(anchor=value) + self.assertEqual(f.place_info()['anchor'], value) + + def test_place_configure_width(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, width=120) + self.root.update() + self.assertEqual(f2.winfo_width(), 120) + f2.place_configure(width='') + self.root.update() + self.assertEqual(f2.winfo_width(), 30) + with self.assertRaisesRegex(TclError, 'bad screen distance "abcd"'): + f2.place_configure(width='abcd') + + def test_place_configure_height(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, height=120) + self.root.update() + self.assertEqual(f2.winfo_height(), 120) + f2.place_configure(height='') + self.root.update() + self.assertEqual(f2.winfo_height(), 60) + with self.assertRaisesRegex(TclError, 'bad screen distance "abcd"'): + f2.place_configure(height='abcd') + + def test_place_configure_relwidth(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, relwidth=0.5) + self.root.update() + self.assertEqual(f2.winfo_width(), 75) + f2.place_configure(relwidth='') + self.root.update() + self.assertEqual(f2.winfo_width(), 30) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "abcd"'): + f2.place_configure(relwidth='abcd') + + def test_place_configure_relheight(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, relheight=0.5) + self.root.update() + self.assertEqual(f2.winfo_height(), 40) + f2.place_configure(relheight='') + self.root.update() + self.assertEqual(f2.winfo_height(), 60) + with self.assertRaisesRegex(TclError, 'expected floating-point number ' + 'but got "abcd"'): + f2.place_configure(relheight='abcd') + + def test_place_configure_bordermode(self): + f = tkinter.Frame(self.root) + with self.assertRaisesRegex(TclError, 'bad bordermode "j"'): + f.place_configure(bordermode='j') + with self.assertRaisesRegex(TclError, 'ambiguous bordermode ""'): + f.place_configure(bordermode='') + for value in 'inside', 'outside', 'ignore': + f.place_configure(bordermode=value) + self.assertEqual(f.place_info()['bordermode'], value) + + def test_place_forget(self): + foo = tkinter.Frame(self.root) + foo.place_configure(width=50, height=50) + self.root.update() + foo.place_forget() + self.root.update() + self.assertFalse(foo.winfo_ismapped()) + with self.assertRaises(TypeError): + foo.place_forget(0) + + def test_place_info(self): + t, f, f2 = self.create2() + f2.place_configure(in_=f, x=1, y=2, width=3, height=4, + relx=0.1, rely=0.2, relwidth=0.3, relheight=0.4, + anchor='se', bordermode='outside') + info = f2.place_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['x'], '1') + self.assertEqual(info['y'], '2') + self.assertEqual(info['width'], '3') + self.assertEqual(info['height'], '4') + self.assertEqual(info['relx'], '0.1') + self.assertEqual(info['rely'], '0.2') + self.assertEqual(info['relwidth'], '0.3') + self.assertEqual(info['relheight'], '0.4') + self.assertEqual(info['anchor'], 'se') + self.assertEqual(info['bordermode'], 'outside') + self.assertEqual(info['x'], '1') + self.assertEqual(info['x'], '1') + with self.assertRaises(TypeError): + f2.place_info(0) + + def test_place_slaves(self): + foo = tkinter.Frame(self.root) + bar = tkinter.Frame(self.root) + self.assertEqual(foo.place_slaves(), []) + bar.place_configure(in_=foo) + self.assertEqual(foo.place_slaves(), [bar]) + with self.assertRaises(TypeError): + foo.place_slaves(0) + + +class GridTest(AbstractWidgetTest, unittest.TestCase): + + def tearDown(self): + super().tearDown() + cols, rows = self.root.grid_size() + for i in range(cols + 1): + self.root.grid_columnconfigure(i, weight=0, minsize=0, pad=0, uniform='') + for i in range(rows + 1): + self.root.grid_rowconfigure(i, weight=0, minsize=0, pad=0, uniform='') + self.root.grid_propagate(1) + if tcl_version >= (8, 5): + self.root.grid_anchor('nw') + + def test_grid_configure(self): + b = tkinter.Button(self.root) + self.addCleanup(b.destroy) + self.assertEqual(b.grid_info(), {}) + b.grid_configure() + self.assertEqual(b.grid_info()['in'], self.root) + self.assertEqual(b.grid_info()['column'], self._str(0)) + self.assertEqual(b.grid_info()['row'], self._str(0)) + b.grid_configure({'column': 1}, row=2) + self.assertEqual(b.grid_info()['column'], self._str(1)) + self.assertEqual(b.grid_info()['row'], self._str(2)) + + def test_grid_configure_column(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad column value "-1": ' + 'must be a non-negative integer'): + b.grid_configure(column=-1) + b.grid_configure(column=2) + self.assertEqual(b.grid_info()['column'], self._str(2)) + + def test_grid_configure_columnspan(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad columnspan value "0": ' + 'must be a positive integer'): + b.grid_configure(columnspan=0) + b.grid_configure(columnspan=2) + self.assertEqual(b.grid_info()['columnspan'], self._str(2)) + + def test_grid_configure_in(self): + f = tkinter.Frame(self.root) + b = tkinter.Button(self.root) + self.assertEqual(b.grid_info(), {}) + b.grid_configure() + self.assertEqual(b.grid_info()['in'], self.root) + b.grid_configure(in_=f) + self.assertEqual(b.grid_info()['in'], f) + b.grid_configure({'in': self.root}) + self.assertEqual(b.grid_info()['in'], self.root) + + def test_grid_configure_ipadx(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad ipadx value "-1": ' + 'must be positive screen distance'): + b.grid_configure(ipadx=-1) + b.grid_configure(ipadx=1) + self.assertEqual(b.grid_info()['ipadx'], self._str(1)) + b.grid_configure(ipadx='.5c') + self.assertEqual(b.grid_info()['ipadx'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_ipady(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad ipady value "-1": ' + 'must be positive screen distance'): + b.grid_configure(ipady=-1) + b.grid_configure(ipady=1) + self.assertEqual(b.grid_info()['ipady'], self._str(1)) + b.grid_configure(ipady='.5c') + self.assertEqual(b.grid_info()['ipady'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_padx(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad pad value "-1": ' + 'must be positive screen distance'): + b.grid_configure(padx=-1) + b.grid_configure(padx=1) + self.assertEqual(b.grid_info()['padx'], self._str(1)) + b.grid_configure(padx=(10, 5)) + self.assertEqual(b.grid_info()['padx'], self._str((10, 5))) + b.grid_configure(padx='.5c') + self.assertEqual(b.grid_info()['padx'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_pady(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad pad value "-1": ' + 'must be positive screen distance'): + b.grid_configure(pady=-1) + b.grid_configure(pady=1) + self.assertEqual(b.grid_info()['pady'], self._str(1)) + b.grid_configure(pady=(10, 5)) + self.assertEqual(b.grid_info()['pady'], self._str((10, 5))) + b.grid_configure(pady='.5c') + self.assertEqual(b.grid_info()['pady'], + self._str(round(pixels_conv('.5c') * self.scaling))) + + def test_grid_configure_row(self): + b = tkinter.Button(self.root) + self.addCleanup(b.destroy) + with self.assertRaisesRegex(TclError, 'bad (row|grid) value "-1": ' + 'must be a non-negative integer'): + b.grid_configure(row=-1) + b.grid_configure(row=2) + self.assertEqual(b.grid_info()['row'], self._str(2)) + + def test_grid_configure_rownspan(self): + b = tkinter.Button(self.root) + with self.assertRaisesRegex(TclError, 'bad rowspan value "0": ' + 'must be a positive integer'): + b.grid_configure(rowspan=0) + b.grid_configure(rowspan=2) + self.assertEqual(b.grid_info()['rowspan'], self._str(2)) + + def test_grid_configure_sticky(self): + f = tkinter.Frame(self.root, bg='red') + with self.assertRaisesRegex(TclError, 'bad stickyness value "glue"'): + f.grid_configure(sticky='glue') + f.grid_configure(sticky='ne') + self.assertEqual(f.grid_info()['sticky'], 'ne') + f.grid_configure(sticky='n,s,e,w') + self.assertEqual(f.grid_info()['sticky'], 'nesw') + + def test_grid_columnconfigure(self): + with self.assertRaises(TypeError): + self.root.grid_columnconfigure() + self.assertEqual(self.root.grid_columnconfigure(0), + {'minsize': 0, 'pad': 0, 'uniform': None, 'weight': 0}) + with self.assertRaisesRegex(TclError, 'bad option "-foo"'): + self.root.grid_columnconfigure(0, 'foo') + self.root.grid_columnconfigure((0, 3), weight=2) + with self.assertRaisesRegex(TclError, + 'must specify a single element on retrieval'): + self.root.grid_columnconfigure((0, 3)) + b = tkinter.Button(self.root) + b.grid_configure(column=0, row=0) + if tcl_version >= (8, 5): + self.root.grid_columnconfigure('all', weight=3) + with self.assertRaisesRegex(TclError, 'expected integer but got "all"'): + self.root.grid_columnconfigure('all') + self.assertEqual(self.root.grid_columnconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_columnconfigure(3, 'weight'), 2) + self.assertEqual(self.root.grid_columnconfigure(265, 'weight'), 0) + if tcl_version >= (8, 5): + self.root.grid_columnconfigure(b, weight=4) + self.assertEqual(self.root.grid_columnconfigure(0, 'weight'), 4) + + def test_grid_columnconfigure_minsize(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_columnconfigure(0, minsize='foo') + self.root.grid_columnconfigure(0, minsize=10) + self.assertEqual(self.root.grid_columnconfigure(0, 'minsize'), 10) + self.assertEqual(self.root.grid_columnconfigure(0)['minsize'], 10) + + def test_grid_columnconfigure_weight(self): + with self.assertRaisesRegex(TclError, 'expected integer but got "bad"'): + self.root.grid_columnconfigure(0, weight='bad') + with self.assertRaisesRegex(TclError, 'invalid arg "-weight": ' + 'should be non-negative'): + self.root.grid_columnconfigure(0, weight=-3) + self.root.grid_columnconfigure(0, weight=3) + self.assertEqual(self.root.grid_columnconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_columnconfigure(0)['weight'], 3) + + def test_grid_columnconfigure_pad(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_columnconfigure(0, pad='foo') + with self.assertRaisesRegex(TclError, 'invalid arg "-pad": ' + 'should be non-negative'): + self.root.grid_columnconfigure(0, pad=-3) + self.root.grid_columnconfigure(0, pad=3) + self.assertEqual(self.root.grid_columnconfigure(0, 'pad'), 3) + self.assertEqual(self.root.grid_columnconfigure(0)['pad'], 3) + + def test_grid_columnconfigure_uniform(self): + self.root.grid_columnconfigure(0, uniform='foo') + self.assertEqual(self.root.grid_columnconfigure(0, 'uniform'), 'foo') + self.assertEqual(self.root.grid_columnconfigure(0)['uniform'], 'foo') + + def test_grid_rowconfigure(self): + with self.assertRaises(TypeError): + self.root.grid_rowconfigure() + self.assertEqual(self.root.grid_rowconfigure(0), + {'minsize': 0, 'pad': 0, 'uniform': None, 'weight': 0}) + with self.assertRaisesRegex(TclError, 'bad option "-foo"'): + self.root.grid_rowconfigure(0, 'foo') + self.root.grid_rowconfigure((0, 3), weight=2) + with self.assertRaisesRegex(TclError, + 'must specify a single element on retrieval'): + self.root.grid_rowconfigure((0, 3)) + b = tkinter.Button(self.root) + b.grid_configure(column=0, row=0) + if tcl_version >= (8, 5): + self.root.grid_rowconfigure('all', weight=3) + with self.assertRaisesRegex(TclError, 'expected integer but got "all"'): + self.root.grid_rowconfigure('all') + self.assertEqual(self.root.grid_rowconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_rowconfigure(3, 'weight'), 2) + self.assertEqual(self.root.grid_rowconfigure(265, 'weight'), 0) + if tcl_version >= (8, 5): + self.root.grid_rowconfigure(b, weight=4) + self.assertEqual(self.root.grid_rowconfigure(0, 'weight'), 4) + + def test_grid_rowconfigure_minsize(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_rowconfigure(0, minsize='foo') + self.root.grid_rowconfigure(0, minsize=10) + self.assertEqual(self.root.grid_rowconfigure(0, 'minsize'), 10) + self.assertEqual(self.root.grid_rowconfigure(0)['minsize'], 10) + + def test_grid_rowconfigure_weight(self): + with self.assertRaisesRegex(TclError, 'expected integer but got "bad"'): + self.root.grid_rowconfigure(0, weight='bad') + with self.assertRaisesRegex(TclError, 'invalid arg "-weight": ' + 'should be non-negative'): + self.root.grid_rowconfigure(0, weight=-3) + self.root.grid_rowconfigure(0, weight=3) + self.assertEqual(self.root.grid_rowconfigure(0, 'weight'), 3) + self.assertEqual(self.root.grid_rowconfigure(0)['weight'], 3) + + def test_grid_rowconfigure_pad(self): + with self.assertRaisesRegex(TclError, 'bad screen distance "foo"'): + self.root.grid_rowconfigure(0, pad='foo') + with self.assertRaisesRegex(TclError, 'invalid arg "-pad": ' + 'should be non-negative'): + self.root.grid_rowconfigure(0, pad=-3) + self.root.grid_rowconfigure(0, pad=3) + self.assertEqual(self.root.grid_rowconfigure(0, 'pad'), 3) + self.assertEqual(self.root.grid_rowconfigure(0)['pad'], 3) + + def test_grid_rowconfigure_uniform(self): + self.root.grid_rowconfigure(0, uniform='foo') + self.assertEqual(self.root.grid_rowconfigure(0, 'uniform'), 'foo') + self.assertEqual(self.root.grid_rowconfigure(0)['uniform'], 'foo') + + def test_grid_forget(self): + b = tkinter.Button(self.root) + c = tkinter.Button(self.root) + b.grid_configure(row=2, column=2, rowspan=2, columnspan=2, + padx=3, pady=4, sticky='ns') + self.assertEqual(self.root.grid_slaves(), [b]) + b.grid_forget() + c.grid_forget() + self.assertEqual(self.root.grid_slaves(), []) + self.assertEqual(b.grid_info(), {}) + b.grid_configure(row=0, column=0) + info = b.grid_info() + self.assertEqual(info['row'], self._str(0)) + self.assertEqual(info['column'], self._str(0)) + self.assertEqual(info['rowspan'], self._str(1)) + self.assertEqual(info['columnspan'], self._str(1)) + self.assertEqual(info['padx'], self._str(0)) + self.assertEqual(info['pady'], self._str(0)) + self.assertEqual(info['sticky'], '') + + def test_grid_remove(self): + b = tkinter.Button(self.root) + c = tkinter.Button(self.root) + b.grid_configure(row=2, column=2, rowspan=2, columnspan=2, + padx=3, pady=4, sticky='ns') + self.assertEqual(self.root.grid_slaves(), [b]) + b.grid_remove() + c.grid_remove() + self.assertEqual(self.root.grid_slaves(), []) + self.assertEqual(b.grid_info(), {}) + b.grid_configure(row=0, column=0) + info = b.grid_info() + self.assertEqual(info['row'], self._str(0)) + self.assertEqual(info['column'], self._str(0)) + self.assertEqual(info['rowspan'], self._str(2)) + self.assertEqual(info['columnspan'], self._str(2)) + self.assertEqual(info['padx'], self._str(3)) + self.assertEqual(info['pady'], self._str(4)) + self.assertEqual(info['sticky'], 'ns') + + def test_grid_info(self): + b = tkinter.Button(self.root) + self.assertEqual(b.grid_info(), {}) + b.grid_configure(row=2, column=2, rowspan=2, columnspan=2, + padx=3, pady=4, sticky='ns') + info = b.grid_info() + self.assertIsInstance(info, dict) + self.assertEqual(info['in'], self.root) + self.assertEqual(info['row'], self._str(2)) + self.assertEqual(info['column'], self._str(2)) + self.assertEqual(info['rowspan'], self._str(2)) + self.assertEqual(info['columnspan'], self._str(2)) + self.assertEqual(info['padx'], self._str(3)) + self.assertEqual(info['pady'], self._str(4)) + self.assertEqual(info['sticky'], 'ns') + + @requires_tcl(8, 5) + def test_grid_anchor(self): + with self.assertRaisesRegex(TclError, 'bad anchor "x"'): + self.root.grid_anchor('x') + with self.assertRaisesRegex(TclError, 'ambiguous anchor ""'): + self.root.grid_anchor('') + with self.assertRaises(TypeError): + self.root.grid_anchor('se', 'nw') + self.root.grid_anchor('se') + self.assertEqual(self.root.tk.call('grid', 'anchor', self.root), 'se') + + def test_grid_bbox(self): + self.assertEqual(self.root.grid_bbox(), (0, 0, 0, 0)) + self.assertEqual(self.root.grid_bbox(0, 0), (0, 0, 0, 0)) + self.assertEqual(self.root.grid_bbox(0, 0, 1, 1), (0, 0, 0, 0)) + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox('x', 0) + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox(0, 'x') + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox(0, 0, 'x', 0) + with self.assertRaisesRegex(TclError, 'expected integer but got "x"'): + self.root.grid_bbox(0, 0, 0, 'x') + with self.assertRaises(TypeError): + self.root.grid_bbox(0, 0, 0, 0, 0) + t = tkinter.Toplevel(self.root) + # de-maximize + t.wm_geometry('1x1+0+0') + t.wm_geometry('') + f1 = tkinter.Frame(t, width=75, height=75, bg='red') + f2 = tkinter.Frame(t, width=90, height=90, bg='blue') + f1.grid_configure(row=0, column=0) + f2.grid_configure(row=1, column=1) + self.root.update() + self.assertEqual(t.grid_bbox(), (0, 0, 165, 165)) + self.assertEqual(t.grid_bbox(0, 0), (0, 0, 75, 75)) + self.assertEqual(t.grid_bbox(0, 0, 1, 1), (0, 0, 165, 165)) + self.assertEqual(t.grid_bbox(1, 1), (75, 75, 90, 90)) + self.assertEqual(t.grid_bbox(10, 10, 0, 0), (0, 0, 165, 165)) + self.assertEqual(t.grid_bbox(-2, -2, -1, -1), (0, 0, 0, 0)) + self.assertEqual(t.grid_bbox(10, 10, 12, 12), (165, 165, 0, 0)) + + def test_grid_location(self): + with self.assertRaises(TypeError): + self.root.grid_location() + with self.assertRaises(TypeError): + self.root.grid_location(0) + with self.assertRaises(TypeError): + self.root.grid_location(0, 0, 0) + with self.assertRaisesRegex(TclError, 'bad screen distance "x"'): + self.root.grid_location('x', 'y') + with self.assertRaisesRegex(TclError, 'bad screen distance "y"'): + self.root.grid_location('1c', 'y') + t = tkinter.Toplevel(self.root) + # de-maximize + t.wm_geometry('1x1+0+0') + t.wm_geometry('') + f = tkinter.Frame(t, width=200, height=100, + highlightthickness=0, bg='red') + self.assertEqual(f.grid_location(10, 10), (-1, -1)) + f.grid_configure() + self.root.update() + self.assertEqual(t.grid_location(-10, -10), (-1, -1)) + self.assertEqual(t.grid_location(-10, 0), (-1, 0)) + self.assertEqual(t.grid_location(-1, 0), (-1, 0)) + self.assertEqual(t.grid_location(0, -10), (0, -1)) + self.assertEqual(t.grid_location(0, -1), (0, -1)) + self.assertEqual(t.grid_location(0, 0), (0, 0)) + self.assertEqual(t.grid_location(200, 0), (0, 0)) + self.assertEqual(t.grid_location(201, 0), (1, 0)) + self.assertEqual(t.grid_location(0, 100), (0, 0)) + self.assertEqual(t.grid_location(0, 101), (0, 1)) + self.assertEqual(t.grid_location(201, 101), (1, 1)) + + def test_grid_propagate(self): + self.assertEqual(self.root.grid_propagate(), True) + with self.assertRaises(TypeError): + self.root.grid_propagate(False, False) + self.root.grid_propagate(False) + self.assertFalse(self.root.grid_propagate()) + f = tkinter.Frame(self.root, width=100, height=100, bg='red') + f.grid_configure(row=0, column=0) + self.root.update() + self.assertEqual(f.winfo_width(), 100) + self.assertEqual(f.winfo_height(), 100) + f.grid_propagate(False) + g = tkinter.Frame(self.root, width=75, height=85, bg='green') + g.grid_configure(in_=f, row=0, column=0) + self.root.update() + self.assertEqual(f.winfo_width(), 100) + self.assertEqual(f.winfo_height(), 100) + f.grid_propagate(True) + self.root.update() + self.assertEqual(f.winfo_width(), 75) + self.assertEqual(f.winfo_height(), 85) + + def test_grid_size(self): + with self.assertRaises(TypeError): + self.root.grid_size(0) + self.assertEqual(self.root.grid_size(), (0, 0)) + f = tkinter.Scale(self.root) + f.grid_configure(row=0, column=0) + self.assertEqual(self.root.grid_size(), (1, 1)) + f.grid_configure(row=4, column=5) + self.assertEqual(self.root.grid_size(), (6, 5)) + + def test_grid_slaves(self): + self.assertEqual(self.root.grid_slaves(), []) + a = tkinter.Label(self.root) + a.grid_configure(row=0, column=1) + b = tkinter.Label(self.root) + b.grid_configure(row=1, column=0) + c = tkinter.Label(self.root) + c.grid_configure(row=1, column=1) + d = tkinter.Label(self.root) + d.grid_configure(row=1, column=1) + self.assertEqual(self.root.grid_slaves(), [d, c, b, a]) + self.assertEqual(self.root.grid_slaves(row=0), [a]) + self.assertEqual(self.root.grid_slaves(row=1), [d, c, b]) + self.assertEqual(self.root.grid_slaves(column=0), [b]) + self.assertEqual(self.root.grid_slaves(column=1), [d, c, a]) + self.assertEqual(self.root.grid_slaves(row=1, column=1), [d, c]) + + +tests_gui = ( + PackTest, PlaceTest, GridTest, +) + +if __name__ == '__main__': + unittest.main() diff --git a/Lib/tkinter/test/widget_tests.py b/Lib/tkinter/test/widget_tests.py --- a/Lib/tkinter/test/widget_tests.py +++ b/Lib/tkinter/test/widget_tests.py @@ -33,6 +33,17 @@ if not self.root.wantobjects(): self.wantobjects = False + def tearDown(self): + for w in self.root.winfo_children(): + w.destroy() + + def _str(self, value): + if self.wantobjects and tcl_version >= (8, 6): + return value + if isinstance(value, tuple): + return ' '.join(map(self._str, value)) + return str(value) + def create(self, **kwargs): widget = self._create(**kwargs) self.addCleanup(widget.destroy) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -208,6 +208,8 @@ Tests ----- +- Issue #20635: Added tests for Tk geometry managers. + - Add test case for freeze. - Issue #20743: Fix a reference leak in test_tcl. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 13 19:17:07 2014 From: python-checkins at python.org (barry.warsaw) Date: Sun, 13 Apr 2014 19:17:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_reference=2E?= Message-ID: <3g6KQC0V5bz7LjQ@mail.python.org> http://hg.python.org/peps/rev/2fa9f100f856 changeset: 5458:2fa9f100f856 user: Barry Warsaw date: Sun Apr 13 13:17:01 2014 -0400 summary: Add reference. files: pep-0373.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0373.txt b/pep-0373.txt --- a/pep-0373.txt +++ b/pep-0373.txt @@ -31,7 +31,7 @@ contribute bugfixes for Python 2.7 and it should satisfy vendors who still have to support Python 2 for years to come. -There will be no Python 2.8. +There will be no Python 2.8 (see PEP 404). Release Manager and Crew -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Apr 14 01:52:25 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 01:52:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_correct_sphinx_mark_up_for?= =?utf-8?q?_cmdline_options_=28closes_=2321210=29?= Message-ID: <3g6VBK6ztDz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/399bf1638911 changeset: 90231:399bf1638911 user: Benjamin Peterson date: Sun Apr 13 19:52:14 2014 -0400 summary: correct sphinx mark up for cmdline options (closes #21210) files: Doc/library/json.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/library/json.rst b/Doc/library/json.rst --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -590,7 +590,7 @@ Command line options ^^^^^^^^^^^^^^^^^^^^ -.. cmdoption:: [] +.. cmdoption:: infile The JSON file to be validated or pretty-printed:: @@ -606,7 +606,9 @@ } ] -.. cmdoption:: [] + If *infile* is not specified, read from :attr:`sys.stdin`. + +.. cmdoption:: outfile Write the output of the *infile* to the given *outfile*. Otherwise, write it to :attr:`sys.stdout`. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:09:46 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 14 Apr 2014 04:09:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzIxMTY5OiBmaXgg?= =?utf-8?q?getpass_to_use_replace_error_handler_on_UnicodeEncodeError=2E?= Message-ID: <3g6YDp1jYgz7LjR@mail.python.org> http://hg.python.org/cpython/rev/f430fdd1628e changeset: 90232:f430fdd1628e branch: 3.4 parent: 90229:e8c184d8407d user: R David Murray date: Sun Apr 13 22:07:39 2014 -0400 summary: #21169: fix getpass to use replace error handler on UnicodeEncodeError. If the input stream encoding couldn't encode one or more of the non-ascii characters in the prompt, it would fail, throwing a UnicodeEncodeError. Now if that happens we re-encoding using the 'replace' error handler. Patch by Kushal Das. files: Lib/getpass.py | 7 ++++++- Lib/test/test_getpass.py | 10 +++++++++- Misc/NEWS | 4 ++++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/getpass.py b/Lib/getpass.py --- a/Lib/getpass.py +++ b/Lib/getpass.py @@ -135,7 +135,12 @@ input = sys.stdin prompt = str(prompt) if prompt: - stream.write(prompt) + try: + stream.write(prompt) + except UnicodeEncodeError: + prompt = prompt.encode(stream.encoding, 'replace') + prompt = prompt.decode(stream.encoding) + stream.write(prompt) stream.flush() # NOTE: The Python C API calls flockfile() (and unlock) during readline. line = input.readline() diff --git a/Lib/test/test_getpass.py b/Lib/test/test_getpass.py --- a/Lib/test/test_getpass.py +++ b/Lib/test/test_getpass.py @@ -1,7 +1,7 @@ import getpass import os import unittest -from io import BytesIO, StringIO +from io import BytesIO, StringIO, TextIOWrapper from unittest import mock from test import support @@ -69,6 +69,14 @@ getpass._raw_input(stream=StringIO()) mock_input.readline.assert_called_once_with() + @mock.patch('sys.stdin') + def test_uses_stdin_as_different_locale(self, mock_input): + stream = TextIOWrapper(BytesIO(), encoding="ascii") + mock_input.readline.return_value = "Has??o: " + getpass._raw_input(prompt="Has??o: ",stream=stream) + mock_input.readline.assert_called_once_with() + + def test_raises_on_empty_input(self): input = StringIO('') self.assertRaises(EOFError, getpass._raw_input, input=input) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,10 @@ Library ------- +- Issue #21169: getpass now handles non-ascii characters that the + input stream encoding cannot encode by re-encoding using the + replace error handler. + - Issue #21171: Fixed undocumented filter API of the rot13 codec. Patch by Berker Peksag. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:09:47 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 14 Apr 2014 04:09:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Mierge_=2321169=3A_fix_getpass_to_use_replace_error_hand?= =?utf-8?q?ler_on_UnicodeEncodeError=2E?= Message-ID: <3g6YDq3vHSz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/461f5863f2aa changeset: 90233:461f5863f2aa parent: 90231:399bf1638911 parent: 90232:f430fdd1628e user: R David Murray date: Sun Apr 13 22:09:29 2014 -0400 summary: Mierge #21169: fix getpass to use replace error handler on UnicodeEncodeError. files: Lib/getpass.py | 7 ++++++- Lib/test/test_getpass.py | 10 +++++++++- Misc/NEWS | 4 ++++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/getpass.py b/Lib/getpass.py --- a/Lib/getpass.py +++ b/Lib/getpass.py @@ -135,7 +135,12 @@ input = sys.stdin prompt = str(prompt) if prompt: - stream.write(prompt) + try: + stream.write(prompt) + except UnicodeEncodeError: + prompt = prompt.encode(stream.encoding, 'replace') + prompt = prompt.decode(stream.encoding) + stream.write(prompt) stream.flush() # NOTE: The Python C API calls flockfile() (and unlock) during readline. line = input.readline() diff --git a/Lib/test/test_getpass.py b/Lib/test/test_getpass.py --- a/Lib/test/test_getpass.py +++ b/Lib/test/test_getpass.py @@ -1,7 +1,7 @@ import getpass import os import unittest -from io import BytesIO, StringIO +from io import BytesIO, StringIO, TextIOWrapper from unittest import mock from test import support @@ -69,6 +69,14 @@ getpass._raw_input(stream=StringIO()) mock_input.readline.assert_called_once_with() + @mock.patch('sys.stdin') + def test_uses_stdin_as_different_locale(self, mock_input): + stream = TextIOWrapper(BytesIO(), encoding="ascii") + mock_input.readline.return_value = "Has??o: " + getpass._raw_input(prompt="Has??o: ",stream=stream) + mock_input.readline.assert_called_once_with() + + def test_raises_on_empty_input(self): input = StringIO('') self.assertRaises(EOFError, getpass._raw_input, input=input) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,10 @@ Library ------- +- Issue #21169: getpass now handles non-ascii characters that the + input stream encoding cannot encode by re-encoding using the + replace error handler. + - Issue #21171: Fixed undocumented filter API of the rot13 codec. Patch by Berker Peksag. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:32:20 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:32:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogaW4gc2Nhbl9vbmNl?= =?utf-8?q?=2C_prevent_the_reading_of_arbitrary_memory_when_passed_a_negat?= =?utf-8?q?ive?= Message-ID: <3g6Ykr6tdMz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/50c07ed1743d changeset: 90234:50c07ed1743d branch: 2.7 parent: 90224:218e28a935ab user: Benjamin Peterson date: Sun Apr 13 22:10:38 2014 -0400 summary: in scan_once, prevent the reading of arbitrary memory when passed a negative index Bug reported by Guido Vranken. files: Lib/json/tests/test_decode.py | 5 +++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 10 ++++++++-- 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/Lib/json/tests/test_decode.py b/Lib/json/tests/test_decode.py --- a/Lib/json/tests/test_decode.py +++ b/Lib/json/tests/test_decode.py @@ -60,5 +60,10 @@ msg = 'escape' self.assertRaisesRegexp(ValueError, msg, self.loads, s) + def test_negative_index(self): + d = self.json.JSONDecoder() + self.assertRaises(ValueError, d.raw_decode, 'a'*42, -50000) + self.assertRaises(ValueError, d.raw_decode, u'a'*42, -50000) + class TestPyDecode(TestDecode, PyTest): pass class TestCDecode(TestDecode, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1369,6 +1369,7 @@ Frank Visser Johannes Vogel Alex Volkov +Guido Vranken Martijn Vries Sjoerd de Vries Niki W. Waibel diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,9 @@ Library ------- +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. + - Issue #21172: isinstance check relaxed from dict to collections.Mapping. - Issue #21191: In os.fdopen, alwyas close the file descriptor when an exception diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1468,7 +1468,10 @@ PyObject *res; char *str = PyString_AS_STRING(pystr); Py_ssize_t length = PyString_GET_SIZE(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with the Python version. */ + idx += length; + if (idx < 0 || idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } @@ -1555,7 +1558,10 @@ PyObject *res; Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with Python version. */ + idx += length; + if (idx < 0 || idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:32:22 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:32:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4xKTogaW4gc2Nhbl9vbmNl?= =?utf-8?q?=2C_prevent_the_reading_of_arbitrary_memory_when_passed_a_negat?= =?utf-8?q?ive?= Message-ID: <3g6Ykt1HT1z7LjZ@mail.python.org> http://hg.python.org/cpython/rev/a8facac493ef changeset: 90235:a8facac493ef branch: 3.1 parent: 89608:23add5382fb3 user: Benjamin Peterson date: Sun Apr 13 22:10:38 2014 -0400 summary: in scan_once, prevent the reading of arbitrary memory when passed a negative index Bug reported by Guido Vranken. files: Lib/json/tests/test_decode.py | 4 ++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 5 ++++- 4 files changed, 12 insertions(+), 1 deletions(-) diff --git a/Lib/json/tests/test_decode.py b/Lib/json/tests/test_decode.py --- a/Lib/json/tests/test_decode.py +++ b/Lib/json/tests/test_decode.py @@ -45,5 +45,9 @@ self.assertEqual(rval, {"key":"value", "k":"v"}) + def test_negative_index(self): + d = self.json.JSONDecoder() + self.assertRaises(ValueError, d.raw_decode, 'a'*42, -50000) + class TestPyDecode(TestDecode, PyTest): pass class TestCDecode(TestDecode, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -842,6 +842,7 @@ Kurt Vile Norman Vine Frank Visser +Guido Vranken Niki W. Waibel Wojtek Walczak Charles Waldman diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. + - Issue #20246: Fix buffer overflow in socket.recvfrom_into. - Issue #19435: Fix directory traversal attack on CGIHttpRequestHandler. diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -902,7 +902,10 @@ PyObject *res; Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with Python version. */ + idx += length; + if (idx < 0 || idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:32:23 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:32:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4yKTogaW4gc2Nhbl9vbmNl?= =?utf-8?q?=2C_prevent_the_reading_of_arbitrary_memory_when_passed_a_negat?= =?utf-8?q?ive?= Message-ID: <3g6Ykv2nkKz7Lks@mail.python.org> http://hg.python.org/cpython/rev/8130b8c06606 changeset: 90236:8130b8c06606 branch: 3.2 parent: 90129:b9913eb96643 user: Benjamin Peterson date: Sun Apr 13 22:10:38 2014 -0400 summary: in scan_once, prevent the reading of arbitrary memory when passed a negative index Bug reported by Guido Vranken. files: Lib/test/json_tests/test_decode.py | 4 ++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 5 ++++- 4 files changed, 12 insertions(+), 1 deletions(-) diff --git a/Lib/test/json_tests/test_decode.py b/Lib/test/json_tests/test_decode.py --- a/Lib/test/json_tests/test_decode.py +++ b/Lib/test/json_tests/test_decode.py @@ -70,5 +70,9 @@ msg = 'escape' self.assertRaisesRegex(ValueError, msg, self.loads, s) + def test_negative_index(self): + d = self.json.JSONDecoder() + self.assertRaises(ValueError, d.raw_decode, 'a'*42, -50000) + class TestPyDecode(TestDecode, PyTest): pass class TestCDecode(TestDecode, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1139,6 +1139,7 @@ Johannes Vogel Martijn Vries Sjoerd de Vries +Guido Vranken Niki W. Waibel Wojtek Walczak Charles Waldman diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Library ------- +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. + - Issue #21082: In os.makedirs, do not set the process-wide umask. Note this changes behavior of makedirs when exist_ok=True. diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -930,7 +930,10 @@ PyObject *res; Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with Python version. */ + idx += length; + if (idx < 0 || idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:32:24 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:32:24 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2?= Message-ID: <3g6Ykw4pYBz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/4f15bd1ab28f changeset: 90237:4f15bd1ab28f branch: 3.3 parent: 90160:b49d990aaa9d parent: 90236:8130b8c06606 user: Benjamin Peterson date: Sun Apr 13 22:28:16 2014 -0400 summary: merge 3.2 files: Lib/test/test_json/test_decode.py | 4 ++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 5 ++++- 4 files changed, 12 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_json/test_decode.py b/Lib/test/test_json/test_decode.py --- a/Lib/test/test_json/test_decode.py +++ b/Lib/test/test_json/test_decode.py @@ -70,5 +70,9 @@ msg = 'escape' self.assertRaisesRegex(ValueError, msg, self.loads, s) + def test_negative_index(self): + d = self.json.JSONDecoder() + self.assertRaises(ValueError, d.raw_decode, 'a'*42, -50000) + class TestPyDecode(TestDecode, PyTest): pass class TestCDecode(TestDecode, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1313,6 +1313,7 @@ Alex Volkov Martijn Vries Sjoerd de Vries +Guido Vranken Niki W. Waibel Wojtek Walczak Charles Waldman diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,9 @@ Library ------- +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. + - Issue #20633: Replace relative import by absolute import. - Issue #21082: In os.makedirs, do not set the process-wide umask. Note this diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -975,7 +975,10 @@ kind = PyUnicode_KIND(pystr); length = PyUnicode_GET_LENGTH(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with Python version. */ + idx += length; + if (idx < 0 || idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:32:25 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:32:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <3g6Ykx6hhRz7LlG@mail.python.org> http://hg.python.org/cpython/rev/7b95540ced5c changeset: 90238:7b95540ced5c branch: 3.4 parent: 90232:f430fdd1628e parent: 90237:4f15bd1ab28f user: Benjamin Peterson date: Sun Apr 13 22:31:42 2014 -0400 summary: merge 3.3 files: Lib/test/test_json/test_decode.py | 4 ++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 5 ++++- 4 files changed, 12 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_json/test_decode.py b/Lib/test/test_json/test_decode.py --- a/Lib/test/test_json/test_decode.py +++ b/Lib/test/test_json/test_decode.py @@ -91,5 +91,9 @@ self.assertEqual(self.loads(bom_in_str), '\ufeff') self.assertEqual(self.json.load(StringIO(bom_in_str)), '\ufeff') + def test_negative_index(self): + d = self.json.JSONDecoder() + self.assertRaises(ValueError, d.raw_decode, 'a'*42, -50000) + class TestPyDecode(TestDecode, PyTest): pass class TestCDecode(TestDecode, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1372,6 +1372,7 @@ Alex Volkov Martijn Vries Sjoerd de Vries +Guido Vranken Niki W. Waibel Wojtek Walczak Charles Waldman diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ Library ------- +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. + - Issue #21169: getpass now handles non-ascii characters that the input stream encoding cannot encode by re-encoding using the replace error handler. diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -941,7 +941,10 @@ kind = PyUnicode_KIND(pystr); length = PyUnicode_GET_LENGTH(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with Python version. */ + idx += length; + if (idx < 0 || idx >= length) { raise_stop_iteration(idx); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:32:27 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:32:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g6Ykz1hcFz7LlJ@mail.python.org> http://hg.python.org/cpython/rev/3a414c709f1f changeset: 90239:3a414c709f1f parent: 90233:461f5863f2aa parent: 90238:7b95540ced5c user: Benjamin Peterson date: Sun Apr 13 22:32:12 2014 -0400 summary: merge 3.4 files: Lib/test/test_json/test_decode.py | 4 ++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 5 ++++- 4 files changed, 12 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_json/test_decode.py b/Lib/test/test_json/test_decode.py --- a/Lib/test/test_json/test_decode.py +++ b/Lib/test/test_json/test_decode.py @@ -91,5 +91,9 @@ self.assertEqual(self.loads(bom_in_str), '\ufeff') self.assertEqual(self.json.load(StringIO(bom_in_str)), '\ufeff') + def test_negative_index(self): + d = self.json.JSONDecoder() + self.assertRaises(ValueError, d.raw_decode, 'a'*42, -50000) + class TestPyDecode(TestDecode, PyTest): pass class TestCDecode(TestDecode, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1373,6 +1373,7 @@ Alex Volkov Martijn Vries Sjoerd de Vries +Guido Vranken Niki W. Waibel Wojtek Walczak Charles Waldman diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,9 @@ Library ------- +- Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second + parameter. Bug reported by Guido Vranken. + - Issue #21169: getpass now handles non-ascii characters that the input stream encoding cannot encode by re-encoding using the replace error handler. diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -941,7 +941,10 @@ kind = PyUnicode_KIND(pystr); length = PyUnicode_GET_LENGTH(pystr); - if (idx >= length) { + if (idx < 0) + /* Compatibility with Python version. */ + idx += length; + if (idx < 0 || idx >= length) { raise_stop_iteration(idx); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 04:47:23 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 04:47:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_specific_2=2E7=2E7_dates?= Message-ID: <3g6Z4C5lZfz7Ljv@mail.python.org> http://hg.python.org/peps/rev/c20c62cca734 changeset: 5459:c20c62cca734 user: Benjamin Peterson date: Sun Apr 13 22:47:18 2014 -0400 summary: specific 2.7.7 dates files: pep-0373.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0373.txt b/pep-0373.txt --- a/pep-0373.txt +++ b/pep-0373.txt @@ -71,7 +71,8 @@ Planned future release dates: -- 2.7.7 May 2014 +- 2.7.7 rc1 2014-05-12 +- 2.7.7 final 2014-05-26 - 2.7.8 November 2014 - 2.7.9 May 2015 - beyond this date, releases as needed -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Mon Apr 14 05:52:53 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 05:52:53 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_sending_tu?= =?utf-8?q?ples_to_custom_generator_objects_with_yield_from_=28closes_=232?= =?utf-8?q?1209=29?= Message-ID: <3g6bWn5tzZz7LjS@mail.python.org> http://hg.python.org/cpython/rev/05b3a23b3836 changeset: 90240:05b3a23b3836 branch: 3.4 parent: 90238:7b95540ced5c user: Benjamin Peterson date: Sun Apr 13 23:52:01 2014 -0400 summary: fix sending tuples to custom generator objects with yield from (closes #21209) Debugged by Victor. files: Lib/test/test_pep380.py | 19 +++++++++++++++++++ Misc/NEWS | 3 +++ Python/ceval.c | 2 +- 3 files changed, 23 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_pep380.py b/Lib/test/test_pep380.py --- a/Lib/test/test_pep380.py +++ b/Lib/test/test_pep380.py @@ -993,6 +993,25 @@ del inner_gen gc_collect() + def test_send_tuple_with_custom_generator(self): + # See issue #21209. + class MyGen: + def __iter__(self): + return self + def __next__(self): + return 42 + def send(self, what): + nonlocal v + v = what + return None + def outer(): + v = yield from MyGen() + g = outer() + next(g) + v = None + g.send((1, 2, 3, 4)) + self.assertEqual(v, (1, 2, 3, 4)) + def test_main(): from test import support diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #21209: Fix sending tuples to custom generator objects with the yield + from syntax. + - Issue #21134: Fix segfault when str is called on an uninitialized UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1902,7 +1902,7 @@ if (v == Py_None) retval = Py_TYPE(reciever)->tp_iternext(reciever); else - retval = _PyObject_CallMethodId(reciever, &PyId_send, "O", v); + retval = _PyObject_CallMethodIdObjArgs(reciever, &PyId_send, v, NULL); } Py_DECREF(v); if (retval == NULL) { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 05:52:55 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 05:52:55 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjEyMDkp?= Message-ID: <3g6bWq0g2hz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/d1eba2645b80 changeset: 90241:d1eba2645b80 parent: 90239:3a414c709f1f parent: 90240:05b3a23b3836 user: Benjamin Peterson date: Sun Apr 13 23:52:43 2014 -0400 summary: merge 3.4 (#21209) files: Lib/test/test_pep380.py | 19 +++++++++++++++++++ Misc/NEWS | 3 +++ Python/ceval.c | 2 +- 3 files changed, 23 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_pep380.py b/Lib/test/test_pep380.py --- a/Lib/test/test_pep380.py +++ b/Lib/test/test_pep380.py @@ -993,6 +993,25 @@ del inner_gen gc_collect() + def test_send_tuple_with_custom_generator(self): + # See issue #21209. + class MyGen: + def __iter__(self): + return self + def __next__(self): + return 42 + def send(self, what): + nonlocal v + v = what + return None + def outer(): + v = yield from MyGen() + g = outer() + next(g) + v = None + g.send((1, 2, 3, 4)) + self.assertEqual(v, (1, 2, 3, 4)) + def test_main(): from test import support diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #21209: Fix sending tuples to custom generator objects with the yield + from syntax. + - Issue #21193: pow(a, b, c) now raises ValueError rather than TypeError when b is negative. Patch by Josh Rosenberg. diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1926,7 +1926,7 @@ if (v == Py_None) retval = Py_TYPE(reciever)->tp_iternext(reciever); else - retval = _PyObject_CallMethodId(reciever, &PyId_send, "O", v); + retval = _PyObject_CallMethodIdObjArgs(reciever, &PyId_send, v, NULL); } Py_DECREF(v); if (retval == NULL) { -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Mon Apr 14 09:55:52 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 14 Apr 2014 09:55:52 +0200 Subject: [Python-checkins] Daily reference leaks (399bf1638911): sum=11 Message-ID: results for 399bf1638911 on branch "default" -------------------------------------------- test_asyncio leaked [4, 0, 0] memory blocks, sum=4 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogpqgHJ0', '-x'] From python-checkins at python.org Mon Apr 14 13:46:46 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 13:46:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_text_about?= =?utf-8?q?_int=28=29_with_octal_numbers=2E_Closes_=2321212=2E?= Message-ID: <3g6p2Z3P1wz7LjN@mail.python.org> http://hg.python.org/cpython/rev/fb7bc8fe0d49 changeset: 90242:fb7bc8fe0d49 branch: 3.4 parent: 90240:05b3a23b3836 user: Eric V. Smith date: Mon Apr 14 07:41:52 2014 -0400 summary: Fix text about int() with octal numbers. Closes #21212. files: Doc/faq/programming.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -711,7 +711,7 @@ 144`` and ``int('0x144')`` raises :exc:`ValueError`. ``int(string, base)`` takes the base to convert from as a second optional argument, so ``int('0x144', 16) == 324``. If the base is specified as 0, the number is interpreted using Python's -rules: a leading '0' indicates octal, and '0x' indicates a hex number. +rules: a leading '0o' indicates octal, and '0x' indicates a hex number. Do not use the built-in function :func:`eval` if all you need is to convert strings to numbers. :func:`eval` will be significantly slower and it presents a -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 13:46:47 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 13:46:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E4=3A_Fix_text_about_int=28=29_with_octal_numbe?= =?utf-8?q?rs=2E_Closes_=2321212=2E?= Message-ID: <3g6p2b4jFQz7LjR@mail.python.org> http://hg.python.org/cpython/rev/6107a727c60a changeset: 90243:6107a727c60a parent: 90241:d1eba2645b80 parent: 90242:fb7bc8fe0d49 user: Eric V. Smith date: Mon Apr 14 07:46:21 2014 -0400 summary: Merge 3.4: Fix text about int() with octal numbers. Closes #21212. files: Doc/faq/programming.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -711,7 +711,7 @@ 144`` and ``int('0x144')`` raises :exc:`ValueError`. ``int(string, base)`` takes the base to convert from as a second optional argument, so ``int('0x144', 16) == 324``. If the base is specified as 0, the number is interpreted using Python's -rules: a leading '0' indicates octal, and '0x' indicates a hex number. +rules: a leading '0o' indicates octal, and '0x' indicates a hex number. Do not use the built-in function :func:`eval` if all you need is to convert strings to numbers. :func:`eval` will be significantly slower and it presents a -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 13:53:42 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 13:53:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_faq_exampl?= =?utf-8?q?e_with_division=2E?= Message-ID: <3g6pBZ4Bh5z7LjX@mail.python.org> http://hg.python.org/cpython/rev/7e66f8ecd360 changeset: 90244:7e66f8ecd360 branch: 3.4 parent: 90242:fb7bc8fe0d49 user: Eric V. Smith date: Mon Apr 14 07:52:53 2014 -0400 summary: Fix faq example with division. files: Doc/faq/programming.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -732,7 +732,7 @@ constructor :func:`str`. If you want a hexadecimal or octal representation, use the built-in functions :func:`hex` or :func:`oct`. For fancy formatting, see the :ref:`string-formatting` section, e.g. ``"{:04d}".format(144)`` yields -``'0144'`` and ``"{:.3f}".format(1/3)`` yields ``'0.333'``. +``'0144'`` and ``"{:.3f}".format(1.0/3.0)`` yields ``'0.333'``. How do I modify a string in place? -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 13:53:43 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 13:53:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E4=3A_Fix_faq_example_with_division=2E?= Message-ID: <3g6pBb5tyRz7LjX@mail.python.org> http://hg.python.org/cpython/rev/62c2b76df993 changeset: 90245:62c2b76df993 parent: 90243:6107a727c60a parent: 90244:7e66f8ecd360 user: Eric V. Smith date: Mon Apr 14 07:53:33 2014 -0400 summary: Merge 3.4: Fix faq example with division. files: Doc/faq/programming.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -732,7 +732,7 @@ constructor :func:`str`. If you want a hexadecimal or octal representation, use the built-in functions :func:`hex` or :func:`oct`. For fancy formatting, see the :ref:`string-formatting` section, e.g. ``"{:04d}".format(144)`` yields -``'0144'`` and ``"{:.3f}".format(1/3)`` yields ``'0.333'``. +``'0144'`` and ``"{:.3f}".format(1.0/3.0)`` yields ``'0.333'``. How do I modify a string in place? -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 16:31:08 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 14 Apr 2014 16:31:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzIxMTY5OiBhZGQg?= =?utf-8?q?comment_and_doc_update_for_getpass_change=2E?= Message-ID: <3g6shD72Dcz7LjP@mail.python.org> http://hg.python.org/cpython/rev/bdde36cd9048 changeset: 90246:bdde36cd9048 branch: 3.4 parent: 90244:7e66f8ecd360 user: R David Murray date: Mon Apr 14 10:28:58 2014 -0400 summary: #21169: add comment and doc update for getpass change. files: Doc/library/getpass.rst | 9 +++++---- Lib/getpass.py | 1 + 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Doc/library/getpass.rst b/Doc/library/getpass.rst --- a/Doc/library/getpass.rst +++ b/Doc/library/getpass.rst @@ -13,10 +13,11 @@ .. function:: getpass(prompt='Password: ', stream=None) Prompt the user for a password without echoing. The user is prompted using - the string *prompt*, which defaults to ``'Password: '``. On Unix, the prompt - is written to the file-like object *stream*. *stream* defaults to the - controlling terminal (:file:`/dev/tty`) or if that is unavailable to - ``sys.stderr`` (this argument is ignored on Windows). + the string *prompt*, which defaults to ``'Password: '``. On Unix, the + prompt is written to the file-like object *stream* using the replace error + handler if needed. *stream* defaults to the controlling terminal + (:file:`/dev/tty`) or if that is unavailable to ``sys.stderr`` (this + argument is ignored on Windows). If echo free input is unavailable getpass() falls back to printing a warning message to *stream* and reading from ``sys.stdin`` and diff --git a/Lib/getpass.py b/Lib/getpass.py --- a/Lib/getpass.py +++ b/Lib/getpass.py @@ -138,6 +138,7 @@ try: stream.write(prompt) except UnicodeEncodeError: + # Use replace error handler to get as much as possible printed. prompt = prompt.encode(stream.encoding, 'replace') prompt = prompt.decode(stream.encoding) stream.write(prompt) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 16:31:10 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 14 Apr 2014 16:31:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2321169=3A_add_comment_and_doc_update_for_getp?= =?utf-8?q?ass_change=2E?= Message-ID: <3g6shG1S6Sz7LkD@mail.python.org> http://hg.python.org/cpython/rev/fe532dccf8f6 changeset: 90247:fe532dccf8f6 parent: 90245:62c2b76df993 parent: 90246:bdde36cd9048 user: R David Murray date: Mon Apr 14 10:30:43 2014 -0400 summary: Merge: #21169: add comment and doc update for getpass change. files: Doc/library/getpass.rst | 9 +++++---- Lib/getpass.py | 1 + 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/Doc/library/getpass.rst b/Doc/library/getpass.rst --- a/Doc/library/getpass.rst +++ b/Doc/library/getpass.rst @@ -13,10 +13,11 @@ .. function:: getpass(prompt='Password: ', stream=None) Prompt the user for a password without echoing. The user is prompted using - the string *prompt*, which defaults to ``'Password: '``. On Unix, the prompt - is written to the file-like object *stream*. *stream* defaults to the - controlling terminal (:file:`/dev/tty`) or if that is unavailable to - ``sys.stderr`` (this argument is ignored on Windows). + the string *prompt*, which defaults to ``'Password: '``. On Unix, the + prompt is written to the file-like object *stream* using the replace error + handler if needed. *stream* defaults to the controlling terminal + (:file:`/dev/tty`) or if that is unavailable to ``sys.stderr`` (this + argument is ignored on Windows). If echo free input is unavailable getpass() falls back to printing a warning message to *stream* and reading from ``sys.stdin`` and diff --git a/Lib/getpass.py b/Lib/getpass.py --- a/Lib/getpass.py +++ b/Lib/getpass.py @@ -138,6 +138,7 @@ try: stream.write(prompt) except UnicodeEncodeError: + # Use replace error handler to get as much as possible printed. prompt = prompt.encode(stream.encoding, 'replace') prompt = prompt.decode(stream.encoding) stream.write(prompt) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:21:00 2014 From: python-checkins at python.org (mark.dickinson) Date: Mon, 14 Apr 2014 17:21:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIwNjI0?= =?utf-8?q?=3A_Exception_docs_wording_tweak_-_clarify_that_it=27s_okay_to_?= =?utf-8?q?inherit?= Message-ID: <3g6tnm1JrRzQ5y@mail.python.org> http://hg.python.org/cpython/rev/8dc1b45bd467 changeset: 90248:8dc1b45bd467 branch: 3.4 parent: 90246:bdde36cd9048 user: Mark Dickinson date: Mon Apr 14 11:20:12 2014 -0400 summary: Issue #20624: Exception docs wording tweak - clarify that it's okay to inherit from a subclass of Exception. files: Doc/library/exceptions.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -28,10 +28,10 @@ interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error. -The built-in exception classes can be sub-classed to define new exceptions; -programmers are encouraged to at least derive new exceptions from the -:exc:`Exception` class and not :exc:`BaseException`. More information on -defining exceptions is available in the Python Tutorial under +The built-in exception classes can be subclassed to define new exceptions; +programmers are encouraged to derive new exceptions from the :exc:`Exception` +class or one of its subclasses, and not from :exc:`BaseException`. More +information on defining exceptions is available in the Python Tutorial under :ref:`tut-userexceptions`. When raising (or re-raising) an exception in an :keyword:`except` clause -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:21:01 2014 From: python-checkins at python.org (mark.dickinson) Date: Mon, 14 Apr 2014 17:21:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320624=3A_Merge_exception_docs_tweak_from_3=2E4_?= =?utf-8?q?branch=2E?= Message-ID: <3g6tnn2bJPz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/262204877004 changeset: 90249:262204877004 parent: 90247:fe532dccf8f6 parent: 90248:8dc1b45bd467 user: Mark Dickinson date: Mon Apr 14 11:20:45 2014 -0400 summary: Issue #20624: Merge exception docs tweak from 3.4 branch. files: Doc/library/exceptions.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -28,10 +28,10 @@ interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error. -The built-in exception classes can be sub-classed to define new exceptions; -programmers are encouraged to at least derive new exceptions from the -:exc:`Exception` class and not :exc:`BaseException`. More information on -defining exceptions is available in the Python Tutorial under +The built-in exception classes can be subclassed to define new exceptions; +programmers are encouraged to derive new exceptions from the :exc:`Exception` +class or one of its subclasses, and not from :exc:`BaseException`. More +information on defining exceptions is available in the Python Tutorial under :ref:`tut-userexceptions`. When raising (or re-raising) an exception in an :keyword:`except` clause -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:25:26 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 17:25:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzEyNTQ2?= =?utf-8?q?=3A_Allow_=5Cx00_as_a_fill_character_for_builtin_type_=5F=5Ffor?= =?utf-8?b?bWF0X18=?= Message-ID: <3g6ttt4RtQzQ5y@mail.python.org> http://hg.python.org/cpython/rev/520ce42ba2b8 changeset: 90250:520ce42ba2b8 branch: 2.7 parent: 90234:50c07ed1743d user: Eric V. Smith date: Mon Apr 14 11:22:33 2014 -0400 summary: Issue #12546: Allow \x00 as a fill character for builtin type __format__ methods. files: Lib/test/test_unicode.py | 21 +++++++++++++++++++++ Misc/NEWS | 3 +++ Objects/stringlib/formatter.h | 18 ++++++++---------- 3 files changed, 32 insertions(+), 10 deletions(-) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -1463,6 +1463,27 @@ self.assertEqual(u'{0:10000}'.format(u''), u' ' * 10000) self.assertEqual(u'{0:10000000}'.format(u''), u' ' * 10000000) + # issue 12546: use \x00 as a fill character + self.assertEqual('{0:\x00<6s}'.format('foo'), 'foo\x00\x00\x00') + self.assertEqual('{0:\x01<6s}'.format('foo'), 'foo\x01\x01\x01') + self.assertEqual('{0:\x00^6s}'.format('foo'), '\x00foo\x00\x00') + self.assertEqual('{0:^6s}'.format('foo'), ' foo ') + + self.assertEqual('{0:\x00<6}'.format(3), '3\x00\x00\x00\x00\x00') + self.assertEqual('{0:\x01<6}'.format(3), '3\x01\x01\x01\x01\x01') + self.assertEqual('{0:\x00^6}'.format(3), '\x00\x003\x00\x00\x00') + self.assertEqual('{0:<6}'.format(3), '3 ') + + self.assertEqual('{0:\x00<6}'.format(3.14), '3.14\x00\x00') + self.assertEqual('{0:\x01<6}'.format(3.14), '3.14\x01\x01') + self.assertEqual('{0:\x00^6}'.format(3.14), '\x003.14\x00') + self.assertEqual('{0:^6}'.format(3.14), ' 3.14 ') + + self.assertEqual('{0:\x00<12}'.format(3+2.0j), '(3+2j)\x00\x00\x00\x00\x00\x00') + self.assertEqual('{0:\x01<12}'.format(3+2.0j), '(3+2j)\x01\x01\x01\x01\x01\x01') + self.assertEqual('{0:\x00^12}'.format(3+2.0j), '\x00\x00\x00(3+2j)\x00\x00\x00') + self.assertEqual('{0:^12}'.format(3+2.0j), ' (3+2j) ') + # format specifiers for user defined type self.assertEqual(u'{0:abc}'.format(C()), u'abc') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,9 @@ - Issue #19638: Fix possible crash / undefined behaviour from huge (more than 2 billion characters) input strings in _Py_dg_strtod. +- Issue #12546: Allow \x00 to be used as a fill character when using str, int, + float, and complex __format__ methods. + Library ------- diff --git a/Objects/stringlib/formatter.h b/Objects/stringlib/formatter.h --- a/Objects/stringlib/formatter.h +++ b/Objects/stringlib/formatter.h @@ -180,8 +180,9 @@ Py_ssize_t consumed; int align_specified = 0; + int fill_char_specified = 0; - format->fill_char = '\0'; + format->fill_char = ' '; format->align = default_align; format->alternate = 0; format->sign = '\0'; @@ -195,6 +196,7 @@ if (end-ptr >= 2 && is_alignment_token(ptr[1])) { format->align = ptr[1]; format->fill_char = ptr[0]; + fill_char_specified = 1; align_specified = 1; ptr += 2; } @@ -218,7 +220,7 @@ } /* The special case for 0-padding (backwards compat) */ - if (format->fill_char == '\0' && end-ptr >= 1 && ptr[0] == '0') { + if (!fill_char_specified && end-ptr >= 1 && ptr[0] == '0') { format->fill_char = '0'; if (!align_specified) { format->align = '='; @@ -715,8 +717,7 @@ /* Write into that space. First the padding. */ p = fill_padding(STRINGLIB_STR(result), len, - format->fill_char=='\0'?' ':format->fill_char, - lpad, rpad); + format->fill_char, lpad, rpad); /* Then the source string. */ memcpy(p, STRINGLIB_STR(value), len * sizeof(STRINGLIB_CHAR)); @@ -893,8 +894,7 @@ /* Populate the memory. */ fill_number(STRINGLIB_STR(result), &spec, pnumeric_chars, n_digits, - prefix, format->fill_char == '\0' ? ' ' : format->fill_char, - &locale, format->type == 'X'); + prefix, format->fill_char, &locale, format->type == 'X'); done: Py_XDECREF(tmp); @@ -1048,8 +1048,7 @@ /* Populate the memory. */ fill_number(STRINGLIB_STR(result), &spec, p, n_digits, NULL, - format->fill_char == '\0' ? ' ' : format->fill_char, &locale, - 0); + format->fill_char, &locale, 0); done: PyMem_Free(buf); @@ -1265,8 +1264,7 @@ /* Populate the memory. First, the padding. */ p = fill_padding(STRINGLIB_STR(result), n_re_total + n_im_total + 1 + add_parens * 2, - format->fill_char=='\0' ? ' ' : format->fill_char, - lpad, rpad); + format->fill_char, lpad, rpad); if (add_parens) *p++ = '('; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:27:04 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 17:27:04 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjA5Njgu?= =?utf-8?q?_unittest=2Emock=2EMagicMock_now_supports_division?= Message-ID: <3g6twm6qvtzQ5y@mail.python.org> http://hg.python.org/cpython/rev/445ef3b58109 changeset: 90251:445ef3b58109 branch: 3.4 parent: 90246:bdde36cd9048 user: Michael Foord date: Mon Apr 14 11:23:48 2014 -0400 summary: Issue 20968. unittest.mock.MagicMock now supports division files: Lib/unittest/mock.py | 4 +- Lib/unittest/test/testmock/testmagicmethods.py | 25 ++++++++++ Misc/NEWS | 3 + 3 files changed, 31 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1634,7 +1634,9 @@ "bool next " ) -numerics = "add sub mul div floordiv mod lshift rshift and xor or pow " +numerics = ( + "add sub mul div floordiv mod lshift rshift and xor or pow truediv" +) inplace = ' '.join('i%s' % n for n in numerics.split()) right = ' '.join('r%s' % n for n in numerics.split()) diff --git a/Lib/unittest/test/testmock/testmagicmethods.py b/Lib/unittest/test/testmock/testmagicmethods.py --- a/Lib/unittest/test/testmock/testmagicmethods.py +++ b/Lib/unittest/test/testmock/testmagicmethods.py @@ -126,6 +126,31 @@ self.assertEqual(7 + mock, mock) self.assertEqual(mock.value, 16) + def test_division(self): + original = mock = Mock() + mock.value = 32 + self.assertRaises(TypeError, lambda: mock / 2) + + def truediv(self, other): + mock.value /= other + return self + mock.__truediv__ = truediv + self.assertEqual(mock / 2, mock) + self.assertEqual(mock.value, 16) + + del mock.__truediv__ + def itruediv(mock): + mock /= 4 + self.assertRaises(TypeError, itruediv, mock) + mock.__itruediv__ = truediv + mock /= 8 + self.assertEqual(mock, original) + self.assertEqual(mock.value, 2) + + self.assertRaises(TypeError, lambda: 8 / mock) + mock.__rtruediv__ = truediv + self.assertEqual(0.5 / mock, mock) + self.assertEqual(mock.value, 4) def test_hash(self): mock = Mock() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,6 +30,9 @@ Library ------- +- Issue #20968: unittest.mock.MagicMock now supports division. + Patch by Johannes Baiter. + - Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second parameter. Bug reported by Guido Vranken. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:27:06 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 17:27:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <3g6twp1Gx0z7Ljj@mail.python.org> http://hg.python.org/cpython/rev/34687e01543f changeset: 90252:34687e01543f parent: 90247:fe532dccf8f6 parent: 90251:445ef3b58109 user: Michael Foord date: Mon Apr 14 11:24:38 2014 -0400 summary: Merge files: Lib/unittest/mock.py | 4 +- Lib/unittest/test/testmock/testmagicmethods.py | 25 ++++++++++ Misc/NEWS | 3 + 3 files changed, 31 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1634,7 +1634,9 @@ "bool next " ) -numerics = "add sub mul div floordiv mod lshift rshift and xor or pow " +numerics = ( + "add sub mul div floordiv mod lshift rshift and xor or pow truediv" +) inplace = ' '.join('i%s' % n for n in numerics.split()) right = ' '.join('r%s' % n for n in numerics.split()) diff --git a/Lib/unittest/test/testmock/testmagicmethods.py b/Lib/unittest/test/testmock/testmagicmethods.py --- a/Lib/unittest/test/testmock/testmagicmethods.py +++ b/Lib/unittest/test/testmock/testmagicmethods.py @@ -126,6 +126,31 @@ self.assertEqual(7 + mock, mock) self.assertEqual(mock.value, 16) + def test_division(self): + original = mock = Mock() + mock.value = 32 + self.assertRaises(TypeError, lambda: mock / 2) + + def truediv(self, other): + mock.value /= other + return self + mock.__truediv__ = truediv + self.assertEqual(mock / 2, mock) + self.assertEqual(mock.value, 16) + + del mock.__truediv__ + def itruediv(mock): + mock /= 4 + self.assertRaises(TypeError, itruediv, mock) + mock.__itruediv__ = truediv + mock /= 8 + self.assertEqual(mock, original) + self.assertEqual(mock.value, 2) + + self.assertRaises(TypeError, lambda: 8 / mock) + mock.__rtruediv__ = truediv + self.assertEqual(0.5 / mock, mock) + self.assertEqual(mock.value, 4) def test_hash(self): mock = Mock() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #20968: unittest.mock.MagicMock now supports division. + Patch by Johannes Baiter. + - Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second parameter. Bug reported by Guido Vranken. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:27:07 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 17:27:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <3g6twq352Zz7Lk2@mail.python.org> http://hg.python.org/cpython/rev/1d7bd04ab6dd changeset: 90253:1d7bd04ab6dd parent: 90252:34687e01543f parent: 90249:262204877004 user: Michael Foord date: Mon Apr 14 11:25:15 2014 -0400 summary: Merge files: Doc/library/exceptions.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -28,10 +28,10 @@ interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error. -The built-in exception classes can be sub-classed to define new exceptions; -programmers are encouraged to at least derive new exceptions from the -:exc:`Exception` class and not :exc:`BaseException`. More information on -defining exceptions is available in the Python Tutorial under +The built-in exception classes can be subclassed to define new exceptions; +programmers are encouraged to derive new exceptions from the :exc:`Exception` +class or one of its subclasses, and not from :exc:`BaseException`. More +information on defining exceptions is available in the Python Tutorial under :ref:`tut-userexceptions`. When raising (or re-raising) an exception in an :keyword:`except` clause -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:27:08 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 17:27:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNCk6?= =?utf-8?q?_Merge?= Message-ID: <3g6twr4vVmz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/6c18b941e850 changeset: 90254:6c18b941e850 branch: 3.4 parent: 90248:8dc1b45bd467 parent: 90251:445ef3b58109 user: Michael Foord date: Mon Apr 14 11:26:38 2014 -0400 summary: Merge files: Lib/unittest/mock.py | 4 +- Lib/unittest/test/testmock/testmagicmethods.py | 25 ++++++++++ Misc/NEWS | 3 + 3 files changed, 31 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1634,7 +1634,9 @@ "bool next " ) -numerics = "add sub mul div floordiv mod lshift rshift and xor or pow " +numerics = ( + "add sub mul div floordiv mod lshift rshift and xor or pow truediv" +) inplace = ' '.join('i%s' % n for n in numerics.split()) right = ' '.join('r%s' % n for n in numerics.split()) diff --git a/Lib/unittest/test/testmock/testmagicmethods.py b/Lib/unittest/test/testmock/testmagicmethods.py --- a/Lib/unittest/test/testmock/testmagicmethods.py +++ b/Lib/unittest/test/testmock/testmagicmethods.py @@ -126,6 +126,31 @@ self.assertEqual(7 + mock, mock) self.assertEqual(mock.value, 16) + def test_division(self): + original = mock = Mock() + mock.value = 32 + self.assertRaises(TypeError, lambda: mock / 2) + + def truediv(self, other): + mock.value /= other + return self + mock.__truediv__ = truediv + self.assertEqual(mock / 2, mock) + self.assertEqual(mock.value, 16) + + del mock.__truediv__ + def itruediv(mock): + mock /= 4 + self.assertRaises(TypeError, itruediv, mock) + mock.__itruediv__ = truediv + mock /= 8 + self.assertEqual(mock, original) + self.assertEqual(mock.value, 2) + + self.assertRaises(TypeError, lambda: 8 / mock) + mock.__rtruediv__ = truediv + self.assertEqual(0.5 / mock, mock) + self.assertEqual(mock.value, 4) def test_hash(self): mock = Mock() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,6 +30,9 @@ Library ------- +- Issue #20968: unittest.mock.MagicMock now supports division. + Patch by Johannes Baiter. + - Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second parameter. Bug reported by Guido Vranken. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:34:04 2014 From: python-checkins at python.org (mark.dickinson) Date: Mon, 14 Apr 2014 17:34:04 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwNjI0?= =?utf-8?q?=3A_Merge_exception_docs_tweak_from_3=2E4_branch=2E?= Message-ID: <3g6v4r6BW4zQ5y@mail.python.org> http://hg.python.org/cpython/rev/f729a0e90c4f changeset: 90255:f729a0e90c4f branch: 2.7 parent: 90250:520ce42ba2b8 user: Mark Dickinson date: Mon Apr 14 11:33:46 2014 -0400 summary: Issue #20624: Merge exception docs tweak from 3.4 branch. files: Doc/library/exceptions.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -38,10 +38,10 @@ interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error. -The built-in exception classes can be sub-classed to define new exceptions; -programmers are encouraged to at least derive new exceptions from the -:exc:`Exception` class and not :exc:`BaseException`. More information on -defining exceptions is available in the Python Tutorial under +The built-in exception classes can be subclassed to define new exceptions; +programmers are encouraged to derive new exceptions from the :exc:`Exception` +class or one of its subclasses, and not from :exc:`BaseException`. More +information on defining exceptions is available in the Python Tutorial under :ref:`tut-userexceptions`. The following exceptions are only used as base classes for other exceptions. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:52:16 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 17:52:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E1=29=3A_disallow_a_neg?= =?utf-8?q?ative_idx_parameter?= Message-ID: <3g6vTr1fhBz7LkJ@mail.python.org> http://hg.python.org/cpython/rev/4bd1fb0f4f44 changeset: 90256:4bd1fb0f4f44 branch: 3.1 parent: 90235:a8facac493ef user: Benjamin Peterson date: Mon Apr 14 11:45:21 2014 -0400 summary: disallow a negative idx parameter files: Modules/_json.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -902,10 +902,11 @@ PyObject *res; Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); - if (idx < 0) - /* Compatibility with Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx canont be negative"); + return NULL; + } + if (idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:52:17 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 17:52:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E2=29=3A_disallow_a_neg?= =?utf-8?q?ative_idx_parameter?= Message-ID: <3g6vTs3LHxz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/95d06fc2616b changeset: 90257:95d06fc2616b branch: 3.2 parent: 90236:8130b8c06606 user: Benjamin Peterson date: Mon Apr 14 11:45:21 2014 -0400 summary: disallow a negative idx parameter files: Modules/_json.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -930,10 +930,11 @@ PyObject *res; Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); - if (idx < 0) - /* Compatibility with Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx canont be negative"); + return NULL; + } + if (idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:52:18 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 17:52:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2?= Message-ID: <3g6vTt4nLYz7LkG@mail.python.org> http://hg.python.org/cpython/rev/37adcd9ffa1c changeset: 90258:37adcd9ffa1c branch: 3.3 parent: 90237:4f15bd1ab28f parent: 90257:95d06fc2616b user: Benjamin Peterson date: Mon Apr 14 11:46:51 2014 -0400 summary: merge 3.2 files: Modules/_json.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -975,10 +975,11 @@ kind = PyUnicode_KIND(pystr); length = PyUnicode_GET_LENGTH(pystr); - if (idx < 0) - /* Compatibility with Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); + return NULL; + } + if (idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:52:19 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 17:52:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <3g6vTv6Wnxz7Lkq@mail.python.org> http://hg.python.org/cpython/rev/ef52ae167555 changeset: 90259:ef52ae167555 branch: 3.4 parent: 90254:6c18b941e850 parent: 90258:37adcd9ffa1c user: Benjamin Peterson date: Mon Apr 14 11:48:21 2014 -0400 summary: merge 3.3 files: Modules/_json.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -941,10 +941,11 @@ kind = PyUnicode_KIND(pystr); length = PyUnicode_GET_LENGTH(pystr); - if (idx < 0) - /* Compatibility with Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); + return NULL; + } + if (idx >= length) { raise_stop_iteration(idx); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:52:21 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 17:52:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g6vTx18N4z7Lk4@mail.python.org> http://hg.python.org/cpython/rev/17d64eb0636a changeset: 90260:17d64eb0636a parent: 90253:1d7bd04ab6dd parent: 90259:ef52ae167555 user: Benjamin Peterson date: Mon Apr 14 11:48:29 2014 -0400 summary: merge 3.4 files: Modules/_json.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -941,10 +941,11 @@ kind = PyUnicode_KIND(pystr); length = PyUnicode_GET_LENGTH(pystr); - if (idx < 0) - /* Compatibility with Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); + return NULL; + } + if (idx >= length) { raise_stop_iteration(idx); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 17:52:22 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 17:52:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_disallow_a_neg?= =?utf-8?q?ative_idx_parameter?= Message-ID: <3g6vTy2x0mz7Lkm@mail.python.org> http://hg.python.org/cpython/rev/c095ff9b0e84 changeset: 90261:c095ff9b0e84 branch: 2.7 parent: 90255:f729a0e90c4f user: Benjamin Peterson date: Mon Apr 14 11:43:09 2014 -0400 summary: disallow a negative idx parameter files: Modules/_json.c | 18 ++++++++++-------- 1 files changed, 10 insertions(+), 8 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1468,10 +1468,11 @@ PyObject *res; char *str = PyString_AS_STRING(pystr); Py_ssize_t length = PyString_GET_SIZE(pystr); - if (idx < 0) - /* Compatibility with the Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); + return NULL; + } + if (idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } @@ -1558,10 +1559,11 @@ PyObject *res; Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); - if (idx < 0) - /* Compatibility with Python version. */ - idx += length; - if (idx < 0 || idx >= length) { + if (idx < 0) { + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); + return NULL; + } + if (idx >= length) { PyErr_SetNone(PyExc_StopIteration); return NULL; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:08:29 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 18:08:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyNTQ2?= =?utf-8?q?=3A_Allow_=5Cx00_as_a_fill_character_for_builtin_type_=5F=5Ffor?= =?utf-8?b?bWF0X18=?= Message-ID: <3g6vrY6dtsz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/7c484551bce1 changeset: 90262:7c484551bce1 branch: 3.4 parent: 90259:ef52ae167555 user: Eric V. Smith date: Mon Apr 14 11:55:10 2014 -0400 summary: Issue #12546: Allow \x00 as a fill character for builtin type __format__ methods. files: Lib/test/test_unicode.py | 21 +++++++++++++++++++++ Misc/NEWS | 3 +++ Python/formatter_unicode.c | 19 ++++++++----------- 3 files changed, 32 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -845,6 +845,27 @@ self.assertEqual('{0:10000}'.format(''), ' ' * 10000) self.assertEqual('{0:10000000}'.format(''), ' ' * 10000000) + # issue 12546: use \x00 as a fill character + self.assertEqual('{0:\x00<6s}'.format('foo'), 'foo\x00\x00\x00') + self.assertEqual('{0:\x01<6s}'.format('foo'), 'foo\x01\x01\x01') + self.assertEqual('{0:\x00^6s}'.format('foo'), '\x00foo\x00\x00') + self.assertEqual('{0:^6s}'.format('foo'), ' foo ') + + self.assertEqual('{0:\x00<6}'.format(3), '3\x00\x00\x00\x00\x00') + self.assertEqual('{0:\x01<6}'.format(3), '3\x01\x01\x01\x01\x01') + self.assertEqual('{0:\x00^6}'.format(3), '\x00\x003\x00\x00\x00') + self.assertEqual('{0:<6}'.format(3), '3 ') + + self.assertEqual('{0:\x00<6}'.format(3.14), '3.14\x00\x00') + self.assertEqual('{0:\x01<6}'.format(3.14), '3.14\x01\x01') + self.assertEqual('{0:\x00^6}'.format(3.14), '\x003.14\x00') + self.assertEqual('{0:^6}'.format(3.14), ' 3.14 ') + + self.assertEqual('{0:\x00<12}'.format(3+2.0j), '(3+2j)\x00\x00\x00\x00\x00\x00') + self.assertEqual('{0:\x01<12}'.format(3+2.0j), '(3+2j)\x01\x01\x01\x01\x01\x01') + self.assertEqual('{0:\x00^12}'.format(3+2.0j), '\x00\x00\x00(3+2j)\x00\x00\x00') + self.assertEqual('{0:^12}'.format(3+2.0j), ' (3+2j) ') + # format specifiers for user defined type self.assertEqual('{0:abc}'.format(C()), 'abc') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ - Issue #20637: Key-sharing now also works for instance dictionaries of subclasses. Patch by Peter Ingebretson. +- Issue #12546: Allow \x00 to be used as a fill character when using str, int, + float, and complex __format__ methods. + Library ------- diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -156,8 +156,9 @@ Py_ssize_t consumed; int align_specified = 0; + int fill_char_specified = 0; - format->fill_char = '\0'; + format->fill_char = ' '; format->align = default_align; format->alternate = 0; format->sign = '\0'; @@ -171,6 +172,7 @@ if (end-pos >= 2 && is_alignment_token(READ_spec(pos+1))) { format->align = READ_spec(pos+1); format->fill_char = READ_spec(pos); + fill_char_specified = 1; align_specified = 1; pos += 2; } @@ -194,7 +196,7 @@ } /* The special case for 0-padding (backwards compat) */ - if (format->fill_char == '\0' && end-pos >= 1 && READ_spec(pos) == '0') { + if (!fill_char_specified && end-pos >= 1 && READ_spec(pos) == '0') { format->fill_char = '0'; if (!align_specified) { format->align = '='; @@ -784,9 +786,7 @@ goto done; /* Write into that space. First the padding. */ - result = fill_padding(writer, len, - format->fill_char=='\0'?' ':format->fill_char, - lpad, rpad); + result = fill_padding(writer, len, format->fill_char, lpad, rpad); if (result == -1) goto done; @@ -956,8 +956,7 @@ /* Populate the memory. */ result = fill_number(writer, &spec, tmp, inumeric_chars, inumeric_chars + n_digits, - tmp, prefix, - format->fill_char == '\0' ? ' ' : format->fill_char, + tmp, prefix, format->fill_char, &locale, format->type == 'X'); done: @@ -1104,8 +1103,7 @@ /* Populate the memory. */ result = fill_number(writer, &spec, unicode_tmp, index, index + n_digits, - NULL, 0, - format->fill_char == '\0' ? ' ' : format->fill_char, + NULL, 0, format->fill_char, &locale, 0); done: @@ -1311,8 +1309,7 @@ /* Populate the memory. First, the padding. */ result = fill_padding(writer, n_re_total + n_im_total + 1 + add_parens * 2, - format->fill_char=='\0' ? ' ' : format->fill_char, - lpad, rpad); + format->fill_char, lpad, rpad); if (result == -1) goto done; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:08:31 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 18:08:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_issue_=2312546=3A_Allow_=5Cx00_as_a_fill_characte?= =?utf-8?q?r_for_builtin_type_=5F=5Fformat=5F=5F?= Message-ID: <3g6vrb2Pxjz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/bd90e68dc81f changeset: 90263:bd90e68dc81f parent: 90260:17d64eb0636a parent: 90262:7c484551bce1 user: Eric V. Smith date: Mon Apr 14 12:08:21 2014 -0400 summary: Closes issue #12546: Allow \x00 as a fill character for builtin type __format__ methods. files: Lib/test/test_unicode.py | 21 +++++++++++++++++++++ Misc/NEWS | 3 +++ Python/formatter_unicode.c | 19 ++++++++----------- 3 files changed, 32 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -874,6 +874,27 @@ self.assertEqual('{0:10000}'.format(''), ' ' * 10000) self.assertEqual('{0:10000000}'.format(''), ' ' * 10000000) + # issue 12546: use \x00 as a fill character + self.assertEqual('{0:\x00<6s}'.format('foo'), 'foo\x00\x00\x00') + self.assertEqual('{0:\x01<6s}'.format('foo'), 'foo\x01\x01\x01') + self.assertEqual('{0:\x00^6s}'.format('foo'), '\x00foo\x00\x00') + self.assertEqual('{0:^6s}'.format('foo'), ' foo ') + + self.assertEqual('{0:\x00<6}'.format(3), '3\x00\x00\x00\x00\x00') + self.assertEqual('{0:\x01<6}'.format(3), '3\x01\x01\x01\x01\x01') + self.assertEqual('{0:\x00^6}'.format(3), '\x00\x003\x00\x00\x00') + self.assertEqual('{0:<6}'.format(3), '3 ') + + self.assertEqual('{0:\x00<6}'.format(3.14), '3.14\x00\x00') + self.assertEqual('{0:\x01<6}'.format(3.14), '3.14\x01\x01') + self.assertEqual('{0:\x00^6}'.format(3.14), '\x003.14\x00') + self.assertEqual('{0:^6}'.format(3.14), ' 3.14 ') + + self.assertEqual('{0:\x00<12}'.format(3+2.0j), '(3+2j)\x00\x00\x00\x00\x00\x00') + self.assertEqual('{0:\x01<12}'.format(3+2.0j), '(3+2j)\x01\x01\x01\x01\x01\x01') + self.assertEqual('{0:\x00^12}'.format(3+2.0j), '\x00\x00\x00(3+2j)\x00\x00\x00') + self.assertEqual('{0:^12}'.format(3+2.0j), ' (3+2j) ') + # format specifiers for user defined type self.assertEqual('{0:abc}'.format(C()), 'abc') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,9 @@ - Issue #19995: %c, %o, %x, and %X now raise TypeError on non-integer input. +- Issue #12546: Allow \x00 to be used as a fill character when using str, int, + float, and complex __format__ methods. + Library ------- diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -156,8 +156,9 @@ Py_ssize_t consumed; int align_specified = 0; + int fill_char_specified = 0; - format->fill_char = '\0'; + format->fill_char = ' '; format->align = default_align; format->alternate = 0; format->sign = '\0'; @@ -171,6 +172,7 @@ if (end-pos >= 2 && is_alignment_token(READ_spec(pos+1))) { format->align = READ_spec(pos+1); format->fill_char = READ_spec(pos); + fill_char_specified = 1; align_specified = 1; pos += 2; } @@ -194,7 +196,7 @@ } /* The special case for 0-padding (backwards compat) */ - if (format->fill_char == '\0' && end-pos >= 1 && READ_spec(pos) == '0') { + if (!fill_char_specified && end-pos >= 1 && READ_spec(pos) == '0') { format->fill_char = '0'; if (!align_specified) { format->align = '='; @@ -784,9 +786,7 @@ goto done; /* Write into that space. First the padding. */ - result = fill_padding(writer, len, - format->fill_char=='\0'?' ':format->fill_char, - lpad, rpad); + result = fill_padding(writer, len, format->fill_char, lpad, rpad); if (result == -1) goto done; @@ -956,8 +956,7 @@ /* Populate the memory. */ result = fill_number(writer, &spec, tmp, inumeric_chars, inumeric_chars + n_digits, - tmp, prefix, - format->fill_char == '\0' ? ' ' : format->fill_char, + tmp, prefix, format->fill_char, &locale, format->type == 'X'); done: @@ -1104,8 +1103,7 @@ /* Populate the memory. */ result = fill_number(writer, &spec, unicode_tmp, index, index + n_digits, - NULL, 0, - format->fill_char == '\0' ? ' ' : format->fill_char, + NULL, 0, format->fill_char, &locale, 0); done: @@ -1311,8 +1309,7 @@ /* Populate the memory. First, the padding. */ result = fill_padding(writer, n_re_total + n_im_total + 1 + add_parens * 2, - format->fill_char=='\0' ? ' ' : format->fill_char, - lpad, rpad); + format->fill_char, lpad, rpad); if (result == -1) goto done; -- Repository URL: http://hg.python.org/cpython From zachary.ware+pydev at gmail.com Mon Apr 14 18:14:23 2014 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Mon, 14 Apr 2014 11:14:23 -0500 Subject: [Python-checkins] cpython (3.1): disallow a negative idx parameter In-Reply-To: <3g6vTr1fhBz7LkJ@mail.python.org> References: <3g6vTr1fhBz7LkJ@mail.python.org> Message-ID: On Mon, Apr 14, 2014 at 10:52 AM, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/4bd1fb0f4f44 > changeset: 90256:4bd1fb0f4f44 > branch: 3.1 > parent: 90235:a8facac493ef > user: Benjamin Peterson > date: Mon Apr 14 11:45:21 2014 -0400 > summary: > disallow a negative idx parameter > > files: > Modules/_json.c | 9 +++++---- > 1 files changed, 5 insertions(+), 4 deletions(-) > > > diff --git a/Modules/_json.c b/Modules/_json.c > --- a/Modules/_json.c > +++ b/Modules/_json.c > @@ -902,10 +902,11 @@ > PyObject *res; > Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); > Py_ssize_t length = PyUnicode_GET_SIZE(pystr); > - if (idx < 0) > - /* Compatibility with Python version. */ > - idx += length; > - if (idx < 0 || idx >= length) { > + if (idx < 0) { > + PyErr_SetString(PyExc_ValueError, "idx canont be negative"); s/canont/cannot/ Also in the comparable 3.2 commit, but not the 3.3+ merge. Regards, -- Zach From python-checkins at python.org Mon Apr 14 18:16:28 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 18:16:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E1=29=3A_fix_poor_spell?= =?utf-8?q?ing?= Message-ID: <3g6w1m5qqwzQXg@mail.python.org> http://hg.python.org/cpython/rev/c7b93519807a changeset: 90264:c7b93519807a branch: 3.1 parent: 90256:4bd1fb0f4f44 user: Benjamin Peterson date: Mon Apr 14 12:15:28 2014 -0400 summary: fix poor spelling files: Modules/_json.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -903,7 +903,7 @@ Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); if (idx < 0) { - PyErr_SetString(PyExc_ValueError, "idx canont be negative"); + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); return NULL; } if (idx >= length) { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:16:30 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 18:16:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E2=29=3A_fix_poor_spell?= =?utf-8?q?ing?= Message-ID: <3g6w1p0Bshz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/0a7d4cdc4c8d changeset: 90265:0a7d4cdc4c8d branch: 3.2 parent: 90257:95d06fc2616b user: Benjamin Peterson date: Mon Apr 14 12:15:28 2014 -0400 summary: fix poor spelling files: Modules/_json.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -931,7 +931,7 @@ Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); Py_ssize_t length = PyUnicode_GET_SIZE(pystr); if (idx < 0) { - PyErr_SetString(PyExc_ValueError, "idx canont be negative"); + PyErr_SetString(PyExc_ValueError, "idx cannot be negative"); return NULL; } if (idx >= length) { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:16:31 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 18:16:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4yIC0+IDMuMyk6?= =?utf-8?q?_merge_3=2E2?= Message-ID: <3g6w1q1TpMz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/a8445ead2f9e changeset: 90266:a8445ead2f9e branch: 3.3 parent: 90258:37adcd9ffa1c parent: 90265:0a7d4cdc4c8d user: Benjamin Peterson date: Mon Apr 14 12:15:58 2014 -0400 summary: merge 3.2 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:16:32 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 18:16:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_merge_3=2E3?= Message-ID: <3g6w1r2tLSz7Lk1@mail.python.org> http://hg.python.org/cpython/rev/fd2c69cedb25 changeset: 90267:fd2c69cedb25 branch: 3.4 parent: 90262:7c484551bce1 parent: 90266:a8445ead2f9e user: Benjamin Peterson date: Mon Apr 14 12:16:07 2014 -0400 summary: merge 3.3 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:16:33 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 18:16:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g6w1s4gX2z7Lk9@mail.python.org> http://hg.python.org/cpython/rev/ab7f57e92629 changeset: 90268:ab7f57e92629 parent: 90263:bd90e68dc81f parent: 90267:fd2c69cedb25 user: Benjamin Peterson date: Mon Apr 14 12:16:18 2014 -0400 summary: merge 3.4 files: -- Repository URL: http://hg.python.org/cpython From benjamin at python.org Mon Apr 14 18:16:33 2014 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 14 Apr 2014 09:16:33 -0700 Subject: [Python-checkins] [Python-Dev] cpython (3.1): disallow a negative idx parameter In-Reply-To: References: <3g6vTr1fhBz7LkJ@mail.python.org> Message-ID: <1397492193.2024.106352309.33F18E5B@webmail.messagingengine.com> On Mon, Apr 14, 2014, at 9:14, Zachary Ware wrote: > On Mon, Apr 14, 2014 at 10:52 AM, benjamin.peterson > wrote: > > http://hg.python.org/cpython/rev/4bd1fb0f4f44 > > changeset: 90256:4bd1fb0f4f44 > > branch: 3.1 > > parent: 90235:a8facac493ef > > user: Benjamin Peterson > > date: Mon Apr 14 11:45:21 2014 -0400 > > summary: > > disallow a negative idx parameter > > > > files: > > Modules/_json.c | 9 +++++---- > > 1 files changed, 5 insertions(+), 4 deletions(-) > > > > > > diff --git a/Modules/_json.c b/Modules/_json.c > > --- a/Modules/_json.c > > +++ b/Modules/_json.c > > @@ -902,10 +902,11 @@ > > PyObject *res; > > Py_UNICODE *str = PyUnicode_AS_UNICODE(pystr); > > Py_ssize_t length = PyUnicode_GET_SIZE(pystr); > > - if (idx < 0) > > - /* Compatibility with Python version. */ > > - idx += length; > > - if (idx < 0 || idx >= length) { > > + if (idx < 0) { > > + PyErr_SetString(PyExc_ValueError, "idx canont be negative"); > > s/canont/cannot/ > > Also in the comparable 3.2 commit, but not the 3.3+ merge. Thanks From python-checkins at python.org Mon Apr 14 18:24:44 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 18:24:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_do_not_generat?= =?utf-8?q?e_pipe_names_in_the_temporary_dir?= Message-ID: <3g6wCJ4vB9z7LjY@mail.python.org> http://hg.python.org/cpython/rev/ea677e26dbeb changeset: 90269:ea677e26dbeb branch: 2.7 parent: 90261:c095ff9b0e84 user: Benjamin Peterson date: Mon Apr 14 12:24:37 2014 -0400 summary: do not generate pipe names in the temporary dir files: Lib/multiprocessing/connection.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py --- a/Lib/multiprocessing/connection.py +++ b/Lib/multiprocessing/connection.py @@ -90,7 +90,7 @@ return tempfile.mktemp(prefix='listener-', dir=get_temp_dir()) elif family == 'AF_PIPE': return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' % - (os.getpid(), _mmap_counter.next())) + (os.getpid(), _mmap_counter.next()), dir="") else: raise ValueError('unrecognized family') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 18:58:31 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 18:58:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320480=3A_Add_ipad?= =?utf-8?q?dress=2Ereverse=5Fpointer=2E_Patch_by_Leon_Weber=2E?= Message-ID: <3g6wyH07V1z7LjS@mail.python.org> http://hg.python.org/cpython/rev/5992d2c9522c changeset: 90270:5992d2c9522c parent: 90268:ab7f57e92629 user: Eric V. Smith date: Mon Apr 14 12:58:07 2014 -0400 summary: Issue #20480: Add ipaddress.reverse_pointer. Patch by Leon Weber. files: Doc/library/ipaddress.rst | 15 +++++++++++++ Lib/ipaddress.py | 29 ++++++++++++++++++++++++++ Lib/test/test_ipaddress.py | 8 +++++++ Misc/ACKS | 1 + Misc/NEWS | 2 + 5 files changed, 55 insertions(+), 0 deletions(-) diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst --- a/Doc/library/ipaddress.rst +++ b/Doc/library/ipaddress.rst @@ -146,6 +146,20 @@ the appropriate length (most significant octet first). This is 4 bytes for IPv4 and 16 bytes for IPv6. + .. attribute:: reverse_pointer + + The name of the reverse DNS PTR record for the IP address, e.g.:: + + >>> ipaddress.ip_address("127.0.0.1").reverse_pointer + '1.0.0.127.in-addr.arpa' + >>> ipaddress.ip_address("2001:db8::1").reverse_pointer + '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa' + + This is the name that could be used for performing a PTR lookup, not the + resolved hostname itself. + + .. versionadded:: 3.5 + .. attribute:: is_multicast ``True`` if the address is reserved for multicast use. See @@ -226,6 +240,7 @@ :class:`IPv4Address` class: .. attribute:: packed + .. attribute:: reverse_pointer .. attribute:: version .. attribute:: max_prefixlen .. attribute:: is_multicast diff --git a/Lib/ipaddress.py b/Lib/ipaddress.py --- a/Lib/ipaddress.py +++ b/Lib/ipaddress.py @@ -436,6 +436,17 @@ return str(self) @property + def reverse_pointer(self): + """The name of the reverse DNS pointer for the IP address, e.g.: + >>> ipaddress.ip_address("127.0.0.1").reverse_pointer + '1.0.0.127.in-addr.arpa' + >>> ipaddress.ip_address("2001:db8::1").reverse_pointer + '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa' + + """ + return self._reverse_pointer() + + @property def version(self): msg = '%200s has no version specified' % (type(self),) raise NotImplementedError(msg) @@ -1221,6 +1232,15 @@ return True return False + def _reverse_pointer(self): + """Return the reverse DNS pointer name for the IPv4 address. + + This implements the method described in RFC1035 3.5. + + """ + reverse_octets = str(self).split('.')[::-1] + return '.'.join(reverse_octets) + '.in-addr.arpa' + @property def max_prefixlen(self): return self._max_prefixlen @@ -1784,6 +1804,15 @@ return '%s/%d' % (':'.join(parts), self._prefixlen) return ':'.join(parts) + def _reverse_pointer(self): + """Return the reverse DNS pointer name for the IPv6 address. + + This implements the method described in RFC3596 2.5. + + """ + reverse_chars = self.exploded[::-1].replace(':', '') + return '.'.join(reverse_chars) + '.ip6.arpa' + @property def max_prefixlen(self): return self._max_prefixlen diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -1593,6 +1593,14 @@ addr3.exploded) self.assertEqual('192.168.178.1', addr4.exploded) + def testReversePointer(self): + addr1 = ipaddress.IPv4Address('127.0.0.1') + addr2 = ipaddress.IPv6Address('2001:db8::1') + self.assertEqual('1.0.0.127.in-addr.arpa', addr1.reverse_pointer) + self.assertEqual('1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.' + + 'b.d.0.1.0.0.2.ip6.arpa', + addr2.reverse_pointer) + def testIntRepresentation(self): self.assertEqual(16909060, int(self.ipv4_address)) self.assertEqual(42540616829182469433547762482097946625, diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1392,6 +1392,7 @@ David Watson Aaron Watters Henrik Weber +Leon Weber Corran Webster Glyn Webster Phil Webster diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,8 @@ - Issue #12546: Allow \x00 to be used as a fill character when using str, int, float, and complex __format__ methods. +- Issue #20480: Add ipaddress.reverse_pointer. Patch by Leon Weber. + Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 19:10:28 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 19:10:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzc3NzY6?= =?utf-8?q?_Fix_=60=60Host=3A=27=27_header_and_reconnection_when_using?= Message-ID: <3g6xD43QzBz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/39ee3286d187 changeset: 90271:39ee3286d187 branch: 3.4 parent: 90267:fd2c69cedb25 user: Senthil Kumaran date: Mon Apr 14 13:07:56 2014 -0400 summary: Issue #7776: Fix ``Host:'' header and reconnection when using http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. files: Lib/http/client.py | 73 ++++++++++++++++++--------- Lib/test/test_httplib.py | 50 ++++++++++++++++++- Misc/NEWS | 3 + 3 files changed, 100 insertions(+), 26 deletions(-) diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -747,14 +747,30 @@ self._tunnel_port = None self._tunnel_headers = {} - self._set_hostport(host, port) + (self.host, self.port) = self._get_hostport(host, port) + + # This is stored as an instance variable to allow unit + # tests to replace it with a suitable mockup + self._create_connection = socket.create_connection def set_tunnel(self, host, port=None, headers=None): - """ Sets up the host and the port for the HTTP CONNECT Tunnelling. + """Set up host and port for HTTP CONNECT tunnelling. - The headers argument should be a mapping of extra HTTP headers - to send with the CONNECT request. + In a connection that uses HTTP CONNECT tunneling, the host passed to the + constructor is used as a proxy server that relays all communication to + the endpoint passed to `set_tunnel`. This done by sending an HTTP + CONNECT request to the proxy server when the connection is established. + + This method must be called before the HTML connection has been + established. + + The headers argument should be a mapping of extra HTTP headers to send + with the CONNECT request. """ + + if self.sock: + raise RuntimeError("Can't set up tunnel for established connection") + self._tunnel_host = host self._tunnel_port = port if headers: @@ -762,7 +778,7 @@ else: self._tunnel_headers.clear() - def _set_hostport(self, host, port): + def _get_hostport(self, host, port): if port is None: i = host.rfind(':') j = host.rfind(']') # ipv6 addresses have [...] @@ -779,15 +795,16 @@ port = self.default_port if host and host[0] == '[' and host[-1] == ']': host = host[1:-1] - self.host = host - self.port = port + + return (host, port) def set_debuglevel(self, level): self.debuglevel = level def _tunnel(self): - self._set_hostport(self._tunnel_host, self._tunnel_port) - connect_str = "CONNECT %s:%d HTTP/1.0\r\n" % (self.host, self.port) + (host, port) = self._get_hostport(self._tunnel_host, + self._tunnel_port) + connect_str = "CONNECT %s:%d HTTP/1.0\r\n" % (host, port) connect_bytes = connect_str.encode("ascii") self.send(connect_bytes) for header, value in self._tunnel_headers.items(): @@ -815,8 +832,9 @@ def connect(self): """Connect to the host and port specified in __init__.""" - self.sock = socket.create_connection((self.host,self.port), - self.timeout, self.source_address) + self.sock = self._create_connection((self.host,self.port), + self.timeout, self.source_address) + if self._tunnel_host: self._tunnel() @@ -985,22 +1003,29 @@ netloc_enc = netloc.encode("idna") self.putheader('Host', netloc_enc) else: + if self._tunnel_host: + host = self._tunnel_host + port = self._tunnel_port + else: + host = self.host + port = self.port + try: - host_enc = self.host.encode("ascii") + host_enc = host.encode("ascii") except UnicodeEncodeError: - host_enc = self.host.encode("idna") + host_enc = host.encode("idna") # As per RFC 273, IPv6 address should be wrapped with [] # when used as Host header - if self.host.find(':') >= 0: + if host.find(':') >= 0: host_enc = b'[' + host_enc + b']' - if self.port == self.default_port: + if port == self.default_port: self.putheader('Host', host_enc) else: host_enc = host_enc.decode("ascii") - self.putheader('Host', "%s:%s" % (host_enc, self.port)) + self.putheader('Host', "%s:%s" % (host_enc, port)) # note: we are assuming that clients will not attempt to set these # headers since *this* library must deal with the @@ -1193,19 +1218,19 @@ def connect(self): "Connect to a host on a given (SSL) port." - sock = socket.create_connection((self.host, self.port), - self.timeout, self.source_address) + super().connect() if self._tunnel_host: - self.sock = sock - self._tunnel() + server_hostname = self._tunnel_host + else: + server_hostname = self.host + sni_hostname = server_hostname if ssl.HAS_SNI else None - server_hostname = self.host if ssl.HAS_SNI else None - self.sock = self._context.wrap_socket(sock, - server_hostname=server_hostname) + self.sock = self._context.wrap_socket(self.sock, + server_hostname=sni_hostname) if not self._context.check_hostname and self._check_hostname: try: - ssl.match_hostname(self.sock.getpeercert(), self.host) + ssl.match_hostname(self.sock.getpeercert(), server_hostname) except Exception: self.sock.shutdown(socket.SHUT_RDWR) self.sock.close() diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -21,13 +21,15 @@ HOST = support.HOST class FakeSocket: - def __init__(self, text, fileclass=io.BytesIO): + def __init__(self, text, fileclass=io.BytesIO, host=None, port=None): if isinstance(text, str): text = text.encode("ascii") self.text = text self.fileclass = fileclass self.data = b'' self.sendall_calls = 0 + self.host = host + self.port = port def sendall(self, data): self.sendall_calls += 1 @@ -38,6 +40,9 @@ raise client.UnimplementedFileMode() return self.fileclass(self.text) + def close(self): + pass + class EPipeSocket(FakeSocket): def __init__(self, text, pipe_trigger): @@ -970,10 +975,51 @@ header = self.resp.getheader('No-Such-Header',default=42) self.assertEqual(header, 42) +class TunnelTests(TestCase): + + def test_connect(self): + response_text = ( + 'HTTP/1.0 200 OK\r\n\r\n' # Reply to CONNECT + 'HTTP/1.1 200 OK\r\n' # Reply to HEAD + 'Content-Length: 42\r\n\r\n' + ) + + def create_connection(address, timeout=None, source_address=None): + return FakeSocket(response_text, host=address[0], + port=address[1]) + + conn = client.HTTPConnection('proxy.com') + conn._create_connection = create_connection + + # Once connected, we shouldn't be able to tunnel anymore + conn.connect() + self.assertRaises(RuntimeError, conn.set_tunnel, + 'destination.com') + + # But if we close the connection, we're good + conn.close() + conn.set_tunnel('destination.com') + conn.request('HEAD', '/', '') + + self.assertEqual(conn.sock.host, 'proxy.com') + self.assertEqual(conn.sock.port, 80) + self.assertTrue(b'CONNECT destination.com' in conn.sock.data) + self.assertTrue(b'Host: destination.com' in conn.sock.data) + + # This test should be removed when CONNECT gets the HTTP/1.1 blessing + self.assertTrue(b'Host: proxy.com' not in conn.sock.data) + + conn.close() + conn.request('PUT', '/', '') + self.assertEqual(conn.sock.host, 'proxy.com') + self.assertEqual(conn.sock.port, 80) + self.assertTrue(b'CONNECT destination.com' in conn.sock.data) + self.assertTrue(b'Host: destination.com' in conn.sock.data) + def test_main(verbose=None): support.run_unittest(HeaderTests, OfflineTest, BasicTest, TimeoutTest, HTTPSTest, RequestBodyTest, SourceAddressTest, - HTTPResponseTest) + HTTPResponseTest, TunnelTests) if __name__ == '__main__': test_main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -33,6 +33,9 @@ Library ------- +- Issue #7776: Fix ``Host:'' header and reconnection when using + http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. + - Issue #20968: unittest.mock.MagicMock now supports division. Patch by Johannes Baiter. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 19:10:29 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 19:10:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_from_3=2E4?= Message-ID: <3g6xD600JPz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/2c9af09ba7b8 changeset: 90272:2c9af09ba7b8 parent: 90270:5992d2c9522c parent: 90271:39ee3286d187 user: Senthil Kumaran date: Mon Apr 14 13:10:05 2014 -0400 summary: merge from 3.4 Issue #7776: Fix ``Host:'' header and reconnection when using http.client.HTTPConnection.set_tunnel() Patch by Nikolaus Rath. files: Lib/http/client.py | 73 ++++++++++++++++++--------- Lib/test/test_httplib.py | 50 ++++++++++++++++++- Misc/NEWS | 3 + 3 files changed, 100 insertions(+), 26 deletions(-) diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -811,14 +811,30 @@ self._tunnel_port = None self._tunnel_headers = {} - self._set_hostport(host, port) + (self.host, self.port) = self._get_hostport(host, port) + + # This is stored as an instance variable to allow unit + # tests to replace it with a suitable mockup + self._create_connection = socket.create_connection def set_tunnel(self, host, port=None, headers=None): - """ Sets up the host and the port for the HTTP CONNECT Tunnelling. + """Set up host and port for HTTP CONNECT tunnelling. - The headers argument should be a mapping of extra HTTP headers - to send with the CONNECT request. + In a connection that uses HTTP CONNECT tunneling, the host passed to the + constructor is used as a proxy server that relays all communication to + the endpoint passed to `set_tunnel`. This done by sending an HTTP + CONNECT request to the proxy server when the connection is established. + + This method must be called before the HTML connection has been + established. + + The headers argument should be a mapping of extra HTTP headers to send + with the CONNECT request. """ + + if self.sock: + raise RuntimeError("Can't set up tunnel for established connection") + self._tunnel_host = host self._tunnel_port = port if headers: @@ -826,7 +842,7 @@ else: self._tunnel_headers.clear() - def _set_hostport(self, host, port): + def _get_hostport(self, host, port): if port is None: i = host.rfind(':') j = host.rfind(']') # ipv6 addresses have [...] @@ -843,15 +859,16 @@ port = self.default_port if host and host[0] == '[' and host[-1] == ']': host = host[1:-1] - self.host = host - self.port = port + + return (host, port) def set_debuglevel(self, level): self.debuglevel = level def _tunnel(self): - self._set_hostport(self._tunnel_host, self._tunnel_port) - connect_str = "CONNECT %s:%d HTTP/1.0\r\n" % (self.host, self.port) + (host, port) = self._get_hostport(self._tunnel_host, + self._tunnel_port) + connect_str = "CONNECT %s:%d HTTP/1.0\r\n" % (host, port) connect_bytes = connect_str.encode("ascii") self.send(connect_bytes) for header, value in self._tunnel_headers.items(): @@ -879,8 +896,9 @@ def connect(self): """Connect to the host and port specified in __init__.""" - self.sock = socket.create_connection((self.host,self.port), - self.timeout, self.source_address) + self.sock = self._create_connection((self.host,self.port), + self.timeout, self.source_address) + if self._tunnel_host: self._tunnel() @@ -1049,22 +1067,29 @@ netloc_enc = netloc.encode("idna") self.putheader('Host', netloc_enc) else: + if self._tunnel_host: + host = self._tunnel_host + port = self._tunnel_port + else: + host = self.host + port = self.port + try: - host_enc = self.host.encode("ascii") + host_enc = host.encode("ascii") except UnicodeEncodeError: - host_enc = self.host.encode("idna") + host_enc = host.encode("idna") # As per RFC 273, IPv6 address should be wrapped with [] # when used as Host header - if self.host.find(':') >= 0: + if host.find(':') >= 0: host_enc = b'[' + host_enc + b']' - if self.port == self.default_port: + if port == self.default_port: self.putheader('Host', host_enc) else: host_enc = host_enc.decode("ascii") - self.putheader('Host', "%s:%s" % (host_enc, self.port)) + self.putheader('Host', "%s:%s" % (host_enc, port)) # note: we are assuming that clients will not attempt to set these # headers since *this* library must deal with the @@ -1257,19 +1282,19 @@ def connect(self): "Connect to a host on a given (SSL) port." - sock = socket.create_connection((self.host, self.port), - self.timeout, self.source_address) + super().connect() if self._tunnel_host: - self.sock = sock - self._tunnel() + server_hostname = self._tunnel_host + else: + server_hostname = self.host + sni_hostname = server_hostname if ssl.HAS_SNI else None - server_hostname = self.host if ssl.HAS_SNI else None - self.sock = self._context.wrap_socket(sock, - server_hostname=server_hostname) + self.sock = self._context.wrap_socket(self.sock, + server_hostname=sni_hostname) if not self._context.check_hostname and self._check_hostname: try: - ssl.match_hostname(self.sock.getpeercert(), self.host) + ssl.match_hostname(self.sock.getpeercert(), server_hostname) except Exception: self.sock.shutdown(socket.SHUT_RDWR) self.sock.close() diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -41,13 +41,15 @@ HOST = support.HOST class FakeSocket: - def __init__(self, text, fileclass=io.BytesIO): + def __init__(self, text, fileclass=io.BytesIO, host=None, port=None): if isinstance(text, str): text = text.encode("ascii") self.text = text self.fileclass = fileclass self.data = b'' self.sendall_calls = 0 + self.host = host + self.port = port def sendall(self, data): self.sendall_calls += 1 @@ -61,6 +63,9 @@ self.file.close = lambda:None #nerf close () return self.file + def close(self): + pass + class EPipeSocket(FakeSocket): def __init__(self, text, pipe_trigger): @@ -1204,11 +1209,52 @@ header = self.resp.getheader('No-Such-Header',default=42) self.assertEqual(header, 42) +class TunnelTests(TestCase): + + def test_connect(self): + response_text = ( + 'HTTP/1.0 200 OK\r\n\r\n' # Reply to CONNECT + 'HTTP/1.1 200 OK\r\n' # Reply to HEAD + 'Content-Length: 42\r\n\r\n' + ) + + def create_connection(address, timeout=None, source_address=None): + return FakeSocket(response_text, host=address[0], + port=address[1]) + + conn = client.HTTPConnection('proxy.com') + conn._create_connection = create_connection + + # Once connected, we shouldn't be able to tunnel anymore + conn.connect() + self.assertRaises(RuntimeError, conn.set_tunnel, + 'destination.com') + + # But if we close the connection, we're good + conn.close() + conn.set_tunnel('destination.com') + conn.request('HEAD', '/', '') + + self.assertEqual(conn.sock.host, 'proxy.com') + self.assertEqual(conn.sock.port, 80) + self.assertTrue(b'CONNECT destination.com' in conn.sock.data) + self.assertTrue(b'Host: destination.com' in conn.sock.data) + + # This test should be removed when CONNECT gets the HTTP/1.1 blessing + self.assertTrue(b'Host: proxy.com' not in conn.sock.data) + + conn.close() + conn.request('PUT', '/', '') + self.assertEqual(conn.sock.host, 'proxy.com') + self.assertEqual(conn.sock.port, 80) + self.assertTrue(b'CONNECT destination.com' in conn.sock.data) + self.assertTrue(b'Host: destination.com' in conn.sock.data) + def test_main(verbose=None): support.run_unittest(HeaderTests, OfflineTest, BasicTest, TimeoutTest, HTTPSTest, RequestBodyTest, SourceAddressTest, HTTPResponseTest, ExtendedReadTest, - ExtendedReadTestChunked) + ExtendedReadTestChunked, TunnelTests) if __name__ == '__main__': test_main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,9 @@ Library ------- +- Issue #7776: Fix ``Host:'' header and reconnection when using + http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. + - Issue #20968: unittest.mock.MagicMock now supports division. Patch by Johannes Baiter. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 19:13:07 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 19:13:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_do_not_generat?= =?utf-8?q?e_pipe_names_in_the_temporary_dir?= Message-ID: <3g6xH75WyQz7Ljj@mail.python.org> http://hg.python.org/cpython/rev/0d64a930e846 changeset: 90273:0d64a930e846 branch: 3.4 parent: 90271:39ee3286d187 user: Benjamin Peterson date: Mon Apr 14 12:24:37 2014 -0400 summary: do not generate pipe names in the temporary dir files: Lib/multiprocessing/connection.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py --- a/Lib/multiprocessing/connection.py +++ b/Lib/multiprocessing/connection.py @@ -76,7 +76,7 @@ return tempfile.mktemp(prefix='listener-', dir=util.get_temp_dir()) elif family == 'AF_PIPE': return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' % - (os.getpid(), next(_mmap_counter))) + (os.getpid(), next(_mmap_counter)), dir="") else: raise ValueError('unrecognized family') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 19:13:09 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 19:13:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g6xH90YP4z7Ljj@mail.python.org> http://hg.python.org/cpython/rev/99db0ca554d9 changeset: 90274:99db0ca554d9 parent: 90272:2c9af09ba7b8 parent: 90273:0d64a930e846 user: Benjamin Peterson date: Mon Apr 14 13:13:01 2014 -0400 summary: merge 3.4 files: Lib/multiprocessing/connection.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py --- a/Lib/multiprocessing/connection.py +++ b/Lib/multiprocessing/connection.py @@ -76,7 +76,7 @@ return tempfile.mktemp(prefix='listener-', dir=util.get_temp_dir()) elif family == 'AF_PIPE': return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' % - (os.getpid(), next(_mmap_counter))) + (os.getpid(), next(_mmap_counter)), dir="") else: raise ValueError('unrecognized family') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 19:40:06 2014 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 14 Apr 2014 19:40:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE4NTE4OiBtZW50?= =?utf-8?q?ion_that_including_a_return_statement_changes/breaks_the_behavi?= =?utf-8?q?our?= Message-ID: <3g6xtG33nYz7LjX@mail.python.org> http://hg.python.org/cpython/rev/7e2708484ea5 changeset: 90275:7e2708484ea5 branch: 3.4 parent: 90273:0d64a930e846 user: Andrew Kuchling date: Mon Apr 14 13:39:43 2014 -0400 summary: #18518: mention that including a return statement changes/breaks the behaviour files: Doc/library/timeit.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -63,6 +63,12 @@ Create a :class:`Timer` instance with the given statement, *setup* code and *timer* function and run its :meth:`.timeit` method with *number* executions. + .. note:: + + Because :meth:`.timeit` is executing *stmt*, placing a return statement + in *stmt* will prevent :meth:`.timeit` from returning execution time. + It will instead return the data specified by your return statement. + .. function:: repeat(stmt='pass', setup='pass', timer=, repeat=3, number=1000000) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 19:52:54 2014 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 14 Apr 2014 19:52:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E4?= Message-ID: <3g6y924Rb9z7LjQ@mail.python.org> http://hg.python.org/cpython/rev/39f60aa96475 changeset: 90276:39f60aa96475 parent: 90274:99db0ca554d9 parent: 90275:7e2708484ea5 user: Andrew Kuchling date: Mon Apr 14 13:52:34 2014 -0400 summary: Merge from 3.4 files: Doc/library/timeit.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -63,6 +63,12 @@ Create a :class:`Timer` instance with the given statement, *setup* code and *timer* function and run its :meth:`.timeit` method with *number* executions. + .. note:: + + Because :meth:`.timeit` is executing *stmt*, placing a return statement + in *stmt* will prevent :meth:`.timeit` from returning execution time. + It will instead return the data specified by your return statement. + .. function:: repeat(stmt='pass', setup='pass', timer=, repeat=3, number=1000000) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 20:20:15 2014 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 14 Apr 2014 20:20:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2311983=3A_update_comment?= =?utf-8?q?_to_describe_which_fields_are_used_and_why=2E?= Message-ID: <3g6ymb6dMyz7LjS@mail.python.org> http://hg.python.org/cpython/rev/0e35cef1d984 changeset: 90277:0e35cef1d984 user: Andrew Kuchling date: Mon Apr 14 14:19:52 2014 -0400 summary: #11983: update comment to describe which fields are used and why. Original patch by Caelyn McAulay; modified after discussion w/ her at the PyCon 2014 sprints. files: Include/code.h | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Include/code.h b/Include/code.h --- a/Include/code.h +++ b/Include/code.h @@ -21,7 +21,12 @@ PyObject *co_varnames; /* tuple of strings (local variable names) */ PyObject *co_freevars; /* tuple of strings (free variable names) */ PyObject *co_cellvars; /* tuple of strings (cell variable names) */ - /* The rest doesn't count for hash or comparisons */ + /* The rest aren't used in either hash or comparisons, except for + co_name (used in both) and co_firstlineno (used only in + comparisons). This is done to preserve the name and line number + for tracebacks and debuggers; otherwise, constant de-duplication + would collapse identical functions/lambdas defined on different lines. + */ unsigned char *co_cell2arg; /* Maps cell vars which are arguments. */ PyObject *co_filename; /* unicode (where it was loaded from) */ PyObject *co_name; /* unicode (name, for reference) */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 20:33:28 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 20:33:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_the_NEWS_E?= =?utf-8?q?ntry_item=2E_=28Thanks_St=C3=A9phane_Wirtel=29?= Message-ID: <3g6z3r3zpQz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/a85606b6de32 changeset: 90278:a85606b6de32 branch: 3.4 parent: 90275:7e2708484ea5 user: Senthil Kumaran date: Mon Apr 14 14:32:20 2014 -0400 summary: Fix the NEWS Entry item. (Thanks St?phane Wirtel) files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -33,7 +33,7 @@ Library ------- -- Issue #7776: Fix ``Host:'' header and reconnection when using +- Issue #7776: Fix ``Host:`` header and reconnection when using http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. - Issue #20968: unittest.mock.MagicMock now supports division. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 20:33:29 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 20:33:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g6z3s5P04z7LjQ@mail.python.org> http://hg.python.org/cpython/rev/fb5516cbc522 changeset: 90279:fb5516cbc522 parent: 90277:0e35cef1d984 parent: 90278:a85606b6de32 user: Senthil Kumaran date: Mon Apr 14 14:33:14 2014 -0400 summary: merge 3.4 files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,7 +42,7 @@ Library ------- -- Issue #7776: Fix ``Host:'' header and reconnection when using +- Issue #7776: Fix ``Host:`` header and reconnection when using http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. - Issue #20968: unittest.mock.MagicMock now supports division. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 21:06:18 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 14 Apr 2014 21:06:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE1MTA0OiBpbXBy?= =?utf-8?q?ove_the_discussion_of_=5F=5Fmain=5F=5F=2E?= Message-ID: <3g6znk0Gldz7Lk9@mail.python.org> http://hg.python.org/cpython/rev/4f23648b7c97 changeset: 90280:4f23648b7c97 branch: 3.4 parent: 90278:a85606b6de32 user: R David Murray date: Mon Apr 14 15:04:47 2014 -0400 summary: #15104: improve the discussion of __main__. Patch by Sam Lucidi. files: Doc/library/__main__.rst | 16 +++++++++++----- 1 files changed, 11 insertions(+), 5 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -5,13 +5,19 @@ .. module:: __main__ :synopsis: The environment where the top-level script is run. +'__main__' is the name of the scope in which top-level code executes. +A module's __name__ is set equal to '__main__' when read from +standard input, a script, or from an interactive prompt. -This module represents the (otherwise anonymous) scope in which the -interpreter's main program executes --- commands read either from standard -input, from a script file, or from an interactive prompt. It is this -environment in which the idiomatic "conditional script" stanza causes a script -to run:: +A module can discover whether or not it is running in the main scope by +checking its own __name__, which allows a common idiom for conditionally +executing code in a module when it is run as a script or with `python +-m` but not when it is imported: if __name__ == "__main__": + # execute only if run as a script main() +For a package, the same effect can be achieved by including a +__main__.py module, the contents of which will be executed when the +module is run with -m. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 21:06:19 2014 From: python-checkins at python.org (r.david.murray) Date: Mon, 14 Apr 2014 21:06:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2315104=3A_improve_the_discussion_of_=5F=5Fmai?= =?utf-8?b?bl9fLg==?= Message-ID: <3g6znl1Y0Wz7LkG@mail.python.org> http://hg.python.org/cpython/rev/94ac365bf1b7 changeset: 90281:94ac365bf1b7 parent: 90279:fb5516cbc522 parent: 90280:4f23648b7c97 user: R David Murray date: Mon Apr 14 15:05:12 2014 -0400 summary: Merge: #15104: improve the discussion of __main__. files: Doc/library/__main__.rst | 16 +++++++++++----- 1 files changed, 11 insertions(+), 5 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -5,13 +5,19 @@ .. module:: __main__ :synopsis: The environment where the top-level script is run. +'__main__' is the name of the scope in which top-level code executes. +A module's __name__ is set equal to '__main__' when read from +standard input, a script, or from an interactive prompt. -This module represents the (otherwise anonymous) scope in which the -interpreter's main program executes --- commands read either from standard -input, from a script file, or from an interactive prompt. It is this -environment in which the idiomatic "conditional script" stanza causes a script -to run:: +A module can discover whether or not it is running in the main scope by +checking its own __name__, which allows a common idiom for conditionally +executing code in a module when it is run as a script or with `python +-m` but not when it is imported: if __name__ == "__main__": + # execute only if run as a script main() +For a package, the same effect can be achieved by including a +__main__.py module, the contents of which will be executed when the +module is run with -m. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 21:08:43 2014 From: python-checkins at python.org (andrew.kuchling) Date: Mon, 14 Apr 2014 21:08:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2310481=3A_describe_unive?= =?utf-8?q?rsal=5Fnewlines=27_effect_on_communicate=28=29/check=5Foutput?= =?utf-8?b?KCk=?= Message-ID: <3g6zrW4jCXz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/267fff541bda changeset: 90282:267fff541bda user: Andrew Kuchling date: Mon Apr 14 15:08:18 2014 -0400 summary: #10481: describe universal_newlines' effect on communicate()/check_output() output (alternately bytes or strings) Patch by Sam Kimbrel. files: Doc/library/subprocess.rst | 1 + Lib/subprocess.py | 38 ++++++++++++++++++------- 2 files changed, 28 insertions(+), 11 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -660,6 +660,7 @@ must be bytes or, if *universal_newlines* was ``True``, a string. :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``. + The data will be bytes or, if *universal_newlines* was ``True``, strings. Note that if you want to send data to the process's stdin, you need to create the Popen object with ``stdin=PIPE``. Similarly, to get anything other than diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -104,17 +104,21 @@ If env is not None, it defines the environment variables for the new process. -If universal_newlines is false, the file objects stdin, stdout and stderr +If universal_newlines is False, the file objects stdin, stdout and stderr are opened as binary files, and no line ending conversion is done. -If universal_newlines is true, the file objects stdout and stderr are -opened as a text files, but lines may be terminated by any of '\n', +If universal_newlines is True, the file objects stdout and stderr are +opened as a text file, but lines may be terminated by any of '\n', the Unix end-of-line convention, '\r', the old Macintosh convention or '\r\n', the Windows convention. All of these external representations are seen as '\n' by the Python program. Also, the newlines attribute of the file objects stdout, stdin and stderr are not updated by the communicate() method. +In either case, the process being communicated with should start up +expecting to receive bytes on its standard input and decode them with +the same encoding they are sent in. + The startupinfo and creationflags, if given, will be passed to the underlying CreateProcess() function. They can specify things such as appearance of the main window and priority for the new process. @@ -184,6 +188,9 @@ pass a string to the subprocess's stdin. If you use this argument you may not also use the Popen constructor's "stdin" argument. + If universal_newlines is set to True, the "input" argument must + be a string rather than bytes, and the return value will be a string. + Exceptions ---------- Exceptions raised in the child process, before the new program has @@ -225,9 +232,13 @@ communicate(input=None) Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to - terminate. The optional input argument should be a string to be + terminate. The optional input argument should be data to be sent to the child process, or None, if no data should be sent to - the child. + the child. If the Popen instance was constructed with universal_newlines + set to True, the input argument should be a string and will be encoded + using the preferred system encoding (see locale.getpreferredencoding); + if universal_newlines is False, the input argument should be a + byte string. communicate() returns a tuple (stdout, stderr). @@ -587,8 +598,8 @@ ... input=b"when in the course of fooman events\n") b'when in the course of barman events\n' - If universal_newlines=True is passed, the return value will be a - string rather than bytes. + If universal_newlines=True is passed, the "input" argument must be a + string and the return value will be a string rather than bytes. """ if 'stdout' in kwargs: raise ValueError('stdout argument not allowed, it will be overridden.') @@ -908,11 +919,16 @@ def communicate(self, input=None, timeout=None): """Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for - process to terminate. The optional input argument should be - bytes to be sent to the child process, or None, if no data - should be sent to the child. + process to terminate. - communicate() returns a tuple (stdout, stderr).""" + The optional "input" argument should be data to be sent to the + child process (if self.universal_newlines is True, this should + be a string; if it is False, "input" should be bytes), or + None, if no data should be sent to the child. + + communicate() returns a tuple (stdout, stderr). These will be + bytes or, if self.universal_newlines was True, a string. + """ if self._communication_started and input: raise ValueError("Cannot send input after starting communication") -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:10:17 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 22:10:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMTc4MjYu?= =?utf-8?q?_Setting_an_iterable_side=5Feffect_on_a_mock_created_by?= Message-ID: <3g71CY4nQvz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/1e3c64470629 changeset: 90283:1e3c64470629 branch: 3.4 parent: 90280:4f23648b7c97 user: Michael Foord date: Mon Apr 14 16:09:42 2014 -0400 summary: Issue 17826. Setting an iterable side_effect on a mock created by create_autospec now works files: Lib/unittest/mock.py | 15 ++++++++++- Lib/unittest/test/testmock/testmock.py | 18 ++++++++++++++ Misc/NEWS | 3 ++ 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -343,7 +343,14 @@ value._mock_name = name return True - +# Internal class to identify if we wrapped an iterator object or not. +class _MockIter(object): + def __init__(self, obj): + self.obj = iter(obj) + def __iter__(self): + return self + def __next__(self): + return next(self.obj) class Base(object): _mock_return_value = DEFAULT @@ -495,7 +502,11 @@ delegated = self._mock_delegate if delegated is None: return self._mock_side_effect - return delegated.side_effect + sf = delegated.side_effect + if sf is not None and not callable(sf) and not isinstance(sf, _MockIter): + sf = _MockIter(sf) + delegated.side_effect = sf + return sf def __set_side_effect(self, value): value = _try_iter(value) diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -154,6 +154,24 @@ mock = Mock(side_effect=side_effect, return_value=sentinel.RETURN) self.assertEqual(mock(), sentinel.RETURN) + def test_autospec_side_effect(self): + # Test for issue17826 + results = [1, 2, 3] + def effect(): + return results.pop() + def f(): + pass + + mock = create_autospec(f) + mock.side_effect = [1, 2, 3] + self.assertEqual([mock(), mock(), mock()], [1, 2, 3], + "side effect not used correctly in create_autospec") + # Test where side effect is a callable + results = [1, 2, 3] + mock = create_autospec(f) + mock.side_effect = effect + self.assertEqual([mock(), mock(), mock()], [3, 2, 1], + "callable side effect not used correctly") @unittest.skipUnless('java' in sys.platform, 'This test only applies to Jython') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -33,6 +33,9 @@ Library ------- +- Issue #17826: setting an iterable side_effect on a mock function created by + create_autospec now works. Patch by Kushal Das. + - Issue #7776: Fix ``Host:`` header and reconnection when using http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:10:18 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 22:10:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <3g71CZ6hbLz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/4681e871c36c changeset: 90284:4681e871c36c parent: 90282:267fff541bda parent: 90283:1e3c64470629 user: Michael Foord date: Mon Apr 14 16:10:02 2014 -0400 summary: Merge files: Lib/unittest/mock.py | 15 ++++++++++- Lib/unittest/test/testmock/testmock.py | 18 ++++++++++++++ Misc/NEWS | 3 ++ 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -343,7 +343,14 @@ value._mock_name = name return True - +# Internal class to identify if we wrapped an iterator object or not. +class _MockIter(object): + def __init__(self, obj): + self.obj = iter(obj) + def __iter__(self): + return self + def __next__(self): + return next(self.obj) class Base(object): _mock_return_value = DEFAULT @@ -495,7 +502,11 @@ delegated = self._mock_delegate if delegated is None: return self._mock_side_effect - return delegated.side_effect + sf = delegated.side_effect + if sf is not None and not callable(sf) and not isinstance(sf, _MockIter): + sf = _MockIter(sf) + delegated.side_effect = sf + return sf def __set_side_effect(self, value): value = _try_iter(value) diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -154,6 +154,24 @@ mock = Mock(side_effect=side_effect, return_value=sentinel.RETURN) self.assertEqual(mock(), sentinel.RETURN) + def test_autospec_side_effect(self): + # Test for issue17826 + results = [1, 2, 3] + def effect(): + return results.pop() + def f(): + pass + + mock = create_autospec(f) + mock.side_effect = [1, 2, 3] + self.assertEqual([mock(), mock(), mock()], [1, 2, 3], + "side effect not used correctly in create_autospec") + # Test where side effect is a callable + results = [1, 2, 3] + mock = create_autospec(f) + mock.side_effect = effect + self.assertEqual([mock(), mock(), mock()], [3, 2, 1], + "callable side effect not used correctly") @unittest.skipUnless('java' in sys.platform, 'This test only applies to Jython') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,9 @@ Library ------- +- Issue #17826: setting an iterable side_effect on a mock function created by + create_autospec now works. Patch by Kushal Das. + - Issue #7776: Fix ``Host:`` header and reconnection when using http.client.HTTPConnection.set_tunnel(). Patch by Nikolaus Rath. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:17:45 2014 From: python-checkins at python.org (terry.reedy) Date: Mon, 14 Apr 2014 22:17:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyMDk1?= =?utf-8?q?6=3A_2=2E7_tokenize_does_not_produce_named_tuples=2E_Patch_by_S?= =?utf-8?q?am_Kimbrel=2E?= Message-ID: <3g71N93XFKz7LjS@mail.python.org> http://hg.python.org/cpython/rev/d4f5a88b94b4 changeset: 90285:d4f5a88b94b4 branch: 2.7 parent: 90269:ea677e26dbeb user: Terry Jan Reedy date: Mon Apr 14 16:17:09 2014 -0400 summary: Closes #20956: 2.7 tokenize does not produce named tuples. Patch by Sam Kimbrel. files: Doc/library/tokenize.rst | 7 ++++--- Misc/ACKS | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -17,9 +17,10 @@ To simplify token stream handling, all :ref:`operators` and :ref:`delimiters` tokens are returned using the generic :data:`token.OP` token type. The exact -type can be determined by checking the token ``string`` field on the -:term:`named tuple` returned from :func:`tokenize.tokenize` for the character -sequence that identifies a specific operator token. +type can be determined by checking the second field (containing the actual +token string matched) of the tuple returned from +:func:`tokenize.generate_tokens` for the character sequence that identifies a +specific operator token. The primary entry point is a :term:`generator`: diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -670,6 +670,7 @@ Jason Killen Jan Kim Taek Joo Kim +Sam Kimbrel W. Trevor King Paul Kippes Steve Kirsch -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:25:32 2014 From: python-checkins at python.org (michael.foord) Date: Mon, 14 Apr 2014 22:25:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_issue_17660=2E_You_?= =?utf-8?q?no_longer_need_to_explicitly_pass_create=3DTrue_when?= Message-ID: <3g71Y80937z7Ljd@mail.python.org> http://hg.python.org/cpython/rev/e457de60028c changeset: 90286:e457de60028c parent: 90284:4681e871c36c user: Michael Foord date: Mon Apr 14 16:25:20 2014 -0400 summary: Closes issue 17660. You no longer need to explicitly pass create=True when patching builtin names. files: Doc/library/unittest.mock.rst | 25 +++++++++++- Lib/unittest/mock.py | 7 +++ Lib/unittest/test/testmock/testpatch.py | 24 +++++++++++- 3 files changed, 52 insertions(+), 4 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -1031,6 +1031,12 @@ default because it can be dangerous. With it switched on you can write passing tests against APIs that don't actually exist! + .. note:: + + .. versionchanged:: 3.5 + If you are patching builtins in a module then you don't + need to pass `create=True`, it will be added by default. + Patch can be used as a `TestCase` class decorator. It works by decorating each test method in the class. This reduces the boilerplate code when your test methods share a common patchings set. `patch` finds @@ -1401,6 +1407,21 @@ Stop all active patches. Only stops patches started with `start`. +.. patch-builtins: + +patch builtins +~~~~~~~~~~~~~~~ +You can patch any builtins within a module. The following example patches +builtin `ord`: + + >>> @patch('__main__.ord') + ... def test(mock_ord): + ... mock_ord.return_value = 101 + ... print(ord('c')) + ... + >>> test() + 101 + TEST_PREFIX ~~~~~~~~~~~ @@ -2011,7 +2032,7 @@ enough that a helper function is useful. >>> m = mock_open() - >>> with patch('__main__.open', m, create=True): + >>> with patch('__main__.open', m): ... with open('foo', 'w') as h: ... h.write('some stuff') ... @@ -2026,7 +2047,7 @@ And for reading files: - >>> with patch('__main__.open', mock_open(read_data='bibble'), create=True) as m: + >>> with patch('__main__.open', mock_open(read_data='bibble')) as m: ... with open('foo') as h: ... result = h.read() ... diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -27,9 +27,13 @@ import inspect import pprint import sys +import builtins +from types import ModuleType from functools import wraps, partial +_builtins = {name for name in dir(builtins) if not name.startswith('_')} + BaseExceptions = (BaseException,) if 'java' in sys.platform: # jython @@ -1166,6 +1170,9 @@ else: local = True + if name in _builtins and isinstance(target, ModuleType): + self.create = True + if not self.create and original is DEFAULT: raise AttributeError( "%s does not have the attribute %r" % (target, name) diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -377,7 +377,7 @@ def test_patchobject_wont_create_by_default(self): try: - @patch.object(SomeClass, 'frooble', sentinel.Frooble) + @patch.object(SomeClass, 'ord', sentinel.Frooble) def test(): self.fail('Patching non existent attributes should fail') @@ -386,7 +386,27 @@ pass else: self.fail('Patching non existent attributes should fail') - self.assertFalse(hasattr(SomeClass, 'frooble')) + self.assertFalse(hasattr(SomeClass, 'ord')) + + + def test_patch_builtins_without_create(self): + @patch(__name__+'.ord') + def test_ord(mock_ord): + mock_ord.return_value = 101 + return ord('c') + + @patch(__name__+'.open') + def test_open(mock_open): + m = mock_open.return_value + m.read.return_value = 'abcd' + + fobj = open('doesnotexists.txt') + data = fobj.read() + fobj.close() + return data + + self.assertEqual(test_ord(), 101) + self.assertEqual(test_open(), 'abcd') def test_patch_with_static_methods(self): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:32:45 2014 From: python-checkins at python.org (gregory.p.smith) Date: Mon, 14 Apr 2014 22:32:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_conditiona?= =?utf-8?q?l_code_for_android=27s_lack_of_definition_of_SYS=5Fgetdent64=2E?= Message-ID: <3g71jT6VBJz7Ljx@mail.python.org> http://hg.python.org/cpython/rev/211eeb97b352 changeset: 90287:211eeb97b352 branch: 3.4 parent: 90283:1e3c64470629 user: Gregory P. Smith date: Mon Apr 14 13:31:21 2014 -0700 summary: Add conditional code for android's lack of definition of SYS_getdent64. Fixes issue20307. No Misc/NEWS entry because frankly this is an esoteric platform for anyone to be figuring out how to cross compile CPython for. files: Modules/_posixsubprocess.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -18,6 +18,12 @@ #include #endif +#if defined(__ANDROID__) && !defined(SYS_getdents64) +/* Android doesn't expose syscalls, add the definition manually. */ +# include +# define SYS_getdents64 __NR_getdents64 +#endif + #if defined(sun) /* readdir64 is used to work around Solaris 9 bug 6395699. */ # define readdir readdir64 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:32:47 2014 From: python-checkins at python.org (gregory.p.smith) Date: Mon, 14 Apr 2014 22:32:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Add_conditional_code_for_android=27s_lack_of_definition_?= =?utf-8?q?of_SYS=5Fgetdent64=2E?= Message-ID: <3g71jW19j0z7Lk4@mail.python.org> http://hg.python.org/cpython/rev/9f89958ded0a changeset: 90288:9f89958ded0a parent: 90284:4681e871c36c parent: 90287:211eeb97b352 user: Gregory P. Smith date: Mon Apr 14 13:31:55 2014 -0700 summary: Add conditional code for android's lack of definition of SYS_getdent64. Fixes issue20307. No Misc/NEWS entry because frankly this is an esoteric platform for anyone to be figuring out how to cross compile CPython for. files: Modules/_posixsubprocess.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -18,6 +18,12 @@ #include #endif +#if defined(__ANDROID__) && !defined(SYS_getdents64) +/* Android doesn't expose syscalls, add the definition manually. */ +# include +# define SYS_getdents64 __NR_getdents64 +#endif + #if defined(sun) /* readdir64 is used to work around Solaris 9 bug 6395699. */ # define readdir readdir64 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:32:48 2014 From: python-checkins at python.org (gregory.p.smith) Date: Mon, 14 Apr 2014 22:32:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <3g71jX3fGLz7Lk0@mail.python.org> http://hg.python.org/cpython/rev/8ab6eedf3a18 changeset: 90289:8ab6eedf3a18 parent: 90288:9f89958ded0a parent: 90286:e457de60028c user: Gregory P. Smith date: Mon Apr 14 13:32:35 2014 -0700 summary: merge heads files: Doc/library/unittest.mock.rst | 25 +++++++++++- Lib/unittest/mock.py | 7 +++ Lib/unittest/test/testmock/testpatch.py | 24 +++++++++++- 3 files changed, 52 insertions(+), 4 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -1031,6 +1031,12 @@ default because it can be dangerous. With it switched on you can write passing tests against APIs that don't actually exist! + .. note:: + + .. versionchanged:: 3.5 + If you are patching builtins in a module then you don't + need to pass `create=True`, it will be added by default. + Patch can be used as a `TestCase` class decorator. It works by decorating each test method in the class. This reduces the boilerplate code when your test methods share a common patchings set. `patch` finds @@ -1401,6 +1407,21 @@ Stop all active patches. Only stops patches started with `start`. +.. patch-builtins: + +patch builtins +~~~~~~~~~~~~~~~ +You can patch any builtins within a module. The following example patches +builtin `ord`: + + >>> @patch('__main__.ord') + ... def test(mock_ord): + ... mock_ord.return_value = 101 + ... print(ord('c')) + ... + >>> test() + 101 + TEST_PREFIX ~~~~~~~~~~~ @@ -2011,7 +2032,7 @@ enough that a helper function is useful. >>> m = mock_open() - >>> with patch('__main__.open', m, create=True): + >>> with patch('__main__.open', m): ... with open('foo', 'w') as h: ... h.write('some stuff') ... @@ -2026,7 +2047,7 @@ And for reading files: - >>> with patch('__main__.open', mock_open(read_data='bibble'), create=True) as m: + >>> with patch('__main__.open', mock_open(read_data='bibble')) as m: ... with open('foo') as h: ... result = h.read() ... diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -27,9 +27,13 @@ import inspect import pprint import sys +import builtins +from types import ModuleType from functools import wraps, partial +_builtins = {name for name in dir(builtins) if not name.startswith('_')} + BaseExceptions = (BaseException,) if 'java' in sys.platform: # jython @@ -1166,6 +1170,9 @@ else: local = True + if name in _builtins and isinstance(target, ModuleType): + self.create = True + if not self.create and original is DEFAULT: raise AttributeError( "%s does not have the attribute %r" % (target, name) diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -377,7 +377,7 @@ def test_patchobject_wont_create_by_default(self): try: - @patch.object(SomeClass, 'frooble', sentinel.Frooble) + @patch.object(SomeClass, 'ord', sentinel.Frooble) def test(): self.fail('Patching non existent attributes should fail') @@ -386,7 +386,27 @@ pass else: self.fail('Patching non existent attributes should fail') - self.assertFalse(hasattr(SomeClass, 'frooble')) + self.assertFalse(hasattr(SomeClass, 'ord')) + + + def test_patch_builtins_without_create(self): + @patch(__name__+'.ord') + def test_ord(mock_ord): + mock_ord.return_value = 101 + return ord('c') + + @patch(__name__+'.open') + def test_open(mock_open): + m = mock_open.return_value + m.read.return_value = 'abcd' + + fobj = open('doesnotexists.txt') + data = fobj.read() + fobj.close() + return data + + self.assertEqual(test_ord(), 101) + self.assertEqual(test_open(), 'abcd') def test_patch_with_static_methods(self): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:44:01 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 22:44:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEzNTk4?= =?utf-8?q?=3A_Add_auto-numbering_of_replacement_fields_to_string=2EFormat?= =?utf-8?q?ter=2E?= Message-ID: <3g71yT4tS9z7Ljq@mail.python.org> http://hg.python.org/cpython/rev/ad74229a6fba changeset: 90290:ad74229a6fba branch: 3.4 parent: 90287:211eeb97b352 user: Eric V. Smith date: Mon Apr 14 16:43:50 2014 -0400 summary: Issue #13598: Add auto-numbering of replacement fields to string.Formatter. files: Lib/string.py | 23 +++++++++++++++++++++-- Lib/test/test_string.py | 17 +++++++++++++++++ Misc/NEWS | 4 ++++ 3 files changed, 42 insertions(+), 2 deletions(-) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -169,7 +169,8 @@ self.check_unused_args(used_args, args, kwargs) return result - def _vformat(self, format_string, args, kwargs, used_args, recursion_depth): + def _vformat(self, format_string, args, kwargs, used_args, recursion_depth, + auto_arg_index=0): if recursion_depth < 0: raise ValueError('Max string recursion exceeded') result = [] @@ -185,6 +186,23 @@ # this is some markup, find the object and do # the formatting + # handle arg indexing when empty field_names are given. + if field_name == '': + if auto_arg_index is False: + raise ValueError('cannot switch from manual field ' + 'specification to automatic field ' + 'numbering') + field_name = str(auto_arg_index) + auto_arg_index += 1 + elif field_name.isdigit(): + if auto_arg_index: + raise ValueError('cannot switch from manual field ' + 'specification to automatic field ' + 'numbering') + # disable auto arg incrementing, if it gets + # used later on, then an exception will be raised + auto_arg_index = False + # given the field_name, find the object it references # and the argument it came from obj, arg_used = self.get_field(field_name, args, kwargs) @@ -195,7 +213,8 @@ # expand the format spec, if needed format_spec = self._vformat(format_spec, args, kwargs, - used_args, recursion_depth-1) + used_args, recursion_depth-1, + auto_arg_index=auto_arg_index) # format the object and append to the result result.append(self.format_field(obj, format_spec)) diff --git a/Lib/test/test_string.py b/Lib/test/test_string.py --- a/Lib/test/test_string.py +++ b/Lib/test/test_string.py @@ -32,6 +32,23 @@ self.assertEqual(fmt.format("foo{0}", "bar"), "foobar") self.assertEqual(fmt.format("foo{1}{0}-{1}", "bar", 6), "foo6bar-6") + def test_auto_numbering(self): + fmt = string.Formatter() + self.assertEqual(fmt.format('foo{}{}', 'bar', 6), + 'foo{}{}'.format('bar', 6)) + self.assertEqual(fmt.format('foo{1}{num}{1}', None, 'bar', num=6), + 'foo{1}{num}{1}'.format(None, 'bar', num=6)) + self.assertEqual(fmt.format('{:^{}}', 'bar', 6), + '{:^{}}'.format('bar', 6)) + self.assertEqual(fmt.format('{:^{pad}}{}', 'foo', 'bar', pad=6), + '{:^{pad}}{}'.format('foo', 'bar', pad=6)) + + with self.assertRaises(ValueError): + fmt.format('foo{1}{}', 'bar', 6) + + with self.assertRaises(ValueError): + fmt.format('foo{}{1}', 'bar', 6) + def test_conversion_specifiers(self): fmt = string.Formatter() self.assertEqual(fmt.format("-{arg!r}-", arg='test'), "-'test'-") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,6 +30,10 @@ - Issue #12546: Allow \x00 to be used as a fill character when using str, int, float, and complex __format__ methods. +- Issue #13598: Modify string.Formatter to support auto-numbering of + replacement fields. It now matches the behavior of str.format() in + this regard. + Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:47:01 2014 From: python-checkins at python.org (eric.smith) Date: Mon, 14 Apr 2014 22:47:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEzNTk4?= =?utf-8?q?=3A_Added_acknowledgements_to_Misc/NEWS=2E?= Message-ID: <3g721x3dxpz7LjX@mail.python.org> http://hg.python.org/cpython/rev/50fe497983fd changeset: 90291:50fe497983fd branch: 3.4 user: Eric V. Smith date: Mon Apr 14 16:46:52 2014 -0400 summary: Issue #13598: Added acknowledgements to Misc/NEWS. files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -32,7 +32,7 @@ - Issue #13598: Modify string.Formatter to support auto-numbering of replacement fields. It now matches the behavior of str.format() in - this regard. + this regard. Patches by Phil Elson and Ramchandra Apte. Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:17 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Convert_urllib?= =?utf-8?q?=2Erequest_parse=5Fproxy_doctests_to_unittests=2E?= Message-ID: <3g724Y1KDWz7LjX@mail.python.org> http://hg.python.org/cpython/rev/29d07cd3d91f changeset: 90292:29d07cd3d91f branch: 3.4 parent: 90283:1e3c64470629 user: Senthil Kumaran date: Mon Apr 14 16:32:20 2014 -0400 summary: Convert urllib.request parse_proxy doctests to unittests. files: Lib/test/test_urllib2.py | 39 +++++++++++++++++++++++- Lib/urllib/request.py | 45 +--------------------------- 2 files changed, 39 insertions(+), 45 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -10,7 +10,7 @@ import urllib.request # The proxy bypass method imported below has logic specific to the OSX # proxy config data structure but is testable on all platforms. -from urllib.request import Request, OpenerDirector, _proxy_bypass_macosx_sysconf +from urllib.request import Request, OpenerDirector, _parse_proxy, _proxy_bypass_macosx_sysconf from urllib.parse import urlparse import urllib.error @@ -1467,6 +1467,43 @@ expected_errmsg = 'HTTP Error %s: %s' % (err.code, err.msg) self.assertEqual(str(err), expected_errmsg) + def test_parse_proxy(self): + parse_proxy_test_cases = [ + ('proxy.example.com', + (None, None, None, 'proxy.example.com')), + ('proxy.example.com:3128', + (None, None, None, 'proxy.example.com:3128')), + ('proxy.example.com', (None, None, None, 'proxy.example.com')), + ('proxy.example.com:3128', + (None, None, None, 'proxy.example.com:3128')), + # The authority component may optionally include userinfo + # (assumed to be # username:password): + ('joe:password at proxy.example.com', + (None, 'joe', 'password', 'proxy.example.com')), + ('joe:password at proxy.example.com:3128', + (None, 'joe', 'password', 'proxy.example.com:3128')), + #Examples with URLS + ('http://proxy.example.com/', + ('http', None, None, 'proxy.example.com')), + ('http://proxy.example.com:3128/', + ('http', None, None, 'proxy.example.com:3128')), + ('http://joe:password at proxy.example.com/', + ('http', 'joe', 'password', 'proxy.example.com')), + ('http://joe:password at proxy.example.com:3128', + ('http', 'joe', 'password', 'proxy.example.com:3128')), + # Everything after the authority is ignored + ('ftp://joe:password at proxy.example.com/rubbish:3128', + ('ftp', 'joe', 'password', 'proxy.example.com')), + # Test for no trailing '/' case + ('http://joe:password at proxy.example.com', + ('http', 'joe', 'password', 'proxy.example.com')) + ] + + for tc, expected in parse_proxy_test_cases: + self.assertEqual(_parse_proxy(tc), expected) + + self.assertRaises(ValueError, _parse_proxy, 'file:/ftp.example.com'), + class RequestTests(unittest.TestCase): class PutRequest(Request): method='PUT' diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -687,50 +687,7 @@ If a URL is supplied, it must have an authority (host:port) component. According to RFC 3986, having an authority component means the URL must - have two slashes after the scheme: - - >>> _parse_proxy('file:/ftp.example.com/') - Traceback (most recent call last): - ValueError: proxy URL with no authority: 'file:/ftp.example.com/' - - The first three items of the returned tuple may be None. - - Examples of authority parsing: - - >>> _parse_proxy('proxy.example.com') - (None, None, None, 'proxy.example.com') - >>> _parse_proxy('proxy.example.com:3128') - (None, None, None, 'proxy.example.com:3128') - - The authority component may optionally include userinfo (assumed to be - username:password): - - >>> _parse_proxy('joe:password at proxy.example.com') - (None, 'joe', 'password', 'proxy.example.com') - >>> _parse_proxy('joe:password at proxy.example.com:3128') - (None, 'joe', 'password', 'proxy.example.com:3128') - - Same examples, but with URLs instead: - - >>> _parse_proxy('http://proxy.example.com/') - ('http', None, None, 'proxy.example.com') - >>> _parse_proxy('http://proxy.example.com:3128/') - ('http', None, None, 'proxy.example.com:3128') - >>> _parse_proxy('http://joe:password at proxy.example.com/') - ('http', 'joe', 'password', 'proxy.example.com') - >>> _parse_proxy('http://joe:password at proxy.example.com:3128') - ('http', 'joe', 'password', 'proxy.example.com:3128') - - Everything after the authority is ignored: - - >>> _parse_proxy('ftp://joe:password at proxy.example.com/rubbish:3128') - ('ftp', 'joe', 'password', 'proxy.example.com') - - Test for no trailing '/' case: - - >>> _parse_proxy('http://joe:password at proxy.example.com') - ('http', 'joe', 'password', 'proxy.example.com') - + have two slashes after the scheme. """ scheme, r_scheme = splittype(proxy) if not r_scheme.startswith("/"): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:18 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g724Z3pJ5z7Ljl@mail.python.org> http://hg.python.org/cpython/rev/f769925b066f changeset: 90293:f769925b066f parent: 90286:e457de60028c parent: 90292:29d07cd3d91f user: Senthil Kumaran date: Mon Apr 14 16:33:21 2014 -0400 summary: merge 3.4 Convert urllib.request parse_proxy doctests to unittests. files: Lib/test/test_urllib2.py | 39 +++++++++++++++++++++++- Lib/urllib/request.py | 45 +--------------------------- 2 files changed, 39 insertions(+), 45 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -10,7 +10,7 @@ import urllib.request # The proxy bypass method imported below has logic specific to the OSX # proxy config data structure but is testable on all platforms. -from urllib.request import Request, OpenerDirector, _proxy_bypass_macosx_sysconf +from urllib.request import Request, OpenerDirector, _parse_proxy, _proxy_bypass_macosx_sysconf from urllib.parse import urlparse import urllib.error @@ -1467,6 +1467,43 @@ expected_errmsg = 'HTTP Error %s: %s' % (err.code, err.msg) self.assertEqual(str(err), expected_errmsg) + def test_parse_proxy(self): + parse_proxy_test_cases = [ + ('proxy.example.com', + (None, None, None, 'proxy.example.com')), + ('proxy.example.com:3128', + (None, None, None, 'proxy.example.com:3128')), + ('proxy.example.com', (None, None, None, 'proxy.example.com')), + ('proxy.example.com:3128', + (None, None, None, 'proxy.example.com:3128')), + # The authority component may optionally include userinfo + # (assumed to be # username:password): + ('joe:password at proxy.example.com', + (None, 'joe', 'password', 'proxy.example.com')), + ('joe:password at proxy.example.com:3128', + (None, 'joe', 'password', 'proxy.example.com:3128')), + #Examples with URLS + ('http://proxy.example.com/', + ('http', None, None, 'proxy.example.com')), + ('http://proxy.example.com:3128/', + ('http', None, None, 'proxy.example.com:3128')), + ('http://joe:password at proxy.example.com/', + ('http', 'joe', 'password', 'proxy.example.com')), + ('http://joe:password at proxy.example.com:3128', + ('http', 'joe', 'password', 'proxy.example.com:3128')), + # Everything after the authority is ignored + ('ftp://joe:password at proxy.example.com/rubbish:3128', + ('ftp', 'joe', 'password', 'proxy.example.com')), + # Test for no trailing '/' case + ('http://joe:password at proxy.example.com', + ('http', 'joe', 'password', 'proxy.example.com')) + ] + + for tc, expected in parse_proxy_test_cases: + self.assertEqual(_parse_proxy(tc), expected) + + self.assertRaises(ValueError, _parse_proxy, 'file:/ftp.example.com'), + class RequestTests(unittest.TestCase): class PutRequest(Request): method='PUT' diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -687,50 +687,7 @@ If a URL is supplied, it must have an authority (host:port) component. According to RFC 3986, having an authority component means the URL must - have two slashes after the scheme: - - >>> _parse_proxy('file:/ftp.example.com/') - Traceback (most recent call last): - ValueError: proxy URL with no authority: 'file:/ftp.example.com/' - - The first three items of the returned tuple may be None. - - Examples of authority parsing: - - >>> _parse_proxy('proxy.example.com') - (None, None, None, 'proxy.example.com') - >>> _parse_proxy('proxy.example.com:3128') - (None, None, None, 'proxy.example.com:3128') - - The authority component may optionally include userinfo (assumed to be - username:password): - - >>> _parse_proxy('joe:password at proxy.example.com') - (None, 'joe', 'password', 'proxy.example.com') - >>> _parse_proxy('joe:password at proxy.example.com:3128') - (None, 'joe', 'password', 'proxy.example.com:3128') - - Same examples, but with URLs instead: - - >>> _parse_proxy('http://proxy.example.com/') - ('http', None, None, 'proxy.example.com') - >>> _parse_proxy('http://proxy.example.com:3128/') - ('http', None, None, 'proxy.example.com:3128') - >>> _parse_proxy('http://joe:password at proxy.example.com/') - ('http', 'joe', 'password', 'proxy.example.com') - >>> _parse_proxy('http://joe:password at proxy.example.com:3128') - ('http', 'joe', 'password', 'proxy.example.com:3128') - - Everything after the authority is ignored: - - >>> _parse_proxy('ftp://joe:password at proxy.example.com/rubbish:3128') - ('ftp', 'joe', 'password', 'proxy.example.com') - - Test for no trailing '/' case: - - >>> _parse_proxy('http://joe:password at proxy.example.com') - ('http', 'joe', 'password', 'proxy.example.com') - + have two slashes after the scheme. """ scheme, r_scheme = splittype(proxy) if not r_scheme.startswith("/"): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:19 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <3g724b59mnz7Lk3@mail.python.org> http://hg.python.org/cpython/rev/886bf974c079 changeset: 90294:886bf974c079 parent: 90293:f769925b066f parent: 90289:8ab6eedf3a18 user: Senthil Kumaran date: Mon Apr 14 16:37:57 2014 -0400 summary: merge heads files: Modules/_posixsubprocess.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -18,6 +18,12 @@ #include #endif +#if defined(__ANDROID__) && !defined(SYS_getdents64) +/* Android doesn't expose syscalls, add the definition manually. */ +# include +# define SYS_getdents64 __NR_getdents64 +#endif + #if defined(sun) /* readdir64 is used to work around Solaris 9 bug 6395699. */ # define readdir readdir64 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:21 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNCk6?= =?utf-8?q?_merge_heads?= Message-ID: <3g724d0MwXz7LkG@mail.python.org> http://hg.python.org/cpython/rev/a153cd9b2e16 changeset: 90295:a153cd9b2e16 branch: 3.4 parent: 90292:29d07cd3d91f parent: 90287:211eeb97b352 user: Senthil Kumaran date: Mon Apr 14 16:43:58 2014 -0400 summary: merge heads files: Modules/_posixsubprocess.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -18,6 +18,12 @@ #include #endif +#if defined(__ANDROID__) && !defined(SYS_getdents64) +/* Android doesn't expose syscalls, add the definition manually. */ +# include +# define SYS_getdents64 __NR_getdents64 +#endif + #if defined(sun) /* readdir64 is used to work around Solaris 9 bug 6395699. */ # define readdir readdir64 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:22 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <3g724f2T7bz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/4c30718c9763 changeset: 90296:4c30718c9763 parent: 90294:886bf974c079 parent: 90295:a153cd9b2e16 user: Senthil Kumaran date: Mon Apr 14 16:44:27 2014 -0400 summary: merge heads files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:23 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNCk6?= =?utf-8?q?_merge_heads?= Message-ID: <3g724g54D1z7Ljy@mail.python.org> http://hg.python.org/cpython/rev/83417ca91956 changeset: 90297:83417ca91956 branch: 3.4 parent: 90295:a153cd9b2e16 parent: 90290:ad74229a6fba user: Senthil Kumaran date: Mon Apr 14 16:45:49 2014 -0400 summary: merge heads files: Lib/string.py | 23 +++++++++++++++++++++-- Lib/test/test_string.py | 17 +++++++++++++++++ Misc/NEWS | 4 ++++ 3 files changed, 42 insertions(+), 2 deletions(-) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -169,7 +169,8 @@ self.check_unused_args(used_args, args, kwargs) return result - def _vformat(self, format_string, args, kwargs, used_args, recursion_depth): + def _vformat(self, format_string, args, kwargs, used_args, recursion_depth, + auto_arg_index=0): if recursion_depth < 0: raise ValueError('Max string recursion exceeded') result = [] @@ -185,6 +186,23 @@ # this is some markup, find the object and do # the formatting + # handle arg indexing when empty field_names are given. + if field_name == '': + if auto_arg_index is False: + raise ValueError('cannot switch from manual field ' + 'specification to automatic field ' + 'numbering') + field_name = str(auto_arg_index) + auto_arg_index += 1 + elif field_name.isdigit(): + if auto_arg_index: + raise ValueError('cannot switch from manual field ' + 'specification to automatic field ' + 'numbering') + # disable auto arg incrementing, if it gets + # used later on, then an exception will be raised + auto_arg_index = False + # given the field_name, find the object it references # and the argument it came from obj, arg_used = self.get_field(field_name, args, kwargs) @@ -195,7 +213,8 @@ # expand the format spec, if needed format_spec = self._vformat(format_spec, args, kwargs, - used_args, recursion_depth-1) + used_args, recursion_depth-1, + auto_arg_index=auto_arg_index) # format the object and append to the result result.append(self.format_field(obj, format_spec)) diff --git a/Lib/test/test_string.py b/Lib/test/test_string.py --- a/Lib/test/test_string.py +++ b/Lib/test/test_string.py @@ -32,6 +32,23 @@ self.assertEqual(fmt.format("foo{0}", "bar"), "foobar") self.assertEqual(fmt.format("foo{1}{0}-{1}", "bar", 6), "foo6bar-6") + def test_auto_numbering(self): + fmt = string.Formatter() + self.assertEqual(fmt.format('foo{}{}', 'bar', 6), + 'foo{}{}'.format('bar', 6)) + self.assertEqual(fmt.format('foo{1}{num}{1}', None, 'bar', num=6), + 'foo{1}{num}{1}'.format(None, 'bar', num=6)) + self.assertEqual(fmt.format('{:^{}}', 'bar', 6), + '{:^{}}'.format('bar', 6)) + self.assertEqual(fmt.format('{:^{pad}}{}', 'foo', 'bar', pad=6), + '{:^{pad}}{}'.format('foo', 'bar', pad=6)) + + with self.assertRaises(ValueError): + fmt.format('foo{1}{}', 'bar', 6) + + with self.assertRaises(ValueError): + fmt.format('foo{}{1}', 'bar', 6) + def test_conversion_specifiers(self): fmt = string.Formatter() self.assertEqual(fmt.format("-{arg!r}-", arg='test'), "-'test'-") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,6 +30,10 @@ - Issue #12546: Allow \x00 to be used as a fill character when using str, int, float, and complex __format__ methods. +- Issue #13598: Modify string.Formatter to support auto-numbering of + replacement fields. It now matches the behavior of str.format() in + this regard. + Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:25 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <3g724j0wSnz7Ljq@mail.python.org> http://hg.python.org/cpython/rev/8a0764a29585 changeset: 90298:8a0764a29585 parent: 90296:4c30718c9763 parent: 90297:83417ca91956 user: Senthil Kumaran date: Mon Apr 14 16:47:05 2014 -0400 summary: merge heads files: Lib/string.py | 23 +++++++++++++++++++++-- Lib/test/test_string.py | 17 +++++++++++++++++ Misc/NEWS | 4 ++++ 3 files changed, 42 insertions(+), 2 deletions(-) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -169,7 +169,8 @@ self.check_unused_args(used_args, args, kwargs) return result - def _vformat(self, format_string, args, kwargs, used_args, recursion_depth): + def _vformat(self, format_string, args, kwargs, used_args, recursion_depth, + auto_arg_index=0): if recursion_depth < 0: raise ValueError('Max string recursion exceeded') result = [] @@ -185,6 +186,23 @@ # this is some markup, find the object and do # the formatting + # handle arg indexing when empty field_names are given. + if field_name == '': + if auto_arg_index is False: + raise ValueError('cannot switch from manual field ' + 'specification to automatic field ' + 'numbering') + field_name = str(auto_arg_index) + auto_arg_index += 1 + elif field_name.isdigit(): + if auto_arg_index: + raise ValueError('cannot switch from manual field ' + 'specification to automatic field ' + 'numbering') + # disable auto arg incrementing, if it gets + # used later on, then an exception will be raised + auto_arg_index = False + # given the field_name, find the object it references # and the argument it came from obj, arg_used = self.get_field(field_name, args, kwargs) @@ -195,7 +213,8 @@ # expand the format spec, if needed format_spec = self._vformat(format_spec, args, kwargs, - used_args, recursion_depth-1) + used_args, recursion_depth-1, + auto_arg_index=auto_arg_index) # format the object and append to the result result.append(self.format_field(obj, format_spec)) diff --git a/Lib/test/test_string.py b/Lib/test/test_string.py --- a/Lib/test/test_string.py +++ b/Lib/test/test_string.py @@ -32,6 +32,23 @@ self.assertEqual(fmt.format("foo{0}", "bar"), "foobar") self.assertEqual(fmt.format("foo{1}{0}-{1}", "bar", 6), "foo6bar-6") + def test_auto_numbering(self): + fmt = string.Formatter() + self.assertEqual(fmt.format('foo{}{}', 'bar', 6), + 'foo{}{}'.format('bar', 6)) + self.assertEqual(fmt.format('foo{1}{num}{1}', None, 'bar', num=6), + 'foo{1}{num}{1}'.format(None, 'bar', num=6)) + self.assertEqual(fmt.format('{:^{}}', 'bar', 6), + '{:^{}}'.format('bar', 6)) + self.assertEqual(fmt.format('{:^{pad}}{}', 'foo', 'bar', pad=6), + '{:^{pad}}{}'.format('foo', 'bar', pad=6)) + + with self.assertRaises(ValueError): + fmt.format('foo{1}{}', 'bar', 6) + + with self.assertRaises(ValueError): + fmt.format('foo{}{1}', 'bar', 6) + def test_conversion_specifiers(self): fmt = string.Formatter() self.assertEqual(fmt.format("-{arg!r}-", arg='test'), "-'test'-") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,10 @@ - Issue #20480: Add ipaddress.reverse_pointer. Patch by Leon Weber. +- Issue #13598: Modify string.Formatter to support auto-numbering of + replacement fields. It now matches the behavior of str.format() in + this regard. + Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:26 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNCk6?= =?utf-8?q?_merge_heads?= Message-ID: <3g724k2rd7z7LkD@mail.python.org> http://hg.python.org/cpython/rev/1574d6314112 changeset: 90299:1574d6314112 branch: 3.4 parent: 90297:83417ca91956 parent: 90291:50fe497983fd user: Senthil Kumaran date: Mon Apr 14 16:48:39 2014 -0400 summary: merge heads files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -32,7 +32,7 @@ - Issue #13598: Modify string.Formatter to support auto-numbering of replacement fields. It now matches the behavior of str.format() in - this regard. + this regard. Patches by Phil Elson and Ramchandra Apte. Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:49:27 2014 From: python-checkins at python.org (senthil.kumaran) Date: Mon, 14 Apr 2014 22:49:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <3g724l4llWz7LkD@mail.python.org> http://hg.python.org/cpython/rev/528234542ff0 changeset: 90300:528234542ff0 parent: 90298:8a0764a29585 parent: 90299:1574d6314112 user: Senthil Kumaran date: Mon Apr 14 16:49:07 2014 -0400 summary: merge heads files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,7 +41,7 @@ - Issue #13598: Modify string.Formatter to support auto-numbering of replacement fields. It now matches the behavior of str.format() in - this regard. + this regard. Patches by Phil Elson and Ramchandra Apte. Library ------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 22:51:38 2014 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 14 Apr 2014 22:51:38 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_temporary_chan?= =?utf-8?q?ge_to_get_better_debugging_from_windows_bots?= Message-ID: <3g727G1DCRz7LjX@mail.python.org> http://hg.python.org/cpython/rev/81abb121dc65 changeset: 90301:81abb121dc65 branch: 2.7 parent: 90285:d4f5a88b94b4 user: Benjamin Peterson date: Mon Apr 14 16:51:29 2014 -0400 summary: temporary change to get better debugging from windows bots files: Tools/buildbot/test.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,3 +1,3 @@ @rem Used by the buildbot "test" step. cd PCbuild -call rt.bat -d -q -uall -rwW %1 %2 %3 %4 %5 %6 %7 %8 %9 +call rt.bat -d -v -uall -rwW %1 %2 %3 %4 %5 %6 %7 %8 %9 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 14 23:18:42 2014 From: python-checkins at python.org (brett.cannon) Date: Mon, 14 Apr 2014 23:18:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Add_note_about_Kushal_Das?= =?utf-8?q?=27_privs?= Message-ID: <3g72kV00hjz7LjZ@mail.python.org> http://hg.python.org/devguide/rev/c14c8a195fec changeset: 686:c14c8a195fec user: Brett Cannon date: Mon Apr 14 17:18:37 2014 -0400 summary: Add note about Kushal Das' privs files: developers.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/developers.rst b/developers.rst --- a/developers.rst +++ b/developers.rst @@ -25,6 +25,9 @@ Permissions History ------------------- +- Kushal Das was given push privileges on Apr 14, 2014 by BAC, for + general patches, on recommendation by Michael Foord. + - Steven d'Aprano was given push privileges on Feb 08 2014 by BAC, for the statistics module, on recommendation by Nick Coghlan. -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Tue Apr 15 00:22:53 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 15 Apr 2014 00:22:53 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE3NDk4OiBEZWZl?= =?utf-8?q?r_SMTPServerDisconnected_errors_until_the_next_command=2E?= Message-ID: <3g748Y35QVz7LjR@mail.python.org> http://hg.python.org/cpython/rev/3c441e9ccf87 changeset: 90302:3c441e9ccf87 branch: 3.4 parent: 90299:1574d6314112 user: R David Murray date: Mon Apr 14 18:21:38 2014 -0400 summary: #17498: Defer SMTPServerDisconnected errors until the next command. Normally an SMTP server will return an error, and smtplib will then issue an RSET to return the connection to the known starting state. Some servers, however, disconnect after issuing certain errors. When we issue the RSET, this would result in raising an SMTPServerDisconnected error, *instead* of returning the error code the user of the library was expecting. This fix makes the internal RSET calls ignore the disconnection so that the error code is returned. The user of the library will then get the SMTPServerDisconnected error the next time they try to talk to the server. Patch by Kushal Das. files: Lib/smtplib.py | 18 +++++++++++++++--- Lib/test/test_smtplib.py | 13 +++++++++++++ Misc/NEWS | 5 +++++ 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/Lib/smtplib.py b/Lib/smtplib.py --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -478,6 +478,18 @@ """SMTP 'rset' command -- resets session.""" return self.docmd("rset") + def _rset(self): + """Internal 'rset' command which ignores any SMTPServerDisconnected error. + + Used internally in the library, since the server disconnected error + should appear to the application when the *next* command is issued, if + we are doing an internal "safety" reset. + """ + try: + self.rset() + except SMTPServerDisconnected: + pass + def noop(self): """SMTP 'noop' command -- doesn't do anything :>""" return self.docmd("noop") @@ -762,7 +774,7 @@ if code == 421: self.close() else: - self.rset() + self._rset() raise SMTPSenderRefused(code, resp, from_addr) senderrs = {} if isinstance(to_addrs, str): @@ -776,14 +788,14 @@ raise SMTPRecipientsRefused(senderrs) if len(senderrs) == len(to_addrs): # the server refused all our recipients - self.rset() + self._rset() raise SMTPRecipientsRefused(senderrs) (code, resp) = self.data(msg) if code != 250: if code == 421: self.close() else: - self.rset() + self._rset() raise SMTPDataError(code, resp) #if we got here then somebody got our mail return senderrs diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -619,6 +619,7 @@ data_response = None rcpt_count = 0 rset_count = 0 + disconnect = 0 def __init__(self, extra_features, *args, **kw): self._extrafeatures = ''.join( @@ -684,6 +685,8 @@ super().smtp_MAIL(arg) else: self.push(self.mail_response) + if self.disconnect: + self.close_when_done() def smtp_RCPT(self, arg): if self.rcpt_response is None: @@ -875,6 +878,16 @@ #TODO: add tests for correct AUTH method fallback now that the #test infrastructure can support it. + # Issue 17498: make sure _rset does not raise SMTPServerDisconnected exception + def test__rest_from_mail_cmd(self): + smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15) + smtp.noop() + self.serv._SMTPchannel.mail_response = '451 Requested action aborted' + self.serv._SMTPchannel.disconnect = True + with self.assertRaises(smtplib.SMTPSenderRefused): + smtp.sendmail('John', 'Sally', 'test message') + self.assertIsNone(smtp.sock) + # Issue 5713: make sure close, not rset, is called if we get a 421 error def test_421_from_mail_cmd(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,11 @@ Library ------- +- Issue #17498: Some SMTP servers disconnect after certain errors, violating + strict RFC conformance. Instead of losing the error code when we issue the + subsequent RSET, smtplib now returns the error code and defers raising the + SMTPServerDisconnected error until the next command is issued. + - Issue #17826: setting an iterable side_effect on a mock function created by create_autospec now works. Patch by Kushal Das. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 00:22:54 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 15 Apr 2014 00:22:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_=2317498=3A_Defer_SMTPServerDisconnected_errors_un?= =?utf-8?q?til_the_next_command=2E?= Message-ID: <3g748Z5ggFz7Ljl@mail.python.org> http://hg.python.org/cpython/rev/842014ab1c06 changeset: 90303:842014ab1c06 parent: 90300:528234542ff0 parent: 90302:3c441e9ccf87 user: R David Murray date: Mon Apr 14 18:22:00 2014 -0400 summary: Merge #17498: Defer SMTPServerDisconnected errors until the next command. files: Lib/smtplib.py | 18 +++++++++++++++--- Lib/test/test_smtplib.py | 13 +++++++++++++ Misc/NEWS | 5 +++++ 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/Lib/smtplib.py b/Lib/smtplib.py --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -478,6 +478,18 @@ """SMTP 'rset' command -- resets session.""" return self.docmd("rset") + def _rset(self): + """Internal 'rset' command which ignores any SMTPServerDisconnected error. + + Used internally in the library, since the server disconnected error + should appear to the application when the *next* command is issued, if + we are doing an internal "safety" reset. + """ + try: + self.rset() + except SMTPServerDisconnected: + pass + def noop(self): """SMTP 'noop' command -- doesn't do anything :>""" return self.docmd("noop") @@ -762,7 +774,7 @@ if code == 421: self.close() else: - self.rset() + self._rset() raise SMTPSenderRefused(code, resp, from_addr) senderrs = {} if isinstance(to_addrs, str): @@ -776,14 +788,14 @@ raise SMTPRecipientsRefused(senderrs) if len(senderrs) == len(to_addrs): # the server refused all our recipients - self.rset() + self._rset() raise SMTPRecipientsRefused(senderrs) (code, resp) = self.data(msg) if code != 250: if code == 421: self.close() else: - self.rset() + self._rset() raise SMTPDataError(code, resp) #if we got here then somebody got our mail return senderrs diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -619,6 +619,7 @@ data_response = None rcpt_count = 0 rset_count = 0 + disconnect = 0 def __init__(self, extra_features, *args, **kw): self._extrafeatures = ''.join( @@ -684,6 +685,8 @@ super().smtp_MAIL(arg) else: self.push(self.mail_response) + if self.disconnect: + self.close_when_done() def smtp_RCPT(self, arg): if self.rcpt_response is None: @@ -875,6 +878,16 @@ #TODO: add tests for correct AUTH method fallback now that the #test infrastructure can support it. + # Issue 17498: make sure _rset does not raise SMTPServerDisconnected exception + def test__rest_from_mail_cmd(self): + smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15) + smtp.noop() + self.serv._SMTPchannel.mail_response = '451 Requested action aborted' + self.serv._SMTPchannel.disconnect = True + with self.assertRaises(smtplib.SMTPSenderRefused): + smtp.sendmail('John', 'Sally', 'test message') + self.assertIsNone(smtp.sock) + # Issue 5713: make sure close, not rset, is called if we get a 421 error def test_421_from_mail_cmd(self): smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,11 @@ Library ------- +- Issue #17498: Some SMTP servers disconnect after certain errors, violating + strict RFC conformance. Instead of losing the error code when we issue the + subsequent RSET, smtplib now returns the error code and defers raising the + SMTPServerDisconnected error until the next command is issued. + - Issue #17826: setting an iterable side_effect on a mock function created by create_autospec now works. Patch by Kushal Das. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 00:54:43 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 15 Apr 2014 00:54:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE1MTA0OiBhZGQg?= =?utf-8?q?backtick_code_markup=2E?= Message-ID: <3g74sH1tPpz7LjS@mail.python.org> http://hg.python.org/cpython/rev/008486e18e90 changeset: 90304:008486e18e90 branch: 3.4 parent: 90302:3c441e9ccf87 user: R David Murray date: Mon Apr 14 18:53:51 2014 -0400 summary: #15104: add backtick code markup. files: Doc/library/__main__.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -5,19 +5,19 @@ .. module:: __main__ :synopsis: The environment where the top-level script is run. -'__main__' is the name of the scope in which top-level code executes. -A module's __name__ is set equal to '__main__' when read from +``'__main__'`` is the name of the scope in which top-level code executes. +A module's __name__ is set equal to ``'__main__'`` when read from standard input, a script, or from an interactive prompt. A module can discover whether or not it is running in the main scope by -checking its own __name__, which allows a common idiom for conditionally -executing code in a module when it is run as a script or with `python --m` but not when it is imported: +checking its own ``__name__``, which allows a common idiom for conditionally +executing code in a module when it is run as a script or with ``python +-m`` but not when it is imported: if __name__ == "__main__": # execute only if run as a script main() For a package, the same effect can be achieved by including a -__main__.py module, the contents of which will be executed when the -module is run with -m. +``__main__.py`` module, the contents of which will be executed when the +module is run with ``-m``. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 00:54:44 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 15 Apr 2014 00:54:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2315104=3A_add_backtick_code_markup=2E?= Message-ID: <3g74sJ3D4wz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/14e874736d3a changeset: 90305:14e874736d3a parent: 90303:842014ab1c06 parent: 90304:008486e18e90 user: R David Murray date: Mon Apr 14 18:54:21 2014 -0400 summary: Merge: #15104: add backtick code markup. files: Doc/library/__main__.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -5,19 +5,19 @@ .. module:: __main__ :synopsis: The environment where the top-level script is run. -'__main__' is the name of the scope in which top-level code executes. -A module's __name__ is set equal to '__main__' when read from +``'__main__'`` is the name of the scope in which top-level code executes. +A module's __name__ is set equal to ``'__main__'`` when read from standard input, a script, or from an interactive prompt. A module can discover whether or not it is running in the main scope by -checking its own __name__, which allows a common idiom for conditionally -executing code in a module when it is run as a script or with `python --m` but not when it is imported: +checking its own ``__name__``, which allows a common idiom for conditionally +executing code in a module when it is run as a script or with ``python +-m`` but not when it is imported: if __name__ == "__main__": # execute only if run as a script main() For a package, the same effect can be achieved by including a -__main__.py module, the contents of which will be executed when the -module is run with -m. +``__main__.py`` module, the contents of which will be executed when the +module is run with ``-m``. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 01:25:42 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 15 Apr 2014 01:25:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backed_out_cha?= =?utf-8?q?ngeset_81abb121dc65?= Message-ID: <3g75Y26127z7Lk8@mail.python.org> http://hg.python.org/cpython/rev/71a0e0c70525 changeset: 90306:71a0e0c70525 branch: 2.7 parent: 90301:81abb121dc65 user: Benjamin Peterson date: Mon Apr 14 19:25:36 2014 -0400 summary: Backed out changeset 81abb121dc65 files: Tools/buildbot/test.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -1,3 +1,3 @@ @rem Used by the buildbot "test" step. cd PCbuild -call rt.bat -d -v -uall -rwW %1 %2 %3 %4 %5 %6 %7 %8 %9 +call rt.bat -d -q -uall -rwW %1 %2 %3 %4 %5 %6 %7 %8 %9 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 01:46:09 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 15 Apr 2014 01:46:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_when_an_except?= =?utf-8?q?ion_is_raised_in_fdopen=2C_never_close_the_fd_=28changing_on_my?= =?utf-8?q?_mind?= Message-ID: <3g760d3x5Fz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/339c79791b65 changeset: 90307:339c79791b65 branch: 2.7 user: Benjamin Peterson date: Mon Apr 14 19:45:46 2014 -0400 summary: when an exception is raised in fdopen, never close the fd (changing on my mind on #21191) files: Doc/library/os.rst | 2 +- Lib/test/test_posix.py | 2 +- Misc/NEWS | 2 +- Modules/posixmodule.c | 41 +++++++++++++++++++---------- 4 files changed, 30 insertions(+), 17 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -465,7 +465,7 @@ Return an open file object connected to the file descriptor *fd*. The *mode* and *bufsize* arguments have the same meaning as the corresponding arguments to the built-in :func:`open` function. If :func:`fdopen` raises an - exception, it closes *fd*. + exception, it leaves *fd* untouched (unclosed). Availability: Unix, Windows. diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -196,7 +196,7 @@ fd = os.open(test_support.TESTFN, os.O_RDONLY) self.assertRaises(OSError, posix.fdopen, fd, 'w') - self.assertRaises(OSError, os.close, fd) # fd should be closed. + os.close(fd) # fd should not be closed. @unittest.skipUnless(hasattr(posix, 'O_EXLOCK'), 'test needs posix.O_EXLOCK') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -51,7 +51,7 @@ - Issue #21172: isinstance check relaxed from dict to collections.Mapping. -- Issue #21191: In os.fdopen, alwyas close the file descriptor when an exception +- Issue #21191: In os.fdopen, never close the file descriptor when an exception happens. - Issue #21149: Improved thread-safety in logging cleanup during interpreter diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -6841,19 +6841,37 @@ /* Sanitize mode. See fileobject.c */ mode = PyMem_MALLOC(strlen(orgmode)+3); if (!mode) { - close(fd); PyErr_NoMemory(); return NULL; } strcpy(mode, orgmode); if (_PyFile_SanitizeMode(mode)) { - close(fd); PyMem_FREE(mode); return NULL; } if (!_PyVerify_fd(fd)) { - posix_error(); - close(fd); + PyMem_FREE(mode); + return posix_error(); + } +#if defined(HAVE_FSTAT) && defined(S_IFDIR) && defined(EISDIR) + { + struct stat buf; + if (fstat(fd, &buf) == 0 && S_ISDIR(buf.st_mode)) { + PyMem_FREE(mode); + char *msg = strerror(EISDIR); + PyObject *exc = PyObject_CallFunction(PyExc_IOError, "(isO)", + EISDIR, msg, ""); + PyErr_SetObject(PyExc_IOError, exc); + Py_XDECREF(exc); + return NULL; + } + } +#endif + /* The dummy filename used here must be kept in sync with the value + tested against in gzip.GzipFile.__init__() - see issue #13781. */ + f = PyFile_FromFile(NULL, "", orgmode, fclose); + if (f == NULL) { + PyMem_FREE(mode); return NULL; } Py_BEGIN_ALLOW_THREADS @@ -6876,16 +6894,11 @@ #endif Py_END_ALLOW_THREADS PyMem_FREE(mode); - if (fp == NULL) { - posix_error(); - close(fd); - return NULL; - } - /* The dummy filename used here must be kept in sync with the value - tested against in gzip.GzipFile.__init__() - see issue #13781. */ - f = PyFile_FromFile(fp, "", orgmode, fclose); - if (f != NULL) - PyFile_SetBufSize(f, bufsize); + if (fp == NULL) + return posix_error(); + /* We now know we will succeed, so initialize the file object. */ + ((PyFileObject *)f)->f_fp = fp; + PyFile_SetBufSize(f, bufsize); return f; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 01:57:58 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 15 Apr 2014 01:57:58 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_add_missing_NU?= =?utf-8?q?LL_check?= Message-ID: <3g76GG4jNqz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/3b988a4e7a55 changeset: 90308:3b988a4e7a55 branch: 2.7 user: Benjamin Peterson date: Mon Apr 14 19:57:52 2014 -0400 summary: add missing NULL check files: Modules/posixmodule.c | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -6856,13 +6856,17 @@ #if defined(HAVE_FSTAT) && defined(S_IFDIR) && defined(EISDIR) { struct stat buf; + const char *msg; + PyObject *exc; if (fstat(fd, &buf) == 0 && S_ISDIR(buf.st_mode)) { PyMem_FREE(mode); - char *msg = strerror(EISDIR); - PyObject *exc = PyObject_CallFunction(PyExc_IOError, "(isO)", - EISDIR, msg, ""); - PyErr_SetObject(PyExc_IOError, exc); - Py_XDECREF(exc); + msg = strerror(EISDIR); + exc = PyObject_CallFunction(PyExc_IOError, "(isO)", + EISDIR, msg, ""); + if (exc) { + PyErr_SetObject(PyExc_IOError, exc); + Py_DECREF(exc); + } return NULL; } } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 02:29:02 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 15 Apr 2014 02:29:02 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2315916=3A_if_there_are_n?= =?utf-8?q?o_docstrings=2C_make_empty_suite=2C_not_an_error=2E?= Message-ID: <3g76y633Rkz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/57fb5441a4aa changeset: 90309:57fb5441a4aa parent: 90305:14e874736d3a user: R David Murray date: Mon Apr 14 20:28:36 2014 -0400 summary: #15916: if there are no docstrings, make empty suite, not an error. This makes doctest work like unittest: if the test case is empty, that just means there are zero tests run, it's not an error. The existing behavior was broken, since it only gave an error if there were *no* docstrings, and zero tests run if there were docstrings but none of them contained tests. So this makes it self-consistent as well. Patch by Glenn Jones. files: Doc/library/doctest.rst | 12 ++------- Doc/whatsnew/3.5.rst | 4 +++ Lib/doctest.py | 9 ------- Lib/test/test_doctest.py | 35 +++++++++++++++------------ Misc/ACKS | 1 + Misc/NEWS | 3 ++ 6 files changed, 30 insertions(+), 34 deletions(-) diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst --- a/Doc/library/doctest.rst +++ b/Doc/library/doctest.rst @@ -1058,15 +1058,9 @@ This function uses the same search technique as :func:`testmod`. - .. note:: - Unlike :func:`testmod` and :class:`DocTestFinder`, this function raises - a :exc:`ValueError` if *module* contains no docstrings. You can prevent - this error by passing a :class:`DocTestFinder` instance as the - *test_finder* argument with its *exclude_empty* keyword argument set - to ``False``:: - - >>> finder = doctest.DocTestFinder(exclude_empty=False) - >>> suite = doctest.DocTestSuite(test_finder=finder) + .. versionchanged:: 3.5 + :func:`DocTestSuite` returns an empty :class:`unittest.TestSuite` if *module* + contains no docstrings instead of raising :exc:`ValueError`. Under the covers, :func:`DocTestSuite` creates a :class:`unittest.TestSuite` out diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -154,6 +154,10 @@ applications where startup time is paramount (contributed by Brett Cannon in :issue:`17621`). +* :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if + *module* contains no docstrings instead of raising :exc:`ValueError` + (contributed by Glenn Jones in :issue:`15916`). + Optimizations ============= diff --git a/Lib/doctest.py b/Lib/doctest.py --- a/Lib/doctest.py +++ b/Lib/doctest.py @@ -2376,15 +2376,6 @@ suite = _DocTestSuite() suite.addTest(SkipDocTestCase(module)) return suite - elif not tests: - # Why do we want to do this? Because it reveals a bug that might - # otherwise be hidden. - # It is probably a bug that this exception is not also raised if the - # number of doctest examples in tests is zero (i.e. if no doctest - # examples were found). However, we should probably not be raising - # an exception at all here, though it is too late to make this change - # for a maintenance release. See also issue #14649. - raise ValueError(module, "has no docstrings") tests.sort() suite = _DocTestSuite() diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -2096,22 +2096,9 @@ >>> suite.run(unittest.TestResult()) - However, if DocTestSuite finds no docstrings, it raises an error: - - >>> try: - ... doctest.DocTestSuite('test.sample_doctest_no_docstrings') - ... except ValueError as e: - ... error = e - - >>> print(error.args[1]) - has no docstrings - - You can prevent this error by passing a DocTestFinder instance with - the `exclude_empty` keyword argument set to False: - - >>> finder = doctest.DocTestFinder(exclude_empty=False) - >>> suite = doctest.DocTestSuite('test.sample_doctest_no_docstrings', - ... test_finder=finder) + The module need not contain any docstrings either: + + >>> suite = doctest.DocTestSuite('test.sample_doctest_no_docstrings') >>> suite.run(unittest.TestResult()) @@ -2121,6 +2108,22 @@ >>> suite.run(unittest.TestResult()) + We can also provide a DocTestFinder: + + >>> finder = doctest.DocTestFinder() + >>> suite = doctest.DocTestSuite('test.sample_doctest', + ... test_finder=finder) + >>> suite.run(unittest.TestResult()) + + + The DocTestFinder need not return any tests: + + >>> finder = doctest.DocTestFinder() + >>> suite = doctest.DocTestSuite('test.sample_doctest_no_docstrings', + ... test_finder=finder) + >>> suite.run(unittest.TestResult()) + + We can supply global variables. If we pass globs, they will be used instead of the module globals. Here we'll pass an empty globals, triggering an extra error: diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -630,6 +630,7 @@ Nicolas Joly Brian K. Jones Evan Jones +Glenn Jones Jeremy Jones Richard Jones Irmen de Jong diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -203,6 +203,9 @@ - Issue #20334: inspect.Signature and inspect.Parameter are now hashable. +- Issue #15916: doctest.DocTestSuite returns an empty unittest.TestSuite instead + of raising ValueError if it finds no tests + IDLE ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 03:27:33 2014 From: python-checkins at python.org (r.david.murray) Date: Tue, 15 Apr 2014 03:27:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_=2321215=3A_update_debian?= =?utf-8?q?/ubuntu_build-dep_instructions=2E?= Message-ID: <3g78Fd0WQrz7Ljs@mail.python.org> http://hg.python.org/devguide/rev/84ccbb961f26 changeset: 687:84ccbb961f26 user: R David Murray date: Mon Apr 14 21:27:22 2014 -0400 summary: #21215: update debian/ubuntu build-dep instructions. This update is from Glenn Jones based on his experience following the instructions while at the PyCon sprints, and also informed by feedback from Barry Warsaw. files: setup.rst | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/setup.rst b/setup.rst --- a/setup.rst +++ b/setup.rst @@ -107,9 +107,13 @@ $ sudo yum install yum-utils $ sudo yum-builddep python3 -Debian, Ubuntu and other ``apt`` based systems:: +Debian, Ubuntu and other ``apt`` based systems, try to get the dependencies for +the Python version that you're working on e.g.:: - $ sudo apt-get build-dep python3 + $ sudo apt-get build-dep python3.4 + +If that package is not available for your system, try reducing the minor +version until you find a package that is available. For Mac OS X systems, it is generally easiest to use the C compiler and other development utilities provided by Apple's Xcode Developer Tools. There are @@ -217,7 +221,7 @@ Windows ''''''' -The readme included in the solution has more details, especially on the +The readme included in the solution has more details, especially on the software needed to resolve the below mentioned build errors. **Python 3.3** and later use Microsoft Visual Studio 2010. You can -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Tue Apr 15 03:32:20 2014 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 15 Apr 2014 03:32:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSW52b2tlIHRlc3Rf?= =?utf-8?q?urllibnet_tests_using_unittest=2Emain_function?= Message-ID: <3g78M86PlWz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/2755403682ad changeset: 90310:2755403682ad branch: 3.4 parent: 90304:008486e18e90 user: Senthil Kumaran date: Mon Apr 14 21:31:41 2014 -0400 summary: Invoke test_urllibnet tests using unittest.main function files: Lib/test/test_urllibnet.py | 10 +++------- 1 files changed, 3 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -10,6 +10,8 @@ import time +support.requires('network') + class URLTimeoutTest(unittest.TestCase): # XXX this test doesn't seem to test anything useful. @@ -206,11 +208,5 @@ " >= total size in %s" % records_repr) -def test_main(): - support.requires('network') - support.run_unittest(URLTimeoutTest, - urlopenNetworkTests, - urlretrieveNetworkTests) - if __name__ == "__main__": - test_main() + unittest.main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 03:32:22 2014 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 15 Apr 2014 03:32:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g78MB0dqmz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/54f1168036f1 changeset: 90311:54f1168036f1 parent: 90309:57fb5441a4aa parent: 90310:2755403682ad user: Senthil Kumaran date: Mon Apr 14 21:32:13 2014 -0400 summary: merge 3.4 files: Lib/test/test_urllibnet.py | 10 +++------- 1 files changed, 3 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -10,6 +10,8 @@ import time +support.requires('network') + class URLTimeoutTest(unittest.TestCase): # XXX this test doesn't seem to test anything useful. @@ -206,11 +208,5 @@ " >= total size in %s" % records_repr) -def test_main(): - support.requires('network') - support.run_unittest(URLTimeoutTest, - urlopenNetworkTests, - urlretrieveNetworkTests) - if __name__ == "__main__": - test_main() + unittest.main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 04:29:47 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 04:29:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpby50YXNr?= =?utf-8?q?s=3A_Fix_CoroWrapper_to_workaround_yield-from_bug_in_CPython_?= =?utf-8?b?PCAzLjQuMQ==?= Message-ID: <3g79dR4KpQz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/0c35d3616df5 changeset: 90312:0c35d3616df5 branch: 3.4 parent: 90310:2755403682ad user: Yury Selivanov date: Mon Apr 14 22:21:52 2014 -0400 summary: asyncio.tasks: Fix CoroWrapper to workaround yield-from bug in CPython < 3.4.1 Closes issue #21209. files: Lib/asyncio/tasks.py | 5 ++- Lib/test/test_asyncio/test_tasks.py | 25 +++++++++++++++++ Misc/NEWS | 3 ++ 3 files changed, 32 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -49,7 +49,10 @@ def __next__(self): return next(self.gen) - def send(self, value): + def send(self, *value): + # We use `*value` because of a bug in CPythons prior + # to 3.4.1. See issue #21209 and test_yield_from_corowrapper + # for details. This workaround should be removed in 3.5.0. return self.gen.send(value) def throw(self, exc): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -1386,6 +1386,31 @@ self.assertRaises(ValueError, self.loop.run_until_complete, asyncio.wait([], loop=self.loop)) + def test_yield_from_corowrapper(self): + old_debug = asyncio.tasks._DEBUG + asyncio.tasks._DEBUG = True + try: + @asyncio.coroutine + def t1(): + return (yield from t2()) + + @asyncio.coroutine + def t2(): + f = asyncio.Future(loop=self.loop) + asyncio.Task(t3(f), loop=self.loop) + return (yield from f) + + @asyncio.coroutine + def t3(f): + f.set_result((1, 2, 3)) + + task = asyncio.Task(t1(), loop=self.loop) + val = self.loop.run_until_complete(task) + self.assertEqual(val, (1, 2, 3)) + finally: + asyncio.tasks._DEBUG = old_debug + + class GatherTestsBase: def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -147,6 +147,9 @@ positional-or-keyword arguments passed as keyword arguments become keyword-only. +- Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug + in yield-from implementation in CPythons prior to 3.4.1. + IDLE ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 04:29:48 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 04:29:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogbWlzYy5uZXdzOiBS?= =?utf-8?q?emove_whitespace?= Message-ID: <3g79dS64Nnz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/2031ff9f13a8 changeset: 90313:2031ff9f13a8 branch: 3.4 user: Yury Selivanov date: Mon Apr 14 22:22:36 2014 -0400 summary: misc.news: Remove whitespace files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -30,7 +30,7 @@ - Issue #12546: Allow \x00 to be used as a fill character when using str, int, float, and complex __format__ methods. -- Issue #13598: Modify string.Formatter to support auto-numbering of +- Issue #13598: Modify string.Formatter to support auto-numbering of replacement fields. It now matches the behavior of str.format() in this regard. Patches by Phil Elson and Ramchandra Apte. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 04:29:50 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 04:29:50 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_syncio=2Etasks=3A_Fix_Coro?= =?utf-8?q?Wrapper_to_workaround_yield-from_bug_in_CPython_=3C_3=2E4=2E1?= Message-ID: <3g79dV0qpvz7Ljy@mail.python.org> http://hg.python.org/cpython/rev/13ff8645be57 changeset: 90314:13ff8645be57 parent: 90311:54f1168036f1 user: Yury Selivanov date: Mon Apr 14 22:24:51 2014 -0400 summary: syncio.tasks: Fix CoroWrapper to workaround yield-from bug in CPython < 3.4.1 Closes issue #21209. files: Lib/asyncio/tasks.py | 5 ++- Lib/test/test_asyncio/test_tasks.py | 25 +++++++++++++++++ Misc/NEWS | 3 ++ 3 files changed, 32 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -49,7 +49,10 @@ def __next__(self): return next(self.gen) - def send(self, value): + def send(self, *value): + # We use `*value` because of a bug in CPythons prior + # to 3.4.1. See issue #21209 and test_yield_from_corowrapper + # for details. This workaround should be removed in 3.5.0. return self.gen.send(value) def throw(self, exc): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -1386,6 +1386,31 @@ self.assertRaises(ValueError, self.loop.run_until_complete, asyncio.wait([], loop=self.loop)) + def test_yield_from_corowrapper(self): + old_debug = asyncio.tasks._DEBUG + asyncio.tasks._DEBUG = True + try: + @asyncio.coroutine + def t1(): + return (yield from t2()) + + @asyncio.coroutine + def t2(): + f = asyncio.Future(loop=self.loop) + asyncio.Task(t3(f), loop=self.loop) + return (yield from f) + + @asyncio.coroutine + def t3(f): + f.set_result((1, 2, 3)) + + task = asyncio.Task(t1(), loop=self.loop) + val = self.loop.run_until_complete(task) + self.assertEqual(val, (1, 2, 3)) + finally: + asyncio.tasks._DEBUG = old_debug + + class GatherTestsBase: def setUp(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -206,6 +206,9 @@ - Issue #15916: doctest.DocTestSuite returns an empty unittest.TestSuite instead of raising ValueError if it finds no tests +- Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug + in yield-from implementation in CPythons prior to 3.4.1. + IDLE ---- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 04:29:51 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 04:29:51 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_misc=2Enews=3A_Remove_trai?= =?utf-8?q?ling_whitespace?= Message-ID: <3g79dW2GVTz7LkT@mail.python.org> http://hg.python.org/cpython/rev/f021049608a1 changeset: 90315:f021049608a1 user: Yury Selivanov date: Mon Apr 14 22:25:24 2014 -0400 summary: misc.news: Remove trailing whitespace files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,7 +39,7 @@ - Issue #20480: Add ipaddress.reverse_pointer. Patch by Leon Weber. -- Issue #13598: Modify string.Formatter to support auto-numbering of +- Issue #13598: Modify string.Formatter to support auto-numbering of replacement fields. It now matches the behavior of str.format() in this regard. Patches by Phil Elson and Ramchandra Apte. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 04:29:52 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 04:29:52 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g79dX3hFMz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/bf65e7db066d changeset: 90316:bf65e7db066d parent: 90315:f021049608a1 parent: 90313:2031ff9f13a8 user: Yury Selivanov date: Mon Apr 14 22:27:27 2014 -0400 summary: merge 3.4 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 05:25:59 2014 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 15 Apr 2014 05:25:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_bump_copyright_year?= Message-ID: <3g7BtH2kYPz7LjX@mail.python.org> http://hg.python.org/devguide/rev/d7f1dc8459aa changeset: 688:d7f1dc8459aa user: Benjamin Peterson date: Mon Apr 14 23:25:54 2014 -0400 summary: bump copyright year files: conf.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/conf.py b/conf.py --- a/conf.py +++ b/conf.py @@ -43,7 +43,7 @@ # General information about the project. project = u'Python Developer\'s Guide' -copyright = u'2011-2012, Python Software Foundation' +copyright = u'2011-2014, Python Software Foundation' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Tue Apr 15 09:05:15 2014 From: python-checkins at python.org (eric.smith) Date: Tue, 15 Apr 2014 09:05:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closed_issue_?= =?utf-8?q?=238931=3A_Make_alternate_formatting_for_=27c=27_raise_an_excep?= =?utf-8?q?tion=2E_Patch?= Message-ID: <3g7HlH1WfHz7LjV@mail.python.org> http://hg.python.org/cpython/rev/16efa8d27e4c changeset: 90317:16efa8d27e4c branch: 3.4 parent: 90313:2031ff9f13a8 user: Eric V. Smith date: Tue Apr 15 03:05:02 2014 -0400 summary: Closed issue #8931: Make alternate formatting for 'c' raise an exception. Patch by Torsten Landschoff. files: Lib/test/test_types.py | 2 ++ Misc/NEWS | 4 ++++ Python/formatter_unicode.c | 7 +++++++ 3 files changed, 13 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -343,6 +343,8 @@ self.assertRaises(ValueError, 3 .__format__, ",n") # can't have ',' with 'c' self.assertRaises(ValueError, 3 .__format__, ",c") + # can't have '#' with 'c' + self.assertRaises(ValueError, 3 .__format__, "#c") # ensure that only int and float type specifiers work for format_spec in ([chr(x) for x in range(ord('a'), ord('z')+1)] + diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,6 +34,10 @@ replacement fields. It now matches the behavior of str.format() in this regard. Patches by Phil Elson and Ramchandra Apte. +- Issue #8931: Make alternate formatting ('#') for type 'c' raise an + exception. It had no effect, now trying to specify it is an error. + Patch by Torsten Landschoff. + Library ------- diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -846,6 +846,13 @@ " format specifier 'c'"); goto done; } + /* error to request alternate format */ + if (format->alternate) { + PyErr_SetString(PyExc_ValueError, + "Alternate form (#) not allowed with integer" + " format specifier 'c'"); + goto done; + } /* taken from unicodeobject.c formatchar() */ /* Integer input truncated to a character */ -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Tue Apr 15 09:49:34 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 15 Apr 2014 09:49:34 +0200 Subject: [Python-checkins] Daily reference leaks (57fb5441a4aa): sum=3 Message-ID: results for 57fb5441a4aa on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogoK5MJy', '-x'] From python-checkins at python.org Tue Apr 15 14:56:48 2014 From: python-checkins at python.org (vinay.sajip) Date: Tue, 15 Apr 2014 14:56:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTk3?= =?utf-8?q?=3A_Add_lib64_-=3E_lib_symlink_in_venvs_on_64-bit_non-OS_X_POSI?= =?utf-8?q?X=2E?= Message-ID: <3g7RXw6zR6z7LkC@mail.python.org> http://hg.python.org/cpython/rev/4d9b24b2f1b8 changeset: 90318:4d9b24b2f1b8 branch: 3.4 user: Vinay Sajip date: Tue Apr 15 11:18:10 2014 +0100 summary: Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. files: Lib/test/test_venv.py | 9 +++++++++ Lib/venv/__init__.py | 13 +++++++++++-- Misc/NEWS | 2 ++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -9,6 +9,7 @@ import os import os.path import shutil +import struct import subprocess import sys import tempfile @@ -87,6 +88,14 @@ self.isdir(self.bindir) self.isdir(self.include) self.isdir(*self.lib) + # Issue 21197 + p = self.get_env_file('lib64') + conditions = ((struct.calcsize('P') == 8) and (os.name == 'posix') and + (sys.platform != 'darwin')) + if conditions: + self.assertTrue(os.path.islink(p)) + else: + self.assertFalse(os.path.exists(p)) data = self.get_text_file_contents('pyvenv.cfg') if sys.platform == 'darwin' and ('__PYVENV_LAUNCHER__' in os.environ): diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -1,7 +1,7 @@ """ Virtual environment (venv) package for Python. Based on PEP 405. -Copyright (C) 2011-2012 Vinay Sajip. +Copyright (C) 2011-2014 Vinay Sajip. Licensed to the PSF under a contributor agreement. usage: python -m venv [-h] [--system-site-packages] [--symlinks] [--clear] @@ -30,6 +30,7 @@ import logging import os import shutil +import struct import subprocess import sys import types @@ -132,10 +133,18 @@ else: binname = 'bin' incpath = 'include' - libpath = os.path.join(env_dir, 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages') + libpath = os.path.join(env_dir, 'lib', + 'python%d.%d' % sys.version_info[:2], + 'site-packages') context.inc_path = path = os.path.join(env_dir, incpath) create_if_needed(path) create_if_needed(libpath) + # Issue 21197: create lib64 as a symlink to lib on 64-bit non-OS X POSIX + if ((struct.calcsize('P') == 8) and (os.name == 'posix') and + (sys.platform != 'darwin')): + p = os.path.join(env_dir, 'lib') + link_path = os.path.join(env_dir, 'lib64') + os.symlink(p, link_path) context.bin_path = binpath = os.path.join(env_dir, binname) context.bin_name = binname context.env_exe = os.path.join(binpath, exename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -41,6 +41,8 @@ Library ------- +- Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. + - Issue #17498: Some SMTP servers disconnect after certain errors, violating strict RFC conformance. Instead of losing the error code when we issue the subsequent RSET, smtplib now returns the error code and defers raising the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 14:56:50 2014 From: python-checkins at python.org (vinay.sajip) Date: Tue, 15 Apr 2014 14:56:50 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Reverted_16efa?= =?utf-8?q?8d27e4c_after_discussion_with_Eric=2E?= Message-ID: <3g7RXy1Yzcz7Ljs@mail.python.org> http://hg.python.org/cpython/rev/4c65f8641d89 changeset: 90319:4c65f8641d89 branch: 3.4 user: Vinay Sajip date: Tue Apr 15 13:52:21 2014 +0100 summary: Reverted 16efa8d27e4c after discussion with Eric. files: Lib/test/test_types.py | 2 -- Misc/NEWS | 4 ---- Python/formatter_unicode.c | 7 ------- 3 files changed, 0 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -343,8 +343,6 @@ self.assertRaises(ValueError, 3 .__format__, ",n") # can't have ',' with 'c' self.assertRaises(ValueError, 3 .__format__, ",c") - # can't have '#' with 'c' - self.assertRaises(ValueError, 3 .__format__, "#c") # ensure that only int and float type specifiers work for format_spec in ([chr(x) for x in range(ord('a'), ord('z')+1)] + diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -34,10 +34,6 @@ replacement fields. It now matches the behavior of str.format() in this regard. Patches by Phil Elson and Ramchandra Apte. -- Issue #8931: Make alternate formatting ('#') for type 'c' raise an - exception. It had no effect, now trying to specify it is an error. - Patch by Torsten Landschoff. - Library ------- diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -846,13 +846,6 @@ " format specifier 'c'"); goto done; } - /* error to request alternate format */ - if (format->alternate) { - PyErr_SetString(PyExc_ValueError, - "Alternate form (#) not allowed with integer" - " format specifier 'c'"); - goto done; - } /* taken from unicodeobject.c formatchar() */ /* Integer input truncated to a character */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 14:56:51 2014 From: python-checkins at python.org (vinay.sajip) Date: Tue, 15 Apr 2014 14:56:51 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2321197=3A_Add_lib64_-=3E_lib_symlink_in_venvs_o?= =?utf-8?q?n_64-bit_non-OS_X_POSIX=2E?= Message-ID: <3g7RXz3Pbgz7LkB@mail.python.org> http://hg.python.org/cpython/rev/947515cc8957 changeset: 90320:947515cc8957 parent: 90316:bf65e7db066d parent: 90319:4c65f8641d89 user: Vinay Sajip date: Tue Apr 15 13:56:36 2014 +0100 summary: Closes #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. files: Lib/test/test_venv.py | 9 +++++++++ Lib/venv/__init__.py | 13 +++++++++++-- Misc/NEWS | 2 ++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -9,6 +9,7 @@ import os import os.path import shutil +import struct import subprocess import sys import tempfile @@ -87,6 +88,14 @@ self.isdir(self.bindir) self.isdir(self.include) self.isdir(*self.lib) + # Issue 21197 + p = self.get_env_file('lib64') + conditions = ((struct.calcsize('P') == 8) and (os.name == 'posix') and + (sys.platform != 'darwin')) + if conditions: + self.assertTrue(os.path.islink(p)) + else: + self.assertFalse(os.path.exists(p)) data = self.get_text_file_contents('pyvenv.cfg') if sys.platform == 'darwin' and ('__PYVENV_LAUNCHER__' in os.environ): diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -1,7 +1,7 @@ """ Virtual environment (venv) package for Python. Based on PEP 405. -Copyright (C) 2011-2012 Vinay Sajip. +Copyright (C) 2011-2014 Vinay Sajip. Licensed to the PSF under a contributor agreement. usage: python -m venv [-h] [--system-site-packages] [--symlinks] [--clear] @@ -30,6 +30,7 @@ import logging import os import shutil +import struct import subprocess import sys import types @@ -132,10 +133,18 @@ else: binname = 'bin' incpath = 'include' - libpath = os.path.join(env_dir, 'lib', 'python%d.%d' % sys.version_info[:2], 'site-packages') + libpath = os.path.join(env_dir, 'lib', + 'python%d.%d' % sys.version_info[:2], + 'site-packages') context.inc_path = path = os.path.join(env_dir, incpath) create_if_needed(path) create_if_needed(libpath) + # Issue 21197: create lib64 as a symlink to lib on 64-bit non-OS X POSIX + if ((struct.calcsize('P') == 8) and (os.name == 'posix') and + (sys.platform != 'darwin')): + p = os.path.join(env_dir, 'lib') + link_path = os.path.join(env_dir, 'lib64') + os.symlink(p, link_path) context.bin_path = binpath = os.path.join(env_dir, binname) context.bin_name = binname context.env_exe = os.path.join(binpath, exename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,8 @@ Library ------- +- Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. + - Issue #17498: Some SMTP servers disconnect after certain errors, violating strict RFC conformance. Instead of losing the error code when we issue the subsequent RSET, smtplib now returns the error code and defers raising the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 15:25:05 2014 From: python-checkins at python.org (vinay.sajip) Date: Tue, 15 Apr 2014 15:25:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2321203=3A_Updated?= =?utf-8?q?_fileConfig_and_dictConfig_to_remove_inconsistencies=2E?= Message-ID: <3g7S9Y5nlnz7LjT@mail.python.org> http://hg.python.org/cpython/rev/2d33cbf02522 changeset: 90321:2d33cbf02522 user: Vinay Sajip date: Tue Apr 15 14:24:53 2014 +0100 summary: Closes #21203: Updated fileConfig and dictConfig to remove inconsistencies. Thanks to Jure Koren for the patch. files: Lib/logging/config.py | 10 ++++++++-- Misc/NEWS | 3 +++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Lib/logging/config.py b/Lib/logging/config.py --- a/Lib/logging/config.py +++ b/Lib/logging/config.py @@ -116,11 +116,12 @@ sectname = "formatter_%s" % form fs = cp.get(sectname, "format", raw=True, fallback=None) dfs = cp.get(sectname, "datefmt", raw=True, fallback=None) + stl = cp.get(sectname, "style", raw=True, fallback='%') c = logging.Formatter class_name = cp[sectname].get("class") if class_name: c = _resolve(class_name) - f = c(fs, dfs) + f = c(fs, dfs, stl) formatters[form] = f return formatters @@ -660,7 +661,12 @@ fmt = config.get('format', None) dfmt = config.get('datefmt', None) style = config.get('style', '%') - result = logging.Formatter(fmt, dfmt, style) + cname = config.get('class', None) + if not cname: + c = logging.Formatter + else: + c = _resolve(cname) + result = c(fmt, dfmt, style) return result def configure_filter(self, config): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #21203: Updated fileConfig and dictConfig to remove inconsistencies. + Thanks to Jure Koren for the patch. + - Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. - Issue #17498: Some SMTP servers disconnect after certain errors, violating -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 17:01:41 2014 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 15 Apr 2014 17:01:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Some_cleanup_o?= =?utf-8?q?f_the_subprocess_docs_in_2=2E7=2E__Adds_a_reference_to_the?= Message-ID: <3g7VK15FH5z7LjP@mail.python.org> http://hg.python.org/cpython/rev/dd52365c8721 changeset: 90322:dd52365c8721 branch: 2.7 parent: 90308:3b988a4e7a55 user: Gregory P. Smith date: Tue Apr 15 07:59:44 2014 -0700 summary: Some cleanup of the subprocess docs in 2.7. Adds a reference to the subprocess32 project in PyPI for posix users. files: Doc/library/subprocess.rst | 59 +++++++++++++------------ 1 files changed, 30 insertions(+), 29 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -20,20 +20,26 @@ popen2.* commands.* -Information about how the :mod:`subprocess` module can be used to replace these -modules and functions can be found in the following sections. +Information about how this module can be used to replace the older +functions can be found in the subprocess-replacements_ section. .. seealso:: + POSIX users (Linux, BSD, etc.) are strongly encouraged to install + and use the much more recent subprocess32_ module instead of the + version included with python 2.7. It is a drop in replacement with + better behavior in many situations. + :pep:`324` -- PEP proposing the subprocess module +.. _subprocess32: https://pypi.python.org/pypi/subprocess32/ Using the :mod:`subprocess` Module ---------------------------------- -The recommended approach to invoking subprocesses is to use the following -convenience functions for all use cases they can handle. For more advanced -use cases, the underlying :class:`Popen` interface can be used directly. +The recommended way to launch subprocesses is to use the following +convenience functions. For more advanced use cases when these do not +meet your needs, use the underlying :class:`Popen` interface. .. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False) @@ -57,16 +63,15 @@ .. warning:: - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. .. note:: - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As - the pipes are not being read in the current process, the child - process may block if it generates enough output to a pipe to fill up - the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function + as that can deadlock based on the child process output volume. + Use :class:`Popen` with the :meth:`communicate` method when you + need pipes. .. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False) @@ -96,16 +101,15 @@ .. warning:: - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. .. note:: - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As - the pipes are not being read in the current process, the child - process may block if it generates enough output to a pipe to fill up - the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function + as that can deadlock based on the child process output volume. + Use :class:`Popen` with the :meth:`communicate` method when you + need pipes. .. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False) @@ -145,19 +149,16 @@ .. versionadded:: 2.7 - .. - .. warning:: - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. .. note:: - Do not use ``stderr=PIPE`` with this function. As the pipe is not being - read in the current process, the child process may block if it - generates enough output to the pipe to fill up the OS pipe buffer. + Do not use ``stderr=PIPE`` with this function as that can deadlock + based on the child process error volume. Use :class:`Popen` with + the :meth:`communicate` method when you need a stderr pipe. .. data:: PIPE @@ -740,9 +741,9 @@ :: - sts = os.system("mycmd" + " myarg") + status = os.system("mycmd" + " myarg") # becomes - sts = call("mycmd" + " myarg", shell=True) + status = subprocess.call("mycmd" + " myarg", shell=True) Notes: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 17:01:42 2014 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 15 Apr 2014 17:01:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_whitespace_cha?= =?utf-8?q?ps_my_hide?= Message-ID: <3g7VK26zhZz7LjP@mail.python.org> http://hg.python.org/cpython/rev/ff5648697ac7 changeset: 90323:ff5648697ac7 branch: 2.7 user: Gregory P. Smith date: Tue Apr 15 08:01:27 2014 -0700 summary: whitespace chaps my hide files: Doc/library/subprocess.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -32,7 +32,7 @@ :pep:`324` -- PEP proposing the subprocess module -.. _subprocess32: https://pypi.python.org/pypi/subprocess32/ +.. _subprocess32: https://pypi.python.org/pypi/subprocess32/ Using the :mod:`subprocess` Module ---------------------------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 18:02:26 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 18:02:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpby50YXNr?= =?utf-8?q?s=3A_Make_sure_CoroWrapper=2Esend_proxies_one_argument_correctl?= =?utf-8?q?y?= Message-ID: <3g7Wg674D0z7LjR@mail.python.org> http://hg.python.org/cpython/rev/2729823525fe changeset: 90324:2729823525fe branch: 3.4 parent: 90319:4c65f8641d89 user: Yury Selivanov date: Tue Apr 15 12:01:16 2014 -0400 summary: asyncio.tasks: Make sure CoroWrapper.send proxies one argument correctly Issue #21209. files: Lib/asyncio/tasks.py | 2 + Lib/test/test_asyncio/test_tasks.py | 18 +++++++++++++++++ 2 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -53,6 +53,8 @@ # We use `*value` because of a bug in CPythons prior # to 3.4.1. See issue #21209 and test_yield_from_corowrapper # for details. This workaround should be removed in 3.5.0. + if len(value) == 1: + value = value[0] return self.gen.send(value) def throw(self, exc): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -1410,6 +1410,24 @@ finally: asyncio.tasks._DEBUG = old_debug + def test_yield_from_corowrapper_send(self): + def foo(): + a = yield + return a + + def call(arg): + cw = asyncio.tasks.CoroWrapper(foo(), foo) + cw.send(None) + try: + cw.send(arg) + except StopIteration as ex: + return ex.args[0] + else: + raise AssertionError('StopIteration was expected') + + self.assertEqual(call((1, 2)), (1, 2)) + self.assertEqual(call('spam'), 'spam') + class GatherTestsBase: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 18:02:28 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 18:02:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_asyncio=2Etasks=3A_Make_su?= =?utf-8?q?re_CoroWrapper=2Esend_proxies_one_argument_correctly?= Message-ID: <3g7Wg81mmsz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/552ee474f3e7 changeset: 90325:552ee474f3e7 parent: 90321:2d33cbf02522 user: Yury Selivanov date: Tue Apr 15 12:01:44 2014 -0400 summary: asyncio.tasks: Make sure CoroWrapper.send proxies one argument correctly Issue #21209. files: Lib/asyncio/tasks.py | 2 + Lib/test/test_asyncio/test_tasks.py | 18 +++++++++++++++++ 2 files changed, 20 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -53,6 +53,8 @@ # We use `*value` because of a bug in CPythons prior # to 3.4.1. See issue #21209 and test_yield_from_corowrapper # for details. This workaround should be removed in 3.5.0. + if len(value) == 1: + value = value[0] return self.gen.send(value) def throw(self, exc): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -1410,6 +1410,24 @@ finally: asyncio.tasks._DEBUG = old_debug + def test_yield_from_corowrapper_send(self): + def foo(): + a = yield + return a + + def call(arg): + cw = asyncio.tasks.CoroWrapper(foo(), foo) + cw.send(None) + try: + cw.send(arg) + except StopIteration as ex: + return ex.args[0] + else: + raise AssertionError('StopIteration was expected') + + self.assertEqual(call((1, 2)), (1, 2)) + self.assertEqual(call('spam'), 'spam') + class GatherTestsBase: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 18:02:29 2014 From: python-checkins at python.org (yury.selivanov) Date: Tue, 15 Apr 2014 18:02:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g7Wg943Pvz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/4ab2d8fac52b changeset: 90326:4ab2d8fac52b parent: 90325:552ee474f3e7 parent: 90324:2729823525fe user: Yury Selivanov date: Tue Apr 15 12:01:57 2014 -0400 summary: merge 3.4 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 20:22:28 2014 From: python-checkins at python.org (kushal.das) Date: Tue, 15 Apr 2014 20:22:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_Issue_17861=3A_Auto?= =?utf-8?q?generate_Include/opcode=2Eh_from_opcode=2Epy=2E?= Message-ID: <3g7Zmh5tzpz7Ljk@mail.python.org> http://hg.python.org/cpython/rev/56e7fa27f979 changeset: 90327:56e7fa27f979 user: Kushal Das date: Tue Apr 15 23:50:06 2014 +0530 summary: Closes Issue 17861: Autogenerate Include/opcode.h from opcode.py. It includes required changes in Makefile.pre.in and configure.ac among other files. files: .hgtouch | 2 + Include/opcode.h | 249 +++++++--------- Makefile.pre.in | 12 +- Misc/NEWS | 3 + Tools/scripts/generate_opcode_h.py | 53 +++ configure | 52 +++ configure.ac | 9 + 7 files changed, 239 insertions(+), 141 deletions(-) diff --git a/.hgtouch b/.hgtouch --- a/.hgtouch +++ b/.hgtouch @@ -4,6 +4,8 @@ Python/importlib.h: Lib/importlib/_bootstrap.py Modules/_freeze_importlib.c +Include/opcode.h: Lib/opcode.py Tools/scripts/generate_opcode_h.py + Include/Python-ast.h: Parser/Python.asdl Parser/asdl.py Parser/asdl_c.py Python/Python-ast.c: Include/Python-ast.h diff --git a/Include/opcode.h b/Include/opcode.h --- a/Include/opcode.h +++ b/Include/opcode.h @@ -1,3 +1,4 @@ +/* Auto-generated by Tools/scripts/generate_opcode_h.py */ #ifndef Py_OPCODE_H #define Py_OPCODE_H #ifdef __cplusplus @@ -5,144 +6,111 @@ #endif -/* Instruction opcodes for compiled code */ - -#define POP_TOP 1 -#define ROT_TWO 2 -#define ROT_THREE 3 -#define DUP_TOP 4 -#define DUP_TOP_TWO 5 -#define NOP 9 - -#define UNARY_POSITIVE 10 -#define UNARY_NEGATIVE 11 -#define UNARY_NOT 12 - -#define UNARY_INVERT 15 - -#define BINARY_MATRIX_MULTIPLY 16 -#define INPLACE_MATRIX_MULTIPLY 17 - -#define BINARY_POWER 19 - -#define BINARY_MULTIPLY 20 - -#define BINARY_MODULO 22 -#define BINARY_ADD 23 -#define BINARY_SUBTRACT 24 -#define BINARY_SUBSCR 25 -#define BINARY_FLOOR_DIVIDE 26 -#define BINARY_TRUE_DIVIDE 27 -#define INPLACE_FLOOR_DIVIDE 28 -#define INPLACE_TRUE_DIVIDE 29 - -#define STORE_MAP 54 -#define INPLACE_ADD 55 -#define INPLACE_SUBTRACT 56 -#define INPLACE_MULTIPLY 57 - -#define INPLACE_MODULO 59 -#define STORE_SUBSCR 60 -#define DELETE_SUBSCR 61 - -#define BINARY_LSHIFT 62 -#define BINARY_RSHIFT 63 -#define BINARY_AND 64 -#define BINARY_XOR 65 -#define BINARY_OR 66 -#define INPLACE_POWER 67 -#define GET_ITER 68 -#define PRINT_EXPR 70 -#define LOAD_BUILD_CLASS 71 -#define YIELD_FROM 72 - -#define INPLACE_LSHIFT 75 -#define INPLACE_RSHIFT 76 -#define INPLACE_AND 77 -#define INPLACE_XOR 78 -#define INPLACE_OR 79 -#define BREAK_LOOP 80 -#define WITH_CLEANUP 81 - -#define RETURN_VALUE 83 -#define IMPORT_STAR 84 - -#define YIELD_VALUE 86 -#define POP_BLOCK 87 -#define END_FINALLY 88 -#define POP_EXCEPT 89 - -#define HAVE_ARGUMENT 90 /* Opcodes from here have an argument: */ - -#define STORE_NAME 90 /* Index in name list */ -#define DELETE_NAME 91 /* "" */ -#define UNPACK_SEQUENCE 92 /* Number of sequence items */ -#define FOR_ITER 93 -#define UNPACK_EX 94 /* Num items before variable part + - (Num items after variable part << 8) */ - -#define STORE_ATTR 95 /* Index in name list */ -#define DELETE_ATTR 96 /* "" */ -#define STORE_GLOBAL 97 /* "" */ -#define DELETE_GLOBAL 98 /* "" */ - -#define LOAD_CONST 100 /* Index in const list */ -#define LOAD_NAME 101 /* Index in name list */ -#define BUILD_TUPLE 102 /* Number of tuple items */ -#define BUILD_LIST 103 /* Number of list items */ -#define BUILD_SET 104 /* Number of set items */ -#define BUILD_MAP 105 /* Always zero for now */ -#define LOAD_ATTR 106 /* Index in name list */ -#define COMPARE_OP 107 /* Comparison operator */ -#define IMPORT_NAME 108 /* Index in name list */ -#define IMPORT_FROM 109 /* Index in name list */ - -#define JUMP_FORWARD 110 /* Number of bytes to skip */ -#define JUMP_IF_FALSE_OR_POP 111 /* Target byte offset from beginning of code */ -#define JUMP_IF_TRUE_OR_POP 112 /* "" */ -#define JUMP_ABSOLUTE 113 /* "" */ -#define POP_JUMP_IF_FALSE 114 /* "" */ -#define POP_JUMP_IF_TRUE 115 /* "" */ - -#define LOAD_GLOBAL 116 /* Index in name list */ - -#define CONTINUE_LOOP 119 /* Start of loop (absolute) */ -#define SETUP_LOOP 120 /* Target address (relative) */ -#define SETUP_EXCEPT 121 /* "" */ -#define SETUP_FINALLY 122 /* "" */ - -#define LOAD_FAST 124 /* Local variable number */ -#define STORE_FAST 125 /* Local variable number */ -#define DELETE_FAST 126 /* Local variable number */ - -#define RAISE_VARARGS 130 /* Number of raise arguments (1, 2 or 3) */ -/* CALL_FUNCTION_XXX opcodes defined below depend on this definition */ -#define CALL_FUNCTION 131 /* #args + (#kwargs<<8) */ -#define MAKE_FUNCTION 132 /* #defaults + #kwdefaults<<8 + #annotations<<16 */ -#define BUILD_SLICE 133 /* Number of items */ - -#define MAKE_CLOSURE 134 /* same as MAKE_FUNCTION */ -#define LOAD_CLOSURE 135 /* Load free variable from closure */ -#define LOAD_DEREF 136 /* Load and dereference from closure cell */ -#define STORE_DEREF 137 /* Store into cell */ -#define DELETE_DEREF 138 /* Delete closure cell */ - -/* The next 3 opcodes must be contiguous and satisfy - (CALL_FUNCTION_VAR - CALL_FUNCTION) & 3 == 1 */ -#define CALL_FUNCTION_VAR 140 /* #args + (#kwargs<<8) */ -#define CALL_FUNCTION_KW 141 /* #args + (#kwargs<<8) */ -#define CALL_FUNCTION_VAR_KW 142 /* #args + (#kwargs<<8) */ - -#define SETUP_WITH 143 - -/* Support for opargs more than 16 bits long */ -#define EXTENDED_ARG 144 - -#define LIST_APPEND 145 -#define SET_ADD 146 -#define MAP_ADD 147 - -#define LOAD_CLASSDEREF 148 + /* Instruction opcodes for compiled code */ +#define POP_TOP 1 +#define ROT_TWO 2 +#define ROT_THREE 3 +#define DUP_TOP 4 +#define DUP_TOP_TWO 5 +#define NOP 9 +#define UNARY_POSITIVE 10 +#define UNARY_NEGATIVE 11 +#define UNARY_NOT 12 +#define UNARY_INVERT 15 +#define BINARY_MATRIX_MULTIPLY 16 +#define INPLACE_MATRIX_MULTIPLY 17 +#define BINARY_POWER 19 +#define BINARY_MULTIPLY 20 +#define BINARY_MODULO 22 +#define BINARY_ADD 23 +#define BINARY_SUBTRACT 24 +#define BINARY_SUBSCR 25 +#define BINARY_FLOOR_DIVIDE 26 +#define BINARY_TRUE_DIVIDE 27 +#define INPLACE_FLOOR_DIVIDE 28 +#define INPLACE_TRUE_DIVIDE 29 +#define STORE_MAP 54 +#define INPLACE_ADD 55 +#define INPLACE_SUBTRACT 56 +#define INPLACE_MULTIPLY 57 +#define INPLACE_MODULO 59 +#define STORE_SUBSCR 60 +#define DELETE_SUBSCR 61 +#define BINARY_LSHIFT 62 +#define BINARY_RSHIFT 63 +#define BINARY_AND 64 +#define BINARY_XOR 65 +#define BINARY_OR 66 +#define INPLACE_POWER 67 +#define GET_ITER 68 +#define PRINT_EXPR 70 +#define LOAD_BUILD_CLASS 71 +#define YIELD_FROM 72 +#define INPLACE_LSHIFT 75 +#define INPLACE_RSHIFT 76 +#define INPLACE_AND 77 +#define INPLACE_XOR 78 +#define INPLACE_OR 79 +#define BREAK_LOOP 80 +#define WITH_CLEANUP 81 +#define RETURN_VALUE 83 +#define IMPORT_STAR 84 +#define YIELD_VALUE 86 +#define POP_BLOCK 87 +#define END_FINALLY 88 +#define POP_EXCEPT 89 +#define HAVE_ARGUMENT 90 +#define STORE_NAME 90 +#define DELETE_NAME 91 +#define UNPACK_SEQUENCE 92 +#define FOR_ITER 93 +#define UNPACK_EX 94 +#define STORE_ATTR 95 +#define DELETE_ATTR 96 +#define STORE_GLOBAL 97 +#define DELETE_GLOBAL 98 +#define LOAD_CONST 100 +#define LOAD_NAME 101 +#define BUILD_TUPLE 102 +#define BUILD_LIST 103 +#define BUILD_SET 104 +#define BUILD_MAP 105 +#define LOAD_ATTR 106 +#define COMPARE_OP 107 +#define IMPORT_NAME 108 +#define IMPORT_FROM 109 +#define JUMP_FORWARD 110 +#define JUMP_IF_FALSE_OR_POP 111 +#define JUMP_IF_TRUE_OR_POP 112 +#define JUMP_ABSOLUTE 113 +#define POP_JUMP_IF_FALSE 114 +#define POP_JUMP_IF_TRUE 115 +#define LOAD_GLOBAL 116 +#define CONTINUE_LOOP 119 +#define SETUP_LOOP 120 +#define SETUP_EXCEPT 121 +#define SETUP_FINALLY 122 +#define LOAD_FAST 124 +#define STORE_FAST 125 +#define DELETE_FAST 126 +#define RAISE_VARARGS 130 +#define CALL_FUNCTION 131 +#define MAKE_FUNCTION 132 +#define BUILD_SLICE 133 +#define MAKE_CLOSURE 134 +#define LOAD_CLOSURE 135 +#define LOAD_DEREF 136 +#define STORE_DEREF 137 +#define DELETE_DEREF 138 +#define CALL_FUNCTION_VAR 140 +#define CALL_FUNCTION_KW 141 +#define CALL_FUNCTION_VAR_KW 142 +#define SETUP_WITH 143 +#define EXTENDED_ARG 144 +#define LIST_APPEND 145 +#define SET_ADD 146 +#define MAP_ADD 147 +#define LOAD_CLASSDEREF 148 /* EXCEPT_HANDLER is a special, implicit block type which is created when entering an except handler. It is not an opcode but we define it here @@ -151,8 +119,9 @@ #define EXCEPT_HANDLER 257 -enum cmp_op {PyCmp_LT=Py_LT, PyCmp_LE=Py_LE, PyCmp_EQ=Py_EQ, PyCmp_NE=Py_NE, PyCmp_GT=Py_GT, PyCmp_GE=Py_GE, - PyCmp_IN, PyCmp_NOT_IN, PyCmp_IS, PyCmp_IS_NOT, PyCmp_EXC_MATCH, PyCmp_BAD}; +enum cmp_op {PyCmp_LT=Py_LT, PyCmp_LE=Py_LE, PyCmp_EQ=Py_EQ, PyCmp_NE=Py_NE, + PyCmp_GT=Py_GT, PyCmp_GE=Py_GE, PyCmp_IN, PyCmp_NOT_IN, + PyCmp_IS, PyCmp_IS_NOT, PyCmp_EXC_MATCH, PyCmp_BAD}; #define HAS_ARG(op) ((op) >= HAVE_ARGUMENT) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -323,6 +323,13 @@ PGENOBJS= $(POBJS) $(PGOBJS) ########################################################################## +# opcode.h generation +OPCODE_H_DIR= Include +OPCODE_H_SCRIPT= Tools/scripts/generate_opcode_h.py +OPCODE_H= $(srcdir)/$(OPCODE_H_DIR)/opcode.h +OPCODE_H_GEN= @OPCODEHGEN@ $(OPCODE_H_SCRIPT) Lib/ $(OPCODE_H) +# +########################################################################## # AST AST_H_DIR= Include AST_H= $(AST_H_DIR)/Python-ast.h @@ -760,6 +767,9 @@ $(MKDIR_P) $(AST_C_DIR) $(ASDLGEN) -c $(AST_C_DIR) $(AST_ASDL) +$(OPCODE_H): $(srcdir)/Lib/opcode.py $(OPCODE_H_SCRIPT) + $(OPCODE_H_GEN) + Python/compile.o Python/symtable.o Python/ast.o: $(GRAMMAR_H) $(AST_H) Python/getplatform.o: $(srcdir)/Python/getplatform.c @@ -871,7 +881,7 @@ $(srcdir)/Include/node.h \ $(srcdir)/Include/object.h \ $(srcdir)/Include/objimpl.h \ - $(srcdir)/Include/opcode.h \ + $(OPCODE_H) \ $(srcdir)/Include/osdefs.h \ $(srcdir)/Include/patchlevel.h \ $(srcdir)/Include/pgen.h \ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -223,6 +223,9 @@ Build ----- +- Issue #17861: Tools/scripts/generate_opcode_h.py automatically regenerates + Include/opcode.h from Lib/opcode.py if the later gets any change. + - Issue #20644: OS X installer build support for documentation build changes in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py new file mode 100644 --- /dev/null +++ b/Tools/scripts/generate_opcode_h.py @@ -0,0 +1,53 @@ +# This script generates the opcode.h header file. + +import sys +if len(sys.argv) > 0: + sys.path.insert(0, sys.argv[1]) +# Importing module from our given src directory. +import opcode +header = """/* Auto-generated by Tools/scripts/generate_opcode_h.py */ +#ifndef Py_OPCODE_H +#define Py_OPCODE_H +#ifdef __cplusplus +extern "C" { +#endif + + + /* Instruction opcodes for compiled code */ +""" + +footer = """ +/* EXCEPT_HANDLER is a special, implicit block type which is created when + entering an except handler. It is not an opcode but we define it here + as we want it to be available to both frameobject.c and ceval.c, while + remaining private.*/ +#define EXCEPT_HANDLER 257 + + +enum cmp_op {PyCmp_LT=Py_LT, PyCmp_LE=Py_LE, PyCmp_EQ=Py_EQ, PyCmp_NE=Py_NE, + PyCmp_GT=Py_GT, PyCmp_GE=Py_GE, PyCmp_IN, PyCmp_NOT_IN, + PyCmp_IS, PyCmp_IS_NOT, PyCmp_EXC_MATCH, PyCmp_BAD}; + +#define HAS_ARG(op) ((op) >= HAVE_ARGUMENT) + +#ifdef __cplusplus +} +#endif +#endif /* !Py_OPCODE_H */ +""" + + +def main(outfile='Include/opcode.h'): + with open(outfile, 'w') as fobj: + fobj.write(header) + for name in opcode.opname: + if name in opcode.opmap: + fobj.write("#define %-20s\t%-3s\n" % (name, opcode.opmap[name])) + if name == 'POP_EXCEPT': # Special entry for HAVE_ARGUMENT + fobj.write("#define %-20s\t%-3d\n" % + ('HAVE_ARGUMENT', opcode.HAVE_ARGUMENT)) + fobj.write(footer) + + +if __name__ == '__main__': + main(sys.argv[2]) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -670,6 +670,7 @@ INSTALL_DATA INSTALL_SCRIPT INSTALL_PROGRAM +OPCODEHGEN PYTHON ASDLGEN ac_ct_READELF @@ -6048,6 +6049,57 @@ fi +for ac_prog in python$PACKAGE_VERSION python3 python +do + # Extract the first word of "$ac_prog", so it can be a program name with args. +set dummy $ac_prog; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_prog_PYTHON+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test -n "$PYTHON"; then + ac_cv_prog_PYTHON="$PYTHON" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_PYTHON="$ac_prog" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + +fi +fi +PYTHON=$ac_cv_prog_PYTHON +if test -n "$PYTHON"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PYTHON" >&5 +$as_echo "$PYTHON" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + + test -n "$PYTHON" && break +done +test -n "$PYTHON" || PYTHON="not-found" + +if test "$PYTHON" = not-found; then + OPCODEHGEN="@echo python: $PYTHON! cannot run Tools/scripts/generate_opcode_h.py" +else + OPCODEHGEN="$PYTHON" +fi + + + case $MACHDEP in bsdos*|hp*|HP*) # install -d does not work on BSDI or HP-UX diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1036,6 +1036,15 @@ ASDLGEN="$PYTHON" fi +AC_SUBST(OPCODEHGEN) +AC_CHECK_PROGS(PYTHON, python$PACKAGE_VERSION python3 python, not-found) +if test "$PYTHON" = not-found; then + OPCODEHGEN="@echo python: $PYTHON! cannot run Tools/scripts/generate_opcode_h.py" +else + OPCODEHGEN="$PYTHON" +fi + + case $MACHDEP in bsdos*|hp*|HP*) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 20:38:41 2014 From: python-checkins at python.org (matthias.klose) Date: Tue, 15 Apr 2014 20:38:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogLSBJc3N1ZSAjMjEy?= =?utf-8?q?23=3A_Pass_test=5Fsite/test=5Fstartup=5Fimports_when_some_of_th?= =?utf-8?q?e_extensions?= Message-ID: <3g7b7P5GT2z7Llp@mail.python.org> http://hg.python.org/cpython/rev/ebb9595af548 changeset: 90328:ebb9595af548 branch: 3.4 parent: 90324:2729823525fe user: doko at ubuntu.com date: Tue Apr 15 20:37:54 2014 +0200 summary: - Issue #21223: Pass test_site/test_startup_imports when some of the extensions are built as builtins. files: Lib/test/test_site.py | 3 ++- Misc/NEWS | 3 +++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -459,7 +459,8 @@ # http://bugs.python.org/issue19218> collection_mods = {'_collections', 'collections', 'functools', 'heapq', 'itertools', 'keyword', 'operator', - 'reprlib', 'types', 'weakref'} + 'reprlib', 'types', 'weakref' + }.difference(sys.builtin_module_names) self.assertFalse(modules.intersection(collection_mods), stderr) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -190,6 +190,9 @@ Tests ----- +- Issue #21223: Pass test_site/test_startup_imports when some of the extensions + are built as builtins. + - Issue #20635: Added tests for Tk geometry managers. - Add test case for freeze. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 20:38:42 2014 From: python-checkins at python.org (matthias.klose) Date: Tue, 15 Apr 2014 20:38:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E4_branch=2E?= Message-ID: <3g7b7Q6zrgz7Llp@mail.python.org> http://hg.python.org/cpython/rev/795d90c7820d changeset: 90329:795d90c7820d parent: 90327:56e7fa27f979 parent: 90328:ebb9595af548 user: doko at ubuntu.com date: Tue Apr 15 20:38:28 2014 +0200 summary: Merge 3.4 branch. files: Lib/test/test_site.py | 3 ++- Misc/NEWS | 3 +++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -459,7 +459,8 @@ # http://bugs.python.org/issue19218> collection_mods = {'_collections', 'collections', 'functools', 'heapq', 'itertools', 'keyword', 'operator', - 'reprlib', 'types', 'weakref'} + 'reprlib', 'types', 'weakref' + }.difference(sys.builtin_module_names) self.assertFalse(modules.intersection(collection_mods), stderr) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -255,6 +255,9 @@ Tests ----- +- Issue #21223: Pass test_site/test_startup_imports when some of the extensions + are built as builtins. + - Issue #20635: Added tests for Tk geometry managers. - Add test case for freeze. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 21:53:10 2014 From: python-checkins at python.org (kushal.das) Date: Tue, 15 Apr 2014 21:53:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closes_Issue_2?= =?utf-8?q?1222=2E?= Message-ID: <3g7cnL0wnVzSRT@mail.python.org> http://hg.python.org/cpython/rev/d471b0d38516 changeset: 90330:d471b0d38516 branch: 3.4 parent: 90328:ebb9595af548 user: Kushal Das date: Wed Apr 16 01:05:50 2014 +0530 summary: Closes Issue 21222. Passing name keyword argument to mock.create_autospec now works. files: Lib/unittest/mock.py | 2 ++ Lib/unittest/test/testmock/testmock.py | 4 ++++ Misc/NEWS | 3 +++ 3 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2096,6 +2096,8 @@ elif is_type and instance and not _instance_callable(spec): Klass = NonCallableMagicMock + _name = _kwargs.pop('name', _name) + _new_name = _name if _parent is None: # for a top level object no _new_name should be set diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -1182,6 +1182,10 @@ func.mock_calls, [call(1, 2), call(3, 4)] ) + #Issue21222 + def test_create_autospec_with_name(self): + m = mock.create_autospec(object(), name='sweet_func') + self.assertIn('sweet_func', repr(m)) def test_mock_add_spec(self): class _One(object): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #21222: Passing name keyword argument to mock.create_autospec now + works. + - Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. - Issue #17498: Some SMTP servers disconnect after certain errors, violating -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 21:53:12 2014 From: python-checkins at python.org (kushal.das) Date: Tue, 15 Apr 2014 21:53:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <3g7cnN03sJz7Ljk@mail.python.org> http://hg.python.org/cpython/rev/ec3ea80758a4 changeset: 90331:ec3ea80758a4 parent: 90329:795d90c7820d parent: 90330:d471b0d38516 user: Kushal Das date: Wed Apr 16 01:22:52 2014 +0530 summary: Merge files: Lib/unittest/mock.py | 2 ++ Lib/unittest/test/testmock/testmock.py | 4 ++++ Misc/NEWS | 3 +++ 3 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2103,6 +2103,8 @@ elif is_type and instance and not _instance_callable(spec): Klass = NonCallableMagicMock + _name = _kwargs.pop('name', _name) + _new_name = _name if _parent is None: # for a top level object no _new_name should be set diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -1182,6 +1182,10 @@ func.mock_calls, [call(1, 2), call(3, 4)] ) + #Issue21222 + def test_create_autospec_with_name(self): + m = mock.create_autospec(object(), name='sweet_func') + self.assertIn('sweet_func', repr(m)) def test_mock_add_spec(self): class _One(object): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -48,6 +48,9 @@ - Issue #21203: Updated fileConfig and dictConfig to remove inconsistencies. Thanks to Jure Koren for the patch. + +- Issue #21222: Passing name keyword argument to mock.create_autospec now + works. - Issue #21197: Add lib64 -> lib symlink in venvs on 64-bit non-OS X POSIX. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 22:16:03 2014 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 15 Apr 2014 22:16:03 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE1ODQwOiBtYWtl?= =?utf-8?q?_docs_consistent_by_saying_operations_on_closed_files_raise?= Message-ID: <3g7dHl3czTz7LjV@mail.python.org> http://hg.python.org/cpython/rev/0c7cf0e598e7 changeset: 90332:0c7cf0e598e7 branch: 2.7 parent: 90323:ff5648697ac7 user: Andrew Kuchling date: Tue Apr 15 16:07:52 2014 -0400 summary: #15840: make docs consistent by saying operations on closed files raise ValueError. Patch by Caelyn McAulay. Neither Caelyn nor I could find any cases in 2.7 or 3.4/5 where an operation on a closed stream raised IOError; generally the C implementation have a macro to check for the stream being closed, and these macros all raised ValueError. If we find any, a new bug should be opened. files: Modules/_io/iobase.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -41,8 +41,8 @@ "bytes. bytearrays are accepted too, and in some cases (such as\n" "readinto) needed. Text I/O classes work with str data.\n" "\n" - "Note that calling any method (even inquiries) on a closed stream is\n" - "undefined. Implementations may raise IOError in this case.\n" + "Note that calling any method (except additional calls to close(),\n" + "which are ignored) on a closed stream should raise a ValueError.\n" "\n" "IOBase (and its subclasses) support the iterator protocol, meaning\n" "that an IOBase object can be iterated over yielding the lines in a\n" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 22:37:22 2014 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 15 Apr 2014 22:37:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogUnVuIHRlc3RfdXJs?= =?utf-8?q?lib2=5Flocalnet_tests_using_unittest=2Emain=28=29=2E?= Message-ID: <3g7dmL1N8vz7LjS@mail.python.org> http://hg.python.org/cpython/rev/9fe4d9686162 changeset: 90333:9fe4d9686162 branch: 3.4 parent: 90330:d471b0d38516 user: Senthil Kumaran date: Tue Apr 15 16:36:43 2014 -0400 summary: Run test_urllib2_localnet tests using unittest.main(). Capture threads in the setUpModule and cleanup threads in the tearDownModule. files: Lib/test/test_urllib2_localnet.py | 24 +++++++++++++----- 1 files changed, 17 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -5,8 +5,11 @@ import http.server import unittest import hashlib + from test import support + threading = support.import_module('threading') + try: import ssl except ImportError: @@ -57,14 +60,11 @@ request_handler.protocol_version = "HTTP/1.0" self.httpd = LoopbackHttpServer(("127.0.0.1", 0), request_handler) - #print "Serving HTTP on %s port %s" % (self.httpd.server_name, - # self.httpd.server_port) self.port = self.httpd.server_port def stop(self): """Stops the webserver if it's currently running.""" - # Set the stop flag. self._stop_server = True self.join() @@ -232,6 +232,7 @@ # Test cases + at unittest.skipUnless(threading, "Threading required for this test.") class ProxyAuthTests(unittest.TestCase): URL = "http://localhost" @@ -343,6 +344,7 @@ return FakeHTTPRequestHandler + at unittest.skipUnless(threading, "Threading required for this test.") class TestUrlopen(unittest.TestCase): """Tests urllib.request.urlopen using the network. @@ -590,9 +592,17 @@ self.assertEqual(index + 1, len(lines)) - at support.reap_threads -def test_main(): - support.run_unittest(ProxyAuthTests, TestUrlopen) +threads_key = None + +def setUpModule(): + # Store the threading_setup in a key and ensure that it is cleaned up + # in the tearDown + global threads_key + threads_key = support.threading_setup() + +def tearDownModule(): + if threads_key: + support.threading_cleanup(threads_key) if __name__ == "__main__": - test_main() + unittest.main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 22:37:23 2014 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 15 Apr 2014 22:37:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g7dmM3WMxz7LjV@mail.python.org> http://hg.python.org/cpython/rev/d9ce31599f76 changeset: 90334:d9ce31599f76 parent: 90331:ec3ea80758a4 parent: 90333:9fe4d9686162 user: Senthil Kumaran date: Tue Apr 15 16:37:14 2014 -0400 summary: merge 3.4 files: Lib/test/test_urllib2_localnet.py | 24 +++++++++++++----- 1 files changed, 17 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -5,8 +5,11 @@ import http.server import unittest import hashlib + from test import support + threading = support.import_module('threading') + try: import ssl except ImportError: @@ -57,14 +60,11 @@ request_handler.protocol_version = "HTTP/1.0" self.httpd = LoopbackHttpServer(("127.0.0.1", 0), request_handler) - #print "Serving HTTP on %s port %s" % (self.httpd.server_name, - # self.httpd.server_port) self.port = self.httpd.server_port def stop(self): """Stops the webserver if it's currently running.""" - # Set the stop flag. self._stop_server = True self.join() @@ -232,6 +232,7 @@ # Test cases + at unittest.skipUnless(threading, "Threading required for this test.") class ProxyAuthTests(unittest.TestCase): URL = "http://localhost" @@ -343,6 +344,7 @@ return FakeHTTPRequestHandler + at unittest.skipUnless(threading, "Threading required for this test.") class TestUrlopen(unittest.TestCase): """Tests urllib.request.urlopen using the network. @@ -590,9 +592,17 @@ self.assertEqual(index + 1, len(lines)) - at support.reap_threads -def test_main(): - support.run_unittest(ProxyAuthTests, TestUrlopen) +threads_key = None + +def setUpModule(): + # Store the threading_setup in a key and ensure that it is cleaned up + # in the tearDown + global threads_key + threads_key = support.threading_setup() + +def tearDownModule(): + if threads_key: + support.threading_cleanup(threads_key) if __name__ == "__main__": - test_main() + unittest.main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 22:42:04 2014 From: python-checkins at python.org (stefan.krah) Date: Tue, 15 Apr 2014 22:42:04 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Skip_test=5Fenums_in_the_a?= =?utf-8?q?bsence_of_threading=2E?= Message-ID: <3g7dsm1PJSz7LjS@mail.python.org> http://hg.python.org/cpython/rev/16243861604f changeset: 90335:16243861604f parent: 90331:ec3ea80758a4 user: Stefan Krah date: Tue Apr 15 22:40:06 2014 +0200 summary: Skip test_enums in the absence of threading. files: Lib/test/test_signal.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py --- a/Lib/test/test_signal.py +++ b/Lib/test/test_signal.py @@ -42,6 +42,7 @@ class GenericTests(unittest.TestCase): + @unittest.skipIf(threading is None, "test needs threading module") def test_enums(self): for name in dir(signal): sig = getattr(signal, name) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 22:42:05 2014 From: python-checkins at python.org (stefan.krah) Date: Tue, 15 Apr 2014 22:42:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?b?KTogTWVyZ2Uu?= Message-ID: <3g7dsn3BQpz7LjS@mail.python.org> http://hg.python.org/cpython/rev/69d479e9aa82 changeset: 90336:69d479e9aa82 parent: 90335:16243861604f parent: 90334:d9ce31599f76 user: Stefan Krah date: Tue Apr 15 22:41:36 2014 +0200 summary: Merge. files: Lib/test/test_urllib2_localnet.py | 24 +++++++++++++----- 1 files changed, 17 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -5,8 +5,11 @@ import http.server import unittest import hashlib + from test import support + threading = support.import_module('threading') + try: import ssl except ImportError: @@ -57,14 +60,11 @@ request_handler.protocol_version = "HTTP/1.0" self.httpd = LoopbackHttpServer(("127.0.0.1", 0), request_handler) - #print "Serving HTTP on %s port %s" % (self.httpd.server_name, - # self.httpd.server_port) self.port = self.httpd.server_port def stop(self): """Stops the webserver if it's currently running.""" - # Set the stop flag. self._stop_server = True self.join() @@ -232,6 +232,7 @@ # Test cases + at unittest.skipUnless(threading, "Threading required for this test.") class ProxyAuthTests(unittest.TestCase): URL = "http://localhost" @@ -343,6 +344,7 @@ return FakeHTTPRequestHandler + at unittest.skipUnless(threading, "Threading required for this test.") class TestUrlopen(unittest.TestCase): """Tests urllib.request.urlopen using the network. @@ -590,9 +592,17 @@ self.assertEqual(index + 1, len(lines)) - at support.reap_threads -def test_main(): - support.run_unittest(ProxyAuthTests, TestUrlopen) +threads_key = None + +def setUpModule(): + # Store the threading_setup in a key and ensure that it is cleaned up + # in the tearDown + global threads_key + threads_key = support.threading_setup() + +def tearDownModule(): + if threads_key: + support.threading_cleanup(threads_key) if __name__ == "__main__": - test_main() + unittest.main() -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 22:44:52 2014 From: python-checkins at python.org (andrew.kuchling) Date: Tue, 15 Apr 2014 22:44:52 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE3MDQ0NzQ6IG1h?= =?utf-8?q?rk_refleak_test_as_specific_to_CPython?= Message-ID: <3g7dx00dT8z7LjS@mail.python.org> http://hg.python.org/cpython/rev/a1ef419c0cfb changeset: 90337:a1ef419c0cfb branch: 2.7 parent: 90332:0c7cf0e598e7 user: Andrew Kuchling date: Tue Apr 15 16:44:43 2014 -0400 summary: #1704474: mark refleak test as specific to CPython Patch by Christian Hudon. files: Lib/test/test_optparse.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_optparse.py b/Lib/test/test_optparse.py --- a/Lib/test/test_optparse.py +++ b/Lib/test/test_optparse.py @@ -383,6 +383,7 @@ self.assertRaises(self.parser.remove_option, ('foo',), None, ValueError, "no such option 'foo'") + @test_support.impl_detail('Relies on sys.getrefcount', cpython=True) def test_refleak(self): # If an OptionParser is carrying around a reference to a large # object, various cycles can prevent it from being GC'd in -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 23:22:30 2014 From: python-checkins at python.org (michael.foord) Date: Tue, 15 Apr 2014 23:22:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closes_issue_2?= =?utf-8?q?1239=2E_unittest=2Emock=2Epatch=2Estopall=28=29_did_not_work?= Message-ID: <3g7fmQ2Sprz7LkV@mail.python.org> http://hg.python.org/cpython/rev/727b7e9c40e3 changeset: 90338:727b7e9c40e3 branch: 3.4 parent: 90333:9fe4d9686162 user: Michael Foord date: Tue Apr 15 17:21:08 2014 -0400 summary: Closes issue 21239. unittest.mock.patch.stopall() did not work deterministically when the same name was patched multiple times. files: Lib/unittest/mock.py | 15 ++++++--- Lib/unittest/test/testmock/testpatch.py | 19 ++++++++++++- Misc/NEWS | 3 ++ 3 files changed, 31 insertions(+), 6 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1050,7 +1050,7 @@ class _patch(object): attribute_name = None - _active_patches = set() + _active_patches = [] def __init__( self, getter, attribute, new, spec, create, @@ -1323,13 +1323,18 @@ def start(self): """Activate a patch, returning any created mock.""" result = self.__enter__() - self._active_patches.add(self) + self._active_patches.append(self) return result def stop(self): """Stop an active patch.""" - self._active_patches.discard(self) + try: + self._active_patches.remove(self) + except ValueError: + # If the patch hasn't been started this will fail + pass + return self.__exit__() @@ -1622,8 +1627,8 @@ def _patch_stopall(): - """Stop all active patches.""" - for patch in list(_patch._active_patches): + """Stop all active patches. LIFO to unroll nested patches.""" + for patch in reversed(_patch._active_patches): patch.stop() diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -12,7 +12,7 @@ from unittest.mock import ( NonCallableMock, CallableMixin, patch, sentinel, MagicMock, Mock, NonCallableMagicMock, patch, _patch, - DEFAULT, call, _get_target + DEFAULT, call, _get_target, _patch ) @@ -1779,6 +1779,23 @@ patched() self.assertIs(os.path, path) + def test_stopall_lifo(self): + stopped = [] + class thing(object): + one = two = three = None + + def get_patch(attribute): + class mypatch(_patch): + def stop(self): + stopped.append(attribute) + return super(mypatch, self).stop() + return mypatch(lambda: thing, attribute, None, None, + False, None, None, None, {}) + [get_patch(val).start() for val in ("one", "two", "three")] + patch.stopall() + + self.assertEqual(stopped, ["three", "two", "one"]) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -37,6 +37,9 @@ Library ------- +- Issue #21239: patch.stopall() didn't work deterministically when the same + name was patched more than once. + - Issue #21222: Passing name keyword argument to mock.create_autospec now works. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 15 23:22:35 2014 From: python-checkins at python.org (michael.foord) Date: Tue, 15 Apr 2014 23:22:35 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <3g7fmW4cWvz7Ljr@mail.python.org> http://hg.python.org/cpython/rev/42d59de284e2 changeset: 90339:42d59de284e2 parent: 90336:69d479e9aa82 parent: 90338:727b7e9c40e3 user: Michael Foord date: Tue Apr 15 17:22:22 2014 -0400 summary: Merge files: Lib/unittest/mock.py | 15 ++++++--- Lib/unittest/test/testmock/testpatch.py | 19 ++++++++++++- Misc/NEWS | 5 ++- 3 files changed, 32 insertions(+), 7 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1054,7 +1054,7 @@ class _patch(object): attribute_name = None - _active_patches = set() + _active_patches = [] def __init__( self, getter, attribute, new, spec, create, @@ -1330,13 +1330,18 @@ def start(self): """Activate a patch, returning any created mock.""" result = self.__enter__() - self._active_patches.add(self) + self._active_patches.append(self) return result def stop(self): """Stop an active patch.""" - self._active_patches.discard(self) + try: + self._active_patches.remove(self) + except ValueError: + # If the patch hasn't been started this will fail + pass + return self.__exit__() @@ -1629,8 +1634,8 @@ def _patch_stopall(): - """Stop all active patches.""" - for patch in list(_patch._active_patches): + """Stop all active patches. LIFO to unroll nested patches.""" + for patch in reversed(_patch._active_patches): patch.stop() diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -12,7 +12,7 @@ from unittest.mock import ( NonCallableMock, CallableMixin, patch, sentinel, MagicMock, Mock, NonCallableMagicMock, patch, _patch, - DEFAULT, call, _get_target + DEFAULT, call, _get_target, _patch ) @@ -1799,6 +1799,23 @@ patched() self.assertIs(os.path, path) + def test_stopall_lifo(self): + stopped = [] + class thing(object): + one = two = three = None + + def get_patch(attribute): + class mypatch(_patch): + def stop(self): + stopped.append(attribute) + return super(mypatch, self).stop() + return mypatch(lambda: thing, attribute, None, None, + False, None, None, None, {}) + [get_patch(val).start() for val in ("one", "two", "three")] + patch.stopall() + + self.assertEqual(stopped, ["three", "two", "one"]) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,9 +46,12 @@ Library ------- +- Issue #21239: patch.stopall() didn't work deterministically when the same + name was patched more than once. + - Issue #21203: Updated fileConfig and dictConfig to remove inconsistencies. Thanks to Jure Koren for the patch. - + - Issue #21222: Passing name keyword argument to mock.create_autospec now works. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 00:14:10 2014 From: python-checkins at python.org (vinay.sajip) Date: Wed, 16 Apr 2014 00:14:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMjQ1?= =?utf-8?q?=3A_updated_documentation_on_exception=28=29_method_and_functio?= =?utf-8?q?n=2E?= Message-ID: <3g7gw22W0Qz7LjM@mail.python.org> http://hg.python.org/cpython/rev/dcb5ca1f1b44 changeset: 90340:dcb5ca1f1b44 branch: 2.7 parent: 90337:a1ef419c0cfb user: Vinay Sajip date: Tue Apr 15 23:11:15 2014 +0100 summary: Issue #21245: updated documentation on exception() method and function. files: Doc/library/logging.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -222,7 +222,7 @@ interpreted as for :meth:`debug`. -.. method:: Logger.exception(msg, *args) +.. method:: Logger.exception(msg, *args, **kwargs) Logs a message with level :const:`ERROR` on this logger. The arguments are interpreted as for :meth:`debug`. Exception info is added to the logging @@ -840,7 +840,7 @@ are interpreted as for :func:`debug`. -.. function:: exception(msg[, *args]) +.. function:: exception(msg[, *args[, **kwargs]]) Logs a message with level :const:`ERROR` on the root logger. The arguments are interpreted as for :func:`debug`. Exception info is added to the logging -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 00:14:11 2014 From: python-checkins at python.org (vinay.sajip) Date: Wed, 16 Apr 2014 00:14:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMjQ1?= =?utf-8?q?=3A_updated_documentation_on_exception=28=29_method_and_functio?= =?utf-8?q?n=2E?= Message-ID: <3g7gw34S2jz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/eee4fd2012ae changeset: 90341:eee4fd2012ae branch: 3.4 parent: 90338:727b7e9c40e3 user: Vinay Sajip date: Tue Apr 15 23:13:12 2014 +0100 summary: Issue #21245: updated documentation on exception() method and function. files: Doc/library/logging.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -250,7 +250,7 @@ interpreted as for :meth:`debug`. -.. method:: Logger.exception(msg, *args) +.. method:: Logger.exception(msg, *args, **kwargs) Logs a message with level :const:`ERROR` on this logger. The arguments are interpreted as for :meth:`debug`. Exception info is added to the logging @@ -992,7 +992,7 @@ are interpreted as for :func:`debug`. -.. function:: exception(msg, *args) +.. function:: exception(msg, *args, **kwargs) Logs a message with level :const:`ERROR` on the root logger. The arguments are interpreted as for :func:`debug`. Exception info is added to the logging -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 00:14:12 2014 From: python-checkins at python.org (vinay.sajip) Date: Wed, 16 Apr 2014 00:14:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2321245=3A_merged_update_from_3=2E4=2E?= Message-ID: <3g7gw45srRz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/a0ec713964b1 changeset: 90342:a0ec713964b1 parent: 90339:42d59de284e2 parent: 90341:eee4fd2012ae user: Vinay Sajip date: Tue Apr 15 23:13:54 2014 +0100 summary: Closes #21245: merged update from 3.4. files: Doc/library/logging.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -250,7 +250,7 @@ interpreted as for :meth:`debug`. -.. method:: Logger.exception(msg, *args) +.. method:: Logger.exception(msg, *args, **kwargs) Logs a message with level :const:`ERROR` on this logger. The arguments are interpreted as for :meth:`debug`. Exception info is added to the logging @@ -992,7 +992,7 @@ are interpreted as for :func:`debug`. -.. function:: exception(msg, *args) +.. function:: exception(msg, *args, **kwargs) Logs a message with level :const:`ERROR` on the root logger. The arguments are interpreted as for :func:`debug`. Exception info is added to the logging -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 02:30:12 2014 From: python-checkins at python.org (r.david.murray) Date: Wed, 16 Apr 2014 02:30:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzIwODc0OiB1cGRh?= =?utf-8?q?te_tutorial_wording=3A_sophisticated_line_editing_is_now_standa?= =?utf-8?b?cmQu?= Message-ID: <3g7kx01vlzz7LjV@mail.python.org> http://hg.python.org/cpython/rev/cbe0cf95fe37 changeset: 90343:cbe0cf95fe37 branch: 3.4 parent: 90341:eee4fd2012ae user: R David Murray date: Tue Apr 15 20:25:18 2014 -0400 summary: #20874: update tutorial wording: sophisticated line editing is now standard. Patch by Rafael Mejia. files: Doc/tutorial/interpreter.rst | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -35,10 +35,9 @@ status. If that doesn't work, you can exit the interpreter by typing the following command: ``quit()``. -The interpreter's line-editing features usually aren't very sophisticated. On -Unix, whoever installed the interpreter may have enabled support for the GNU -readline library, which adds more elaborate interactive editing and history -features. Perhaps the quickest check to see whether command line editing is +The interpreter's line-editing features include interactive editing, history +substitution and code completion on systems that support readline. +Perhaps the quickest check to see whether command line editing is supported is typing Control-P to the first Python prompt you get. If it beeps, you have command line editing; see Appendix :ref:`tut-interacting` for an introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 02:30:13 2014 From: python-checkins at python.org (r.david.murray) Date: Wed, 16 Apr 2014 02:30:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzIwODc0OiByZWZs?= =?utf-8?q?ow_paragraph=2E?= Message-ID: <3g7kx13ckVz7Ljr@mail.python.org> http://hg.python.org/cpython/rev/b5f4ab357ff9 changeset: 90344:b5f4ab357ff9 branch: 3.4 user: R David Murray date: Tue Apr 15 20:26:54 2014 -0400 summary: #20874: reflow paragraph. files: Doc/tutorial/interpreter.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -36,13 +36,13 @@ following command: ``quit()``. The interpreter's line-editing features include interactive editing, history -substitution and code completion on systems that support readline. -Perhaps the quickest check to see whether command line editing is -supported is typing Control-P to the first Python prompt you get. If it beeps, -you have command line editing; see Appendix :ref:`tut-interacting` for an -introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, -command line editing isn't available; you'll only be able to use backspace to -remove characters from the current line. +substitution and code completion on systems that support readline. Perhaps the +quickest check to see whether command line editing is supported is typing +Control-P to the first Python prompt you get. If it beeps, you have command +line editing; see Appendix :ref:`tut-interacting` for an introduction to the +keys. If nothing appears to happen, or if ``^P`` is echoed, command line +editing isn't available; you'll only be able to use backspace to remove +characters from the current line. The interpreter operates somewhat like the Unix shell: when called with standard input connected to a tty device, it reads and executes commands interactively; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 02:30:14 2014 From: python-checkins at python.org (r.david.murray) Date: Wed, 16 Apr 2014 02:30:14 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_=2320874_fix=2E?= Message-ID: <3g7kx25VKSz7Lk0@mail.python.org> http://hg.python.org/cpython/rev/d4e3bea03f9f changeset: 90345:d4e3bea03f9f parent: 90342:a0ec713964b1 parent: 90344:b5f4ab357ff9 user: R David Murray date: Tue Apr 15 20:30:00 2014 -0400 summary: Merge #20874 fix. files: Doc/tutorial/interpreter.rst | 17 ++++++++--------- 1 files changed, 8 insertions(+), 9 deletions(-) diff --git a/Doc/tutorial/interpreter.rst b/Doc/tutorial/interpreter.rst --- a/Doc/tutorial/interpreter.rst +++ b/Doc/tutorial/interpreter.rst @@ -35,15 +35,14 @@ status. If that doesn't work, you can exit the interpreter by typing the following command: ``quit()``. -The interpreter's line-editing features usually aren't very sophisticated. On -Unix, whoever installed the interpreter may have enabled support for the GNU -readline library, which adds more elaborate interactive editing and history -features. Perhaps the quickest check to see whether command line editing is -supported is typing Control-P to the first Python prompt you get. If it beeps, -you have command line editing; see Appendix :ref:`tut-interacting` for an -introduction to the keys. If nothing appears to happen, or if ``^P`` is echoed, -command line editing isn't available; you'll only be able to use backspace to -remove characters from the current line. +The interpreter's line-editing features include interactive editing, history +substitution and code completion on systems that support readline. Perhaps the +quickest check to see whether command line editing is supported is typing +Control-P to the first Python prompt you get. If it beeps, you have command +line editing; see Appendix :ref:`tut-interacting` for an introduction to the +keys. If nothing appears to happen, or if ``^P`` is echoed, command line +editing isn't available; you'll only be able to use backspace to remove +characters from the current line. The interpreter operates somewhat like the Unix shell: when called with standard input connected to a tty device, it reads and executes commands interactively; -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 03:13:21 2014 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 16 Apr 2014 03:13:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE1ODQwOiBtYWtl?= =?utf-8?q?_docs_consistent_by_saying_operations_on_closed_files_raise?= Message-ID: <3g7ltn0Dm5z7LjR@mail.python.org> http://hg.python.org/cpython/rev/7a27039e4a42 changeset: 90346:7a27039e4a42 branch: 3.4 parent: 90344:b5f4ab357ff9 user: Andrew Kuchling date: Tue Apr 15 21:11:36 2014 -0400 summary: #15840: make docs consistent by saying operations on closed files raise ValueError. Patch by Caelyn McAulay. Neither Caelyn nor I could find any cases in 2.7 or 3.4/5 where an operation on a closed stream raised IOError; generally the C implementations have a macro to check for the stream being closed, and these macros all raised ValueError. If we find any, a new bug should be opened. files: Modules/_io/iobase.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -42,8 +42,8 @@ "bytes. bytearrays are accepted too, and in some cases (such as\n" "readinto) needed. Text I/O classes work with str data.\n" "\n" - "Note that calling any method (even inquiries) on a closed stream is\n" - "undefined. Implementations may raise IOError in this case.\n" + "Note that calling any method (except additional calls to close(),\n" + "which are ignored) on a closed stream should raise a ValueError.\n" "\n" "IOBase (and its subclasses) support the iterator protocol, meaning\n" "that an IOBase object can be iterated over yielding the lines in a\n" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 03:16:58 2014 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 16 Apr 2014 03:16:58 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE3MDQ0NzQ6IG1h?= =?utf-8?q?rk_refleak_test_as_specific_to_CPython?= Message-ID: <3g7lyy3f2Bz7LjR@mail.python.org> http://hg.python.org/cpython/rev/c76d782cb9df changeset: 90347:c76d782cb9df branch: 3.4 user: Andrew Kuchling date: Tue Apr 15 16:44:43 2014 -0400 summary: #1704474: mark refleak test as specific to CPython Patch by Christian Hudon. files: Lib/test/test_optparse.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_optparse.py b/Lib/test/test_optparse.py --- a/Lib/test/test_optparse.py +++ b/Lib/test/test_optparse.py @@ -395,6 +395,7 @@ self.assertRaises(self.parser.remove_option, ('foo',), None, ValueError, "no such option 'foo'") + @support.impl_detail('Relies on sys.getrefcount', cpython=True) def test_refleak(self): # If an OptionParser is carrying around a reference to a large # object, various cycles can prevent it from being GC'd in -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 03:22:10 2014 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 16 Apr 2014 03:22:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E4?= Message-ID: <3g7m4y1N70z7LkV@mail.python.org> http://hg.python.org/cpython/rev/e50417fc4b51 changeset: 90348:e50417fc4b51 parent: 90345:d4e3bea03f9f parent: 90347:c76d782cb9df user: Andrew Kuchling date: Tue Apr 15 21:21:56 2014 -0400 summary: Merge from 3.4 files: Lib/test/test_optparse.py | 1 + Modules/_io/iobase.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_optparse.py b/Lib/test/test_optparse.py --- a/Lib/test/test_optparse.py +++ b/Lib/test/test_optparse.py @@ -395,6 +395,7 @@ self.assertRaises(self.parser.remove_option, ('foo',), None, ValueError, "no such option 'foo'") + @support.impl_detail('Relies on sys.getrefcount', cpython=True) def test_refleak(self): # If an OptionParser is carrying around a reference to a large # object, various cycles can prevent it from being GC'd in diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -42,8 +42,8 @@ "bytes. bytearrays are accepted too, and in some cases (such as\n" "readinto) needed. Text I/O classes work with str data.\n" "\n" - "Note that calling any method (even inquiries) on a closed stream is\n" - "undefined. Implementations may raise IOError in this case.\n" + "Note that calling any method (except additional calls to close(),\n" + "which are ignored) on a closed stream should raise a ValueError.\n" "\n" "IOBase (and its subclasses) support the iterator protocol, meaning\n" "that an IOBase object can be iterated over yielding the lines in a\n" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 04:29:16 2014 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 16 Apr 2014 04:29:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2320103=3A_Rewrite_descri?= =?utf-8?q?ption_of_itertools=2Eaccumulate=28=29=2E?= Message-ID: <3g7nZN0Cmhz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/9e1d2150fff2 changeset: 90349:9e1d2150fff2 user: Andrew Kuchling date: Tue Apr 15 22:28:40 2014 -0400 summary: #20103: Rewrite description of itertools.accumulate(). Patch by Christian Hudon. files: Doc/library/itertools.rst | 13 +++++++++---- 1 files changed, 9 insertions(+), 4 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -87,10 +87,15 @@ .. function:: accumulate(iterable[, func]) - Make an iterator that returns accumulated sums. Elements may be any addable - type including :class:`~decimal.Decimal` or :class:`~fractions.Fraction`. - If the optional *func* argument is supplied, it should be a function of two - arguments and it will be used instead of addition. + Make an iterator that returns accumulated sums, or accumulated + results of other binary functions (specified via the optional + *func* argument). If *func* is supplied, it should be a function + of two arguments. Elements of the input *iterable* may be any type + that can be accepted as arguments to *func*. (For example, with + the default operation of addition, elements may be any addable + type including :class:`~decimal.Decimal` or + :class:`~fractions.Fraction`.) If the input iterable is empty, the + output iterable will also be empty. Equivalent to:: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 04:38:07 2014 From: python-checkins at python.org (eric.smith) Date: Wed, 16 Apr 2014 04:38:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_issue_=238931=3A_Mak?= =?utf-8?q?e_alternate_formatting_for_=27c=27_raise_an_exception=2E_Patch?= Message-ID: <3g7nmb36Pgz7Lk8@mail.python.org> http://hg.python.org/cpython/rev/b1aba042b36c changeset: 90350:b1aba042b36c user: Eric V. Smith date: Tue Apr 15 22:37:55 2014 -0400 summary: Close issue #8931: Make alternate formatting for 'c' raise an exception. Patch by Torsten Landschoff. files: Lib/test/test_types.py | 2 ++ Misc/NEWS | 4 ++++ Python/formatter_unicode.c | 7 +++++++ 3 files changed, 13 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -343,6 +343,8 @@ self.assertRaises(ValueError, 3 .__format__, ",n") # can't have ',' with 'c' self.assertRaises(ValueError, 3 .__format__, ",c") + # can't have '#' with 'c' + self.assertRaises(ValueError, 3 .__format__, "#c") # ensure that only int and float type specifiers work for format_spec in ([chr(x) for x in range(ord('a'), ord('z')+1)] + diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -43,6 +43,10 @@ replacement fields. It now matches the behavior of str.format() in this regard. Patches by Phil Elson and Ramchandra Apte. +- Issue #8931: Make alternate formatting ('#') for type 'c' raise an + exception. In versions prior to 3.5, '#' with 'c' had no effect. Now + specifying it is an error. Patch by Torsten Landschoff. + Library ------- diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -846,6 +846,13 @@ " format specifier 'c'"); goto done; } + /* error to request alternate format */ + if (format->alternate) { + PyErr_SetString(PyExc_ValueError, + "Alternate form (#) not allowed with integer" + " format specifier 'c'"); + goto done; + } /* taken from unicodeobject.c formatchar() */ /* Integer input truncated to a character */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 05:45:23 2014 From: python-checkins at python.org (terry.reedy) Date: Wed, 16 Apr 2014 05:45:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4NTY2?= =?utf-8?q?=3A_Clarify_unittest_setUp=2C_tearDown_doc=2E_Patch_by_Nitika_A?= =?utf-8?q?garwal=2E?= Message-ID: <3g7qGC363Cz7LjM@mail.python.org> http://hg.python.org/cpython/rev/fdadc152b964 changeset: 90351:fdadc152b964 branch: 2.7 parent: 90340:dcb5ca1f1b44 user: Terry Jan Reedy date: Tue Apr 15 23:38:11 2014 -0400 summary: Issue #18566: Clarify unittest setUp, tearDown doc. Patch by Nitika Agarwal. files: Doc/library/unittest.rst | 14 +++++++------- Misc/ACKS | 1 + 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -719,9 +719,9 @@ .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; any exception raised by this method will - be considered an error rather than a test failure. The default - implementation does nothing. + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than + a test failure. The default implementation does nothing. .. method:: tearDown() @@ -729,10 +729,10 @@ Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception raised by this - method will be considered an error rather than a test failure. This - method will only be called if the :meth:`setUp` succeeds, regardless of - the outcome of the test method. The default implementation does nothing. + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, + regardless of the outcome of the test method. The default implementation does nothing. .. method:: setUpClass() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -18,6 +18,7 @@ Marc Abramowitz Ron Adam Ali Afshar +Nitika Agarwal Jim Ahlstrom Farhan Ahmad Matthew Ahrens -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 05:45:24 2014 From: python-checkins at python.org (terry.reedy) Date: Wed, 16 Apr 2014 05:45:24 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE4NTY2?= =?utf-8?q?=3A_Clarify_unittest_setUp=2C_tearDown_doc=2E_Patch_by_Nitika_A?= =?utf-8?q?garwal=2E?= Message-ID: <3g7qGD4rQgz7LjS@mail.python.org> http://hg.python.org/cpython/rev/9ab66a7f654a changeset: 90352:9ab66a7f654a branch: 3.4 parent: 90347:c76d782cb9df user: Terry Jan Reedy date: Tue Apr 15 23:38:18 2014 -0400 summary: Issue #18566: Clarify unittest setUp, tearDown doc. Patch by Nitika Agarwal. files: Doc/library/unittest.rst | 14 +++++++------- Misc/ACKS | 1 + 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -674,9 +674,9 @@ .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; any exception raised by this method will - be considered an error rather than a test failure. The default - implementation does nothing. + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than + a test failure. The default implementation does nothing. .. method:: tearDown() @@ -684,10 +684,10 @@ Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception raised by this - method will be considered an error rather than a test failure. This - method will only be called if the :meth:`setUp` succeeds, regardless of - the outcome of the test method. The default implementation does nothing. + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, + regardless of the outcome of the test method. The default implementation does nothing. .. method:: setUpClass() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -18,6 +18,7 @@ Marc Abramowitz Ron Adam Ali Afshar +Nitika Agarwal Jim Ahlstrom Farhan Ahmad Matthew Ahrens -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 05:45:25 2014 From: python-checkins at python.org (terry.reedy) Date: Wed, 16 Apr 2014 05:45:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3g7qGF74Byz7Lk3@mail.python.org> http://hg.python.org/cpython/rev/90bf8f791ce4 changeset: 90353:90bf8f791ce4 parent: 90350:b1aba042b36c parent: 90352:9ab66a7f654a user: Terry Jan Reedy date: Tue Apr 15 23:38:44 2014 -0400 summary: Merge with 3.4 files: Doc/library/unittest.rst | 14 +++++++------- Misc/ACKS | 1 + 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -674,9 +674,9 @@ .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; any exception raised by this method will - be considered an error rather than a test failure. The default - implementation does nothing. + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than + a test failure. The default implementation does nothing. .. method:: tearDown() @@ -684,10 +684,10 @@ Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception raised by this - method will be considered an error rather than a test failure. This - method will only be called if the :meth:`setUp` succeeds, regardless of - the outcome of the test method. The default implementation does nothing. + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, + regardless of the outcome of the test method. The default implementation does nothing. .. method:: setUpClass() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -18,6 +18,7 @@ Marc Abramowitz Ron Adam Ali Afshar +Nitika Agarwal Jim Ahlstrom Farhan Ahmad Matthew Ahrens -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 05:45:27 2014 From: python-checkins at python.org (terry.reedy) Date: Wed, 16 Apr 2014 05:45:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4NTY2OiBXaGl0?= =?utf-8?q?espace?= Message-ID: <3g7qGH2PCHz7LkV@mail.python.org> http://hg.python.org/cpython/rev/72b1715e18c1 changeset: 90354:72b1715e18c1 branch: 2.7 parent: 90351:fdadc152b964 user: Terry Jan Reedy date: Tue Apr 15 23:44:10 2014 -0400 summary: #18566: Whitespace files: Doc/library/unittest.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -719,8 +719,8 @@ .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, - any exception raised by this method will be considered an error rather than + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing. @@ -729,9 +729,9 @@ Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception, other than :exc:`AssertionError` - or :exc:`SkipTest`, raised by this method will be considered an error rather than a - test failure. This method will only be called if the :meth:`setUp` succeeds, + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, regardless of the outcome of the test method. The default implementation does nothing. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 05:45:28 2014 From: python-checkins at python.org (terry.reedy) Date: Wed, 16 Apr 2014 05:45:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE4NTY2OiBXaGl0?= =?utf-8?q?espace?= Message-ID: <3g7qGJ4W83z7Lkg@mail.python.org> http://hg.python.org/cpython/rev/a37440dec1b6 changeset: 90355:a37440dec1b6 branch: 3.4 parent: 90352:9ab66a7f654a user: Terry Jan Reedy date: Tue Apr 15 23:44:14 2014 -0400 summary: #18566: Whitespace files: Doc/library/unittest.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -674,8 +674,8 @@ .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, - any exception raised by this method will be considered an error rather than + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing. @@ -684,9 +684,9 @@ Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception, other than :exc:`AssertionError` - or :exc:`SkipTest`, raised by this method will be considered an error rather than a - test failure. This method will only be called if the :meth:`setUp` succeeds, + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, regardless of the outcome of the test method. The default implementation does nothing. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 05:45:29 2014 From: python-checkins at python.org (terry.reedy) Date: Wed, 16 Apr 2014 05:45:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3g7qGK6GkHz7LkW@mail.python.org> http://hg.python.org/cpython/rev/1b25b2fd2803 changeset: 90356:1b25b2fd2803 parent: 90353:90bf8f791ce4 parent: 90355:a37440dec1b6 user: Terry Jan Reedy date: Tue Apr 15 23:44:37 2014 -0400 summary: Merge with 3.4 files: Doc/library/unittest.rst | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -674,8 +674,8 @@ .. method:: setUp() Method called to prepare the test fixture. This is called immediately - before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, - any exception raised by this method will be considered an error rather than + before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, + any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing. @@ -684,9 +684,9 @@ Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly - careful about checking internal state. Any exception, other than :exc:`AssertionError` - or :exc:`SkipTest`, raised by this method will be considered an error rather than a - test failure. This method will only be called if the :meth:`setUp` succeeds, + careful about checking internal state. Any exception, other than :exc:`AssertionError` + or :exc:`SkipTest`, raised by this method will be considered an error rather than a + test failure. This method will only be called if the :meth:`setUp` succeeds, regardless of the outcome of the test method. The default implementation does nothing. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Wed Apr 16 09:41:17 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 16 Apr 2014 09:41:17 +0200 Subject: [Python-checkins] Daily reference leaks (e50417fc4b51): sum=-1 Message-ID: results for e50417fc4b51 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, -2, 0] references, sum=-2 test_site leaked [0, -2, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogk1YWly', '-x'] From python-checkins at python.org Wed Apr 16 15:11:08 2014 From: python-checkins at python.org (andrew.kuchling) Date: Wed, 16 Apr 2014 15:11:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=236490=3A_Expand_document?= =?utf-8?b?YXRpb24gZm9yIG9zLnBvcGVuKCku?= Message-ID: <3g83q051Nqz7LkW@mail.python.org> http://hg.python.org/cpython/rev/3417a95df7e2 changeset: 90357:3417a95df7e2 user: Andrew Kuchling date: Wed Apr 16 09:10:53 2014 -0400 summary: #6490: Expand documentation for os.popen(). Patch by Sam Kimbrel. files: Doc/library/os.rst | 25 +++++++++++++++++++++---- 1 files changed, 21 insertions(+), 4 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -2730,10 +2730,27 @@ Availability: Unix. -.. function:: popen(...) - - Run child processes, returning opened pipes for communications. These functions - are described in section :ref:`os-newstreams`. +.. function:: popen(command, mode='r', buffering=-1) + + Open a pipe to or from *command*. The return value is an open file object + connected to the pipe, which can be read or written depending on whether *mode* + is ``'r'`` (default) or ``'w'``. The *buffering* argument has the same meaning as + the corresponding argument to the built-in :func:`open` function. The + returned file object reads or writes text strings rather than bytes. + + The ``close`` method returns :const:`None` if the subprocess exited + successfully, or the subprocess's return code if there was an + error. On POSIX systems, if the return code is positive it + represents the return value of the process left-shifted by one + byte. If the return code is negative, the process was terminated + by the signal given by the negated value of the return code. (For + example, the return value might be ``- signal.SIGKILL`` if the + subprocess was killed.) On Windows systems, the return value + contains the signed integer return code from the child process. + + This is implemented using :class:`subprocess.Popen`; see that class's + documentation for more powerful ways to manage and communicate with + subprocesses. .. function:: spawnl(mode, path, ...) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 18:34:08 2014 From: python-checkins at python.org (antoine.pitrou) Date: Wed, 16 Apr 2014 18:34:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Try_to_fix_bui?= =?utf-8?q?ldbot_failures_on_old_OpenSSLs_=28=3C_1=2E0=2E0=29_-_followup_t?= =?utf-8?q?o_issue?= Message-ID: <3g88KD4bBLz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/d6501421b86b changeset: 90358:d6501421b86b branch: 3.4 parent: 90355:a37440dec1b6 user: Antoine Pitrou date: Wed Apr 16 18:33:39 2014 +0200 summary: Try to fix buildbot failures on old OpenSSLs (< 1.0.0) - followup to issue #21015 files: Lib/test/test_ssl.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -2593,7 +2593,12 @@ # should be enabled by default on SSL contexts. context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) context.load_cert_chain(CERTFILE) - context.set_ciphers("ECDH") + # Prior to OpenSSL 1.0.0, ECDH ciphers have to be enabled + # explicitly using the 'ECCdraft' cipher alias. Otherwise, + # our default cipher list should prefer ECDH-based ciphers + # automatically. + if ssl.OPENSSL_VERSION_INFO < (1, 0, 0): + context.set_ciphers("ECCdraft:ECDH") with ThreadedEchoServer(context=context) as server: with context.wrap_socket(socket.socket()) as s: s.connect((HOST, server.port)) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 18:34:09 2014 From: python-checkins at python.org (antoine.pitrou) Date: Wed, 16 Apr 2014 18:34:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Try_to_fix_buildbot_failures_on_old_OpenSSLs_=28=3C_1=2E?= =?utf-8?q?0=2E0=29_-_followup_to_issue?= Message-ID: <3g88KF6GSKz7Ll1@mail.python.org> http://hg.python.org/cpython/rev/1305410bff2d changeset: 90359:1305410bff2d parent: 90357:3417a95df7e2 parent: 90358:d6501421b86b user: Antoine Pitrou date: Wed Apr 16 18:34:01 2014 +0200 summary: Try to fix buildbot failures on old OpenSSLs (< 1.0.0) - followup to issue #21015 files: Lib/test/test_ssl.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -2593,7 +2593,12 @@ # should be enabled by default on SSL contexts. context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) context.load_cert_chain(CERTFILE) - context.set_ciphers("ECDH") + # Prior to OpenSSL 1.0.0, ECDH ciphers have to be enabled + # explicitly using the 'ECCdraft' cipher alias. Otherwise, + # our default cipher list should prefer ECDH-based ciphers + # automatically. + if ssl.OPENSSL_VERSION_INFO < (1, 0, 0): + context.set_ciphers("ECCdraft:ECDH") with ThreadedEchoServer(context=context) as server: with context.wrap_socket(socket.socket()) as s: s.connect((HOST, server.port)) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 18:57:35 2014 From: python-checkins at python.org (antoine.pitrou) Date: Wed, 16 Apr 2014 18:57:35 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320896=3A_ssl=2Ege?= =?utf-8?q?t=5Fserver=5Fcertificate=28=29_now_uses_PROTOCOL=5FSSLv23=2C_no?= =?utf-8?q?t?= Message-ID: <3g88rH0Wvkz7Lkp@mail.python.org> http://hg.python.org/cpython/rev/55f62fa5bebc changeset: 90360:55f62fa5bebc user: Antoine Pitrou date: Wed Apr 16 18:56:28 2014 +0200 summary: Issue #20896: ssl.get_server_certificate() now uses PROTOCOL_SSLv23, not PROTOCOL_SSLv3, for maximum compatibility. files: Doc/library/ssl.rst | 6 +++++- Lib/ssl.py | 2 +- Lib/test/test_ssl.py | 5 +---- Misc/NEWS | 3 +++ 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -387,7 +387,7 @@ >>> time.ctime(ssl.cert_time_to_seconds("May 9 00:00:00 2007 GMT")) 'Wed May 9 00:00:00 2007' -.. function:: get_server_certificate(addr, ssl_version=PROTOCOL_SSLv3, ca_certs=None) +.. function:: get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None) Given the address ``addr`` of an SSL-protected server, as a (*hostname*, *port-number*) pair, fetches the server's certificate, and returns it as a @@ -401,6 +401,10 @@ .. versionchanged:: 3.3 This function is now IPv6-compatible. + .. versionchanged:: 3.5 + The default *ssl_version* is changed from :data:`PROTOCOL_SSLv3` to + :data:`PROTOCOL_SSLv23` for maximum compatibility with modern servers. + .. function:: DER_cert_to_PEM_cert(DER_cert_bytes) Given a certificate as a DER-encoded blob of bytes, returns a PEM-encoded diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -922,7 +922,7 @@ d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)] return base64.decodebytes(d.encode('ASCII', 'strict')) -def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv3, ca_certs=None): +def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None): """Retrieve the certificate from the server at the specified address, and return it as a PEM-encoded string. If 'ca_certs' is specified, validate the server cert against it. diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1371,14 +1371,12 @@ def test_get_server_certificate(self): def _test_get_server_certificate(host, port, cert=None): with support.transient_internet(host): - pem = ssl.get_server_certificate((host, port), - ssl.PROTOCOL_SSLv23) + pem = ssl.get_server_certificate((host, port)) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) try: pem = ssl.get_server_certificate((host, port), - ssl.PROTOCOL_SSLv23, ca_certs=CERTFILE) except ssl.SSLError as x: #should fail @@ -1388,7 +1386,6 @@ self.fail("Got server certificate %s for %s:%s!" % (pem, host, port)) pem = ssl.get_server_certificate((host, port), - ssl.PROTOCOL_SSLv23, ca_certs=cert) if not pem: self.fail("No server certificate on %s:%s!" % (host, port)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,6 +50,9 @@ Library ------- +- Issue #20896: ssl.get_server_certificate() now uses PROTOCOL_SSLv23, not + PROTOCOL_SSLv3, for maximum compatibility. + - Issue #21239: patch.stopall() didn't work deterministically when the same name was patched more than once. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 19:56:56 2014 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 16 Apr 2014 19:56:56 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Address_issue_?= =?utf-8?q?18229_-_Explain_http=2Eserver=2EBaseHTTPRequestHandler=27s_=2Eh?= =?utf-8?q?eaders?= Message-ID: <3g8B8m0Kvrz7Lky@mail.python.org> http://hg.python.org/cpython/rev/104fab0143e9 changeset: 90361:104fab0143e9 branch: 3.4 parent: 90358:d6501421b86b user: Senthil Kumaran date: Wed Apr 16 13:56:19 2014 -0400 summary: Address issue 18229 - Explain http.server.BaseHTTPRequestHandler's .headers attribute further. Initial patch by Caelyn McAulay. files: Doc/library/http.server.rst | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Doc/library/http.server.rst b/Doc/library/http.server.rst --- a/Doc/library/http.server.rst +++ b/Doc/library/http.server.rst @@ -81,7 +81,10 @@ Holds an instance of the class specified by the :attr:`MessageClass` class variable. This instance parses and manages the headers in the HTTP - request. + request. The :func:`~http.client.parse_headers` function from + :mod:`http.client` is used to parse the headers and it requires that the + HTTP request provide a valid :rfc:`2822` style header. + .. attribute:: rfile -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 19:56:57 2014 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 16 Apr 2014 19:56:57 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g8B8n24vkz7Lkx@mail.python.org> http://hg.python.org/cpython/rev/110385183d16 changeset: 90362:110385183d16 parent: 90360:55f62fa5bebc parent: 90361:104fab0143e9 user: Senthil Kumaran date: Wed Apr 16 13:56:41 2014 -0400 summary: merge 3.4 files: Doc/library/http.server.rst | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Doc/library/http.server.rst b/Doc/library/http.server.rst --- a/Doc/library/http.server.rst +++ b/Doc/library/http.server.rst @@ -81,7 +81,10 @@ Holds an instance of the class specified by the :attr:`MessageClass` class variable. This instance parses and manages the headers in the HTTP - request. + request. The :func:`~http.client.parse_headers` function from + :mod:`http.client` is used to parse the headers and it requires that the + HTTP request provide a valid :rfc:`2822` style header. + .. attribute:: rfile -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 20:06:59 2014 From: python-checkins at python.org (kushal.das) Date: Wed, 16 Apr 2014 20:06:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_Issue_21238=3A_New_?= =?utf-8?q?keyword_argument_=60unsafe=60_to_Mock=2E?= Message-ID: <3g8BNM2Bz7z7LjW@mail.python.org> http://hg.python.org/cpython/rev/e4ee0b15cc4f changeset: 90363:e4ee0b15cc4f user: Kushal Das date: Wed Apr 16 23:32:21 2014 +0530 summary: Closes Issue 21238: New keyword argument `unsafe` to Mock. It raises `AttributeError` incase of an attribute startswith assert or assret. files: Doc/library/unittest.mock.rst | 8 +++++++- Lib/unittest/mock.py | 8 ++++++-- Lib/unittest/test/testmock/testmock.py | 11 +++++++++++ Misc/NEWS | 3 +++ 4 files changed, 27 insertions(+), 3 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -198,7 +198,7 @@ the `new_callable` argument to `patch`. -.. class:: Mock(spec=None, side_effect=None, return_value=DEFAULT, wraps=None, name=None, spec_set=None, **kwargs) +.. class:: Mock(spec=None, side_effect=None, return_value=DEFAULT, wraps=None, name=None, spec_set=None, unsafe=False, **kwargs) Create a new `Mock` object. `Mock` takes several optional arguments that specify the behaviour of the Mock object: @@ -235,6 +235,12 @@ this is a new Mock (created on first access). See the :attr:`return_value` attribute. + * `unsafe`: By default if any attribute starts with *assert* or + *assret* will raise an `AttributeError`. Passing `unsafe=True` will allow + access to these attributes. + + .. versionadded:: 3.5 + * `wraps`: Item for the mock object to wrap. If `wraps` is not None then calling the Mock will pass the call through to the wrapped object (returning the real result). Attribute access on the mock will return a diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -379,7 +379,7 @@ def __init__( self, spec=None, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, - _spec_as_instance=False, _eat_self=None, **kwargs + _spec_as_instance=False, _eat_self=None, unsafe=False, **kwargs ): if _new_parent is None: _new_parent = parent @@ -409,6 +409,7 @@ __dict__['_mock_mock_calls'] = _CallList() __dict__['method_calls'] = _CallList() + __dict__['_mock_unsafe'] = unsafe if kwargs: self.configure_mock(**kwargs) @@ -565,13 +566,16 @@ def __getattr__(self, name): - if name == '_mock_methods': + if name in {'_mock_methods', '_mock_unsafe'}: raise AttributeError(name) elif self._mock_methods is not None: if name not in self._mock_methods or name in _all_magics: raise AttributeError("Mock object has no attribute %r" % name) elif _is_magic(name): raise AttributeError(name) + if not self._mock_unsafe: + if name.startswith(('assert', 'assret')): + raise AttributeError(name) result = self._mock_children.get(name) if result is _deleted: diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -1187,6 +1187,17 @@ m = mock.create_autospec(object(), name='sweet_func') self.assertIn('sweet_func', repr(m)) + #Issue21238 + def test_mock_unsafe(self): + m = Mock() + with self.assertRaises(AttributeError): + m.assert_foo_call() + with self.assertRaises(AttributeError): + m.assret_foo_call() + m = Mock(unsafe=True) + m.assert_foo_call() + m.assret_foo_call() + def test_mock_add_spec(self): class _One(object): one = 1 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,6 +50,9 @@ Library ------- +- Issue #21238: New keyword argument `unsafe` to Mock. It raises + `AttributeError` incase of an attribute startswith assert or assret. + - Issue #20896: ssl.get_server_certificate() now uses PROTOCOL_SSLv23, not PROTOCOL_SSLv3, for maximum compatibility. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 22:06:23 2014 From: python-checkins at python.org (kushal.das) Date: Wed, 16 Apr 2014 22:06:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_Issue_21262=3A_New_?= =?utf-8?q?method_assert=5Fnot=5Fcalled_for_Mock=2E?= Message-ID: <3g8F272wZ3z7Ljj@mail.python.org> http://hg.python.org/cpython/rev/9e5cbc46e916 changeset: 90364:9e5cbc46e916 user: Kushal Das date: Thu Apr 17 01:36:14 2014 +0530 summary: Closes Issue 21262: New method assert_not_called for Mock. It raises AssertionError if the mock has been called. files: Doc/library/unittest.mock.rst | 14 ++++++++++++++ Lib/unittest/mock.py | 8 ++++++++ Lib/unittest/test/testmock/testmock.py | 9 +++++++++ Misc/NEWS | 3 +++ 4 files changed, 34 insertions(+), 0 deletions(-) diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -321,6 +321,20 @@ >>> calls = [call(4), call(2), call(3)] >>> mock.assert_has_calls(calls, any_order=True) + .. method:: assert_not_called(*args, **kwargs) + + Assert the mock was never called. + + >>> m = Mock() + >>> m.hello.assert_not_called() + >>> obj = m.hello() + >>> m.hello.assert_not_called() + Traceback (most recent call last): + ... + AssertionError: Expected 'hello' to not have been called. Called 1 times. + + .. versionadded:: 3.5 + .. method:: reset_mock() diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -758,6 +758,14 @@ else: return _call + def assert_not_called(_mock_self, *args, **kwargs): + """assert that the mock was never called. + """ + self = _mock_self + if self.call_count != 0: + msg = ("Expected '%s' to not have been called. Called %s times." % + (self._mock_name or 'mock', self.call_count)) + raise AssertionError(msg) def assert_called_with(_mock_self, *args, **kwargs): """assert that the mock was called with the specified arguments. diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -1198,6 +1198,15 @@ m.assert_foo_call() m.assret_foo_call() + #Issue21262 + def test_assert_not_called(self): + m = Mock() + m.hello.assert_not_called() + m.hello() + with self.assertRaises(AssertionError): + m.hello.assert_not_called() + + def test_mock_add_spec(self): class _One(object): one = 1 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,6 +50,9 @@ Library ------- +- Issue #21262: New method assert_not_called for Mock. + It raises AssertionError if the mock has been called. + - Issue #21238: New keyword argument `unsafe` to Mock. It raises `AttributeError` incase of an attribute startswith assert or assret. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 22:07:55 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 16 Apr 2014 22:07:55 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_use_the_called?= =?utf-8?q?_property_of_the_run=5Fpip_mock_rather_than_an_assertion_method?= Message-ID: <3g8F3v5Mflz7Ljt@mail.python.org> http://hg.python.org/cpython/rev/2e54b9e205c8 changeset: 90365:2e54b9e205c8 branch: 3.4 parent: 90361:104fab0143e9 user: Benjamin Peterson date: Wed Apr 16 16:06:39 2014 -0400 summary: use the called property of the run_pip mock rather than an assertion method that doesn't exist files: Lib/test/test_ensurepip.py | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_ensurepip.py b/Lib/test/test_ensurepip.py --- a/Lib/test/test_ensurepip.py +++ b/Lib/test/test_ensurepip.py @@ -153,7 +153,7 @@ def test_altinstall_default_pip_conflict(self): with self.assertRaises(ValueError): ensurepip.bootstrap(altinstall=True, default_pip=True) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip def test_pip_environment_variables_removed(self): @@ -194,7 +194,7 @@ def test_uninstall_skipped_when_not_installed(self): with fake_pip(None): ensurepip._uninstall_helper() - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) def test_uninstall_skipped_with_warning_for_wrong_version(self): with fake_pip("not a valid version"): @@ -202,7 +202,7 @@ ensurepip._uninstall_helper() warning = stderr.getvalue().strip() self.assertIn("only uninstall a matching version", warning) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip @@ -272,7 +272,7 @@ self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder" with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"): ensurepip_no_ssl.bootstrap() - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ) def test_uninstall_requires_ssl(self): @@ -280,7 +280,7 @@ with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"): with fake_pip(): ensurepip_no_ssl._uninstall_helper() - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ) def test_main_exits_early_with_warning(self): @@ -288,7 +288,7 @@ ensurepip_no_ssl._main(["--version"]) warning = stderr.getvalue().strip() self.assertTrue(warning.endswith("requires SSL/TLS"), warning) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) # Basic testing of the main functions and their argument parsing @@ -303,7 +303,7 @@ ensurepip._main(["--version"]) result = stdout.getvalue().strip() self.assertEqual(result, EXPECTED_VERSION_OUTPUT) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip def test_basic_bootstrapping(self): @@ -328,7 +328,7 @@ ensurepip._uninstall._main(["--version"]) result = stdout.getvalue().strip() self.assertEqual(result, EXPECTED_VERSION_OUTPUT) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip def test_basic_uninstall(self): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 22:07:56 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 16 Apr 2014 22:07:56 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g8F3w73sMz7Ljt@mail.python.org> http://hg.python.org/cpython/rev/578671ede1be changeset: 90366:578671ede1be parent: 90364:9e5cbc46e916 parent: 90365:2e54b9e205c8 user: Benjamin Peterson date: Wed Apr 16 16:07:49 2014 -0400 summary: merge 3.4 files: Lib/test/test_ensurepip.py | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_ensurepip.py b/Lib/test/test_ensurepip.py --- a/Lib/test/test_ensurepip.py +++ b/Lib/test/test_ensurepip.py @@ -153,7 +153,7 @@ def test_altinstall_default_pip_conflict(self): with self.assertRaises(ValueError): ensurepip.bootstrap(altinstall=True, default_pip=True) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip def test_pip_environment_variables_removed(self): @@ -194,7 +194,7 @@ def test_uninstall_skipped_when_not_installed(self): with fake_pip(None): ensurepip._uninstall_helper() - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) def test_uninstall_skipped_with_warning_for_wrong_version(self): with fake_pip("not a valid version"): @@ -202,7 +202,7 @@ ensurepip._uninstall_helper() warning = stderr.getvalue().strip() self.assertIn("only uninstall a matching version", warning) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip @@ -272,7 +272,7 @@ self.os_environ["PIP_THIS_SHOULD_STAY"] = "test fodder" with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"): ensurepip_no_ssl.bootstrap() - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ) def test_uninstall_requires_ssl(self): @@ -280,7 +280,7 @@ with self.assertRaisesRegex(RuntimeError, "requires SSL/TLS"): with fake_pip(): ensurepip_no_ssl._uninstall_helper() - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) self.assertIn("PIP_THIS_SHOULD_STAY", self.os_environ) def test_main_exits_early_with_warning(self): @@ -288,7 +288,7 @@ ensurepip_no_ssl._main(["--version"]) warning = stderr.getvalue().strip() self.assertTrue(warning.endswith("requires SSL/TLS"), warning) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) # Basic testing of the main functions and their argument parsing @@ -303,7 +303,7 @@ ensurepip._main(["--version"]) result = stdout.getvalue().strip() self.assertEqual(result, EXPECTED_VERSION_OUTPUT) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip def test_basic_bootstrapping(self): @@ -328,7 +328,7 @@ ensurepip._uninstall._main(["--version"]) result = stdout.getvalue().strip() self.assertEqual(result, EXPECTED_VERSION_OUTPUT) - self.run_pip.assert_not_called() + self.assertFalse(self.run_pip.called) @requires_usable_pip def test_basic_uninstall(self): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 22:17:22 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 16 Apr 2014 22:17:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_remove_superfl?= =?utf-8?q?uous_and_useless_line?= Message-ID: <3g8FGp6dZJz7LjP@mail.python.org> http://hg.python.org/cpython/rev/30a7e37b8441 changeset: 90367:30a7e37b8441 branch: 3.4 parent: 90365:2e54b9e205c8 user: Benjamin Peterson date: Wed Apr 16 16:16:37 2014 -0400 summary: remove superfluous and useless line files: Lib/test/test_asyncio/test_unix_events.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py --- a/Lib/test/test_asyncio/test_unix_events.py +++ b/Lib/test/test_asyncio/test_unix_events.py @@ -1335,7 +1335,6 @@ with self.ignore_warnings: self.watcher._sig_chld() - callback.assert_called(m.waitpid) if isinstance(self.watcher, asyncio.FastChildWatcher): # here the FastChildWatche enters a deadlock # (there is no way to prevent it) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 22:17:24 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 16 Apr 2014 22:17:24 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g8FGr1WGbz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/53ba8ec5ece6 changeset: 90368:53ba8ec5ece6 parent: 90366:578671ede1be parent: 90367:30a7e37b8441 user: Benjamin Peterson date: Wed Apr 16 16:17:16 2014 -0400 summary: merge 3.4 files: Lib/test/test_asyncio/test_unix_events.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py --- a/Lib/test/test_asyncio/test_unix_events.py +++ b/Lib/test/test_asyncio/test_unix_events.py @@ -1335,7 +1335,6 @@ with self.ignore_warnings: self.watcher._sig_chld() - callback.assert_called(m.waitpid) if isinstance(self.watcher, asyncio.FastChildWatcher): # here the FastChildWatche enters a deadlock # (there is no way to prevent it) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 22:24:15 2014 From: python-checkins at python.org (kushal.das) Date: Wed, 16 Apr 2014 22:24:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Removes_unused_varargs_and?= =?utf-8?q?_varkwargs_from_assert=5Fnot=5Fcalled=28=29=2E?= Message-ID: <3g8FQl20m1z7Lk6@mail.python.org> http://hg.python.org/cpython/rev/01ae1ac2daf4 changeset: 90369:01ae1ac2daf4 user: Kushal Das date: Thu Apr 17 01:54:07 2014 +0530 summary: Removes unused varargs and varkwargs from assert_not_called(). files: Lib/unittest/mock.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -758,7 +758,7 @@ else: return _call - def assert_not_called(_mock_self, *args, **kwargs): + def assert_not_called(_mock_self): """assert that the mock was never called. """ self = _mock_self -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 16 23:58:16 2014 From: python-checkins at python.org (nick.coghlan) Date: Wed, 16 Apr 2014 23:58:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_466=3A_make_proposal_even?= =?utf-8?q?_more_specific?= Message-ID: <3g8HWD0dwFz7LjP@mail.python.org> http://hg.python.org/peps/rev/ba563b4ccbdd changeset: 5460:ba563b4ccbdd user: Nick Coghlan date: Wed Apr 16 17:58:06 2014 -0400 summary: PEP 466: make proposal even more specific files: pep-0466.txt | 203 +++++++++++++++----------------------- 1 files changed, 81 insertions(+), 122 deletions(-) diff --git a/pep-0466.txt b/pep-0466.txt --- a/pep-0466.txt +++ b/pep-0466.txt @@ -1,5 +1,5 @@ PEP: 466 -Title: Network Security Enhancement Exception for Python 2.7 +Title: Network Security Enhancements for Python 2.7.7 Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , @@ -7,7 +7,7 @@ Type: Informational Content-Type: text/x-rst Created: 23-Mar-2014 -Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014 +Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014, 16-Apr-2014 Abstract @@ -26,14 +26,9 @@ Python 3 in the near term may not be feasible. In recognition of the additional practical considerations that have arisen -during the 4+ year maintenance cycle for Python 2.7, this PEP allows -Python 2.7 standard library components that have implications for the -overall security of the internet to be updated in line with the -corresponding Python 3 feature releases. - -Specifically, the exception allows a critical set of network security -related features to be backported from Python 3.4 to the upcoming Python -2.7.7 maintenance release. +during the 4+ year maintenance cycle for Python 2.7, this PEP allows a +critical set of network security related features to be backported from +Python 3.4 to the upcoming Python 2.7.7 maintenance release. While this PEP does not make any changes to the core development team's handling of security-fix-only branches that are no longer in active @@ -44,10 +39,10 @@ directly to the public internet. -Exemption Policy -================ +New security related features in Python 2.7.7 +============================================= -Under this policy, the following features SHOULD be backported from Python +Under this proposal, the following features will be backported from Python 3.4 to the upcoming Python 2.7.7 maintenance release: * in the ``os`` module: @@ -67,20 +62,26 @@ * in the ``ssl`` module: * this module is almost entirely synchronised with its Python 3 - counterpart, bringing TLSv1.2, SSLContext manipulation, Server Name - Identification, access to platform certificate stores, standard + counterpart, bringing TLSv1.x settings, SSLContext manipulation, Server + Name Indication, access to platform certificate stores, standard library support for peer hostname validation and more to the Python 2 series. * the only ``ssl`` module features *not* backported under this policy are the ``ssl.RAND_*`` functions that provide access to OpenSSL's random number generation capabilities - use ``os.urandom()`` instead. -As part of this policy, permission is also granted to upgrade to newer -feature releases of OpenSSL when preparing the binary installers -for new maintenance releases of Python 2.7. +As a general change in maintenance policy, permission is also granted to +upgrade to newer feature releases of OpenSSL when preparing the binary +installers for new maintenance releases of Python 2.7. +This PEP does NOT propose a general exception for backporting new features +to Python 2.7 - every new feature proposed for backporting will still need +to be justified independently. In particular, it will need to be explained +why relying on and independently updated backport on the Python Package Index +instead is not an acceptable solution. -Backwards Compatibility Considerations + +Backwards compatibility considerations ====================================== As in the Python 3 series, the backported ``ssl.create_default_context()`` @@ -98,7 +99,7 @@ limiting the risk of breaking currently working software when upgrading to a new Python 2.7 maintenance release. -In all cases where this policy allows new features to be backported to +In all cases where this proposal allows new features to be backported to the Python 2.7 release series, it is possible to write cross-version compatible code that operates by "feature detection" (for example, checking for particular attributes in a module), without needing to explicitly check @@ -137,7 +138,7 @@ OpenSSL compatibility --------------------- -Under this policy, OpenSSL may be upgraded to more recent feature releases +Under this proposal, OpenSSL may be upgraded to more recent feature releases in Python 2.7 maintenance releases. On Linux and most other POSIX systems, the specific version of OpenSSL used already varies, as CPython dynamically links to the system provided OpenSSL library by default. @@ -169,6 +170,10 @@ policy, and assisting with any additional maintenance burdens that arise in the Python 2 series as a result. +Steve Dower and Brian Curtin have offered to help with the creation of the +Windows installers, allowing Martin von L?wis the opportunity to step back +from the task of maintaining the 2.7 Windows installer. + This PEP is primarily about establishing the consensus needed to allow them to carry out this work. For other core developers, this policy change shouldn't impose any additional effort beyond potentially reviewing the @@ -176,18 +181,6 @@ affected modules. -Documentation -------------- - -All modules covered by this policy MUST include a "Security Considerations" -section in their documentation in order to take advantage of this policy. - -In addition to any other module specific contents, this section SHOULD -enumerate key security enhancements and fixes (with CVE identifiers where -applicable), along with the feature and maintenance releases that first -included them. - - Security releases ----------------- @@ -197,8 +190,8 @@ However, the recommendations for library and application developers are deliberately designed to accommodate commercial redistributors that choose -to apply this policy to additional Python release series that are either in -security fix only mode, or have been declared "end of life" by the core +to apply these changes to additional Python release series that are either +in security fix only mode, or have been declared "end of life" by the core development team. Whether or not redistributors choose to exercise that option will be up @@ -209,12 +202,12 @@ ------------------- Third party integration testing services should offer users the ability -to test against specific Python 2.7 maintenance releases, to ensure that -libraries, frameworks and applications can still test their handling of the -legacy security infrastructure correctly (either failing or degrading -gracefully, depending on the security sensitivity of the software), even -after the features covered in this policy have been backported to the -Python 2.7 series. +to test against multiple Python 2.7 maintenance releases (at least 2.7.6 +and 2.7.7+), to ensure that libraries, frameworks and applications can still +test their handling of the legacy security infrastructure correctly (either +failing or degrading gracefully, depending on the security sensitivity of +the software), even after the features covered in this proposal have been +backported to the Python 2.7 series. Handling lower security environments with low risk tolerance @@ -222,7 +215,7 @@ For better or for worse (mostly worse), there are some environments where the risk of latent security defects is more tolerated than even a slightly -increased risk of regressions in maintenance releases. This policy largely +increased risk of regressions in maintenance releases. This proposal largely excludes these environments from consideration where the modules covered by the exemption are concerned - this approach is entirely inappropriate for software connected to the public internet, and defence in depth security @@ -236,28 +229,6 @@ scenario. -Evolution of this Policy -======================== - -The key requirement for a feature to be considered for inclusion in this -policy is that it must have security implications *beyond* the specific -application that is written in Python and the system that application is -running on. Thus the focus on network security protocols, password storage -and related cryptographic infrastructure - Python is a popular choice for -the development of web services and clients, and thus the capabilities of -widely used Python versions have implications for the security design of -other services that may themselves be using newer versions of Python or -other development languages, but need to interoperate with clients or -servers written using older versions of Python. - -The intent behind this requirement is to minimise any impact that the -introduction of this policy may have on the stability and compatibility of -maintenance releases. It would be thoroughly counterproductive if end -users became as cautious about updating to new Python 2.7 maintenance -releases as they are about updating to new feature releases within the -same release series. - - Motivation and Rationale ======================== @@ -280,19 +251,26 @@ While the use of the system OpenSSL installation addresses many of these concerns on Linux platforms, it doesn't address all of them (in particular, it is still difficult for sotware to explicitly require some higher level -security settings). In the case of the binary installers for Windows and -Mac OS X that are published on python.org, the version of OpenSSL used is -entirely within the control of the Python core development team, but is -currently limited to OpenSSL maintenance releases for the version initially -shipped with the corresponding Python feature release. +security settings). The standard library support can be bypassed by using a +third party library like PyOpenSSL or Pycurl, but this still results in a +security problem, as these can be difficult dependencies to deploy, and many +users will remain unaware that they might want them. Rather than explaining +to potentially naive users how to obtain and use these libraries, it seems +better to just fix the included batteries. -With increased popularity comes increased responsibility, and this policy +In the case of the binary installers for Windows and Mac OS X that are +published on python.org, the version of OpenSSL used is entirely within +the control of the Python core development team, but is currently limited +to OpenSSL maintenance releases for the version initially shipped with the +corresponding Python feature release. + +With increased popularity comes increased responsibility, and this proposal aims to acknowledge the fact that Python's popularity and adoption is at a sufficiently high level that some of our design and policy decisions have significant implications beyond the Python development community. As one example, the Python 2 ``ssl`` module does not support the Server -Name Identification standard. While it is possible to obtain SNI support +Name Indication standard. While it is possible to obtain SNI support by using the third party ``requests`` client library, actually doing so currently requires using not only ``requests`` and its embedded dependencies, but also half a dozen or more additional libraries. The lack of support @@ -315,19 +293,6 @@ it *does* make the barrier to resolution much lower once the problem is pointed out. -My position on the ongoing transition from Python 2 to Python 3 has long -been that Python 2 remains a supported platform for the core development -team, and that commercial support will remain available well after upstream -maintenance ends. However, in the absence of this network security -enhancement policy, that position is difficult to justify when it comes to -software that operates over the public internet. Just as many developers -consider it too difficult to develop truly secure modern networked software -in C/C++ (largely due to the challenges associated with manual -memory management), I anticipate that in the not too distant future, it -will be considered too difficult to develop truly secure modern networked -software using the Python 2 series (some developers would argue that we -have already reached that point). - Python 2.7 represents the only long term maintenance release the core development team has provided, and it is natural that there will be things that worked over a historically shorter maintenance lifespan that don't work @@ -337,22 +302,23 @@ the ability to add new features, even while retaining backwards compatibility for existing interfaces. -It is worth comparing the approach described in this PEP with Red Hat's -handling of its long term support commitments: it isn't the RHEL 6.0 release -itself that receives 10 years worth of support, but the overall RHEL 6 -*series*. The individual RHEL 6.x point releases within the series then -receive a wide variety of new features, including security enhancements, -all while meeting strict backwards compatibility guarantees for existing -software. The policy described in this PEP brings our approach to long term -maintenance more into line with this precedent - we retain our strict -backwards compatibility requirements, but slightly relax the restrictions -against adding new features. +For those familiar with it, it is worth comparing the approach described in +this PEP with Red Hat's handling of its long term open source support +commitments: it isn't the RHEL 6.0 release itself that receives 10 years +worth of support, but the overall RHEL 6 *series*. The individual RHEL 6.x +point releases within the series then receive a wide variety of new +features, including security enhancements, all while meeting strict +backwards compatibility guarantees for existing software. The proposal +covered in this PEP brings our approach to long term maintenance more into +line with this precedent - we retain our strict backwards compatibility +requirements, but make an exception to the restriction against adding new +features. To date, downstream redistributors have respected our upstream policy of "no new features in Python maintenance releases". This PEP explicitly accepts that a more nuanced policy is appropriate in the case of network -security related features, and the specific one it describes is deliberately -designed such that it at least has some chance of being applied to Red Hat +security related features, and the specific change it describes is +deliberately designed such that it is potentially suitable for Red Hat Enterprise Linux and its downstream derivatives. @@ -515,19 +481,19 @@ branch that preserved the exact feature set of the Python 2.7.6 network security infrastructure. -It is the opinion of the PEP author that anyone that actually wants this is -almost certainly making a mistake, and if they insist they really do want -it in their specific situation, they're welcome to either make it themselves -or arrange for a downstream redistributor to make it for them. +In my opinion, anyone that actually wants this is almost certainly making a +mistake, and if they insist they really do want it in their specific +situation, they're welcome to either make it themselves or arrange for a +downstream redistributor to make it for them. If they are made publicly available, any such rebuilds should be referred to as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official Python 2.7 releases that include more up to date network security infrastructure. -After the first Python 2.7 maintenance release that has the security -infrastructure updated to match Python 3.4, it would also be appropriate to -refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL". +After the first Python 2.7 maintenance release that implements this PEP, it +would also be appropriate to refer to Python 2.7.6 and earlier releases as +"Python 2.7 with Legacy SSL". Rejected variant: synchronise particular modules entirely with Python 3 @@ -540,26 +506,18 @@ and has thus been replaced by the current more explicit proposal. -Open Questions -============== +Rejected variant: open ended backport policy +-------------------------------------------- -* MvL has indicated he is not prepared to tackle the task of trying to - integrate a newer OpenSSL into the also aging Python 2.7 build - infrastructure on Windows (unfortunately, we've looked into upgrading - that build infrastructure, and the backwards compatibility issues - appear to be effectively insurmountable). We would require a commitment - from another trusted contributor to handle at least this task, and - potentially also taking over the task of creating the official - Python 2.7 Windows installers for the remaining Python 2.7 maintenance - releases. +Earlier versions of this PEP suggested a general policy change related to +future Python 3 enhancements that impact the general security of the +internet. -* We would need commitments to create and review full backports of the - components covered by this policy from Python 3.4 to Python 2.7, as well - as support for handling any more specific security issues affecting these - modules. - -* Did I miss anything important in the switch to a more restrictive - proposal? +That approach created unnecessary uncertainty, so it has been simplified to +propose backport a specific concrete set of changes. Future feature +backport proposals can refer back to this PEP as precedent, but it will +still be necessary to make a specific case for each feature addition to +the Python 2.7 long term support release. Disclosure of Interest @@ -591,7 +549,8 @@ draft of this proposal. Thanks also to participants in the python-dev mailing list threads -[1,2,5,6]_ +[1,2,5,6]_, as well as the various folks I discussed this issue with at +PyCon 2014 in Montreal. References -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Thu Apr 17 01:15:23 2014 From: python-checkins at python.org (thomas.wouters) Date: Thu, 17 Apr 2014 01:15:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_Tools/scripts/generate?= =?utf-8?q?=5Fopcode=5Fh=2Epy_from_issue_=2317861_to_work_correctly?= Message-ID: <3g8KDC128Xz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/2b187c9e3e92 changeset: 90370:2b187c9e3e92 user: Thomas Wouters date: Thu Apr 17 01:13:29 2014 +0200 summary: Fix Tools/scripts/generate_opcode_h.py from issue #17861 to work correctly when building in a separate object tree. More people should build this way. This may still fail if the source is unwritable, I haven't tested that yet. files: Makefile.pre.in | 8 +++--- Tools/scripts/generate_opcode_h.py | 19 ++++++++--------- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -324,10 +324,10 @@ ########################################################################## # opcode.h generation -OPCODE_H_DIR= Include -OPCODE_H_SCRIPT= Tools/scripts/generate_opcode_h.py -OPCODE_H= $(srcdir)/$(OPCODE_H_DIR)/opcode.h -OPCODE_H_GEN= @OPCODEHGEN@ $(OPCODE_H_SCRIPT) Lib/ $(OPCODE_H) +OPCODE_H_DIR= $(srcdir)/Include +OPCODE_H_SCRIPT= $(srcdir)/Tools/scripts/generate_opcode_h.py +OPCODE_H= $(OPCODE_H_DIR)/opcode.h +OPCODE_H_GEN= @OPCODEHGEN@ $(OPCODE_H_SCRIPT) $(srcdir)/Lib/opcode.py $(OPCODE_H) # ########################################################################## # AST diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py --- a/Tools/scripts/generate_opcode_h.py +++ b/Tools/scripts/generate_opcode_h.py @@ -1,10 +1,6 @@ # This script generates the opcode.h header file. import sys -if len(sys.argv) > 0: - sys.path.insert(0, sys.argv[1]) -# Importing module from our given src directory. -import opcode header = """/* Auto-generated by Tools/scripts/generate_opcode_h.py */ #ifndef Py_OPCODE_H #define Py_OPCODE_H @@ -37,17 +33,20 @@ """ -def main(outfile='Include/opcode.h'): +def main(opcode_py, outfile='Include/opcode.h'): + opcode = {} + exec(open(opcode_py).read(), opcode) + opmap = opcode['opmap'] with open(outfile, 'w') as fobj: fobj.write(header) - for name in opcode.opname: - if name in opcode.opmap: - fobj.write("#define %-20s\t%-3s\n" % (name, opcode.opmap[name])) + for name in opcode['opname']: + if name in opmap: + fobj.write("#define %-20s\t%-3s\n" % (name, opmap[name])) if name == 'POP_EXCEPT': # Special entry for HAVE_ARGUMENT fobj.write("#define %-20s\t%-3d\n" % - ('HAVE_ARGUMENT', opcode.HAVE_ARGUMENT)) + ('HAVE_ARGUMENT', opcode['HAVE_ARGUMENT'])) fobj.write(footer) if __name__ == '__main__': - main(sys.argv[2]) + main(sys.argv[1], sys.argv[2]) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 03:58:22 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 17 Apr 2014 03:58:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE4NjI4OiBjbGFy?= =?utf-8?q?ify_index_entry_for_source_file_encoding_declaration=2E?= Message-ID: <3g8NrG2Kwhz7LjS@mail.python.org> http://hg.python.org/cpython/rev/0413e0b1f76d changeset: 90371:0413e0b1f76d branch: 3.4 parent: 90367:30a7e37b8441 user: R David Murray date: Wed Apr 16 21:48:04 2014 -0400 summary: #18628: clarify index entry for source file encoding declaration. Patch by Sam Lucidi. files: Doc/reference/lexical_analysis.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -76,7 +76,7 @@ Encoding declarations --------------------- -.. index:: source character set, encodings +.. index:: source character set, encoding declarations (source file) If a comment in the first or second line of the Python script matches the regular expression ``coding[=:]\s*([-\w.]+)``, this comment is processed as an -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 03:58:23 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 17 Apr 2014 03:58:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2318628=3A_clarify_index_entry_for_source_file?= =?utf-8?q?_encoding_declaration=2E?= Message-ID: <3g8NrH3zjbz7LjS@mail.python.org> http://hg.python.org/cpython/rev/7c2dcb18146c changeset: 90372:7c2dcb18146c parent: 90370:2b187c9e3e92 parent: 90371:0413e0b1f76d user: R David Murray date: Wed Apr 16 21:50:25 2014 -0400 summary: Merge: #18628: clarify index entry for source file encoding declaration. files: Doc/reference/lexical_analysis.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -76,7 +76,7 @@ Encoding declarations --------------------- -.. index:: source character set, encodings +.. index:: source character set, encoding declarations (source file) If a comment in the first or second line of the Python script matches the regular expression ``coding[=:]\s*([-\w.]+)``, this comment is processed as an -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 03:58:24 2014 From: python-checkins at python.org (r.david.murray) Date: Thu, 17 Apr 2014 03:58:24 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4NjI4OiBjbGFy?= =?utf-8?q?ify_index_entry_for_source_file_encoding_declaration=2E?= Message-ID: <3g8NrJ5j4Sz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/2a793df32be5 changeset: 90373:2a793df32be5 branch: 2.7 parent: 90354:72b1715e18c1 user: R David Murray date: Wed Apr 16 21:57:38 2014 -0400 summary: #18628: clarify index entry for source file encoding declaration. files: Doc/reference/lexical_analysis.rst | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -104,9 +104,7 @@ Encoding declarations --------------------- -.. index:: - single: source character set - single: encodings +.. index:: source character set, encoding declarations (source file) If a comment in the first or second line of the Python script matches the regular expression ``coding[=:]\s*([-\w.]+)``, this comment is processed as an -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 05:33:40 2014 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 17 Apr 2014 05:33:40 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Correct_the_UR?= =?utf-8?q?L_in_the_http=2Eclient_example=2E_Noted_by_Evens_Fortun=C3=A9?= =?utf-8?q?=2E_Closes?= Message-ID: <3g8QyD57Kfz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/57c66f85942d changeset: 90374:57c66f85942d branch: 3.4 parent: 90371:0413e0b1f76d user: Senthil Kumaran date: Wed Apr 16 23:33:02 2014 -0400 summary: Correct the URL in the http.client example. Noted by Evens Fortun?. Closes #21229 files: Doc/library/http.client.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -660,7 +660,7 @@ >>> # This creates an HTTP message >>> # with the content of BODY as the enclosed representation - >>> # for the resource http://localhost:8080/foobar + >>> # for the resource http://localhost:8080/file ... >>> import http.client >>> BODY = "***filecontents***" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 05:33:41 2014 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 17 Apr 2014 05:33:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g8QyF6SLTz7Lkc@mail.python.org> http://hg.python.org/cpython/rev/c99b4341330c changeset: 90375:c99b4341330c parent: 90372:7c2dcb18146c parent: 90374:57c66f85942d user: Senthil Kumaran date: Wed Apr 16 23:33:30 2014 -0400 summary: merge 3.4 files: Doc/library/http.client.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -660,7 +660,7 @@ >>> # This creates an HTTP message >>> # with the content of BODY as the enclosed representation - >>> # for the resource http://localhost:8080/foobar + >>> # for the resource http://localhost:8080/file ... >>> import http.client >>> BODY = "***filecontents***" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 05:44:12 2014 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 17 Apr 2014 05:44:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Clarify_BROWSE?= =?utf-8?q?R_envar_behavior_in_webbrowser=2Epy=2E_Noted_by_David_Turner=2E?= =?utf-8?q?_Closes?= Message-ID: <3g8RBN5ZgVz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/503bf9dee28e changeset: 90376:503bf9dee28e branch: 3.4 parent: 90374:57c66f85942d user: Senthil Kumaran date: Wed Apr 16 23:43:34 2014 -0400 summary: Clarify BROWSER envar behavior in webbrowser.py. Noted by David Turner. Closes #21248 files: Doc/library/webbrowser.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst --- a/Doc/library/webbrowser.rst +++ b/Doc/library/webbrowser.rst @@ -19,12 +19,12 @@ available. If text-mode browsers are used, the calling process will block until the user exits the browser. -If the environment variable :envvar:`BROWSER` exists, it is interpreted to -override the platform default list of browsers, as a :data:`os.pathsep`-separated -list of browsers to try in order. When the value of a list part contains the -string ``%s``, then it is interpreted as a literal browser command line to be -used with the argument URL substituted for ``%s``; if the part does not contain -``%s``, it is simply interpreted as the name of the browser to launch. [1]_ +If the environment variable :envvar:`BROWSER` exists, it is interpreted as the +:data:`os.pathsep`-separated list of browsers to try ahead of the the platform +defaults. When the value of a list part contains the string ``%s``, then it is +interpreted as a literal browser command line to be used with the argument URL +substituted for ``%s``; if the part does not contain ``%s``, it is simply +interpreted as the name of the browser to launch. [1]_ For non-Unix platforms, or when a remote browser is available on Unix, the controlling process will not wait for the user to finish with the browser, but -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 05:44:13 2014 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 17 Apr 2014 05:44:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g8RBP6wNkz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/3441b4edb779 changeset: 90377:3441b4edb779 parent: 90375:c99b4341330c parent: 90376:503bf9dee28e user: Senthil Kumaran date: Wed Apr 16 23:44:03 2014 -0400 summary: merge 3.4 files: Doc/library/webbrowser.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/webbrowser.rst b/Doc/library/webbrowser.rst --- a/Doc/library/webbrowser.rst +++ b/Doc/library/webbrowser.rst @@ -19,12 +19,12 @@ available. If text-mode browsers are used, the calling process will block until the user exits the browser. -If the environment variable :envvar:`BROWSER` exists, it is interpreted to -override the platform default list of browsers, as a :data:`os.pathsep`-separated -list of browsers to try in order. When the value of a list part contains the -string ``%s``, then it is interpreted as a literal browser command line to be -used with the argument URL substituted for ``%s``; if the part does not contain -``%s``, it is simply interpreted as the name of the browser to launch. [1]_ +If the environment variable :envvar:`BROWSER` exists, it is interpreted as the +:data:`os.pathsep`-separated list of browsers to try ahead of the the platform +defaults. When the value of a list part contains the string ``%s``, then it is +interpreted as a literal browser command line to be used with the argument URL +substituted for ``%s``; if the part does not contain ``%s``, it is simply +interpreted as the name of the browser to launch. [1]_ For non-Unix platforms, or when a remote browser is available on Unix, the controlling process will not wait for the user to finish with the browser, but -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 06:00:55 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 17 Apr 2014 06:00:55 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_support_setting_fpu_precis?= =?utf-8?q?ion_on_m68k_=28closes_=2320904=29?= Message-ID: <3g8RYg35DVz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/c2f6551c9eaf changeset: 90378:c2f6551c9eaf user: Benjamin Peterson date: Thu Apr 17 00:00:31 2014 -0400 summary: support setting fpu precision on m68k (closes #20904) Patch from Andreas Schwab. files: Include/pyport.h | 19 +++++++++++++++++++ Misc/NEWS | 2 ++ configure | 32 ++++++++++++++++++++++++++++++++ configure.ac | 13 +++++++++++++ pyconfig.h.in | 3 +++ 5 files changed, 69 insertions(+), 0 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -588,6 +588,25 @@ } while (0) #endif +#ifdef HAVE_GCC_ASM_FOR_MC68881 +#define HAVE_PY_SET_53BIT_PRECISION 1 +#define _Py_SET_53BIT_PRECISION_HEADER \ + unsigned int old_fpcr, new_fpcr +#define _Py_SET_53BIT_PRECISION_START \ + do { \ + __asm__ ("fmove.l %%fpcr,%0" : "=g" (old_fpcr)); \ + /* Set double precision / round to nearest. */ \ + new_fpcr = (old_fpcr & ~0xf0) | 0x80; \ + if (new_fpcr != old_fpcr) \ + __asm__ volatile ("fmove.l %0,%%fpcr" : : "g" (new_fpcr)); \ + } while (0) +#define _Py_SET_53BIT_PRECISION_END \ + do { \ + if (new_fpcr != old_fpcr) \ + __asm__ volatile ("fmove.l %0,%%fpcr" : : "g" (old_fpcr)); \ + } while (0) +#endif + /* default definitions are empty */ #ifndef HAVE_PY_SET_53BIT_PRECISION #define _Py_SET_53BIT_PRECISION_HEADER diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #20904: Support setting FPU precision on m68k. + - Issue #21209: Fix sending tuples to custom generator objects with the yield from syntax. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -13244,6 +13244,38 @@ fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we can use gcc inline assembler to get and set mc68881 fpcr" >&5 +$as_echo_n "checking whether we can use gcc inline assembler to get and set mc68881 fpcr... " >&6; } +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + unsigned int fpcr; + __asm__ __volatile__ ("fmove.l %%fpcr,%0" : "=g" (fpcr)); + __asm__ __volatile__ ("fmove.l %0,%%fpcr" : : "g" (fpcr)); + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + have_gcc_asm_for_mc68881=yes +else + have_gcc_asm_for_mc68881=no +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $have_gcc_asm_for_mc68881" >&5 +$as_echo "$have_gcc_asm_for_mc68881" >&6; } +if test "$have_gcc_asm_for_mc68881" = yes +then + +$as_echo "#define HAVE_GCC_ASM_FOR_MC68881 1" >>confdefs.h + +fi + # Detect whether system arithmetic is subject to x87-style double # rounding issues. The result of this test has little meaning on non # IEEE 754 platforms. On IEEE 754, test should return 1 if rounding diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3810,6 +3810,19 @@ [Define if we can use gcc inline assembler to get and set x87 control word]) fi +AC_MSG_CHECKING(whether we can use gcc inline assembler to get and set mc68881 fpcr) +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[ + unsigned int fpcr; + __asm__ __volatile__ ("fmove.l %%fpcr,%0" : "=g" (fpcr)); + __asm__ __volatile__ ("fmove.l %0,%%fpcr" : : "g" (fpcr)); +]])],[have_gcc_asm_for_mc68881=yes],[have_gcc_asm_for_mc68881=no]) +AC_MSG_RESULT($have_gcc_asm_for_mc68881) +if test "$have_gcc_asm_for_mc68881" = yes +then + AC_DEFINE(HAVE_GCC_ASM_FOR_MC68881, 1, + [Define if we can use gcc inline assembler to get and set mc68881 fpcr]) +fi + # Detect whether system arithmetic is subject to x87-style double # rounding issues. The result of this test has little meaning on non # IEEE 754 platforms. On IEEE 754, test should return 1 if rounding diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -313,6 +313,9 @@ /* Define to 1 if you have the `gamma' function. */ #undef HAVE_GAMMA +/* Define if we can use gcc inline assembler to get and set mc68881 fpcr */ +#undef HAVE_GCC_ASM_FOR_MC68881 + /* Define if we can use x64 gcc inline assembler */ #undef HAVE_GCC_ASM_FOR_X64 -- Repository URL: http://hg.python.org/cpython From bcannon at gmail.com Fri Apr 4 16:43:56 2014 From: bcannon at gmail.com (Brett Cannon) Date: Fri, 04 Apr 2014 14:43:56 +0000 Subject: [Python-checkins] [Python-Dev] cpython: fix #21076: turn signal module constants into enums References: <3g0hvS5h5Wz7LjM@mail.python.org> Message-ID: On Fri Apr 04 2014 at 10:34:06 AM, Victor Stinner wrote: > 2014-04-04 16:21 GMT+02:00 Brett Cannon : > > Fix is in rev c6e63bb132fb. > > Hum, this one was not enough for me. I also modified Modules/ > Setup.config.in: > Wasn't for me either in the end as it failed when I did a distclean. The unfortunately bit me in the middle of trying to get a 3.4->default merge so I just tried to do the best I could. This might take a little while to clean up as the Windows 7 buildbot now builds -- which it has been doing for a while -- but claims it can't find _signal.sigpending which I can at least see on OS X so that bit of code my need tweaking. -Brett > > changeset: 90137:df5120efb86e > tag: tip > user: Victor Stinner > date: Fri Apr 04 16:30:04 2014 +0200 > files: Modules/Setup.config.in > description: > Issue #21076: the C signal module has been renamed to _signal > > > diff -r c6e63bb132fb -r df5120efb86e Modules/Setup.config.in > --- a/Modules/Setup.config.in Fri Apr 04 10:20:28 2014 -0400 > +++ b/Modules/Setup.config.in Fri Apr 04 16:30:04 2014 +0200 > @@ -7,7 +7,7 @@ > @USE_THREAD_MODULE at _thread _threadmodule.c > > # The signal module > - at USE_SIGNAL_MODULE@signal signalmodule.c > + at USE_SIGNAL_MODULE@_signal signalmodule.c > > # The rest of the modules previously listed in this file are built > # by the setup.py script in Python 2.1 and later. > > Victor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Fri Apr 4 17:08:37 2014 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Fri, 4 Apr 2014 17:08:37 +0200 Subject: [Python-checkins] [Python-Dev] cpython: fix #21076: turn signal module constants into enums In-Reply-To: References: <3g0hvS5h5Wz7LjM@mail.python.org> Message-ID: Sorry for the troubles. :( I committed this because it worked on my local copy of Python 3.5 but after I tried a brand new "hg clone" it didnt. On Fri, Apr 4, 2014 at 4:43 PM, Brett Cannon wrote: > > > On Fri Apr 04 2014 at 10:34:06 AM, Victor Stinner < > victor.stinner at gmail.com> wrote: > >> 2014-04-04 16:21 GMT+02:00 Brett Cannon : >> > Fix is in rev c6e63bb132fb. >> >> Hum, this one was not enough for me. I also modified Modules/ >> Setup.config.in: >> > > Wasn't for me either in the end as it failed when I did a distclean. The > unfortunately bit me in the middle of trying to get a 3.4->default merge so > I just tried to do the best I could. This might take a little while to > clean up as the Windows 7 buildbot now builds -- which it has been doing > for a while -- but claims it can't find _signal.sigpending which I can at > least see on OS X so that bit of code my need tweaking. > > -Brett > > >> >> changeset: 90137:df5120efb86e >> tag: tip >> user: Victor Stinner >> date: Fri Apr 04 16:30:04 2014 +0200 >> files: Modules/Setup.config.in >> description: >> Issue #21076: the C signal module has been renamed to _signal >> >> >> diff -r c6e63bb132fb -r df5120efb86e Modules/Setup.config.in >> --- a/Modules/Setup.config.in Fri Apr 04 10:20:28 2014 -0400 >> +++ b/Modules/Setup.config.in Fri Apr 04 16:30:04 2014 +0200 >> @@ -7,7 +7,7 @@ >> @USE_THREAD_MODULE at _thread _threadmodule.c >> >> # The signal module >> - at USE_SIGNAL_MODULE@signal signalmodule.c >> + at USE_SIGNAL_MODULE@_signal signalmodule.c >> >> # The rest of the modules previously listed in this file are built >> # by the setup.py script in Python 2.1 and later. >> >> Victor >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > > -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Apr 17 09:59:59 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 17 Apr 2014 09:59:59 +0200 Subject: [Python-checkins] Daily reference leaks (2b187c9e3e92): sum=7 Message-ID: results for 2b187c9e3e92 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 0, 2] references, sum=2 test_site leaked [0, 0, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogRAnzvj', '-x'] From python-checkins at python.org Thu Apr 17 12:25:49 2014 From: python-checkins at python.org (antoine.pitrou) Date: Thu, 17 Apr 2014 12:25:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_ssl=2Ecreate=5Fdefault?= =?utf-8?q?=5Fcontext_in_Lib/test/ssl=5Fservers=2Epy?= Message-ID: <3g8c5n4CsKz7Ljc@mail.python.org> http://hg.python.org/cpython/rev/42b2a51761b4 changeset: 90379:42b2a51761b4 user: Antoine Pitrou date: Thu Apr 17 12:21:36 2014 +0200 summary: Use ssl.create_default_context in Lib/test/ssl_servers.py files: Lib/test/ssl_servers.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/ssl_servers.py b/Lib/test/ssl_servers.py --- a/Lib/test/ssl_servers.py +++ b/Lib/test/ssl_servers.py @@ -150,7 +150,7 @@ def make_https_server(case, *, context=None, certfile=CERTFILE, host=HOST, handler_class=None): if context is None: - context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) # We assume the certfile contains both private key and certificate context.load_cert_chain(certfile) server = HTTPSServerThread(context, host, handler_class) @@ -192,7 +192,7 @@ else: handler_class = RootedHTTPRequestHandler handler_class.root = os.getcwd() - context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) context.load_cert_chain(CERTFILE) if args.curve_name: context.set_ecdh_curve(args.curve_name) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 12:30:20 2014 From: python-checkins at python.org (antoine.pitrou) Date: Thu, 17 Apr 2014 12:30:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_--ciphers_option_to_?= =?utf-8?q?Lib/test/ssl=5Fservers=2Epy?= Message-ID: <3g8cC05zYZz7Ljc@mail.python.org> http://hg.python.org/cpython/rev/517de1983677 changeset: 90380:517de1983677 user: Antoine Pitrou date: Thu Apr 17 12:30:14 2014 +0200 summary: Add a --ciphers option to Lib/test/ssl_servers.py files: Lib/test/ssl_servers.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/test/ssl_servers.py b/Lib/test/ssl_servers.py --- a/Lib/test/ssl_servers.py +++ b/Lib/test/ssl_servers.py @@ -182,6 +182,8 @@ parser.add_argument('--curve-name', dest='curve_name', type=str, action='store', help='curve name for EC-based Diffie-Hellman') + parser.add_argument('--ciphers', dest='ciphers', type=str, + help='allowed cipher list') parser.add_argument('--dh', dest='dh_file', type=str, action='store', help='PEM file containing DH parameters') args = parser.parse_args() @@ -198,6 +200,8 @@ context.set_ecdh_curve(args.curve_name) if args.dh_file: context.load_dh_params(args.dh_file) + if args.ciphers: + context.set_ciphers(args.ciphers) server = HTTPSServer(("", args.port), handler_class, context) if args.verbose: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 14:57:53 2014 From: python-checkins at python.org (eric.smith) Date: Thu, 17 Apr 2014 14:57:53 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Fix_typo=2E?= Message-ID: <3g8gTF5shLz7Ljc@mail.python.org> http://hg.python.org/peps/rev/e1d2784451ac changeset: 5461:e1d2784451ac user: Eric V. Smith date: Thu Apr 17 08:57:48 2014 -0400 summary: Fix typo. files: pep-0466.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0466.txt b/pep-0466.txt --- a/pep-0466.txt +++ b/pep-0466.txt @@ -77,7 +77,7 @@ This PEP does NOT propose a general exception for backporting new features to Python 2.7 - every new feature proposed for backporting will still need to be justified independently. In particular, it will need to be explained -why relying on and independently updated backport on the Python Package Index +why relying on an independently updated backport on the Python Package Index instead is not an acceptable solution. -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Thu Apr 17 17:55:49 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 17:55:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogLSBJc3N1ZSAjMTUy?= =?utf-8?q?34=3A_For_BerkelyDB_and_Sqlite=2C_only_add_the_found_library_an?= =?utf-8?q?d?= Message-ID: <3g8lQY33mjz7Lk2@mail.python.org> http://hg.python.org/cpython/rev/1a00e04a233d changeset: 90381:1a00e04a233d branch: 3.4 parent: 90376:503bf9dee28e user: doko at ubuntu.com date: Thu Apr 17 17:52:48 2014 +0200 summary: - Issue #15234: For BerkelyDB and Sqlite, only add the found library and include directories if they aren't already being searched. This avoids an explicit runtime library dependency. files: Misc/NEWS | 4 ++++ setup.py | 15 +++++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -167,6 +167,10 @@ Build ----- +- Issue #15234: For BerkelyDB and Sqlite, only add the found library and + include directories if they aren't already being searched. This avoids + an explicit runtime library dependency. + - Issue #20644: OS X installer build support for documentation build changes in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1022,8 +1022,16 @@ if db_setup_debug: print("bsddb using BerkeleyDB lib:", db_ver, dblib) print("bsddb lib dir:", dblib_dir, " inc dir:", db_incdir) - db_incs = [db_incdir] dblibs = [dblib] + # Only add the found library and include directories if they aren't + # already being searched. This avoids an explicit runtime library + # dependency. + if db_incdir in inc_dirs: + db_incs = None + else: + db_incs = [db_incdir] + if dblib_dir[0] in lib_dirs: + dblib_dir = None else: if db_setup_debug: print("db: no appropriate library found") db_incs = None @@ -1134,6 +1142,9 @@ # can end up with a bad search path order. if sqlite_incdir not in self.compiler.include_dirs: include_dirs.append(sqlite_incdir) + # avoid a runtime library path for a system library dir + if sqlite_libdir and sqlite_libdir[0] in lib_dirs: + sqlite_libdir = None exts.append(Extension('_sqlite3', sqlite_srcs, define_macros=sqlite_defines, include_dirs=include_dirs, @@ -1202,7 +1213,7 @@ libraries = gdbm_libs) break elif cand == "bdb": - if db_incs is not None: + if dblibs: if dbm_setup_debug: print("building dbm using bdb") dbmext = Extension('_dbm', ['_dbmmodule.c'], library_dirs=dblib_dir, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 17:55:50 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 17:55:50 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy40?= Message-ID: <3g8lQZ4rQQz7Lkh@mail.python.org> http://hg.python.org/cpython/rev/1d1aefd00f07 changeset: 90382:1d1aefd00f07 parent: 90380:517de1983677 parent: 90381:1a00e04a233d user: doko at ubuntu.com date: Thu Apr 17 17:55:03 2014 +0200 summary: Merge 3.4 files: Misc/NEWS | 6 ++++++ setup.py | 15 +++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -244,8 +244,14 @@ Build ----- +<<<<<<< local - Issue #17861: Tools/scripts/generate_opcode_h.py automatically regenerates Include/opcode.h from Lib/opcode.py if the later gets any change. +======= +- Issue #15234: For BerkelyDB and Sqlite, only add the found library and + include directories if they aren't already being searched. This avoids + an explicit runtime library dependency. +>>>>>>> other - Issue #20644: OS X installer build support for documentation build changes in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1022,8 +1022,16 @@ if db_setup_debug: print("bsddb using BerkeleyDB lib:", db_ver, dblib) print("bsddb lib dir:", dblib_dir, " inc dir:", db_incdir) - db_incs = [db_incdir] dblibs = [dblib] + # Only add the found library and include directories if they aren't + # already being searched. This avoids an explicit runtime library + # dependency. + if db_incdir in inc_dirs: + db_incs = None + else: + db_incs = [db_incdir] + if dblib_dir[0] in lib_dirs: + dblib_dir = None else: if db_setup_debug: print("db: no appropriate library found") db_incs = None @@ -1134,6 +1142,9 @@ # can end up with a bad search path order. if sqlite_incdir not in self.compiler.include_dirs: include_dirs.append(sqlite_incdir) + # avoid a runtime library path for a system library dir + if sqlite_libdir and sqlite_libdir[0] in lib_dirs: + sqlite_libdir = None exts.append(Extension('_sqlite3', sqlite_srcs, define_macros=sqlite_defines, include_dirs=include_dirs, @@ -1202,7 +1213,7 @@ libraries = gdbm_libs) break elif cand == "bdb": - if db_incs is not None: + if dblibs: if dbm_setup_debug: print("building dbm using bdb") dbmext = Extension('_dbm', ['_dbmmodule.c'], library_dirs=dblib_dir, -- Repository URL: http://hg.python.org/cpython From benjamin at python.org Thu Apr 17 18:42:31 2014 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 17 Apr 2014 09:42:31 -0700 Subject: [Python-checkins] cpython (merge 3.4 -> default): Merge 3.4 In-Reply-To: <3g8lQZ4rQQz7Lkh@mail.python.org> References: <3g8lQZ4rQQz7Lkh@mail.python.org> Message-ID: <1397752951.29059.107630649.01472991@webmail.messagingengine.com> On Thu, Apr 17, 2014, at 8:55, matthias.klose wrote: > http://hg.python.org/cpython/rev/1d1aefd00f07 > changeset: 90382:1d1aefd00f07 > parent: 90380:517de1983677 > parent: 90381:1a00e04a233d > user: doko at ubuntu.com > date: Thu Apr 17 17:55:03 2014 +0200 > summary: > Merge 3.4 > > files: > Misc/NEWS | 6 ++++++ > setup.py | 15 +++++++++++++-- > 2 files changed, 19 insertions(+), 2 deletions(-) > > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -244,8 +244,14 @@ > Build > ----- > > +<<<<<<< local > - Issue #17861: Tools/scripts/generate_opcode_h.py automatically > regenerates > Include/opcode.h from Lib/opcode.py if the later gets any change. > +======= > +- Issue #15234: For BerkelyDB and Sqlite, only add the found library and > + include directories if they aren't already being searched. This avoids > + an explicit runtime library dependency. > +>>>>>>> other You might want to actually resolve this conflict. From python-checkins at python.org Thu Apr 17 18:47:01 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 18:47:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_-_fix_merge_conflict?= Message-ID: <3g8mYd2Y2jz7Lkv@mail.python.org> http://hg.python.org/cpython/rev/4e4eb69a1724 changeset: 90383:4e4eb69a1724 user: doko at ubuntu.com date: Thu Apr 17 18:46:53 2014 +0200 summary: - fix merge conflict files: Misc/NEWS | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -244,14 +244,12 @@ Build ----- -<<<<<<< local -- Issue #17861: Tools/scripts/generate_opcode_h.py automatically regenerates - Include/opcode.h from Lib/opcode.py if the later gets any change. -======= - Issue #15234: For BerkelyDB and Sqlite, only add the found library and include directories if they aren't already being searched. This avoids an explicit runtime library dependency. ->>>>>>> other + +- Issue #17861: Tools/scripts/generate_opcode_h.py automatically regenerates + Include/opcode.h from Lib/opcode.py if the later gets any change. - Issue #20644: OS X installer build support for documentation build changes in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 19:49:11 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 19:49:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fixes_for_KFre?= =?utf-8?q?eBSD_and_the_Hurd=3A?= Message-ID: <3g8nxM5Yxnz7LlQ@mail.python.org> http://hg.python.org/cpython/rev/ca2edbefca35 changeset: 90384:ca2edbefca35 branch: 3.4 parent: 90381:1a00e04a233d user: doko at ubuntu.com date: Thu Apr 17 19:47:16 2014 +0200 summary: Fixes for KFreeBSD and the Hurd: - Issue #21274: Define PATH_MAX for GNU/Hurd in Python/pythonrun.c. - Issue #21276: posixmodule: Don't define USE_XATTRS on KFreeBSD and the Hurd. - Issue #21275: Fix a socket test on KFreeBSD. files: Lib/test/test_socket.py | 2 +- Misc/NEWS | 9 +++++++++ Modules/posixmodule.c | 2 +- Python/pythonrun.c | 4 ++++ 4 files changed, 15 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -867,7 +867,7 @@ # Find one service that exists, then check all the related interfaces. # I've ordered this by protocols that have both a tcp and udp # protocol, at least for modern Linuxes. - if (sys.platform.startswith(('freebsd', 'netbsd')) + if (sys.platform.startswith(('freebsd', 'netbsd', 'gnukfreebsd')) or sys.platform in ('linux', 'darwin')): # avoid the 'echo' service on this platform, as there is an # assumption breaking non-standard port/protocol entry diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #21274: Define PATH_MAX for GNU/Hurd in Python/pythonrun.c. + - Issue #21209: Fix sending tuples to custom generator objects with the yield from syntax. @@ -158,6 +160,11 @@ - Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug in yield-from implementation in CPythons prior to 3.4.1. +Extension Modules +----------------- + +- Issue #21276: posixmodule: Don't define USE_XATTRS on KFreeBSD and the Hurd. + IDLE ---- @@ -200,6 +207,8 @@ Tests ----- +- Issue #21275: Fix a socket test on KFreeBSD. + - Issue #21223: Pass test_site/test_startup_imports when some of the extensions are built as builtins. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -92,7 +92,7 @@ #undef HAVE_SCHED_SETAFFINITY #endif -#if defined(HAVE_SYS_XATTR_H) && defined(__GLIBC__) +#if defined(HAVE_SYS_XATTR_H) && defined(__GLIBC__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) #define USE_XATTRS #endif diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -35,6 +35,10 @@ #define PATH_MAX MAXPATHLEN #endif +#ifdef __gnu_hurd__ +#define PATH_MAX MAXPATHLEN +#endif + _Py_IDENTIFIER(builtins); _Py_IDENTIFIER(excepthook); _Py_IDENTIFIER(flush); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 19:49:13 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 19:49:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_-_Merge_3=2E4?= Message-ID: <3g8nxP0Y30z7Ljp@mail.python.org> http://hg.python.org/cpython/rev/bdf002b9c20c changeset: 90385:bdf002b9c20c parent: 90383:4e4eb69a1724 parent: 90384:ca2edbefca35 user: doko at ubuntu.com date: Thu Apr 17 19:49:00 2014 +0200 summary: - Merge 3.4 files: Lib/test/test_socket.py | 2 +- Misc/NEWS | 9 +++++++++ Modules/posixmodule.c | 2 +- Python/pythonrun.c | 4 ++++ 4 files changed, 15 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -867,7 +867,7 @@ # Find one service that exists, then check all the related interfaces. # I've ordered this by protocols that have both a tcp and udp # protocol, at least for modern Linuxes. - if (sys.platform.startswith(('freebsd', 'netbsd')) + if (sys.platform.startswith(('freebsd', 'netbsd', 'gnukfreebsd')) or sys.platform in ('linux', 'darwin')): # avoid the 'echo' service on this platform, as there is an # assumption breaking non-standard port/protocol entry diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #21274: Define PATH_MAX for GNU/Hurd in Python/pythonrun.c. + - Issue #20904: Support setting FPU precision on m68k. - Issue #21209: Fix sending tuples to custom generator objects with the yield @@ -235,6 +237,11 @@ - Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug in yield-from implementation in CPythons prior to 3.4.1. +Extension Modules +----------------- + +- Issue #21276: posixmodule: Don't define USE_XATTRS on KFreeBSD and the Hurd. + IDLE ---- @@ -280,6 +287,8 @@ Tests ----- +- Issue #21275: Fix a socket test on KFreeBSD. + - Issue #21223: Pass test_site/test_startup_imports when some of the extensions are built as builtins. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -92,7 +92,7 @@ #undef HAVE_SCHED_SETAFFINITY #endif -#if defined(HAVE_SYS_XATTR_H) && defined(__GLIBC__) +#if defined(HAVE_SYS_XATTR_H) && defined(__GLIBC__) && !defined(__FreeBSD_kernel__) && !defined(__GNU__) #define USE_XATTRS #endif diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -35,6 +35,10 @@ #define PATH_MAX MAXPATHLEN #endif +#ifdef __gnu_hurd__ +#define PATH_MAX MAXPATHLEN +#endif + _Py_IDENTIFIER(builtins); _Py_IDENTIFIER(excepthook); _Py_IDENTIFIER(flush); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 20:14:42 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 20:14:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogLSBJc3N1ZSAjMjEy?= =?utf-8?q?85=3A_Refactor_and_fix_curses_configure_check_to_always_search?= Message-ID: <3g8pVp2Fppz7LkG@mail.python.org> http://hg.python.org/cpython/rev/1bc0a8310b9f changeset: 90386:1bc0a8310b9f branch: 2.7 parent: 90373:2a793df32be5 user: doko at ubuntu.com date: Thu Apr 17 20:11:19 2014 +0200 summary: - Issue #21285: Refactor and fix curses configure check to always search in a ncursesw directory. files: Misc/NEWS | 6 +++ configure | 62 ++++++++++++++++++++++++++------------- configure.ac | 26 ++++++++++----- setup.py | 5 +++ 4 files changed, 69 insertions(+), 30 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -367,6 +367,12 @@ - Issue #21093: Prevent failures of ctypes test_macholib on OS X if a copy of libz exists in $HOME/lib or /usr/local/lib. +Build +----- + +- Issue #21285: Refactor and fix curses configure check to always search + in a ncursesw directory. + Documentation ------------- diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6651,9 +6651,9 @@ fi -for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +for ac_header in asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ -ieeefp.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ +ieeefp.h io.h langinfo.h libintl.h poll.h process.h pthread.h \ shadow.h signal.h stdint.h stropts.h termios.h thread.h \ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/epoll.h sys/event.h sys/file.h sys/loadavg.h \ @@ -6884,25 +6884,6 @@ fi -# On Solaris, term.h requires curses.h -for ac_header in term.h -do : - ac_fn_c_check_header_compile "$LINENO" "term.h" "ac_cv_header_term_h" " -#ifdef HAVE_CURSES_H -#include -#endif - -" -if test "x$ac_cv_header_term_h" = xyes; then : - cat >>confdefs.h <<_ACEOF -#define HAVE_TERM_H 1 -_ACEOF - -fi - -done - - # On Linux, netlink.h requires asm/types.h for ac_header in linux/netlink.h do : @@ -14132,6 +14113,43 @@ fi +# first curses configure check +ac_save_cppflags="$CPPFLAGS" +CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" + +for ac_header in curses.h ncurses.h +do : + as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` +ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" +if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : + cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 +_ACEOF + +fi + +done + + +# On Solaris, term.h requires curses.h +for ac_header in term.h +do : + ac_fn_c_check_header_compile "$LINENO" "term.h" "ac_cv_header_term_h" " +#ifdef HAVE_CURSES_H +#include +#endif + +" +if test "x$ac_cv_header_term_h" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_TERM_H 1 +_ACEOF + +fi + +done + + # On HP/UX 11.0, mvwdelch is a block with a return statement { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether mvwdelch is an expression" >&5 $as_echo_n "checking whether mvwdelch is an expression... " >&6; } @@ -14285,6 +14303,8 @@ fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +# last curses configure check +CPPFLAGS=$ac_save_cppflags { $as_echo "$as_me:${as_lineno-$LINENO}: checking for device files" >&5 $as_echo "$as_me: checking for device files" >&6;} diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1508,9 +1508,9 @@ # checks for header files AC_HEADER_STDC -AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +AC_CHECK_HEADERS(asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ -ieeefp.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ +ieeefp.h io.h langinfo.h libintl.h poll.h process.h pthread.h \ shadow.h signal.h stdint.h stropts.h termios.h thread.h \ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/epoll.h sys/event.h sys/file.h sys/loadavg.h \ @@ -1523,13 +1523,6 @@ AC_HEADER_DIRENT AC_HEADER_MAJOR -# On Solaris, term.h requires curses.h -AC_CHECK_HEADERS(term.h,,,[ -#ifdef HAVE_CURSES_H -#include -#endif -]) - # On Linux, netlink.h requires asm/types.h AC_CHECK_HEADERS(linux/netlink.h,,,[ #ifdef HAVE_ASM_TYPES_H @@ -4296,6 +4289,19 @@ [Define if you have struct stat.st_mtimensec]) fi +# first curses configure check +ac_save_cppflags="$CPPFLAGS" +CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" + +AC_CHECK_HEADERS(curses.h ncurses.h) + +# On Solaris, term.h requires curses.h +AC_CHECK_HEADERS(term.h,,,[ +#ifdef HAVE_CURSES_H +#include +#endif +]) + # On HP/UX 11.0, mvwdelch is a block with a return statement AC_MSG_CHECKING(whether mvwdelch is an expression) AC_CACHE_VAL(ac_cv_mvwdelch_is_expression, @@ -4350,6 +4356,8 @@ AC_MSG_RESULT(yes)], [AC_MSG_RESULT(no)] ) +# last curses configure check +CPPFLAGS=$ac_save_cppflags AC_MSG_NOTICE([checking for device files]) diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1339,13 +1339,17 @@ # Curses support, requiring the System V version of curses, often # provided by the ncurses library. panel_library = 'panel' + curses_incs = None if curses_library.startswith('ncurses'): if curses_library == 'ncursesw': # Bug 1464056: If _curses.so links with ncursesw, # _curses_panel.so must link with panelw. panel_library = 'panelw' curses_libs = [curses_library] + curses_incs = find_file('curses.h', inc_dirs, + [os.path.join(d, 'ncursesw') for d in inc_dirs]) exts.append( Extension('_curses', ['_cursesmodule.c'], + include_dirs = curses_incs, libraries = curses_libs) ) elif curses_library == 'curses' and host_platform != 'darwin': # OSX has an old Berkeley curses, not good enough for @@ -1366,6 +1370,7 @@ if (module_enabled(exts, '_curses') and self.compiler.find_library_file(lib_dirs, panel_library)): exts.append( Extension('_curses_panel', ['_curses_panel.c'], + include_dirs = curses_incs, libraries = [panel_library] + curses_libs) ) else: missing.append('_curses_panel') -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 20:14:43 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 20:14:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogLSBJc3N1ZSAjMjEy?= =?utf-8?q?85=3A_Refactor_and_fix_curses_configure_check_to_always_search?= Message-ID: <3g8pVq5SqKz7LmY@mail.python.org> http://hg.python.org/cpython/rev/635817da596d changeset: 90387:635817da596d branch: 3.4 parent: 90384:ca2edbefca35 user: doko at ubuntu.com date: Thu Apr 17 20:13:44 2014 +0200 summary: - Issue #21285: Refactor and fix curses configure check to always search in a ncursesw directory. files: Misc/NEWS | 3 + configure | 62 +++++++++++++++++++++++---------------- configure.ac | 26 ++++++++-------- 3 files changed, 53 insertions(+), 38 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -174,6 +174,9 @@ Build ----- +- Issue #21285: Refactor and fix curses configure check to always search + in a ncursesw directory. + - Issue #15234: For BerkelyDB and Sqlite, only add the found library and include directories if they aren't already being searched. This avoids an explicit runtime library dependency. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6969,11 +6969,9 @@ fi -ac_save_cppflags="$CPPFLAGS" -CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" -for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +for ac_header in asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ -ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \ +ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ sched.h shadow.h signal.h stdint.h stropts.h termios.h \ unistd.h utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ @@ -6997,7 +6995,6 @@ done -CPPFLAGS=$ac_save_cppflags ac_header_dirent=no for ac_hdr in dirent.h sys/ndir.h sys/dir.h ndir.h; do as_ac_Header=`$as_echo "ac_cv_header_dirent_$ac_hdr" | $as_tr_sh` @@ -7234,26 +7231,6 @@ done - -# On Solaris, term.h requires curses.h -for ac_header in term.h -do : - ac_fn_c_check_header_compile "$LINENO" "term.h" "ac_cv_header_term_h" " -#ifdef HAVE_CURSES_H -#include -#endif - -" -if test "x$ac_cv_header_term_h" = xyes; then : - cat >>confdefs.h <<_ACEOF -#define HAVE_TERM_H 1 -_ACEOF - -fi - -done - - # On Linux, netlink.h requires asm/types.h for ac_header in linux/netlink.h do : @@ -14677,8 +14654,43 @@ fi +# first curses header check ac_save_cppflags="$CPPFLAGS" CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" + +for ac_header in curses.h ncurses.h +do : + as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` +ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" +if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : + cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 +_ACEOF + +fi + +done + + +# On Solaris, term.h requires curses.h +for ac_header in term.h +do : + ac_fn_c_check_header_compile "$LINENO" "term.h" "ac_cv_header_term_h" " +#ifdef HAVE_CURSES_H +#include +#endif + +" +if test "x$ac_cv_header_term_h" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_TERM_H 1 +_ACEOF + +fi + +done + + # On HP/UX 11.0, mvwdelch is a block with a return statement { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether mvwdelch is an expression" >&5 $as_echo_n "checking whether mvwdelch is an expression... " >&6; } diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1546,11 +1546,9 @@ # checks for header files AC_HEADER_STDC -ac_save_cppflags="$CPPFLAGS" -CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" -AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +AC_CHECK_HEADERS(asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ -ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \ +ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ sched.h shadow.h signal.h stdint.h stropts.h termios.h \ unistd.h utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ @@ -1562,7 +1560,6 @@ libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h endian.h \ sys/endian.h) -CPPFLAGS=$ac_save_cppflags AC_HEADER_DIRENT AC_HEADER_MAJOR @@ -1582,14 +1579,6 @@ #endif ]) - -# On Solaris, term.h requires curses.h -AC_CHECK_HEADERS(term.h,,,[ -#ifdef HAVE_CURSES_H -#include -#endif -]) - # On Linux, netlink.h requires asm/types.h AC_CHECK_HEADERS(linux/netlink.h,,,[ #ifdef HAVE_ASM_TYPES_H @@ -4391,8 +4380,19 @@ [Define if you have struct stat.st_mtimensec]) fi +# first curses header check ac_save_cppflags="$CPPFLAGS" CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" + +AC_CHECK_HEADERS(curses.h ncurses.h) + +# On Solaris, term.h requires curses.h +AC_CHECK_HEADERS(term.h,,,[ +#ifdef HAVE_CURSES_H +#include +#endif +]) + # On HP/UX 11.0, mvwdelch is a block with a return statement AC_MSG_CHECKING(whether mvwdelch is an expression) AC_CACHE_VAL(ac_cv_mvwdelch_is_expression, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 17 20:14:45 2014 From: python-checkins at python.org (matthias.klose) Date: Thu, 17 Apr 2014 20:14:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_-_Merge_3=2E4?= Message-ID: <3g8pVs2L9jz7Ln1@mail.python.org> http://hg.python.org/cpython/rev/85fd955c6fc8 changeset: 90388:85fd955c6fc8 parent: 90385:bdf002b9c20c parent: 90387:635817da596d user: doko at ubuntu.com date: Thu Apr 17 20:14:29 2014 +0200 summary: - Merge 3.4 files: Misc/NEWS | 3 + configure | 62 +++++++++++++++++++++++---------------- configure.ac | 26 ++++++++-------- 3 files changed, 53 insertions(+), 38 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -251,6 +251,9 @@ Build ----- +- Issue #21285: Refactor and fix curses configure check to always search + in a ncursesw directory. + - Issue #15234: For BerkelyDB and Sqlite, only add the found library and include directories if they aren't already being searched. This avoids an explicit runtime library dependency. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -7021,11 +7021,9 @@ fi -ac_save_cppflags="$CPPFLAGS" -CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" -for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +for ac_header in asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ -ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \ +ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ sched.h shadow.h signal.h stdint.h stropts.h termios.h \ unistd.h utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ @@ -7049,7 +7047,6 @@ done -CPPFLAGS=$ac_save_cppflags ac_header_dirent=no for ac_hdr in dirent.h sys/ndir.h sys/dir.h ndir.h; do as_ac_Header=`$as_echo "ac_cv_header_dirent_$ac_hdr" | $as_tr_sh` @@ -7286,26 +7283,6 @@ done - -# On Solaris, term.h requires curses.h -for ac_header in term.h -do : - ac_fn_c_check_header_compile "$LINENO" "term.h" "ac_cv_header_term_h" " -#ifdef HAVE_CURSES_H -#include -#endif - -" -if test "x$ac_cv_header_term_h" = xyes; then : - cat >>confdefs.h <<_ACEOF -#define HAVE_TERM_H 1 -_ACEOF - -fi - -done - - # On Linux, netlink.h requires asm/types.h for ac_header in linux/netlink.h do : @@ -14761,8 +14738,43 @@ fi +# first curses header check ac_save_cppflags="$CPPFLAGS" CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" + +for ac_header in curses.h ncurses.h +do : + as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` +ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" +if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : + cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 +_ACEOF + +fi + +done + + +# On Solaris, term.h requires curses.h +for ac_header in term.h +do : + ac_fn_c_check_header_compile "$LINENO" "term.h" "ac_cv_header_term_h" " +#ifdef HAVE_CURSES_H +#include +#endif + +" +if test "x$ac_cv_header_term_h" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_TERM_H 1 +_ACEOF + +fi + +done + + # On HP/UX 11.0, mvwdelch is a block with a return statement { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether mvwdelch is an expression" >&5 $as_echo_n "checking whether mvwdelch is an expression... " >&6; } diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1555,11 +1555,9 @@ # checks for header files AC_HEADER_STDC -ac_save_cppflags="$CPPFLAGS" -CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" -AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +AC_CHECK_HEADERS(asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ -ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \ +ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ sched.h shadow.h signal.h stdint.h stropts.h termios.h \ unistd.h utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ @@ -1571,7 +1569,6 @@ libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h endian.h \ sys/endian.h) -CPPFLAGS=$ac_save_cppflags AC_HEADER_DIRENT AC_HEADER_MAJOR @@ -1591,14 +1588,6 @@ #endif ]) - -# On Solaris, term.h requires curses.h -AC_CHECK_HEADERS(term.h,,,[ -#ifdef HAVE_CURSES_H -#include -#endif -]) - # On Linux, netlink.h requires asm/types.h AC_CHECK_HEADERS(linux/netlink.h,,,[ #ifdef HAVE_ASM_TYPES_H @@ -4413,8 +4402,19 @@ [Define if you have struct stat.st_mtimensec]) fi +# first curses header check ac_save_cppflags="$CPPFLAGS" CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" + +AC_CHECK_HEADERS(curses.h ncurses.h) + +# On Solaris, term.h requires curses.h +AC_CHECK_HEADERS(term.h,,,[ +#ifdef HAVE_CURSES_H +#include +#endif +]) + # On HP/UX 11.0, mvwdelch is a block with a return statement AC_MSG_CHECKING(whether mvwdelch is an expression) AC_CACHE_VAL(ac_cv_mvwdelch_is_expression, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 00:29:35 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 18 Apr 2014 00:29:35 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_ref_count_?= =?utf-8?q?annotations_on_sphinx_=3E=3D_1=2E2=2E1_=28closes_=2321286=29?= Message-ID: <3g8w8v0By3z7Ljf@mail.python.org> http://hg.python.org/cpython/rev/8c12d3e0f1de changeset: 90389:8c12d3e0f1de branch: 3.4 parent: 90387:635817da596d user: Benjamin Peterson date: Thu Apr 17 18:29:01 2014 -0400 summary: fix ref count annotations on sphinx >= 1.2.1 (closes #21286) files: Doc/tools/sphinxext/c_annotations.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Doc/tools/sphinxext/c_annotations.py b/Doc/tools/sphinxext/c_annotations.py --- a/Doc/tools/sphinxext/c_annotations.py +++ b/Doc/tools/sphinxext/c_annotations.py @@ -81,7 +81,10 @@ continue if not par[0].has_key('names') or not par[0]['names']: continue - entry = self.get(par[0]['names'][0]) + name = par[0]['names'][0] + if name.startswith("c."): + name = name[2:] + entry = self.get(name) if not entry: continue elif entry.result_type not in ("PyObject*", "PyVarObject*"): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 00:29:36 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 18 Apr 2014 00:29:36 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g8w8w1cwBz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/1122b85c5f7a changeset: 90390:1122b85c5f7a parent: 90388:85fd955c6fc8 parent: 90389:8c12d3e0f1de user: Benjamin Peterson date: Thu Apr 17 18:29:27 2014 -0400 summary: merge 3.4 files: Doc/tools/sphinxext/c_annotations.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Doc/tools/sphinxext/c_annotations.py b/Doc/tools/sphinxext/c_annotations.py --- a/Doc/tools/sphinxext/c_annotations.py +++ b/Doc/tools/sphinxext/c_annotations.py @@ -81,7 +81,10 @@ continue if not par[0].has_key('names') or not par[0]['names']: continue - entry = self.get(par[0]['names'][0]) + name = par[0]['names'][0] + if name.startswith("c."): + name = name[2:] + entry = self.get(name) if not entry: continue elif entry.result_type not in ("PyObject*", "PyVarObject*"): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 07:04:16 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 18 Apr 2014 07:04:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_correct_len_si?= =?utf-8?q?gnature_in_docstring_=28closes_=2321294=29?= Message-ID: <3g94wJ5M9wz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/679319e3f42b changeset: 90391:679319e3f42b branch: 3.4 parent: 90389:8c12d3e0f1de user: Benjamin Peterson date: Fri Apr 18 01:03:59 2014 -0400 summary: correct len signature in docstring (closes #21294) files: Python/bltinmodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1325,7 +1325,7 @@ } PyDoc_STRVAR(len_doc, -"len(module, object)\n\ +"len(object)\n\ \n\ Return the number of items of a sequence or mapping."); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 07:04:18 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 18 Apr 2014 07:04:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3g94wL113Mz7Lk6@mail.python.org> http://hg.python.org/cpython/rev/7433f7bce880 changeset: 90392:7433f7bce880 parent: 90390:1122b85c5f7a parent: 90391:679319e3f42b user: Benjamin Peterson date: Fri Apr 18 01:04:07 2014 -0400 summary: merge 3.4 files: Python/bltinmodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1325,7 +1325,7 @@ } PyDoc_STRVAR(len_doc, -"len(module, object)\n\ +"len(object)\n\ \n\ Return the number of items of a sequence or mapping."); -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Fri Apr 18 09:42:08 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 18 Apr 2014 09:42:08 +0200 Subject: [Python-checkins] Daily reference leaks (1122b85c5f7a): sum=-1 Message-ID: results for 1122b85c5f7a on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogAGCMgN', '-x'] From python-checkins at python.org Fri Apr 18 16:17:44 2014 From: python-checkins at python.org (zach.ware) Date: Fri, 18 Apr 2014 16:17:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_a_typo_in_?= =?utf-8?q?the_signature_for_object=2E=5F=5Fge=5F=5F?= Message-ID: <3g9KBw5t0Wz7LjS@mail.python.org> http://hg.python.org/cpython/rev/7b7885a1573c changeset: 90393:7b7885a1573c branch: 3.4 parent: 90391:679319e3f42b user: Zachary Ware date: Fri Apr 18 09:14:31 2014 -0500 summary: Fix a typo in the signature for object.__ge__ files: Objects/typeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6180,7 +6180,7 @@ TPSLOT("__gt__", tp_richcompare, slot_tp_richcompare, richcmp_gt, "__gt__($self, value, /)\n--\n\nReturn self>value."), TPSLOT("__ge__", tp_richcompare, slot_tp_richcompare, richcmp_ge, - "__ge__=($self, value, /)\n--\n\nReturn self>=value."), + "__ge__($self, value, /)\n--\n\nReturn self>=value."), TPSLOT("__iter__", tp_iter, slot_tp_iter, wrap_unaryfunc, "__iter__($self, /)\n--\n\nImplement iter(self)."), TPSLOT("__next__", tp_iternext, slot_tp_iternext, wrap_next, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 16:17:46 2014 From: python-checkins at python.org (zach.ware) Date: Fri, 18 Apr 2014 16:17:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_typo_fix_from_3=2E4?= Message-ID: <3g9KBy0Zpkz7Ljk@mail.python.org> http://hg.python.org/cpython/rev/4abd734cf1fb changeset: 90394:4abd734cf1fb parent: 90392:7433f7bce880 parent: 90393:7b7885a1573c user: Zachary Ware date: Fri Apr 18 09:17:04 2014 -0500 summary: Merge typo fix from 3.4 files: Objects/typeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6184,7 +6184,7 @@ TPSLOT("__gt__", tp_richcompare, slot_tp_richcompare, richcmp_gt, "__gt__($self, value, /)\n--\n\nReturn self>value."), TPSLOT("__ge__", tp_richcompare, slot_tp_richcompare, richcmp_ge, - "__ge__=($self, value, /)\n--\n\nReturn self>=value."), + "__ge__($self, value, /)\n--\n\nReturn self>=value."), TPSLOT("__iter__", tp_iter, slot_tp_iter, wrap_unaryfunc, "__iter__($self, /)\n--\n\nImplement iter(self)."), TPSLOT("__next__", tp_iternext, slot_tp_iternext, wrap_next, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 16:23:59 2014 From: python-checkins at python.org (zach.ware) Date: Fri, 18 Apr 2014 16:23:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Fix_a_typo_in_?= =?utf-8?q?the_docstring_of_nb=5Findex=2E?= Message-ID: <3g9KL73G09z7LjS@mail.python.org> http://hg.python.org/cpython/rev/57f914880078 changeset: 90395:57f914880078 branch: 3.4 parent: 90393:7b7885a1573c user: Zachary Ware date: Fri Apr 18 09:23:14 2014 -0500 summary: Fix a typo in the docstring of nb_index. files: Objects/typeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6276,7 +6276,7 @@ slot_nb_inplace_true_divide, wrap_binaryfunc, "/"), NBSLOT("__index__", nb_index, slot_nb_index, wrap_unaryfunc, "__index__($self, /)\n--\n\n" - "Return self converted to an integer, if self is suitable" + "Return self converted to an integer, if self is suitable " "for use as an index into a list."), MPSLOT("__len__", mp_length, slot_mp_length, wrap_lenfunc, "__len__($self, /)\n--\n\nReturn len(self)."), -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 16:24:00 2014 From: python-checkins at python.org (zach.ware) Date: Fri, 18 Apr 2014 16:24:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_typo_fix_from_3=2E4?= Message-ID: <3g9KL854G2z7LjS@mail.python.org> http://hg.python.org/cpython/rev/00488e3cdc36 changeset: 90396:00488e3cdc36 parent: 90394:4abd734cf1fb parent: 90395:57f914880078 user: Zachary Ware date: Fri Apr 18 09:23:35 2014 -0500 summary: Merge typo fix from 3.4 files: Objects/typeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6280,7 +6280,7 @@ slot_nb_inplace_true_divide, wrap_binaryfunc, "/"), NBSLOT("__index__", nb_index, slot_nb_index, wrap_unaryfunc, "__index__($self, /)\n--\n\n" - "Return self converted to an integer, if self is suitable" + "Return self converted to an integer, if self is suitable " "for use as an index into a list."), BINSLOT("__matmul__", nb_matrix_multiply, slot_nb_matrix_multiply, "@"), -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 17:09:10 2014 From: python-checkins at python.org (mark.dickinson) Date: Fri, 18 Apr 2014 17:09:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_typo_in_test_method_na?= =?utf-8?b?bWUu?= Message-ID: <3g9LLG651Qz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/b64be5cb67a6 changeset: 90397:b64be5cb67a6 user: Mark Dickinson date: Fri Apr 18 16:08:47 2014 +0100 summary: Fix typo in test method name. files: Lib/unittest/test/test_case.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/unittest/test/test_case.py b/Lib/unittest/test/test_case.py --- a/Lib/unittest/test/test_case.py +++ b/Lib/unittest/test/test_case.py @@ -1080,7 +1080,7 @@ # so can't use assertEqual either. Just use assertTrue. self.assertTrue(sample_text_error == error) - def testAsertEqualSingleLine(self): + def testAssertEqualSingleLine(self): sample_text = "laden swallows fly slowly" revised_sample_text = "unladen swallows fly quickly" sample_text_error = """\ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 18:06:09 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 18 Apr 2014 18:06:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_asyncio=3A_Add_gi=5F=7Bfra?= =?utf-8?q?me=2Crunning=2Ccode=7D_properties_to_CoroWrapper_=28upstream_?= =?utf-8?b?IzE2Myku?= Message-ID: <3g9Mc16NJZz7LjS@mail.python.org> http://hg.python.org/cpython/rev/c776ed8a8e75 changeset: 90398:c776ed8a8e75 user: Guido van Rossum date: Tue Apr 15 12:06:34 2014 -0700 summary: asyncio: Add gi_{frame,running,code} properties to CoroWrapper (upstream #163). files: Lib/asyncio/tasks.py | 12 ++++ Lib/test/test_asyncio/test_tasks.py | 47 +++++++++++++++++ 2 files changed, 59 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -63,6 +63,18 @@ def close(self): return self.gen.close() + @property + def gi_frame(self): + return self.gen.gi_frame + + @property + def gi_running(self): + return self.gen.gi_running + + @property + def gi_code(self): + return self.gen.gi_code + def __del__(self): frame = self.gen.gi_frame if frame is not None and frame.f_lasti == -1: diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -2,6 +2,7 @@ import gc import os.path +import types import unittest from test.script_helper import assert_python_ok @@ -1386,6 +1387,52 @@ self.assertRaises(ValueError, self.loop.run_until_complete, asyncio.wait([], loop=self.loop)) + def test_corowrapper_mocks_generator(self): + + def check(): + # A function that asserts various things. + # Called twice, with different debug flag values. + + @asyncio.coroutine + def coro(): + # The actual coroutine. + self.assertTrue(gen.gi_running) + yield from fut + + # A completed Future used to run the coroutine. + fut = asyncio.Future(loop=self.loop) + fut.set_result(None) + + # Call the coroutine. + gen = coro() + + # Check some properties. + self.assertTrue(asyncio.iscoroutine(gen)) + self.assertIsInstance(gen.gi_frame, types.FrameType) + self.assertFalse(gen.gi_running) + self.assertIsInstance(gen.gi_code, types.CodeType) + + # Run it. + self.loop.run_until_complete(gen) + + # The frame should have changed. + self.assertIsNone(gen.gi_frame) + + # Save debug flag. + old_debug = asyncio.tasks._DEBUG + try: + # Test with debug flag cleared. + asyncio.tasks._DEBUG = False + check() + + # Test with debug flag set. + asyncio.tasks._DEBUG = True + check() + + finally: + # Restore original debug flag. + asyncio.tasks._DEBUG = old_debug + def test_yield_from_corowrapper(self): old_debug = asyncio.tasks._DEBUG asyncio.tasks._DEBUG = True -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 18:13:28 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 18 Apr 2014 18:13:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbzogQWRk?= =?utf-8?q?_gi=5F=7Bframe=2Crunning=2Ccode=7D_properties_to_CoroWrapper_?= =?utf-8?q?=28upstream_=23163=29=2E?= Message-ID: <3g9MmS6zrTz7LjN@mail.python.org> http://hg.python.org/cpython/rev/6278ceba94ac changeset: 90399:6278ceba94ac branch: 3.4 parent: 90395:57f914880078 user: Guido van Rossum date: Tue Apr 15 12:06:34 2014 -0700 summary: asyncio: Add gi_{frame,running,code} properties to CoroWrapper (upstream #163). files: Lib/asyncio/tasks.py | 12 ++++ Lib/test/test_asyncio/test_tasks.py | 47 +++++++++++++++++ 2 files changed, 59 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -63,6 +63,18 @@ def close(self): return self.gen.close() + @property + def gi_frame(self): + return self.gen.gi_frame + + @property + def gi_running(self): + return self.gen.gi_running + + @property + def gi_code(self): + return self.gen.gi_code + def __del__(self): frame = self.gen.gi_frame if frame is not None and frame.f_lasti == -1: diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -2,6 +2,7 @@ import gc import os.path +import types import unittest from test.script_helper import assert_python_ok @@ -1386,6 +1387,52 @@ self.assertRaises(ValueError, self.loop.run_until_complete, asyncio.wait([], loop=self.loop)) + def test_corowrapper_mocks_generator(self): + + def check(): + # A function that asserts various things. + # Called twice, with different debug flag values. + + @asyncio.coroutine + def coro(): + # The actual coroutine. + self.assertTrue(gen.gi_running) + yield from fut + + # A completed Future used to run the coroutine. + fut = asyncio.Future(loop=self.loop) + fut.set_result(None) + + # Call the coroutine. + gen = coro() + + # Check some properties. + self.assertTrue(asyncio.iscoroutine(gen)) + self.assertIsInstance(gen.gi_frame, types.FrameType) + self.assertFalse(gen.gi_running) + self.assertIsInstance(gen.gi_code, types.CodeType) + + # Run it. + self.loop.run_until_complete(gen) + + # The frame should have changed. + self.assertIsNone(gen.gi_frame) + + # Save debug flag. + old_debug = asyncio.tasks._DEBUG + try: + # Test with debug flag cleared. + asyncio.tasks._DEBUG = False + check() + + # Test with debug flag set. + asyncio.tasks._DEBUG = True + check() + + finally: + # Restore original debug flag. + asyncio.tasks._DEBUG = old_debug + def test_yield_from_corowrapper(self): old_debug = asyncio.tasks._DEBUG asyncio.tasks._DEBUG = True -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 18:13:30 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 18 Apr 2014 18:13:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy40?= Message-ID: <3g9MmV1TGwz7LjS@mail.python.org> http://hg.python.org/cpython/rev/93f29fa758e0 changeset: 90400:93f29fa758e0 parent: 90398:c776ed8a8e75 parent: 90399:6278ceba94ac user: Guido van Rossum date: Fri Apr 18 09:13:12 2014 -0700 summary: Merge 3.4 files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 18:22:09 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 18 Apr 2014 18:22:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_NEWS_for_lates?= =?utf-8?q?t_asyncio_commit=2E?= Message-ID: <3g9MyT0qZqz7Ljq@mail.python.org> http://hg.python.org/cpython/rev/64782d0d16fe changeset: 90401:64782d0d16fe branch: 3.4 parent: 90399:6278ceba94ac user: Guido van Rossum date: Fri Apr 18 09:21:26 2014 -0700 summary: NEWS for latest asyncio commit. files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -160,6 +160,9 @@ - Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug in yield-from implementation in CPythons prior to 3.4.1. +- asyncio: Add gi_{frame,running,code} properties to CoroWrapper + (upstream issue #163). + Extension Modules ----------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 18:22:10 2014 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 18 Apr 2014 18:22:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_news_item_from_3=2E4?= Message-ID: <3g9MyV2Fn7z7LmC@mail.python.org> http://hg.python.org/cpython/rev/6f133133c5da changeset: 90402:6f133133c5da parent: 90400:93f29fa758e0 parent: 90401:64782d0d16fe user: Guido van Rossum date: Fri Apr 18 09:21:55 2014 -0700 summary: Merge news item from 3.4 files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -237,6 +237,9 @@ - Issue #21209: Fix asyncio.tasks.CoroWrapper to workaround a bug in yield-from implementation in CPythons prior to 3.4.1. +- asyncio: Add gi_{frame,running,code} properties to CoroWrapper + (upstream issue #163). + Extension Modules ----------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 20:11:04 2014 From: python-checkins at python.org (philip.jenvey) Date: Fri, 18 Apr 2014 20:11:04 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_the_description_of_byt?= =?utf-8?q?es_tests?= Message-ID: <3g9QN83b98z7LjS@mail.python.org> http://hg.python.org/cpython/rev/1adcc23de886 changeset: 90403:1adcc23de886 user: Philip Jenvey date: Fri Apr 18 11:10:50 2014 -0700 summary: fix the description of bytes tests files: Lib/test/test_set.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -929,7 +929,7 @@ class TestBasicOpsBytes(TestBasicOps, unittest.TestCase): def setUp(self): - self.case = "string set" + self.case = "bytes set" self.values = [b"a", b"b", b"c"] self.set = set(self.values) self.dup = set(self.values) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 20:33:15 2014 From: python-checkins at python.org (antoine.pitrou) Date: Fri, 18 Apr 2014 20:33:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321068=3A_The_ssl?= =?utf-8?q?=2EPROTOCOL*_constants_are_now_enum_members=2E?= Message-ID: <3g9Qsl3j71z7LjN@mail.python.org> http://hg.python.org/cpython/rev/f776771ab0ee changeset: 90404:f776771ab0ee user: Antoine Pitrou date: Fri Apr 18 20:33:08 2014 +0200 summary: Issue #21068: The ssl.PROTOCOL* constants are now enum members. files: Lib/ssl.py | 29 +++++++++-------------------- Lib/test/test_ssl.py | 8 ++++++++ Misc/NEWS | 2 ++ 3 files changed, 19 insertions(+), 20 deletions(-) diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -92,7 +92,7 @@ import sys import os from collections import namedtuple -from enum import Enum as _Enum +from enum import Enum as _Enum, IntEnum as _IntEnum import _ssl # if we can't import it, let the error propagate @@ -119,30 +119,19 @@ from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN -from _ssl import PROTOCOL_SSLv3, PROTOCOL_SSLv23, PROTOCOL_TLSv1 from _ssl import _OPENSSL_API_VERSION +_SSLMethod = _IntEnum('_SSLMethod', + {name: value for name, value in vars(_ssl).items() + if name.startswith('PROTOCOL_')}) +globals().update(_SSLMethod.__members__) -_PROTOCOL_NAMES = { - PROTOCOL_TLSv1: "TLSv1", - PROTOCOL_SSLv23: "SSLv23", - PROTOCOL_SSLv3: "SSLv3", -} -try: - from _ssl import PROTOCOL_SSLv2 - _SSLv2_IF_EXISTS = PROTOCOL_SSLv2 -except ImportError: - _SSLv2_IF_EXISTS = None -else: - _PROTOCOL_NAMES[PROTOCOL_SSLv2] = "SSLv2" +_PROTOCOL_NAMES = {value: name for name, value in _SSLMethod.__members__.items()} try: - from _ssl import PROTOCOL_TLSv1_1, PROTOCOL_TLSv1_2 -except ImportError: - pass -else: - _PROTOCOL_NAMES[PROTOCOL_TLSv1_1] = "TLSv1.1" - _PROTOCOL_NAMES[PROTOCOL_TLSv1_2] = "TLSv1.2" + _SSLv2_IF_EXISTS = PROTOCOL_SSLv2 +except NameError: + _SSLv2_IF_EXISTS = None if sys.platform == "win32": from _ssl import enum_certificates, enum_crls diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -134,6 +134,14 @@ self.assertIn(ssl.HAS_SNI, {True, False}) self.assertIn(ssl.HAS_ECDH, {True, False}) + def test_str_for_enums(self): + # Make sure that the PROTOCOL_* constants have enum-like string + # reprs. + proto = ssl.PROTOCOL_SSLv3 + self.assertEqual(str(proto), '_SSLMethod.PROTOCOL_SSLv3') + ctx = ssl.SSLContext(proto) + self.assertIs(ctx.protocol, proto) + def test_random(self): v = ssl.RAND_status() if support.verbose: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -54,6 +54,8 @@ Library ------- +- Issue #21068: The ssl.PROTOCOL* constants are now enum members. + - Issue #21262: New method assert_not_called for Mock. It raises AssertionError if the mock has been called. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 22:45:47 2014 From: python-checkins at python.org (zach.ware) Date: Fri, 18 Apr 2014 22:45:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMjg5?= =?utf-8?q?=3A_Fix_documentation_building_on_Windows_using_Doc/make=2Ebat?= =?utf-8?q?=2E?= Message-ID: <3g9Tpg6ZSJz7LjS@mail.python.org> http://hg.python.org/cpython/rev/02fec733f760 changeset: 90405:02fec733f760 branch: 3.4 parent: 90401:64782d0d16fe user: Zachary Ware date: Fri Apr 18 15:10:40 2014 -0500 summary: Issue #21289: Fix documentation building on Windows using Doc/make.bat. Also fixes a long-standing typo in Doc/README.txt. Initial patch by Dave Sawyer. files: Doc/README.txt | 2 +- Doc/make.bat | 6 +++--- Misc/ACKS | 1 + 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -81,7 +81,7 @@ Install the Sphinx package and its dependencies from PyPI. -Then, from the ``Docs`` directory, run :: +Then, from the ``Doc`` directory, run :: sphinx-build -b . build/ diff --git a/Doc/make.bat b/Doc/make.bat --- a/Doc/make.bat +++ b/Doc/make.bat @@ -1,7 +1,8 @@ @@echo off setlocal -if "%PYTHON%" EQU "" set PYTHON=py -2 +if "%SPHINXBUILD%" EQU "" set SPHINXBUILD=sphinx-build +if "%PYTHON%" EQU "" set PYTHON=py if "%HTMLHELP%" EQU "" set HTMLHELP=%ProgramFiles%\HTML Help Workshop\hhc.exe if "%DISTVERSION%" EQU "" for /f "usebackq" %%v in (`%PYTHON% tools/sphinxext/patchlevel.py`) do set DISTVERSION=%%v @@ -32,8 +33,7 @@ if not exist build mkdir build if not exist build\%1 mkdir build\%1 if not exist build\doctrees mkdir build\doctrees -cmd /C %PYTHON% --version -cmd /C %PYTHON% tools\sphinx-build.py -b%1 -dbuild\doctrees . build\%* +cmd /C %SPHINXBUILD% -b%1 -dbuild\doctrees . build\%* if "%1" EQU "htmlhelp" "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp goto end diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1152,6 +1152,7 @@ Ty Sarna Hugh Sasse Bob Savage +Dave Sawyer Ben Sayer sbt Marco Scataglini -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 22:45:49 2014 From: python-checkins at python.org (zach.ware) Date: Fri, 18 Apr 2014 22:45:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3g9Tpj1DhDz7Ljp@mail.python.org> http://hg.python.org/cpython/rev/761785106b31 changeset: 90406:761785106b31 parent: 90404:f776771ab0ee parent: 90405:02fec733f760 user: Zachary Ware date: Fri Apr 18 15:45:02 2014 -0500 summary: Merge with 3.4 files: Doc/README.txt | 2 +- Doc/make.bat | 6 +++--- Misc/ACKS | 1 + 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -81,7 +81,7 @@ Install the Sphinx package and its dependencies from PyPI. -Then, from the ``Docs`` directory, run :: +Then, from the ``Doc`` directory, run :: sphinx-build -b . build/ diff --git a/Doc/make.bat b/Doc/make.bat --- a/Doc/make.bat +++ b/Doc/make.bat @@ -1,7 +1,8 @@ @@echo off setlocal -if "%PYTHON%" EQU "" set PYTHON=py -2 +if "%SPHINXBUILD%" EQU "" set SPHINXBUILD=sphinx-build +if "%PYTHON%" EQU "" set PYTHON=py if "%HTMLHELP%" EQU "" set HTMLHELP=%ProgramFiles%\HTML Help Workshop\hhc.exe if "%DISTVERSION%" EQU "" for /f "usebackq" %%v in (`%PYTHON% tools/sphinxext/patchlevel.py`) do set DISTVERSION=%%v @@ -32,8 +33,7 @@ if not exist build mkdir build if not exist build\%1 mkdir build\%1 if not exist build\doctrees mkdir build\doctrees -cmd /C %PYTHON% --version -cmd /C %PYTHON% tools\sphinx-build.py -b%1 -dbuild\doctrees . build\%* +cmd /C %SPHINXBUILD% -b%1 -dbuild\doctrees . build\%* if "%1" EQU "htmlhelp" "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp goto end diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1154,6 +1154,7 @@ Ty Sarna Hugh Sasse Bob Savage +Dave Sawyer Ben Sayer sbt Marco Scataglini -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 23:01:33 2014 From: python-checkins at python.org (terry.reedy) Date: Fri, 18 Apr 2014 23:01:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMjMy?= =?utf-8?q?=3A_Replace_=2Esplitlines_arg_=271=27_with_=27keepends=3DTrue?= =?utf-8?b?Jy4=?= Message-ID: <3g9V8s6plVz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/604b74f9a07d changeset: 90407:604b74f9a07d branch: 3.4 parent: 90405:02fec733f760 user: Terry Jan Reedy date: Fri Apr 18 17:00:19 2014 -0400 summary: Issue #21232: Replace .splitlines arg '1' with 'keepends=True'. files: Doc/library/difflib.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -226,8 +226,8 @@ :file:`Tools/scripts/ndiff.py` is a command-line front-end to this function. - >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(1), - ... 'ore\ntree\nemu\n'.splitlines(1)) + >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True), + ... 'ore\ntree\nemu\n'.splitlines(keepends=True)) >>> print(''.join(diff), end="") - one ? ^ @@ -250,8 +250,8 @@ Example: - >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(1), - ... 'ore\ntree\nemu\n'.splitlines(1)) + >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True), + ... 'ore\ntree\nemu\n'.splitlines(keepends=True)) >>> diff = list(diff) # materialize the generated delta into a list >>> print(''.join(restore(diff, 1)), end="") one @@ -650,7 +650,7 @@ ... 2. Explicit is better than implicit. ... 3. Simple is better than complex. ... 4. Complex is better than complicated. - ... '''.splitlines(1) + ... '''.splitlines(keepends=True) >>> len(text1) 4 >>> text1[0][-1] @@ -659,7 +659,7 @@ ... 3. Simple is better than complex. ... 4. Complicated is better than complex. ... 5. Flat is better than nested. - ... '''.splitlines(1) + ... '''.splitlines(keepends=True) Next we instantiate a Differ object: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 18 23:01:35 2014 From: python-checkins at python.org (terry.reedy) Date: Fri, 18 Apr 2014 23:01:35 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogTWVyZ2Ugd2l0aCAzLjQuIENsb3NlcyAjMjEyMzIu?= Message-ID: <3g9V8v1NCWz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/c82dcad83438 changeset: 90408:c82dcad83438 parent: 90406:761785106b31 parent: 90407:604b74f9a07d user: Terry Jan Reedy date: Fri Apr 18 17:00:50 2014 -0400 summary: Merge with 3.4. Closes #21232. files: Doc/library/difflib.rst | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -230,8 +230,8 @@ :file:`Tools/scripts/ndiff.py` is a command-line front-end to this function. - >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(1), - ... 'ore\ntree\nemu\n'.splitlines(1)) + >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True), + ... 'ore\ntree\nemu\n'.splitlines(keepends=True)) >>> print(''.join(diff), end="") - one ? ^ @@ -254,8 +254,8 @@ Example: - >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(1), - ... 'ore\ntree\nemu\n'.splitlines(1)) + >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True), + ... 'ore\ntree\nemu\n'.splitlines(keepends=True)) >>> diff = list(diff) # materialize the generated delta into a list >>> print(''.join(restore(diff, 1)), end="") one @@ -660,7 +660,7 @@ ... 2. Explicit is better than implicit. ... 3. Simple is better than complex. ... 4. Complex is better than complicated. - ... '''.splitlines(1) + ... '''.splitlines(keepends=True) >>> len(text1) 4 >>> text1[0][-1] @@ -669,7 +669,7 @@ ... 3. Simple is better than complex. ... 4. Complicated is better than complex. ... 5. Flat is better than nested. - ... '''.splitlines(1) + ... '''.splitlines(keepends=True) Next we instantiate a Differ object: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 02:48:05 2014 From: python-checkins at python.org (nick.coghlan) Date: Sat, 19 Apr 2014 02:48:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Guido_has_accepted_PEP_466?= Message-ID: <3g9bBF16Qkz7LjN@mail.python.org> http://hg.python.org/peps/rev/d905b6f9c6a9 changeset: 5462:d905b6f9c6a9 user: Nick Coghlan date: Fri Apr 18 20:47:56 2014 -0400 summary: Guido has accepted PEP 466 files: pep-0466.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0466.txt b/pep-0466.txt --- a/pep-0466.txt +++ b/pep-0466.txt @@ -3,12 +3,12 @@ Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , -Status: Draft -Type: Informational +Status: Accepted +Type: Standards Track Content-Type: text/x-rst Created: 23-Mar-2014 Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014, 16-Apr-2014 - +Resolution: https://mail.python.org/pipermail/python-dev/2014-April/134163.html Abstract ======== -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sat Apr 19 04:07:33 2014 From: python-checkins at python.org (nick.coghlan) Date: Sat, 19 Apr 2014 04:07:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_The_folks_at_the_Twisted_spri?= =?utf-8?q?nt_made_a_good_case_=3A=29?= Message-ID: <3g9cxx4xR5z7LjN@mail.python.org> http://hg.python.org/peps/rev/27829cec6787 changeset: 5463:27829cec6787 user: Nick Coghlan date: Fri Apr 18 22:07:27 2014 -0400 summary: The folks at the Twisted sprint made a good case :) files: pep-0469.txt | 126 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 126 insertions(+), 0 deletions(-) diff --git a/pep-0469.txt b/pep-0469.txt new file mode 100644 --- /dev/null +++ b/pep-0469.txt @@ -0,0 +1,126 @@ +PEP: 469 +Title: Simplified migration of iterator-based mapping code to Python 3 +Version: $Revision$ +Last-Modified: $Date$ +Author: Nick Coghlan +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 2014-04-18 +Python-Version: 3.5 +Post-History: 2014-04-18 + + +Abstract +======== + +For Python 3, PEP 3106 changed the design of the ``dict`` builtin and the +mapping API in general to replace the separate list based and iterator based +APIs in Python 2 with a merged, memory efficient set and multiset view +based API. + +This means that Python 3 code always requires an additional qualifier to +reliably reproduce classic Python 2 mapping semantics: + + * List based (e.g. ``d.keys()``): ``list(d.keys())`` + * Iterator based (e.g. ``d.iterkeys()``): ``iter(d.keys())`` + +Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3 +(or the common subset of Python 2 and Python 3) without alteration, but +*all* code using the iterator based API requires modification. Code that +is migrating to the common subset of Python 2 and 3 and needs to retain the +memory efficient implementation that avoids creating an unnecessary list +object must switch away from using a method to instead using a helper +function (such as those provided by the ``six`` module) + +To simplify the process of migrating Python 2 code that uses the existing +iterator based APIs to Python 3, this PEP proposes the reintroduction +of the Python 2 spelling of the iterator based semantics in Python 3.5, by +restoring the following methods to the builtin ``dict`` API and the +``collections.abc.Mapping`` ABC definition: + + * ``iterkeys()`` + * ``itervalues()`` + * ``iteritems()`` + + +Proposal +======== + +Methods with the following exact semantics will be added to the builtin +``dict`` type and ``collections.abc.Mapping`` ABC:: + + def iterkeys(self): + return iter(self.keys()) + + def itervalues(self): + return iter(self.values()) + + def iteritems(self): + return iter(self.items()) + +These semantics ensure that the methods also work as expected for subclasses +of these base types. + + +Rationale +========= + +Similar in spirit to PEP 414 (which restored explicit Unicode literal +support in Python 3.3), this PEP is aimed primarily at helping users +that currently feel punished for making use of a feature that needed to be +requested explicitly in Python 2, but was effectively made the default +behaviour in Python 3. + +Users of list-based iteration in Python 2 that aren't actually relying on +those semantics get a free memory efficiency improvement when migrating to +Python 3, and face no additional difficulties when migrating via the common +subset of Python 2 and 3. + +By contrast, users that actually want the increased efficiency will have +faced a three phase migration process by the time they have fully migrated +to Python 3: + +* original migration to the iterator based APIs after they were added in + Python 2.2 +* migration to a separate function based API in order to run in the common + subset of Python 2 and 3 +* eventual migration back to unprefixed method APIs when eventually dropping + Python 2.7 support + +The view based APIs that were added to Python 2.7 don't actually help with +the transition process, as they don't exist in Python 3 and hence aren't +part of the common subset of Python 2 and Python 3, and also aren't supported +by most Python 2 mappings (including the collection ABCs). + +This PEP proposes to just eliminate all that annoyance by making the iterator +based APIs work again in Python 3.5+. As with the restoration of Unicode +literals, it does add a bit of additional noise to the definition of Python +3, but it does so while bringing a significant benefit in increasing the size +of the common subset of Python 2 and Python 3 and so simplifying the process +of migrating to Python 3 for affected Python 2 users. + + +Acknowledgements +================ + +Thanks to the folks at the Twisted sprint table at PyCon for a very +vigorous discussion of this idea (and several other topics), and especially +to Hynek Schlawack for acting as a moderator when things got a little too +heated :) + + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sat Apr 19 04:25:59 2014 From: python-checkins at python.org (nick.coghlan) Date: Sat, 19 Apr 2014 04:25:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_469_wording_tweaks?= Message-ID: <3g9dMC0qbDz7Ljh@mail.python.org> http://hg.python.org/peps/rev/c506ce4f1966 changeset: 5464:c506ce4f1966 user: Nick Coghlan date: Fri Apr 18 22:25:50 2014 -0400 summary: PEP 469 wording tweaks files: pep-0469.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0469.txt b/pep-0469.txt --- a/pep-0469.txt +++ b/pep-0469.txt @@ -77,7 +77,7 @@ Python 3, and face no additional difficulties when migrating via the common subset of Python 2 and 3. -By contrast, users that actually want the increased efficiency will have +By contrast, users that actually want the increased efficiency may have faced a three phase migration process by the time they have fully migrated to Python 3: @@ -85,8 +85,8 @@ Python 2.2 * migration to a separate function based API in order to run in the common subset of Python 2 and 3 -* eventual migration back to unprefixed method APIs when eventually dropping - Python 2.7 support +* eventual migration back to unprefixed method APIs when finally dropping + Python 2.7 support at some point in the future The view based APIs that were added to Python 2.7 don't actually help with the transition process, as they don't exist in Python 3 and hence aren't -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sat Apr 19 06:35:41 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 19 Apr 2014 06:35:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_Update_issue_tracker_sect?= =?utf-8?q?ion_with_revised_=22stage=22_and_=22resolution=22_names=2E?= Message-ID: <3g9hDs47GPz7LjS@mail.python.org> http://hg.python.org/devguide/rev/f4679382ab71 changeset: 689:f4679382ab71 user: Ned Deily date: Fri Apr 18 21:35:25 2014 -0700 summary: Update issue tracker section with revised "stage" and "resolution" names. files: triaging.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/triaging.rst b/triaging.rst --- a/triaging.rst +++ b/triaging.rst @@ -60,7 +60,7 @@ A triager performed a patch review and it looks good to them, but a core developer needs to commit the patch (and do a quick once-over to make sure nothing was overlooked). -committed/rejected +resolved The issue is considered closed and dealt with. Components @@ -235,11 +235,11 @@ Duplicate of another issue; should have the Superseder field filled out. fixed A fix for the issue was committed. -invalid +later + Issue is to be worked on at a later date. +not a bug For some reason the issue is invalid (e.g. the perceived problem is not a bug in Python). -later - Issue is to be worked on at a later date. out of date The issue has already been fixed, or the problem doesn't exist anymore for other reasons. -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Sat Apr 19 08:17:48 2014 From: python-checkins at python.org (eric.snow) Date: Sat, 19 Apr 2014 08:17:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMjAw?= =?utf-8?q?=3A_Return_None_from_pkgutil=2Eget=5Floader=28=29_when_=5F=5Fsp?= =?utf-8?q?ec=5F=5F_is_missing=2E?= Message-ID: <3g9kVh4M7Tz7LjN@mail.python.org> http://hg.python.org/cpython/rev/bc4eb1b3db5d changeset: 90409:bc4eb1b3db5d branch: 3.4 parent: 90407:604b74f9a07d user: Eric Snow date: Sat Apr 19 00:13:23 2014 -0600 summary: Issue #21200: Return None from pkgutil.get_loader() when __spec__ is missing. files: Lib/pkgutil.py | 2 ++ Lib/test/test_pkgutil.py | 19 ++++++++++++++++++- Misc/NEWS | 2 ++ 3 files changed, 22 insertions(+), 1 deletions(-) diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -461,6 +461,8 @@ loader = getattr(module, '__loader__', None) if loader is not None: return loader + if getattr(module, '__spec__', None) is None: + return None fullname = module.__name__ else: fullname = module_or_name diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py --- a/Lib/test/test_pkgutil.py +++ b/Lib/test/test_pkgutil.py @@ -1,4 +1,4 @@ -from test.support import run_unittest, unload, check_warnings +from test.support import run_unittest, unload, check_warnings, CleanImport import unittest import sys import importlib @@ -345,6 +345,23 @@ finally: __loader__ = this_loader + def test_get_loader_handles_missing_spec_attribute(self): + name = 'spam' + mod = type(sys)(name) + del mod.__spec__ + with CleanImport(name): + sys.modules[name] = mod + loader = pkgutil.get_loader(name) + self.assertIsNone(loader) + + def test_get_loader_handles_spec_attribute_none(self): + name = 'spam' + mod = type(sys)(name) + mod.__spec__ = None + with CleanImport(name): + sys.modules[name] = mod + loader = pkgutil.get_loader(name) + self.assertIsNone(loader) def test_find_loader_avoids_emulation(self): with check_warnings() as w: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -86,6 +86,8 @@ :func:`tempfile.NamedTemporaryFile`, close the file descriptor if :func:`io.open` fails +- Issue #21200: Return None from pkgutil.get_loader() when __spec__ is missing. + - Issue #21013: Enhance ssl.create_default_context() when used for server side sockets to provide better security by default. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 08:17:49 2014 From: python-checkins at python.org (eric.snow) Date: Sat, 19 Apr 2014 08:17:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgZnJvbSAzLjQu?= Message-ID: <3g9kVj62f8z7Lk6@mail.python.org> http://hg.python.org/cpython/rev/ffb1845e96e6 changeset: 90410:ffb1845e96e6 parent: 90408:c82dcad83438 parent: 90409:bc4eb1b3db5d user: Eric Snow date: Sat Apr 19 00:14:54 2014 -0600 summary: Merge from 3.4. files: Lib/pkgutil.py | 2 ++ Lib/test/test_pkgutil.py | 19 ++++++++++++++++++- Misc/NEWS | 2 ++ 3 files changed, 22 insertions(+), 1 deletions(-) diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -461,6 +461,8 @@ loader = getattr(module, '__loader__', None) if loader is not None: return loader + if getattr(module, '__spec__', None) is None: + return None fullname = module.__name__ else: fullname = module_or_name diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py --- a/Lib/test/test_pkgutil.py +++ b/Lib/test/test_pkgutil.py @@ -1,4 +1,4 @@ -from test.support import run_unittest, unload, check_warnings +from test.support import run_unittest, unload, check_warnings, CleanImport import unittest import sys import importlib @@ -345,6 +345,23 @@ finally: __loader__ = this_loader + def test_get_loader_handles_missing_spec_attribute(self): + name = 'spam' + mod = type(sys)(name) + del mod.__spec__ + with CleanImport(name): + sys.modules[name] = mod + loader = pkgutil.get_loader(name) + self.assertIsNone(loader) + + def test_get_loader_handles_spec_attribute_none(self): + name = 'spam' + mod = type(sys)(name) + mod.__spec__ = None + with CleanImport(name): + sys.modules[name] = mod + loader = pkgutil.get_loader(name) + self.assertIsNone(loader) def test_find_loader_avoids_emulation(self): with check_warnings() as w: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -129,6 +129,8 @@ :func:`tempfile.NamedTemporaryFile`, close the file descriptor if :func:`io.open` fails +- Issue #21200: Return None from pkgutil.get_loader() when __spec__ is missing. + - Issue #21013: Enhance ssl.create_default_context() when used for server side sockets to provide better security by default. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Apr 19 10:03:33 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 19 Apr 2014 10:03:33 +0200 Subject: [Python-checkins] Daily reference leaks (c82dcad83438): sum=3 Message-ID: results for c82dcad83438 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 2, -2] references, sum=0 test_site leaked [0, 2, -2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogHH0sW8', '-x'] From python-checkins at python.org Sat Apr 19 19:06:47 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 19 Apr 2014 19:06:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzkzNjQ6IEltcHJv?= =?utf-8?q?ve_the_text_printed_by_help=28pydoc=29_and_help=28help=29=2E?= Message-ID: <3gB0vW630kz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/16207b8495bf changeset: 90411:16207b8495bf branch: 3.4 parent: 90409:bc4eb1b3db5d user: R David Murray date: Sat Apr 19 12:59:30 2014 -0400 summary: #9364: Improve the text printed by help(pydoc) and help(help). files: Lib/_sitebuiltins.py | 6 +++++- Lib/pydoc.py | 7 ++++--- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/Lib/_sitebuiltins.py b/Lib/_sitebuiltins.py --- a/Lib/_sitebuiltins.py +++ b/Lib/_sitebuiltins.py @@ -87,8 +87,12 @@ class _Helper(object): """Define the builtin 'help'. - This is a wrapper around pydoc.help (with a twist). + This is a wrapper around pydoc.help that provides a helpful message + when 'help' is typed at the Python interactive prompt. + + Calling help() at the Python prompt starts an interactive help session. + Calling help(thing) prints help for the python object 'thing'. """ def __repr__(self): diff --git a/Lib/pydoc.py b/Lib/pydoc.py --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -1,8 +1,9 @@ #!/usr/bin/env python3 """Generate Python documentation in HTML or text for interactive use. -In the Python interpreter, do "from pydoc import help" to provide -help. Calling help(thing) on a Python object documents the object. +At the Python interactive prompt, calling help(thing) on a Python object +documents the object, and calling help() starts up an interactive +help session. Or, at the shell command line outside of Python: @@ -1865,7 +1866,7 @@ def intro(self): self.output.write(''' -Welcome to Python %s! This is the interactive help utility. +Welcome to Python %s's help utility! If this is your first time using Python, you should definitely check out the tutorial on the Internet at http://docs.python.org/%s/tutorial/. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 19:06:49 2014 From: python-checkins at python.org (r.david.murray) Date: Sat, 19 Apr 2014 19:06:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=239364=3A_Improve_the_text_printed_by_help=28p?= =?utf-8?b?eWRvYykgYW5kIGhlbHAoaGVscCku?= Message-ID: <3gB0vY0fpkz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/256c782ab078 changeset: 90412:256c782ab078 parent: 90410:ffb1845e96e6 parent: 90411:16207b8495bf user: R David Murray date: Sat Apr 19 13:01:21 2014 -0400 summary: Merge: #9364: Improve the text printed by help(pydoc) and help(help). files: Lib/_sitebuiltins.py | 6 +++++- Lib/pydoc.py | 7 ++++--- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/Lib/_sitebuiltins.py b/Lib/_sitebuiltins.py --- a/Lib/_sitebuiltins.py +++ b/Lib/_sitebuiltins.py @@ -87,8 +87,12 @@ class _Helper(object): """Define the builtin 'help'. - This is a wrapper around pydoc.help (with a twist). + This is a wrapper around pydoc.help that provides a helpful message + when 'help' is typed at the Python interactive prompt. + + Calling help() at the Python prompt starts an interactive help session. + Calling help(thing) prints help for the python object 'thing'. """ def __repr__(self): diff --git a/Lib/pydoc.py b/Lib/pydoc.py --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -1,8 +1,9 @@ #!/usr/bin/env python3 """Generate Python documentation in HTML or text for interactive use. -In the Python interpreter, do "from pydoc import help" to provide -help. Calling help(thing) on a Python object documents the object. +At the Python interactive prompt, calling help(thing) on a Python object +documents the object, and calling help() starts up an interactive +help session. Or, at the shell command line outside of Python: @@ -1865,7 +1866,7 @@ def intro(self): self.output.write(''' -Welcome to Python %s! This is the interactive help utility. +Welcome to Python %s's help utility! If this is your first time using Python, you should definitely check out the tutorial on the Internet at http://docs.python.org/%s/tutorial/. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 19:23:48 2014 From: python-checkins at python.org (ezio.melotti) Date: Sat, 19 Apr 2014 19:23:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_missing_parenthesis_=28no?= =?utf-8?q?ticed_by_Jurko_Gospodneti=C4=87=29=2E?= Message-ID: <3gB1H83Zjvz7Ljf@mail.python.org> http://hg.python.org/peps/rev/b87e19488fb3 changeset: 5465:b87e19488fb3 user: Ezio Melotti date: Sat Apr 19 20:23:34 2014 +0300 summary: Add missing parenthesis (noticed by Jurko Gospodneti?). files: pep-0453.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-0453.txt b/pep-0453.txt --- a/pep-0453.txt +++ b/pep-0453.txt @@ -389,7 +389,7 @@ the improved distribution capabilities offered by the CPython installers to include private copies of ``pip`` and ``setuptools`` as wheel files (rather than as embedded base64 encoded data), and would not try to -contact PyPI (instead installing directly from the private wheel files. +contact PyPI (instead installing directly from the private wheel files). Rather than including separate code to handle the bootstrapping, the ``ensurepip`` module will manipulate ``sys.path`` appropriately to allow -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sat Apr 19 21:07:34 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sat, 19 Apr 2014 21:07:34 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=237221=3A_remove_re?= =?utf-8?q?dundant_tests_in_test=5Fasyncore=2E__Patch_by_Jessica?= Message-ID: <3gB3Zt0wj4z7Ljf@mail.python.org> http://hg.python.org/cpython/rev/cbe7b5a5a110 changeset: 90413:cbe7b5a5a110 user: Antoine Pitrou date: Sat Apr 19 21:07:16 2014 +0200 summary: Issue #7221: remove redundant tests in test_asyncore. Patch by Jessica McKellar. files: Lib/test/test_asyncore.py | 7 +------ 1 files changed, 1 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_asyncore.py b/Lib/test/test_asyncore.py --- a/Lib/test/test_asyncore.py +++ b/Lib/test/test_asyncore.py @@ -349,9 +349,8 @@ def handle_connect(self): pass + class DispatcherWithSendTests(unittest.TestCase): - usepoll = False - def setUp(self): pass @@ -401,10 +400,6 @@ self.fail("join() timed out") - -class DispatcherWithSendTests_UsePoll(DispatcherWithSendTests): - usepoll = True - @unittest.skipUnless(hasattr(asyncore, 'file_wrapper'), 'asyncore.file_wrapper required') class FileWrapperTest(unittest.TestCase): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 22:27:47 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 19 Apr 2014 22:27:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMzEx?= =?utf-8?q?=3A_Avoid_exception_in_=5Fosx=5Fsupport_with_non-standard_compi?= =?utf-8?q?ler?= Message-ID: <3gB5MR0NKHz7LjN@mail.python.org> http://hg.python.org/cpython/rev/e3217efa6edd changeset: 90414:e3217efa6edd branch: 2.7 parent: 90386:1bc0a8310b9f user: Ned Deily date: Sat Apr 19 13:24:03 2014 -0700 summary: Issue #21311: Avoid exception in _osx_support with non-standard compiler configurations. Patch by John Szakmeister. files: Lib/_osx_support.py | 2 +- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 3 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/_osx_support.py b/Lib/_osx_support.py --- a/Lib/_osx_support.py +++ b/Lib/_osx_support.py @@ -182,7 +182,7 @@ # Compiler is GCC, check if it is LLVM-GCC data = _read_output("'%s' --version" % (cc.replace("'", "'\"'\"'"),)) - if 'llvm-gcc' in data: + if data and 'llvm-gcc' in data: # Found LLVM-GCC, fall back to clang cc = _find_build_tool('clang') diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1280,6 +1280,7 @@ Paul Swartz Thenault Sylvain P?ter Szab? +John Szakmeister Amir Szekely Arfrever Frehtes Taifersar Arahesis Hideaki Takahashi diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -271,6 +271,9 @@ (Original patches by Hirokazu Yamamoto and Amaury Forgeot d'Arc, with suggested wording by David Gutteridge) +- Issue #21311: Avoid exception in _osx_support with non-standard compiler + configurations. Patch by John Szakmeister. + Tools/Demos ----------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 22:27:48 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 19 Apr 2014 22:27:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMzEx?= =?utf-8?q?=3A_Avoid_exception_in_=5Fosx=5Fsupport_with_non-standard_compi?= =?utf-8?q?ler?= Message-ID: <3gB5MS2GxFz7Lk3@mail.python.org> http://hg.python.org/cpython/rev/3d1578c705c9 changeset: 90415:3d1578c705c9 branch: 3.4 parent: 90411:16207b8495bf user: Ned Deily date: Sat Apr 19 13:25:29 2014 -0700 summary: Issue #21311: Avoid exception in _osx_support with non-standard compiler configurations. Patch by John Szakmeister. files: Lib/_osx_support.py | 2 +- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 3 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/_osx_support.py b/Lib/_osx_support.py --- a/Lib/_osx_support.py +++ b/Lib/_osx_support.py @@ -182,7 +182,7 @@ # Compiler is GCC, check if it is LLVM-GCC data = _read_output("'%s' --version" % (cc.replace("'", "'\"'\"'"),)) - if 'llvm-gcc' in data: + if data and 'llvm-gcc' in data: # Found LLVM-GCC, fall back to clang cc = _find_build_tool('clang') diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1281,6 +1281,7 @@ Paul Swartz Thenault Sylvain P?ter Szab? +John Szakmeister Amir Szekely Arfrever Frehtes Taifersar Arahesis Hideaki Takahashi diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -165,6 +165,9 @@ - asyncio: Add gi_{frame,running,code} properties to CoroWrapper (upstream issue #163). +- Issue #21311: Avoid exception in _osx_support with non-standard compiler + configurations. Patch by John Szakmeister. + Extension Modules ----------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 19 22:27:49 2014 From: python-checkins at python.org (ned.deily) Date: Sat, 19 Apr 2014 22:27:49 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321311=3A_merge_with_3=2E4?= Message-ID: <3gB5MT3j5hz7Lk5@mail.python.org> http://hg.python.org/cpython/rev/2a4401109672 changeset: 90416:2a4401109672 parent: 90413:cbe7b5a5a110 parent: 90415:3d1578c705c9 user: Ned Deily date: Sat Apr 19 13:27:06 2014 -0700 summary: Issue #21311: merge with 3.4 files: Lib/_osx_support.py | 2 +- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 3 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/_osx_support.py b/Lib/_osx_support.py --- a/Lib/_osx_support.py +++ b/Lib/_osx_support.py @@ -182,7 +182,7 @@ # Compiler is GCC, check if it is LLVM-GCC data = _read_output("'%s' --version" % (cc.replace("'", "'\"'\"'"),)) - if 'llvm-gcc' in data: + if data and 'llvm-gcc' in data: # Found LLVM-GCC, fall back to clang cc = _find_build_tool('clang') diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1283,6 +1283,7 @@ Paul Swartz Thenault Sylvain P?ter Szab? +John Szakmeister Amir Szekely Arfrever Frehtes Taifersar Arahesis Hideaki Takahashi diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -244,6 +244,9 @@ - asyncio: Add gi_{frame,running,code} properties to CoroWrapper (upstream issue #163). +- Issue #21311: Avoid exception in _osx_support with non-standard compiler + configurations. Patch by John Szakmeister. + Extension Modules ----------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 04:15:34 2014 From: python-checkins at python.org (ned.deily) Date: Sun, 20 Apr 2014 04:15:34 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzExNTcx?= =?utf-8?q?=3A_Ensure_that_the_turtle_window_becomes_the_topmost_window?= Message-ID: <3gBF4k6ggdz7Ljm@mail.python.org> http://hg.python.org/cpython/rev/1f3946b22e64 changeset: 90417:1f3946b22e64 branch: 3.4 parent: 90415:3d1578c705c9 user: Ned Deily date: Sat Apr 19 19:11:14 2014 -0700 summary: Issue #11571: Ensure that the turtle window becomes the topmost window when launched on OS X. (Original patch by Ronald Oussoren) files: Lib/turtle.py | 7 +++++++ Misc/NEWS | 3 +++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/turtle.py b/Lib/turtle.py --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -109,6 +109,7 @@ import math import time import inspect +import sys from os.path import isfile, split, join from copy import deepcopy @@ -992,6 +993,12 @@ self._colormode = _CFG["colormode"] self._keys = [] self.clear() + if sys.platform == 'darwin': + # Force Turtle window to the front on OS X. This is needed because + # the Turtle window will show behind the Terminal window when you + # start the demo from the command line. + cv._rootwindow.call('wm', 'attributes', '.', '-topmost', '1') + cv._rootwindow.call('wm', 'attributes', '.', '-topmost', '0') def clear(self): """Delete all drawings and all turtles from the TurtleScreen. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -168,6 +168,9 @@ - Issue #21311: Avoid exception in _osx_support with non-standard compiler configurations. Patch by John Szakmeister. +- Issue #11571: Ensure that the turtle window becomes the topmost window + when launched on OS X. + Extension Modules ----------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 04:15:36 2014 From: python-checkins at python.org (ned.deily) Date: Sun, 20 Apr 2014 04:15:36 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2311571=3A_merge_with_3=2E4?= Message-ID: <3gBF4m1KPqz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/01228d7b5e01 changeset: 90418:01228d7b5e01 parent: 90416:2a4401109672 parent: 90417:1f3946b22e64 user: Ned Deily date: Sat Apr 19 19:12:31 2014 -0700 summary: Issue #11571: merge with 3.4 files: Lib/turtle.py | 7 +++++++ Misc/NEWS | 3 +++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/turtle.py b/Lib/turtle.py --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -109,6 +109,7 @@ import math import time import inspect +import sys from os.path import isfile, split, join from copy import deepcopy @@ -992,6 +993,12 @@ self._colormode = _CFG["colormode"] self._keys = [] self.clear() + if sys.platform == 'darwin': + # Force Turtle window to the front on OS X. This is needed because + # the Turtle window will show behind the Terminal window when you + # start the demo from the command line. + cv._rootwindow.call('wm', 'attributes', '.', '-topmost', '1') + cv._rootwindow.call('wm', 'attributes', '.', '-topmost', '0') def clear(self): """Delete all drawings and all turtles from the TurtleScreen. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -247,6 +247,9 @@ - Issue #21311: Avoid exception in _osx_support with non-standard compiler configurations. Patch by John Szakmeister. +- Issue #11571: Ensure that the turtle window becomes the topmost window + when launched on OS X. + Extension Modules ----------------- -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 04:15:37 2014 From: python-checkins at python.org (ned.deily) Date: Sun, 20 Apr 2014 04:15:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2310291=3A_Cleanup_?= =?utf-8?q?turtledemo_to_use_docstrings_for_help=2E?= Message-ID: <3gBF4n4F64z7Lk5@mail.python.org> http://hg.python.org/cpython/rev/004fe3449193 changeset: 90419:004fe3449193 user: Ned Deily date: Sat Apr 19 19:14:29 2014 -0700 summary: Issue #10291: Cleanup turtledemo to use docstrings for help. (Original patch by Alexander Belopolsky, refreshed by Jessica McKellar.) files: Lib/turtledemo/__init__.py | 14 ++ Lib/turtledemo/__main__.py | 80 +++++++++++++++- Lib/turtledemo/about_turtle.txt | 76 ---------------- Lib/turtledemo/about_turtledemo.txt | 13 -- Lib/turtledemo/demohelp.txt | 70 -------------- 5 files changed, 87 insertions(+), 166 deletions(-) diff --git a/Lib/turtledemo/__init__.py b/Lib/turtledemo/__init__.py --- a/Lib/turtledemo/__init__.py +++ b/Lib/turtledemo/__init__.py @@ -0,0 +1,14 @@ +""" + -------------------------------------- + About this viewer + -------------------------------------- + + Tiny demo viewer to view turtle graphics example scripts. + + Quickly and dirtyly assembled by Gregor Lingl. + June, 2006 + + For more information see: turtledemo - Help + + Have fun! +""" diff --git a/Lib/turtledemo/__main__.py b/Lib/turtledemo/__main__.py --- a/Lib/turtledemo/__main__.py +++ b/Lib/turtledemo/__main__.py @@ -1,12 +1,81 @@ #!/usr/bin/env python3 + +""" + ---------------------------------------------- + + turtledemo - Help + + ---------------------------------------------- + + This document has two sections: + + (1) How to use the demo viewer + (2) How to add your own demos to the demo repository + + + (1) How to use the demo viewer. + + Select a demoscript from the example menu. + The (syntax coloured) source code appears in the left + source code window. IT CANNOT BE EDITED, but ONLY VIEWED! + + - Press START button to start the demo. + - Stop execution by pressing the STOP button. + - Clear screen by pressing the CLEAR button. + - Restart by pressing the START button again. + + SPECIAL demos are those which run EVENTDRIVEN. + (For example clock.py - or oldTurtleDemo.py which + in the end expects a mouse click.): + + Press START button to start the demo. + + - Until the EVENTLOOP is entered everything works + as in an ordinary demo script. + + - When the EVENTLOOP is entered, you control the + application by using the mouse and/or keys (or it's + controlled by some timer events) + To stop it you can and must press the STOP button. + + While the EVENTLOOP is running, the examples menu is disabled. + + - Only after having pressed the STOP button, you may + restart it or choose another example script. + + * * * * * * * * + In some rare situations there may occur interferences/conflicts + between events concerning the demo script and those concerning the + demo-viewer. (They run in the same process.) Strange behaviour may be + the consequence and in the worst case you must close and restart the + viewer. + * * * * * * * * + + + (2) How to add your own demos to the demo repository + + - place: same directory as turtledemo/__main__.py + + - requirements on source code: + code must contain a main() function which will + be executed by the viewer (see provided example scripts) + main() may return a string which will be displayed + in the Label below the source code window (when execution + has finished.) + + !! For programs, which are EVENT DRIVEN, main must return + !! the string "EVENTLOOP". This informs the viewer, that the + !! script is still running and must be stopped by the user! +""" import sys import os from tkinter import * from idlelib.Percolator import Percolator from idlelib.ColorDelegator import ColorDelegator -from idlelib.textView import view_file # TextViewer +from idlelib.textView import view_text # TextViewer from importlib import reload +from turtledemo import __doc__ as about_turtledemo import turtle import time @@ -28,16 +97,13 @@ entry.endswith(".py") and entry[0] != '_'] def showDemoHelp(): - view_file(demo.root, "Help on turtleDemo", - os.path.join(demo_dir, "demohelp.txt")) + view_text(demo.root, "Help on turtledemo", __doc__) def showAboutDemo(): - view_file(demo.root, "About turtleDemo", - os.path.join(demo_dir, "about_turtledemo.txt")) + view_text(demo.root, "About turtledemo", about_turtledemo) def showAboutTurtle(): - view_file(demo.root, "About the new turtle module.", - os.path.join(demo_dir, "about_turtle.txt")) + view_text(demo.root, "About the turtle module.", turtle.__doc__) class DemoWindow(object): diff --git a/Lib/turtledemo/about_turtle.txt b/Lib/turtledemo/about_turtle.txt deleted file mode 100644 --- a/Lib/turtledemo/about_turtle.txt +++ /dev/null @@ -1,76 +0,0 @@ - -======================================================== - A new turtle module for Python -======================================================== - -Turtle graphics is a popular way for introducing programming to -kids. It was part of the original Logo programming language developed -by Wally Feurzig and Seymour Papert in 1966. - -Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an ``import turtle``, give it -the command turtle.forward(15), and it moves (on-screen!) 15 pixels in -the direction it is facing, drawing a line as it moves. Give it the -command turtle.right(25), and it rotates in-place 25 degrees clockwise. - -By combining together these and similar commands, intricate shapes and -pictures can easily be drawn. - ------ turtle.py - -This module is an extended reimplementation of turtle.py from the -Python standard distribution up to Python 2.5. (See: http:\\www.python.org) - -It tries to keep the merits of turtle.py and to be (nearly) 100% -compatible with it. This means in the first place to enable the -learning programmer to use all the commands, classes and methods -interactively when using the module from within IDLE run with -the -n switch. - -Roughly it has the following features added: - -- Better animation of the turtle movements, especially of turning the - turtle. So the turtles can more easily be used as a visual feedback - instrument by the (beginning) programmer. - -- Different turtle shapes, gif-images as turtle shapes, user defined - and user controllable turtle shapes, among them compound - (multicolored) shapes. Turtle shapes can be stgretched and tilted, which - makes turtles zu very versatile geometrical objects. - -- Fine control over turtle movement and screen updates via delay(), - and enhanced tracer() and speed() methods. - -- Aliases for the most commonly used commands, like fd for forward etc., - following the early Logo traditions. This reduces the boring work of - typing long sequences of commands, which often occur in a natural way - when kids try to program fancy pictures on their first encounter with - turtle graphcis. - -- Turtles now have an undo()-method with configurable undo-buffer. - -- Some simple commands/methods for creating event driven programs - (mouse-, key-, timer-events). Especially useful for programming games. - -- A scrollable Canvas class. The default scrollable Canvas can be - extended interactively as needed while playing around with the turtle(s). - -- A TurtleScreen class with methods controlling background color or - background image, window and canvas size and other properties of the - TurtleScreen. - -- There is a method, setworldcoordinates(), to install a user defined - coordinate-system for the TurtleScreen. - -- The implementation uses a 2-vector class named Vec2D, derived from tuple. - This class is public, so it can be imported by the application programmer, - which makes certain types of computations very natural and compact. - -- Appearance of the TurtleScreen and the Turtles at startup/import can be - configured by means of a turtle.cfg configuration file. - The default configuration mimics the appearance of the old turtle module. - -- If configured appropriately the module reads in docstrings from a docstring - dictionary in some different language, supplied separately and replaces - the english ones by those read in. There is a utility function - write_docstringdict() to write a dictionary with the original (english) - docstrings to disc, so it can serve as a template for translations. diff --git a/Lib/turtledemo/about_turtledemo.txt b/Lib/turtledemo/about_turtledemo.txt deleted file mode 100644 --- a/Lib/turtledemo/about_turtledemo.txt +++ /dev/null @@ -1,13 +0,0 @@ - - -------------------------------------- - About this viewer - -------------------------------------- - - Tiny demo viewer to view turtle graphics example scripts. - - Quickly and dirtyly assembled by Gregor Lingl. - June, 2006 - - For more information see: turtleDemo - Help - - Have fun! diff --git a/Lib/turtledemo/demohelp.txt b/Lib/turtledemo/demohelp.txt deleted file mode 100644 --- a/Lib/turtledemo/demohelp.txt +++ /dev/null @@ -1,70 +0,0 @@ - - - ---------------------------------------------- - - turtleDemo - Help - - ---------------------------------------------- - - This document has two sections: - - (1) How to use the demo viewer - (2) How to add your own demos to the demo repository - - - (1) How to use the demo viewer. - - Select a demoscript from the example menu. - The (syntax coloured) source code appears in the left - source code window. IT CANNOT BE EDITED, but ONLY VIEWED! - - - Press START button to start the demo. - - Stop execution by pressing the STOP button. - - Clear screen by pressing the CLEAR button. - - Restart by pressing the START button again. - - SPECIAL demos are those which run EVENTDRIVEN. - (For example clock.py - or oldTurtleDemo.py which - in the end expects a mouse click.): - - Press START button to start the demo. - - - Until the EVENTLOOP is entered everything works - as in an ordinary demo script. - - - When the EVENTLOOP is entered, you control the - application by using the mouse and/or keys (or it's - controlled by some timer events) - To stop it you can and must press the STOP button. - - While the EVENTLOOP is running, the examples menu is disabled. - - - Only after having pressed the STOP button, you may - restart it or choose another example script. - - * * * * * * * * - In some rare situations there may occur interferences/conflicts - between events concerning the demo script and those concerning the - demo-viewer. (They run in the same process.) Strange behaviour may be - the consequence and in the worst case you must close and restart the - viewer. - * * * * * * * * - - - (2) How to add your own demos to the demo repository - - - place: same directory as turtledemo/__main__.py - - - requirements on source code: - code must contain a main() function which will - be executed by the viewer (see provided example scripts) - main() may return a string which will be displayed - in the Label below the source code window (when execution - has finished.) - - !! For programs, which are EVENT DRIVEN, main must return - !! the string "EVENTLOOP". This informs the viewer, that the - !! script is still running and must be stopped by the user! - - - -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 06:48:41 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 20 Apr 2014 06:48:41 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzEyMjIwOiBpbXBy?= =?utf-8?q?ove_minidom_error_when_URI_contains_spaces=2E?= Message-ID: <3gBJTP3DbMz7LjV@mail.python.org> http://hg.python.org/cpython/rev/13c1c5e3d2ee changeset: 90420:13c1c5e3d2ee branch: 3.4 parent: 90417:1f3946b22e64 user: R David Murray date: Sun Apr 20 00:46:05 2014 -0400 summary: #12220: improve minidom error when URI contains spaces. Fix by 'amathew', test by Marek Stepniowski. files: Lib/test/test_minidom.py | 4 ++++ Lib/xml/dom/expatbuilder.py | 4 +++- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -1518,6 +1518,10 @@ doc2 = parseString(doc.toxml()) self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) + def testExceptionOnSpacesInXMLNSValue(self): + with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + parseString('') + def testDocRemoveChild(self): doc = parse(tstfile) title_tag = doc.documentElement.getElementsByTagName("TITLE")[0] diff --git a/Lib/xml/dom/expatbuilder.py b/Lib/xml/dom/expatbuilder.py --- a/Lib/xml/dom/expatbuilder.py +++ b/Lib/xml/dom/expatbuilder.py @@ -121,10 +121,12 @@ qname = "%s:%s" % (prefix, localname) qname = intern(qname, qname) localname = intern(localname, localname) - else: + elif len(parts) == 2: uri, localname = parts prefix = EMPTY_PREFIX qname = localname = intern(localname, localname) + else: + raise ValueError("Unsupported syntax: spaces in URIs not supported: %r" % name) return intern(uri, uri), localname, prefix, qname diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1255,6 +1255,7 @@ Anthony Starks Oliver Steele Greg Stein +Marek Stepniowski Baruch Sterin Chris Stern Alex Stewart diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #12220: mindom now raises a custom ValueError indicating it doesn't + support spaces in URIs instead of letting a 'split' ValueError bubble up. + - Issue #21239: patch.stopall() didn't work deterministically when the same name was patched more than once. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 06:48:42 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 20 Apr 2014 06:48:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2312220=3A_improve_minidom_error_when_URI_cont?= =?utf-8?q?ains_spaces=2E?= Message-ID: <3gBJTQ55j7z7Ljl@mail.python.org> http://hg.python.org/cpython/rev/3e67d923a0df changeset: 90421:3e67d923a0df parent: 90419:004fe3449193 parent: 90420:13c1c5e3d2ee user: R David Murray date: Sun Apr 20 00:48:25 2014 -0400 summary: Merge: #12220: improve minidom error when URI contains spaces. files: Lib/test/test_minidom.py | 4 ++++ Lib/xml/dom/expatbuilder.py | 4 +++- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -1518,6 +1518,10 @@ doc2 = parseString(doc.toxml()) self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) + def testExceptionOnSpacesInXMLNSValue(self): + with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + parseString('') + def testDocRemoveChild(self): doc = parse(tstfile) title_tag = doc.documentElement.getElementsByTagName("TITLE")[0] diff --git a/Lib/xml/dom/expatbuilder.py b/Lib/xml/dom/expatbuilder.py --- a/Lib/xml/dom/expatbuilder.py +++ b/Lib/xml/dom/expatbuilder.py @@ -121,10 +121,12 @@ qname = "%s:%s" % (prefix, localname) qname = intern(qname, qname) localname = intern(localname, localname) - else: + elif len(parts) == 2: uri, localname = parts prefix = EMPTY_PREFIX qname = localname = intern(localname, localname) + else: + raise ValueError("Unsupported syntax: spaces in URIs not supported: %r" % name) return intern(uri, uri), localname, prefix, qname diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1257,6 +1257,7 @@ Anthony Starks Oliver Steele Greg Stein +Marek Stepniowski Baruch Sterin Chris Stern Alex Stewart diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -54,6 +54,9 @@ Library ------- +- Issue #12220: mindom now raises a custom ValueError indicating it doesn't + support spaces in URIs instead of letting a 'split' ValueError bubble up. + - Issue #21068: The ssl.PROTOCOL* constants are now enum members. - Issue #21262: New method assert_not_called for Mock. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sun Apr 20 09:59:47 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 20 Apr 2014 09:59:47 +0200 Subject: [Python-checkins] Daily reference leaks (2a4401109672): sum=-1 Message-ID: results for 2a4401109672 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog6q3J6R', '-x'] From python-checkins at python.org Sun Apr 20 18:45:09 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 20 Apr 2014 18:45:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogdXJsbGliLnJlc3Bv?= =?utf-8?q?nse_object_to_use_=5FTemporaryFileWrapper_=28and_=5FTemporaryFi?= =?utf-8?q?leCloser=29?= Message-ID: <3gBcN574hLz7Ll1@mail.python.org> http://hg.python.org/cpython/rev/bb71b71322a3 changeset: 90422:bb71b71322a3 branch: 3.4 parent: 90420:13c1c5e3d2ee user: Senthil Kumaran date: Sun Apr 20 09:41:29 2014 -0700 summary: urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) facility. Provides a better way to handle file descriptor close. Address issue #15002 . Patch contributed by Christian Theune. files: Lib/test/test_urllib_response.py | 65 ++++++++++++------- Lib/urllib/response.py | 61 +++++------------ 2 files changed, 60 insertions(+), 66 deletions(-) diff --git a/Lib/test/test_urllib_response.py b/Lib/test/test_urllib_response.py --- a/Lib/test/test_urllib_response.py +++ b/Lib/test/test_urllib_response.py @@ -1,42 +1,59 @@ """Unit tests for code in urllib.response.""" -import test.support +import socket +import tempfile import urllib.response import unittest -class TestFile(object): - - def __init__(self): - self.closed = False - - def read(self, bytes): - pass - - def readline(self): - pass - - def close(self): - self.closed = True - -class Testaddbase(unittest.TestCase): - - # TODO(jhylton): Write tests for other functionality of addbase() +class TestResponse(unittest.TestCase): def setUp(self): - self.fp = TestFile() - self.addbase = urllib.response.addbase(self.fp) + self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.fp = self.sock.makefile('rb') + self.test_headers = {"Host": "www.python.org", + "Connection": "close"} def test_with(self): + addbase = urllib.response.addbase(self.fp) + + self.assertIsInstance(addbase, tempfile._TemporaryFileWrapper) + def f(): - with self.addbase as spam: + with addbase as spam: pass self.assertFalse(self.fp.closed) f() self.assertTrue(self.fp.closed) self.assertRaises(ValueError, f) -def test_main(): - test.support.run_unittest(Testaddbase) + def test_addclosehook(self): + closehook_called = False + + def closehook(): + nonlocal closehook_called + closehook_called = True + + closehook = urllib.response.addclosehook(self.fp, closehook) + closehook.close() + + self.assertTrue(self.fp.closed) + self.assertTrue(closehook_called) + + def test_addinfo(self): + info = urllib.response.addinfo(self.fp, self.test_headers) + self.assertEqual(info.info(), self.test_headers) + + def test_addinfourl(self): + url = "http://www.python.org" + code = 200 + infourl = urllib.response.addinfourl(self.fp, self.test_headers, + url, code) + self.assertEqual(infourl.info(), self.test_headers) + self.assertEqual(infourl.geturl(), url) + self.assertEqual(infourl.getcode(), code) + + def tearDown(self): + self.sock.close() if __name__ == '__main__': - test_main() + unittest.main() diff --git a/Lib/urllib/response.py b/Lib/urllib/response.py --- a/Lib/urllib/response.py +++ b/Lib/urllib/response.py @@ -6,60 +6,39 @@ headers and a geturl() method that returns the url. """ -class addbase(object): - """Base class for addinfo and addclosehook.""" +import tempfile + +__all__ = ['addbase', 'addclosehook', 'addinfo', 'addinfourl'] + + +class addbase(tempfile._TemporaryFileWrapper): + """Base class for addinfo and addclosehook. Is a good idea for garbage collection.""" # XXX Add a method to expose the timeout on the underlying socket? def __init__(self, fp): - # TODO(jhylton): Is there a better way to delegate using io? + super(addbase, self).__init__(fp, '', delete=False) + # Keep reference around as this was part of the original API. self.fp = fp - self.read = self.fp.read - self.readline = self.fp.readline - # TODO(jhylton): Make sure an object with readlines() is also iterable - if hasattr(self.fp, "readlines"): - self.readlines = self.fp.readlines - if hasattr(self.fp, "fileno"): - self.fileno = self.fp.fileno - else: - self.fileno = lambda: None - - def __iter__(self): - # Assigning `__iter__` to the instance doesn't work as intended - # because the iter builtin does something like `cls.__iter__(obj)` - # and thus fails to find the _bound_ method `obj.__iter__`. - # Returning just `self.fp` works for built-in file objects but - # might not work for general file-like objects. - return iter(self.fp) def __repr__(self): return '<%s at %r whose fp = %r>' % (self.__class__.__name__, - id(self), self.fp) - - def close(self): - if self.fp: - self.fp.close() - self.fp = None - self.read = None - self.readline = None - self.readlines = None - self.fileno = None - self.__iter__ = None - self.__next__ = None + id(self), self.file) def __enter__(self): - if self.fp is None: + if self.fp.closed: raise ValueError("I/O operation on closed file") return self def __exit__(self, type, value, traceback): self.close() + class addclosehook(addbase): """Class to add a close hook to an open file.""" def __init__(self, fp, closehook, *hookargs): - addbase.__init__(self, fp) + super(addclosehook, self).__init__(fp) self.closehook = closehook self.hookargs = hookargs @@ -68,30 +47,28 @@ self.closehook(*self.hookargs) self.closehook = None self.hookargs = None - addbase.close(self) + super(addclosehook, self).close() + class addinfo(addbase): """class to add an info() method to an open file.""" def __init__(self, fp, headers): - addbase.__init__(self, fp) + super(addinfo, self).__init__(fp) self.headers = headers def info(self): return self.headers -class addinfourl(addbase): + +class addinfourl(addinfo): """class to add info() and geturl() methods to an open file.""" def __init__(self, fp, headers, url, code=None): - addbase.__init__(self, fp) - self.headers = headers + super(addinfourl, self).__init__(fp, headers) self.url = url self.code = code - def info(self): - return self.headers - def getcode(self): return self.code -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 18:45:11 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 20 Apr 2014 18:45:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_NEWS_entry_for?= =?utf-8?q?_=2315002?= Message-ID: <3gBcN71l97z7Lky@mail.python.org> http://hg.python.org/cpython/rev/72fe23edfec6 changeset: 90423:72fe23edfec6 branch: 3.4 user: Senthil Kumaran date: Sun Apr 20 09:44:11 2014 -0700 summary: NEWS entry for #15002 files: Misc/NEWS | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,10 @@ Library ------- +- Issue #15002: urllib.response object to use _TemporaryFileWrapper (and + _TemporaryFileCloser) facility. Provides a better way to handle file + descriptor close. Patch contributed by Christian Theune. + - Issue #12220: mindom now raises a custom ValueError indicating it doesn't support spaces in URIs instead of letting a 'split' ValueError bubble up. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 20 18:45:12 2014 From: python-checkins at python.org (senthil.kumaran) Date: Sun, 20 Apr 2014 18:45:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40?= Message-ID: <3gBcN84cr7z7LkY@mail.python.org> http://hg.python.org/cpython/rev/8c8315bac6a8 changeset: 90424:8c8315bac6a8 parent: 90421:3e67d923a0df parent: 90423:72fe23edfec6 user: Senthil Kumaran date: Sun Apr 20 09:45:00 2014 -0700 summary: merge 3.4 urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) facility. Provides a better way to handle file descriptor close. Address issue #15002 . Patch contributed by Christian Theune. files: Lib/test/test_urllib_response.py | 65 ++++++++++++------- Lib/urllib/response.py | 61 +++++------------ Misc/NEWS | 4 + 3 files changed, 64 insertions(+), 66 deletions(-) diff --git a/Lib/test/test_urllib_response.py b/Lib/test/test_urllib_response.py --- a/Lib/test/test_urllib_response.py +++ b/Lib/test/test_urllib_response.py @@ -1,42 +1,59 @@ """Unit tests for code in urllib.response.""" -import test.support +import socket +import tempfile import urllib.response import unittest -class TestFile(object): - - def __init__(self): - self.closed = False - - def read(self, bytes): - pass - - def readline(self): - pass - - def close(self): - self.closed = True - -class Testaddbase(unittest.TestCase): - - # TODO(jhylton): Write tests for other functionality of addbase() +class TestResponse(unittest.TestCase): def setUp(self): - self.fp = TestFile() - self.addbase = urllib.response.addbase(self.fp) + self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.fp = self.sock.makefile('rb') + self.test_headers = {"Host": "www.python.org", + "Connection": "close"} def test_with(self): + addbase = urllib.response.addbase(self.fp) + + self.assertIsInstance(addbase, tempfile._TemporaryFileWrapper) + def f(): - with self.addbase as spam: + with addbase as spam: pass self.assertFalse(self.fp.closed) f() self.assertTrue(self.fp.closed) self.assertRaises(ValueError, f) -def test_main(): - test.support.run_unittest(Testaddbase) + def test_addclosehook(self): + closehook_called = False + + def closehook(): + nonlocal closehook_called + closehook_called = True + + closehook = urllib.response.addclosehook(self.fp, closehook) + closehook.close() + + self.assertTrue(self.fp.closed) + self.assertTrue(closehook_called) + + def test_addinfo(self): + info = urllib.response.addinfo(self.fp, self.test_headers) + self.assertEqual(info.info(), self.test_headers) + + def test_addinfourl(self): + url = "http://www.python.org" + code = 200 + infourl = urllib.response.addinfourl(self.fp, self.test_headers, + url, code) + self.assertEqual(infourl.info(), self.test_headers) + self.assertEqual(infourl.geturl(), url) + self.assertEqual(infourl.getcode(), code) + + def tearDown(self): + self.sock.close() if __name__ == '__main__': - test_main() + unittest.main() diff --git a/Lib/urllib/response.py b/Lib/urllib/response.py --- a/Lib/urllib/response.py +++ b/Lib/urllib/response.py @@ -6,60 +6,39 @@ headers and a geturl() method that returns the url. """ -class addbase(object): - """Base class for addinfo and addclosehook.""" +import tempfile + +__all__ = ['addbase', 'addclosehook', 'addinfo', 'addinfourl'] + + +class addbase(tempfile._TemporaryFileWrapper): + """Base class for addinfo and addclosehook. Is a good idea for garbage collection.""" # XXX Add a method to expose the timeout on the underlying socket? def __init__(self, fp): - # TODO(jhylton): Is there a better way to delegate using io? + super(addbase, self).__init__(fp, '', delete=False) + # Keep reference around as this was part of the original API. self.fp = fp - self.read = self.fp.read - self.readline = self.fp.readline - # TODO(jhylton): Make sure an object with readlines() is also iterable - if hasattr(self.fp, "readlines"): - self.readlines = self.fp.readlines - if hasattr(self.fp, "fileno"): - self.fileno = self.fp.fileno - else: - self.fileno = lambda: None - - def __iter__(self): - # Assigning `__iter__` to the instance doesn't work as intended - # because the iter builtin does something like `cls.__iter__(obj)` - # and thus fails to find the _bound_ method `obj.__iter__`. - # Returning just `self.fp` works for built-in file objects but - # might not work for general file-like objects. - return iter(self.fp) def __repr__(self): return '<%s at %r whose fp = %r>' % (self.__class__.__name__, - id(self), self.fp) - - def close(self): - if self.fp: - self.fp.close() - self.fp = None - self.read = None - self.readline = None - self.readlines = None - self.fileno = None - self.__iter__ = None - self.__next__ = None + id(self), self.file) def __enter__(self): - if self.fp is None: + if self.fp.closed: raise ValueError("I/O operation on closed file") return self def __exit__(self, type, value, traceback): self.close() + class addclosehook(addbase): """Class to add a close hook to an open file.""" def __init__(self, fp, closehook, *hookargs): - addbase.__init__(self, fp) + super(addclosehook, self).__init__(fp) self.closehook = closehook self.hookargs = hookargs @@ -68,30 +47,28 @@ self.closehook(*self.hookargs) self.closehook = None self.hookargs = None - addbase.close(self) + super(addclosehook, self).close() + class addinfo(addbase): """class to add an info() method to an open file.""" def __init__(self, fp, headers): - addbase.__init__(self, fp) + super(addinfo, self).__init__(fp) self.headers = headers def info(self): return self.headers -class addinfourl(addbase): + +class addinfourl(addinfo): """class to add info() and geturl() methods to an open file.""" def __init__(self, fp, headers, url, code=None): - addbase.__init__(self, fp) - self.headers = headers + super(addinfourl, self).__init__(fp, headers) self.url = url self.code = code - def info(self): - return self.headers - def getcode(self): return self.code diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -54,6 +54,10 @@ Library ------- +- Issue #15002: urllib.response object to use _TemporaryFileWrapper (and + _TemporaryFileCloser) facility. Provides a better way to handle file + descriptor close. Patch contributed by Christian Theune. + - Issue #12220: mindom now raises a custom ValueError indicating it doesn't support spaces in URIs instead of letting a 'split' ValueError bubble up. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 21 06:20:06 2014 From: python-checkins at python.org (nick.coghlan) Date: Mon, 21 Apr 2014 06:20:06 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_465=3A_updated_=26_withdr?= =?utf-8?q?awn_based_on_feedback?= Message-ID: <3gBvny2dctz7Lk5@mail.python.org> http://hg.python.org/peps/rev/d47a3fc5f95b changeset: 5466:d47a3fc5f95b user: Nick Coghlan date: Mon Apr 21 00:19:58 2014 -0400 summary: PEP 465: updated & withdrawn based on feedback This PEP now reviews exactly what is involved in migrating mapping iteration code to Python 3, as well as to the hybrid 2/3 subset. It is now withdrawn, as I now believe enhancements to migration tools and libraries are a better option than making changes to Python 3.5+ files: pep-0469.txt | 382 ++++++++++++++++++++++++++++++++------ 1 files changed, 317 insertions(+), 65 deletions(-) diff --git a/pep-0469.txt b/pep-0469.txt --- a/pep-0469.txt +++ b/pep-0469.txt @@ -1,14 +1,14 @@ PEP: 469 -Title: Simplified migration of iterator-based mapping code to Python 3 +Title: Migration of dict iteration code to Python 3 Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan -Status: Draft +Status: Withdrawn Type: Standards Track Content-Type: text/x-rst Created: 2014-04-18 Python-Version: 3.5 -Post-History: 2014-04-18 +Post-History: 2014-04-18, 2014-04-21 Abstract @@ -17,88 +17,335 @@ For Python 3, PEP 3106 changed the design of the ``dict`` builtin and the mapping API in general to replace the separate list based and iterator based APIs in Python 2 with a merged, memory efficient set and multiset view -based API. +based API. This new style of dict iteration was also added to the Python 2.7 +``dict`` type as a new set of iteration methods. -This means that Python 3 code always requires an additional qualifier to -reliably reproduce classic Python 2 mapping semantics: +This means that there are now 3 different kinds of dict iteration that may +need to be migrated to Python 3 when an application makes the transition: - * List based (e.g. ``d.keys()``): ``list(d.keys())`` - * Iterator based (e.g. ``d.iterkeys()``): ``iter(d.keys())`` +* Lists as mutable snapshots: ``d.items()`` -> ``list(d.items())`` +* Iterator objects: ``d.iteritems()`` -> ``iter(d.items())`` +* Set based dynamic views: ``d.viewitems()`` -> ``d.items()`` -Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3 -(or the common subset of Python 2 and Python 3) without alteration, but -*all* code using the iterator based API requires modification. Code that -is migrating to the common subset of Python 2 and 3 and needs to retain the -memory efficient implementation that avoids creating an unnecessary list -object must switch away from using a method to instead using a helper -function (such as those provided by the ``six`` module) +There is currently no widely agreed best practice on how to reliably convert +all Python 2 dict iteration code to the common subset of Python 2 and 3, +especially when test coverage of the ported code is limited. This PEP +reviews the various ways the Python 2 iteration APIs may be accessed, and +looks at the available options for migrating that code to Python 3 by way of +the common subset of Python 2.6+ and Python 3.0+. -To simplify the process of migrating Python 2 code that uses the existing -iterator based APIs to Python 3, this PEP proposes the reintroduction -of the Python 2 spelling of the iterator based semantics in Python 3.5, by -restoring the following methods to the builtin ``dict`` API and the -``collections.abc.Mapping`` ABC definition: +The PEP also considers the question of whether or not there are any +additions that may be worth making to Python 3.5 that may ease the +transition process for application code that doesn't need to worry about +supporting earlier versions when eventually making the leap to Python 3. - * ``iterkeys()`` - * ``itervalues()`` - * ``iteritems()`` +PEP Withdrawal +============== -Proposal -======== +In writing the second draft of this PEP, I came to the conclusion that +the readability of hybrid Python 2/3 mapping code can actually be best +enhanced by better helper functions rather than by making changes to +Python 3.5+. The main value I now see in this PEP is as a clear record +of the recommended approaches to migrating mapping iteration code from +Python 2 to Python 3, as well as suggesting ways to keep things readable +and maintainable when writing hybrid code that supports both versions. -Methods with the following exact semantics will be added to the builtin -``dict`` type and ``collections.abc.Mapping`` ABC:: +Notably, I recommend that hybrid code avoid calling mapping iteration +methods directly, and instead rely on builtin functions where possible, +and some additional helper functions for cases that would be a simple +combination of a builtin and a mapping method in pure Python 3 code, but +need to be handled slightly differently to get the exact same semantics in +Python 2. - def iterkeys(self): - return iter(self.keys()) +Static code checkers like pylint could potentially be extended with an +optional warning regarding direct use of the mapping iteration methods in +a hybrid code base. - def itervalues(self): - return iter(self.values()) - def iteritems(self): - return iter(self.items()) +Mapping iteration models +======================== -These semantics ensure that the methods also work as expected for subclasses -of these base types. +Python 2.7 provides three different sets of methods to extract the keys, +values and items from a ``dict`` instance, accounting for 9 out of the +18 public methods of the ``dict`` type. +In Python 3, this has been rationalised to just 3 out of 11 public methods +(as the ``has_key`` method has also been removed). -Rationale -========= -Similar in spirit to PEP 414 (which restored explicit Unicode literal -support in Python 3.3), this PEP is aimed primarily at helping users -that currently feel punished for making use of a feature that needed to be -requested explicitly in Python 2, but was effectively made the default -behaviour in Python 3. +Lists as mutable snapshots +-------------------------- -Users of list-based iteration in Python 2 that aren't actually relying on -those semantics get a free memory efficiency improvement when migrating to -Python 3, and face no additional difficulties when migrating via the common -subset of Python 2 and 3. +This is the oldest of the three styles of dict iteration, and hence the +one implemented by the ``d.keys()``, ``d.values()`` and ``d.items()`` +methods in Python 2. -By contrast, users that actually want the increased efficiency may have -faced a three phase migration process by the time they have fully migrated -to Python 3: +These methods all return lists that are snapshots of the state of the +mapping at the time the method was called. This has a few consequences: -* original migration to the iterator based APIs after they were added in - Python 2.2 -* migration to a separate function based API in order to run in the common - subset of Python 2 and 3 -* eventual migration back to unprefixed method APIs when finally dropping - Python 2.7 support at some point in the future +* the original object can be mutated freely without affecting iteration + over the snapshot +* the snapshot can be modified independently of the original object +* the snapshot consumes memory proportional to the size of the original + mapping -The view based APIs that were added to Python 2.7 don't actually help with -the transition process, as they don't exist in Python 3 and hence aren't -part of the common subset of Python 2 and Python 3, and also aren't supported -by most Python 2 mappings (including the collection ABCs). +The semantic equivalent of these operations in Python 3 are +``list(d.keys())``, ``list(d.values())`` and ``list(d.iteritems())``. -This PEP proposes to just eliminate all that annoyance by making the iterator -based APIs work again in Python 3.5+. As with the restoration of Unicode -literals, it does add a bit of additional noise to the definition of Python -3, but it does so while bringing a significant benefit in increasing the size -of the common subset of Python 2 and Python 3 and so simplifying the process -of migrating to Python 3 for affected Python 2 users. + +Iterator objects +---------------- + +In Python 2.2, ``dict`` objects gained support for the then-new iterator +protocol, allowing direct iteration over the keys stored in the dictionary, +thus avoiding the need to build a list just to iterate over the dictionary +contents one entry at a time. ``iter(d)`` provides direct access to the +iterator object for the keys. + +Python 2 also provides a ``d.iterkeys()`` method that is essentially +synonymous with ``iter(d)``, along with ``d.itervalues()`` and +``d.iteritems()`` methods. + +These iterators provide live views of the underlying object, and hence may +fail if the set of keys in the underlying object is changed during +iteration:: + + >>> d = dict(a=1) + >>> for k in d: + ... del d[k] + ... + Traceback (most recent call last): + File "", line 1, in + RuntimeError: dictionary changed size during iteration + +As iterators, iteration over these objects is also a one-time operation: +once the iterator is exhausted, you have to go back to the original mapping +in order to iterate again. + +In Python 3, direct iteration over mappings works the same way as it does +in Python 2. There are no method based equivalents - the semantic equivalents +of ``d.itervalues()`` and ``d.iteritems()`` in Python 3 are +``iter(d.values())`` and ``iter(d.iteritems())``. + +The ``six`` and ``future.utils`` compatibility modules also both provide +``iterkeys()``, ``itervalues()`` and ``iteritems()`` helper functions that +provide efficient iterator semantics in both Python 2 and 3. + + +Set based dynamic views +----------------------- + +The model that is provided in Python 3 as a method based API is that of set +based dynamic views (technically multisets in the case of the ``values()`` +view). + +In Python 3, the objects returned by ``d.keys()``, ``d.values()`` and +``d. items()`` provide a live view of the current state of +the underlying object, rather than taking a full snapshot of the current +state as they did in Python 2. This change is safe in many circumstances, +but does mean that, as with the direct iteration API, it is necessary to +avoid adding or removing keys during iteration, in order to avoid +encountering the following error:: + + >>> d = dict(a=1) + >>> for k, v in d.items(): + ... del d[k] + ... + Traceback (most recent call last): + File "", line 1, in + RuntimeError: dictionary changed size during iteration + +Unlike the iteration API, these objects are iterables, rather than iterators: +you can iterate over them multiple times, and each time they will iterate +over the entire underlying mapping. + +These semantics are also available in Python 2.7 as the ``d.viewkeys()``, +``d.viewvalues()`` and ```d.viewitems()`` methods. + +The ``future.utils`` compatibility module also provides +``viewkeys()``, ``viewvalues()`` and ``viewitems()`` helper functions +when running on Python 2.7 or Python 3.x. + + +Migrating directly to Python 3 +============================== + +The ``2to3`` migration tool handles direct migrations to Python 3 in +accordance with the semantic equivalents described above: + +* ``d.keys()`` -> ``list(d.keys())`` +* ``d.values()`` -> ``list(d.values())`` +* ``d.items()`` -> ``list(d.items())`` +* ``d.iterkeys()`` -> ``iter(d.keys())`` +* ``d.itervalues()`` -> ``iter(d.values())`` +* ``d.iteritems()`` -> ``iter(d.items())`` +* ``d.viewkeys()`` -> ``d.keys()`` +* ``d.viewvalues()`` -> ``d.values()`` +* ``d.viewitems()`` -> ``d.items()`` + +Rather than 9 distinct mapping methods for iteration, there are now only the +3 view methods, which combine in straightforward ways with the two relevant +builtin functions to cover all of the behaviours that are available as +``dict`` methods in Python 2.7. + +Note that in many cases ``d.keys()`` can be replaced by just ``d``, but the +``2to3`` migration tool doesn't attempt that replacement. + +The ``2to3`` migration tool also *does not* provide any automatic assistance +for migrating references to these objects as bound or unbound methods - it +only automates conversions where the API is called immediately. + + +Migrating to the common subset of Python 2 and 3 +================================================ + +When migrating to the common subset of Python 2 and 3, the above +transformations are not generally appropriate, as they all either result in +the creation of a redundant list in Python 2, have unexpectedly different +semantics in at least some cases, or both. + +Since most code running in the common subset of Python 2 and 3 supports +at least as far back as Python 2.6, the currently recommended approach to +conversion of mapping iteration operation depends on two helper functions +for efficient iteration over mapping values and mapping item tuples: + +* ``d.keys()`` -> ``list(d)`` +* ``d.values()`` -> ``list(itervalues(d))`` +* ``d.items()`` -> ``list(iteritems(d))`` +* ``d.iterkeys()`` -> ``iter(d)`` +* ``d.itervalues()`` -> ``itervalues(d)`` +* ``d.iteritems()`` -> ``iteritems(d)`` + +Both ``six`` and ``future.utils`` provide appropriate definitions of +``itervalues()`` and ``iteritems()`` (along with essentially redundant +definitions of ``iterkeys()``). Creating your own definitions of these +functions in a custom compatibility module is also relatively +straightforward:: + + try: + dict.iteritems + except AttributeError: + # Python 3 + def itervalues(d): + return iter(d.values()) + def iteritems(d): + return iter(d.items()) + else: + # Python 2 + def itervalues(d): + return d.itervalues() + def iteritems(d): + return d.iteritems() + +The greatest loss of readability currently arises when converting code that +actually *needs* the list based snapshots that were the default in Python +2. This readability loss could likely be mitigated by also providing +``listvalues`` and ``listitems`` helper functions, allowing the affected +conversions to be simplified to: + +* ``d.values()`` -> ``listvalues(d)`` +* ``d.items()`` -> ``listitems(d)`` + +The corresponding compatibility function definitions are as straightforward +as their iterator counterparts:: + + try: + dict.iteritems + except AttributeError: + # Python 3 + def listvalues(d): + return list(d.values()) + def listitems(d): + return list(d.items()) + else: + # Python 2 + def listvalues(d): + return d.values() + def listitems(d): + return d.items() + +With that expanded set of compatibility functions, Python 2 code would +then be converted to "idiomatic" hybrid 2/3 code as: + +* ``d.keys()`` -> ``list(d)`` +* ``d.values()`` -> ``listvalues(d)`` +* ``d.items()`` -> ``listitems(d)`` +* ``d.iterkeys()`` -> ``iter(d)`` +* ``d.itervalues()`` -> ``itervalues(d)`` +* ``d.iteritems()`` -> ``iteritems(d)`` + +This compares well for readability with the idiomatic pure Python 3 +code that uses the mapping methods and builtins directly: + +* ``d.keys()`` -> ``list(d)`` +* ``d.values()`` -> ``list(d.values())`` +* ``d.items()`` -> ``list(d.items())`` +* ``d.iterkeys()`` -> ``iter(d)`` +* ``d.itervalues()`` -> ``iter(d.values())`` +* ``d.iteritems()`` -> ``iter(d.items())`` + +It's also notable that when using this approach, hybrid code would *never* +invoke the mapping methods directly: it would always invoke either a +builtin or helper function instead, in order to ensure the exact same +semantics on both Python 2 and 3. + + +Possible changes to Python 3.5+ +=============================== + +The main proposal put forward to potentially aid migration of existing +Python 2 code to Python 3 is the restoration of some or all of the +alternate iteration APIs to the Python 3 mapping API. In particular, +the initial draft of this PEP proposed making the following conversions +possible when migrating to the common subset of Python 2 and Python 3.5+: + +* ``d.keys()`` -> ``list(d)`` +* ``d.values()`` -> ``list(d.itervalues())`` +* ``d.items()`` -> ``list(d.iteritems())`` +* ``d.iterkeys()`` -> ``d.iterkeys()`` +* ``d.itervalues()`` -> ``d.itervalues()`` +* ``d.iteritems()`` -> ``d.iteritems()`` + +Possible mitigations of the additional language complexity in Python 3 +created by restoring these methods included immediately deprecating them, +as well as potentially hiding them from the ``dir()`` function (or perhaps +even defining a way to make ``pydoc`` aware of function deprecations). + +However, in the case where the list output is actually desired, the end +result of that proposal is actually less readable than an appropriately +defined helper function, and the function and method forms of the iterator +versions are pretty much equivalent from a readability perspective. + +So unless I've missed something critical, readily available ``listvalues()`` +and ``listitems()`` helper functions look like they will improve the +readability of hybrid code more than anything we could add back to the +Python 3.5+ mapping API, and won't have any long term impact on the +complexity of Python 3 itself. + + +Discussion +========== + +The fact that 5 years in to the Python 3 migration we still have users +considering the dict API changes a significant barrier to migration suggests +that there are problems with previously recommended approaches. This PEP +attempts to explore those issues and tries to isolate those cases where +previous advice (such as it was) could prove problematic. + +My assessment (largely based on feedback from Twisted devs) is that +problems are most likely to arise when attempting to use ``d.keys()``, +``d.values()``, and ``d.items()`` in hybrid code. While superficially it +seems as though there should be cases where it is safe to ignore the +semantic differences, in practice, the change from "mutable snapshot" to +"dynamic view" is significant enough that it is likely better +to just force the use of either list or iterator semantics for hybrid code, +and leave the use of the view semantics to pure Python 3 code. + +This approach also creates rules that are simple enough and safe enough that +it should be possible to automate them in code modernisation scripts that +target the common subset of Python 2 and Python 3, just as ``2to3`` converts +them automatically when targeting pure Python 3 code. Acknowledgements @@ -109,6 +356,11 @@ to Hynek Schlawack for acting as a moderator when things got a little too heated :) +Thanks also to JP Calderone and Itamar Turner-Trauring for their email +feedback, as well to the participants in the `python-dev review +`__ of +the initial version of the PEP. + Copyright ========= -- Repository URL: http://hg.python.org/peps From solipsis at pitrou.net Mon Apr 21 09:58:27 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 21 Apr 2014 09:58:27 +0200 Subject: [Python-checkins] Daily reference leaks (8c8315bac6a8): sum=-1 Message-ID: results for 8c8315bac6a8 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, -2, 0] references, sum=-2 test_site leaked [0, -2, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogYvgWZy', '-x'] From python-checkins at python.org Tue Apr 22 03:04:59 2014 From: python-checkins at python.org (nick.coghlan) Date: Tue, 22 Apr 2014 03:04:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_469=3A_also_cover_reverse?= =?utf-8?q?_migrations_from_3_-=3E_2/3?= Message-ID: <3gCRQM1fhYz7LkP@mail.python.org> http://hg.python.org/peps/rev/da9254eba5dd changeset: 5467:da9254eba5dd user: Nick Coghlan date: Mon Apr 21 21:04:52 2014 -0400 summary: PEP 469: also cover reverse migrations from 3 -> 2/3 files: pep-0469.txt | 34 ++++++++++++++++++++++++++++++++++ 1 files changed, 34 insertions(+), 0 deletions(-) diff --git a/pep-0469.txt b/pep-0469.txt --- a/pep-0469.txt +++ b/pep-0469.txt @@ -291,6 +291,40 @@ semantics on both Python 2 and 3. +Migrating from Python 3 to the common subset with Python 2.7 +============================================================ + +While the majority of migrations are currently from Python 2 either directly +to Python 3 or to the common subset of Python 2 and Python 3, there are also +some migrations of newer projects that start in Python 3 and then later +add Python 2 support, either due to user demand, or to gain access to +Python 2 libraries that are not yet available in Python 3 (and porting them +to Python 3 or creating a Python 3 compatible replacement is not a trivial +exercise). + +In these cases, Python 2.7 compatibility is often sufficient, and the 2.7+ +only view based helper functions provided by ``future.utils`` allow the bare +accesses to the Python 3 mapping view methods to be replaced with code that +is compatible with both Python 2.7 and Python 3 (note, this is the only +migration chart in the PEP that has Python 3 code on the left of the +conversion): + +* ``d.keys()`` -> ``viewkeys(d)`` +* ``d.values()`` -> ``viewvalues(d)`` +* ``d.items()`` -> ``viewitems(d)`` +* ``list(d.keys())`` -> ``list(d)`` +* ``list(d.values())`` -> ``listvalues(d)`` +* ``list(d.items())`` -> ``listitems(d)`` +* ``iter(d.keys())`` -> ``iter(d)`` +* ``iter(d.values())`` -> ``itervalues(d)`` +* ``iter(d.items())`` -> ``iteritems(d)`` + +As with migrations from Python 2 to the common subset, note that the hybrid +code ends up never invoking the mapping methods directly - it only calls +builtins and helper methods, with the latter addressing the semantic +differences between Python 2 and Python 3. + + Possible changes to Python 3.5+ =============================== -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Tue Apr 22 07:12:43 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Apr 2014 07:12:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMjEyODQ6?= =?utf-8?q?_Idle=3A_make_test=5Fformatparagraph_pass_even_when_a_user_chan?= =?utf-8?q?ges_the?= Message-ID: <3gCXwC6XGZz7LjY@mail.python.org> http://hg.python.org/cpython/rev/4ff2c0a637cf changeset: 90425:4ff2c0a637cf branch: 2.7 parent: 90414:e3217efa6edd user: Terry Jan Reedy date: Tue Apr 22 01:10:57 2014 -0400 summary: Issue 21284: Idle: make test_formatparagraph pass even when a user changes the reformat width in the configuration menu. files: Lib/idlelib/FormatParagraph.py | 13 ++++++--- Lib/idlelib/idle_test/test_formatparagraph.py | 10 +++--- Misc/NEWS | 2 + 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/Lib/idlelib/FormatParagraph.py b/Lib/idlelib/FormatParagraph.py --- a/Lib/idlelib/FormatParagraph.py +++ b/Lib/idlelib/FormatParagraph.py @@ -32,7 +32,7 @@ def close(self): self.editwin = None - def format_paragraph_event(self, event): + def format_paragraph_event(self, event, limit=None): """Formats paragraph to a max width specified in idleConf. If text is selected, format_paragraph_event will start breaking lines @@ -41,9 +41,12 @@ If no text is selected, format_paragraph_event uses the current cursor location to determine the paragraph (lines of text surrounded by blank lines) and formats it. + + The length limit parameter is for testing with a known value. """ - maxformatwidth = idleConf.GetOption( - 'main', 'FormatParagraph', 'paragraph', type='int') + if limit == None: + limit = idleConf.GetOption( + 'main', 'FormatParagraph', 'paragraph', type='int') text = self.editwin.text first, last = self.editwin.get_selection_indices() if first and last: @@ -53,9 +56,9 @@ first, last, comment_header, data = \ find_paragraph(text, text.index("insert")) if comment_header: - newdata = reformat_comment(data, maxformatwidth, comment_header) + newdata = reformat_comment(data, limit, comment_header) else: - newdata = reformat_paragraph(data, maxformatwidth) + newdata = reformat_paragraph(data, limit) text.tag_remove("sel", "1.0", "end") if newdata != data: diff --git a/Lib/idlelib/idle_test/test_formatparagraph.py b/Lib/idlelib/idle_test/test_formatparagraph.py --- a/Lib/idlelib/idle_test/test_formatparagraph.py +++ b/Lib/idlelib/idle_test/test_formatparagraph.py @@ -293,7 +293,7 @@ # Set cursor ('insert' mark) to '1.0', within text. text.insert('1.0', self.test_string) text.mark_set('insert', '1.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') # find function includes \n expected = ( @@ -305,7 +305,7 @@ # Select from 1.11 to line end. text.insert('1.0', self.test_string) text.tag_add('sel', '1.11', '1.end') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') # selection excludes \n expected = ( @@ -319,7 +319,7 @@ # Select 2 long lines. text.insert('1.0', self.multiline_test_string) text.tag_add('sel', '2.0', '4.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('2.0', 'insert') expected = ( " The second line's length is way over the max width. It goes on and\n" @@ -334,7 +334,7 @@ # Set cursor ('insert') to '1.0', within block. text.insert('1.0', self.multiline_test_comment) - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') expected = ( "# The first line is under the max width. The second line's length is\n" @@ -348,7 +348,7 @@ # Select line 2, verify line 1 unaffected. text.insert('1.0', self.multiline_test_comment) text.tag_add('sel', '2.0', '3.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') expected = ( "# The first line is under the max width.\n" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -289,6 +289,8 @@ IDLE ---- +- Issue 21284: Paragraph reformat test passes after user changes reformat width. + - Issue #20406: Use Python application icons for Idle window title bars. Patch mostly by Serhiy Storchaka. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 22 07:12:45 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Apr 2014 07:12:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgMjEyODQ6?= =?utf-8?q?_Idle=3A_make_test=5Fformatparagraph_pass_even_when_a_user_chan?= =?utf-8?q?ges_the?= Message-ID: <3gCXwF2pLJz7Lk0@mail.python.org> http://hg.python.org/cpython/rev/fe067339af80 changeset: 90426:fe067339af80 branch: 3.4 parent: 90423:72fe23edfec6 user: Terry Jan Reedy date: Tue Apr 22 01:11:03 2014 -0400 summary: Issue 21284: Idle: make test_formatparagraph pass even when a user changes the reformat width in the configuration menu. files: Lib/idlelib/FormatParagraph.py | 13 ++++++--- Lib/idlelib/idle_test/test_formatparagraph.py | 10 +++--- Misc/NEWS | 2 + 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/Lib/idlelib/FormatParagraph.py b/Lib/idlelib/FormatParagraph.py --- a/Lib/idlelib/FormatParagraph.py +++ b/Lib/idlelib/FormatParagraph.py @@ -32,7 +32,7 @@ def close(self): self.editwin = None - def format_paragraph_event(self, event): + def format_paragraph_event(self, event, limit=None): """Formats paragraph to a max width specified in idleConf. If text is selected, format_paragraph_event will start breaking lines @@ -41,9 +41,12 @@ If no text is selected, format_paragraph_event uses the current cursor location to determine the paragraph (lines of text surrounded by blank lines) and formats it. + + The length limit parameter is for testing with a known value. """ - maxformatwidth = idleConf.GetOption( - 'main', 'FormatParagraph', 'paragraph', type='int') + if limit == None: + limit = idleConf.GetOption( + 'main', 'FormatParagraph', 'paragraph', type='int') text = self.editwin.text first, last = self.editwin.get_selection_indices() if first and last: @@ -53,9 +56,9 @@ first, last, comment_header, data = \ find_paragraph(text, text.index("insert")) if comment_header: - newdata = reformat_comment(data, maxformatwidth, comment_header) + newdata = reformat_comment(data, limit, comment_header) else: - newdata = reformat_paragraph(data, maxformatwidth) + newdata = reformat_paragraph(data, limit) text.tag_remove("sel", "1.0", "end") if newdata != data: diff --git a/Lib/idlelib/idle_test/test_formatparagraph.py b/Lib/idlelib/idle_test/test_formatparagraph.py --- a/Lib/idlelib/idle_test/test_formatparagraph.py +++ b/Lib/idlelib/idle_test/test_formatparagraph.py @@ -293,7 +293,7 @@ # Set cursor ('insert' mark) to '1.0', within text. text.insert('1.0', self.test_string) text.mark_set('insert', '1.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') # find function includes \n expected = ( @@ -305,7 +305,7 @@ # Select from 1.11 to line end. text.insert('1.0', self.test_string) text.tag_add('sel', '1.11', '1.end') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') # selection excludes \n expected = ( @@ -319,7 +319,7 @@ # Select 2 long lines. text.insert('1.0', self.multiline_test_string) text.tag_add('sel', '2.0', '4.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('2.0', 'insert') expected = ( " The second line's length is way over the max width. It goes on and\n" @@ -334,7 +334,7 @@ # Set cursor ('insert') to '1.0', within block. text.insert('1.0', self.multiline_test_comment) - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') expected = ( "# The first line is under the max width. The second line's length is\n" @@ -348,7 +348,7 @@ # Select line 2, verify line 1 unaffected. text.insert('1.0', self.multiline_test_comment) text.tag_add('sel', '2.0', '3.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') expected = ( "# The first line is under the max width.\n" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -186,6 +186,8 @@ IDLE ---- +- Issue 21284: Paragraph reformat test passes after user changes reformat width. + - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 22 07:12:46 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Apr 2014 07:12:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3gCXwG5mT2z7LlX@mail.python.org> http://hg.python.org/cpython/rev/db4367f73b6c changeset: 90427:db4367f73b6c parent: 90424:8c8315bac6a8 parent: 90426:fe067339af80 user: Terry Jan Reedy date: Tue Apr 22 01:11:49 2014 -0400 summary: Merge with 3.4 files: Lib/idlelib/FormatParagraph.py | 13 ++++++--- Lib/idlelib/idle_test/test_formatparagraph.py | 10 +++--- Misc/NEWS | 2 + 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/Lib/idlelib/FormatParagraph.py b/Lib/idlelib/FormatParagraph.py --- a/Lib/idlelib/FormatParagraph.py +++ b/Lib/idlelib/FormatParagraph.py @@ -32,7 +32,7 @@ def close(self): self.editwin = None - def format_paragraph_event(self, event): + def format_paragraph_event(self, event, limit=None): """Formats paragraph to a max width specified in idleConf. If text is selected, format_paragraph_event will start breaking lines @@ -41,9 +41,12 @@ If no text is selected, format_paragraph_event uses the current cursor location to determine the paragraph (lines of text surrounded by blank lines) and formats it. + + The length limit parameter is for testing with a known value. """ - maxformatwidth = idleConf.GetOption( - 'main', 'FormatParagraph', 'paragraph', type='int') + if limit == None: + limit = idleConf.GetOption( + 'main', 'FormatParagraph', 'paragraph', type='int') text = self.editwin.text first, last = self.editwin.get_selection_indices() if first and last: @@ -53,9 +56,9 @@ first, last, comment_header, data = \ find_paragraph(text, text.index("insert")) if comment_header: - newdata = reformat_comment(data, maxformatwidth, comment_header) + newdata = reformat_comment(data, limit, comment_header) else: - newdata = reformat_paragraph(data, maxformatwidth) + newdata = reformat_paragraph(data, limit) text.tag_remove("sel", "1.0", "end") if newdata != data: diff --git a/Lib/idlelib/idle_test/test_formatparagraph.py b/Lib/idlelib/idle_test/test_formatparagraph.py --- a/Lib/idlelib/idle_test/test_formatparagraph.py +++ b/Lib/idlelib/idle_test/test_formatparagraph.py @@ -293,7 +293,7 @@ # Set cursor ('insert' mark) to '1.0', within text. text.insert('1.0', self.test_string) text.mark_set('insert', '1.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') # find function includes \n expected = ( @@ -305,7 +305,7 @@ # Select from 1.11 to line end. text.insert('1.0', self.test_string) text.tag_add('sel', '1.11', '1.end') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') # selection excludes \n expected = ( @@ -319,7 +319,7 @@ # Select 2 long lines. text.insert('1.0', self.multiline_test_string) text.tag_add('sel', '2.0', '4.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('2.0', 'insert') expected = ( " The second line's length is way over the max width. It goes on and\n" @@ -334,7 +334,7 @@ # Set cursor ('insert') to '1.0', within block. text.insert('1.0', self.multiline_test_comment) - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') expected = ( "# The first line is under the max width. The second line's length is\n" @@ -348,7 +348,7 @@ # Select line 2, verify line 1 unaffected. text.insert('1.0', self.multiline_test_comment) text.tag_add('sel', '2.0', '3.0') - self.formatter('ParameterDoesNothing') + self.formatter('ParameterDoesNothing', limit=70) result = text.get('1.0', 'insert') expected = ( "# The first line is under the max width.\n" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -265,6 +265,8 @@ IDLE ---- +- Issue 21284: Paragraph reformat test passes after user changes reformat width. + - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 22 07:28:10 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Apr 2014 07:28:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMTM4?= =?utf-8?q?=3A_Change_default_reformat_paragraph_width_to_PEP_8=27s_72=2E?= Message-ID: <3gCYG21Mqjz7LjY@mail.python.org> http://hg.python.org/cpython/rev/374746c5dedc changeset: 90428:374746c5dedc branch: 2.7 parent: 90425:4ff2c0a637cf user: Terry Jan Reedy date: Tue Apr 22 01:26:35 2014 -0400 summary: Issue #21138: Change default reformat paragraph width to PEP 8's 72. files: Lib/idlelib/config-main.def | 2 +- Misc/NEWS | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def --- a/Lib/idlelib/config-main.def +++ b/Lib/idlelib/config-main.def @@ -59,7 +59,7 @@ encoding= none [FormatParagraph] -paragraph=70 +paragraph=72 [Indent] use-spaces= 1 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -289,7 +289,9 @@ IDLE ---- -- Issue 21284: Paragraph reformat test passes after user changes reformat width. +- Issue #21139: Change default paragraph width to 72, the PEP 8 recommendation. + +- Issue #21284: Paragraph reformat test passes after user changes reformat width. - Issue #20406: Use Python application icons for Idle window title bars. Patch mostly by Serhiy Storchaka. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 22 07:28:11 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Apr 2014 07:28:11 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTM4?= =?utf-8?q?=3A_Change_default_reformat_paragraph_width_to_PEP_8=27s_72=2E?= Message-ID: <3gCYG3334cz7LkP@mail.python.org> http://hg.python.org/cpython/rev/dd24099c0cf6 changeset: 90429:dd24099c0cf6 branch: 3.4 parent: 90426:fe067339af80 user: Terry Jan Reedy date: Tue Apr 22 01:26:41 2014 -0400 summary: Issue #21138: Change default reformat paragraph width to PEP 8's 72. files: Lib/idlelib/config-main.def | 2 +- Misc/NEWS | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def --- a/Lib/idlelib/config-main.def +++ b/Lib/idlelib/config-main.def @@ -59,7 +59,7 @@ encoding= none [FormatParagraph] -paragraph=70 +paragraph=72 [Indent] use-spaces= 1 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -186,7 +186,9 @@ IDLE ---- -- Issue 21284: Paragraph reformat test passes after user changes reformat width. +- Issue #21139: Change default paragraph width to 72, the PEP 8 recommendation. + +- Issue #21284: Paragraph reformat test passes after user changes reformat width. - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 22 07:28:12 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 22 Apr 2014 07:28:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3gCYG44y9qz7LlK@mail.python.org> http://hg.python.org/cpython/rev/a15bc8c12370 changeset: 90430:a15bc8c12370 parent: 90427:db4367f73b6c parent: 90429:dd24099c0cf6 user: Terry Jan Reedy date: Tue Apr 22 01:27:06 2014 -0400 summary: Merge with 3.4 files: Lib/idlelib/config-main.def | 2 +- Misc/NEWS | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/idlelib/config-main.def b/Lib/idlelib/config-main.def --- a/Lib/idlelib/config-main.def +++ b/Lib/idlelib/config-main.def @@ -59,7 +59,7 @@ encoding= none [FormatParagraph] -paragraph=70 +paragraph=72 [Indent] use-spaces= 1 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -265,7 +265,9 @@ IDLE ---- -- Issue 21284: Paragraph reformat test passes after user changes reformat width. +- Issue #21139: Change default paragraph width to 72, the PEP 8 recommendation. + +- Issue #21284: Paragraph reformat test passes after user changes reformat width. - Issue #17654: Ensure IDLE menus are customized properly on OS X for non-framework builds and for all variants of Tk. -- Repository URL: http://hg.python.org/cpython From tjreedy at udel.edu Tue Apr 22 07:34:51 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 22 Apr 2014 01:34:51 -0400 Subject: [Python-checkins] cpython (2.7): Issue #21138: Change default reformat paragraph width to PEP 8's 72. In-Reply-To: <3gCYG21Mqjz7LjY@mail.python.org> References: <3gCYG21Mqjz7LjY@mail.python.org> Message-ID: <5355FF7B.8080702@udel.edu> On 4/22/2014 1:28 AM, terry.reedy wrote: > http://hg.python.org/cpython/rev/374746c5dedc > changeset: 90428:374746c5dedc > branch: 2.7 > parent: 90425:4ff2c0a637cf > user: Terry Jan Reedy > date: Tue Apr 22 01:26:35 2014 -0400 > summary: > Issue #21138: Change default reformat paragraph width to PEP 8's 72. I already noticed the typo: #21139, and unlinked msg from 21138. From solipsis at pitrou.net Tue Apr 22 10:03:27 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 22 Apr 2014 10:03:27 +0200 Subject: [Python-checkins] Daily reference leaks (8c8315bac6a8): sum=1 Message-ID: results for 8c8315bac6a8 on branch "default" -------------------------------------------- test_collections leaked [-4, 0, 0] references, sum=-4 test_collections leaked [-2, 0, 0] memory blocks, sum=-2 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [2, -2, 2] references, sum=2 test_site leaked [2, -2, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogNHROb1', '-x'] From python-checkins at python.org Tue Apr 22 22:36:50 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 22 Apr 2014 22:36:50 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMzAz?= =?utf-8?q?=2C_=2320565=3A_Updated_the_version_of_Tcl/Tk_used_on_Windows?= Message-ID: <3gCxQV3wZqz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/2b8d9276ad5b changeset: 90431:2b8d9276ad5b branch: 2.7 parent: 90428:374746c5dedc user: Zachary Ware date: Tue Apr 22 15:36:11 2014 -0500 summary: Issue #21303, #20565: Updated the version of Tcl/Tk used on Windows from 8.5.2 to 8.5.15. files: Misc/NEWS | 6 ++++++ Tools/buildbot/external-amd64.bat | 10 +++++----- Tools/buildbot/external-common.bat | 14 +++++++++----- Tools/buildbot/external.bat | 12 ++++++------ 4 files changed, 26 insertions(+), 16 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -390,6 +390,12 @@ - Issue #19795: Improved markup of True/False constants. +Windows +------- + +- Issue #21303, #20565: Updated the version of Tcl/Tk included in the + installer from 8.5.2 to 8.5.15. + Whats' New in Python 2.7.6? =========================== diff --git a/Tools/buildbot/external-amd64.bat b/Tools/buildbot/external-amd64.bat --- a/Tools/buildbot/external-amd64.bat +++ b/Tools/buildbot/external-amd64.bat @@ -5,16 +5,16 @@ call "%VS90COMNTOOLS%\..\..\VC\vcvarsall.bat" x86_amd64 if not exist tcltk64\bin\tcl85g.dll ( - cd tcl-8.5.2.1\win + cd tcl-8.5.15.0\win nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 clean all nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 install cd ..\.. ) if not exist tcltk64\bin\tk85g.dll ( - cd tk-8.5.2.0\win - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.2.1 clean - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.2.1 all - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.2.1 install + cd tk-8.5.15.0\win + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.15.0 clean + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.15.0 all + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 MACHINE=AMD64 INSTALLDIR=..\..\tcltk64 TCLDIR=..\..\tcl-8.5.15.0 install cd ..\.. ) diff --git a/Tools/buildbot/external-common.bat b/Tools/buildbot/external-common.bat --- a/Tools/buildbot/external-common.bat +++ b/Tools/buildbot/external-common.bat @@ -10,13 +10,17 @@ @rem if exist tcl8.4.12 rd /s/q tcl8.4.12 @rem if exist tcl8.4.16 rd /s/q tcl8.4.16 @rem if exist tcl-8.4.18.1 rd /s/q tcl-8.4.18.1 + at rem if exist tcl-8.5.2.1 rd /s/q tcl-8.5.2.1 + at rem if exist tcl-8.5.15.0 rd /s/q tcl-8.5.15.0 @rem if exist tk8.4.12 rd /s/q tk8.4.12 @rem if exist tk8.4.16 rd /s/q tk8.4.16 @rem if exist tk-8.4.18.1 rd /s/q tk-8.4.18.1 + at rem if exist tk-8.5.2.0 rd /s/q tk-8.5.2.0 + at rem if exist tk-8.5.15.0 rd /s/q tk-8.5.15.0 @rem if exist db-4.4.20 rd /s/q db-4.4.20 @rem if exist db-4.7.25.0 rd /s/q db-4.7.25.0 @rem if exist openssl-0.9.8y rd /s/q openssl-0.9.8y - at rem if exist sqlite-3.6.21 rd /s/q sqlite-3.6.21 + at rem if exist sqlite-3.6.21 rd /s/q sqlite-3.6.21 @rem bzip if not exist bzip2-1.0.6 ( @@ -32,11 +36,11 @@ if not exist openssl-0.9.8y svn export http://svn.python.org/projects/external/openssl-0.9.8y @rem tcl/tk -if not exist tcl-8.5.2.1 ( - rd /s/q tcltk tcltk64 - svn export http://svn.python.org/projects/external/tcl-8.5.2.1 +if not exist tcl-8.5.15.0 ( + rd /s/q tcltk tcltk64 tcl-8.5.2.1 tk-8.5.2.0 + svn export http://svn.python.org/projects/external/tcl-8.5.15.0 ) -if not exist tk-8.5.2.0 svn export http://svn.python.org/projects/external/tk-8.5.2.0 +if not exist tk-8.5.15.0 svn export http://svn.python.org/projects/external/tk-8.5.15.0 @rem sqlite3 if not exist sqlite-3.6.21 ( diff --git a/Tools/buildbot/external.bat b/Tools/buildbot/external.bat --- a/Tools/buildbot/external.bat +++ b/Tools/buildbot/external.bat @@ -6,16 +6,16 @@ if not exist tcltk\bin\tcl85g.dll ( @rem all and install need to be separate invocations, otherwise nmakehlp is not found on install - cd tcl-8.5.2.1\win - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 DEBUG=1 INSTALLDIR=..\..\tcltk clean all + cd tcl-8.5.15.0\win + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 DEBUG=1 INSTALLDIR=..\..\tcltk clean all nmake -f makefile.vc DEBUG=1 INSTALLDIR=..\..\tcltk install cd ..\.. ) if not exist tcltk\bin\tk85g.dll ( - cd tk-8.5.2.0\win - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.2.1 clean - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.2.1 all - nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.2.1 install + cd tk-8.5.15.0\win + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.15.0 clean + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.15.0 all + nmake -f makefile.vc COMPILERFLAGS=-DWINVER=0x0500 OPTS=noxp DEBUG=1 INSTALLDIR=..\..\tcltk TCLDIR=..\..\tcl-8.5.15.0 install cd ..\.. ) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 00:34:58 2014 From: python-checkins at python.org (antoine.pitrou) Date: Wed, 23 Apr 2014 00:34:58 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMTI3?= =?utf-8?q?=3A_Path_objects_can_now_be_instantiated_from_str_subclass_inst?= =?utf-8?q?ances?= Message-ID: <3gD02p3HFwz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/c24cbd9bd63b changeset: 90432:c24cbd9bd63b branch: 3.4 parent: 90429:dd24099c0cf6 user: Antoine Pitrou date: Wed Apr 23 00:34:15 2014 +0200 summary: Issue #21127: Path objects can now be instantiated from str subclass instances (such as numpy.str_). Thanks to Antony Lee for the report and preliminary patch. files: Lib/pathlib.py | 4 +- Lib/test/test_pathlib.py | 30 ++++++++++++++++++++++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 ++ 4 files changed, 36 insertions(+), 2 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -574,8 +574,8 @@ if isinstance(a, PurePath): parts += a._parts elif isinstance(a, str): - # Assuming a str - parts.append(a) + # Force-cast str subclasses to str (issue #21127) + parts.append(str(a)) else: raise TypeError( "argument should be a path or str object, not %r" diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -197,6 +197,25 @@ self.assertEqual(P(P('a'), 'b'), P('a/b')) self.assertEqual(P(P('a'), P('b')), P('a/b')) + def _check_str_subclass(self, *args): + # Issue #21127: it should be possible to construct a PurePath object + # from an str subclass instance, and it then gets converted to + # a pure str object. + class StrSubclass(str): + pass + P = self.cls + p = P(*(StrSubclass(x) for x in args)) + self.assertEqual(p, P(*args)) + for part in p.parts: + self.assertIs(type(part), str) + + def test_str_subclass_common(self): + self._check_str_subclass('') + self._check_str_subclass('.') + self._check_str_subclass('a') + self._check_str_subclass('a/b.txt') + self._check_str_subclass('/a/b.txt') + def test_join_common(self): P = self.cls p = P('a/b') @@ -690,6 +709,17 @@ p = self.cls('//a/b/c/d') self.assertEqual(str(p), '\\\\a\\b\\c\\d') + def test_str_subclass(self): + self._check_str_subclass('c:') + self._check_str_subclass('c:a') + self._check_str_subclass('c:a\\b.txt') + self._check_str_subclass('c:\\') + self._check_str_subclass('c:\\a') + self._check_str_subclass('c:\\a\\b.txt') + self._check_str_subclass('\\\\some\\share') + self._check_str_subclass('\\\\some\\share\\a') + self._check_str_subclass('\\\\some\\share\\a\\b.txt') + def test_eq(self): P = self.cls self.assertEqual(P('c:a/b'), P('c:a/b')) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -743,6 +743,7 @@ Chris Lawrence Brian Leair Mathieu Leduc-Hamel +Antony Lee Christopher Lee Inyeol Lee James Lee diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #21127: Path objects can now be instantiated from str subclass + instances (such as numpy.str_). + - Issue #15002: urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) facility. Provides a better way to handle file descriptor close. Patch contributed by Christian Theune. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 00:34:59 2014 From: python-checkins at python.org (antoine.pitrou) Date: Wed, 23 Apr 2014 00:34:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321127=3A_Path_objects_can_now_be_instantiated_f?= =?utf-8?q?rom_str_subclass_instances?= Message-ID: <3gD02q58YPz7Ljx@mail.python.org> http://hg.python.org/cpython/rev/aad6d6b819ed changeset: 90433:aad6d6b819ed parent: 90430:a15bc8c12370 parent: 90432:c24cbd9bd63b user: Antoine Pitrou date: Wed Apr 23 00:34:51 2014 +0200 summary: Issue #21127: Path objects can now be instantiated from str subclass instances (such as numpy.str_). Thanks to Antony Lee for the report and preliminary patch. files: Lib/pathlib.py | 4 +- Lib/test/test_pathlib.py | 30 ++++++++++++++++++++++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 ++ 4 files changed, 36 insertions(+), 2 deletions(-) diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -574,8 +574,8 @@ if isinstance(a, PurePath): parts += a._parts elif isinstance(a, str): - # Assuming a str - parts.append(a) + # Force-cast str subclasses to str (issue #21127) + parts.append(str(a)) else: raise TypeError( "argument should be a path or str object, not %r" diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -197,6 +197,25 @@ self.assertEqual(P(P('a'), 'b'), P('a/b')) self.assertEqual(P(P('a'), P('b')), P('a/b')) + def _check_str_subclass(self, *args): + # Issue #21127: it should be possible to construct a PurePath object + # from an str subclass instance, and it then gets converted to + # a pure str object. + class StrSubclass(str): + pass + P = self.cls + p = P(*(StrSubclass(x) for x in args)) + self.assertEqual(p, P(*args)) + for part in p.parts: + self.assertIs(type(part), str) + + def test_str_subclass_common(self): + self._check_str_subclass('') + self._check_str_subclass('.') + self._check_str_subclass('a') + self._check_str_subclass('a/b.txt') + self._check_str_subclass('/a/b.txt') + def test_join_common(self): P = self.cls p = P('a/b') @@ -690,6 +709,17 @@ p = self.cls('//a/b/c/d') self.assertEqual(str(p), '\\\\a\\b\\c\\d') + def test_str_subclass(self): + self._check_str_subclass('c:') + self._check_str_subclass('c:a') + self._check_str_subclass('c:a\\b.txt') + self._check_str_subclass('c:\\') + self._check_str_subclass('c:\\a') + self._check_str_subclass('c:\\a\\b.txt') + self._check_str_subclass('\\\\some\\share') + self._check_str_subclass('\\\\some\\share\\a') + self._check_str_subclass('\\\\some\\share\\a\\b.txt') + def test_eq(self): P = self.cls self.assertEqual(P('c:a/b'), P('c:a/b')) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -744,6 +744,7 @@ Chris Lawrence Brian Leair Mathieu Leduc-Hamel +Antony Lee Christopher Lee Inyeol Lee James Lee diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -54,6 +54,9 @@ Library ------- +- Issue #21127: Path objects can now be instantiated from str subclass + instances (such as numpy.str_). + - Issue #15002: urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) facility. Provides a better way to handle file descriptor close. Patch contributed by Christian Theune. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 03:55:16 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 23 Apr 2014 03:55:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_off-by-one?= =?utf-8?q?_error_=28closes_=2321330=29?= Message-ID: <3gD4Tw5lcBz7LjW@mail.python.org> http://hg.python.org/cpython/rev/b428b803f71f changeset: 90434:b428b803f71f branch: 2.7 parent: 90431:2b8d9276ad5b user: Benjamin Peterson date: Tue Apr 22 21:54:10 2014 -0400 summary: fix off-by-one error (closes #21330) files: Doc/howto/unicode.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -49,7 +49,7 @@ 255 characters aren't very many. For example, you can't fit both the accented characters used in Western Europe and the Cyrillic alphabet used for Russian -into the 128-255 range because there are more than 127 such characters. +into the 128-255 range because there are more than 128 such characters. You could write files using different codes (all your Russian files in a coding system called KOI8, all your French files in a different coding system called -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 03:55:18 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 23 Apr 2014 03:55:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_off-by-one?= =?utf-8?q?_error_=28closes_=2321330=29?= Message-ID: <3gD4Ty0ZJNz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/660d53bfb332 changeset: 90435:660d53bfb332 branch: 3.4 parent: 90432:c24cbd9bd63b user: Benjamin Peterson date: Tue Apr 22 21:54:10 2014 -0400 summary: fix off-by-one error (closes #21330) files: Doc/howto/unicode.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -49,7 +49,7 @@ 255 characters aren't very many. For example, you can't fit both the accented characters used in Western Europe and the Cyrillic alphabet used for Russian -into the 128--255 range because there are more than 127 such characters. +into the 128--255 range because there are more than 128 such characters. You could write files using different codes (all your Russian files in a coding system called KOI8, all your French files in a different coding system called -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 03:55:19 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 23 Apr 2014 03:55:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjEzMzAp?= Message-ID: <3gD4Tz2LQwz7Llb@mail.python.org> http://hg.python.org/cpython/rev/ae4a9000e925 changeset: 90436:ae4a9000e925 parent: 90433:aad6d6b819ed parent: 90435:660d53bfb332 user: Benjamin Peterson date: Tue Apr 22 21:55:07 2014 -0400 summary: merge 3.4 (#21330) files: Doc/howto/unicode.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -49,7 +49,7 @@ 255 characters aren't very many. For example, you can't fit both the accented characters used in Western Europe and the Cyrillic alphabet used for Russian -into the 128--255 range because there are more than 127 such characters. +into the 128--255 range because there are more than 128 such characters. You could write files using different codes (all your Russian files in a coding system called KOI8, all your French files in a different coding system called -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Wed Apr 23 08:44:57 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 23 Apr 2014 08:44:57 +0200 Subject: [Python-checkins] Daily reference leaks (aad6d6b819ed): sum=3 Message-ID: results for aad6d6b819ed on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog3oWVb_', '-x'] From python-checkins at python.org Wed Apr 23 09:38:33 2014 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 23 Apr 2014 09:38:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogc3VicHJvY2Vzcydz?= =?utf-8?q?_Popen=2Ewait=28=29_is_now_thread_safe_so_that_multiple_threads?= Message-ID: <3gDD611Cxbz7LjX@mail.python.org> http://hg.python.org/cpython/rev/5d745d97b7da changeset: 90437:5d745d97b7da branch: 3.4 parent: 90435:660d53bfb332 user: Gregory P. Smith date: Wed Apr 23 00:27:17 2014 -0700 summary: subprocess's Popen.wait() is now thread safe so that multiple threads may be calling wait() or poll() on a Popen instance at the same time without losing the Popen.returncode value. Fixes issue #21291. files: Lib/subprocess.py | 50 ++++++++++++++++++++----- Lib/test/test_subprocess.py | 48 ++++++++++++++++++++++++ Misc/NEWS | 4 ++ 3 files changed, 92 insertions(+), 10 deletions(-) diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -405,6 +405,10 @@ import _posixsubprocess import select import selectors + try: + import threading + except ImportError: + import dummy_threading as threading # When select or poll has indicated that the file is writable, # we can write up to _PIPE_BUF bytes without risk of blocking. @@ -748,6 +752,12 @@ pass_fds=()): """Create new Popen instance.""" _cleanup() + # Held while anything is calling waitpid before returncode has been + # updated to prevent clobbering returncode if wait() or poll() are + # called from multiple threads at once. After acquiring the lock, + # code must re-check self.returncode to see if another thread just + # finished a waitpid() call. + self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False @@ -1450,6 +1460,7 @@ def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED, _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED, _WEXITSTATUS=os.WEXITSTATUS): + """All callers to this function MUST hold self._waitpid_lock.""" # This method is called (indirectly) by __del__, so it cannot # refer to anything outside of its local scope. if _WIFSIGNALED(sts): @@ -1471,7 +1482,13 @@ """ if self.returncode is None: + if not self._waitpid_lock.acquire(False): + # Something else is busy calling waitpid. Don't allow two + # at once. We know nothing yet. + return None try: + if self.returncode is not None: + return self.returncode # Another thread waited. pid, sts = _waitpid(self.pid, _WNOHANG) if pid == self.pid: self._handle_exitstatus(sts) @@ -1485,10 +1502,13 @@ # can't get the status. # http://bugs.python.org/issue15756 self.returncode = 0 + finally: + self._waitpid_lock.release() return self.returncode def _try_wait(self, wait_flags): + """All callers to this function MUST hold self._waitpid_lock.""" try: (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags) except OSError as e: @@ -1521,11 +1541,17 @@ # cribbed from Lib/threading.py in Thread.wait() at r71065. delay = 0.0005 # 500 us -> initial delay of 1 ms while True: - (pid, sts) = self._try_wait(os.WNOHANG) - assert pid == self.pid or pid == 0 - if pid == self.pid: - self._handle_exitstatus(sts) - break + if self._waitpid_lock.acquire(False): + try: + if self.returncode is not None: + break # Another thread waited. + (pid, sts) = self._try_wait(os.WNOHANG) + assert pid == self.pid or pid == 0 + if pid == self.pid: + self._handle_exitstatus(sts) + break + finally: + self._waitpid_lock.release() remaining = self._remaining_time(endtime) if remaining <= 0: raise TimeoutExpired(self.args, timeout) @@ -1533,11 +1559,15 @@ time.sleep(delay) else: while self.returncode is None: - (pid, sts) = self._try_wait(0) - # Check the pid and loop as waitpid has been known to return - # 0 even without WNOHANG in odd situations. issue14396. - if pid == self.pid: - self._handle_exitstatus(sts) + with self._waitpid_lock: + if self.returncode is not None: + break # Another thread waited. + (pid, sts) = self._try_wait(0) + # Check the pid and loop as waitpid has been known to + # return 0 even without WNOHANG in odd situations. + # http://bugs.python.org/issue14396. + if pid == self.pid: + self._handle_exitstatus(sts) return self.returncode diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -1052,6 +1052,54 @@ if exc is not None: raise exc + @unittest.skipIf(threading is None, "threading required") + def test_threadsafe_wait(self): + """Issue21291: Popen.wait() needs to be threadsafe for returncode.""" + proc = subprocess.Popen([sys.executable, '-c', + 'import time; time.sleep(12)']) + self.assertEqual(proc.returncode, None) + results = [] + + def kill_proc_timer_thread(): + results.append(('thread-start-poll-result', proc.poll())) + # terminate it from the thread and wait for the result. + proc.kill() + proc.wait() + results.append(('thread-after-kill-and-wait', proc.returncode)) + # this wait should be a no-op given the above. + proc.wait() + results.append(('thread-after-second-wait', proc.returncode)) + + # This is a timing sensitive test, the failure mode is + # triggered when both the main thread and this thread are in + # the wait() call at once. The delay here is to allow the + # main thread to most likely be blocked in its wait() call. + t = threading.Timer(0.2, kill_proc_timer_thread) + t.start() + + # Wait for the process to finish; the thread should kill it + # long before it finishes on its own. Supplying a timeout + # triggers a different code path for better coverage. + proc.wait(timeout=20) + # Should be -9 because of the proc.kill() from the thread. + self.assertEqual(proc.returncode, -9, + msg="unexpected result in wait from main thread") + + # This should be a no-op with no change in returncode. + proc.wait() + self.assertEqual(proc.returncode, -9, + msg="unexpected result in second main wait.") + + t.join() + # Ensure that all of the thread results are as expected. + # When a race condition occurs in wait(), the returncode could + # be set by the wrong thread that doesn't actually have it + # leading to an incorrect value. + self.assertEqual([('thread-start-poll-result', None), + ('thread-after-kill-and-wait', -9), + ('thread-after-second-wait', -9)], + results) + def test_issue8780(self): # Ensure that stdout is inherited from the parent # if stdout=PIPE is not used diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,10 @@ Library ------- +- Issue #21291: subprocess's Popen.wait() is now thread safe so that + multiple threads may be calling wait() or poll() on a Popen instance + at the same time without losing the Popen.returncode value. + - Issue #21127: Path objects can now be instantiated from str subclass instances (such as numpy.str_). -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 09:38:34 2014 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 23 Apr 2014 09:38:34 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_subprocess=27s_Popen=2Ewait=28=29_is_now_thread_safe_so_?= =?utf-8?q?that_multiple_threads?= Message-ID: <3gDD6248rnz7LkB@mail.python.org> http://hg.python.org/cpython/rev/df45d0336dad changeset: 90438:df45d0336dad parent: 90436:ae4a9000e925 parent: 90437:5d745d97b7da user: Gregory P. Smith date: Wed Apr 23 00:38:22 2014 -0700 summary: subprocess's Popen.wait() is now thread safe so that multiple threads may be calling wait() or poll() on a Popen instance at the same time without losing the Popen.returncode value. Fixes issue #21291. files: Lib/subprocess.py | 50 ++++++++++++++++++++----- Lib/test/test_subprocess.py | 48 ++++++++++++++++++++++++ Misc/NEWS | 4 ++ 3 files changed, 92 insertions(+), 10 deletions(-) diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -416,6 +416,10 @@ import _posixsubprocess import select import selectors + try: + import threading + except ImportError: + import dummy_threading as threading # When select or poll has indicated that the file is writable, # we can write up to _PIPE_BUF bytes without risk of blocking. @@ -759,6 +763,12 @@ pass_fds=()): """Create new Popen instance.""" _cleanup() + # Held while anything is calling waitpid before returncode has been + # updated to prevent clobbering returncode if wait() or poll() are + # called from multiple threads at once. After acquiring the lock, + # code must re-check self.returncode to see if another thread just + # finished a waitpid() call. + self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False @@ -1466,6 +1476,7 @@ def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED, _WTERMSIG=os.WTERMSIG, _WIFEXITED=os.WIFEXITED, _WEXITSTATUS=os.WEXITSTATUS): + """All callers to this function MUST hold self._waitpid_lock.""" # This method is called (indirectly) by __del__, so it cannot # refer to anything outside of its local scope. if _WIFSIGNALED(sts): @@ -1487,7 +1498,13 @@ """ if self.returncode is None: + if not self._waitpid_lock.acquire(False): + # Something else is busy calling waitpid. Don't allow two + # at once. We know nothing yet. + return None try: + if self.returncode is not None: + return self.returncode # Another thread waited. pid, sts = _waitpid(self.pid, _WNOHANG) if pid == self.pid: self._handle_exitstatus(sts) @@ -1501,10 +1518,13 @@ # can't get the status. # http://bugs.python.org/issue15756 self.returncode = 0 + finally: + self._waitpid_lock.release() return self.returncode def _try_wait(self, wait_flags): + """All callers to this function MUST hold self._waitpid_lock.""" try: (pid, sts) = _eintr_retry_call(os.waitpid, self.pid, wait_flags) except OSError as e: @@ -1537,11 +1557,17 @@ # cribbed from Lib/threading.py in Thread.wait() at r71065. delay = 0.0005 # 500 us -> initial delay of 1 ms while True: - (pid, sts) = self._try_wait(os.WNOHANG) - assert pid == self.pid or pid == 0 - if pid == self.pid: - self._handle_exitstatus(sts) - break + if self._waitpid_lock.acquire(False): + try: + if self.returncode is not None: + break # Another thread waited. + (pid, sts) = self._try_wait(os.WNOHANG) + assert pid == self.pid or pid == 0 + if pid == self.pid: + self._handle_exitstatus(sts) + break + finally: + self._waitpid_lock.release() remaining = self._remaining_time(endtime) if remaining <= 0: raise TimeoutExpired(self.args, timeout) @@ -1549,11 +1575,15 @@ time.sleep(delay) else: while self.returncode is None: - (pid, sts) = self._try_wait(0) - # Check the pid and loop as waitpid has been known to return - # 0 even without WNOHANG in odd situations. issue14396. - if pid == self.pid: - self._handle_exitstatus(sts) + with self._waitpid_lock: + if self.returncode is not None: + break # Another thread waited. + (pid, sts) = self._try_wait(0) + # Check the pid and loop as waitpid has been known to + # return 0 even without WNOHANG in odd situations. + # http://bugs.python.org/issue14396. + if pid == self.pid: + self._handle_exitstatus(sts) return self.returncode diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -1052,6 +1052,54 @@ if exc is not None: raise exc + @unittest.skipIf(threading is None, "threading required") + def test_threadsafe_wait(self): + """Issue21291: Popen.wait() needs to be threadsafe for returncode.""" + proc = subprocess.Popen([sys.executable, '-c', + 'import time; time.sleep(12)']) + self.assertEqual(proc.returncode, None) + results = [] + + def kill_proc_timer_thread(): + results.append(('thread-start-poll-result', proc.poll())) + # terminate it from the thread and wait for the result. + proc.kill() + proc.wait() + results.append(('thread-after-kill-and-wait', proc.returncode)) + # this wait should be a no-op given the above. + proc.wait() + results.append(('thread-after-second-wait', proc.returncode)) + + # This is a timing sensitive test, the failure mode is + # triggered when both the main thread and this thread are in + # the wait() call at once. The delay here is to allow the + # main thread to most likely be blocked in its wait() call. + t = threading.Timer(0.2, kill_proc_timer_thread) + t.start() + + # Wait for the process to finish; the thread should kill it + # long before it finishes on its own. Supplying a timeout + # triggers a different code path for better coverage. + proc.wait(timeout=20) + # Should be -9 because of the proc.kill() from the thread. + self.assertEqual(proc.returncode, -9, + msg="unexpected result in wait from main thread") + + # This should be a no-op with no change in returncode. + proc.wait() + self.assertEqual(proc.returncode, -9, + msg="unexpected result in second main wait.") + + t.join() + # Ensure that all of the thread results are as expected. + # When a race condition occurs in wait(), the returncode could + # be set by the wrong thread that doesn't actually have it + # leading to an incorrect value. + self.assertEqual([('thread-start-poll-result', None), + ('thread-after-kill-and-wait', -9), + ('thread-after-second-wait', -9)], + results) + def test_issue8780(self): # Ensure that stdout is inherited from the parent # if stdout=PIPE is not used diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -54,6 +54,10 @@ Library ------- +- Issue #21291: subprocess's Popen.wait() is now thread safe so that + multiple threads may be calling wait() or poll() on a Popen instance + at the same time without losing the Popen.returncode value. + - Issue #21127: Path objects can now be instantiated from str subclass instances (such as numpy.str_). -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 09:59:23 2014 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 23 Apr 2014 09:59:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_implementation_notes?= Message-ID: <3gDDZ33hMhz7Ljd@mail.python.org> http://hg.python.org/cpython/rev/1baa02e1de82 changeset: 90439:1baa02e1de82 user: Raymond Hettinger date: Wed Apr 23 00:58:48 2014 -0700 summary: Add implementation notes files: Modules/_collectionsmodule.c | 33 +++++++++++++++++++++++- 1 files changed, 32 insertions(+), 1 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -3,7 +3,7 @@ /* collections module implementation of a deque() datatype Written and maintained by Raymond D. Hettinger - Copyright (c) 2004-2013 Python Software Foundation. + Copyright (c) 2004-2014 Python Software Foundation. All rights reserved. */ @@ -145,6 +145,12 @@ static PyTypeObject deque_type; +/* XXX Todo: + If aligned memory allocations become available, make the + deque object 64 byte aligned so that all of the fields + can be retrieved or updated in a single cache line. +*/ + static PyObject * deque_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { @@ -454,6 +460,31 @@ return (PyObject *)deque; } +/* The rotate() method is part of the public API and is used internally +as a primitive for other methods. + +Rotation by 1 or -1 is a common case, so any optimizations for high +volume rotations should take care not to penalize the common case. + +Conceptually, a rotate by one is equivalent to a pop on one side and an +append on the other. However, a pop/append pair is unnecessarily slow +because it requires a incref/decref pair for an object located randomly +in memory. It is better to just move the object pointer from one block +to the next without changing the reference count. + +When moving batches of pointers, it is tempting to use memcpy() but that +proved to be slower than a simple loop for a variety of reasons. +Memcpy() cannot know in advance that we're copying pointers instead of +bytes, that the source and destination are pointer aligned and +non-overlapping, that moving just one pointer is a common case, that we +never need to move more than BLOCKLEN pointers, and that at least one +pointer is always moved. + +For high volume rotations, newblock() and freeblock() are never called +more than once. Previously emptied blocks are immediately reused as a +destination block. If a block is left-over at the end, it is freed. +*/ + static int _deque_rotate(dequeobject *deque, Py_ssize_t n) { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 17:05:08 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 17:05:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Prevent_Sphinx?= =?utf-8?q?_error_on_whatsnew/changelog?= Message-ID: <3gDQ1J1DHsz7Lkd@mail.python.org> http://hg.python.org/cpython/rev/75419257fec3 changeset: 90440:75419257fec3 branch: 3.4 parent: 90437:5d745d97b7da user: Zachary Ware date: Wed Apr 23 10:04:20 2014 -0500 summary: Prevent Sphinx error on whatsnew/changelog files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -44,7 +44,7 @@ at the same time without losing the Popen.returncode value. - Issue #21127: Path objects can now be instantiated from str subclass - instances (such as numpy.str_). + instances (such as ``numpy.str_``). - Issue #15002: urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) facility. Provides a better way to handle file -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 17:05:09 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 17:05:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E4?= Message-ID: <3gDQ1K2xYmz7Ljk@mail.python.org> http://hg.python.org/cpython/rev/a70cf4a35376 changeset: 90441:a70cf4a35376 parent: 90439:1baa02e1de82 parent: 90440:75419257fec3 user: Zachary Ware date: Wed Apr 23 10:04:55 2014 -0500 summary: Merge from 3.4 files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -59,7 +59,7 @@ at the same time without losing the Popen.returncode value. - Issue #21127: Path objects can now be instantiated from str subclass - instances (such as numpy.str_). + instances (such as ``numpy.str_``). - Issue #15002: urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) facility. Provides a better way to handle file -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 17:39:13 2014 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 23 Apr 2014 17:39:13 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_fix_the_test_o?= =?utf-8?q?n_windows_which_has_different_return_codes_from_killed?= Message-ID: <3gDQmd2lP2z7LjV@mail.python.org> http://hg.python.org/cpython/rev/2d1049ebb588 changeset: 90442:2d1049ebb588 branch: 3.4 parent: 90440:75419257fec3 user: Gregory P. Smith date: Wed Apr 23 08:38:36 2014 -0700 summary: fix the test on windows which has different return codes from killed children. files: Lib/test/test_subprocess.py | 15 ++++++++++----- 1 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -1077,17 +1077,22 @@ t = threading.Timer(0.2, kill_proc_timer_thread) t.start() + if mswindows: + expected_errorcode = 1 + else: + # Should be -9 because of the proc.kill() from the thread. + expected_errorcode = -9 + # Wait for the process to finish; the thread should kill it # long before it finishes on its own. Supplying a timeout # triggers a different code path for better coverage. proc.wait(timeout=20) - # Should be -9 because of the proc.kill() from the thread. - self.assertEqual(proc.returncode, -9, + self.assertEqual(proc.returncode, expected_errorcode, msg="unexpected result in wait from main thread") # This should be a no-op with no change in returncode. proc.wait() - self.assertEqual(proc.returncode, -9, + self.assertEqual(proc.returncode, expected_errorcode, msg="unexpected result in second main wait.") t.join() @@ -1096,8 +1101,8 @@ # be set by the wrong thread that doesn't actually have it # leading to an incorrect value. self.assertEqual([('thread-start-poll-result', None), - ('thread-after-kill-and-wait', -9), - ('thread-after-second-wait', -9)], + ('thread-after-kill-and-wait', expected_errorcode), + ('thread-after-second-wait', expected_errorcode)], results) def test_issue8780(self): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 17:39:14 2014 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 23 Apr 2014 17:39:14 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_fix_the_test_on_windows_which_has_different_return_codes?= =?utf-8?q?_from_killed?= Message-ID: <3gDQmf4VXWz7Ljk@mail.python.org> http://hg.python.org/cpython/rev/5e978d499066 changeset: 90443:5e978d499066 parent: 90441:a70cf4a35376 parent: 90442:2d1049ebb588 user: Gregory P. Smith date: Wed Apr 23 08:39:02 2014 -0700 summary: fix the test on windows which has different return codes from killed children. files: Lib/test/test_subprocess.py | 15 ++++++++++----- 1 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -1077,17 +1077,22 @@ t = threading.Timer(0.2, kill_proc_timer_thread) t.start() + if mswindows: + expected_errorcode = 1 + else: + # Should be -9 because of the proc.kill() from the thread. + expected_errorcode = -9 + # Wait for the process to finish; the thread should kill it # long before it finishes on its own. Supplying a timeout # triggers a different code path for better coverage. proc.wait(timeout=20) - # Should be -9 because of the proc.kill() from the thread. - self.assertEqual(proc.returncode, -9, + self.assertEqual(proc.returncode, expected_errorcode, msg="unexpected result in wait from main thread") # This should be a no-op with no change in returncode. proc.wait() - self.assertEqual(proc.returncode, -9, + self.assertEqual(proc.returncode, expected_errorcode, msg="unexpected result in second main wait.") t.join() @@ -1096,8 +1101,8 @@ # be set by the wrong thread that doesn't actually have it # leading to an incorrect value. self.assertEqual([('thread-start-poll-result', None), - ('thread-after-kill-and-wait', -9), - ('thread-after-second-wait', -9)], + ('thread-after-kill-and-wait', expected_errorcode), + ('thread-after-second-wait', expected_errorcode)], results) def test_issue8780(self): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 19:12:47 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 19:12:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_=22Escape=22_a_character_i?= =?utf-8?q?n_the_middle_of_=22warning=22?= Message-ID: <3gDSrb5RrXz7LjW@mail.python.org> http://hg.python.org/cpython/rev/23666151612f changeset: 90444:23666151612f user: Zachary Ware date: Wed Apr 23 12:09:54 2014 -0500 summary: "Escape" a character in the middle of "warning" This should stop Buildbot from falsely reporting a warning on Windows bots. files: PCbuild/pythoncore.vcxproj | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -184,7 +184,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -219,7 +219,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -254,7 +254,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -292,7 +292,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -325,7 +325,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -361,7 +361,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -394,7 +394,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -430,7 +430,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 19:18:00 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 19:18:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIkVzY2FwZSIgYSBj?= =?utf-8?q?haracter_in_the_middle_of_=22warning=22?= Message-ID: <3gDSyc0K7zz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/9c5c9d4ed9ba changeset: 90445:9c5c9d4ed9ba branch: 3.4 parent: 90442:2d1049ebb588 user: Zachary Ware date: Wed Apr 23 12:09:54 2014 -0500 summary: "Escape" a character in the middle of "warning" This should stop Buildbot from falsely reporting a warning on Windows bots. files: PCbuild/pythoncore.vcxproj | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -184,7 +184,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -219,7 +219,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -254,7 +254,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -292,7 +292,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -325,7 +325,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -361,7 +361,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -394,7 +394,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) @@ -430,7 +430,7 @@ $(KillPythonExe) IF %ERRORLEVEL% NEQ 0 ( - echo kill_python: warning: could not kill running Pythons, exit code %ERRORLEVEL% + echo kill_python: war^ning: could not kill running Pythons, exit code %ERRORLEVEL% exit /b 0 ) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 19:18:01 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 19:18:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Null_merge?= Message-ID: <3gDSyd269rz7LkB@mail.python.org> http://hg.python.org/cpython/rev/ed291f85434b changeset: 90446:ed291f85434b parent: 90444:23666151612f parent: 90445:9c5c9d4ed9ba user: Zachary Ware date: Wed Apr 23 12:17:25 2014 -0500 summary: Null merge files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 20:54:44 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 20:54:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_compiler_warning_on_Wi?= =?utf-8?q?ndows?= Message-ID: <3gDW6D1PHrz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/1ff53fd593ed changeset: 90447:1ff53fd593ed user: Zachary Ware date: Wed Apr 23 13:51:27 2014 -0500 summary: Fix compiler warning on Windows ..\Modules\_testcapimodule.c(3320): warning C4098: 'matmulType_dealloc' : 'void' function returning a value files: Modules/_testcapimodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3317,7 +3317,7 @@ static void matmulType_dealloc(PyObject *self) { - return Py_TYPE(self)->tp_free(self); + Py_TYPE(self)->tp_free(self); } static PyNumberMethods matmulType_as_number = { -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 23 22:38:05 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 23 Apr 2014 22:38:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239765=3A_Adjust_wh?= =?utf-8?q?ere_Tools/msi/msi=2Epy_looks_for_Tcl/Tk_license_terms=2E?= Message-ID: <3gDYPT3lMLz7LjP@mail.python.org> http://hg.python.org/cpython/rev/4ff37fbcd4e8 changeset: 90448:4ff37fbcd4e8 user: Zachary Ware date: Wed Apr 23 15:37:37 2014 -0500 summary: Issue #9765: Adjust where Tools/msi/msi.py looks for Tcl/Tk license terms. Patch by Daniel Stutzbach. files: Tools/msi/msi.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -925,8 +925,8 @@ shutil.copyfileobj(open("crtlicense.txt"), out) for name, pat, file in (("bzip2","bzip2-*", "LICENSE"), ("openssl", "openssl-*", "LICENSE"), - ("Tcl", "tcl8*", "license.terms"), - ("Tk", "tk8*", "license.terms"), + ("Tcl", "tcl-8*", "license.terms"), + ("Tk", "tk-8*", "license.terms"), ("Tix", "tix-*", "license.terms")): out.write("\nThis copy of Python includes a copy of %s, which is licensed under the following terms:\n\n" % name) dirs = glob.glob(srcdir+"/../"+pat) -- Repository URL: http://hg.python.org/cpython From tjreedy at udel.edu Thu Apr 24 00:24:53 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 23 Apr 2014 18:24:53 -0400 Subject: [Python-checkins] cpython (3.4): Prevent Sphinx error on whatsnew/changelog In-Reply-To: <3gDQ1J1DHsz7Lkd@mail.python.org> References: <3gDQ1J1DHsz7Lkd@mail.python.org> Message-ID: <53583DB5.8070809@udel.edu> On 4/23/2014 11:05 AM, zach.ware wrote: > http://hg.python.org/cpython/rev/75419257fec3 > changeset: 90440:75419257fec3 > branch: 3.4 > parent: 90437:5d745d97b7da > user: Zachary Ware > date: Wed Apr 23 10:04:20 2014 -0500 > summary: > Prevent Sphinx error on whatsnew/changelog > > files: > Misc/NEWS | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -44,7 +44,7 @@ > at the same time without losing the Popen.returncode value. > > - Issue #21127: Path objects can now be instantiated from str subclass > - instances (such as numpy.str_). > + instances (such as ``numpy.str_``). Could you explain this? I thought NEWS was a plain text file without .rst markup. > - Issue #15002: urllib.response object to use _TemporaryFileWrapper (and > _TemporaryFileCloser) facility. Provides a better way to handle file From berker.peksag at gmail.com Thu Apr 24 01:53:44 2014 From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=) Date: Thu, 24 Apr 2014 02:53:44 +0300 Subject: [Python-checkins] cpython (3.4): Prevent Sphinx error on whatsnew/changelog In-Reply-To: <53583DB5.8070809@udel.edu> References: <3gDQ1J1DHsz7Lkd@mail.python.org> <53583DB5.8070809@udel.edu> Message-ID: On Thu, Apr 24, 2014 at 1:24 AM, Terry Reedy wrote: > On 4/23/2014 11:05 AM, zach.ware wrote: >> >> http://hg.python.org/cpython/rev/75419257fec3 >> changeset: 90440:75419257fec3 >> branch: 3.4 >> parent: 90437:5d745d97b7da >> user: Zachary Ware >> date: Wed Apr 23 10:04:20 2014 -0500 >> summary: >> Prevent Sphinx error on whatsnew/changelog >> >> files: >> Misc/NEWS | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> diff --git a/Misc/NEWS b/Misc/NEWS >> --- a/Misc/NEWS >> +++ b/Misc/NEWS >> @@ -44,7 +44,7 @@ >> at the same time without losing the Popen.returncode value. >> >> - Issue #21127: Path objects can now be instantiated from str subclass >> - instances (such as numpy.str_). >> + instances (such as ``numpy.str_``). > > > Could you explain this? I thought NEWS was a plain text file without .rst > markup. Hi, Doc/whatsnew/changelog.rst [1] using it via a custom Sphinx directive: http://hg.python.org/cpython/file/4ff37fbcd4e8/Doc/tools/sphinxext/pyspecific.py#l200 --Berker [1] http://hg.python.org/cpython/file/4ff37fbcd4e8/Doc/whatsnew/changelog.rst > > >> - Issue #15002: urllib.response object to use _TemporaryFileWrapper (and >> _TemporaryFileCloser) facility. Provides a better way to handle file > > > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > https://mail.python.org/mailman/listinfo/python-checkins From zachary.ware+pydev at gmail.com Thu Apr 24 03:30:09 2014 From: zachary.ware+pydev at gmail.com (Zachary Ware) Date: Wed, 23 Apr 2014 20:30:09 -0500 Subject: [Python-checkins] cpython (3.4): Prevent Sphinx error on whatsnew/changelog In-Reply-To: <53583DB5.8070809@udel.edu> References: <3gDQ1J1DHsz7Lkd@mail.python.org> <53583DB5.8070809@udel.edu> Message-ID: On April 23, 2014 5:24:53 PM CDT, Terry Reedy wrote: >On 4/23/2014 11:05 AM, zach.ware wrote: >> http://hg.python.org/cpython/rev/75419257fec3 >> changeset: 90440:75419257fec3 >> branch: 3.4 >> parent: 90437:5d745d97b7da >> user: Zachary Ware >> date: Wed Apr 23 10:04:20 2014 -0500 >> summary: >> Prevent Sphinx error on whatsnew/changelog >> >> files: >> Misc/NEWS | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> >> diff --git a/Misc/NEWS b/Misc/NEWS >> --- a/Misc/NEWS >> +++ b/Misc/NEWS >> @@ -44,7 +44,7 @@ >> at the same time without losing the Popen.returncode value. >> >> - Issue #21127: Path objects can now be instantiated from str >subclass >> - instances (such as numpy.str_). >> + instances (such as ``numpy.str_``). > >Could you explain this? I thought NEWS was a plain text file without >.rst markup. Doc/whatsnew/changelog.rst includes the full text of Misc/NEWS, which means Sphinx doesn't like markup issues in NEWS. -- Zach (on a phone) From solipsis at pitrou.net Thu Apr 24 09:58:59 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Thu, 24 Apr 2014 09:58:59 +0200 Subject: [Python-checkins] Daily reference leaks (4ff37fbcd4e8): sum=-1 Message-ID: results for 4ff37fbcd4e8 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogcVO8Z5', '-x'] From python-checkins at python.org Thu Apr 24 18:09:39 2014 From: python-checkins at python.org (giampaolo.rodola) Date: Thu, 24 Apr 2014 18:09:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_update_os=2Esendfile=28=29?= =?utf-8?q?_doc_signaling_that_cross-platform_apps_should_not_use?= Message-ID: <3gF3PH6zc8z7Lmx@mail.python.org> http://hg.python.org/cpython/rev/ebbbbbb96624 changeset: 90449:ebbbbbb96624 user: Giampaolo Rodola' date: Thu Apr 24 18:09:21 2014 +0200 summary: update os.sendfile() doc signaling that cross-platform apps should not use headers, trailers and flags arguments files: Doc/library/os.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1087,6 +1087,9 @@ All platforms support sockets as *out* file descriptor, and some platforms allow other types (e.g. regular file, pipe) as well. + Cross-platform applications should not use *headers*, *trailers* and *flags* + arguments. + Availability: Unix. .. versionadded:: 3.3 -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 24 20:23:25 2014 From: python-checkins at python.org (zach.ware) Date: Thu, 24 Apr 2014 20:23:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMzQ2?= =?utf-8?q?=3A_Fix_typos_in_test=5Fitertools=2E__Patch_by_Brian_Kearns=2E?= Message-ID: <3gF6Md1fmlz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/1beb3e0507fa changeset: 90450:1beb3e0507fa branch: 2.7 parent: 90434:b428b803f71f user: Zachary Ware date: Thu Apr 24 13:20:27 2014 -0500 summary: Issue #21346: Fix typos in test_itertools. Patch by Brian Kearns. files: Lib/test/test_itertools.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -274,7 +274,7 @@ self.assertEqual(result, list(permutations(values, None))) # test r as None self.assertEqual(result, list(permutations(values))) # test default r - @test_support.impl_detail("tuple resuse is CPython specific") + @test_support.impl_detail("tuple reuse is specific to CPython") def test_permutations_tuple_reuse(self): self.assertEqual(len(set(map(id, permutations('abcde', 3)))), 1) self.assertNotEqual(len(set(map(id, list(permutations('abcde', 3))))), 1) @@ -536,7 +536,7 @@ zip('abc', 'def')) @test_support.impl_detail("tuple reuse is specific to CPython") - def test_izip_tuple_resuse(self): + def test_izip_tuple_reuse(self): ids = map(id, izip('abc', 'def')) self.assertEqual(min(ids), max(ids)) ids = map(id, list(izip('abc', 'def'))) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 24 20:23:26 2014 From: python-checkins at python.org (zach.ware) Date: Thu, 24 Apr 2014 20:23:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMzQ2?= =?utf-8?q?=3A_Fix_typo=2C_make_message_consistent_in_test=5Fitertools=2E?= Message-ID: <3gF6Mf3YhVz7Llh@mail.python.org> http://hg.python.org/cpython/rev/901b9afc918e changeset: 90451:901b9afc918e branch: 3.4 parent: 90445:9c5c9d4ed9ba user: Zachary Ware date: Thu Apr 24 13:22:05 2014 -0500 summary: Issue #21346: Fix typo, make message consistent in test_itertools. Pointed out by Brian Kearns. files: Lib/test/test_itertools.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -411,7 +411,7 @@ self.pickletest(permutations(values, r)) # test pickling - @support.impl_detail("tuple resuse is CPython specific") + @support.impl_detail("tuple reuse is specific to CPython") def test_permutations_tuple_reuse(self): self.assertEqual(len(set(map(id, permutations('abcde', 3)))), 1) self.assertNotEqual(len(set(map(id, list(permutations('abcde', 3))))), 1) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 24 20:23:27 2014 From: python-checkins at python.org (zach.ware) Date: Thu, 24 Apr 2014 20:23:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2321346=3A_Merge_with_3=2E4?= Message-ID: <3gF6Mg53BJz7LmR@mail.python.org> http://hg.python.org/cpython/rev/21012515c249 changeset: 90452:21012515c249 parent: 90449:ebbbbbb96624 parent: 90451:901b9afc918e user: Zachary Ware date: Thu Apr 24 13:22:47 2014 -0500 summary: Closes #21346: Merge with 3.4 files: Lib/test/test_itertools.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -411,7 +411,7 @@ self.pickletest(permutations(values, r)) # test pickling - @support.impl_detail("tuple resuse is CPython specific") + @support.impl_detail("tuple reuse is specific to CPython") def test_permutations_tuple_reuse(self): self.assertEqual(len(set(map(id, permutations('abcde', 3)))), 1) self.assertNotEqual(len(set(map(id, list(permutations('abcde', 3))))), 1) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Thu Apr 24 21:54:45 2014 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 24 Apr 2014 21:54:45 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?hooks=3A_committed/rejected_is_now_ca?= =?utf-8?q?lled_resolved?= Message-ID: <3gF8P15wCrz7Lk5@mail.python.org> http://hg.python.org/hooks/rev/d4f26b92b1f3 changeset: 88:d4f26b92b1f3 user: Benjamin Peterson date: Thu Apr 24 15:54:41 2014 -0400 summary: committed/rejected is now called resolved files: hgroundup.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/hgroundup.py b/hgroundup.py --- a/hgroundup.py +++ b/hgroundup.py @@ -126,7 +126,7 @@ issues[key]['properties'].update({ 'status': 'closed', 'resolution': 'fixed', - 'stage': 'committed/rejected', + 'stage': 'resolved', }) def send_comments(mailrelay, fromaddr, toaddr, issues): -- Repository URL: http://hg.python.org/hooks From python-checkins at python.org Thu Apr 24 23:48:01 2014 From: python-checkins at python.org (ethan.furman) Date: Thu, 24 Apr 2014 23:48:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue8297=3A_module_attrib?= =?utf-8?q?ute_lookup_failures_now_include_module_name_in_error?= Message-ID: <3gFBvj18sjz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/d84a69b7ba72 changeset: 90453:d84a69b7ba72 user: Ethan Furman date: Thu Apr 24 14:47:47 2014 -0700 summary: Issue8297: module attribute lookup failures now include module name in error message. files: Lib/test/test_doctest.py | 4 +- Lib/test/test_module.py | 16 +++++++++++ Lib/unittest/test/test_loader.py | 10 +++--- Misc/NEWS | 3 ++ Objects/moduleobject.c | 28 ++++++++++++++++++- 5 files changed, 52 insertions(+), 9 deletions(-) diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -2171,7 +2171,7 @@ >>> test.test_doctest.sillySetup Traceback (most recent call last): ... - AttributeError: 'module' object has no attribute 'sillySetup' + AttributeError: module 'test.test_doctest' has no attribute 'sillySetup' The setUp and tearDown funtions are passed test objects. Here we'll use the setUp function to supply the missing variable y: @@ -2317,7 +2317,7 @@ >>> test.test_doctest.sillySetup Traceback (most recent call last): ... - AttributeError: 'module' object has no attribute 'sillySetup' + AttributeError: module 'test.test_doctest' has no attribute 'sillySetup' The setUp and tearDown funtions are passed test objects. Here, we'll use a setUp function to set the favorite color in diff --git a/Lib/test/test_module.py b/Lib/test/test_module.py --- a/Lib/test/test_module.py +++ b/Lib/test/test_module.py @@ -30,6 +30,22 @@ pass self.assertEqual(foo.__doc__, ModuleType.__doc__) + def test_unintialized_missing_getattr(self): + # Issue 8297 + # test the text in the AttributeError of an uninitialized module + foo = ModuleType.__new__(ModuleType) + self.assertRaisesRegex( + AttributeError, "module has no attribute 'not_here'", + getattr, foo, "not_here") + + def test_missing_getattr(self): + # Issue 8297 + # test the text in the AttributeError + foo = ModuleType("foo") + self.assertRaisesRegex( + AttributeError, "module 'foo' has no attribute 'not_here'", + getattr, foo, "not_here") + def test_no_docstring(self): # Regularly initialized module, no docstring foo = ModuleType("foo") diff --git a/Lib/unittest/test/test_loader.py b/Lib/unittest/test/test_loader.py --- a/Lib/unittest/test/test_loader.py +++ b/Lib/unittest/test/test_loader.py @@ -255,7 +255,7 @@ try: loader.loadTestsFromName('unittest.sdasfasfasdf') except AttributeError as e: - self.assertEqual(str(e), "'module' object has no attribute 'sdasfasfasdf'") + self.assertEqual(str(e), "module 'unittest' has no attribute 'sdasfasfasdf'") else: self.fail("TestLoader.loadTestsFromName failed to raise AttributeError") @@ -272,7 +272,7 @@ try: loader.loadTestsFromName('sdasfasfasdf', unittest) except AttributeError as e: - self.assertEqual(str(e), "'module' object has no attribute 'sdasfasfasdf'") + self.assertEqual(str(e), "module 'unittest' has no attribute 'sdasfasfasdf'") else: self.fail("TestLoader.loadTestsFromName failed to raise AttributeError") @@ -635,7 +635,7 @@ try: loader.loadTestsFromNames(['unittest.sdasfasfasdf', 'unittest']) except AttributeError as e: - self.assertEqual(str(e), "'module' object has no attribute 'sdasfasfasdf'") + self.assertEqual(str(e), "module 'unittest' has no attribute 'sdasfasfasdf'") else: self.fail("TestLoader.loadTestsFromNames failed to raise AttributeError") @@ -654,7 +654,7 @@ try: loader.loadTestsFromNames(['sdasfasfasdf'], unittest) except AttributeError as e: - self.assertEqual(str(e), "'module' object has no attribute 'sdasfasfasdf'") + self.assertEqual(str(e), "module 'unittest' has no attribute 'sdasfasfasdf'") else: self.fail("TestLoader.loadTestsFromName failed to raise AttributeError") @@ -673,7 +673,7 @@ try: loader.loadTestsFromNames(['TestCase', 'sdasfasfasdf'], unittest) except AttributeError as e: - self.assertEqual(str(e), "'module' object has no attribute 'sdasfasfasdf'") + self.assertEqual(str(e), "module 'unittest' has no attribute 'sdasfasfasdf'") else: self.fail("TestLoader.loadTestsFromName failed to raise AttributeError") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,9 @@ - Issue #20637: Key-sharing now also works for instance dictionaries of subclasses. Patch by Peter Ingebretson. +- Issue #8297: Attributes missing from modules now include the module name + in the error text. Original patch by ysj.ray. + - Issue #19995: %c, %o, %x, and %X now raise TypeError on non-integer input. - Issue #12546: Allow \x00 to be used as a fill character when using str, int, diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -411,6 +411,31 @@ return PyObject_CallMethod(interp->importlib, "_module_repr", "O", m); } +static PyObject* +module_getattr(PyObject *m, PyObject *name) +{ + PyModuleObject *module; + PyObject *attr, *mod_name; + attr = PyObject_GenericGetAttr(m, name); + if (attr != NULL) + return attr; + PyErr_Clear(); + module = (PyModuleObject*)m; + if (module->md_dict != NULL) { + mod_name = PyDict_GetItemString(module->md_dict, "__name__"); + if (mod_name != NULL) { + PyErr_Format(PyExc_AttributeError, + "module '%U' has no attribute '%U'", mod_name, name); + return NULL; + } + else if (PyErr_Occurred()) + PyErr_Clear(); + } + PyErr_Format(PyExc_AttributeError, + "module has no attribute '%U'", name); + return NULL; +} + static int module_traverse(PyModuleObject *m, visitproc visit, void *arg) { @@ -464,7 +489,6 @@ {0} }; - PyDoc_STRVAR(module_doc, "module(name[, doc])\n\ \n\ @@ -488,7 +512,7 @@ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ + module_getattr, /* tp_getattro */ PyObject_GenericSetAttr, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 25 01:39:25 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 25 Apr 2014 01:39:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_do_not_override_errors_fro?= =?utf-8?q?m_descriptors_on_modules?= Message-ID: <3gFFNF3rqPz7Ljh@mail.python.org> http://hg.python.org/cpython/rev/01edd9116b02 changeset: 90454:01edd9116b02 user: Benjamin Peterson date: Thu Apr 24 19:29:23 2014 -0400 summary: do not override errors from descriptors on modules files: Lib/test/test_module.py | 8 ++++++++ Objects/moduleobject.c | 19 +++++++++---------- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/Lib/test/test_module.py b/Lib/test/test_module.py --- a/Lib/test/test_module.py +++ b/Lib/test/test_module.py @@ -227,6 +227,14 @@ b"len = len", b"shutil.rmtree = rmtree"}) + def test_descriptor_errors_propogate(self): + class Descr: + def __get__(self, o, t): + raise RuntimeError + class M(ModuleType): + melon = Descr() + self.assertRaises(RuntimeError, getattr, M("mymod"), "melon") + # frozen and namespace module reprs are tested in importlib. diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -412,24 +412,23 @@ } static PyObject* -module_getattr(PyObject *m, PyObject *name) +module_getattro(PyModuleObject *m, PyObject *name) { - PyModuleObject *module; PyObject *attr, *mod_name; - attr = PyObject_GenericGetAttr(m, name); - if (attr != NULL) + attr = PyObject_GenericGetAttr((PyObject *)m, name); + if (attr || !PyErr_ExceptionMatches(PyExc_AttributeError)) return attr; PyErr_Clear(); - module = (PyModuleObject*)m; - if (module->md_dict != NULL) { - mod_name = PyDict_GetItemString(module->md_dict, "__name__"); - if (mod_name != NULL) { + if (m->md_dict) { + mod_name = PyDict_GetItemString(m->md_dict, "__name__"); + if (mod_name) { PyErr_Format(PyExc_AttributeError, "module '%U' has no attribute '%U'", mod_name, name); return NULL; } - else if (PyErr_Occurred()) + else if (PyErr_Occurred()) { PyErr_Clear(); + } } PyErr_Format(PyExc_AttributeError, "module has no attribute '%U'", name); @@ -512,7 +511,7 @@ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ - module_getattr, /* tp_getattro */ + (getattrofunc)module_getattro, /* tp_getattro */ PyObject_GenericSetAttr, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 25 01:39:26 2014 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 25 Apr 2014 01:39:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_sprinkle_some_=5FPyId_good?= =?utf-8?q?ness_around_moduleobject=2Ec?= Message-ID: <3gFFNG5ggMz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/c8d8a6cc5586 changeset: 90455:c8d8a6cc5586 user: Benjamin Peterson date: Thu Apr 24 19:39:18 2014 -0400 summary: sprinkle some _PyId goodness around moduleobject.c files: Objects/moduleobject.c | 28 +++++++++++++++++++--------- 1 files changed, 19 insertions(+), 9 deletions(-) diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -32,20 +32,26 @@ module_init_dict(PyModuleObject *mod, PyObject *md_dict, PyObject *name, PyObject *doc) { + _Py_IDENTIFIER(__name__); + _Py_IDENTIFIER(__doc__); + _Py_IDENTIFIER(__package__); + _Py_IDENTIFIER(__loader__); + _Py_IDENTIFIER(__spec__); + if (md_dict == NULL) return -1; if (doc == NULL) doc = Py_None; - if (PyDict_SetItemString(md_dict, "__name__", name) != 0) + if (_PyDict_SetItemId(md_dict, &PyId___name__, name) != 0) return -1; - if (PyDict_SetItemString(md_dict, "__doc__", doc) != 0) + if (_PyDict_SetItemId(md_dict, &PyId___doc__, doc) != 0) return -1; - if (PyDict_SetItemString(md_dict, "__package__", Py_None) != 0) + if (_PyDict_SetItemId(md_dict, &PyId___package__, Py_None) != 0) return -1; - if (PyDict_SetItemString(md_dict, "__loader__", Py_None) != 0) + if (_PyDict_SetItemId(md_dict, &PyId___loader__, Py_None) != 0) return -1; - if (PyDict_SetItemString(md_dict, "__spec__", Py_None) != 0) + if (_PyDict_SetItemId(md_dict, &PyId___spec__, Py_None) != 0) return -1; if (PyUnicode_CheckExact(name)) { Py_INCREF(name); @@ -184,8 +190,9 @@ Py_DECREF(n); } if (module->m_doc != NULL) { + _Py_IDENTIFIER(__doc__); v = PyUnicode_FromString(module->m_doc); - if (v == NULL || PyDict_SetItemString(d, "__doc__", v) != 0) { + if (v == NULL || _PyDict_SetItemId(d, &PyId___doc__, v) != 0) { Py_XDECREF(v); Py_DECREF(m); return NULL; @@ -214,6 +221,7 @@ PyObject* PyModule_GetNameObject(PyObject *m) { + _Py_IDENTIFIER(__name__); PyObject *d; PyObject *name; if (!PyModule_Check(m)) { @@ -222,7 +230,7 @@ } d = ((PyModuleObject *)m)->md_dict; if (d == NULL || - (name = PyDict_GetItemString(d, "__name__")) == NULL || + (name = _PyDict_GetItemId(d, &PyId___name__)) == NULL || !PyUnicode_Check(name)) { PyErr_SetString(PyExc_SystemError, "nameless module"); @@ -245,6 +253,7 @@ PyObject* PyModule_GetFilenameObject(PyObject *m) { + _Py_IDENTIFIER(__file__); PyObject *d; PyObject *fileobj; if (!PyModule_Check(m)) { @@ -253,7 +262,7 @@ } d = ((PyModuleObject *)m)->md_dict; if (d == NULL || - (fileobj = PyDict_GetItemString(d, "__file__")) == NULL || + (fileobj = _PyDict_GetItemId(d, &PyId___file__)) == NULL || !PyUnicode_Check(fileobj)) { PyErr_SetString(PyExc_SystemError, "module filename missing"); @@ -420,7 +429,8 @@ return attr; PyErr_Clear(); if (m->md_dict) { - mod_name = PyDict_GetItemString(m->md_dict, "__name__"); + _Py_IDENTIFIER(__name__); + mod_name = _PyDict_GetItemId(m->md_dict, &PyId___name__); if (mod_name) { PyErr_Format(PyExc_AttributeError, "module '%U' has no attribute '%U'", mod_name, name); -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Fri Apr 25 10:02:12 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Fri, 25 Apr 2014 10:02:12 +0200 Subject: [Python-checkins] Daily reference leaks (c8d8a6cc5586): sum=3 Message-ID: results for c8d8a6cc5586 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 2, 0] references, sum=0 test_site leaked [-2, 2, 0] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogDwoTUB', '-x'] From python-checkins at python.org Fri Apr 25 11:54:00 2014 From: python-checkins at python.org (kristjan.jonsson) Date: Fri, 25 Apr 2014 11:54:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIwNDM0?= =?utf-8?q?_Correct_error_handlin_of_=5FPyString=5FResize_and_=5FPyBytes?= =?utf-8?q?=5FResize?= Message-ID: <3gFW1N0Kjgz7LjN@mail.python.org> http://hg.python.org/cpython/rev/4f79c3827adc changeset: 90456:4f79c3827adc branch: 2.7 parent: 90450:1beb3e0507fa user: Kristj?n Valur J?nsson date: Fri Apr 25 09:51:21 2014 +0000 summary: Issue #20434 Correct error handlin of _PyString_Resize and _PyBytes_Resize files: Modules/_io/fileio.c | 15 +----- Modules/binascii.c | 55 +++++++++----------------- Modules/bz2module.c | 3 +- Objects/bytearrayobject.c | 6 +- Objects/stringobject.c | 4 +- PC/_subprocess.c | 5 +- 6 files changed, 32 insertions(+), 56 deletions(-) diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -549,14 +549,8 @@ } if (PyBytes_GET_SIZE(result) < (Py_ssize_t)newsize) { - if (_PyBytes_Resize(&result, newsize) < 0) { - if (total == 0) { - Py_DECREF(result); - return NULL; - } - PyErr_Clear(); - break; - } + if (_PyBytes_Resize(&result, newsize) < 0) + return NULL; /* result has been freed */ } Py_BEGIN_ALLOW_THREADS errno = 0; @@ -599,7 +593,6 @@ if (PyBytes_GET_SIZE(result) > total) { if (_PyBytes_Resize(&result, total) < 0) { /* This should never happen, but just in case */ - Py_DECREF(result); return NULL; } } @@ -656,10 +649,8 @@ } if (n != size) { - if (_PyBytes_Resize(&bytes, n) < 0) { - Py_DECREF(bytes); + if (_PyBytes_Resize(&bytes, n) < 0) return NULL; - } } return (PyObject *) bytes; diff --git a/Modules/binascii.c b/Modules/binascii.c --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -320,12 +320,10 @@ } *ascii_data++ = '\n'; /* Append a courtesy newline */ - if (_PyString_Resize(&rv, + /* rv is cleared on error */ + (void)_PyString_Resize(&rv, (ascii_data - - (unsigned char *)PyString_AS_STRING(rv))) < 0) { - Py_DECREF(rv); - rv = NULL; - } + (unsigned char *)PyString_AS_STRING(rv))); PyBuffer_Release(&pbin); return rv; } @@ -452,10 +450,8 @@ ** string instead; _PyString_Resize() won't do this for us. */ if (bin_len > 0) { - if (_PyString_Resize(&rv, bin_len) < 0) { - Py_DECREF(rv); - rv = NULL; - } + /* rv is cleared on error */ + (void)_PyString_Resize(&rv, bin_len); } else { Py_DECREF(rv); @@ -522,12 +518,10 @@ } *ascii_data++ = '\n'; /* Append a courtesy newline */ - if (_PyString_Resize(&rv, + /* rv is cleared on error */ + (void)_PyString_Resize(&rv, (ascii_data - - (unsigned char *)PyString_AS_STRING(rv))) < 0) { - Py_DECREF(rv); - rv = NULL; - } + (unsigned char *)PyString_AS_STRING(rv))); PyBuffer_Release(&pbuf); return rv; } @@ -601,13 +595,10 @@ Py_DECREF(rv); return NULL; } + /* rv is cleared on error */ if (_PyString_Resize(&rv, (bin_data - - (unsigned char *)PyString_AS_STRING(rv))) < 0) { - Py_DECREF(rv); - rv = NULL; - } - if (rv) { + (unsigned char *)PyString_AS_STRING(rv))) == 0) { PyObject *rrv = Py_BuildValue("Oi", rv, done); PyBuffer_Release(&pascii); Py_DECREF(rv); @@ -672,12 +663,10 @@ } } } - if (_PyString_Resize(&rv, + /* rv is cleared on error */ + (void)_PyString_Resize(&rv, (out_data - - (unsigned char *)PyString_AS_STRING(rv))) < 0) { - Py_DECREF(rv); - rv = NULL; - } + (unsigned char *)PyString_AS_STRING(rv))); PyBuffer_Release(&pbuf); return rv; } @@ -729,12 +718,10 @@ leftchar <<= (6-leftbits); *ascii_data++ = table_b2a_hqx[leftchar & 0x3f]; } - if (_PyString_Resize(&rv, + /* rv is cleared on error */ + (void)_PyString_Resize(&rv, (ascii_data - - (unsigned char *)PyString_AS_STRING(rv))) < 0) { - Py_DECREF(rv); - rv = NULL; - } + (unsigned char *)PyString_AS_STRING(rv))); PyBuffer_Release(&pbin); return rv; } @@ -796,7 +783,7 @@ if ( --out_len_left < 0 ) { \ if ( out_len > PY_SSIZE_T_MAX / 2) return PyErr_NoMemory(); \ if (_PyString_Resize(&rv, 2*out_len) < 0) \ - { Py_DECREF(rv); PyBuffer_Release(&pin); return NULL; } \ + { PyBuffer_Release(&pin); return NULL; } \ out_data = (unsigned char *)PyString_AS_STRING(rv) \ + out_len; \ out_len_left = out_len-1; \ @@ -846,12 +833,10 @@ OUTBYTE(in_byte); } } - if (_PyString_Resize(&rv, + /* rv is cleared on error */ + (void)_PyString_Resize(&rv, (out_data - - (unsigned char *)PyString_AS_STRING(rv))) < 0) { - Py_DECREF(rv); - rv = NULL; - } + (unsigned char *)PyString_AS_STRING(rv))); PyBuffer_Release(&pin); return rv; } diff --git a/Modules/bz2module.c b/Modules/bz2module.c --- a/Modules/bz2module.c +++ b/Modules/bz2module.c @@ -732,7 +732,8 @@ } else { /* Grow the big buffer */ - _PyString_Resize(&big_buffer, buffersize); + if (_PyString_Resize(&big_buffer, buffersize)) + goto error; buffer = PyString_AS_STRING(big_buffer); } continue; diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -994,10 +994,8 @@ *p++ = *quote_postfix++; } *p = '\0'; - if (_PyString_Resize(&v, (p - PyString_AS_STRING(v)))) { - Py_DECREF(v); - return NULL; - } + /* v is cleared on error */ + (void)_PyString_Resize(&v, (p - PyString_AS_STRING(v))); return v; } } diff --git a/Objects/stringobject.c b/Objects/stringobject.c --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -748,8 +748,8 @@ UTF-8 bytes may follow. */ } } - if (p-buf < newlen && _PyString_Resize(&v, p - buf)) - goto failed; + if (p-buf < newlen) + _PyString_Resize(&v, p - buf); /* v is cleared on error */ return v; failed: Py_DECREF(v); diff --git a/PC/_subprocess.c b/PC/_subprocess.c --- a/PC/_subprocess.c +++ b/PC/_subprocess.c @@ -367,7 +367,8 @@ vsize + 1 + 1; if (totalsize > PyString_GET_SIZE(out)) { int offset = p - PyString_AS_STRING(out); - _PyString_Resize(&out, totalsize + 1024); + if (_PyString_Resize(&out, totalsize + 1024)) + goto exit; p = PyString_AS_STRING(out) + offset; } memcpy(p, PyString_AS_STRING(key), ksize); @@ -383,7 +384,7 @@ _PyString_Resize(&out, p - PyString_AS_STRING(out)); /* PyObject_Print(out, stdout, 0); */ - +exit: Py_XDECREF(keys); Py_XDECREF(values); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 25 15:29:46 2014 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 25 Apr 2014 15:29:46 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzIxMjI1OiBjb3B5?= =?utf-8?q?_docstrings_from_base_classes?= Message-ID: <3gFbpL1yzBz7Ljb@mail.python.org> http://hg.python.org/cpython/rev/e33a036fd784 changeset: 90457:e33a036fd784 branch: 3.4 parent: 90451:901b9afc918e user: Andrew Kuchling date: Fri Apr 25 09:29:30 2014 -0400 summary: #21225: copy docstrings from base classes files: Lib/io.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/io.py b/Lib/io.py --- a/Lib/io.py +++ b/Lib/io.py @@ -70,16 +70,16 @@ # Method descriptions and default implementations are inherited from the C # version however. class IOBase(_io._IOBase, metaclass=abc.ABCMeta): - pass + __doc__ = _io._IOBase.__doc__ class RawIOBase(_io._RawIOBase, IOBase): - pass + __doc__ = _io._RawIOBase.__doc__ class BufferedIOBase(_io._BufferedIOBase, IOBase): - pass + __doc__ = _io._BufferedIOBase.__doc__ class TextIOBase(_io._TextIOBase, IOBase): - pass + __doc__ = _io._TextIOBase.__doc__ RawIOBase.register(FileIO) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Fri Apr 25 15:36:20 2014 From: python-checkins at python.org (andrew.kuchling) Date: Fri, 25 Apr 2014 15:36:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E4?= Message-ID: <3gFbxw5610z7Lkj@mail.python.org> http://hg.python.org/cpython/rev/9fc4a1ebe652 changeset: 90458:9fc4a1ebe652 parent: 90455:c8d8a6cc5586 parent: 90457:e33a036fd784 user: Andrew Kuchling date: Fri Apr 25 09:35:26 2014 -0400 summary: Merge from 3.4 files: Lib/io.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/io.py b/Lib/io.py --- a/Lib/io.py +++ b/Lib/io.py @@ -70,16 +70,16 @@ # Method descriptions and default implementations are inherited from the C # version however. class IOBase(_io._IOBase, metaclass=abc.ABCMeta): - pass + __doc__ = _io._IOBase.__doc__ class RawIOBase(_io._RawIOBase, IOBase): - pass + __doc__ = _io._RawIOBase.__doc__ class BufferedIOBase(_io._BufferedIOBase, IOBase): - pass + __doc__ = _io._BufferedIOBase.__doc__ class TextIOBase(_io._TextIOBase, IOBase): - pass + __doc__ = _io._TextIOBase.__doc__ RawIOBase.register(FileIO) -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sat Apr 26 10:01:33 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sat, 26 Apr 2014 10:01:33 +0200 Subject: [Python-checkins] Daily reference leaks (9fc4a1ebe652): sum=3 Message-ID: results for 9fc4a1ebe652 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogcWbG7R', '-x'] From python-checkins at python.org Sat Apr 26 14:36:08 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sat, 26 Apr 2014 14:36:08 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMjA3?= =?utf-8?q?=3A_Detect_when_the_os=2Eurandom_cached_fd_has_been_closed_or_r?= =?utf-8?q?eplaced=2C?= Message-ID: <3gGBZ04JRkz7Lk6@mail.python.org> http://hg.python.org/cpython/rev/a66524ce9551 changeset: 90459:a66524ce9551 branch: 3.4 parent: 90457:e33a036fd784 user: Antoine Pitrou date: Sat Apr 26 14:33:03 2014 +0200 summary: Issue #21207: Detect when the os.urandom cached fd has been closed or replaced, and open it anew. files: Lib/test/test_os.py | 43 ++++++++++++++++++++++++++++ Misc/NEWS | 3 ++ Python/random.c | 49 ++++++++++++++++++++++++++------ 3 files changed, 85 insertions(+), 10 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1070,6 +1070,49 @@ """ assert_python_ok('-c', code) + def test_urandom_fd_closed(self): + # Issue #21207: urandom() should reopen its fd to /dev/urandom if + # closed. + code = """if 1: + import os + import sys + os.urandom(4) + os.closerange(3, 256) + sys.stdout.buffer.write(os.urandom(4)) + """ + rc, out, err = assert_python_ok('-Sc', code) + + def test_urandom_fd_reopened(self): + # Issue #21207: urandom() should detect its fd to /dev/urandom + # changed to something else, and reopen it. + with open(support.TESTFN, 'wb') as f: + f.write(b"x" * 256) + self.addCleanup(os.unlink, support.TESTFN) + code = """if 1: + import os + import sys + os.urandom(4) + for fd in range(3, 256): + try: + os.close(fd) + except OSError: + pass + else: + # Found the urandom fd (XXX hopefully) + break + os.closerange(3, 256) + with open({TESTFN!r}, 'rb') as f: + os.dup2(f.fileno(), fd) + sys.stdout.buffer.write(os.urandom(4)) + sys.stdout.buffer.write(os.urandom(4)) + """.format(TESTFN=support.TESTFN) + rc, out, err = assert_python_ok('-Sc', code) + self.assertEqual(len(out), 8) + self.assertNotEqual(out[0:4], out[4:8]) + rc, out2, err2 = assert_python_ok('-Sc', code) + self.assertEqual(len(out2), 8) + self.assertNotEqual(out2, out) + @contextlib.contextmanager def _execvpe_mockup(defpath=None): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #21207: Detect when the os.urandom cached fd has been closed or + replaced, and open it anew. + - Issue #21291: subprocess's Popen.wait() is now thread safe so that multiple threads may be calling wait() or poll() on a Popen instance at the same time without losing the Popen.returncode value. diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -3,6 +3,9 @@ #include #else #include +#ifdef HAVE_SYS_STAT_H +#include +#endif #endif #ifdef Py_DEBUG @@ -69,7 +72,11 @@ #ifndef MS_WINDOWS -static int urandom_fd = -1; +static struct { + int fd; + dev_t st_dev; + ino_t st_ino; +} urandom_cache = { -1 }; /* Read size bytes from /dev/urandom into buffer. Call Py_FatalError() on error. */ @@ -109,12 +116,24 @@ { int fd; Py_ssize_t n; + struct stat st; if (size <= 0) return 0; - if (urandom_fd >= 0) - fd = urandom_fd; + if (urandom_cache.fd >= 0) { + /* Does the fd point to the same thing as before? (issue #21207) */ + if (fstat(urandom_cache.fd, &st) + || st.st_dev != urandom_cache.st_dev + || st.st_ino != urandom_cache.st_ino) { + /* Something changed: forget the cached fd (but don't close it, + since it probably points to something important for some + third-party code). */ + urandom_cache.fd = -1; + } + } + if (urandom_cache.fd >= 0) + fd = urandom_cache.fd; else { Py_BEGIN_ALLOW_THREADS fd = _Py_open("/dev/urandom", O_RDONLY); @@ -129,14 +148,24 @@ PyErr_SetFromErrno(PyExc_OSError); return -1; } - if (urandom_fd >= 0) { + if (urandom_cache.fd >= 0) { /* urandom_fd was initialized by another thread while we were not holding the GIL, keep it. */ close(fd); - fd = urandom_fd; + fd = urandom_cache.fd; } - else - urandom_fd = fd; + else { + if (fstat(fd, &st)) { + PyErr_SetFromErrno(PyExc_OSError); + close(fd); + return -1; + } + else { + urandom_cache.fd = fd; + urandom_cache.st_dev = st.st_dev; + urandom_cache.st_ino = st.st_ino; + } + } } Py_BEGIN_ALLOW_THREADS @@ -168,9 +197,9 @@ static void dev_urandom_close(void) { - if (urandom_fd >= 0) { - close(urandom_fd); - urandom_fd = -1; + if (urandom_cache.fd >= 0) { + close(urandom_cache.fd); + urandom_cache.fd = -1; } } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 14:36:10 2014 From: python-checkins at python.org (antoine.pitrou) Date: Sat, 26 Apr 2014 14:36:10 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321207=3A_Detect_when_the_os=2Eurandom_cached_fd?= =?utf-8?q?_has_been_closed_or_replaced=2C?= Message-ID: <3gGBZ205jVz7LkZ@mail.python.org> http://hg.python.org/cpython/rev/d3e8db93dc18 changeset: 90460:d3e8db93dc18 parent: 90458:9fc4a1ebe652 parent: 90459:a66524ce9551 user: Antoine Pitrou date: Sat Apr 26 14:35:19 2014 +0200 summary: Issue #21207: Detect when the os.urandom cached fd has been closed or replaced, and open it anew. files: Lib/test/test_os.py | 43 ++++++++++++++++++++++++++++ Misc/NEWS | 3 ++ Python/random.c | 49 ++++++++++++++++++++++++++------ 3 files changed, 85 insertions(+), 10 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1070,6 +1070,49 @@ """ assert_python_ok('-c', code) + def test_urandom_fd_closed(self): + # Issue #21207: urandom() should reopen its fd to /dev/urandom if + # closed. + code = """if 1: + import os + import sys + os.urandom(4) + os.closerange(3, 256) + sys.stdout.buffer.write(os.urandom(4)) + """ + rc, out, err = assert_python_ok('-Sc', code) + + def test_urandom_fd_reopened(self): + # Issue #21207: urandom() should detect its fd to /dev/urandom + # changed to something else, and reopen it. + with open(support.TESTFN, 'wb') as f: + f.write(b"x" * 256) + self.addCleanup(os.unlink, support.TESTFN) + code = """if 1: + import os + import sys + os.urandom(4) + for fd in range(3, 256): + try: + os.close(fd) + except OSError: + pass + else: + # Found the urandom fd (XXX hopefully) + break + os.closerange(3, 256) + with open({TESTFN!r}, 'rb') as f: + os.dup2(f.fileno(), fd) + sys.stdout.buffer.write(os.urandom(4)) + sys.stdout.buffer.write(os.urandom(4)) + """.format(TESTFN=support.TESTFN) + rc, out, err = assert_python_ok('-Sc', code) + self.assertEqual(len(out), 8) + self.assertNotEqual(out[0:4], out[4:8]) + rc, out2, err2 = assert_python_ok('-Sc', code) + self.assertEqual(len(out2), 8) + self.assertNotEqual(out2, out) + @contextlib.contextmanager def _execvpe_mockup(defpath=None): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,9 @@ Library ------- +- Issue #21207: Detect when the os.urandom cached fd has been closed or + replaced, and open it anew. + - Issue #21291: subprocess's Popen.wait() is now thread safe so that multiple threads may be calling wait() or poll() on a Popen instance at the same time without losing the Popen.returncode value. diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -3,6 +3,9 @@ #include #else #include +#ifdef HAVE_SYS_STAT_H +#include +#endif #endif #ifdef Py_DEBUG @@ -69,7 +72,11 @@ #ifndef MS_WINDOWS -static int urandom_fd = -1; +static struct { + int fd; + dev_t st_dev; + ino_t st_ino; +} urandom_cache = { -1 }; /* Read size bytes from /dev/urandom into buffer. Call Py_FatalError() on error. */ @@ -109,12 +116,24 @@ { int fd; Py_ssize_t n; + struct stat st; if (size <= 0) return 0; - if (urandom_fd >= 0) - fd = urandom_fd; + if (urandom_cache.fd >= 0) { + /* Does the fd point to the same thing as before? (issue #21207) */ + if (fstat(urandom_cache.fd, &st) + || st.st_dev != urandom_cache.st_dev + || st.st_ino != urandom_cache.st_ino) { + /* Something changed: forget the cached fd (but don't close it, + since it probably points to something important for some + third-party code). */ + urandom_cache.fd = -1; + } + } + if (urandom_cache.fd >= 0) + fd = urandom_cache.fd; else { Py_BEGIN_ALLOW_THREADS fd = _Py_open("/dev/urandom", O_RDONLY); @@ -129,14 +148,24 @@ PyErr_SetFromErrno(PyExc_OSError); return -1; } - if (urandom_fd >= 0) { + if (urandom_cache.fd >= 0) { /* urandom_fd was initialized by another thread while we were not holding the GIL, keep it. */ close(fd); - fd = urandom_fd; + fd = urandom_cache.fd; } - else - urandom_fd = fd; + else { + if (fstat(fd, &st)) { + PyErr_SetFromErrno(PyExc_OSError); + close(fd); + return -1; + } + else { + urandom_cache.fd = fd; + urandom_cache.st_dev = st.st_dev; + urandom_cache.st_ino = st.st_ino; + } + } } Py_BEGIN_ALLOW_THREADS @@ -168,9 +197,9 @@ static void dev_urandom_close(void) { - if (urandom_fd >= 0) { - close(urandom_fd); - urandom_fd = -1; + if (urandom_cache.fd >= 0) { + close(urandom_cache.fd); + urandom_cache.fd = -1; } } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 16:49:01 2014 From: python-checkins at python.org (tim.golden) Date: Sat, 26 Apr 2014 16:49:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue21349_Pas?= =?utf-8?q?sing_a_memoryview_to_=5Fwinreg=2ESetValueEx_now_correctly_raise?= =?utf-8?q?s_a?= Message-ID: <3gGFWK0w8Cz7LjW@mail.python.org> http://hg.python.org/cpython/rev/061db174baad changeset: 90461:061db174baad branch: 2.7 parent: 90456:4f79c3827adc user: Tim Golden date: Sat Apr 26 15:47:08 2014 +0100 summary: Issue21349 Passing a memoryview to _winreg.SetValueEx now correctly raises a TypeError where it previously crashed the interpreter. Patch by Brian Kearns files: Lib/test/test_winreg.py | 9 +++++++++ Misc/ACKS | 1 + PC/_winreg.c | 10 ++++++---- 3 files changed, 16 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -329,6 +329,15 @@ finally: DeleteKey(HKEY_CURRENT_USER, test_key_name) + def test_setvalueex_with_memoryview(self): + try: + with CreateKey(HKEY_CURRENT_USER, test_key_name) as ck: + self.assertNotEqual(ck.handle, 0) + with self.assertRaises(TypeError): + SetValueEx(ck, "test_name", None, REG_BINARY, memoryview('val')) + finally: + DeleteKey(HKEY_CURRENT_USER, test_key_name) + def test_queryvalueex_return_value(self): # Test for Issue #16759, return unsigned int from QueryValueEx. # Reg2Py, which gets called by QueryValueEx, was returning a value diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -657,6 +657,7 @@ Anton Kasyanov Lou Kates Hiroaki Kawai +Brian Kearns Sebastien Keim Ryan Kelly Dan Kenigsberg diff --git a/PC/_winreg.c b/PC/_winreg.c --- a/PC/_winreg.c +++ b/PC/_winreg.c @@ -888,7 +888,7 @@ else { void *src_buf; PyBufferProcs *pb = value->ob_type->tp_as_buffer; - if (pb==NULL) { + if (pb == NULL || pb->bf_getreadbuffer == NULL) { PyErr_Format(PyExc_TypeError, "Objects of type '%s' can not " "be used as binary registry values", @@ -896,9 +896,11 @@ return FALSE; } *retDataSize = (*pb->bf_getreadbuffer)(value, 0, &src_buf); - *retDataBuf = (BYTE *)PyMem_NEW(char, - *retDataSize); - if (*retDataBuf==NULL){ + if (*retDataSize < 0) { + return FALSE; + } + *retDataBuf = (BYTE *)PyMem_NEW(char, *retDataSize); + if (*retDataBuf == NULL){ PyErr_NoMemory(); return FALSE; } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 17:02:28 2014 From: python-checkins at python.org (tim.golden) Date: Sat, 26 Apr 2014 17:02:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Add_NEWS_entry?= =?utf-8?q?_for_issue21349?= Message-ID: <3gGFpr2BwJz7LjP@mail.python.org> http://hg.python.org/cpython/rev/9c38cfc7bed7 changeset: 90462:9c38cfc7bed7 branch: 2.7 user: Tim Golden date: Sat Apr 26 16:02:38 2014 +0100 summary: Add NEWS entry for issue21349 files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #21349: Passing a memoryview to _winreg.SetValueEx now correctly raises + a TypeError where it previously crashed the interpreter. Patch by Brian Kearns + - Fix arbitrary memory access in JSONDecoder.raw_decode with a negative second parameter. Bug reported by Guido Vranken. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 19:36:43 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Apr 2014 19:36:43 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_shallow_defaul?= =?utf-8?b?dHMgdG8gJ1RydWUnIG5vdCAnMScgKGNsb3NlcyAjMjEzNTUp?= Message-ID: <3gGKDq2HVZz7Ljg@mail.python.org> http://hg.python.org/cpython/rev/b7fd640fb159 changeset: 90463:b7fd640fb159 branch: 3.4 parent: 90459:a66524ce9551 user: Benjamin Peterson date: Sat Apr 26 13:36:21 2014 -0400 summary: shallow defaults to 'True' not '1' (closes #21355) Patch by Diana Clarke. files: Lib/filecmp.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/filecmp.py b/Lib/filecmp.py --- a/Lib/filecmp.py +++ b/Lib/filecmp.py @@ -36,7 +36,7 @@ f2 -- Second file name shallow -- Just check stat signature (do not read the files). - defaults to 1. + defaults to True. Return value: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 19:36:44 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Apr 2014 19:36:44 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMjEzNTUp?= Message-ID: <3gGKDr3fRCz7Ljf@mail.python.org> http://hg.python.org/cpython/rev/9ab6d13553ef changeset: 90464:9ab6d13553ef parent: 90460:d3e8db93dc18 parent: 90463:b7fd640fb159 user: Benjamin Peterson date: Sat Apr 26 13:36:35 2014 -0400 summary: merge 3.4 (#21355) files: Lib/filecmp.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/filecmp.py b/Lib/filecmp.py --- a/Lib/filecmp.py +++ b/Lib/filecmp.py @@ -36,7 +36,7 @@ f2 -- Second file name shallow -- Just check stat signature (do not read the files). - defaults to 1. + defaults to True. Return value: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 19:56:48 2014 From: python-checkins at python.org (terry.reedy) Date: Sat, 26 Apr 2014 19:56:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4OTQ0?= =?utf-8?q?=3A_backport_typo_fix?= Message-ID: <3gGKh04PPJz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/de6047ea33e6 changeset: 90465:de6047ea33e6 branch: 2.7 parent: 90462:9c38cfc7bed7 user: Terry Jan Reedy date: Sat Apr 26 13:55:59 2014 -0400 summary: Issue #18944: backport typo fix files: Lib/test/test_set.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -1615,7 +1615,7 @@ for meth in (s.union, s.intersection, s.difference, s.symmetric_difference, s.isdisjoint): for g in (G, I, Ig, L, R): expected = meth(data) - actual = meth(G(data)) + actual = meth(g(data)) if isinstance(expected, bool): self.assertEqual(actual, expected) else: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 22:51:09 2014 From: python-checkins at python.org (stefan.krah) Date: Sat, 26 Apr 2014 22:51:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE3MTQ1?= =?utf-8?q?=3A__Document_array=2Earray_buffer_interface_limitations=2E?= Message-ID: <3gGPY909hjz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/0a2ac61729d2 changeset: 90466:0a2ac61729d2 branch: 2.7 user: Stefan Krah date: Sat Apr 26 22:50:25 2014 +0200 summary: Issue #17145: Document array.array buffer interface limitations. files: Doc/c-api/buffer.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -21,8 +21,10 @@ Two examples of objects that support the buffer interface are strings and arrays. The string object exposes the character contents in the buffer -interface's byte-oriented form. An array can also expose its contents, but it -should be noted that array elements may be multi-byte values. +interface's byte-oriented form. An array can only expose its contents via the +old-style buffer interface. This limitation does not apply to Python 3, +where :class:`memoryview` objects can be constructed from arrays, too. +Array elements may be multi-byte values. An example user of the buffer interface is the file object's :meth:`write` method. Any object that can export a series of bytes through the buffer -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 22:57:20 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Apr 2014 22:57:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_make_operation?= =?utf-8?q?s_on_closed_dumb_databases_raise_a_consistent_exception_=28clos?= =?utf-8?q?es?= Message-ID: <3gGPhJ4RlJz7LjM@mail.python.org> http://hg.python.org/cpython/rev/e8343cb98cc3 changeset: 90467:e8343cb98cc3 branch: 3.4 parent: 90463:b7fd640fb159 user: Benjamin Peterson date: Sat Apr 26 16:56:52 2014 -0400 summary: make operations on closed dumb databases raise a consistent exception (closes #19385) Patch by Claudiu Popa. files: Lib/dbm/dumb.py | 12 +++++++++++ Lib/test/test_dbm_dumb.py | 29 +++++++++++++++++++++++--- Misc/NEWS | 3 ++ 3 files changed, 40 insertions(+), 4 deletions(-) diff --git a/Lib/dbm/dumb.py b/Lib/dbm/dumb.py --- a/Lib/dbm/dumb.py +++ b/Lib/dbm/dumb.py @@ -118,9 +118,14 @@ sync = _commit + def _verify_open(self): + if self._index is None: + raise error('DBM object has already been closed') + def __getitem__(self, key): if isinstance(key, str): key = key.encode('utf-8') + self._verify_open() pos, siz = self._index[key] # may raise KeyError f = _io.open(self._datfile, 'rb') f.seek(pos) @@ -173,6 +178,7 @@ val = val.encode('utf-8') elif not isinstance(val, (bytes, bytearray)): raise TypeError("values must be bytes or strings") + self._verify_open() if key not in self._index: self._addkey(key, self._addval(val)) else: @@ -200,6 +206,7 @@ def __delitem__(self, key): if isinstance(key, str): key = key.encode('utf-8') + self._verify_open() # The blocks used by the associated value are lost. del self._index[key] # XXX It's unclear why we do a _commit() here (the code always @@ -209,21 +216,26 @@ self._commit() def keys(self): + self._verify_open() return list(self._index.keys()) def items(self): + self._verify_open() return [(key, self[key]) for key in self._index.keys()] def __contains__(self, key): if isinstance(key, str): key = key.encode('utf-8') + self._verify_open() return key in self._index def iterkeys(self): + self._verify_open() return iter(self._index.keys()) __iter__ = iterkeys def __len__(self): + self._verify_open() return len(self._index) def close(self): diff --git a/Lib/test/test_dbm_dumb.py b/Lib/test/test_dbm_dumb.py --- a/Lib/test/test_dbm_dumb.py +++ b/Lib/test/test_dbm_dumb.py @@ -3,10 +3,12 @@ """ import io +import operator import os import unittest import dbm.dumb as dumbdbm from test import support +from functools import partial _fname = support.TESTFN @@ -190,12 +192,31 @@ with dumbdbm.open(_fname, 'r') as db: self.assertEqual(list(db.keys()), [b"dumbdbm context manager"]) - # This currently just raises AttributeError rather than a specific - # exception like the GNU or NDBM based implementations. See - # http://bugs.python.org/issue19385 for details. - with self.assertRaises(Exception): + with self.assertRaises(dumbdbm.error): db.keys() + def test_check_closed(self): + f = dumbdbm.open(_fname, 'c') + f.close() + + for meth in (partial(operator.delitem, f), + partial(operator.setitem, f, 'b'), + partial(operator.getitem, f), + partial(operator.contains, f)): + with self.assertRaises(dumbdbm.error) as cm: + meth('test') + self.assertEqual(str(cm.exception), + "DBM object has already been closed") + + for meth in (operator.methodcaller('keys'), + operator.methodcaller('iterkeys'), + operator.methodcaller('items'), + len): + with self.assertRaises(dumbdbm.error) as cm: + meth(f) + self.assertEqual(str(cm.exception), + "DBM object has already been closed") + def tearDown(self): _delete_files() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #19385: Make operations on a closed dbm.dumb database always raise the + same exception. + - Issue #21207: Detect when the os.urandom cached fd has been closed or replaced, and open it anew. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sat Apr 26 22:57:21 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 26 Apr 2014 22:57:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy40ICgjMTkzODUp?= Message-ID: <3gGPhK6zjQz7Lkv@mail.python.org> http://hg.python.org/cpython/rev/dbceba88b96e changeset: 90468:dbceba88b96e parent: 90464:9ab6d13553ef parent: 90467:e8343cb98cc3 user: Benjamin Peterson date: Sat Apr 26 16:57:11 2014 -0400 summary: merge 3.4 (#19385) files: Lib/dbm/dumb.py | 12 +++++++++++ Lib/test/test_dbm_dumb.py | 29 +++++++++++++++++++++++--- Misc/NEWS | 3 ++ 3 files changed, 40 insertions(+), 4 deletions(-) diff --git a/Lib/dbm/dumb.py b/Lib/dbm/dumb.py --- a/Lib/dbm/dumb.py +++ b/Lib/dbm/dumb.py @@ -118,9 +118,14 @@ sync = _commit + def _verify_open(self): + if self._index is None: + raise error('DBM object has already been closed') + def __getitem__(self, key): if isinstance(key, str): key = key.encode('utf-8') + self._verify_open() pos, siz = self._index[key] # may raise KeyError f = _io.open(self._datfile, 'rb') f.seek(pos) @@ -173,6 +178,7 @@ val = val.encode('utf-8') elif not isinstance(val, (bytes, bytearray)): raise TypeError("values must be bytes or strings") + self._verify_open() if key not in self._index: self._addkey(key, self._addval(val)) else: @@ -200,6 +206,7 @@ def __delitem__(self, key): if isinstance(key, str): key = key.encode('utf-8') + self._verify_open() # The blocks used by the associated value are lost. del self._index[key] # XXX It's unclear why we do a _commit() here (the code always @@ -209,21 +216,26 @@ self._commit() def keys(self): + self._verify_open() return list(self._index.keys()) def items(self): + self._verify_open() return [(key, self[key]) for key in self._index.keys()] def __contains__(self, key): if isinstance(key, str): key = key.encode('utf-8') + self._verify_open() return key in self._index def iterkeys(self): + self._verify_open() return iter(self._index.keys()) __iter__ = iterkeys def __len__(self): + self._verify_open() return len(self._index) def close(self): diff --git a/Lib/test/test_dbm_dumb.py b/Lib/test/test_dbm_dumb.py --- a/Lib/test/test_dbm_dumb.py +++ b/Lib/test/test_dbm_dumb.py @@ -3,10 +3,12 @@ """ import io +import operator import os import unittest import dbm.dumb as dumbdbm from test import support +from functools import partial _fname = support.TESTFN @@ -190,12 +192,31 @@ with dumbdbm.open(_fname, 'r') as db: self.assertEqual(list(db.keys()), [b"dumbdbm context manager"]) - # This currently just raises AttributeError rather than a specific - # exception like the GNU or NDBM based implementations. See - # http://bugs.python.org/issue19385 for details. - with self.assertRaises(Exception): + with self.assertRaises(dumbdbm.error): db.keys() + def test_check_closed(self): + f = dumbdbm.open(_fname, 'c') + f.close() + + for meth in (partial(operator.delitem, f), + partial(operator.setitem, f, 'b'), + partial(operator.getitem, f), + partial(operator.contains, f)): + with self.assertRaises(dumbdbm.error) as cm: + meth('test') + self.assertEqual(str(cm.exception), + "DBM object has already been closed") + + for meth in (operator.methodcaller('keys'), + operator.methodcaller('iterkeys'), + operator.methodcaller('items'), + len): + with self.assertRaises(dumbdbm.error) as cm: + meth(f) + self.assertEqual(str(cm.exception), + "DBM object has already been closed") + def tearDown(self): _delete_files() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,9 @@ Library ------- +- Issue #19385: Make operations on a closed dbm.dumb database always raise the + same exception. + - Issue #21207: Detect when the os.urandom cached fd has been closed or replaced, and open it anew. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 01:05:28 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 27 Apr 2014 01:05:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogIzE4MjQzOiBSZW1v?= =?utf-8?q?ve_obsolete_cautionary_note_from_email_mktime=5Ftz_docs=2E?= Message-ID: <3gGSX84NWpz7Ljn@mail.python.org> http://hg.python.org/cpython/rev/d24f1fb256a3 changeset: 90469:d24f1fb256a3 branch: 3.4 parent: 90467:e8343cb98cc3 user: R David Murray date: Sat Apr 26 19:01:18 2014 -0400 summary: #18243: Remove obsolete cautionary note from email mktime_tz docs. files: Doc/library/email.util.rst | 10 +++------- 1 files changed, 3 insertions(+), 7 deletions(-) diff --git a/Doc/library/email.util.rst b/Doc/library/email.util.rst --- a/Doc/library/email.util.rst +++ b/Doc/library/email.util.rst @@ -98,12 +98,9 @@ .. function:: mktime_tz(tuple) - Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC timestamp. It - the timezone item in the tuple is ``None``, assume local time. Minor - deficiency: :func:`mktime_tz` interprets the first 8 elements of *tuple* as a - local time and then compensates for the timezone difference. This may yield a - slight error around changes in daylight savings time, though not worth worrying - about for common use. + Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC + timestamp (seconds since the Epoch). If the timezone item in the + tuple is ``None``, assume local time. .. function:: formatdate(timeval=None, localtime=False, usegmt=False) @@ -210,4 +207,3 @@ .. [#] Note that the sign of the timezone offset is the opposite of the sign of the ``time.timezone`` variable for the same timezone; the latter variable follows the POSIX standard while this module follows :rfc:`2822`. - -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 01:05:29 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 27 Apr 2014 01:05:29 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2318243=3A_Remove_obsolete_cautionary_note_fro?= =?utf-8?q?m_email_mktime=5Ftz_docs=2E?= Message-ID: <3gGSX95pZ8z7Lk8@mail.python.org> http://hg.python.org/cpython/rev/462470859e57 changeset: 90470:462470859e57 parent: 90468:dbceba88b96e parent: 90469:d24f1fb256a3 user: R David Murray date: Sat Apr 26 19:01:47 2014 -0400 summary: Merge: #18243: Remove obsolete cautionary note from email mktime_tz docs. files: Doc/library/email.util.rst | 10 +++------- 1 files changed, 3 insertions(+), 7 deletions(-) diff --git a/Doc/library/email.util.rst b/Doc/library/email.util.rst --- a/Doc/library/email.util.rst +++ b/Doc/library/email.util.rst @@ -98,12 +98,9 @@ .. function:: mktime_tz(tuple) - Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC timestamp. It - the timezone item in the tuple is ``None``, assume local time. Minor - deficiency: :func:`mktime_tz` interprets the first 8 elements of *tuple* as a - local time and then compensates for the timezone difference. This may yield a - slight error around changes in daylight savings time, though not worth worrying - about for common use. + Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC + timestamp (seconds since the Epoch). If the timezone item in the + tuple is ``None``, assume local time. .. function:: formatdate(timeval=None, localtime=False, usegmt=False) @@ -210,4 +207,3 @@ .. [#] Note that the sign of the timezone offset is the opposite of the sign of the ``time.timezone`` variable for the same timezone; the latter variable follows the POSIX standard while this module follows :rfc:`2822`. - -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 01:05:31 2014 From: python-checkins at python.org (r.david.murray) Date: Sun, 27 Apr 2014 01:05:31 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MjQzOiBSZW1v?= =?utf-8?q?ve_obsolete_cautionary_note_from_email_mktime=5Ftz_docs=2E?= Message-ID: <3gGSXC0fdjz7Lkx@mail.python.org> http://hg.python.org/cpython/rev/d1a641ecbe33 changeset: 90471:d1a641ecbe33 branch: 2.7 parent: 90466:0a2ac61729d2 user: R David Murray date: Sat Apr 26 19:05:03 2014 -0400 summary: #18243: Remove obsolete cautionary note from email mktime_tz docs. files: Doc/library/email.util.rst | 10 +++------- 1 files changed, 3 insertions(+), 7 deletions(-) diff --git a/Doc/library/email.util.rst b/Doc/library/email.util.rst --- a/Doc/library/email.util.rst +++ b/Doc/library/email.util.rst @@ -76,12 +76,9 @@ .. function:: mktime_tz(tuple) - Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC timestamp. It - the timezone item in the tuple is ``None``, assume local time. Minor - deficiency: :func:`mktime_tz` interprets the first 8 elements of *tuple* as a - local time and then compensates for the timezone difference. This may yield a - slight error around changes in daylight savings time, though not worth worrying - about for common use. + Turn a 10-tuple as returned by :func:`parsedate_tz` into a UTC + timestamp (seconds since the Epoch). If the timezone item in the + tuple is ``None``, assume local time. .. function:: formatdate([timeval[, localtime][, usegmt]]) @@ -165,4 +162,3 @@ .. [#] Note that the sign of the timezone offset is the opposite of the sign of the ``time.timezone`` variable for the same timezone; the latter variable follows the POSIX standard while this module follows :rfc:`2822`. - -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Sun Apr 27 10:00:06 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Sun, 27 Apr 2014 10:00:06 +0200 Subject: [Python-checkins] Daily reference leaks (462470859e57): sum=7 Message-ID: results for 462470859e57 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 2, 0] references, sum=2 test_site leaked [0, 2, 0] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogmgqrWt', '-x'] From python-checkins at python.org Sun Apr 27 16:44:54 2014 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 27 Apr 2014 16:44:54 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_say_how_to_run_one_test_?= =?utf-8?b?KGNsb3NlcyAjMjEzNjEp?= Message-ID: <3gGsN60l7zz7LjN@mail.python.org> http://hg.python.org/devguide/rev/6b912c90de72 changeset: 690:6b912c90de72 user: Benjamin Peterson date: Sun Apr 27 10:44:46 2014 -0400 summary: say how to run one test (closes #21361) Patch from Jessica McKellar. files: runtests.rst | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/runtests.rst b/runtests.rst --- a/runtests.rst +++ b/runtests.rst @@ -39,12 +39,17 @@ ./python -m test -h -If you want to run a single test, simply specify the test name as an argument. -You also probably want to enable verbose mode (using ``-v``), so that individual -failures are detailed:: +If you want to run a single test file, simply specify the test file name +(without the extension) as an argument. You also probably want to enable +verbose mode (using ``-v``), so that individual failures are detailed:: ./python -m test -v test_abc +To run a single test case, use the ``unittest`` module, providing the import +path to the test case:: + + ./python -m unittest -v test.test_abc.TestABC + If you have a multi-core or multi-CPU machine, you can enable parallel testing using several Python processes so as to speed up things:: -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Sun Apr 27 17:37:07 2014 From: python-checkins at python.org (tim.golden) Date: Sun, 27 Apr 2014 17:37:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzkyOTEg?= =?utf-8?q?Do_not_attempt_to_re-encode_mimetype_data_read_from_registry_in?= Message-ID: <3gGtXM5cYMz7LkZ@mail.python.org> http://hg.python.org/cpython/rev/18cfc2a42772 changeset: 90472:18cfc2a42772 branch: 2.7 user: Tim Golden date: Sun Apr 27 16:36:47 2014 +0100 summary: Issue #9291 Do not attempt to re-encode mimetype data read from registry in ANSI mode. Initial patches by Dmitry Jemerov & Vladimir Iofik files: Lib/mimetypes.py | 38 +++++++------------ Lib/test/test_mimetypes.py | 50 ++++++++++++++++++++++--- 2 files changed, 58 insertions(+), 30 deletions(-) diff --git a/Lib/mimetypes.py b/Lib/mimetypes.py --- a/Lib/mimetypes.py +++ b/Lib/mimetypes.py @@ -242,38 +242,28 @@ i = 0 while True: try: - ctype = _winreg.EnumKey(mimedb, i) + yield _winreg.EnumKey(mimedb, i) except EnvironmentError: break - try: - ctype = ctype.encode(default_encoding) # omit in 3.x! - except UnicodeEncodeError: - pass - else: - yield ctype i += 1 - default_encoding = sys.getdefaultencoding() with _winreg.OpenKey(_winreg.HKEY_CLASSES_ROOT, '') as hkcr: for subkeyname in enum_types(hkcr): - try: - with _winreg.OpenKey(hkcr, subkeyname) as subkey: - # Only check file extensions - if not subkeyname.startswith("."): - continue - # raises EnvironmentError if no 'Content Type' value + # Only check file extensions, not all possible classes + if not subkeyname.startswith("."): + continue + + with _winreg.OpenKey(hkcr, subkeyname) as subkey: + # If there is no "Content Type" value, or if it is not + # a simple string, simply skip + try: mimetype, datatype = _winreg.QueryValueEx( subkey, 'Content Type') - if datatype != _winreg.REG_SZ: - continue - try: - mimetype = mimetype.encode(default_encoding) - subkeyname = subkeyname.encode(default_encoding) - except UnicodeEncodeError: - continue - self.add_type(mimetype, subkeyname, strict) - except EnvironmentError: - continue + except EnvironmentError: + continue + if datatype != _winreg.REG_SZ: + continue + self.add_type(mimetype, subkeyname, strict) def guess_type(url, strict=True): """Guess the type of a file based on its URL. diff --git a/Lib/test/test_mimetypes.py b/Lib/test/test_mimetypes.py --- a/Lib/test/test_mimetypes.py +++ b/Lib/test/test_mimetypes.py @@ -71,8 +71,6 @@ # ensure all entries actually come from the Windows registry self.original_types_map = mimetypes.types_map.copy() mimetypes.types_map.clear() - mimetypes.init() - self.db = mimetypes.MimeTypes() def tearDown(self): # restore default settings @@ -84,14 +82,54 @@ # Windows registry is undocumented AFAIK. # Use file types that should *always* exist: eq = self.assertEqual - eq(self.db.guess_type("foo.txt"), ("text/plain", None)) - eq(self.db.guess_type("image.jpg"), ("image/jpeg", None)) - eq(self.db.guess_type("image.png"), ("image/png", None)) + mimetypes.init() + db = mimetypes.MimeTypes() + eq(db.guess_type("foo.txt"), ("text/plain", None)) + eq(db.guess_type("image.jpg"), ("image/jpeg", None)) + eq(db.guess_type("image.png"), ("image/png", None)) + + def test_non_latin_extension(self): + import _winreg + + class MockWinreg(object): + def __getattr__(self, name): + if name == 'EnumKey': + return lambda key, i: _winreg.EnumKey(key, i) + "\xa3" + elif name == "OpenKey": + return lambda key, name: _winreg.OpenKey(key, name.rstrip("\xa3")) + elif name == 'QueryValueEx': + return lambda subkey, label: (label + "\xa3", _winreg.REG_SZ) + return getattr(_winreg, name) + + mimetypes._winreg = MockWinreg() + try: + # this used to throw an exception if registry contained non-Latin + # characters in extensions (issue #9291) + mimetypes.init() + finally: + mimetypes._winreg = _winreg + + def test_non_latin_type(self): + import _winreg + + class MockWinreg(object): + def __getattr__(self, name): + if name == 'QueryValueEx': + return lambda subkey, label: (label + "\xa3", _winreg.REG_SZ) + return getattr(_winreg, name) + + mimetypes._winreg = MockWinreg() + try: + # this used to throw an exception if registry contained non-Latin + # characters in content types (issue #9291) + mimetypes.init() + finally: + mimetypes._winreg = _winreg def test_main(): test_support.run_unittest(MimeTypesTestCase, Win32MimeTypesTestCase - ) + ) if __name__ == "__main__": -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 17:39:48 2014 From: python-checkins at python.org (tim.golden) Date: Sun, 27 Apr 2014 17:39:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzkyOTEg?= =?utf-8?q?Add_ACKS_=26_NEWS?= Message-ID: <3gGtbS6blZz7Lkg@mail.python.org> http://hg.python.org/cpython/rev/0c8a7299c7e3 changeset: 90473:0c8a7299c7e3 branch: 2.7 user: Tim Golden date: Sun Apr 27 16:39:33 2014 +0100 summary: Issue #9291 Add ACKS & NEWS files: Misc/ACKS | 2 ++ Misc/NEWS | 3 +++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -593,6 +593,7 @@ Peter Ingebretson Tony Ingraldi John Interrante +Vladimir Iofik Bob Ippolito Roger Irwin Atsuo Ishimoto @@ -618,6 +619,7 @@ Philip Jenvey MunSic Jeong Chris Jerdonek +Dmitry Jeremov Jim Jewett Pedro Diaz Jimenez Orjan Johansen diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #9291: Do not attempt to re-encode mimetype data read from registry in + ANSI mode. Initial patches by Dmitry Jemerov & Vladimir Iofik. + - Issue #21349: Passing a memoryview to _winreg.SetValueEx now correctly raises a TypeError where it previously crashed the interpreter. Patch by Brian Kearns -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:00:30 2014 From: python-checkins at python.org (tim.golden) Date: Sun, 27 Apr 2014 19:00:30 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318314_os=2Eunlink?= =?utf-8?q?_will_now_remove_junction_points_on_Windows=2E_Patch_by_Kim?= Message-ID: <3gGwNZ0nprz7Lm7@mail.python.org> http://hg.python.org/cpython/rev/17df50df62c7 changeset: 90474:17df50df62c7 parent: 90470:462470859e57 user: Tim Golden date: Sun Apr 27 18:00:10 2014 +0100 summary: Issue #18314 os.unlink will now remove junction points on Windows. Patch by Kim Gr?sman. files: Lib/test/test_os.py | 36 +++++++++ Misc/ACKS | 1 + Modules/_winapi.c | 109 ++++++++++++++++++++++++++++++ Modules/posixmodule.c | 45 ++--------- Modules/winreparse.h | 53 ++++++++++++++ 5 files changed, 208 insertions(+), 36 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -39,6 +39,10 @@ import fcntl except ImportError: fcntl = None +try: + import _winapi +except ImportError: + _winapi = None from test.script_helper import assert_python_ok @@ -1773,6 +1777,37 @@ shutil.rmtree(level1) + at unittest.skipUnless(sys.platform == "win32", "Win32 specific tests") +class Win32JunctionTests(unittest.TestCase): + junction = 'junctiontest' + junction_target = os.path.dirname(os.path.abspath(__file__)) + + def setUp(self): + assert os.path.exists(self.junction_target) + assert not os.path.exists(self.junction) + + def tearDown(self): + if os.path.exists(self.junction): + # os.rmdir delegates to Windows' RemoveDirectoryW, + # which removes junction points safely. + os.rmdir(self.junction) + + def test_create_junction(self): + _winapi.CreateJunction(self.junction_target, self.junction) + self.assertTrue(os.path.exists(self.junction)) + self.assertTrue(os.path.isdir(self.junction)) + + # Junctions are not recognized as links. + self.assertFalse(os.path.islink(self.junction)) + + def test_unlink_removes_junction(self): + _winapi.CreateJunction(self.junction_target, self.junction) + self.assertTrue(os.path.exists(self.junction)) + + os.unlink(self.junction) + self.assertFalse(os.path.exists(self.junction)) + + @support.skip_unless_symlink class NonLocalSymlinkTests(unittest.TestCase): @@ -2544,6 +2579,7 @@ RemoveDirsTests, CPUCountTests, FDInheritanceTests, + Win32JunctionTests, ) if __name__ == "__main__": diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -468,6 +468,7 @@ Shelley Gooch David Goodger Hans de Graaff +Kim Gr?sman Nathaniel Gray Eddy De Greef Grant Griffin diff --git a/Modules/_winapi.c b/Modules/_winapi.c --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -40,6 +40,7 @@ #define WINDOWS_LEAN_AND_MEAN #include "windows.h" #include +#include "winreparse.h" #if defined(MS_WIN32) && !defined(MS_WIN64) #define HANDLE_TO_PYNUM(handle) \ @@ -401,6 +402,112 @@ } static PyObject * +winapi_CreateJunction(PyObject *self, PyObject *args) +{ + /* Input arguments */ + LPWSTR src_path = NULL; + LPWSTR dst_path = NULL; + + /* Privilege adjustment */ + HANDLE token = NULL; + TOKEN_PRIVILEGES tp; + + /* Reparse data buffer */ + const USHORT prefix_len = 4; + USHORT print_len = 0; + USHORT rdb_size = 0; + PREPARSE_DATA_BUFFER rdb = NULL; + + /* Junction point creation */ + HANDLE junction = NULL; + DWORD ret = 0; + + if (!PyArg_ParseTuple(args, "uu", + &src_path, &dst_path)) + return NULL; + + if (memcmp(src_path, L"\\??\\", prefix_len * sizeof(WCHAR)) == 0) + return PyErr_SetFromWindowsErr(ERROR_INVALID_PARAMETER); + + /* Adjust privileges to allow rewriting directory entry as a + junction point. */ + if (!OpenProcessToken(GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES, &token)) + goto cleanup; + + if (!LookupPrivilegeValue(NULL, SE_RESTORE_NAME, &tp.Privileges[0].Luid)) + goto cleanup; + + tp.PrivilegeCount = 1; + tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED; + if (!AdjustTokenPrivileges(token, FALSE, &tp, sizeof(TOKEN_PRIVILEGES), + NULL, NULL)) + goto cleanup; + + if (GetFileAttributesW(src_path) == INVALID_FILE_ATTRIBUTES) + goto cleanup; + + print_len = (USHORT)GetFullPathNameW(src_path, 0, NULL, NULL); + if (print_len == 0) + goto cleanup; + + /* NUL terminator should not be part of print_len */ + --print_len; + + rdb_size = REPARSE_DATA_BUFFER_HEADER_SIZE + + sizeof(rdb->MountPointReparseBuffer) - + sizeof(rdb->MountPointReparseBuffer.PathBuffer) + + /* Two +1's for NUL terminators. */ + (prefix_len + print_len + 1 + print_len + 1) * sizeof(WCHAR); + rdb = (PREPARSE_DATA_BUFFER)PyMem_RawMalloc(rdb_size, 1); + + rdb->ReparseTag = IO_REPARSE_TAG_MOUNT_POINT; + rdb->ReparseDataLength = rdb_size - REPARSE_DATA_BUFFER_HEADER_SIZE; + rdb->MountPointReparseBuffer.SubstituteNameOffset = 0; + rdb->MountPointReparseBuffer.SubstituteNameLength = + (prefix_len + print_len) * sizeof(WCHAR); + rdb->MountPointReparseBuffer.PrintNameOffset = + rdb->MountPointReparseBuffer.SubstituteNameLength + sizeof(WCHAR); + rdb->MountPointReparseBuffer.PrintNameLength = print_len * sizeof(WCHAR); + + lstrcpyW(rdb->MountPointReparseBuffer.PathBuffer, L"\\??\\"); + if (GetFullPathNameW(src_path, print_len + 1, + rdb->MountPointReparseBuffer.PathBuffer + prefix_len, + NULL) == 0) + goto cleanup; + + lstrcpyW(rdb->MountPointReparseBuffer.PathBuffer + + prefix_len + print_len + 1, + rdb->MountPointReparseBuffer.PathBuffer + prefix_len); + + /* Create a directory for the junction point. */ + if (!CreateDirectoryW(dst_path, NULL)) + goto cleanup; + + junction = CreateFileW(dst_path, GENERIC_READ | GENERIC_WRITE, 0, NULL, + OPEN_EXISTING, + FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTICS, NULL); + if (junction == INVALID_HANDLE_VALUE) + goto cleanup; + + /* Make the directory entry a junction point. */ + if (!DeviceIoControl(junction, FSCTL_SET_REPARSE_POINT, rdb, rdb_size, + NULL, 0, &ret, NULL)) + goto cleanup; + +cleanup: + ret = GetLastError(); + + CloseHandle(token); + CloseHandle(junction); + free(rdb); + + if (ret != 0) + return PyErr_SetFromWindowsErr(ret); + + Py_RETURN_NONE; +} + +static PyObject * winapi_CreateNamedPipe(PyObject *self, PyObject *args) { LPCTSTR lpName; @@ -1225,6 +1332,8 @@ METH_VARARGS | METH_KEYWORDS, ""}, {"CreateFile", winapi_CreateFile, METH_VARARGS, ""}, + {"CreateJunction", winapi_CreateJunction, METH_VARARGS, + ""}, {"CreateNamedPipe", winapi_CreateNamedPipe, METH_VARARGS, ""}, {"CreatePipe", winapi_CreatePipe, METH_VARARGS, diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -27,6 +27,8 @@ #include "Python.h" #ifndef MS_WINDOWS #include "posixmodule.h" +#else +#include "winreparse.h" #endif #ifdef __cplusplus @@ -301,6 +303,9 @@ #ifndef IO_REPARSE_TAG_SYMLINK #define IO_REPARSE_TAG_SYMLINK (0xA000000CL) #endif +#ifndef IO_REPARSE_TAG_MOUNT_POINT +#define IO_REPARSE_TAG_MOUNT_POINT (0xA0000003L) +#endif #include "osdefs.h" #include #include @@ -1109,41 +1114,6 @@ #endif #ifdef MS_WINDOWS -/* The following structure was copied from - http://msdn.microsoft.com/en-us/library/ms791514.aspx as the required - include doesn't seem to be present in the Windows SDK (at least as included - with Visual Studio Express). */ -typedef struct _REPARSE_DATA_BUFFER { - ULONG ReparseTag; - USHORT ReparseDataLength; - USHORT Reserved; - union { - struct { - USHORT SubstituteNameOffset; - USHORT SubstituteNameLength; - USHORT PrintNameOffset; - USHORT PrintNameLength; - ULONG Flags; - WCHAR PathBuffer[1]; - } SymbolicLinkReparseBuffer; - - struct { - USHORT SubstituteNameOffset; - USHORT SubstituteNameLength; - USHORT PrintNameOffset; - USHORT PrintNameLength; - WCHAR PathBuffer[1]; - } MountPointReparseBuffer; - - struct { - UCHAR DataBuffer[1]; - } GenericReparseBuffer; - }; -} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; - -#define REPARSE_DATA_BUFFER_HEADER_SIZE FIELD_OFFSET(REPARSE_DATA_BUFFER,\ - GenericReparseBuffer) -#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE ( 16 * 1024 ) static int win32_get_reparse_tag(HANDLE reparse_point_handle, ULONG *reparse_tag) @@ -4492,7 +4462,10 @@ find_data_handle = FindFirstFileW(lpFileName, &find_data); if(find_data_handle != INVALID_HANDLE_VALUE) { - is_link = find_data.dwReserved0 == IO_REPARSE_TAG_SYMLINK; + /* IO_REPARSE_TAG_SYMLINK if it is a symlink and + IO_REPARSE_TAG_MOUNT_POINT if it is a junction point. */ + is_link = find_data.dwReserved0 == IO_REPARSE_TAG_SYMLINK || + find_data.dwReserved0 == IO_REPARSE_TAG_MOUNT_POINT; FindClose(find_data_handle); } } diff --git a/Modules/winreparse.h b/Modules/winreparse.h new file mode 100644 --- /dev/null +++ b/Modules/winreparse.h @@ -0,0 +1,53 @@ +#ifndef Py_WINREPARSE_H +#define Py_WINREPARSE_H + +#ifdef MS_WINDOWS +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* The following structure was copied from + http://msdn.microsoft.com/en-us/library/ff552012.aspx as the required + include doesn't seem to be present in the Windows SDK (at least as included + with Visual Studio Express). */ +typedef struct _REPARSE_DATA_BUFFER { + ULONG ReparseTag; + USHORT ReparseDataLength; + USHORT Reserved; + union { + struct { + USHORT SubstituteNameOffset; + USHORT SubstituteNameLength; + USHORT PrintNameOffset; + USHORT PrintNameLength; + ULONG Flags; + WCHAR PathBuffer[1]; + } SymbolicLinkReparseBuffer; + + struct { + USHORT SubstituteNameOffset; + USHORT SubstituteNameLength; + USHORT PrintNameOffset; + USHORT PrintNameLength; + WCHAR PathBuffer[1]; + } MountPointReparseBuffer; + + struct { + UCHAR DataBuffer[1]; + } GenericReparseBuffer; + }; +} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; + +#define REPARSE_DATA_BUFFER_HEADER_SIZE FIELD_OFFSET(REPARSE_DATA_BUFFER,\ + GenericReparseBuffer) +#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE ( 16 * 1024 ) + +#ifdef __cplusplus +} +#endif + +#endif /* MS_WINDOWS */ + +#endif /* !Py_WINREPARSE_H */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:03:02 2014 From: python-checkins at python.org (tim.golden) Date: Sun, 27 Apr 2014 19:03:02 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318314_Add_NEWS_it?= =?utf-8?b?ZW0u?= Message-ID: <3gGwRV0gMsz7Lky@mail.python.org> http://hg.python.org/cpython/rev/4b97092aa4bd changeset: 90475:4b97092aa4bd user: Tim Golden date: Sun Apr 27 18:02:35 2014 +0100 summary: Issue #18314 Add NEWS item. files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,9 @@ Library ------- +- Issue #18314: os.unlink now removes junction points on Windows. + Kim Gr?sman + - Issue #19385: Make operations on a closed dbm.dumb database always raise the same exception. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:34:38 2014 From: python-checkins at python.org (barry.warsaw) Date: Sun, 27 Apr 2014 19:34:38 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Additional_clarifications_bas?= =?utf-8?q?ed_on_discussion_amongst_the_PEP_authors=2E?= Message-ID: <3gGx7y5S0sz7Lmv@mail.python.org> http://hg.python.org/peps/rev/559bba5f24c0 changeset: 5468:559bba5f24c0 user: Barry Warsaw date: Sun Apr 27 13:34:33 2014 -0400 summary: Additional clarifications based on discussion amongst the PEP authors. files: pep-0008.txt | 58 +++++++++++++++++++++++++++++++++------ 1 files changed, 49 insertions(+), 9 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -78,14 +78,14 @@ Continuation lines should align wrapped elements either vertically using Python's implicit line joining inside parentheses, brackets and -braces, or using a hanging indent. When using a hanging indent the -following considerations should be applied; there should be no -arguments on the first line and further indentation should be used to -clearly distinguish itself as a continuation line. +braces, or using a *hanging indent* [#fn-hi]_. When using a hanging +indent the following considerations should be applied; there should be +no arguments on the first line and further indentation should be used +to clearly distinguish itself as a continuation line. Yes:: - # Aligned with opening delimiter + # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) @@ -95,25 +95,54 @@ var_four): print(var_one) + # Hanging indents should add a level. + foo = long_function_name( + var_one, var_two, + var_three, var_four) + No:: - # Arguments on first line forbidden when not using vertical alignment + # Arguments on first line forbidden when not using vertical alignment. foo = long_function_name(var_one, var_two, var_three, var_four) - # Further indentation required as indentation is not distinguishable + # Further indentation required as indentation is not distinguishable. def long_function_name( var_one, var_two, var_three, var_four): print(var_one) +The 4-space rule is optional for indentation lines. + Optional:: - # Extra indentation is not necessary. + # Hanging indents *may* be indented to other than 4 spaces. foo = long_function_name( var_one, var_two, var_three, var_four) +This PEP explicitly takes no position on how or whether to further +visually distinguish continuation lines after multi-line ``if`` +statements. Some acceptable options include:: + + # No extra indentation. + if (this + and that): + do_something() + + # Add a comment, which will provide some distinction in editors + # supporting syntax highlighting. + if (this + and that): + # Since both conditions are true, we can frobnicate. + do_something() + + + # Add some extra indentation on the conditional continuation line. + if (this + and that): + do_something() + The closing brace/bracket/parenthesis on multi-line constructs may either line up under the first non-whitespace character of the last line of list, as in:: @@ -183,7 +212,7 @@ issue, it is okay to increase the nominal line length from 80 to 100 characters (effectively increasing the maximum length to 99 characters), provided that comments and docstrings are still wrapped -at 72 characters. +at 72 characters. The Python standard library is conservative and requires limiting lines to 79 characters (and docstrings/comments to 72). @@ -1155,6 +1184,16 @@ automated utilities could be supported. +.. rubric:: Footnotes + +.. [#fn-hi] *Hanging indentation* is a type-setting style where all + the lines in a paragraph are indented except the first line. In + the context of Python, the term is used to describe a style where + the opening parenthesis of a parenthesized statement is the last + non-whitespace character of the line, with subsequent lines being + indented until the closing parenthesis. + + References ========== @@ -1168,6 +1207,7 @@ .. [4] PEP 8 modernisation, July 2013 http://bugs.python.org/issue18472 + Copyright ========= -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Apr 27 19:36:57 2014 From: python-checkins at python.org (tim.golden) Date: Sun, 27 Apr 2014 19:36:57 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset=3A_17?= =?utf-8?q?df50df62c7?= Message-ID: <3gGxBd4DYpz7Lk2@mail.python.org> http://hg.python.org/cpython/rev/afbfccaab27c changeset: 90476:afbfccaab27c user: Tim Golden date: Sun Apr 27 18:35:36 2014 +0100 summary: Backed out changeset: 17df50df62c7 files: Lib/test/test_os.py | 36 --------- Misc/ACKS | 1 - Modules/_winapi.c | 109 ------------------------------ Modules/posixmodule.c | 45 +++++++++-- Modules/winreparse.h | 53 -------------- 5 files changed, 36 insertions(+), 208 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -39,10 +39,6 @@ import fcntl except ImportError: fcntl = None -try: - import _winapi -except ImportError: - _winapi = None from test.script_helper import assert_python_ok @@ -1777,37 +1773,6 @@ shutil.rmtree(level1) - at unittest.skipUnless(sys.platform == "win32", "Win32 specific tests") -class Win32JunctionTests(unittest.TestCase): - junction = 'junctiontest' - junction_target = os.path.dirname(os.path.abspath(__file__)) - - def setUp(self): - assert os.path.exists(self.junction_target) - assert not os.path.exists(self.junction) - - def tearDown(self): - if os.path.exists(self.junction): - # os.rmdir delegates to Windows' RemoveDirectoryW, - # which removes junction points safely. - os.rmdir(self.junction) - - def test_create_junction(self): - _winapi.CreateJunction(self.junction_target, self.junction) - self.assertTrue(os.path.exists(self.junction)) - self.assertTrue(os.path.isdir(self.junction)) - - # Junctions are not recognized as links. - self.assertFalse(os.path.islink(self.junction)) - - def test_unlink_removes_junction(self): - _winapi.CreateJunction(self.junction_target, self.junction) - self.assertTrue(os.path.exists(self.junction)) - - os.unlink(self.junction) - self.assertFalse(os.path.exists(self.junction)) - - @support.skip_unless_symlink class NonLocalSymlinkTests(unittest.TestCase): @@ -2579,7 +2544,6 @@ RemoveDirsTests, CPUCountTests, FDInheritanceTests, - Win32JunctionTests, ) if __name__ == "__main__": diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -468,7 +468,6 @@ Shelley Gooch David Goodger Hans de Graaff -Kim Gr?sman Nathaniel Gray Eddy De Greef Grant Griffin diff --git a/Modules/_winapi.c b/Modules/_winapi.c --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -40,7 +40,6 @@ #define WINDOWS_LEAN_AND_MEAN #include "windows.h" #include -#include "winreparse.h" #if defined(MS_WIN32) && !defined(MS_WIN64) #define HANDLE_TO_PYNUM(handle) \ @@ -402,112 +401,6 @@ } static PyObject * -winapi_CreateJunction(PyObject *self, PyObject *args) -{ - /* Input arguments */ - LPWSTR src_path = NULL; - LPWSTR dst_path = NULL; - - /* Privilege adjustment */ - HANDLE token = NULL; - TOKEN_PRIVILEGES tp; - - /* Reparse data buffer */ - const USHORT prefix_len = 4; - USHORT print_len = 0; - USHORT rdb_size = 0; - PREPARSE_DATA_BUFFER rdb = NULL; - - /* Junction point creation */ - HANDLE junction = NULL; - DWORD ret = 0; - - if (!PyArg_ParseTuple(args, "uu", - &src_path, &dst_path)) - return NULL; - - if (memcmp(src_path, L"\\??\\", prefix_len * sizeof(WCHAR)) == 0) - return PyErr_SetFromWindowsErr(ERROR_INVALID_PARAMETER); - - /* Adjust privileges to allow rewriting directory entry as a - junction point. */ - if (!OpenProcessToken(GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES, &token)) - goto cleanup; - - if (!LookupPrivilegeValue(NULL, SE_RESTORE_NAME, &tp.Privileges[0].Luid)) - goto cleanup; - - tp.PrivilegeCount = 1; - tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED; - if (!AdjustTokenPrivileges(token, FALSE, &tp, sizeof(TOKEN_PRIVILEGES), - NULL, NULL)) - goto cleanup; - - if (GetFileAttributesW(src_path) == INVALID_FILE_ATTRIBUTES) - goto cleanup; - - print_len = (USHORT)GetFullPathNameW(src_path, 0, NULL, NULL); - if (print_len == 0) - goto cleanup; - - /* NUL terminator should not be part of print_len */ - --print_len; - - rdb_size = REPARSE_DATA_BUFFER_HEADER_SIZE + - sizeof(rdb->MountPointReparseBuffer) - - sizeof(rdb->MountPointReparseBuffer.PathBuffer) + - /* Two +1's for NUL terminators. */ - (prefix_len + print_len + 1 + print_len + 1) * sizeof(WCHAR); - rdb = (PREPARSE_DATA_BUFFER)PyMem_RawMalloc(rdb_size, 1); - - rdb->ReparseTag = IO_REPARSE_TAG_MOUNT_POINT; - rdb->ReparseDataLength = rdb_size - REPARSE_DATA_BUFFER_HEADER_SIZE; - rdb->MountPointReparseBuffer.SubstituteNameOffset = 0; - rdb->MountPointReparseBuffer.SubstituteNameLength = - (prefix_len + print_len) * sizeof(WCHAR); - rdb->MountPointReparseBuffer.PrintNameOffset = - rdb->MountPointReparseBuffer.SubstituteNameLength + sizeof(WCHAR); - rdb->MountPointReparseBuffer.PrintNameLength = print_len * sizeof(WCHAR); - - lstrcpyW(rdb->MountPointReparseBuffer.PathBuffer, L"\\??\\"); - if (GetFullPathNameW(src_path, print_len + 1, - rdb->MountPointReparseBuffer.PathBuffer + prefix_len, - NULL) == 0) - goto cleanup; - - lstrcpyW(rdb->MountPointReparseBuffer.PathBuffer + - prefix_len + print_len + 1, - rdb->MountPointReparseBuffer.PathBuffer + prefix_len); - - /* Create a directory for the junction point. */ - if (!CreateDirectoryW(dst_path, NULL)) - goto cleanup; - - junction = CreateFileW(dst_path, GENERIC_READ | GENERIC_WRITE, 0, NULL, - OPEN_EXISTING, - FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTICS, NULL); - if (junction == INVALID_HANDLE_VALUE) - goto cleanup; - - /* Make the directory entry a junction point. */ - if (!DeviceIoControl(junction, FSCTL_SET_REPARSE_POINT, rdb, rdb_size, - NULL, 0, &ret, NULL)) - goto cleanup; - -cleanup: - ret = GetLastError(); - - CloseHandle(token); - CloseHandle(junction); - free(rdb); - - if (ret != 0) - return PyErr_SetFromWindowsErr(ret); - - Py_RETURN_NONE; -} - -static PyObject * winapi_CreateNamedPipe(PyObject *self, PyObject *args) { LPCTSTR lpName; @@ -1332,8 +1225,6 @@ METH_VARARGS | METH_KEYWORDS, ""}, {"CreateFile", winapi_CreateFile, METH_VARARGS, ""}, - {"CreateJunction", winapi_CreateJunction, METH_VARARGS, - ""}, {"CreateNamedPipe", winapi_CreateNamedPipe, METH_VARARGS, ""}, {"CreatePipe", winapi_CreatePipe, METH_VARARGS, diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -27,8 +27,6 @@ #include "Python.h" #ifndef MS_WINDOWS #include "posixmodule.h" -#else -#include "winreparse.h" #endif #ifdef __cplusplus @@ -303,9 +301,6 @@ #ifndef IO_REPARSE_TAG_SYMLINK #define IO_REPARSE_TAG_SYMLINK (0xA000000CL) #endif -#ifndef IO_REPARSE_TAG_MOUNT_POINT -#define IO_REPARSE_TAG_MOUNT_POINT (0xA0000003L) -#endif #include "osdefs.h" #include #include @@ -1114,6 +1109,41 @@ #endif #ifdef MS_WINDOWS +/* The following structure was copied from + http://msdn.microsoft.com/en-us/library/ms791514.aspx as the required + include doesn't seem to be present in the Windows SDK (at least as included + with Visual Studio Express). */ +typedef struct _REPARSE_DATA_BUFFER { + ULONG ReparseTag; + USHORT ReparseDataLength; + USHORT Reserved; + union { + struct { + USHORT SubstituteNameOffset; + USHORT SubstituteNameLength; + USHORT PrintNameOffset; + USHORT PrintNameLength; + ULONG Flags; + WCHAR PathBuffer[1]; + } SymbolicLinkReparseBuffer; + + struct { + USHORT SubstituteNameOffset; + USHORT SubstituteNameLength; + USHORT PrintNameOffset; + USHORT PrintNameLength; + WCHAR PathBuffer[1]; + } MountPointReparseBuffer; + + struct { + UCHAR DataBuffer[1]; + } GenericReparseBuffer; + }; +} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; + +#define REPARSE_DATA_BUFFER_HEADER_SIZE FIELD_OFFSET(REPARSE_DATA_BUFFER,\ + GenericReparseBuffer) +#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE ( 16 * 1024 ) static int win32_get_reparse_tag(HANDLE reparse_point_handle, ULONG *reparse_tag) @@ -4462,10 +4492,7 @@ find_data_handle = FindFirstFileW(lpFileName, &find_data); if(find_data_handle != INVALID_HANDLE_VALUE) { - /* IO_REPARSE_TAG_SYMLINK if it is a symlink and - IO_REPARSE_TAG_MOUNT_POINT if it is a junction point. */ - is_link = find_data.dwReserved0 == IO_REPARSE_TAG_SYMLINK || - find_data.dwReserved0 == IO_REPARSE_TAG_MOUNT_POINT; + is_link = find_data.dwReserved0 == IO_REPARSE_TAG_SYMLINK; FindClose(find_data_handle); } } diff --git a/Modules/winreparse.h b/Modules/winreparse.h deleted file mode 100644 --- a/Modules/winreparse.h +++ /dev/null @@ -1,53 +0,0 @@ -#ifndef Py_WINREPARSE_H -#define Py_WINREPARSE_H - -#ifdef MS_WINDOWS -#include - -#ifdef __cplusplus -extern "C" { -#endif - -/* The following structure was copied from - http://msdn.microsoft.com/en-us/library/ff552012.aspx as the required - include doesn't seem to be present in the Windows SDK (at least as included - with Visual Studio Express). */ -typedef struct _REPARSE_DATA_BUFFER { - ULONG ReparseTag; - USHORT ReparseDataLength; - USHORT Reserved; - union { - struct { - USHORT SubstituteNameOffset; - USHORT SubstituteNameLength; - USHORT PrintNameOffset; - USHORT PrintNameLength; - ULONG Flags; - WCHAR PathBuffer[1]; - } SymbolicLinkReparseBuffer; - - struct { - USHORT SubstituteNameOffset; - USHORT SubstituteNameLength; - USHORT PrintNameOffset; - USHORT PrintNameLength; - WCHAR PathBuffer[1]; - } MountPointReparseBuffer; - - struct { - UCHAR DataBuffer[1]; - } GenericReparseBuffer; - }; -} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER; - -#define REPARSE_DATA_BUFFER_HEADER_SIZE FIELD_OFFSET(REPARSE_DATA_BUFFER,\ - GenericReparseBuffer) -#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE ( 16 * 1024 ) - -#ifdef __cplusplus -} -#endif - -#endif /* MS_WINDOWS */ - -#endif /* !Py_WINREPARSE_H */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:36:58 2014 From: python-checkins at python.org (tim.golden) Date: Sun, 27 Apr 2014 19:36:58 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset=3A_4b?= =?utf-8?q?97092aa4bd?= Message-ID: <3gGxBf60d2z7LlY@mail.python.org> http://hg.python.org/cpython/rev/05d162756946 changeset: 90477:05d162756946 user: Tim Golden date: Sun Apr 27 18:35:51 2014 +0100 summary: Backed out changeset: 4b97092aa4bd files: Misc/NEWS | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,9 +57,6 @@ Library ------- -- Issue #18314: os.unlink now removes junction points on Windows. - Kim Gr?sman - - Issue #19385: Make operations on a closed dbm.dumb database always raise the same exception. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:37:09 2014 From: python-checkins at python.org (barry.warsaw) Date: Sun, 27 Apr 2014 19:37:09 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Wording=2E?= Message-ID: <3gGxBs2Bzlz7LnG@mail.python.org> http://hg.python.org/peps/rev/9d2a53b0df9b changeset: 5469:9d2a53b0df9b user: Barry Warsaw date: Sun Apr 27 13:37:04 2014 -0400 summary: Wording. files: pep-0008.txt | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -112,7 +112,8 @@ var_four): print(var_one) -The 4-space rule is optional for indentation lines. +The 4-space rule is optional for the indented lines in a multi-line +construct. Optional:: -- Repository URL: http://hg.python.org/peps From python-checkins at python.org Sun Apr 27 19:47:37 2014 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 27 Apr 2014 19:47:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbzogQmUg?= =?utf-8?q?careful_accessing_instance_variables_in_=5F=5Fdel=5F=5F_=28clos?= =?utf-8?b?ZXMgIzIxMzQwKS4=?= Message-ID: <3gGxQx027vz7Lnh@mail.python.org> http://hg.python.org/cpython/rev/d42d3d3f9c41 changeset: 90478:d42d3d3f9c41 branch: 3.4 parent: 90469:d24f1fb256a3 user: Guido van Rossum date: Sun Apr 27 10:33:58 2014 -0700 summary: asyncio: Be careful accessing instance variables in __del__ (closes #21340). files: Lib/asyncio/tasks.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -76,7 +76,9 @@ return self.gen.gi_code def __del__(self): - frame = self.gen.gi_frame + # Be careful accessing self.gen.frame -- self.gen might not exist. + gen = getattr(self, 'gen', None) + frame = getattr(gen, 'gi_frame', None) if frame is not None and frame.f_lasti == -1: func = self.func code = func.__code__ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:47:38 2014 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 27 Apr 2014 19:47:38 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogYXN5bmNpbzogQWRk?= =?utf-8?q?_=5F=5Fweakref=5F=5F_slots_to_Handle_and_CoroWrapper=2E_Upstrea?= =?utf-8?q?m_issue_=23166=2E?= Message-ID: <3gGxQy1dWbz7LpQ@mail.python.org> http://hg.python.org/cpython/rev/c27cb341de7d changeset: 90479:c27cb341de7d branch: 3.4 user: Guido van Rossum date: Sun Apr 27 10:44:22 2014 -0700 summary: asyncio: Add __weakref__ slots to Handle and CoroWrapper. Upstream issue #166. files: Lib/asyncio/events.py | 2 +- Lib/asyncio/tasks.py | 2 +- Lib/test/test_asyncio/test_events.py | 6 ++++++ Lib/test/test_asyncio/test_tasks.py | 8 ++++++++ 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -16,7 +16,7 @@ class Handle: """Object returned by callback registration methods.""" - __slots__ = ['_callback', '_args', '_cancelled', '_loop'] + __slots__ = ['_callback', '_args', '_cancelled', '_loop', '__weakref__'] def __init__(self, callback, args, loop): assert not isinstance(callback, Handle), 'A Handle is not a callback' diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -36,7 +36,7 @@ class CoroWrapper: # Wrapper for coroutine in _DEBUG mode. - __slots__ = ['gen', 'func', '__name__', '__doc__'] + __slots__ = ['gen', 'func', '__name__', '__doc__', '__weakref__'] def __init__(self, gen, func): assert inspect.isgenerator(gen), gen diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -21,6 +21,7 @@ import errno import unittest from unittest import mock +import weakref from test import support # find_unused_port, IPV6_ENABLED, TEST_HOME_DIR @@ -1786,6 +1787,11 @@ 'handle': h }) + def test_handle_weakref(self): + wd = weakref.WeakValueDictionary() + h = asyncio.Handle(lambda: None, (), object()) + wd['h'] = h # Would fail without __weakref__ slot. + class TimerTests(unittest.TestCase): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -4,6 +4,7 @@ import os.path import types import unittest +import weakref from test.script_helper import assert_python_ok import asyncio @@ -1475,6 +1476,13 @@ self.assertEqual(call((1, 2)), (1, 2)) self.assertEqual(call('spam'), 'spam') + def test_corowrapper_weakref(self): + wd = weakref.WeakValueDictionary() + def foo(): yield from [] + cw = asyncio.tasks.CoroWrapper(foo(), foo) + wd['cw'] = cw # Would fail without __weakref__ slot. + cw.gen = None # Suppress warning from __del__. + class GatherTestsBase: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:47:39 2014 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 27 Apr 2014 19:47:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E4_-=3E_default=3A_asyncio=3A_Be_careful_access?= =?utf-8?q?ing_instance_variables_in?= Message-ID: <3gGxQz3Vkjz7LpV@mail.python.org> http://hg.python.org/cpython/rev/0cb436c6f082 changeset: 90480:0cb436c6f082 parent: 90477:05d162756946 parent: 90478:d42d3d3f9c41 user: Guido van Rossum date: Sun Apr 27 10:34:39 2014 -0700 summary: Merge 3.4 -> default: asyncio: Be careful accessing instance variables in __del__ (closes #21340). files: Lib/asyncio/tasks.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -76,7 +76,9 @@ return self.gen.gi_code def __del__(self): - frame = self.gen.gi_frame + # Be careful accessing self.gen.frame -- self.gen might not exist. + gen = getattr(self, 'gen', None) + frame = getattr(gen, 'gi_frame', None) if frame is not None and frame.f_lasti == -1: func = self.func code = func.__code__ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 19:47:40 2014 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 27 Apr 2014 19:47:40 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E4_-=3E_default=3A_asyncio=3A_Add_=5F=5Fweakref?= =?utf-8?q?=5F=5F_slots_to_Handle_and_CoroWrapper=2E?= Message-ID: <3gGxR05Nprz7LpW@mail.python.org> http://hg.python.org/cpython/rev/e159cb0d955b changeset: 90481:e159cb0d955b parent: 90480:0cb436c6f082 parent: 90479:c27cb341de7d user: Guido van Rossum date: Sun Apr 27 10:44:46 2014 -0700 summary: Merge 3.4 -> default: asyncio: Add __weakref__ slots to Handle and CoroWrapper. Upstream issue #166. files: Lib/asyncio/events.py | 2 +- Lib/asyncio/tasks.py | 2 +- Lib/test/test_asyncio/test_events.py | 6 ++++++ Lib/test/test_asyncio/test_tasks.py | 8 ++++++++ 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -16,7 +16,7 @@ class Handle: """Object returned by callback registration methods.""" - __slots__ = ['_callback', '_args', '_cancelled', '_loop'] + __slots__ = ['_callback', '_args', '_cancelled', '_loop', '__weakref__'] def __init__(self, callback, args, loop): assert not isinstance(callback, Handle), 'A Handle is not a callback' diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -36,7 +36,7 @@ class CoroWrapper: # Wrapper for coroutine in _DEBUG mode. - __slots__ = ['gen', 'func', '__name__', '__doc__'] + __slots__ = ['gen', 'func', '__name__', '__doc__', '__weakref__'] def __init__(self, gen, func): assert inspect.isgenerator(gen), gen diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -21,6 +21,7 @@ import errno import unittest from unittest import mock +import weakref from test import support # find_unused_port, IPV6_ENABLED, TEST_HOME_DIR @@ -1786,6 +1787,11 @@ 'handle': h }) + def test_handle_weakref(self): + wd = weakref.WeakValueDictionary() + h = asyncio.Handle(lambda: None, (), object()) + wd['h'] = h # Would fail without __weakref__ slot. + class TimerTests(unittest.TestCase): diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -4,6 +4,7 @@ import os.path import types import unittest +import weakref from test.script_helper import assert_python_ok import asyncio @@ -1475,6 +1476,13 @@ self.assertEqual(call((1, 2)), (1, 2)) self.assertEqual(call('spam'), 'spam') + def test_corowrapper_weakref(self): + wd = weakref.WeakValueDictionary() + def foo(): yield from [] + cw = asyncio.tasks.CoroWrapper(foo(), foo) + wd['cw'] = cw # Would fail without __weakref__ slot. + cw.gen = None # Suppress warning from __del__. + class GatherTestsBase: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Sun Apr 27 22:26:23 2014 From: python-checkins at python.org (barry.warsaw) Date: Sun, 27 Apr 2014 22:26:23 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?peps=3A_Nice_simplification_from_Nick?= =?utf-8?q?=2E?= Message-ID: <3gH0y749P8z7LlY@mail.python.org> http://hg.python.org/peps/rev/e7bf114ee31a changeset: 5470:e7bf114ee31a user: Barry Warsaw date: Sun Apr 27 16:26:18 2014 -0400 summary: Nice simplification from Nick. files: pep-0008.txt | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/pep-0008.txt b/pep-0008.txt --- a/pep-0008.txt +++ b/pep-0008.txt @@ -112,8 +112,7 @@ var_four): print(var_one) -The 4-space rule is optional for the indented lines in a multi-line -construct. +The 4-space rule is optional for continuation lines. Optional:: -- Repository URL: http://hg.python.org/peps From solipsis at pitrou.net Mon Apr 28 09:31:20 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Mon, 28 Apr 2014 09:31:20 +0200 Subject: [Python-checkins] Daily reference leaks (e159cb0d955b): sum=-1 Message-ID: results for e159cb0d955b on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 0] references, sum=-2 test_site leaked [-2, 0, 0] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogPWJyXF', '-x'] From python-checkins at python.org Mon Apr 28 13:08:36 2014 From: python-checkins at python.org (antoine.pitrou) Date: Mon, 28 Apr 2014 13:08:36 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEzMjA0?= =?utf-8?q?=3A_Calling_sys=2Eflags=2E=5F=5Fnew=5F=5F_would_crash_the_inter?= =?utf-8?q?preter=2C_now_it?= Message-ID: <3gHNX41PNDz7LjR@mail.python.org> http://hg.python.org/cpython/rev/7052fdd90a11 changeset: 90482:7052fdd90a11 branch: 3.4 parent: 90479:c27cb341de7d user: Antoine Pitrou date: Mon Apr 28 13:07:06 2014 +0200 summary: Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. files: Lib/test/test_sys.py | 20 ++++++++++++++++++++ Misc/NEWS | 3 +++ Python/sysmodule.c | 13 ++++++++++--- 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -519,6 +519,26 @@ self.assertTrue(repr(sys.flags)) self.assertEqual(len(sys.flags), len(attrs)) + def assert_raise_on_new_sys_type(self, sys_attr): + # Users are intentionally prevented from creating new instances of + # sys.flags, sys.version_info, and sys.getwindowsversion. + attr_type = type(sys_attr) + with self.assertRaises(TypeError): + attr_type() + with self.assertRaises(TypeError): + attr_type.__new__(attr_type) + + def test_sys_flags_no_instantiation(self): + self.assert_raise_on_new_sys_type(sys.flags) + + def test_sys_version_info_no_instantiation(self): + self.assert_raise_on_new_sys_type(sys.version_info) + + def test_sys_getwindowsversion_no_instantiation(self): + # Skip if not being run on Windows. + test.support.get_attribute(sys, "getwindowsversion") + self.assert_raise_on_new_sys_type(sys.getwindowsversion()) + def test_clear_type_cache(self): sys._clear_type_cache() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #13204: Calling sys.flags.__new__ would crash the interpreter, + now it raises a TypeError. + - Issue #19385: Make operations on a closed dbm.dumb database always raise the same exception. diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1622,6 +1622,7 @@ _PySys_Init(void) { PyObject *m, *sysdict, *version_info; + int res; m = PyModule_Create(&sysmodule); if (m == NULL) @@ -1629,7 +1630,6 @@ sysdict = PyModule_GetDict(m); #define SET_SYS_FROM_STRING_BORROW(key, value) \ do { \ - int res; \ PyObject *v = (value); \ if (v == NULL) \ return NULL; \ @@ -1640,7 +1640,6 @@ } while (0) #define SET_SYS_FROM_STRING(key, value) \ do { \ - int res; \ PyObject *v = (value); \ if (v == NULL) \ return NULL; \ @@ -1759,6 +1758,9 @@ /* prevent user from creating new instances */ VersionInfoType.tp_init = NULL; VersionInfoType.tp_new = NULL; + res = PyDict_DelItemString(VersionInfoType.tp_dict, "__new__"); + if (res < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) + PyErr_Clear(); /* implementation */ SET_SYS_FROM_STRING("implementation", make_impl_info(version_info)); @@ -1772,7 +1774,9 @@ /* prevent user from creating new instances */ FlagsType.tp_init = NULL; FlagsType.tp_new = NULL; - + res = PyDict_DelItemString(FlagsType.tp_dict, "__new__"); + if (res < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) + PyErr_Clear(); #if defined(MS_WINDOWS) /* getwindowsversion */ @@ -1783,6 +1787,9 @@ /* prevent user from creating new instances */ WindowsVersionType.tp_init = NULL; WindowsVersionType.tp_new = NULL; + res = PyDict_DelItemString(WindowsVersionType.tp_dict, "__new__"); + if (res < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) + PyErr_Clear(); #endif /* float repr style: 0.03 (short) vs 0.029999999999999999 (legacy) */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 28 13:08:37 2014 From: python-checkins at python.org (antoine.pitrou) Date: Mon, 28 Apr 2014 13:08:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?b?KTogSXNzdWUgIzEzMjA0OiBDYWxsaW5nIHN5cy5mbGFncy5fX25ld19fIHdv?= =?utf-8?q?uld_crash_the_interpreter=2C_now_it?= Message-ID: <3gHNX54Mh0z7Lkp@mail.python.org> http://hg.python.org/cpython/rev/a14012352f65 changeset: 90483:a14012352f65 parent: 90481:e159cb0d955b parent: 90482:7052fdd90a11 user: Antoine Pitrou date: Mon Apr 28 13:08:28 2014 +0200 summary: Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. files: Lib/test/test_sys.py | 20 ++++++++++++++++++++ Misc/NEWS | 3 +++ Python/sysmodule.c | 13 ++++++++++--- 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -519,6 +519,26 @@ self.assertTrue(repr(sys.flags)) self.assertEqual(len(sys.flags), len(attrs)) + def assert_raise_on_new_sys_type(self, sys_attr): + # Users are intentionally prevented from creating new instances of + # sys.flags, sys.version_info, and sys.getwindowsversion. + attr_type = type(sys_attr) + with self.assertRaises(TypeError): + attr_type() + with self.assertRaises(TypeError): + attr_type.__new__(attr_type) + + def test_sys_flags_no_instantiation(self): + self.assert_raise_on_new_sys_type(sys.flags) + + def test_sys_version_info_no_instantiation(self): + self.assert_raise_on_new_sys_type(sys.version_info) + + def test_sys_getwindowsversion_no_instantiation(self): + # Skip if not being run on Windows. + test.support.get_attribute(sys, "getwindowsversion") + self.assert_raise_on_new_sys_type(sys.getwindowsversion()) + def test_clear_type_cache(self): sys._clear_type_cache() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,9 @@ Library ------- +- Issue #13204: Calling sys.flags.__new__ would crash the interpreter, + now it raises a TypeError. + - Issue #19385: Make operations on a closed dbm.dumb database always raise the same exception. diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1622,6 +1622,7 @@ _PySys_Init(void) { PyObject *m, *sysdict, *version_info; + int res; m = PyModule_Create(&sysmodule); if (m == NULL) @@ -1629,7 +1630,6 @@ sysdict = PyModule_GetDict(m); #define SET_SYS_FROM_STRING_BORROW(key, value) \ do { \ - int res; \ PyObject *v = (value); \ if (v == NULL) \ return NULL; \ @@ -1640,7 +1640,6 @@ } while (0) #define SET_SYS_FROM_STRING(key, value) \ do { \ - int res; \ PyObject *v = (value); \ if (v == NULL) \ return NULL; \ @@ -1759,6 +1758,9 @@ /* prevent user from creating new instances */ VersionInfoType.tp_init = NULL; VersionInfoType.tp_new = NULL; + res = PyDict_DelItemString(VersionInfoType.tp_dict, "__new__"); + if (res < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) + PyErr_Clear(); /* implementation */ SET_SYS_FROM_STRING("implementation", make_impl_info(version_info)); @@ -1772,7 +1774,9 @@ /* prevent user from creating new instances */ FlagsType.tp_init = NULL; FlagsType.tp_new = NULL; - + res = PyDict_DelItemString(FlagsType.tp_dict, "__new__"); + if (res < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) + PyErr_Clear(); #if defined(MS_WINDOWS) /* getwindowsversion */ @@ -1783,6 +1787,9 @@ /* prevent user from creating new instances */ WindowsVersionType.tp_init = NULL; WindowsVersionType.tp_new = NULL; + res = PyDict_DelItemString(WindowsVersionType.tp_dict, "__new__"); + if (res < 0 && PyErr_ExceptionMatches(PyExc_KeyError)) + PyErr_Clear(); #endif /* float repr style: 0.03 (short) vs 0.029999999999999999 (legacy) */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 28 20:57:47 2014 From: python-checkins at python.org (antoine.pitrou) Date: Mon, 28 Apr 2014 20:57:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319940=3A_ssl=2Ece?= =?utf-8?q?rt=5Ftime=5Fto=5Fseconds=28=29_now_interprets_the_given_time_st?= =?utf-8?q?ring?= Message-ID: <3gHZxR4nzmz7Ljx@mail.python.org> http://hg.python.org/cpython/rev/7191c37238d5 changeset: 90484:7191c37238d5 user: Antoine Pitrou date: Mon Apr 28 20:57:36 2014 +0200 summary: Issue #19940: ssl.cert_time_to_seconds() now interprets the given time string in the UTC timezone (as specified in RFC 5280), not the local timezone. Patch by Akira. files: Doc/library/ssl.rst | 34 ++++++++++---- Lib/ssl.py | 32 ++++++++++++-- Lib/test/test_ssl.py | 71 ++++++++++++++++++++++++++++++++ Misc/ACKS | 1 + Misc/NEWS | 4 + 5 files changed, 126 insertions(+), 16 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -372,20 +372,32 @@ IDN A-labels such as ``www*.xn--pthon-kva.org`` are still supported, but ``x*.python.org`` no longer matches ``xn--tda.python.org``. -.. function:: cert_time_to_seconds(timestring) +.. function:: cert_time_to_seconds(cert_time) - Returns a floating-point value containing a normal seconds-after-the-epoch - time value, given the time-string representing the "notBefore" or "notAfter" - date from a certificate. + Return the time in seconds since the Epoch, given the ``cert_time`` + string representing the "notBefore" or "notAfter" date from a + certificate in ``"%b %d %H:%M:%S %Y %Z"`` strptime format (C + locale). - Here's an example:: + Here's an example: - >>> import ssl - >>> ssl.cert_time_to_seconds("May 9 00:00:00 2007 GMT") - 1178694000.0 - >>> import time - >>> time.ctime(ssl.cert_time_to_seconds("May 9 00:00:00 2007 GMT")) - 'Wed May 9 00:00:00 2007' + .. doctest:: newcontext + + >>> import ssl + >>> timestamp = ssl.cert_time_to_seconds("Jan 5 09:34:43 2018 GMT") + >>> timestamp + 1515144883 + >>> from datetime import datetime + >>> print(datetime.utcfromtimestamp(timestamp)) + 2018-01-05 09:34:43 + + "notBefore" or "notAfter" dates must use GMT (:rfc:`5280`). + + .. versionchanged:: 3.5 + Interpret the input time as a time in UTC as specified by 'GMT' + timezone in the input string. Local timezone was used + previously. Return an integer (no fractions of a second in the + input format) .. function:: get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None) diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -879,12 +879,34 @@ # some utility functions def cert_time_to_seconds(cert_time): - """Takes a date-time string in standard ASN1_print form - ("MON DAY 24HOUR:MINUTE:SEC YEAR TIMEZONE") and return - a Python time value in seconds past the epoch.""" + """Return the time in seconds since the Epoch, given the timestring + representing the "notBefore" or "notAfter" date from a certificate + in ``"%b %d %H:%M:%S %Y %Z"`` strptime format (C locale). - import time - return time.mktime(time.strptime(cert_time, "%b %d %H:%M:%S %Y GMT")) + "notBefore" or "notAfter" dates must use UTC (RFC 5280). + + Month is one of: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec + UTC should be specified as GMT (see ASN1_TIME_print()) + """ + from time import strptime + from calendar import timegm + + months = ( + "Jan","Feb","Mar","Apr","May","Jun", + "Jul","Aug","Sep","Oct","Nov","Dec" + ) + time_format = ' %d %H:%M:%S %Y GMT' # NOTE: no month, fixed GMT + try: + month_number = months.index(cert_time[:3].title()) + 1 + except ValueError: + raise ValueError('time data %r does not match ' + 'format "%%b%s"' % (cert_time, time_format)) + else: + # found valid month + tt = strptime(cert_time[3:], time_format) + # return an integer, the previous mktime()-based implementation + # returned a float (fractional seconds are always zero here). + return timegm((tt[0], month_number) + tt[2:6]) PEM_HEADER = "-----BEGIN CERTIFICATE-----" PEM_FOOTER = "-----END CERTIFICATE-----" diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -86,6 +86,12 @@ # 0.9.8 or higher return ssl.OPENSSL_VERSION_INFO >= (0, 9, 8, 0, 15) +def utc_offset(): #NOTE: ignore issues like #1647654 + # local time = utc time + utc offset + if time.daylight and time.localtime().tm_isdst > 0: + return -time.altzone # seconds + return -time.timezone + def asn1time(cert_time): # Some versions of OpenSSL ignore seconds, see #18207 # 0.9.8.i @@ -651,6 +657,71 @@ ctx.wrap_socket(s) self.assertEqual(str(cx.exception), "only stream sockets are supported") + def cert_time_ok(self, timestring, timestamp): + self.assertEqual(ssl.cert_time_to_seconds(timestring), timestamp) + + def cert_time_fail(self, timestring): + with self.assertRaises(ValueError): + ssl.cert_time_to_seconds(timestring) + + @unittest.skipUnless(utc_offset(), + 'local time needs to be different from UTC') + def test_cert_time_to_seconds_timezone(self): + # Issue #19940: ssl.cert_time_to_seconds() returns wrong + # results if local timezone is not UTC + self.cert_time_ok("May 9 00:00:00 2007 GMT", 1178668800.0) + self.cert_time_ok("Jan 5 09:34:43 2018 GMT", 1515144883.0) + + def test_cert_time_to_seconds(self): + timestring = "Jan 5 09:34:43 2018 GMT" + ts = 1515144883.0 + self.cert_time_ok(timestring, ts) + # accept keyword parameter, assert its name + self.assertEqual(ssl.cert_time_to_seconds(cert_time=timestring), ts) + # accept both %e and %d (space or zero generated by strftime) + self.cert_time_ok("Jan 05 09:34:43 2018 GMT", ts) + # case-insensitive + self.cert_time_ok("JaN 5 09:34:43 2018 GmT", ts) + self.cert_time_fail("Jan 5 09:34 2018 GMT") # no seconds + self.cert_time_fail("Jan 5 09:34:43 2018") # no GMT + self.cert_time_fail("Jan 5 09:34:43 2018 UTC") # not GMT timezone + self.cert_time_fail("Jan 35 09:34:43 2018 GMT") # invalid day + self.cert_time_fail("Jon 5 09:34:43 2018 GMT") # invalid month + self.cert_time_fail("Jan 5 24:00:00 2018 GMT") # invalid hour + self.cert_time_fail("Jan 5 09:60:43 2018 GMT") # invalid minute + + newyear_ts = 1230768000.0 + # leap seconds + self.cert_time_ok("Dec 31 23:59:60 2008 GMT", newyear_ts) + # same timestamp + self.cert_time_ok("Jan 1 00:00:00 2009 GMT", newyear_ts) + + self.cert_time_ok("Jan 5 09:34:59 2018 GMT", 1515144899) + # allow 60th second (even if it is not a leap second) + self.cert_time_ok("Jan 5 09:34:60 2018 GMT", 1515144900) + # allow 2nd leap second for compatibility with time.strptime() + self.cert_time_ok("Jan 5 09:34:61 2018 GMT", 1515144901) + self.cert_time_fail("Jan 5 09:34:62 2018 GMT") # invalid seconds + + # no special treatement for the special value: + # 99991231235959Z (rfc 5280) + self.cert_time_ok("Dec 31 23:59:59 9999 GMT", 253402300799.0) + + @support.run_with_locale('LC_ALL', '') + def test_cert_time_to_seconds_locale(self): + # `cert_time_to_seconds()` should be locale independent + + def local_february_name(): + return time.strftime('%b', (1, 2, 3, 4, 5, 6, 0, 0, 0)) + + if local_february_name().lower() == 'feb': + self.skipTest("locale-specific month name needs to be " + "different from C locale") + + # locale-independent + self.cert_time_ok("Feb 9 00:00:00 2007 GMT", 1170979200.0) + self.cert_time_fail(local_february_name() + " 9 00:00:00 2007 GMT") + class ContextTests(unittest.TestCase): diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -23,6 +23,7 @@ Farhan Ahmad Matthew Ahrens Nir Aides +Akira Yaniv Aknin Jyrki Alakuijala Steve Alexander diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -57,6 +57,10 @@ Library ------- +- Issue #19940: ssl.cert_time_to_seconds() now interprets the given time + string in the UTC timezone (as specified in RFC 5280), not the local + timezone. + - Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 28 21:12:38 2014 From: python-checkins at python.org (antoine.pitrou) Date: Mon, 28 Apr 2014 21:12:38 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321312=3A_Update_t?= =?utf-8?q?he_thread=5Ffoobar=2Eh_template_file_to_include_newer?= Message-ID: <3gHbGZ2871z7Llp@mail.python.org> http://hg.python.org/cpython/rev/7bb1bda5dcef changeset: 90485:7bb1bda5dcef user: Antoine Pitrou date: Mon Apr 28 21:12:11 2014 +0200 summary: Issue #21312: Update the thread_foobar.h template file to include newer threading APIs. Patch by Jack McCracken. files: Misc/ACKS | 1 + Misc/NEWS | 3 + Python/thread_foobar.h | 63 ++++++++++++++++++++++++++++- 3 files changed, 64 insertions(+), 3 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -847,6 +847,7 @@ Lucas Maystre Arnaud Mazin Matt McClure +Jack McCracken Rebecca McCreary Kirk McDonald Chris McDonough diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -320,6 +320,9 @@ Documentation ------------- +- Issue #21312: Update the thread_foobar.h template file to include newer + threading APIs. Patch by Jack McCracken. + - Issue #21043: Remove the recommendation for specific CA organizations and to mention the ability to load the OS certificates. diff --git a/Python/thread_foobar.h b/Python/thread_foobar.h --- a/Python/thread_foobar.h +++ b/Python/thread_foobar.h @@ -1,4 +1,3 @@ - /* * Initialization. */ @@ -61,10 +60,18 @@ int PyThread_acquire_lock(PyThread_type_lock lock, int waitflag) { + return PyThread_acquire_lock_timed(lock, waitflag ? -1 : 0, 0); +} + +PyLockStatus +PyThread_acquire_lock_timed(PyThread_type_lock lock, PY_TIMEOUT_T microseconds, + int intr_flag) +{ int success; - dprintf(("PyThread_acquire_lock(%p, %d) called\n", lock, waitflag)); - dprintf(("PyThread_acquire_lock(%p, %d) -> %d\n", lock, waitflag, success)); + dprintf(("PyThread_acquire_lock_timed(%p, %lld, %d) called\n", lock, microseconds, intr_flag)); + dprintf(("PyThread_acquire_lock_timed(%p, %lld, %d) -> %d\n", + lock, microseconds, intr_flag, success)); return success; } @@ -73,3 +80,53 @@ { dprintf(("PyThread_release_lock(%p) called\n", lock)); } + +/* The following are only needed if native TLS support exists */ +#define Py_HAVE_NATIVE_TLS + +#ifdef Py_HAVE_NATIVE_TLS +int +PyThread_create_key(void) +{ + int result; + return result; +} + +void +PyThread_delete_key(int key) +{ + +} + +int +PyThread_set_key_value(int key, void *value) +{ + int ok; + + /* A failure in this case returns -1 */ + if (!ok) + return -1; + return 0; +} + +void * +PyThread_get_key_value(int key) +{ + void *result; + + return result; +} + +void +PyThread_delete_key_value(int key) +{ + +} + +void +PyThread_ReInitTLS(void) +{ + +} + +#endif -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Mon Apr 28 22:47:26 2014 From: python-checkins at python.org (ned.deily) Date: Mon, 28 Apr 2014 22:47:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317861=3A_Allow_ge?= =?utf-8?q?nerate=5Fopcode=5Fh_to_run_with_a_system_Python_2=2E5=2E?= Message-ID: <3gHdMy1Y9Zz7LjS@mail.python.org> http://hg.python.org/cpython/rev/1fd9c3f6cf68 changeset: 90486:1fd9c3f6cf68 user: Ned Deily date: Mon Apr 28 13:46:36 2014 -0700 summary: Issue #17861: Allow generate_opcode_h to run with a system Python 2.5. Patch by David Bolen. files: Tools/scripts/generate_opcode_h.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py --- a/Tools/scripts/generate_opcode_h.py +++ b/Tools/scripts/generate_opcode_h.py @@ -1,5 +1,7 @@ # This script generates the opcode.h header file. +from __future__ import with_statement + import sys header = """/* Auto-generated by Tools/scripts/generate_opcode_h.py */ #ifndef Py_OPCODE_H -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 00:21:05 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 00:21:05 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzIxMjI1OiBjb3B5?= =?utf-8?q?_docstrings_from_base_classes?= Message-ID: <3gHgS16rL7z7LjV@mail.python.org> http://hg.python.org/cpython/rev/6e23afdee4e4 changeset: 90487:6e23afdee4e4 branch: 2.7 parent: 90473:0c8a7299c7e3 user: Andrew Kuchling date: Fri Apr 25 09:29:30 2014 -0400 summary: #21225: copy docstrings from base classes files: Lib/io.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/io.py b/Lib/io.py --- a/Lib/io.py +++ b/Lib/io.py @@ -68,15 +68,16 @@ # version however. class IOBase(_io._IOBase): __metaclass__ = abc.ABCMeta + __doc__ = _io._IOBase.__doc__ class RawIOBase(_io._RawIOBase, IOBase): - pass + __doc__ = _io._RawIOBase.__doc__ class BufferedIOBase(_io._BufferedIOBase, IOBase): - pass + __doc__ = _io._BufferedIOBase.__doc__ class TextIOBase(_io._TextIOBase, IOBase): - pass + __doc__ = _io._TextIOBase.__doc__ RawIOBase.register(FileIO) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 00:58:20 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 00:58:20 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320355=3A_-W_comma?= =?utf-8?q?nd_line_options_now_have_higher_priority_than_the?= Message-ID: <3gHhH06vC6z7Lpl@mail.python.org> http://hg.python.org/cpython/rev/78c0d80a04f9 changeset: 90488:78c0d80a04f9 parent: 90486:1fd9c3f6cf68 user: Antoine Pitrou date: Tue Apr 29 00:56:08 2014 +0200 summary: Issue #20355: -W command line options now have higher priority than the PYTHONWARNINGS environment variable. Patch by Arfrever. files: Lib/test/test_warnings.py | 16 ++++++++++++++-- Misc/NEWS | 3 +++ Modules/main.c | 19 ++++++++++++++++++- 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py --- a/Lib/test/test_warnings.py +++ b/Lib/test/test_warnings.py @@ -5,7 +5,7 @@ import sys import unittest from test import support -from test.script_helper import assert_python_ok +from test.script_helper import assert_python_ok, assert_python_failure from test import warning_tests @@ -748,7 +748,19 @@ "import sys; sys.stdout.write(str(sys.warnoptions))", PYTHONWARNINGS="ignore::DeprecationWarning") self.assertEqual(stdout, - b"['ignore::UnicodeWarning', 'ignore::DeprecationWarning']") + b"['ignore::DeprecationWarning', 'ignore::UnicodeWarning']") + + def test_conflicting_envvar_and_command_line(self): + rc, stdout, stderr = assert_python_failure("-Werror::DeprecationWarning", "-c", + "import sys, warnings; sys.stdout.write(str(sys.warnoptions)); " + "warnings.warn('Message', DeprecationWarning)", + PYTHONWARNINGS="default::DeprecationWarning") + self.assertEqual(stdout, + b"['default::DeprecationWarning', 'error::DeprecationWarning']") + self.assertEqual(stderr.splitlines(), + [b"Traceback (most recent call last):", + b" File \"\", line 1, in ", + b"DeprecationWarning: Message"]) @unittest.skipUnless(sys.getfilesystemencoding() != 'ascii', 'requires non-ascii filesystemencoding') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #20355: -W command line options now have higher priority than the + PYTHONWARNINGS environment variable. Patch by Arfrever. + - Issue #21274: Define PATH_MAX for GNU/Hurd in Python/pythonrun.c. - Issue #20904: Support setting FPU precision on m68k. diff --git a/Modules/main.c b/Modules/main.c --- a/Modules/main.c +++ b/Modules/main.c @@ -343,6 +343,8 @@ int version = 0; int saw_unbuffered_flag = 0; PyCompilerFlags cf; + PyObject *warning_option = NULL; + PyObject *warning_options = NULL; cf.cf_flags = 0; @@ -465,7 +467,15 @@ break; case 'W': - PySys_AddWarnOption(_PyOS_optarg); + if (warning_options == NULL) + warning_options = PyList_New(0); + if (warning_options == NULL) + Py_FatalError("failure in handling of -W argument"); + warning_option = PyUnicode_FromWideChar(_PyOS_optarg, -1); + if (warning_option == NULL) + Py_FatalError("failure in handling of -W argument"); + PyList_Append(warning_options, warning_option); + Py_DECREF(warning_option); break; case 'X': @@ -559,6 +569,12 @@ PyMem_RawFree(buf); } #endif + if (warning_options != NULL) { + Py_ssize_t i; + for (i = 0; i < PyList_GET_SIZE(warning_options); i++) { + PySys_AddWarnOptionUnicode(PyList_GET_ITEM(warning_options, i)); + } + } if (command == NULL && module == NULL && _PyOS_optind < argc && wcscmp(argv[_PyOS_optind], L"-") != 0) @@ -652,6 +668,7 @@ Py_SetProgramName(argv[0]); #endif Py_Initialize(); + Py_XDECREF(warning_options); if (!Py_QuietFlag && (Py_VerboseFlag || (command == NULL && filename == NULL && -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 00:58:22 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 00:58:22 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_a_workaround_for_fi?= =?utf-8?q?xed_issue_=2320355=2E?= Message-ID: <3gHhH21Sjcz7Lpk@mail.python.org> http://hg.python.org/cpython/rev/925c0b611c02 changeset: 90489:925c0b611c02 user: Antoine Pitrou date: Tue Apr 29 00:58:02 2014 +0200 summary: Remove a workaround for fixed issue #20355. files: Tools/scripts/run_tests.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Tools/scripts/run_tests.py b/Tools/scripts/run_tests.py --- a/Tools/scripts/run_tests.py +++ b/Tools/scripts/run_tests.py @@ -33,8 +33,6 @@ # Allow user-specified interpreter options to override our defaults. args.extend(test.support.args_from_interpreter_flags()) - # Workaround for issue #20355 - os.environ.pop("PYTHONWARNINGS", None) # Workaround for issue #20361 args.extend(['-W', 'error::BytesWarning']) -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 01:25:47 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 01:25:47 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzk4MTU6?= =?utf-8?q?_assertRaises_now_tries_to_clear_references_to_local_variables_?= =?utf-8?q?in?= Message-ID: <3gHhtg4rx0z7LjZ@mail.python.org> http://hg.python.org/cpython/rev/6ab3193e890e changeset: 90490:6ab3193e890e branch: 3.4 parent: 90482:7052fdd90a11 user: Antoine Pitrou date: Tue Apr 29 01:23:50 2014 +0200 summary: Issue #9815: assertRaises now tries to clear references to local variables in the exception's traceback. files: Lib/unittest/case.py | 3 + Lib/unittest/test/test_assertions.py | 31 ++++++++++++++++ Misc/NEWS | 3 + 3 files changed, 37 insertions(+), 0 deletions(-) diff --git a/Lib/unittest/case.py b/Lib/unittest/case.py --- a/Lib/unittest/case.py +++ b/Lib/unittest/case.py @@ -9,6 +9,7 @@ import warnings import collections import contextlib +import traceback from . import result from .util import (strclass, safe_repr, _count_diff_all_purpose, @@ -178,6 +179,8 @@ self.obj_name)) else: self._raiseFailure("{} not raised".format(exc_name)) + else: + traceback.clear_frames(tb) if not issubclass(exc_type, self.expected): # let unexpected exceptions pass through return False diff --git a/Lib/unittest/test/test_assertions.py b/Lib/unittest/test/test_assertions.py --- a/Lib/unittest/test/test_assertions.py +++ b/Lib/unittest/test/test_assertions.py @@ -1,5 +1,6 @@ import datetime import warnings +import weakref import unittest from itertools import product @@ -97,6 +98,36 @@ else: self.fail("assertRaises() didn't let exception pass through") + def test_assertRaises_frames_survival(self): + # Issue #9815: assertRaises should avoid keeping local variables + # in a traceback alive. + class A: + pass + wr = None + + class Foo(unittest.TestCase): + + def foo(self): + nonlocal wr + a = A() + wr = weakref.ref(a) + try: + raise IOError + except IOError: + raise ValueError + + def test_functional(self): + self.assertRaises(ValueError, self.foo) + + def test_with(self): + with self.assertRaises(ValueError): + self.foo() + + Foo("test_functional").run() + self.assertIsNone(wr()) + Foo("test_with").run() + self.assertIsNone(wr()) + def testAssertNotRegex(self): self.assertNotRegex('Ala ma kota', r'r+') try: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #9815: assertRaises now tries to clear references to local variables + in the exception's traceback. + - Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 01:25:48 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 01:25:48 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=239815=3A_assertRaises_now_tries_to_clear_referen?= =?utf-8?q?ces_to_local_variables_in?= Message-ID: <3gHhth6f9Mz7LjZ@mail.python.org> http://hg.python.org/cpython/rev/553fe27521be changeset: 90491:553fe27521be parent: 90489:925c0b611c02 parent: 90490:6ab3193e890e user: Antoine Pitrou date: Tue Apr 29 01:25:07 2014 +0200 summary: Issue #9815: assertRaises now tries to clear references to local variables in the exception's traceback. files: Lib/unittest/case.py | 3 + Lib/unittest/test/test_assertions.py | 31 ++++++++++++++++ Misc/NEWS | 3 + 3 files changed, 37 insertions(+), 0 deletions(-) diff --git a/Lib/unittest/case.py b/Lib/unittest/case.py --- a/Lib/unittest/case.py +++ b/Lib/unittest/case.py @@ -9,6 +9,7 @@ import warnings import collections import contextlib +import traceback from . import result from .util import (strclass, safe_repr, _count_diff_all_purpose, @@ -178,6 +179,8 @@ self.obj_name)) else: self._raiseFailure("{} not raised".format(exc_name)) + else: + traceback.clear_frames(tb) if not issubclass(exc_type, self.expected): # let unexpected exceptions pass through return False diff --git a/Lib/unittest/test/test_assertions.py b/Lib/unittest/test/test_assertions.py --- a/Lib/unittest/test/test_assertions.py +++ b/Lib/unittest/test/test_assertions.py @@ -1,5 +1,6 @@ import datetime import warnings +import weakref import unittest from itertools import product @@ -97,6 +98,36 @@ else: self.fail("assertRaises() didn't let exception pass through") + def test_assertRaises_frames_survival(self): + # Issue #9815: assertRaises should avoid keeping local variables + # in a traceback alive. + class A: + pass + wr = None + + class Foo(unittest.TestCase): + + def foo(self): + nonlocal wr + a = A() + wr = weakref.ref(a) + try: + raise IOError + except IOError: + raise ValueError + + def test_functional(self): + self.assertRaises(ValueError, self.foo) + + def test_with(self): + with self.assertRaises(ValueError): + self.foo() + + Foo("test_functional").run() + self.assertIsNone(wr()) + Foo("test_with").run() + self.assertIsNone(wr()) + def testAssertNotRegex(self): self.assertNotRegex('Ala ma kota', r'r+') try: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,9 @@ Library ------- +- Issue #9815: assertRaises now tries to clear references to local variables + in the exception's traceback. + - Issue #19940: ssl.cert_time_to_seconds() now interprets the given time string in the UTC timezone (as specified in RFC 5280), not the local timezone. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 01:39:32 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 01:39:32 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzkzMDc6?= =?utf-8?q?_document_the_various_Py=5FTPFLAGS=5F*=5FSUBCLASS_flags=2E__Pat?= =?utf-8?q?ch_by_Yury?= Message-ID: <3gHjBX3xKrz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/37786ae8cc1c changeset: 90492:37786ae8cc1c branch: 3.4 parent: 90490:6ab3193e890e user: Antoine Pitrou date: Tue Apr 29 01:39:03 2014 +0200 summary: Issue #9307: document the various Py_TPFLAGS_*_SUBCLASS flags. Patch by Yury V. Zaytsev. files: Doc/c-api/typeobj.rst | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -464,6 +464,24 @@ :const:`Py_TPFLAGS_HAVE_VERSION_TAG`. + .. data:: Py_TPFLAGS_LONG_SUBCLASS + .. data:: Py_TPFLAGS_LIST_SUBCLASS + .. data:: Py_TPFLAGS_TUPLE_SUBCLASS + .. data:: Py_TPFLAGS_BYTES_SUBCLASS + .. data:: Py_TPFLAGS_UNICODE_SUBCLASS + .. data:: Py_TPFLAGS_DICT_SUBCLASS + .. data:: Py_TPFLAGS_BASE_EXC_SUBCLASS + .. data:: Py_TPFLAGS_TYPE_SUBCLASS + + These flags are used by functions such as + :c:func:`PyLong_Check` to quickly determine if a type is a subclass + of a built-in type; such specific checks are faster than a generic + check, like :c:func:`PyObject_IsInstance`. Custom types that inherit + from built-ins should have their :c:member:`~PyTypeObject.tp_flags` + set appropriately, or the code that interacts with such types + will behave differently depending on what kind of check is used. + + .. data:: Py_TPFLAGS_HAVE_FINALIZE This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 01:39:33 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 01:39:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=239307=3A_document_the_various_Py=5FTPFLAGS=5F*?= =?utf-8?q?=5FSUBCLASS_flags=2E__Patch_by_Yury?= Message-ID: <3gHjBY5dYHz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/d1a03834cec7 changeset: 90493:d1a03834cec7 parent: 90491:553fe27521be parent: 90492:37786ae8cc1c user: Antoine Pitrou date: Tue Apr 29 01:39:26 2014 +0200 summary: Issue #9307: document the various Py_TPFLAGS_*_SUBCLASS flags. Patch by Yury V. Zaytsev. files: Doc/c-api/typeobj.rst | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -464,6 +464,24 @@ :const:`Py_TPFLAGS_HAVE_VERSION_TAG`. + .. data:: Py_TPFLAGS_LONG_SUBCLASS + .. data:: Py_TPFLAGS_LIST_SUBCLASS + .. data:: Py_TPFLAGS_TUPLE_SUBCLASS + .. data:: Py_TPFLAGS_BYTES_SUBCLASS + .. data:: Py_TPFLAGS_UNICODE_SUBCLASS + .. data:: Py_TPFLAGS_DICT_SUBCLASS + .. data:: Py_TPFLAGS_BASE_EXC_SUBCLASS + .. data:: Py_TPFLAGS_TYPE_SUBCLASS + + These flags are used by functions such as + :c:func:`PyLong_Check` to quickly determine if a type is a subclass + of a built-in type; such specific checks are faster than a generic + check, like :c:func:`PyObject_IsInstance`. Custom types that inherit + from built-ins should have their :c:member:`~PyTypeObject.tp_flags` + set appropriately, or the code that interacts with such types + will behave differently depending on what kind of check is used. + + .. data:: Py_TPFLAGS_HAVE_FINALIZE This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 01:58:39 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 01:58:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318727=3A_improve_?= =?utf-8?q?test_coverage_of_the_csv_module_by_testing_for?= Message-ID: <3gHjcb2vgPz7LjS@mail.python.org> http://hg.python.org/cpython/rev/2502843dbedf changeset: 90494:2502843dbedf user: Antoine Pitrou date: Tue Apr 29 01:58:06 2014 +0200 summary: Issue #18727: improve test coverage of the csv module by testing for DictWriter.writerows. Patch by Muhammad Jehanzeb. files: Lib/test/test_csv.py | 10 ++++++++++ Misc/ACKS | 1 + 2 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -575,6 +575,16 @@ fileobj.readline() # header self.assertEqual(fileobj.read(), "10,,abc\r\n") + def test_write_multiple_dict_rows(self): + fileobj = StringIO() + writer = csv.DictWriter(fileobj, fieldnames=["f1", "f2", "f3"]) + writer.writeheader() + self.assertEqual(fileobj.getvalue(), "f1,f2,f3\r\n") + writer.writerows([{"f1": 1, "f2": "abc", "f3": "f"}, + {"f1": 2, "f2": 5, "f3": "xyz"}]) + self.assertEqual(fileobj.getvalue(), + "f1,f2,f3\r\n1,abc,f\r\n2,5,xyz\r\n") + def test_write_no_fields(self): fileobj = StringIO() self.assertRaises(TypeError, csv.DictWriter, fileobj) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -613,6 +613,7 @@ Rajagopalasarma Jayakrishnan Zbigniew J?drzejewski-Szmek Julien Jehannet +Muhammad Jehanzeb Drew Jenkins Flemming Kj?r Jensen Philip H. Jensen -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 02:04:24 2014 From: python-checkins at python.org (giampaolo.rodola) Date: Tue, 29 Apr 2014 02:04:24 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_isuse_=2313248=3A_remo?= =?utf-8?q?ve_previously_deprecated_asyncore=2Edispatcher_=5F=5Fgetattr=5F?= =?utf-8?q?=5F?= Message-ID: <3gHjlD6wxkz7LjW@mail.python.org> http://hg.python.org/cpython/rev/2cceb8cb552b changeset: 90495:2cceb8cb552b parent: 90493:d1a03834cec7 user: Giampaolo Rodola' date: Tue Apr 29 02:03:40 2014 +0200 summary: fix isuse #13248: remove previously deprecated asyncore.dispatcher __getattr__ cheap inheritance hack. files: Lib/asyncore.py | 14 -------------- Lib/test/test_asyncore.py | 17 ----------------- Misc/NEWS | 3 +++ 3 files changed, 3 insertions(+), 31 deletions(-) diff --git a/Lib/asyncore.py b/Lib/asyncore.py --- a/Lib/asyncore.py +++ b/Lib/asyncore.py @@ -404,20 +404,6 @@ if why.args[0] not in (ENOTCONN, EBADF): raise - # cheap inheritance, used to pass all other attribute - # references to the underlying socket object. - def __getattr__(self, attr): - try: - retattr = getattr(self.socket, attr) - except AttributeError: - raise AttributeError("%s instance has no attribute '%s'" - %(self.__class__.__name__, attr)) - else: - msg = "%(me)s.%(attr)s is deprecated; use %(me)s.socket.%(attr)s " \ - "instead" % {'me' : self.__class__.__name__, 'attr' : attr} - warnings.warn(msg, DeprecationWarning, stacklevel=2) - return retattr - # log and log_info may be overridden to provide more sophisticated # logging and warning methods. In general, log is for 'hit' logging # and 'log_info' is for informational, warning and error logging. diff --git a/Lib/test/test_asyncore.py b/Lib/test/test_asyncore.py --- a/Lib/test/test_asyncore.py +++ b/Lib/test/test_asyncore.py @@ -316,23 +316,6 @@ 'warning: unhandled connect event'] self.assertEqual(lines, expected) - def test_issue_8594(self): - # XXX - this test is supposed to be removed in next major Python - # version - d = asyncore.dispatcher(socket.socket()) - # make sure the error message no longer refers to the socket - # object but the dispatcher instance instead - self.assertRaisesRegex(AttributeError, 'dispatcher instance', - getattr, d, 'foo') - # cheap inheritance with the underlying socket is supposed - # to still work but a DeprecationWarning is expected - with warnings.catch_warnings(record=True) as w: - warnings.simplefilter("always") - family = d.family - self.assertEqual(family, socket.AF_INET) - self.assertEqual(len(w), 1) - self.assertTrue(issubclass(w[0].category, DeprecationWarning)) - def test_strerror(self): # refers to bug #8573 err = asyncore._strerror(errno.EPERM) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,9 @@ Library ------- +- Issue #13248: removed previously deprecated asyncore.dispatcher __getattr__ + cheap inheritance hack. + - Issue #9815: assertRaises now tries to clear references to local variables in the exception's traceback. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 02:04:26 2014 From: python-checkins at python.org (giampaolo.rodola) Date: Tue, 29 Apr 2014 02:04:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <3gHjlG1VK4z7Ljp@mail.python.org> http://hg.python.org/cpython/rev/3b203f000256 changeset: 90496:3b203f000256 parent: 90495:2cceb8cb552b parent: 90494:2502843dbedf user: Giampaolo Rodola' date: Tue Apr 29 02:04:14 2014 +0200 summary: merge heads files: Lib/test/test_csv.py | 10 ++++++++++ Misc/ACKS | 1 + 2 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -575,6 +575,16 @@ fileobj.readline() # header self.assertEqual(fileobj.read(), "10,,abc\r\n") + def test_write_multiple_dict_rows(self): + fileobj = StringIO() + writer = csv.DictWriter(fileobj, fieldnames=["f1", "f2", "f3"]) + writer.writeheader() + self.assertEqual(fileobj.getvalue(), "f1,f2,f3\r\n") + writer.writerows([{"f1": 1, "f2": "abc", "f3": "f"}, + {"f1": 2, "f2": 5, "f3": "xyz"}]) + self.assertEqual(fileobj.getvalue(), + "f1,f2,f3\r\n1,abc,f\r\n2,5,xyz\r\n") + def test_write_no_fields(self): fileobj = StringIO() self.assertRaises(TypeError, csv.DictWriter, fileobj) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -613,6 +613,7 @@ Rajagopalasarma Jayakrishnan Zbigniew J?drzejewski-Szmek Julien Jehannet +Muhammad Jehanzeb Drew Jenkins Flemming Kj?r Jensen Philip H. Jensen -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 06:33:14 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 06:33:14 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyMTAy?= =?utf-8?q?6=3A_Augment_site_doc_based_on_experiments=2E_Patch_by_Carol_Wi?= =?utf-8?q?lling=2E?= Message-ID: <3gHqjQ18BTz7LjX@mail.python.org> http://hg.python.org/cpython/rev/79a4560a702f changeset: 90497:79a4560a702f branch: 2.7 parent: 90487:6e23afdee4e4 user: Terry Jan Reedy date: Tue Apr 29 00:31:46 2014 -0400 summary: Closes #21026: Augment site doc based on experiments. Patch by Carol Willing. files: Doc/library/site.rst | 6 +++++- Misc/ACKS | 1 + 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/site.rst b/Doc/library/site.rst --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -83,7 +83,11 @@ :mod:`sitecustomize`, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an :exc:`ImportError` exception, it is -silently ignored. +silently ignored. If Python is started without output streams available, as +with :file:`pythonw.exe` on Windows (which is used by default to start IDLE), +attempted output from :mod:`sitecustomize` is ignored. Any exception other +than :exc:`ImportError` causes a silent and perhaps mysterious failure of the +process. .. index:: module: usercustomize diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1421,6 +1421,7 @@ Jason Williams John Williams Sue Williams +Carol Willing Steven Willis Frank Willison Geoff Wilson -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 06:33:15 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 06:33:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMDI2?= =?utf-8?q?=3A_Augment_site_doc_based_on_experiments=2E_Patch_by_Carol_Wil?= =?utf-8?q?ling=2E?= Message-ID: <3gHqjR30fKz7LjX@mail.python.org> http://hg.python.org/cpython/rev/3fef95842314 changeset: 90498:3fef95842314 branch: 3.4 parent: 90492:37786ae8cc1c user: Terry Jan Reedy date: Tue Apr 29 00:31:53 2014 -0400 summary: Issue #21026: Augment site doc based on experiments. Patch by Carol Willing. files: Doc/library/site.rst | 6 +++++- Misc/ACKS | 1 + 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/site.rst b/Doc/library/site.rst --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -99,7 +99,11 @@ :mod:`sitecustomize`, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an :exc:`ImportError` exception, it is -silently ignored. +silently ignored. If Python is started without output streams available, as +with :file:`pythonw.exe` on Windows (which is used by default to start IDLE), +attempted output from :mod:`sitecustomize` is ignored. Any exception other +than :exc:`ImportError` causes a silent and perhaps mysterious failure of the +process. .. index:: module: usercustomize diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1421,6 +1421,7 @@ Jason Williams John Williams Sue Williams +Carol Willing Steven Willis Frank Willison Geoff Wilson -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 06:33:16 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 06:33:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3gHqjS4YySz7Lk9@mail.python.org> http://hg.python.org/cpython/rev/b82299695616 changeset: 90499:b82299695616 parent: 90496:3b203f000256 parent: 90498:3fef95842314 user: Terry Jan Reedy date: Tue Apr 29 00:32:17 2014 -0400 summary: Merge with 3.4 files: Doc/library/site.rst | 6 +++++- Misc/ACKS | 1 + 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Doc/library/site.rst b/Doc/library/site.rst --- a/Doc/library/site.rst +++ b/Doc/library/site.rst @@ -99,7 +99,11 @@ :mod:`sitecustomize`, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an :exc:`ImportError` exception, it is -silently ignored. +silently ignored. If Python is started without output streams available, as +with :file:`pythonw.exe` on Windows (which is used by default to start IDLE), +attempted output from :mod:`sitecustomize` is ignored. Any exception other +than :exc:`ImportError` causes a silent and perhaps mysterious failure of the +process. .. index:: module: usercustomize diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1427,6 +1427,7 @@ Jason Williams John Williams Sue Williams +Carol Willing Steven Willis Frank Willison Geoff Wilson -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 07:00:15 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 07:00:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Closes_21048?= =?utf-8?q?=3A_Index_=27as=27_in_import_and_with_statements=2E?= Message-ID: <3gHrJb1Yfcz7LjX@mail.python.org> http://hg.python.org/cpython/rev/016d326c5aac changeset: 90500:016d326c5aac branch: 2.7 parent: 90497:79a4560a702f user: Terry Jan Reedy date: Tue Apr 29 00:58:48 2014 -0400 summary: Closes 21048: Index 'as' in import and with statements. files: Doc/reference/compound_stmts.rst | 4 +++- Doc/reference/simple_stmts.rst | 1 + 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -331,7 +331,9 @@ The :keyword:`with` statement ============================= -.. index:: statement: with +.. index:: + statement: with + single: as; with statement .. versionadded:: 2.5 diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -653,6 +653,7 @@ single: module; importing pair: name; binding keyword: from + single: as; import statement .. productionlist:: import_stmt: "import" `module` ["as" `name`] ( "," `module` ["as" `name`] )* -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 07:00:16 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 07:00:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closes_21048?= =?utf-8?q?=3A_Index_=27as=27_in_import_and_with_statements=2E?= Message-ID: <3gHrJc3Vgqz7LkJ@mail.python.org> http://hg.python.org/cpython/rev/9a0fc12991e2 changeset: 90501:9a0fc12991e2 branch: 3.4 parent: 90498:3fef95842314 user: Terry Jan Reedy date: Tue Apr 29 00:58:56 2014 -0400 summary: Closes 21048: Index 'as' in import and with statements. files: Doc/reference/compound_stmts.rst | 4 +++- Doc/reference/simple_stmts.rst | 2 ++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -348,7 +348,9 @@ The :keyword:`with` statement ============================= -.. index:: statement: with +.. index:: + statement: with + single: as; with statement The :keyword:`with` statement is used to wrap the execution of a block with methods defined by a context manager (see section :ref:`context-managers`). diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -656,6 +656,8 @@ If the requested module is retrieved successfully, it will be made available in the local namespace in one of three ways: +.. index:: single: as; import statement + * If the module name is followed by :keyword:`as`, then the name following :keyword:`as` is bound directly to the imported module. * If no other name is specified, and the module being imported is a top -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 07:00:17 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 07:00:17 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3gHrJd5cp4z7LjX@mail.python.org> http://hg.python.org/cpython/rev/3b9a03afed0e changeset: 90502:3b9a03afed0e parent: 90499:b82299695616 parent: 90501:9a0fc12991e2 user: Terry Jan Reedy date: Tue Apr 29 00:59:19 2014 -0400 summary: Merge with 3.4 files: Doc/reference/compound_stmts.rst | 4 +++- Doc/reference/simple_stmts.rst | 2 ++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -348,7 +348,9 @@ The :keyword:`with` statement ============================= -.. index:: statement: with +.. index:: + statement: with + single: as; with statement The :keyword:`with` statement is used to wrap the execution of a block with methods defined by a context manager (see section :ref:`context-managers`). diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -656,6 +656,8 @@ If the requested module is retrieved successfully, it will be made available in the local namespace in one of three ways: +.. index:: single: as; import statement + * If the module name is followed by :keyword:`as`, then the name following :keyword:`as` is bound directly to the imported module. * If no other name is specified, and the module being imported is a top -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 07:20:39 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 07:20:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogQ2xvc2VzICMyMTA1?= =?utf-8?q?5=3A_Index_=28augmented=29_assignment_symbols=2E?= Message-ID: <3gHrm74LDRz7Lk1@mail.python.org> http://hg.python.org/cpython/rev/d32b6900da6f changeset: 90503:d32b6900da6f branch: 2.7 parent: 90500:016d326c5aac user: Terry Jan Reedy date: Tue Apr 29 01:19:05 2014 -0400 summary: Closes #21055: Index (augmented) assignment symbols. files: Doc/reference/simple_stmts.rst | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -72,6 +72,7 @@ ===================== .. index:: + single: =; assignment statement pair: assignment; statement pair: binding; name pair: rebinding; name @@ -241,6 +242,18 @@ .. index:: pair: augmented; assignment single: statement; assignment, augmented + single: +=; augmented assignment + single: -=; augmented assignment + single: *=; augmented assignment + single: /=; augmented assignment + single: %=; augmented assignment + single: &=; augmented assignment + single: ^=; augmented assignment + single: |=; augmented assignment + single: **=; augmented assignment + single: //=; augmented assignment + single: >>=; augmented assignment + single: <<=; augmented assignment Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 07:20:40 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 07:20:40 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMDU1?= =?utf-8?q?=3A_Index_=28augmented=29_assignment_symbols=2E?= Message-ID: <3gHrm86wGZz7Lk9@mail.python.org> http://hg.python.org/cpython/rev/0b946c8d7837 changeset: 90504:0b946c8d7837 branch: 3.4 parent: 90501:9a0fc12991e2 user: Terry Jan Reedy date: Tue Apr 29 01:19:17 2014 -0400 summary: Issue #21055: Index (augmented) assignment symbols. files: Doc/reference/simple_stmts.rst | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -70,6 +70,7 @@ ===================== .. index:: + single: =; assignment statement pair: assignment; statement pair: binding; name pair: rebinding; name @@ -260,6 +261,18 @@ .. index:: pair: augmented; assignment single: statement; assignment, augmented + single: +=; augmented assignment + single: -=; augmented assignment + single: *=; augmented assignment + single: /=; augmented assignment + single: %=; augmented assignment + single: &=; augmented assignment + single: ^=; augmented assignment + single: |=; augmented assignment + single: **=; augmented assignment + single: //=; augmented assignment + single: >>=; augmented assignment + single: <<=; augmented assignment Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 07:20:42 2014 From: python-checkins at python.org (terry.reedy) Date: Tue, 29 Apr 2014 07:20:42 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E4?= Message-ID: <3gHrmB2WDrz7Lk1@mail.python.org> http://hg.python.org/cpython/rev/817d86f627c4 changeset: 90505:817d86f627c4 parent: 90502:3b9a03afed0e parent: 90504:0b946c8d7837 user: Terry Jan Reedy date: Tue Apr 29 01:19:41 2014 -0400 summary: Merge with 3.4 files: Doc/reference/simple_stmts.rst | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -70,6 +70,7 @@ ===================== .. index:: + single: =; assignment statement pair: assignment; statement pair: binding; name pair: rebinding; name @@ -260,6 +261,18 @@ .. index:: pair: augmented; assignment single: statement; assignment, augmented + single: +=; augmented assignment + single: -=; augmented assignment + single: *=; augmented assignment + single: /=; augmented assignment + single: %=; augmented assignment + single: &=; augmented assignment + single: ^=; augmented assignment + single: |=; augmented assignment + single: **=; augmented assignment + single: //=; augmented assignment + single: >>=; augmented assignment + single: <<=; augmented assignment Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 10:03:37 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 10:03:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2320951=3A_SSLSocke?= =?utf-8?q?t=2Esend=28=29_now_raises_either_SSLWantReadError_or?= Message-ID: <3gHwN92l34z7LjX@mail.python.org> http://hg.python.org/cpython/rev/3cf067049211 changeset: 90506:3cf067049211 user: Antoine Pitrou date: Tue Apr 29 10:03:28 2014 +0200 summary: Issue #20951: SSLSocket.send() now raises either SSLWantReadError or SSLWantWriteError on a non-blocking socket if the operation would block. Previously, it would return 0. Patch by Nikolaus Rath. files: Doc/library/ssl.rst | 21 +++++++++++++++++++-- Lib/ssl.py | 12 +----------- Lib/test/test_ssl.py | 30 ++++++++++++++++++++++++++++++ Misc/NEWS | 4 ++++ 4 files changed, 54 insertions(+), 13 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1604,8 +1604,25 @@ Notes on non-blocking sockets ----------------------------- -When working with non-blocking sockets, there are several things you need -to be aware of: +SSL sockets behave slightly different than regular sockets in +non-blocking mode. When working with non-blocking sockets, there are +thus several things you need to be aware of: + +- Most :class:`SSLSocket` methods will raise either + :exc:`SSLWantWriteError` or :exc:`SSLWantReadError` instead of + :exc:`BlockingIOError` if an I/O operation would + block. :exc:`SSLWantReadError` will be raised if a read operation on + the underlying socket is necessary, and :exc:`SSLWantWriteError` for + a write operation on the underlying socket. Note that attempts to + *write* to an SSL socket may require *reading* from the underlying + socket first, and attempts to *read* from the SSL socket may require + a prior *write* to the underlying socket. + + .. versionchanged:: 3.5 + + In earlier Python versions, the :meth:`!SSLSocket.send` method + returned zero instead of raising :exc:`SSLWantWriteError` or + :exc:`SSLWantReadError`. - Calling :func:`~select.select` tells you that the OS-level socket can be read from (or written to), but it does not imply that there is sufficient diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -664,17 +664,7 @@ raise ValueError( "non-zero flags not allowed in calls to send() on %s" % self.__class__) - try: - v = self._sslobj.write(data) - except SSLError as x: - if x.args[0] == SSL_ERROR_WANT_READ: - return 0 - elif x.args[0] == SSL_ERROR_WANT_WRITE: - return 0 - else: - raise - else: - return v + return self._sslobj.write(data) else: return socket.send(self, data, flags) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -2547,6 +2547,36 @@ s.write(b"over\n") s.close() + def test_nonblocking_send(self): + server = ThreadedEchoServer(CERTFILE, + certreqs=ssl.CERT_NONE, + ssl_version=ssl.PROTOCOL_TLSv1, + cacerts=CERTFILE, + chatty=True, + connectionchatty=False) + with server: + s = ssl.wrap_socket(socket.socket(), + server_side=False, + certfile=CERTFILE, + ca_certs=CERTFILE, + cert_reqs=ssl.CERT_NONE, + ssl_version=ssl.PROTOCOL_TLSv1) + s.connect((HOST, server.port)) + s.setblocking(False) + + # If we keep sending data, at some point the buffers + # will be full and the call will block + buf = bytearray(8192) + def fill_buffer(): + while True: + s.send(buf) + self.assertRaises((ssl.SSLWantWriteError, + ssl.SSLWantReadError), fill_buffer) + + # Now read all the output and discard it + s.setblocking(True) + s.close() + def test_handshake_timeout(self): # Issue #5103: SSL handshake must respect the socket timeout server = socket.socket(socket.AF_INET) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,10 @@ Library ------- +- Issue #20951: SSLSocket.send() now raises either SSLWantReadError or + SSLWantWriteError on a non-blocking socket if the operation would block. + Previously, it would return 0. Patch by Nikolaus Rath. + - Issue #13248: removed previously deprecated asyncore.dispatcher __getattr__ cheap inheritance hack. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 10:06:07 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 10:06:07 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_porting_note_for_issue?= =?utf-8?q?_=2320951=2E?= Message-ID: <3gHwR36rY3z7LjQ@mail.python.org> http://hg.python.org/cpython/rev/b0f6983d63df changeset: 90507:b0f6983d63df user: Antoine Pitrou date: Tue Apr 29 10:05:59 2014 +0200 summary: Add porting note for issue #20951. files: Doc/whatsnew/3.5.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -213,3 +213,7 @@ if it represented midnight in UTC. This behavior was considered obscure and error-prone and has been removed in Python 3.5. See :issue:`13936` for full details. + +* :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` + or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation + would block. Previously, it would return 0. See :issue:`20951`. -- Repository URL: http://hg.python.org/cpython From solipsis at pitrou.net Tue Apr 29 10:00:39 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Tue, 29 Apr 2014 10:00:39 +0200 Subject: [Python-checkins] Daily reference leaks (3b203f000256): sum=20 Message-ID: results for 3b203f000256 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_nntplib leaked [13, 0, 0] references, sum=13 test_nntplib leaked [4, 0, 0] memory blocks, sum=4 test_site leaked [0, 2, -2] references, sum=0 test_site leaked [0, 2, -2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog4FX3zy', '-x'] From python-checkins at python.org Tue Apr 29 10:27:26 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 10:27:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321057=3A_TextIOWr?= =?utf-8?q?apper_now_allows_the_underlying_binary_stream=27s_read=28=29_or?= Message-ID: <3gHwvf0zXBz7LkF@mail.python.org> http://hg.python.org/cpython/rev/2a1d63f09560 changeset: 90508:2a1d63f09560 user: Antoine Pitrou date: Tue Apr 29 10:14:02 2014 +0200 summary: Issue #21057: TextIOWrapper now allows the underlying binary stream's read() or read1() method to return an arbitrary bytes-like object (such as a memoryview). Patch by Nikolaus Rath. files: Lib/test/test_io.py | 28 ++++++++++++++++++++++++++ Misc/NEWS | 4 +++ Modules/_io/textio.c | 34 ++++++++++++++++++------------- 3 files changed, 52 insertions(+), 14 deletions(-) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -2681,6 +2681,34 @@ self.assertFalse(err) self.assertEqual("ok", out.decode().strip()) + def test_read_byteslike(self): + r = MemviewBytesIO(b'Just some random string\n') + t = self.TextIOWrapper(r, 'utf-8') + + # TextIOwrapper will not read the full string, because + # we truncate it to a multiple of the native int size + # so that we can construct a more complex memoryview. + bytes_val = _to_memoryview(r.getvalue()).tobytes() + + self.assertEqual(t.read(200), bytes_val.decode('utf-8')) + +class MemviewBytesIO(io.BytesIO): + '''A BytesIO object whose read method returns memoryviews + rather than bytes''' + + def read1(self, len_): + return _to_memoryview(super().read1(len_)) + + def read(self, len_): + return _to_memoryview(super().read(len_)) + +def _to_memoryview(buf): + '''Convert bytes-object *buf* to a non-trivial memoryview''' + + arr = array.array('i') + idx = len(buf) - len(buf) % arr.itemsize + arr.frombytes(buf[:idx]) + return memoryview(arr) class CTextIOWrapperTest(TextIOWrapperTest): io = io diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,10 @@ Library ------- +- Issue #21057: TextIOWrapper now allows the underlying binary stream's + read() or read1() method to return an arbitrary bytes-like object + (such as a memoryview). Patch by Nikolaus Rath. + - Issue #20951: SSLSocket.send() now raises either SSLWantReadError or SSLWantWriteError on a non-blocking socket if the operation would block. Previously, it would return 0. Patch by Nikolaus Rath. diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -1439,6 +1439,7 @@ PyObject *dec_buffer = NULL; PyObject *dec_flags = NULL; PyObject *input_chunk = NULL; + Py_buffer input_chunk_buf; PyObject *decoded_chars, *chunk_size; Py_ssize_t nbytes, nchars; int eof; @@ -1470,6 +1471,15 @@ Py_DECREF(state); return -1; } + + if (!PyBytes_Check(dec_buffer)) { + PyErr_Format(PyExc_TypeError, + "decoder getstate() should have returned a bytes " + "object, not '%.200s'", + Py_TYPE(dec_buffer)->tp_name); + Py_DECREF(state); + return -1; + } Py_INCREF(dec_buffer); Py_INCREF(dec_flags); Py_DECREF(state); @@ -1482,23 +1492,24 @@ chunk_size = PyLong_FromSsize_t(Py_MAX(self->chunk_size, size_hint)); if (chunk_size == NULL) goto fail; + input_chunk = PyObject_CallMethodObjArgs(self->buffer, (self->has_read1 ? _PyIO_str_read1: _PyIO_str_read), chunk_size, NULL); Py_DECREF(chunk_size); if (input_chunk == NULL) goto fail; - if (!PyBytes_Check(input_chunk)) { + + if (PyObject_GetBuffer(input_chunk, &input_chunk_buf, 0) != 0) { PyErr_Format(PyExc_TypeError, - "underlying %s() should have returned a bytes object, " + "underlying %s() should have returned a bytes-like object, " "not '%.200s'", (self->has_read1 ? "read1": "read"), Py_TYPE(input_chunk)->tp_name); goto fail; } - nbytes = PyBytes_Size(input_chunk); + nbytes = input_chunk_buf.len; eof = (nbytes == 0); - if (Py_TYPE(self->decoder) == &PyIncrementalNewlineDecoder_Type) { decoded_chars = _PyIncrementalNewlineDecoder_decode( self->decoder, input_chunk, eof); @@ -1507,6 +1518,7 @@ decoded_chars = PyObject_CallMethodObjArgs(self->decoder, _PyIO_str_decode, input_chunk, eof ? Py_True : Py_False, NULL); } + PyBuffer_Release(&input_chunk_buf); if (check_decoded(decoded_chars) < 0) goto fail; @@ -1523,18 +1535,12 @@ /* At the snapshot point, len(dec_buffer) bytes before the read, the * next input to be decoded is dec_buffer + input_chunk. */ - PyObject *next_input = PyNumber_Add(dec_buffer, input_chunk); - if (next_input == NULL) - goto fail; - if (!PyBytes_Check(next_input)) { - PyErr_Format(PyExc_TypeError, - "decoder getstate() should have returned a bytes " - "object, not '%.200s'", - Py_TYPE(next_input)->tp_name); - Py_DECREF(next_input); + PyObject *next_input = dec_buffer; + PyBytes_Concat(&next_input, input_chunk); + if (next_input == NULL) { + dec_buffer = NULL; /* Reference lost to PyBytes_Concat */ goto fail; } - Py_DECREF(dec_buffer); Py_CLEAR(self->snapshot); self->snapshot = Py_BuildValue("NN", dec_flags, next_input); } -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 10:27:27 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 10:27:27 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_failure_in_test=5Fpopl?= =?utf-8?q?ib_after_issue_=2320951=2E?= Message-ID: <3gHwvg2jQvz7Ljv@mail.python.org> http://hg.python.org/cpython/rev/7f50e1836ddb changeset: 90509:7f50e1836ddb user: Antoine Pitrou date: Tue Apr 29 10:27:09 2014 +0200 summary: Fix failure in test_poplib after issue #20951. files: Lib/test/test_poplib.py | 15 +++++---------- 1 files changed, 5 insertions(+), 10 deletions(-) diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py --- a/Lib/test/test_poplib.py +++ b/Lib/test/test_poplib.py @@ -349,23 +349,18 @@ if SUPPORTS_SSL: + from test.test_ftplib import SSLConnection - class DummyPOP3_SSLHandler(DummyPOP3Handler): + class DummyPOP3_SSLHandler(SSLConnection, DummyPOP3Handler): def __init__(self, conn): asynchat.async_chat.__init__(self, conn) - ssl_socket = ssl.wrap_socket(self.socket, certfile=CERTFILE, - server_side=True, - do_handshake_on_connect=False) - self.del_channel() - self.set_socket(ssl_socket) - # Must try handshake before calling push() - self.tls_active = True - self.tls_starting = True - self._do_tls_handshake() + self.secure_connection() self.set_terminator(b"\r\n") self.in_buffer = [] self.push('+OK dummy pop3 server ready. ') + self.tls_active = True + self.tls_starting = False @requires_ssl -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 12:14:59 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 12:14:59 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMzIx?= =?utf-8?q?=3A_itertools=2Eislice=28=29_now_releases_the_reference_to_the_?= =?utf-8?q?source?= Message-ID: <3gHzHl6pSXz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/b795105db23a changeset: 90510:b795105db23a branch: 3.4 parent: 90504:0b946c8d7837 user: Antoine Pitrou date: Tue Apr 29 12:13:46 2014 +0200 summary: Issue #21321: itertools.islice() now releases the reference to the source iterator when the slice is exhausted. Patch by Anton Afanasyev. files: Lib/test/test_itertools.py | 13 +++++++++++-- Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/itertoolsmodule.c | 25 ++++++++++++++++++++++--- 4 files changed, 37 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1,7 +1,7 @@ import unittest from test import support from itertools import * -from weakref import proxy +import weakref from decimal import Decimal from fractions import Fraction import sys @@ -1087,6 +1087,15 @@ list(range(*args))) self.pickletest(islice(range(100), *args)) + # Issue #21321: check source iterator is not referenced + # from islice() after the latter has been exhausted + it = (x for x in (1, 2)) + wr = weakref.ref(it) + it = islice(it, 1) + self.assertIsNotNone(wr()) + list(it) # exhaust the iterator + self.assertIsNone(wr()) + def test_takewhile(self): data = [1, 3, 5, 20, 2, 4, 6, 8] self.assertEqual(list(takewhile(underten, data)), [1, 3, 5]) @@ -1203,7 +1212,7 @@ # test that tee objects are weak referencable a, b = tee(range(10)) - p = proxy(a) + p = weakref.proxy(a) self.assertEqual(getattr(p, '__class__'), type(b)) del a self.assertRaises(ReferenceError, getattr, p, '__class__') diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -17,6 +17,7 @@ David Abrahams Marc Abramowitz Ron Adam +Anton Afanasyev Ali Afshar Nitika Agarwal Jim Ahlstrom diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -39,6 +39,9 @@ Library ------- +- Issue #21321: itertools.islice() now releases the reference to the source + iterator when the slice is exhausted. Patch by Anton Afanasyev. + - Issue #9815: assertRaises now tries to clear references to local variables in the exception's traceback. diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -1492,19 +1492,22 @@ Py_ssize_t oldnext; PyObject *(*iternext)(PyObject *); + if (it == NULL) + return NULL; + iternext = *Py_TYPE(it)->tp_iternext; while (lz->cnt < lz->next) { item = iternext(it); if (item == NULL) - return NULL; + goto empty; Py_DECREF(item); lz->cnt++; } if (stop != -1 && lz->cnt >= stop) - return NULL; + goto empty; item = iternext(it); if (item == NULL) - return NULL; + goto empty; lz->cnt++; oldnext = lz->next; /* The (size_t) cast below avoids the danger of undefined @@ -1513,6 +1516,10 @@ if (lz->next < oldnext || (stop != -1 && lz->next > stop)) lz->next = stop; return item; + +empty: + Py_CLEAR(lz->it); + return NULL; } static PyObject * @@ -1522,6 +1529,18 @@ * then 'setstate' with the next and count */ PyObject *stop; + if (lz->it == NULL) { + PyObject *empty_list; + PyObject *empty_it; + empty_list = PyList_New(0); + if (empty_list == NULL) + return NULL; + empty_it = PyObject_GetIter(empty_list); + Py_DECREF(empty_list); + if (empty_it == NULL) + return NULL; + return Py_BuildValue("O(Nn)n", Py_TYPE(lz), empty_it, 0, 0); + } if (lz->stop == -1) { stop = Py_None; Py_INCREF(stop); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 12:15:01 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 12:15:01 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321321=3A_itertools=2Eislice=28=29_now_releases_?= =?utf-8?q?the_reference_to_the_source?= Message-ID: <3gHzHn2dVWz7Lk9@mail.python.org> http://hg.python.org/cpython/rev/a627b3e3c9c8 changeset: 90511:a627b3e3c9c8 parent: 90509:7f50e1836ddb parent: 90510:b795105db23a user: Antoine Pitrou date: Tue Apr 29 12:14:47 2014 +0200 summary: Issue #21321: itertools.islice() now releases the reference to the source iterator when the slice is exhausted. Patch by Anton Afanasyev. files: Lib/test/test_itertools.py | 13 +++++++++++-- Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/itertoolsmodule.c | 25 ++++++++++++++++++++++--- 4 files changed, 37 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1,7 +1,7 @@ import unittest from test import support from itertools import * -from weakref import proxy +import weakref from decimal import Decimal from fractions import Fraction import sys @@ -1087,6 +1087,15 @@ list(range(*args))) self.pickletest(islice(range(100), *args)) + # Issue #21321: check source iterator is not referenced + # from islice() after the latter has been exhausted + it = (x for x in (1, 2)) + wr = weakref.ref(it) + it = islice(it, 1) + self.assertIsNotNone(wr()) + list(it) # exhaust the iterator + self.assertIsNone(wr()) + def test_takewhile(self): data = [1, 3, 5, 20, 2, 4, 6, 8] self.assertEqual(list(takewhile(underten, data)), [1, 3, 5]) @@ -1203,7 +1212,7 @@ # test that tee objects are weak referencable a, b = tee(range(10)) - p = proxy(a) + p = weakref.proxy(a) self.assertEqual(getattr(p, '__class__'), type(b)) del a self.assertRaises(ReferenceError, getattr, p, '__class__') diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -17,6 +17,7 @@ David Abrahams Marc Abramowitz Ron Adam +Anton Afanasyev Ali Afshar Nitika Agarwal Jim Ahlstrom diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,9 @@ Library ------- +- Issue #21321: itertools.islice() now releases the reference to the source + iterator when the slice is exhausted. Patch by Anton Afanasyev. + - Issue #21057: TextIOWrapper now allows the underlying binary stream's read() or read1() method to return an arbitrary bytes-like object (such as a memoryview). Patch by Nikolaus Rath. diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -1492,19 +1492,22 @@ Py_ssize_t oldnext; PyObject *(*iternext)(PyObject *); + if (it == NULL) + return NULL; + iternext = *Py_TYPE(it)->tp_iternext; while (lz->cnt < lz->next) { item = iternext(it); if (item == NULL) - return NULL; + goto empty; Py_DECREF(item); lz->cnt++; } if (stop != -1 && lz->cnt >= stop) - return NULL; + goto empty; item = iternext(it); if (item == NULL) - return NULL; + goto empty; lz->cnt++; oldnext = lz->next; /* The (size_t) cast below avoids the danger of undefined @@ -1513,6 +1516,10 @@ if (lz->next < oldnext || (stop != -1 && lz->next > stop)) lz->next = stop; return item; + +empty: + Py_CLEAR(lz->it); + return NULL; } static PyObject * @@ -1522,6 +1529,18 @@ * then 'setstate' with the next and count */ PyObject *stop; + if (lz->it == NULL) { + PyObject *empty_list; + PyObject *empty_it; + empty_list = PyList_New(0); + if (empty_list == NULL) + return NULL; + empty_it = PyObject_GetIter(empty_list); + Py_DECREF(empty_list); + if (empty_it == NULL) + return NULL; + return Py_BuildValue("O(Nn)n", Py_TYPE(lz), empty_it, 0, 0); + } if (lz->stop == -1) { stop = Py_None; Py_INCREF(stop); -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 12:27:00 2014 From: python-checkins at python.org (antoine.pitrou) Date: Tue, 29 Apr 2014 12:27:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMzIx?= =?utf-8?q?=3A_itertools=2Eislice=28=29_now_releases_the_reference_to_the_?= =?utf-8?q?source?= Message-ID: <3gHzYc54Cwz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/8ee76e1b5aa6 changeset: 90512:8ee76e1b5aa6 branch: 2.7 parent: 90503:d32b6900da6f user: Antoine Pitrou date: Tue Apr 29 12:13:46 2014 +0200 summary: Issue #21321: itertools.islice() now releases the reference to the source iterator when the slice is exhausted. Patch by Anton Afanasyev. files: Lib/test/test_itertools.py | 13 +++++++++++-- Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/itertoolsmodule.c | 13 ++++++++++--- 4 files changed, 25 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -1,7 +1,7 @@ import unittest from test import test_support from itertools import * -from weakref import proxy +import weakref from decimal import Decimal from fractions import Fraction import sys @@ -792,6 +792,15 @@ self.assertEqual(list(islice(c, 1, 3, 50)), [1]) self.assertEqual(next(c), 3) + # Issue #21321: check source iterator is not referenced + # from islice() after the latter has been exhausted + it = (x for x in (1, 2)) + wr = weakref.ref(it) + it = islice(it, 1) + self.assertIsNotNone(wr()) + list(it) # exhaust the iterator + self.assertIsNone(wr()) + def test_takewhile(self): data = [1, 3, 5, 20, 2, 4, 6, 8] underten = lambda x: x<10 @@ -901,7 +910,7 @@ # test that tee objects are weak referencable a, b = tee(xrange(10)) - p = proxy(a) + p = weakref.proxy(a) self.assertEqual(getattr(p, '__class__'), type(b)) del a self.assertRaises(ReferenceError, getattr, p, '__class__') diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -17,6 +17,7 @@ David Abrahams Marc Abramowitz Ron Adam +Anton Afanasyev Ali Afshar Nitika Agarwal Jim Ahlstrom diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,9 @@ Library ------- +- Issue #21321: itertools.islice() now releases the reference to the source + iterator when the slice is exhausted. Patch by Anton Afanasyev. + - Issue #9291: Do not attempt to re-encode mimetype data read from registry in ANSI mode. Initial patches by Dmitry Jemerov & Vladimir Iofik. diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -1241,19 +1241,22 @@ Py_ssize_t oldnext; PyObject *(*iternext)(PyObject *); + if (it == NULL) + return NULL; + iternext = *Py_TYPE(it)->tp_iternext; while (lz->cnt < lz->next) { item = iternext(it); if (item == NULL) - return NULL; + goto empty; Py_DECREF(item); lz->cnt++; } if (stop != -1 && lz->cnt >= stop) - return NULL; + goto empty; item = iternext(it); if (item == NULL) - return NULL; + goto empty; lz->cnt++; oldnext = lz->next; /* The (size_t) cast below avoids the danger of undefined @@ -1262,6 +1265,10 @@ if (lz->next < oldnext || (stop != -1 && lz->next > stop)) lz->next = stop; return item; + +empty: + Py_CLEAR(lz->it); + return NULL; } PyDoc_STRVAR(islice_doc, -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 16:47:56 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 29 Apr 2014 16:47:56 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE3Mzg2?= =?utf-8?q?=3A_List_the_=27htmlview=27_target_in_the_Doc/Makefile_help_out?= =?utf-8?q?put=2E?= Message-ID: <3gJ5Lh0gVBz7Ljw@mail.python.org> http://hg.python.org/cpython/rev/c75a2282166f changeset: 90513:c75a2282166f branch: 3.4 parent: 90510:b795105db23a user: Zachary Ware date: Tue Apr 29 09:24:40 2014 -0500 summary: Issue #17386: List the 'htmlview' target in the Doc/Makefile help output. files: Doc/Makefile | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Doc/Makefile b/Doc/Makefile --- a/Doc/Makefile +++ b/Doc/Makefile @@ -22,6 +22,7 @@ @echo "Please use \`make ' where is one of" @echo " clean to remove build files" @echo " html to make standalone HTML files" + @echo " htmlview to open the index page built by the html target in your browser" @echo " htmlhelp to make HTML files and a HTML help project" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " text to make plain text files" -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 16:47:57 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 29 Apr 2014 16:47:57 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE3Mzg2?= =?utf-8?q?=3A_Update_Doc/README=2Etxt_to_list_all_targets?= Message-ID: <3gJ5Lj2L8lz7LkK@mail.python.org> http://hg.python.org/cpython/rev/c378c67c4170 changeset: 90514:c378c67c4170 branch: 3.4 user: Zachary Ware date: Tue Apr 29 09:26:56 2014 -0500 summary: Issue #17386: Update Doc/README.txt to list all targets supported by Doc/Makefile, as well as a bit of modernization, cleanup, and notes about the difference between Unix and Windows. files: Doc/README.txt | 29 +++++++++++++++++++++++------ 1 files changed, 23 insertions(+), 6 deletions(-) diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -5,7 +5,7 @@ documentation. You don't need to build them yourself, prebuilt versions are available at . -Documentation on the authoring Python documentation, including information about +Documentation on authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the developers guide . @@ -26,8 +26,7 @@ make html -to build the HTML output files. To view the generated HTML, point your favorite -browser at the top-level index `build/html/index.html` after running "make". +to build the HTML output files. On Windows, we try to emulate the Makefile as closely as possible with a ``make.bat`` file. @@ -35,18 +34,29 @@ To use a Python interpreter that's not called ``python``, use the standard way to set Makefile variables, using e.g. :: - make html PYTHON=/usr/bin/python2.5 + make html PYTHON=python3 + +On Windows, set the PYTHON environment variable instead. + +To use a specific sphinx-build (something other than ``sphinx-build``), set +the SPHINXBUILD variable. Available make targets are: + * "clean", which removes all build files. + * "html", which builds standalone HTML files for offline viewing. + * "htmlview", which re-uses the "html" builder, but then opens the main page + in your default web browser. + * "htmlhelp", which builds HTML files and a HTML Help project file usable to convert them into a single Compiled HTML (.chm) file -- these are popular under Microsoft Windows, but very handy on every platform. - To create the CHM file, you need to run the Microsoft HTML Help Workshop over - the generated project (.hhp) file. + To create the CHM file, you need to run the Microsoft HTML Help Workshop + over the generated project (.hhp) file. The make.bat script does this for + you on Windows. * "latex", which builds LaTeX source files as input to "pdflatex" to produce PDF documents. @@ -75,6 +85,13 @@ * "suspicious", which checks the parsed markup for text that looks like malformed and thus unconverted reST. + * "check", which checks for frequent markup errors. + + * "serve", which serves the build/html directory on port 8000. + + * "dist", (Unix only) which creates distributable archives of HTML, text, + PDF, and EPUB builds. + Without make ------------ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 16:47:58 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 29 Apr 2014 16:47:58 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzE3Mzg2?= =?utf-8?q?=3A_Expand_Doc/make=2Ebat_to_be_much_more_similar_to_Doc/Makefi?= =?utf-8?q?le?= Message-ID: <3gJ5Lk4mBpz7LlD@mail.python.org> http://hg.python.org/cpython/rev/a172f26195f6 changeset: 90515:a172f26195f6 branch: 3.4 user: Zachary Ware date: Tue Apr 29 09:45:33 2014 -0500 summary: Issue #17386: Expand Doc/make.bat to be much more similar to Doc/Makefile files: Doc/make.bat | 121 +++++++++++++++++++++++++++++++------- Misc/NEWS | 3 + 2 files changed, 101 insertions(+), 23 deletions(-) diff --git a/Doc/make.bat b/Doc/make.bat --- a/Doc/make.bat +++ b/Doc/make.bat @@ -1,40 +1,115 @@ -@@echo off + at echo off setlocal +pushd %~dp0 + +set this=%~n0 + if "%SPHINXBUILD%" EQU "" set SPHINXBUILD=sphinx-build if "%PYTHON%" EQU "" set PYTHON=py -if "%HTMLHELP%" EQU "" set HTMLHELP=%ProgramFiles%\HTML Help Workshop\hhc.exe + +if DEFINED ProgramFiles(x86) set _PRGMFLS=%ProgramFiles(x86)% +if NOT DEFINED ProgramFiles(x86) set _PRGMFLS=%ProgramFiles% +if "%HTMLHELP%" EQU "" set HTMLHELP=%_PRGMFLS%\HTML Help Workshop\hhc.exe + if "%DISTVERSION%" EQU "" for /f "usebackq" %%v in (`%PYTHON% tools/sphinxext/patchlevel.py`) do set DISTVERSION=%%v +if "%BUILDDIR%" EQU "" set BUILDDIR=build + +rem Targets that don't require sphinx-build if "%1" EQU "" goto help -if "%1" EQU "html" goto build -if "%1" EQU "htmlhelp" goto build -if "%1" EQU "latex" goto build -if "%1" EQU "text" goto build -if "%1" EQU "suspicious" goto build -if "%1" EQU "linkcheck" goto build -if "%1" EQU "changes" goto build +if "%1" EQU "help" goto help +if "%1" EQU "check" goto check +if "%1" EQU "serve" goto serve +if "%1" == "clean" ( + rmdir /q /s %BUILDDIR% + goto end +) + +%SPHINXBUILD% 2> nul +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + goto end +) + +rem Targets that do require sphinx-build and have their own label +if "%1" EQU "htmlview" goto htmlview + +rem Everything else +goto build :help -set this=%~n0 -echo HELP +echo.usage: %this% BUILDER [filename ...] echo. -echo %this% html -echo %this% htmlhelp -echo %this% latex -echo %this% text -echo %this% suspicious -echo %this% linkcheck -echo %this% changes +echo.Call %this% with the desired Sphinx builder as the first argument, e.g. +echo.``%this% html`` or ``%this% doctest``. Interesting targets that are +echo.always available include: echo. +echo. Provided by Sphinx: +echo. html, htmlhelp, latex, text +echo. suspicious, linkcheck, changes, doctest +echo. Provided by this script: +echo. clean, check, serve, htmlview +echo. +echo.All arguments past the first one are passed through to sphinx-build as +echo.filenames to build or are ignored. See README.txt in this directory or +echo.the documentation for your version of Sphinx for more exhaustive lists +echo.of available targets and descriptions of each. +echo. +echo.This script assumes that the SPHINXBUILD environment variable contains +echo.a legitimate command for calling sphinx-build, or that sphinx-build is +echo.on your PATH if SPHINXBUILD is not set. Options for sphinx-build can +echo.be passed by setting the SPHINXOPTS environment variable. goto end :build -if not exist build mkdir build -if not exist build\%1 mkdir build\%1 -if not exist build\doctrees mkdir build\doctrees -cmd /C %SPHINXBUILD% -b%1 -dbuild\doctrees . build\%* -if "%1" EQU "htmlhelp" "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp +if NOT "%PAPER%" == "" ( + set SPHINXOPTS=-D latex_paper_size=%PAPER% %SPHINXOPTS% +) +cmd /C %SPHINXBUILD% %SPHINXOPTS% -b%1 -dbuild\doctrees . %BUILDDIR%\%* + +if "%1" EQU "htmlhelp" ( + cmd /C "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp + rem hhc.exe seems to always exit with code 1, reset to 0 for less than 2 + if not errorlevel 2 cmd /C exit /b 0 +) + +echo. +if errorlevel 1 ( + echo.Build failed (exit code %ERRORLEVEL%^), check for error messages + echo.above. Any output will be found in %BUILDDIR%\%1 +) else ( + echo.Build succeeded. All output should be in %BUILDDIR%\%1 +) +goto end + +:htmlview +if NOT "%2" EQU "" ( + echo.Can't specify filenames to build with htmlview target, ignoring. +) +cmd /C %this% html + +if EXIST %BUILDDIR%\html\index.html ( + echo.Opening %BUILDDIR%\html\index.html in the default web browser... + start %BUILDDIR%\html\index.html +) + +goto end + +:check +cmd /C %PYTHON% tools\rstlint.py -i tools +goto end + +:serve +cmd /C %PYTHON% ..\Tools\scripts\serve.py %BUILDDIR%\html goto end :end +popd diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -237,6 +237,9 @@ Documentation ------------- +- Issue #17386: Expanded functionality of the ``Doc/make.bat`` script to make + it much more comparable to ``Doc/Makefile``. + - Issue #21043: Remove the recommendation for specific CA organizations and to mention the ability to load the OS certificates. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 16:48:00 2014 From: python-checkins at python.org (zach.ware) Date: Tue, 29 Apr 2014 16:48:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2317386=3A_Merge_with_3=2E4?= Message-ID: <3gJ5Lm0Y3Vz7LkK@mail.python.org> http://hg.python.org/cpython/rev/2b2577d79e80 changeset: 90516:2b2577d79e80 parent: 90511:a627b3e3c9c8 parent: 90515:a172f26195f6 user: Zachary Ware date: Tue Apr 29 09:47:39 2014 -0500 summary: Closes #17386: Merge with 3.4 files: Doc/Makefile | 1 + Doc/README.txt | 29 +++++++- Doc/make.bat | 121 +++++++++++++++++++++++++++++------- Misc/NEWS | 3 + 4 files changed, 125 insertions(+), 29 deletions(-) diff --git a/Doc/Makefile b/Doc/Makefile --- a/Doc/Makefile +++ b/Doc/Makefile @@ -22,6 +22,7 @@ @echo "Please use \`make ' where is one of" @echo " clean to remove build files" @echo " html to make standalone HTML files" + @echo " htmlview to open the index page built by the html target in your browser" @echo " htmlhelp to make HTML files and a HTML help project" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " text to make plain text files" diff --git a/Doc/README.txt b/Doc/README.txt --- a/Doc/README.txt +++ b/Doc/README.txt @@ -5,7 +5,7 @@ documentation. You don't need to build them yourself, prebuilt versions are available at . -Documentation on the authoring Python documentation, including information about +Documentation on authoring Python documentation, including information about both style and markup, is available in the "Documenting Python" chapter of the developers guide . @@ -26,8 +26,7 @@ make html -to build the HTML output files. To view the generated HTML, point your favorite -browser at the top-level index `build/html/index.html` after running "make". +to build the HTML output files. On Windows, we try to emulate the Makefile as closely as possible with a ``make.bat`` file. @@ -35,18 +34,29 @@ To use a Python interpreter that's not called ``python``, use the standard way to set Makefile variables, using e.g. :: - make html PYTHON=/usr/bin/python2.5 + make html PYTHON=python3 + +On Windows, set the PYTHON environment variable instead. + +To use a specific sphinx-build (something other than ``sphinx-build``), set +the SPHINXBUILD variable. Available make targets are: + * "clean", which removes all build files. + * "html", which builds standalone HTML files for offline viewing. + * "htmlview", which re-uses the "html" builder, but then opens the main page + in your default web browser. + * "htmlhelp", which builds HTML files and a HTML Help project file usable to convert them into a single Compiled HTML (.chm) file -- these are popular under Microsoft Windows, but very handy on every platform. - To create the CHM file, you need to run the Microsoft HTML Help Workshop over - the generated project (.hhp) file. + To create the CHM file, you need to run the Microsoft HTML Help Workshop + over the generated project (.hhp) file. The make.bat script does this for + you on Windows. * "latex", which builds LaTeX source files as input to "pdflatex" to produce PDF documents. @@ -75,6 +85,13 @@ * "suspicious", which checks the parsed markup for text that looks like malformed and thus unconverted reST. + * "check", which checks for frequent markup errors. + + * "serve", which serves the build/html directory on port 8000. + + * "dist", (Unix only) which creates distributable archives of HTML, text, + PDF, and EPUB builds. + Without make ------------ diff --git a/Doc/make.bat b/Doc/make.bat --- a/Doc/make.bat +++ b/Doc/make.bat @@ -1,40 +1,115 @@ -@@echo off + at echo off setlocal +pushd %~dp0 + +set this=%~n0 + if "%SPHINXBUILD%" EQU "" set SPHINXBUILD=sphinx-build if "%PYTHON%" EQU "" set PYTHON=py -if "%HTMLHELP%" EQU "" set HTMLHELP=%ProgramFiles%\HTML Help Workshop\hhc.exe + +if DEFINED ProgramFiles(x86) set _PRGMFLS=%ProgramFiles(x86)% +if NOT DEFINED ProgramFiles(x86) set _PRGMFLS=%ProgramFiles% +if "%HTMLHELP%" EQU "" set HTMLHELP=%_PRGMFLS%\HTML Help Workshop\hhc.exe + if "%DISTVERSION%" EQU "" for /f "usebackq" %%v in (`%PYTHON% tools/sphinxext/patchlevel.py`) do set DISTVERSION=%%v +if "%BUILDDIR%" EQU "" set BUILDDIR=build + +rem Targets that don't require sphinx-build if "%1" EQU "" goto help -if "%1" EQU "html" goto build -if "%1" EQU "htmlhelp" goto build -if "%1" EQU "latex" goto build -if "%1" EQU "text" goto build -if "%1" EQU "suspicious" goto build -if "%1" EQU "linkcheck" goto build -if "%1" EQU "changes" goto build +if "%1" EQU "help" goto help +if "%1" EQU "check" goto check +if "%1" EQU "serve" goto serve +if "%1" == "clean" ( + rmdir /q /s %BUILDDIR% + goto end +) + +%SPHINXBUILD% 2> nul +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + goto end +) + +rem Targets that do require sphinx-build and have their own label +if "%1" EQU "htmlview" goto htmlview + +rem Everything else +goto build :help -set this=%~n0 -echo HELP +echo.usage: %this% BUILDER [filename ...] echo. -echo %this% html -echo %this% htmlhelp -echo %this% latex -echo %this% text -echo %this% suspicious -echo %this% linkcheck -echo %this% changes +echo.Call %this% with the desired Sphinx builder as the first argument, e.g. +echo.``%this% html`` or ``%this% doctest``. Interesting targets that are +echo.always available include: echo. +echo. Provided by Sphinx: +echo. html, htmlhelp, latex, text +echo. suspicious, linkcheck, changes, doctest +echo. Provided by this script: +echo. clean, check, serve, htmlview +echo. +echo.All arguments past the first one are passed through to sphinx-build as +echo.filenames to build or are ignored. See README.txt in this directory or +echo.the documentation for your version of Sphinx for more exhaustive lists +echo.of available targets and descriptions of each. +echo. +echo.This script assumes that the SPHINXBUILD environment variable contains +echo.a legitimate command for calling sphinx-build, or that sphinx-build is +echo.on your PATH if SPHINXBUILD is not set. Options for sphinx-build can +echo.be passed by setting the SPHINXOPTS environment variable. goto end :build -if not exist build mkdir build -if not exist build\%1 mkdir build\%1 -if not exist build\doctrees mkdir build\doctrees -cmd /C %SPHINXBUILD% -b%1 -dbuild\doctrees . build\%* -if "%1" EQU "htmlhelp" "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp +if NOT "%PAPER%" == "" ( + set SPHINXOPTS=-D latex_paper_size=%PAPER% %SPHINXOPTS% +) +cmd /C %SPHINXBUILD% %SPHINXOPTS% -b%1 -dbuild\doctrees . %BUILDDIR%\%* + +if "%1" EQU "htmlhelp" ( + cmd /C "%HTMLHELP%" build\htmlhelp\python%DISTVERSION:.=%.hhp + rem hhc.exe seems to always exit with code 1, reset to 0 for less than 2 + if not errorlevel 2 cmd /C exit /b 0 +) + +echo. +if errorlevel 1 ( + echo.Build failed (exit code %ERRORLEVEL%^), check for error messages + echo.above. Any output will be found in %BUILDDIR%\%1 +) else ( + echo.Build succeeded. All output should be in %BUILDDIR%\%1 +) +goto end + +:htmlview +if NOT "%2" EQU "" ( + echo.Can't specify filenames to build with htmlview target, ignoring. +) +cmd /C %this% html + +if EXIST %BUILDDIR%\html\index.html ( + echo.Opening %BUILDDIR%\html\index.html in the default web browser... + start %BUILDDIR%\html\index.html +) + +goto end + +:check +cmd /C %PYTHON% tools\rstlint.py -i tools +goto end + +:serve +cmd /C %PYTHON% ..\Tools\scripts\serve.py %BUILDDIR%\html goto end :end +popd diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -340,6 +340,9 @@ Documentation ------------- +- Issue #17386: Expanded functionality of the ``Doc/make.bat`` script to make + it much more comparable to ``Doc/Makefile``. + - Issue #21312: Update the thread_foobar.h template file to include newer threading APIs. Patch by Jack McCracken. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 17:11:33 2014 From: python-checkins at python.org (tim.golden) Date: Tue, 29 Apr 2014 17:11:33 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE5NjMw?= =?utf-8?q?_Emphasise_that_the_file_parameter_to_marshal=2Edump_must_be_a_?= =?utf-8?q?real?= Message-ID: <3gJ5sx0PNWz7LjQ@mail.python.org> http://hg.python.org/cpython/rev/0f6bdc2b0e38 changeset: 90517:0f6bdc2b0e38 branch: 2.7 parent: 90512:8ee76e1b5aa6 user: Tim Golden date: Tue Apr 29 16:11:18 2014 +0100 summary: Issue #19630 Emphasise that the file parameter to marshal.dump must be a real file object files: Doc/library/marshal.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/library/marshal.rst b/Doc/library/marshal.rst --- a/Doc/library/marshal.rst +++ b/Doc/library/marshal.rst @@ -66,8 +66,9 @@ .. function:: dump(value, file[, version]) Write the value on the open file. The value must be a supported type. The - file must be an open file object such as ``sys.stdout`` or returned by - :func:`open` or :func:`os.popen`. It must be opened in binary mode (``'wb'`` + file must be a open file object such as ``sys.stdout`` or returned by + :func:`open` or :func:`os.popen`. It may not be a wrapper such as + TemporaryFile on Windows. It must be opened in binary mode (``'wb'`` or ``'w+b'``). If the value has (or contains an object that has) an unsupported type, a -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 18:19:15 2014 From: python-checkins at python.org (jesus.cea) Date: Tue, 29 Apr 2014 18:19:15 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Closes_issue_?= =?utf-8?q?=2321316=3A_mark_test=5Fdevpoll_to_be_meaningfull_only_for_Sola?= =?utf-8?q?ris?= Message-ID: <3gJ7N33LLWz7LkB@mail.python.org> http://hg.python.org/cpython/rev/cc2345e6e9ff changeset: 90518:cc2345e6e9ff branch: 3.4 parent: 90515:a172f26195f6 user: Jesus Cea date: Tue Apr 29 18:18:37 2014 +0200 summary: Closes issue #21316: mark test_devpoll to be meaningfull only for Solaris files: Lib/test/test_devpoll.py | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_devpoll.py b/Lib/test/test_devpoll.py --- a/Lib/test/test_devpoll.py +++ b/Lib/test/test_devpoll.py @@ -9,10 +9,8 @@ import unittest from test.support import TESTFN, run_unittest, cpython_only -try: - select.devpoll -except AttributeError: - raise unittest.SkipTest("select.devpoll not defined") +if not hasattr(select, 'devpoll') : + raise unittest.SkipTest('test works only on Solaris OS family') def find_ready_matching(ready, flag): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 18:19:16 2014 From: python-checkins at python.org (jesus.cea) Date: Tue, 29 Apr 2014 18:19:16 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_MERGE=3A_Closes_issue_=2321316=3A_mark_test=5Fdevpoll_to?= =?utf-8?q?_be_meaningfull_only_for_Solaris?= Message-ID: <3gJ7N451Npz7Ljw@mail.python.org> http://hg.python.org/cpython/rev/825c67196aac changeset: 90519:825c67196aac parent: 90516:2b2577d79e80 parent: 90518:cc2345e6e9ff user: Jesus Cea date: Tue Apr 29 18:18:58 2014 +0200 summary: MERGE: Closes issue #21316: mark test_devpoll to be meaningfull only for Solaris files: Lib/test/test_devpoll.py | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_devpoll.py b/Lib/test/test_devpoll.py --- a/Lib/test/test_devpoll.py +++ b/Lib/test/test_devpoll.py @@ -9,10 +9,8 @@ import unittest from test.support import TESTFN, run_unittest, cpython_only -try: - select.devpoll -except AttributeError: - raise unittest.SkipTest("select.devpoll not defined") +if not hasattr(select, 'devpoll') : + raise unittest.SkipTest('test works only on Solaris OS family') def find_ready_matching(ready, flag): -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 18:26:18 2014 From: python-checkins at python.org (stefan.krah) Date: Tue, 29 Apr 2014 18:26:18 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321374=3A_Fix_pick?= =?utf-8?q?ling_of_DecimalTuple=2E?= Message-ID: <3gJ7XB4f2Cz7LjT@mail.python.org> http://hg.python.org/cpython/rev/bf64a5f7c1af changeset: 90520:bf64a5f7c1af user: Stefan Krah date: Tue Apr 29 18:23:35 2014 +0200 summary: Issue #21374: Fix pickling of DecimalTuple. files: Lib/test/test_decimal.py | 17 +++++++++++++++++ Modules/_decimal/_decimal.c | 13 +++++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -2431,6 +2431,23 @@ self.assertIsInstance(r, C.Decimal) self.assertEqual(r, x) + x = C.Decimal('-3.123e81723').as_tuple() + y = P.Decimal('-3.123e81723').as_tuple() + + sys.modules['decimal'] = C + sx = pickle.dumps(x) + sys.modules['decimal'] = P + r = pickle.loads(sx) + self.assertIsInstance(r, P.DecimalTuple) + self.assertEqual(r, y) + + sys.modules['decimal'] = P + sy = pickle.dumps(y) + sys.modules['decimal'] = C + r = pickle.loads(sy) + self.assertIsInstance(r, C.DecimalTuple) + self.assertEqual(r, x) + sys.modules['decimal'] = savedecimal def test_int(self): diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -3542,7 +3542,7 @@ } } -static PyObject *DecimalTuple = NULL; +static PyTypeObject *DecimalTuple = NULL; /* Return the DecimalTuple representation of a PyDecObject. */ static PyObject * PyDec_AsTuple(PyObject *dec, PyObject *dummy UNUSED) @@ -3625,7 +3625,7 @@ } } - result = PyObject_CallFunctionObjArgs(DecimalTuple, + result = PyObject_CallFunctionObjArgs((PyObject *)DecimalTuple, sign, coeff, expt, NULL); out: @@ -5562,9 +5562,14 @@ /* DecimalTuple */ ASSIGN_PTR(collections, PyImport_ImportModule("collections")); - ASSIGN_PTR(DecimalTuple, PyObject_CallMethod(collections, + ASSIGN_PTR(DecimalTuple, (PyTypeObject *)PyObject_CallMethod(collections, "namedtuple", "(ss)", "DecimalTuple", "sign digits exponent")); + + ASSIGN_PTR(obj, PyUnicode_FromString("decimal")); + CHECK_INT(PyDict_SetItemString(DecimalTuple->tp_dict, "__module__", obj)); + Py_CLEAR(obj); + /* MutableMapping */ ASSIGN_PTR(MutableMapping, PyObject_GetAttrString(collections, "MutableMapping")); @@ -5591,7 +5596,7 @@ CHECK_INT(PyModule_AddObject(m, "Context", (PyObject *)&PyDecContext_Type)); Py_INCREF(DecimalTuple); - CHECK_INT(PyModule_AddObject(m, "DecimalTuple", DecimalTuple)); + CHECK_INT(PyModule_AddObject(m, "DecimalTuple", (PyObject *)DecimalTuple)); /* Create top level exception */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 18:26:19 2014 From: python-checkins at python.org (stefan.krah) Date: Tue, 29 Apr 2014 18:26:19 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzIxMzc0?= =?utf-8?q?=3A_Fix_pickling_of_DecimalTuple=2E?= Message-ID: <3gJ7XC6Rnsz7LkD@mail.python.org> http://hg.python.org/cpython/rev/25919f35241e changeset: 90521:25919f35241e branch: 3.4 parent: 90518:cc2345e6e9ff user: Stefan Krah date: Tue Apr 29 18:24:50 2014 +0200 summary: Issue #21374: Fix pickling of DecimalTuple. files: Lib/test/test_decimal.py | 17 +++++++++++++++++ Modules/_decimal/_decimal.c | 13 +++++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -2431,6 +2431,23 @@ self.assertIsInstance(r, C.Decimal) self.assertEqual(r, x) + x = C.Decimal('-3.123e81723').as_tuple() + y = P.Decimal('-3.123e81723').as_tuple() + + sys.modules['decimal'] = C + sx = pickle.dumps(x) + sys.modules['decimal'] = P + r = pickle.loads(sx) + self.assertIsInstance(r, P.DecimalTuple) + self.assertEqual(r, y) + + sys.modules['decimal'] = P + sy = pickle.dumps(y) + sys.modules['decimal'] = C + r = pickle.loads(sy) + self.assertIsInstance(r, C.DecimalTuple) + self.assertEqual(r, x) + sys.modules['decimal'] = savedecimal def test_int(self): diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -3542,7 +3542,7 @@ } } -static PyObject *DecimalTuple = NULL; +static PyTypeObject *DecimalTuple = NULL; /* Return the DecimalTuple representation of a PyDecObject. */ static PyObject * PyDec_AsTuple(PyObject *dec, PyObject *dummy UNUSED) @@ -3625,7 +3625,7 @@ } } - result = PyObject_CallFunctionObjArgs(DecimalTuple, + result = PyObject_CallFunctionObjArgs((PyObject *)DecimalTuple, sign, coeff, expt, NULL); out: @@ -5562,9 +5562,14 @@ /* DecimalTuple */ ASSIGN_PTR(collections, PyImport_ImportModule("collections")); - ASSIGN_PTR(DecimalTuple, PyObject_CallMethod(collections, + ASSIGN_PTR(DecimalTuple, (PyTypeObject *)PyObject_CallMethod(collections, "namedtuple", "(ss)", "DecimalTuple", "sign digits exponent")); + + ASSIGN_PTR(obj, PyUnicode_FromString("decimal")); + CHECK_INT(PyDict_SetItemString(DecimalTuple->tp_dict, "__module__", obj)); + Py_CLEAR(obj); + /* MutableMapping */ ASSIGN_PTR(MutableMapping, PyObject_GetAttrString(collections, "MutableMapping")); @@ -5591,7 +5596,7 @@ CHECK_INT(PyModule_AddObject(m, "Context", (PyObject *)&PyDecContext_Type)); Py_INCREF(DecimalTuple); - CHECK_INT(PyModule_AddObject(m, "DecimalTuple", DecimalTuple)); + CHECK_INT(PyModule_AddObject(m, "DecimalTuple", (PyObject *)DecimalTuple)); /* Create top level exception */ -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 18:26:21 2014 From: python-checkins at python.org (stefan.krah) Date: Tue, 29 Apr 2014 18:26:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Null_merge=2E?= Message-ID: <3gJ7XF0yxrz7Lkf@mail.python.org> http://hg.python.org/cpython/rev/684a6929e5ea changeset: 90522:684a6929e5ea parent: 90520:bf64a5f7c1af parent: 90521:25919f35241e user: Stefan Krah date: Tue Apr 29 18:25:14 2014 +0200 summary: Null merge. files: -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 20:34:25 2014 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 29 Apr 2014 20:34:25 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Document_the_s?= =?utf-8?q?ubprocess_Popen=2Eargs_attribute_=28issue21353=29?= Message-ID: <3gJBN14Gfgz7LjT@mail.python.org> http://hg.python.org/cpython/rev/0a4b211b927e changeset: 90523:0a4b211b927e branch: 3.3 parent: 90266:a8445ead2f9e user: Gregory P. Smith date: Tue Apr 29 11:33:23 2014 -0700 summary: Document the subprocess Popen.args attribute (issue21353) files: Doc/library/subprocess.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -702,6 +702,12 @@ deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. +.. attribute:: Popen.args + + The *args* argument as it was passed to :class:`Popen` -- a + sequence of program arguments or else a single string. + + .. versionadded:: 3.3 .. attribute:: Popen.stdin -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 20:34:26 2014 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 29 Apr 2014 20:34:26 +0200 (CEST) Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuNCk6?= =?utf-8?q?_Document_the_subprocess_Popen=2Eargs_attribute_=28issue21353?= =?utf-8?q?=29?= Message-ID: <3gJBN25pW8z7Ljb@mail.python.org> http://hg.python.org/cpython/rev/182b869283a5 changeset: 90524:182b869283a5 branch: 3.4 parent: 90521:25919f35241e parent: 90523:0a4b211b927e user: Gregory P. Smith date: Tue Apr 29 11:33:56 2014 -0700 summary: Document the subprocess Popen.args attribute (issue21353) files: Doc/library/subprocess.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -723,6 +723,12 @@ deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. +.. attribute:: Popen.args + + The *args* argument as it was passed to :class:`Popen` -- a + sequence of program arguments or else a single string. + + .. versionadded:: 3.3 .. attribute:: Popen.stdin -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Tue Apr 29 20:34:28 2014 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 29 Apr 2014 20:34:28 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E4_-=3E_default?= =?utf-8?q?=29=3A_Document_the_subprocess_Popen=2Eargs_attribute_=28issue2?= =?utf-8?q?1353=29?= Message-ID: <3gJBN40Gd0z7LkM@mail.python.org> http://hg.python.org/cpython/rev/0b2e199ad088 changeset: 90525:0b2e199ad088 parent: 90522:684a6929e5ea parent: 90524:182b869283a5 user: Gregory P. Smith date: Tue Apr 29 11:34:15 2014 -0700 summary: Document the subprocess Popen.args attribute (issue21353) files: Doc/library/subprocess.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -724,6 +724,12 @@ deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. +.. attribute:: Popen.args + + The *args* argument as it was passed to :class:`Popen` -- a + sequence of program arguments or else a single string. + + .. versionadded:: 3.3 .. attribute:: Popen.stdin -- Repository URL: http://hg.python.org/cpython From root at python.org Wed Apr 30 05:35:22 2014 From: root at python.org (Cron Daemon) Date: Wed, 30 Apr 2014 05:35:22 +0200 Subject: [Python-checkins] Cron /home/docs/build-devguide Message-ID: abort: error: Connection timed out From solipsis at pitrou.net Wed Apr 30 10:01:40 2014 From: solipsis at pitrou.net (solipsis at pitrou.net) Date: Wed, 30 Apr 2014 10:01:40 +0200 Subject: [Python-checkins] Daily reference leaks (0b2e199ad088): sum=3 Message-ID: results for 0b2e199ad088 on branch "default" -------------------------------------------- test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [-2, 0, 2] references, sum=0 test_site leaked [-2, 0, 2] memory blocks, sum=0 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogjnOhO7', '-x'] From python-checkins at python.org Wed Apr 30 15:58:37 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 30 Apr 2014 15:58:37 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?devguide=3A_add_link_to_compiling_doc?= =?utf-8?q?umentation_=28closes_=2321395=29?= Message-ID: <3gJhCK3Kjgz7Ljy@mail.python.org> http://hg.python.org/devguide/rev/678310833d85 changeset: 691:678310833d85 user: Benjamin Peterson date: Wed Apr 30 09:58:31 2014 -0400 summary: add link to compiling documentation (closes #21395) Patch by Brian Gesiak. files: runtests.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/runtests.rst b/runtests.rst --- a/runtests.rst +++ b/runtests.rst @@ -14,7 +14,8 @@ ------- The shortest, simplest way of running the test suite is the following command -from the root directory of your checkout (after you have built Python):: +from the root directory of your checkout (after you have +:ref:`built Python `):: ./python -m test -- Repository URL: http://hg.python.org/devguide From python-checkins at python.org Wed Apr 30 17:06:39 2014 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 30 Apr 2014 17:06:39 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_setup=2Epy=3A_report_modul?= =?utf-8?q?es_which_built_but_import_failed_=28closes_=2321282=29?= Message-ID: <3gJjjq5Vz3z7Lnc@mail.python.org> http://hg.python.org/cpython/rev/981242c8fc86 changeset: 90526:981242c8fc86 user: Benjamin Peterson date: Wed Apr 30 11:06:16 2014 -0400 summary: setup.py: report modules which built but import failed (closes #21282) Patch by Lukas Vacek. files: Misc/ACKS | 1 + setup.py | 16 +++++++++++++--- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1356,6 +1356,7 @@ Daniel Urban Michael Urman Hector Urtubia +Lukas Vacek Ville Vainio Andi Vajda Case Van Horsen diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -167,6 +167,7 @@ def __init__(self, dist): build_ext.__init__(self, dist) self.failed = [] + self.failed_on_import = [] def build_extensions(self): @@ -247,8 +248,9 @@ build_ext.build_extensions(self) longest = max([len(e.name) for e in self.extensions]) - if self.failed: - longest = max(longest, max([len(name) for name in self.failed])) + if self.failed or self.failed_on_import: + all_failed = self.failed + self.failed_on_import + longest = max(longest, max([len(name) for name in all_failed])) def print_three_column(lst): lst.sort(key=str.lower) @@ -276,6 +278,14 @@ print_three_column(failed) print() + if self.failed_on_import: + failed = self.failed_on_import[:] + print() + print("Following modules built successfully" + " but were removed because they could not be imported:") + print_three_column(failed) + print() + def build_extension(self, ext): if ext.name == '_ctypes': @@ -334,7 +344,7 @@ try: importlib._bootstrap._SpecMethods(spec).load() except ImportError as why: - self.failed.append(ext.name) + self.failed_on_import.append(ext.name) self.announce('*** WARNING: renaming "%s" since importing it' ' failed: %s' % (ext.name, why), level=3) assert not self.inplace -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 30 19:16:21 2014 From: python-checkins at python.org (stefan.krah) Date: Wed, 30 Apr 2014 19:16:21 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2310650=3A_Remove_t?= =?utf-8?q?he_non-standard_=27watchexp=27_parameter_from_the?= Message-ID: <3gJmbT3jnhz7LjP@mail.python.org> http://hg.python.org/cpython/rev/c2f827af02a2 changeset: 90527:c2f827af02a2 user: Stefan Krah date: Wed Apr 30 19:15:38 2014 +0200 summary: Issue #10650: Remove the non-standard 'watchexp' parameter from the Decimal.quantize() method in the Python version. It had never been present in the C version. files: Doc/library/decimal.rst | 12 +++--------- Lib/decimal.py | 12 +----------- Lib/test/test_decimal.py | 12 ------------ Misc/NEWS | 4 ++++ 4 files changed, 8 insertions(+), 32 deletions(-) diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -744,7 +744,7 @@ * ``"NaN"``, indicating that the operand is a quiet NaN (Not a Number). * ``"sNaN"``, indicating that the operand is a signaling NaN. - .. method:: quantize(exp, rounding=None, context=None, watchexp=True) + .. method:: quantize(exp, rounding=None, context=None) Return a value equal to the first operand after rounding and having the exponent of the second operand. @@ -767,14 +767,8 @@ ``context`` argument; if neither argument is given the rounding mode of the current thread's context is used. - If *watchexp* is set (default), then an error is returned whenever the - resulting exponent is greater than :attr:`Emax` or less than - :attr:`Etiny`. - - .. deprecated:: 3.3 - *watchexp* is an implementation detail from the pure Python version - and is not present in the C version. It will be removed in version - 3.4, where it defaults to ``True``. + An error is returned whenever the resulting exponent is greater than + :attr:`Emax` or less than :attr:`Etiny`. .. method:: radix() diff --git a/Lib/decimal.py b/Lib/decimal.py --- a/Lib/decimal.py +++ b/Lib/decimal.py @@ -2523,7 +2523,7 @@ end -= 1 return _dec_from_triple(dup._sign, dup._int[:end], exp) - def quantize(self, exp, rounding=None, context=None, watchexp=True): + def quantize(self, exp, rounding=None, context=None): """Quantize self so its exponent is the same as that of exp. Similar to self._rescale(exp._exp) but with error checking. @@ -2546,16 +2546,6 @@ return context._raise_error(InvalidOperation, 'quantize with one INF') - # if we're not watching exponents, do a simple rescale - if not watchexp: - ans = self._rescale(exp._exp, rounding) - # raise Inexact and Rounded where appropriate - if ans._exp > self._exp: - context._raise_error(Rounded) - if ans != self: - context._raise_error(Inexact) - return ans - # exp._exp should be between Etiny and Emax if not (context.Etiny() <= exp._exp <= context.Emax): return context._raise_error(InvalidOperation, diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -4448,18 +4448,6 @@ class PyFunctionality(unittest.TestCase): """Extra functionality in decimal.py""" - def test_py_quantize_watchexp(self): - # watchexp functionality - Decimal = P.Decimal - localcontext = P.localcontext - - with localcontext() as c: - c.prec = 1 - c.Emax = 1 - c.Emin = -1 - x = Decimal(99999).quantize(Decimal("1e3"), watchexp=False) - self.assertEqual(x, Decimal('1.00E+5')) - def test_py_alternate_formatting(self): # triples giving a format, a Decimal, and the expected result Decimal = P.Decimal diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,10 @@ Library ------- +- Issue #10650: Remove the non-standard 'watchexp' parameter from the + Decimal.quantize() method in the Python version. It had never been + present in the C version. + - Issue #21321: itertools.islice() now releases the reference to the source iterator when the slice is exhausted. Patch by Anton Afanasyev. -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 30 22:50:00 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 30 Apr 2014 22:50:00 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319962=3A_The_Wind?= =?utf-8?q?ows_build_process_now_creates_=22python=2Ebat=22?= Message-ID: <3gJsL04YJWz7Ljr@mail.python.org> http://hg.python.org/cpython/rev/de35f6a3b292 changeset: 90528:de35f6a3b292 user: Zachary Ware date: Wed Apr 30 15:47:53 2014 -0500 summary: Issue #19962: The Windows build process now creates "python.bat" in the root of the source tree, which passes all arguments through to the most recently built interpreter. files: .hgignore | 1 + Misc/NEWS | 4 + PCbuild/python.vcxproj | 162 ++++++++++++++++++++++++++++- 3 files changed, 166 insertions(+), 1 deletions(-) diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -18,6 +18,7 @@ platform$ pyconfig.h$ python$ +python.bat$ python.exe$ python-config$ python-config.py$ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -322,6 +322,10 @@ Build ----- +- Issue #19962: The Windows build process now creates "python.bat" in the + root of the source tree, which passes all arguments through to the most + recently built interpreter. + - Issue #21285: Refactor and fix curses configure check to always search in a ncursesw directory. diff --git a/PCbuild/python.vcxproj b/PCbuild/python.vcxproj --- a/PCbuild/python.vcxproj +++ b/PCbuild/python.vcxproj @@ -156,6 +156,14 @@ AllRules.ruleset + Link + Link + Link + Link + Link + Link + Link + Link @@ -176,6 +184,25 @@ 2000000 0x1d000000 + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -199,6 +226,25 @@ 2000000 0x1d000000 + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -221,6 +267,25 @@ 2000000 0x1d000000 + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -246,6 +311,25 @@ 4194304 0x1d000000 + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -268,6 +352,25 @@ + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -294,6 +397,25 @@ MachineX64 + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -316,6 +438,25 @@ + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -342,6 +483,25 @@ MachineX64 + + echo @rem This script invokes the most recently built Python with all arguments> "$(SolutionDir)..\python.bat" +echo @rem passed through to the interpreter. This file is generated by the>> "$(SolutionDir)..\python.bat" +echo @rem build process and any changes *will* be thrown away by the next>> "$(SolutionDir)..\python.bat" +echo @rem rebuild.>> "$(SolutionDir)..\python.bat" +echo @rem This is only meant as a convenience for developing CPython>> "$(SolutionDir)..\python.bat" +echo @rem and using it outside of that context is ill-advised.>> "$(SolutionDir)..\python.bat" +echo @echo Running $(Configuration)^^^|$(Platform) interpreter...>> "$(SolutionDir)..\python.bat" +echo @"$(OutDir)python$(PyDebugExt).exe" %%*>> "$(SolutionDir)..\python.bat" + + + Creating convenience batch file for easily invoking the newly built interpreter. + + + $(SolutionDir)..\python.bat + + + $(OutDir)python$(PyDebugExt).exe;%(Inputs) + @@ -361,4 +521,4 @@ - + \ No newline at end of file -- Repository URL: http://hg.python.org/cpython From python-checkins at python.org Wed Apr 30 23:30:12 2014 From: python-checkins at python.org (zach.ware) Date: Wed, 30 Apr 2014 23:30:12 +0200 (CEST) Subject: [Python-checkins] =?utf-8?q?cpython=3A_Unite_=27Build=27_sections?= =?utf-8?q?_of_Misc/NEWS?= Message-ID: <3gJtDN1K7hz7Ljr@mail.python.org> http://hg.python.org/cpython/rev/2a56d3896d50 changeset: 90529:2a56d3896d50 user: Zachary Ware date: Wed Apr 30 16:29:51 2014 -0500 summary: Unite 'Build' sections of Misc/NEWS files: Misc/NEWS | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -339,6 +339,11 @@ - Issue #20644: OS X installer build support for documentation build changes in 3.4.1: assume externally supplied sphinx-build is available in /usr/bin. +- Issue #20022: Eliminate use of deprecated bundlebuilder in OS X builds. + +- Issue #15968: Incorporated Tcl, Tk, and Tix builds into the Windows build + solution. + C API ----- - Issue #20942: PyImport_ImportFrozenModuleObject() no longer sets __file__ to @@ -409,13 +414,6 @@ - Issue #20535: PYTHONWARNING no longer affects the run_tests.py script. Patch by Arfrever Frehtes Taifersar Arahesis. -Build ------ - -- Issue #15968: Incorporated Tcl, Tk, and Tix builds into the Windows build - solution. - -- Issue #20022: Eliminate use of deprecated bundlebuilder in OS X builds. What's New in Python 3.4.0? =========================== -- Repository URL: http://hg.python.org/cpython