From python-checkins at python.org  Sat Jun  1 00:03:29 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:03:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_reference_to_module?=
	=?utf-8?q?=5Fto=5Fload?=
Message-ID: <3bMfmx0hMLz7LjN@mail.python.org>

http://hg.python.org/cpython/rev/ea3af5f5e6bc
changeset:   83996:ea3af5f5e6bc
parent:      83994:ebd11a19d830
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:00:56 2013 -0400
summary:
  Add a reference to module_to_load

files:
  Doc/library/importlib.rst |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -242,8 +242,7 @@
         from the import. If the loader inserted a module and the load fails, it
         must be removed by the loader from :data:`sys.modules`; modules already
         in :data:`sys.modules` before the loader began execution should be left
-        alone. The :func:`importlib.util.module_for_loader` decorator handles
-        all of these details.
+        alone (see :func:`importlib.util.module_to_load`).
 
         The loader should set several attributes on the module.
         (Note that some of these attributes can change when a module is

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 00:03:30 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:03:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_What=27s_New_for_im?=
 =?utf-8?q?portlib=2Eutil=2Emodule=5Fto=5Fload_name_change?=
Message-ID: <3bMfmy2Rb2z7M05@mail.python.org>

http://hg.python.org/cpython/rev/16598c43d3e8
changeset:   83997:16598c43d3e8
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:02:11 2013 -0400
summary:
  Update What's New for importlib.util.module_to_load name change

files:
  Doc/whatsnew/3.4.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -255,4 +255,4 @@
 * :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and
   ``__package__`` unconditionally to properly support reloading. If this is not
   desired then you will need to set these attributes manually. You can use
-  :class:`importlib.util.ModuleManager` for module management.
+  :func:`importlib.util.module_to_load` for module management.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 00:11:26 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:11:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_reset=5Fname_argumen?=
 =?utf-8?q?t_to_importlib=2Eutil=2Emodule=5Fto=5Fload_in_order_to?=
Message-ID: <3bMfy630Xkz7LpQ@mail.python.org>

http://hg.python.org/cpython/rev/39cc1b04713e
changeset:   83998:39cc1b04713e
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:11:17 2013 -0400
summary:
  Add a reset_name argument to importlib.util.module_to_load in order to
control whether to reset the module's __name__ attribute in case a
reload is being done.

files:
  Doc/library/importlib.rst            |   6 +++++-
  Lib/importlib/_bootstrap.py          |  14 +++++++++++++-
  Lib/test/test_importlib/test_util.py |  12 ++++++++++++
  3 files changed, 30 insertions(+), 2 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -788,7 +788,7 @@
 
    .. versionadded:: 3.3
 
-.. function:: module_to_load(name)
+.. function:: module_to_load(name, *, reset_name=True)
 
     Returns a :term:`context manager` which provides the module to load. The
     module will either come from :attr:`sys.modules` in the case of reloading or
@@ -796,6 +796,10 @@
     :attr:`sys.modules` occurs if the module was new and an exception was
     raised.
 
+    If **reset_name** is true and the module requested is being reloaded then
+    the module's :attr:`__name__` attribute will
+    be reset to **name**, else it will be left untouched.
+
     .. versionadded:: 3.4
 
 .. decorator:: module_for_loader
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -493,8 +493,14 @@
 
     """
 
-    def __init__(self, name):
+    def __init__(self, name, *, reset_name=True):
+        """Prepare the context manager.
+
+        The reset_name argument specifies whether to unconditionally reset
+        the __name__ attribute if the module is found to be a reload.
+        """
         self._name = name
+        self._reset_name = reset_name
 
     def __enter__(self):
         self._module = sys.modules.get(self._name)
@@ -508,6 +514,12 @@
             # (otherwise an optimization shortcut in import.c becomes wrong)
             self._module.__initializing__ = True
             sys.modules[self._name] = self._module
+        elif self._reset_name:
+            try:
+                self._module.__name__ = self._name
+            except AttributeError:
+                pass
+
         return self._module
 
     def __exit__(self, *args):
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -55,6 +55,18 @@
         else:
             self.fail('importlib.util.module_to_load swallowed an exception')
 
+    def test_reset_name(self):
+        # If reset_name is true then module.__name__ = name, else leave it be.
+        odd_name = 'not your typical name'
+        created_module = imp.new_module(self.module_name)
+        created_module.__name__ = odd_name
+        sys.modules[self.module_name] = created_module
+        with util.module_to_load(self.module_name) as module:
+            self.assertEqual(module.__name__, self.module_name)
+        created_module.__name__ = odd_name
+        with util.module_to_load(self.module_name, reset_name=False) as module:
+            self.assertEqual(module.__name__, odd_name)
+
 
 class ModuleForLoaderTests(unittest.TestCase):
 

-- 
Repository URL: http://hg.python.org/cpython

From brett at python.org  Sat Jun  1 00:37:13 2013
From: brett at python.org (Brett Cannon)
Date: Fri, 31 May 2013 18:37:13 -0400
Subject: [Python-checkins] cpython: Add a reset_name argument to
 importlib.util.module_to_load in order to
In-Reply-To: <3bMfy630Xkz7LpQ@mail.python.org>
References: <3bMfy630Xkz7LpQ@mail.python.org>
Message-ID: <CAP1=2W7saV5G3RNyW4pBQcGOqZFySF=_J2oLcxgUwZnMyG48gw@mail.gmail.com>

I realize this broke the buildbots. Missed part of a diff in the commit.
I'm trying to split a massive CL into reasonable commit sizes, so please be
patient.


On Fri, May 31, 2013 at 6:11 PM, brett.cannon <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/39cc1b04713e
> changeset:   83998:39cc1b04713e
> user:        Brett Cannon <brett at python.org>
> date:        Fri May 31 18:11:17 2013 -0400
> summary:
>   Add a reset_name argument to importlib.util.module_to_load in order to
> control whether to reset the module's __name__ attribute in case a
> reload is being done.
>
> files:
>   Doc/library/importlib.rst            |   6 +++++-
>   Lib/importlib/_bootstrap.py          |  14 +++++++++++++-
>   Lib/test/test_importlib/test_util.py |  12 ++++++++++++
>   3 files changed, 30 insertions(+), 2 deletions(-)
>
>
> diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
> --- a/Doc/library/importlib.rst
> +++ b/Doc/library/importlib.rst
> @@ -788,7 +788,7 @@
>
>     .. versionadded:: 3.3
>
> -.. function:: module_to_load(name)
> +.. function:: module_to_load(name, *, reset_name=True)
>
>      Returns a :term:`context manager` which provides the module to load.
> The
>      module will either come from :attr:`sys.modules` in the case of
> reloading or
> @@ -796,6 +796,10 @@
>      :attr:`sys.modules` occurs if the module was new and an exception was
>      raised.
>
> +    If **reset_name** is true and the module requested is being reloaded
> then
> +    the module's :attr:`__name__` attribute will
> +    be reset to **name**, else it will be left untouched.
> +
>      .. versionadded:: 3.4
>
>  .. decorator:: module_for_loader
> diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
> --- a/Lib/importlib/_bootstrap.py
> +++ b/Lib/importlib/_bootstrap.py
> @@ -493,8 +493,14 @@
>
>      """
>
> -    def __init__(self, name):
> +    def __init__(self, name, *, reset_name=True):
> +        """Prepare the context manager.
> +
> +        The reset_name argument specifies whether to unconditionally reset
> +        the __name__ attribute if the module is found to be a reload.
> +        """
>          self._name = name
> +        self._reset_name = reset_name
>
>      def __enter__(self):
>          self._module = sys.modules.get(self._name)
> @@ -508,6 +514,12 @@
>              # (otherwise an optimization shortcut in import.c becomes
> wrong)
>              self._module.__initializing__ = True
>              sys.modules[self._name] = self._module
> +        elif self._reset_name:
> +            try:
> +                self._module.__name__ = self._name
> +            except AttributeError:
> +                pass
> +
>          return self._module
>
>      def __exit__(self, *args):
> diff --git a/Lib/test/test_importlib/test_util.py
> b/Lib/test/test_importlib/test_util.py
> --- a/Lib/test/test_importlib/test_util.py
> +++ b/Lib/test/test_importlib/test_util.py
> @@ -55,6 +55,18 @@
>          else:
>              self.fail('importlib.util.module_to_load swallowed an
> exception')
>
> +    def test_reset_name(self):
> +        # If reset_name is true then module.__name__ = name, else leave
> it be.
> +        odd_name = 'not your typical name'
> +        created_module = imp.new_module(self.module_name)
> +        created_module.__name__ = odd_name
> +        sys.modules[self.module_name] = created_module
> +        with util.module_to_load(self.module_name) as module:
> +            self.assertEqual(module.__name__, self.module_name)
> +        created_module.__name__ = odd_name
> +        with util.module_to_load(self.module_name, reset_name=False) as
> module:
> +            self.assertEqual(module.__name__, odd_name)
> +
>
>  class ModuleForLoaderTests(unittest.TestCase):
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130531/c1bf41a0/attachment-0001.html>

From python-checkins at python.org  Sat Jun  1 00:57:56 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:57:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_for_last_commit_on_add?=
 =?utf-8?q?ing_reset=5Fname_to_module=5Fto=5Fload?=
Message-ID: <3bMgzm2W71z7Lnn@mail.python.org>

http://hg.python.org/cpython/rev/130dfd6b3428
changeset:   83999:130dfd6b3428
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:37:44 2013 -0400
summary:
  Fix for last commit on adding reset_name to module_to_load

files:
  Lib/importlib/_bootstrap.py |  9 ++++++---
  1 files changed, 6 insertions(+), 3 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -529,10 +529,13 @@
             del sys.modules[self._name]
 
 
-def module_to_load(name):
-    """Return a context manager which provides the module object to load."""
+def module_to_load(name, *, reset_name=True):
+    """Return a context manager which provides the module object to load.
+
+    If reset_name is true, reset the module's __name__ to 'name'.
+    """
     # Hiding _ModuleManager behind a function for better naming.
-    return _ModuleManager(name)
+    return _ModuleManager(name, reset_name=reset_name)
 
 
 def set_package(fxn):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 00:57:57 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:57:57 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Docstring_cleanup?=
Message-ID: <3bMgzn4Djmz7M17@mail.python.org>

http://hg.python.org/cpython/rev/42a3d182d906
changeset:   84000:42a3d182d906
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:39:07 2013 -0400
summary:
  Docstring cleanup

files:
  Lib/importlib/abc.py |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/importlib/abc.py b/Lib/importlib/abc.py
--- a/Lib/importlib/abc.py
+++ b/Lib/importlib/abc.py
@@ -71,7 +71,7 @@
         a possible part of a namespace.
         The fullname is a str.  Returns a 2-tuple of (Loader, portion) where
         portion is a sequence of file system locations contributing to part of
-        a namespace package.  The sequence may be empty and the loader may be
+        a namespace package. The sequence may be empty and the loader may be
         None.
         """
         return None, []
@@ -108,7 +108,8 @@
     def module_repr(self, module):
         """Return a module's repr.
 
-        Used by the module type when implemented without raising an exception.
+        Used by the module type when the method does not raise
+        NotImplementedError.
         """
         raise NotImplementedError
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 00:57:59 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:57:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issues_=2318088=2C_18089?=
	=?utf-8?q?=3A_Introduce?=
Message-ID: <3bMgzq2q03z7M11@mail.python.org>

http://hg.python.org/cpython/rev/e873f2e67353
changeset:   84001:e873f2e67353
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:56:47 2013 -0400
summary:
  Issues #18088, 18089: Introduce
importlib.abc.Loader.init_module_attrs() and implement
importlib.abc.InspectLoader.load_module().

The importlib.abc.Loader.init_module_attrs() method sets the various
attributes on the module being loaded. It is done unconditionally to
support reloading. Typically people used
importlib.util.module_for_loader, but since that's a decorator there
was no way to override it's actions, so init_module_attrs() came into
existence to allow for overriding. This is also why module_for_loader
is now pending deprecation (having its other use replaced by
importlib.util.module_to_load).

All of this allowed for importlib.abc.InspectLoader.load_module() to
be implemented. At this point you can now implement a loader with
nothing more than get_code() (which only requires get_source();
package support requires is_package()). Thanks to init_module_attrs()
the implementation of load_module() is basically a context manager
containing 2 methods calls, a call to exec(), and a return statement.

files:
  Doc/library/importlib.rst                          |    61 +-
  Doc/whatsnew/3.4.rst                               |     5 +-
  Lib/importlib/_bootstrap.py                        |   125 +-
  Lib/importlib/abc.py                               |    32 +-
  Lib/importlib/util.py                              |    45 +-
  Lib/test/test_importlib/source/test_file_loader.py |    33 +-
  Lib/test/test_importlib/test_abc.py                |   202 +
  Lib/test/test_importlib/test_util.py               |    24 +-
  Misc/NEWS                                          |     8 +-
  Python/importlib.h                                 |  7034 ++++-----
  10 files changed, 3933 insertions(+), 3636 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -246,7 +246,7 @@
 
         The loader should set several attributes on the module.
         (Note that some of these attributes can change when a module is
-        reloaded.)
+        reloaded; see :meth:`init_module_attrs`):
 
         - :attr:`__name__`
             The name of the module.
@@ -289,6 +289,17 @@
         .. versionchanged:: 3.4
            Made optional instead of an abstractmethod.
 
+    .. method:: init_module_attrs(module)
+
+        Set the :attr:`__loader__` attribute on the module.
+
+        Subclasses overriding this method should set whatever appropriate
+        attributes it can, getting the module's name from :attr:`__name__` when
+        needed. All values should also be overridden so that reloading works as
+        expected.
+
+        .. versionadded:: 3.4
+
 
 .. class:: ResourceLoader
 
@@ -363,6 +374,18 @@
 
         .. versionadded:: 3.4
 
+    .. method:: init_module_attrs(module)
+
+        Set the :attr:`__package__` attribute and :attr:`__path__` attribute to
+        the empty list if appropriate along with what
+        :meth:`importlib.abc.Loader.init_module_attrs` sets.
+
+        .. versionadded:: 3.4
+
+    .. method:: load_module(fullname)
+
+        Implementation of :meth:`Loader.load_module`.
+
 
 .. class:: ExecutionLoader
 
@@ -383,6 +406,15 @@
         .. versionchanged:: 3.4
            Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
 
+    .. method:: init_module_attrs(module)
+
+        Set :attr:`__file__` and if initializing a package then set
+        :attr:`__path__` to ``[os.path.dirname(__file__)]`` along with
+        all attributes set by
+        :meth:`importlib.abc.InspectLoader.init_module_attrs`.
+
+        .. versionadded:: 3.4
+
 
 .. class:: FileLoader(fullname, path)
 
@@ -500,6 +532,14 @@
         ``__init__`` when the file extension is removed **and** the module name
         itself does not end in ``__init__``.
 
+    .. method:: init_module_attr(module)
+
+        Set :attr:`__cached__` using :func:`imp.cache_from_source`. Other
+        attributes set by
+        :meth:`importlib.abc.ExecutionLoader.init_module_attrs`.
+
+        .. versionadded:: 3.4
+
 
 :mod:`importlib.machinery` -- Importers and path hooks
 ------------------------------------------------------
@@ -826,17 +866,18 @@
     module from being in left in :data:`sys.modules`. If the module was already
     in :data:`sys.modules` then it is left alone.
 
-    .. note::
-       :func:`module_to_load` subsumes the module management aspect of this
-       decorator.
-
     .. versionchanged:: 3.3
        :attr:`__loader__` and :attr:`__package__` are automatically set
        (when possible).
 
     .. versionchanged:: 3.4
-       Set :attr:`__loader__` :attr:`__package__` unconditionally to support
-       reloading.
+       Set :attr:`__name__`, :attr:`__loader__` :attr:`__package__`
+       unconditionally to support reloading.
+
+    .. deprecated:: 3.4
+        For the benefit of :term:`loader` subclasses, please use
+        :func:`module_to_load` and
+        :meth:`importlib.abc.Loader.init_module_attrs` instead.
 
 .. decorator:: set_loader
 
@@ -849,7 +890,8 @@
 
    .. note::
       As this decorator sets :attr:`__loader__` after loading the module, it is
-      recommended to use :func:`module_for_loader` instead when appropriate.
+      recommended to use :meth:`importlib.abc.Loader.init_module_attrs` instead
+      when appropriate.
 
    .. versionchanged:: 3.4
       Set ``__loader__`` if set to ``None``, as if the attribute does not
@@ -862,4 +904,5 @@
 
    .. note::
       As this decorator sets :attr:`__package__` after loading the module, it is
-      recommended to use :func:`module_for_loader` instead when appropriate.
+      recommended to use :meth:`importlib.abc.Loader.init_module_attrs` instead
+      when appropriate.
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -232,7 +232,10 @@
 Deprecated features
 -------------------
 
-* None yet.
+* :func:`importlib.util.module_for_loader` is pending deprecation. Using
+  :func:`importlib.util.module_to_load` and
+  :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
+  to more easily customize module loading.
 
 
 Porting to Python 3.4
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -538,6 +538,32 @@
     return _ModuleManager(name, reset_name=reset_name)
 
 
+def _init_package_attrs(loader, module):
+    """Set __package__ and __path__ based on what loader.is_package() says."""
+    name = module.__name__
+    try:
+        is_package = loader.is_package(name)
+    except ImportError:
+        pass
+    else:
+        if is_package:
+            module.__package__ = name
+            module.__path__ = []
+        else:
+            module.__package__ = name.rpartition('.')[0]
+
+
+def _init_file_attrs(loader, module):
+    """Set __file__ and __path__ based on loader.get_filename()."""
+    try:
+        module.__file__ = loader.get_filename(module.__name__)
+    except ImportError:
+        pass
+    else:
+        if module.__name__ == module.__package__:
+            module.__path__.append(_path_split(module.__file__)[0])
+
+
 def set_package(fxn):
     """Set __package__ on the returned module."""
     def set_package_wrapper(*args, **kwargs):
@@ -562,42 +588,6 @@
     return set_loader_wrapper
 
 
-def module_for_loader(fxn):
-    """Decorator to handle selecting the proper module for loaders.
-
-    The decorated function is passed the module to use instead of the module
-    name. The module passed in to the function is either from sys.modules if
-    it already exists or is a new module. If the module is new, then __name__
-    is set the first argument to the method, __loader__ is set to self, and
-    __package__ is set accordingly (if self.is_package() is defined) will be set
-    before it is passed to the decorated function (if self.is_package() does
-    not work for the module it will be set post-load).
-
-    If an exception is raised and the decorator created the module it is
-    subsequently removed from sys.modules.
-
-    The decorator assumes that the decorated function takes the module name as
-    the second argument.
-
-    """
-    def module_for_loader_wrapper(self, fullname, *args, **kwargs):
-        with module_to_load(fullname) as module:
-            module.__loader__ = self
-            try:
-                is_package = self.is_package(fullname)
-            except (ImportError, AttributeError):
-                pass
-            else:
-                if is_package:
-                    module.__package__ = fullname
-                else:
-                    module.__package__ = fullname.rpartition('.')[0]
-            # If __package__ was not set above, __import__() will do it later.
-            return fxn(self, module, *args, **kwargs)
-    _wrap(module_for_loader_wrapper, fxn)
-    return module_for_loader_wrapper
-
-
 def _check_name(method):
     """Decorator to verify that the module being requested matches the one the
     loader can handle.
@@ -904,25 +894,32 @@
         tail_name = fullname.rpartition('.')[2]
         return filename_base == '__init__' and tail_name != '__init__'
 
-    @module_for_loader
-    def _load_module(self, module, *, sourceless=False):
-        """Helper for load_module able to handle either source or sourceless
-        loading."""
-        name = module.__name__
-        code_object = self.get_code(name)
-        module.__file__ = self.get_filename(name)
-        if not sourceless:
+    def init_module_attrs(self, module):
+        """Set various attributes on the module.
+
+        ExecutionLoader.init_module_attrs() is used to set __loader__,
+        __package__, __file__, and optionally __path__. The __cached__ attribute
+        is set using imp.cache_from_source() and __file__.
+        """
+        module.__loader__ = self  # Loader
+        _init_package_attrs(self, module)  # InspectLoader
+        _init_file_attrs(self, module)  # ExecutionLoader
+        if hasattr(module, '__file__'):  # SourceLoader
             try:
                 module.__cached__ = cache_from_source(module.__file__)
             except NotImplementedError:
-                module.__cached__ = module.__file__
-        else:
-            module.__cached__ = module.__file__
-        if self.is_package(name):
-            module.__path__ = [_path_split(module.__file__)[0]]
-        # __package__ and __loader set by @module_for_loader.
-        _call_with_frames_removed(exec, code_object, module.__dict__)
-        return module
+                pass
+
+    def load_module(self, fullname):
+        """Load the specified module into sys.modules and return it."""
+        with module_to_load(fullname) as module:
+            self.init_module_attrs(module)
+            code = self.get_code(fullname)
+            if code is None:
+                raise ImportError('cannot load module {!r} when get_code() '
+                                  'returns None'.format(fullname))
+            _call_with_frames_removed(exec, code, module.__dict__)
+            return module
 
 
 class SourceLoader(_LoaderBasics):
@@ -1046,16 +1043,6 @@
                 pass
         return code_object
 
-    def load_module(self, fullname):
-        """Concrete implementation of Loader.load_module.
-
-        Requires ExecutionLoader.get_filename and ResourceLoader.get_data to be
-        implemented to load source code. Use of bytecode is dictated by whether
-        get_code uses/writes bytecode.
-
-        """
-        return self._load_module(fullname)
-
 
 class FileLoader:
 
@@ -1133,8 +1120,9 @@
 
     """Loader which handles sourceless file imports."""
 
-    def load_module(self, fullname):
-        return self._load_module(fullname, sourceless=True)
+    def init_module_attrs(self, module):
+        super().init_module_attrs(module)
+        module.__cached__ = module.__file__
 
     def get_code(self, fullname):
         path = self.get_filename(fullname)
@@ -1259,12 +1247,13 @@
     def module_repr(cls, module):
         return "<module '{}' (namespace)>".format(module.__name__)
 
-    @module_for_loader
-    def load_module(self, module):
+    def load_module(self, fullname):
         """Load a namespace module."""
         _verbose_message('namespace module loaded with path {!r}', self._path)
-        module.__path__ = self._path
-        return module
+        with module_to_load(fullname) as module:
+            module.__path__ = self._path
+            module.__package__ = fullname
+            return module
 
 
 # Finders #####################################################################
diff --git a/Lib/importlib/abc.py b/Lib/importlib/abc.py
--- a/Lib/importlib/abc.py
+++ b/Lib/importlib/abc.py
@@ -8,11 +8,6 @@
         raise
     _frozen_importlib = None
 import abc
-import imp
-import marshal
-import sys
-import tokenize
-import warnings
 
 
 def _register(abstract_cls, *classes):
@@ -113,6 +108,10 @@
         """
         raise NotImplementedError
 
+    def init_module_attrs(self, module):
+        """Set the module's __loader__ attribute."""
+        module.__loader__ = self
+
 
 class ResourceLoader(Loader):
 
@@ -177,6 +176,17 @@
         argument should be where the data was retrieved (when applicable)."""
         return compile(data, path, 'exec', dont_inherit=True)
 
+    def init_module_attrs(self, module):
+        """Initialize the __loader__ and __package__ attributes of the module.
+
+        The name of the module is gleaned from module.__name__. The __package__
+        attribute is set based on self.is_package().
+        """
+        super().init_module_attrs(module)
+        _bootstrap._init_package_attrs(self, module)
+
+    load_module = _bootstrap._LoaderBasics.load_module
+
 _register(InspectLoader, machinery.BuiltinImporter, machinery.FrozenImporter,
             machinery.ExtensionFileLoader)
 
@@ -215,6 +225,18 @@
         else:
             return self.source_to_code(source, path)
 
+    def init_module_attrs(self, module):
+        """Initialize the module's attributes.
+
+        It is assumed that the module's name has been set on module.__name__.
+        It is also assumed that any path returned by self.get_filename() uses
+        (one of) the operating system's path separator(s) to separate filenames
+        from directories in order to set __path__ intelligently.
+        InspectLoader.init_module_attrs() sets __loader__ and __package__.
+        """
+        super().init_module_attrs(module)
+        _bootstrap._init_file_attrs(self, module)
+
 
 class FileLoader(_bootstrap.FileLoader, ResourceLoader, ExecutionLoader):
 
diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py
--- a/Lib/importlib/util.py
+++ b/Lib/importlib/util.py
@@ -1,11 +1,13 @@
 """Utility code for constructing importers, etc."""
 
 from ._bootstrap import module_to_load
-from ._bootstrap import module_for_loader
 from ._bootstrap import set_loader
 from ._bootstrap import set_package
 from ._bootstrap import _resolve_name
 
+import functools
+import warnings
+
 
 def resolve_name(name, package):
     """Resolve a relative module name to an absolute one."""
@@ -20,3 +22,44 @@
             break
         level += 1
     return _resolve_name(name[level:], package, level)
+
+
+def module_for_loader(fxn):
+    """Decorator to handle selecting the proper module for loaders.
+
+    The decorated function is passed the module to use instead of the module
+    name. The module passed in to the function is either from sys.modules if
+    it already exists or is a new module. If the module is new, then __name__
+    is set the first argument to the method, __loader__ is set to self, and
+    __package__ is set accordingly (if self.is_package() is defined) will be set
+    before it is passed to the decorated function (if self.is_package() does
+    not work for the module it will be set post-load).
+
+    If an exception is raised and the decorator created the module it is
+    subsequently removed from sys.modules.
+
+    The decorator assumes that the decorated function takes the module name as
+    the second argument.
+
+    """
+    warnings.warn('To make it easier for subclasses, please use '
+                  'importlib.util.module_to_load() and '
+                  'importlib.abc.Loader.init_module_attrs()',
+                  PendingDeprecationWarning, stacklevel=2)
+    @functools.wraps(fxn)
+    def module_for_loader_wrapper(self, fullname, *args, **kwargs):
+        with module_to_load(fullname) as module:
+            module.__loader__ = self
+            try:
+                is_package = self.is_package(fullname)
+            except (ImportError, AttributeError):
+                pass
+            else:
+                if is_package:
+                    module.__package__ = fullname
+                else:
+                    module.__package__ = fullname.rpartition('.')[0]
+            # If __package__ was not set above, __import__() will do it later.
+            return fxn(self, module, *args, **kwargs)
+
+    return module_for_loader_wrapper
\ No newline at end of file
diff --git a/Lib/test/test_importlib/source/test_file_loader.py b/Lib/test/test_importlib/source/test_file_loader.py
--- a/Lib/test/test_importlib/source/test_file_loader.py
+++ b/Lib/test/test_importlib/source/test_file_loader.py
@@ -15,7 +15,7 @@
 import sys
 import unittest
 
-from test.support import make_legacy_pyc
+from test.support import make_legacy_pyc, unload
 
 
 class SimpleTest(unittest.TestCase):
@@ -26,23 +26,13 @@
     """
 
     def test_load_module_API(self):
-        # If fullname is not specified that assume self.name is desired.
-        class TesterMixin(importlib.abc.Loader):
-            def load_module(self, fullname): return fullname
-            def module_repr(self, module): return '<module>'
+        class Tester(importlib.abc.FileLoader):
+            def get_source(self, _): return 'attr = 42'
+            def is_package(self, _): return False
 
-        class Tester(importlib.abc.FileLoader, TesterMixin):
-            def get_code(self, _): pass
-            def get_source(self, _): pass
-            def is_package(self, _): pass
-
-        name = 'mod_name'
-        loader = Tester(name, 'some_path')
-        self.assertEqual(name, loader.load_module())
-        self.assertEqual(name, loader.load_module(None))
-        self.assertEqual(name, loader.load_module(name))
-        with self.assertRaises(ImportError):
-            loader.load_module(loader.name + 'XXX')
+        loader = Tester('blah', 'blah.py')
+        self.addCleanup(unload, 'blah')
+        module = loader.load_module()  # Should not raise an exception.
 
     def test_get_filename_API(self):
         # If fullname is not set then assume self.path is desired.
@@ -473,13 +463,6 @@
         self._test_non_code_marshal(del_source=True)
 
 
-def test_main():
-    from test.support import run_unittest
-    run_unittest(SimpleTest,
-                 SourceLoaderBadBytecodeTest,
-                 SourcelessLoaderBadBytecodeTest
-                )
-
 
 if __name__ == '__main__':
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_importlib/test_abc.py b/Lib/test/test_importlib/test_abc.py
--- a/Lib/test/test_importlib/test_abc.py
+++ b/Lib/test/test_importlib/test_abc.py
@@ -2,12 +2,14 @@
 from importlib import abc
 from importlib import machinery
 
+import contextlib
 import imp
 import inspect
 import io
 import marshal
 import os
 import sys
+from test import support
 import unittest
 from unittest import mock
 
@@ -198,6 +200,15 @@
         with self.assertRaises(ImportError):
             self.ins.get_filename('blah')
 
+##### Loader concrete methods ##################################################
+class LoaderConcreteMethodTests(unittest.TestCase):
+
+    def test_init_module_attrs(self):
+        loader = LoaderSubclass()
+        module = imp.new_module('blah')
+        loader.init_module_attrs(module)
+        self.assertEqual(module.__loader__, loader)
+
 
 ##### InspectLoader concrete methods ###########################################
 class InspectLoaderSourceToCodeTests(unittest.TestCase):
@@ -269,6 +280,93 @@
             loader.get_code('blah')
 
 
+class InspectLoaderInitModuleTests(unittest.TestCase):
+
+    @staticmethod
+    def mock_is_package(return_value):
+        return mock.patch.object(InspectLoaderSubclass, 'is_package',
+                                 return_value=return_value)
+
+    def init_module_attrs(self, name):
+        loader = InspectLoaderSubclass()
+        module = imp.new_module(name)
+        loader.init_module_attrs(module)
+        self.assertEqual(module.__loader__, loader)
+        return module
+
+    def test_package(self):
+        # If a package, then __package__ == __name__, __path__ == []
+        with self.mock_is_package(True):
+            name = 'blah'
+            module = self.init_module_attrs(name)
+            self.assertEqual(module.__package__, name)
+            self.assertEqual(module.__path__, [])
+
+    def test_toplevel(self):
+        # If a module is top-level, __package__ == ''
+        with self.mock_is_package(False):
+            name = 'blah'
+            module = self.init_module_attrs(name)
+            self.assertEqual(module.__package__, '')
+
+    def test_submodule(self):
+        # If a module is contained within a package then set __package__ to the
+        # package name.
+        with self.mock_is_package(False):
+            name = 'pkg.mod'
+            module = self.init_module_attrs(name)
+            self.assertEqual(module.__package__, 'pkg')
+
+    def test_is_package_ImportError(self):
+        # If is_package() raises ImportError, __package__ should be None and
+        # __path__ should not be set.
+        with self.mock_is_package(False) as mocked_method:
+            mocked_method.side_effect = ImportError
+            name = 'mod'
+            module = self.init_module_attrs(name)
+            self.assertIsNone(module.__package__)
+            self.assertFalse(hasattr(module, '__path__'))
+
+
+class InspectLoaderLoadModuleTests(unittest.TestCase):
+
+    """Test InspectLoader.load_module()."""
+
+    module_name = 'blah'
+
+    def setUp(self):
+        support.unload(self.module_name)
+        self.addCleanup(support.unload, self.module_name)
+
+    def mock_get_code(self):
+        return mock.patch.object(InspectLoaderSubclass, 'get_code')
+
+    def test_get_code_ImportError(self):
+        # If get_code() raises ImportError, it should propagate.
+        with self.mock_get_code() as mocked_get_code:
+            mocked_get_code.side_effect = ImportError
+            with self.assertRaises(ImportError):
+                loader = InspectLoaderSubclass()
+                loader.load_module(self.module_name)
+
+    def test_get_code_None(self):
+        # If get_code() returns None, raise ImportError.
+        with self.mock_get_code() as mocked_get_code:
+            mocked_get_code.return_value = None
+            with self.assertRaises(ImportError):
+                loader = InspectLoaderSubclass()
+                loader.load_module(self.module_name)
+
+    def test_module_returned(self):
+        # The loaded module should be returned.
+        code = compile('attr = 42', '<string>', 'exec')
+        with self.mock_get_code() as mocked_get_code:
+            mocked_get_code.return_value = code
+            loader = InspectLoaderSubclass()
+            module = loader.load_module(self.module_name)
+            self.assertEqual(module, sys.modules[self.module_name])
+
+
 ##### ExecutionLoader concrete methods #########################################
 class ExecutionLoaderGetCodeTests(unittest.TestCase):
 
@@ -327,6 +425,69 @@
         self.assertEqual(module.attr, 42)
 
 
+class ExecutionLoaderInitModuleTests(unittest.TestCase):
+
+    @staticmethod
+    @contextlib.contextmanager
+    def mock_methods(is_package, filename):
+        is_package_manager = InspectLoaderInitModuleTests.mock_is_package(is_package)
+        get_filename_manager = mock.patch.object(ExecutionLoaderSubclass,
+                'get_filename', return_value=filename)
+        with is_package_manager as mock_is_package:
+            with get_filename_manager as mock_get_filename:
+                yield {'is_package': mock_is_package,
+                       'get_filename': mock_get_filename}
+
+    def test_toplevel(self):
+        # Verify __loader__, __file__, and __package__; no __path__.
+        name = 'blah'
+        path = os.path.join('some', 'path', '{}.py'.format(name))
+        with self.mock_methods(False, path):
+            loader = ExecutionLoaderSubclass()
+            module = imp.new_module(name)
+            loader.init_module_attrs(module)
+        self.assertIs(module.__loader__, loader)
+        self.assertEqual(module.__file__, path)
+        self.assertEqual(module.__package__, '')
+        self.assertFalse(hasattr(module, '__path__'))
+
+    def test_package(self):
+        # Verify __loader__, __file__, __package__, and __path__.
+        name = 'pkg'
+        path = os.path.join('some', 'pkg', '__init__.py')
+        with self.mock_methods(True, path):
+            loader = ExecutionLoaderSubclass()
+            module = imp.new_module(name)
+            loader.init_module_attrs(module)
+        self.assertIs(module.__loader__, loader)
+        self.assertEqual(module.__file__, path)
+        self.assertEqual(module.__package__, 'pkg')
+        self.assertEqual(module.__path__, [os.path.dirname(path)])
+
+    def test_submodule(self):
+        # Verify __package__ and not __path__; test_toplevel() takes care of
+        # other attributes.
+        name = 'pkg.submodule'
+        path = os.path.join('some', 'pkg', 'submodule.py')
+        with self.mock_methods(False, path):
+            loader = ExecutionLoaderSubclass()
+            module = imp.new_module(name)
+            loader.init_module_attrs(module)
+        self.assertEqual(module.__package__, 'pkg')
+        self.assertEqual(module.__file__, path)
+        self.assertFalse(hasattr(module, '__path__'))
+
+    def test_get_filename_ImportError(self):
+        # If get_filename() raises ImportError, don't set __file__.
+        name = 'blah'
+        path = 'blah.py'
+        with self.mock_methods(False, path) as mocked_methods:
+            mocked_methods['get_filename'].side_effect = ImportError
+            loader = ExecutionLoaderSubclass()
+            module = imp.new_module(name)
+            loader.init_module_attrs(module)
+        self.assertFalse(hasattr(module, '__file__'))
+
 
 ##### SourceLoader concrete methods ############################################
 class SourceOnlyLoaderMock(abc.SourceLoader):
@@ -621,6 +782,47 @@
         self.assertEqual(mock.get_source(name), expect)
 
 
+class SourceLoaderInitModuleAttrTests(unittest.TestCase):
+
+    """Tests for importlib.abc.SourceLoader.init_module_attrs()."""
+
+    def test_init_module_attrs(self):
+        # If __file__ set, __cached__ == imp.cached_from_source(__file__).
+        name = 'blah'
+        path = 'blah.py'
+        loader = SourceOnlyLoaderMock(path)
+        module = imp.new_module(name)
+        loader.init_module_attrs(module)
+        self.assertEqual(module.__loader__, loader)
+        self.assertEqual(module.__package__, '')
+        self.assertEqual(module.__file__, path)
+        self.assertEqual(module.__cached__, imp.cache_from_source(path))
+
+    @mock.patch('importlib._bootstrap.cache_from_source')
+    def test_cache_from_source_NotImplementedError(self, mock_cache_from_source):
+        # If imp.cache_from_source() raises NotImplementedError don't set
+        # __cached__.
+        mock_cache_from_source.side_effect = NotImplementedError
+        name = 'blah'
+        path = 'blah.py'
+        loader = SourceOnlyLoaderMock(path)
+        module = imp.new_module(name)
+        loader.init_module_attrs(module)
+        self.assertEqual(module.__file__, path)
+        self.assertFalse(hasattr(module, '__cached__'))
+
+    def test_no_get_filename(self):
+        # No __file__, no __cached__.
+        with mock.patch.object(SourceOnlyLoaderMock, 'get_filename') as mocked:
+            mocked.side_effect = ImportError
+            name = 'blah'
+            loader = SourceOnlyLoaderMock('blah.py')
+            module = imp.new_module(name)
+            loader.init_module_attrs(module)
+        self.assertFalse(hasattr(module, '__file__'))
+        self.assertFalse(hasattr(module, '__cached__'))
+
+
 
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -5,6 +5,7 @@
 from test import support
 import types
 import unittest
+import warnings
 
 
 class ModuleToLoadTests(unittest.TestCase):
@@ -72,14 +73,27 @@
 
     """Tests for importlib.util.module_for_loader."""
 
+    @staticmethod
+    def module_for_loader(func):
+        with warnings.catch_warnings():
+            warnings.simplefilter('ignore', PendingDeprecationWarning)
+            return util.module_for_loader(func)
+
+    def test_warning(self):
+        # Should raise a PendingDeprecationWarning when used.
+        with warnings.catch_warnings():
+            warnings.simplefilter('error', PendingDeprecationWarning)
+            with self.assertRaises(PendingDeprecationWarning):
+                func = util.module_for_loader(lambda x: x)
+
     def return_module(self, name):
-        fxn = util.module_for_loader(lambda self, module: module)
+        fxn = self.module_for_loader(lambda self, module: module)
         return fxn(self, name)
 
     def raise_exception(self, name):
         def to_wrap(self, module):
             raise ImportError
-        fxn = util.module_for_loader(to_wrap)
+        fxn = self.module_for_loader(to_wrap)
         try:
             fxn(self, name)
         except ImportError:
@@ -100,7 +114,7 @@
         class FakeLoader:
             def is_package(self, name):
                 return True
-            @util.module_for_loader
+            @self.module_for_loader
             def load_module(self, module):
                 return module
         name = 'a.b.c'
@@ -134,7 +148,7 @@
 
     def test_decorator_attrs(self):
         def fxn(self, module): pass
-        wrapped = util.module_for_loader(fxn)
+        wrapped = self.module_for_loader(fxn)
         self.assertEqual(wrapped.__name__, fxn.__name__)
         self.assertEqual(wrapped.__qualname__, fxn.__qualname__)
 
@@ -160,7 +174,7 @@
                 self._pkg = is_package
             def is_package(self, name):
                 return self._pkg
-            @util.module_for_loader
+            @self.module_for_loader
             def load_module(self, module):
                 return module
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -97,6 +97,12 @@
 Library
 -------
 
+- Issue #18089: Implement importlib.abc.InspectLoader.load_module.
+
+- Issue #18088: Introduce importlib.abc.Loader.init_module_attrs for setting
+  module attributes. Leads to the pending deprecation of
+  importlib.util.module_for_loader.
+
 - Issue #17403: urllib.parse.robotparser normalizes the urls before adding to
   ruleline. This helps in handling certain types invalid urls in a conservative
   manner. Patch contributed by Mher Movsisyan.
@@ -104,7 +110,7 @@
 - Issue #18070: Have importlib.util.module_for_loader() set attributes
   unconditionally in order to properly support reloading.
 
-- Add importlib.util.module_to_load to return a context manager to provide the
+- Added importlib.util.module_to_load to return a context manager to provide the
   proper module object to load.
 
 - Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 00:58:00 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 00:58:00 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_whitespace?=
Message-ID: <3bMgzr4Ww2z7M11@mail.python.org>

http://hg.python.org/cpython/rev/6a90817fb437
changeset:   84002:6a90817fb437
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 18:57:45 2013 -0400
summary:
  fix whitespace

files:
  Lib/importlib/util.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py
--- a/Lib/importlib/util.py
+++ b/Lib/importlib/util.py
@@ -62,4 +62,4 @@
             # If __package__ was not set above, __import__() will do it later.
             return fxn(self, module, *args, **kwargs)
 
-    return module_for_loader_wrapper
\ No newline at end of file
+    return module_for_loader_wrapper

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 05:18:49 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat,  1 Jun 2013 05:18:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318065=3A_For_froz?=
 =?utf-8?b?ZW4gcGFja2FnZXMgc2V0IF9fcGF0aF9fIHRvIFtdLg==?=
Message-ID: <3bMnmn3r0dz7LlV@mail.python.org>

http://hg.python.org/cpython/rev/82db02a2e023
changeset:   84003:82db02a2e023
user:        Brett Cannon <brett at python.org>
date:        Fri May 31 23:18:39 2013 -0400
summary:
  Issue #18065: For frozen packages set __path__ to [].

Previously __path__ was set to [__name__], but that could lead to bad
results if someone managed to circumvent the frozen importer and
somehow ended up with a finder that thought __name__ was a legit
directory/location.

files:
  Doc/whatsnew/3.4.rst                          |  7 +++++++
  Lib/test/test_importlib/frozen/test_loader.py |  2 +-
  Misc/NEWS                                     |  6 ++++++
  Python/import.c                               |  6 ++----
  4 files changed, 16 insertions(+), 5 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -259,3 +259,10 @@
   ``__package__`` unconditionally to properly support reloading. If this is not
   desired then you will need to set these attributes manually. You can use
   :func:`importlib.util.module_to_load` for module management.
+
+* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``,
+  ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading.
+
+* Frozen packages no longer set ``__path__`` to a list containg the package name
+  but an empty list instead. Determing if a module is a package should be done
+  using ``hasattr(module, '__path__')``.
diff --git a/Lib/test/test_importlib/frozen/test_loader.py b/Lib/test/test_importlib/frozen/test_loader.py
--- a/Lib/test/test_importlib/frozen/test_loader.py
+++ b/Lib/test/test_importlib/frozen/test_loader.py
@@ -24,7 +24,7 @@
             module = machinery.FrozenImporter.load_module('__phello__')
             check = {'__name__': '__phello__',
                      '__package__': '__phello__',
-                     '__path__': ['__phello__'],
+                     '__path__': [],
                      '__loader__': machinery.FrozenImporter,
                      }
             for attr, value in check.items():
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,12 @@
 Core and Builtins
 -----------------
 
+- Issue #18065: Don't set __path__ to the package name for frozen packages.
+
+- Issue #18088: When reloading a module, unconditionally reset all relevant
+  attributes on the module (e.g. __name__, __loader__, __package__, __file__,
+  __cached__).
+
 - Issue #17937: Try harder to collect cyclic garbage at shutdown.
 
 - Issue #12370: Prevent class bodies from interfering with the __class__
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -1107,19 +1107,17 @@
         goto err_return;
     }
     if (ispackage) {
-        /* Set __path__ to the package name */
+        /* Set __path__ to the empty list */
         PyObject *d, *l;
         int err;
         m = PyImport_AddModuleObject(name);
         if (m == NULL)
             goto err_return;
         d = PyModule_GetDict(m);
-        l = PyList_New(1);
+        l = PyList_New(0);
         if (l == NULL) {
             goto err_return;
         }
-        Py_INCREF(name);
-        PyList_SET_ITEM(l, 0, name);
         err = PyDict_SetItemString(d, "__path__", l);
         Py_DECREF(l);
         if (err != 0)

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Jun  1 05:51:08 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 01 Jun 2013 05:51:08 +0200
Subject: [Python-checkins] Daily reference leaks (6a90817fb437): sum=0
Message-ID: <E1UicqS-0004Qr-PG@ap.vmr.nerim.net>

results for 6a90817fb437 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloge5rj8r', '-x']

From python-checkins at python.org  Sat Jun  1 07:18:37 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Sat,  1 Jun 2013 07:18:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_frozen_modules_now_apparen?=
 =?utf-8?q?tly_have_empty_=5F=5Fpath=5F=5F?=
Message-ID: <3bMrR11Z6Vz7Ll2@mail.python.org>

http://hg.python.org/cpython/rev/35da9e3ba697
changeset:   84004:35da9e3ba697
user:        Benjamin Peterson <benjamin at python.org>
date:        Fri May 31 22:18:26 2013 -0700
summary:
  frozen modules now apparently have empty __path__

files:
  Lib/test/test_frozen.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_frozen.py b/Lib/test/test_frozen.py
--- a/Lib/test/test_frozen.py
+++ b/Lib/test/test_frozen.py
@@ -36,7 +36,7 @@
             else:
                 expect.add('spam')
                 self.assertEqual(set(dir(__phello__)), expect)
-            self.assertEqual(__phello__.__path__, [__phello__.__name__])
+            self.assertEqual(__phello__.__path__, [])
             self.assertEqual(stdout.getvalue(), 'Hello world!\n')
 
         with captured_stdout() as stdout:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 16:59:25 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 16:59:25 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4ICMxNjQ1MCB0?=
 =?utf-8?q?est=5Fmissing=5Flocalfile_testcase_fails_on_misconfigured_hostn?=
 =?utf-8?q?ame=2E?=
Message-ID: <3bN5K9251PzShL@mail.python.org>

http://hg.python.org/cpython/rev/60c195e89c88
changeset:   84005:60c195e89c88
branch:      2.7
parent:      83995:6ceb5bf24da8
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 07:59:10 2013 -0700
summary:
  Fix #16450 test_missing_localfile testcase fails on misconfigured hostname.
Refactor test to accomodate that and exercise the needed functionality.

files:
  Lib/test/test_urllib.py |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py
--- a/Lib/test/test_urllib.py
+++ b/Lib/test/test_urllib.py
@@ -227,13 +227,13 @@
                 'file://localhost/a/missing/file.py')
         fd, tmp_file = tempfile.mkstemp()
         tmp_fileurl = 'file://localhost/' + tmp_file.replace(os.path.sep, '/')
+        self.assertTrue(os.path.exists(tmp_file))
         try:
-            self.assertTrue(os.path.exists(tmp_file))
             fp = urllib.urlopen(tmp_fileurl)
+            fp.close()
         finally:
             os.close(fd)
-            fp.close()
-        os.unlink(tmp_file)
+            os.unlink(tmp_file)
 
         self.assertFalse(os.path.exists(tmp_file))
         self.assertRaises(IOError, urllib.urlopen, tmp_fileurl)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 17:28:00 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 17:28:00 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4ICMxNzk2Nzog?=
 =?utf-8?q?For_ftp_urls_CWD_to_target_instead_of_hopping_to_each_directory?=
Message-ID: <3bN5y81h4XzT0K@mail.python.org>

http://hg.python.org/cpython/rev/0a544bb539e6
changeset:   84006:0a544bb539e6
branch:      2.7
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 08:24:31 2013 -0700
summary:
  Fix #17967: For ftp urls CWD to target instead of hopping to each directory
towards target. This fixes a bug where target is accessible, but parent
directories are restricted.

files:
  Lib/urllib.py |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Lib/urllib.py b/Lib/urllib.py
--- a/Lib/urllib.py
+++ b/Lib/urllib.py
@@ -870,8 +870,7 @@
         self.ftp = ftplib.FTP()
         self.ftp.connect(self.host, self.port, self.timeout)
         self.ftp.login(self.user, self.passwd)
-        for dir in self.dirs:
-            self.ftp.cwd(dir)
+        self.ftp.cwd(os.path.join(*self.dirs))
 
     def retrfile(self, file, type):
         import ftplib

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 17:28:01 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 17:28:01 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogRml4ICMxNzk2Nzog?=
 =?utf-8?q?For_ftp_urls_CWD_to_target_instead_of_hopping_to_each_directory?=
Message-ID: <3bN5y93ZPmz7Ljd@mail.python.org>

http://hg.python.org/cpython/rev/dbfbdf2b5c19
changeset:   84007:dbfbdf2b5c19
branch:      3.3
parent:      83993:81c02d2c830d
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 08:27:06 2013 -0700
summary:
  Fix #17967: For ftp urls CWD to target instead of hopping to each directory
towards target. This fixes a bug where target is accessible, but parent
directories are restricted.

files:
  Lib/urllib/request.py |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -2273,8 +2273,7 @@
         self.ftp = ftplib.FTP()
         self.ftp.connect(self.host, self.port, self.timeout)
         self.ftp.login(self.user, self.passwd)
-        for dir in self.dirs:
-            self.ftp.cwd(dir)
+        self.ftp.cwd(os.path.join(*self.dirs))
 
     def retrfile(self, file, type):
         import ftplib

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 17:28:02 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 17:28:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_merge_from_3=2E3?=
Message-ID: <3bN5yB5Q6Dz7Ljq@mail.python.org>

http://hg.python.org/cpython/rev/c1101f0d6c29
changeset:   84008:c1101f0d6c29
parent:      84004:35da9e3ba697
parent:      84007:dbfbdf2b5c19
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 08:27:53 2013 -0700
summary:
  merge from 3.3

Fix #17967: For ftp urls CWD to target instead of hopping to each directory
towards target. This fixes a bug where target is accessible, but parent
directories are restricted.

files:
  Lib/urllib/request.py |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -2303,8 +2303,7 @@
         self.ftp = ftplib.FTP()
         self.ftp.connect(self.host, self.port, self.timeout)
         self.ftp.login(self.user, self.passwd)
-        for dir in self.dirs:
-            self.ftp.cwd(dir)
+        self.ftp.cwd(os.path.join(*self.dirs))
 
     def retrfile(self, file, type):
         import ftplib

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 19:53:47 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat,  1 Jun 2013 19:53:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2318066=3A_remove_vestigi?=
 =?utf-8?q?al_code_depending_on_the_sgi_module?=
Message-ID: <3bN9BM0cTdzQ8b@mail.python.org>

http://hg.python.org/cpython/rev/a678f139510b
changeset:   84009:a678f139510b
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 01 13:52:30 2013 -0400
summary:
  #18066: remove vestigial code depending on the sgi module

files:
  Lib/pty.py |  13 +------------
  1 files changed, 1 insertions(+), 12 deletions(-)


diff --git a/Lib/pty.py b/Lib/pty.py
--- a/Lib/pty.py
+++ b/Lib/pty.py
@@ -47,18 +47,7 @@
     return _open_terminal()
 
 def _open_terminal():
-    """Open pty master and return (master_fd, tty_name).
-    SGI and generic BSD version, for when openpty() fails."""
-    try:
-        import sgi
-    except ImportError:
-        pass
-    else:
-        try:
-            tty_name, master_fd = sgi._getpty(os.O_RDWR, 0o666, 0)
-        except OSError as msg:
-            raise OSError(msg)
-        return master_fd, tty_name
+    """Open pty master and return (master_fd, tty_name)."""
     for x in 'pqrstuvwxyzPQRST':
         for y in '0123456789abcdef':
             pty_name = '/dev/pty' + x + y

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 20:12:58 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 20:12:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_thishost_h?=
 =?utf-8?q?elper_funtion_in_urllib=2E_Returns_the_ipaddress_of_localhost_w?=
 =?utf-8?q?hen?=
Message-ID: <3bN9cV3Z2kz7LjQ@mail.python.org>

http://hg.python.org/cpython/rev/4657d0eebe42
changeset:   84010:4657d0eebe42
branch:      2.7
parent:      84006:0a544bb539e6
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 11:11:30 2013 -0700
summary:
  Fix thishost helper funtion in urllib. Returns the ipaddress of localhost when
hostname is resolvable by socket.gethostname for local machine. This all fixes
certain freebsd builtbot failures.

files:
  Lib/urllib.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/urllib.py b/Lib/urllib.py
--- a/Lib/urllib.py
+++ b/Lib/urllib.py
@@ -819,7 +819,10 @@
     """Return the IP address of the current host."""
     global _thishost
     if _thishost is None:
-        _thishost = socket.gethostbyname(socket.gethostname())
+        try:
+            _thishost = socket.gethostbyname(socket.gethostname())
+        except socket.gaierror:
+            _thishost = socket.gethostbyname('localhost')
     return _thishost
 
 _ftperrors = None

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 20:12:59 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 20:12:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fix_thishost_h?=
 =?utf-8?q?elper_funtion_in_urllib=2E_Returns_the_ipaddress_of_localhost_w?=
 =?utf-8?q?hen?=
Message-ID: <3bN9cW5Svvz7LjQ@mail.python.org>

http://hg.python.org/cpython/rev/b6464827bddb
changeset:   84011:b6464827bddb
branch:      3.3
parent:      84007:dbfbdf2b5c19
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 11:12:17 2013 -0700
summary:
  Fix thishost helper funtion in urllib. Returns the ipaddress of localhost when
hostname is resolvable by socket.gethostname for local machine. This all fixes
certain freebsd builtbot failures.

files:
  Lib/urllib/request.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -2229,7 +2229,10 @@
     """Return the IP addresses of the current host."""
     global _thishost
     if _thishost is None:
-        _thishost = tuple(socket.gethostbyname_ex(socket.gethostname())[2])
+        try:
+            _thishost = tuple(socket.gethostbyname_ex(socket.gethostname())[2])
+        except socket.gaierror:
+            _thishost = tuple(socket.gethostbyname_ex('localhost')[2])
     return _thishost
 
 _ftperrors = None

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 20:13:01 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sat,  1 Jun 2013 20:13:01 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_merge_from_3=2E3?=
Message-ID: <3bN9cY0LhWz7LjY@mail.python.org>

http://hg.python.org/cpython/rev/25450fff5b90
changeset:   84012:25450fff5b90
parent:      84009:a678f139510b
parent:      84011:b6464827bddb
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sat Jun 01 11:12:52 2013 -0700
summary:
  merge from 3.3

Fix thishost helper funtion in urllib. Returns the ipaddress of localhost when
hostname is resolvable by socket.gethostname for local machine. This all fixes
certain freebsd builtbot failures.

files:
  Lib/urllib/request.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -2259,7 +2259,10 @@
     """Return the IP addresses of the current host."""
     global _thishost
     if _thishost is None:
-        _thishost = tuple(socket.gethostbyname_ex(socket.gethostname())[2])
+        try:
+            _thishost = tuple(socket.gethostbyname_ex(socket.gethostname())[2])
+        except socket.gaierror:
+            _thishost = tuple(socket.gethostbyname_ex('localhost')[2])
     return _thishost
 
 _ftperrors = None

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  1 21:21:39 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Sat,  1 Jun 2013 21:21:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?devguide=3A_Don=27t_ask_contributors_?=
 =?utf-8?q?to_add_themselves_to_Misc/ACKS=2C_it_is_the_core?=
Message-ID: <3bNC7l0Dwpz7Lk5@mail.python.org>

http://hg.python.org/devguide/rev/51cbc121628e
changeset:   621:51cbc121628e
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Jun 01 21:21:31 2013 +0200
summary:
  Don't ask contributors to add themselves to Misc/ACKS, it is the core developer's job.
(as discussed on core-mentorship, with Nick's and Terry's approval)

files:
  patch.rst |  38 +++++++++++++++++++++++++++-----------
  1 files changed, 27 insertions(+), 11 deletions(-)


diff --git a/patch.rst b/patch.rst
--- a/patch.rst
+++ b/patch.rst
@@ -74,17 +74,6 @@
 Fifth, proper :ref:`documentation <documenting>`
 additions/changes should be included.
 
-Sixth, if you are not already in the ``Misc/ACKS`` file then add your name. If
-you have taken the time to diagnose a problem, invent a solution, code it up,
-and submit a patch you deserve to be recognized as having contributed to
-Python. This also means you need to fill out a `contributor form`_ which
-allows the `Python Software Foundation`_ to license your code for use with
-Python (you retain the copyright).
-
-
-.. _contributor form: http://www.python.org/psf/contrib/
-.. _Python Software Foundation: http://www.python.org/psf/
-
 
 .. _patch-generation:
 
@@ -113,6 +102,24 @@
 **single, condensed** patch rather than a series of several changesets.
 
 
+Licensing
+---------
+
+For non-trivial changes, we must have your formal approval for distributing
+your work under the `PSF license`_.  Therefore, you need to fill out a
+`contributor form`_ which allows the `Python Software Foundation`_ to
+license your code for use with Python (you retain the copyright).
+
+.. note::
+   You only have to sign this document once, it will then apply to all
+   your further contributions to Python.
+
+
+.. _PSF license: http://docs.python.org/3.4/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python
+.. _contributor form: http://www.python.org/psf/contrib/
+.. _Python Software Foundation: http://www.python.org/psf/
+
+
 Submitting
 ----------
 
@@ -166,3 +173,12 @@
 with the next major release of Python. It may also be backported to older
 versions of Python as a bugfix if the core developer doing the commit believes
 it is warranted.
+
+
+Crediting
+---------
+
+Non-trivial contributions are credited in the ``Misc/ACKS`` file (and, most
+often, in a contribution's ``Misc/NEWS`` entry as well).  This is something
+the core developer will do when committing your patch, you don't have to
+propose the addition by yourself.

-- 
Repository URL: http://hg.python.org/devguide

From solipsis at pitrou.net  Sun Jun  2 05:49:50 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 02 Jun 2013 05:49:50 +0200
Subject: [Python-checkins] Daily reference leaks (25450fff5b90): sum=4
Message-ID: <E1UizIk-0000Bc-BT@ap.vmr.nerim.net>

results for 25450fff5b90 on branch "default"
--------------------------------------------

test_support leaked [-1, 1, 0] references, sum=0
test_support leaked [-1, 3, 2] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloghWGbq0', '-x']

From python-checkins at python.org  Sun Jun  2 19:05:16 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Sun,  2 Jun 2013 19:05:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Clarify_which_?=
 =?utf-8?q?dictionaries_are_updateable?=
Message-ID: <3bNm3w3sSLz7LmK@mail.python.org>

http://hg.python.org/cpython/rev/4c8426acd2cf
changeset:   84013:4c8426acd2cf
branch:      3.3
parent:      84011:b6464827bddb
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Jun 02 10:03:05 2013 -0700
summary:
  Clarify which dictionaries are updateable
by using the wording from the Py2.7 docs.

files:
  Doc/library/functions.rst |  20 ++++++++++----------
  1 files changed, 10 insertions(+), 10 deletions(-)


diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -1355,14 +1355,18 @@
 
 .. function:: vars([object])
 
-   Without an argument, act like :func:`locals`.
+   Return the :attr:`__dict__` attribute for a module, class, instance,
+   or any other object with a :attr:`__dict__` attribute.
 
-   With a module, class or class instance object as argument (or anything else that
-   has a :attr:`__dict__` attribute), return that attribute.
+   Objects such as modules and instances have an updateable :attr:`__dict__`
+   attribute; however, other objects may have write restrictions on their
+   :attr:`__dict__` attributes (for example, classes use a
+   dictproxy to prevent direct dictionary updates).
 
-   .. note::
-      The returned dictionary should not be modified:
-      the effects on the corresponding symbol table are undefined. [#]_
+   Without an argument, :func:`vars` acts like :func:`locals`.  Note, the
+   locals dictionary is only useful for reads since updates to the locals
+   dictionary are ignored.
+
 
 .. function:: zip(*iterables)
 
@@ -1481,7 +1485,3 @@
 .. [#] Note that the parser only accepts the Unix-style end of line convention.
    If you are reading the code from a file, make sure to use newline conversion
    mode to convert Windows or Mac-style newlines.
-
-.. [#] In the current implementation, local variable bindings cannot normally be
-   affected this way, but variables retrieved from other scopes (such as modules)
-   can be.  This may change.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  2 19:05:17 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Sun,  2 Jun 2013 19:05:17 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bNm3x5jLvz7Llm@mail.python.org>

http://hg.python.org/cpython/rev/49bcd1af0f49
changeset:   84014:49bcd1af0f49
parent:      84012:25450fff5b90
parent:      84013:4c8426acd2cf
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Jun 02 10:04:59 2013 -0700
summary:
  merge

files:
  Doc/library/functions.rst |  20 ++++++++++----------
  1 files changed, 10 insertions(+), 10 deletions(-)


diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -1365,14 +1365,18 @@
 
 .. function:: vars([object])
 
-   Without an argument, act like :func:`locals`.
+   Return the :attr:`__dict__` attribute for a module, class, instance,
+   or any other object with a :attr:`__dict__` attribute.
 
-   With a module, class or class instance object as argument (or anything else that
-   has a :attr:`__dict__` attribute), return that attribute.
+   Objects such as modules and instances have an updateable :attr:`__dict__`
+   attribute; however, other objects may have write restrictions on their
+   :attr:`__dict__` attributes (for example, classes use a
+   dictproxy to prevent direct dictionary updates).
 
-   .. note::
-      The returned dictionary should not be modified:
-      the effects on the corresponding symbol table are undefined. [#]_
+   Without an argument, :func:`vars` acts like :func:`locals`.  Note, the
+   locals dictionary is only useful for reads since updates to the locals
+   dictionary are ignored.
+
 
 .. function:: zip(*iterables)
 
@@ -1491,7 +1495,3 @@
 .. [#] Note that the parser only accepts the Unix-style end of line convention.
    If you are reading the code from a file, make sure to use newline conversion
    mode to convert Windows or Mac-style newlines.
-
-.. [#] In the current implementation, local variable bindings cannot normally be
-   affected this way, but variables retrieved from other scopes (such as modules)
-   can be.  This may change.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  2 21:00:52 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sun,  2 Jun 2013 21:00:52 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4ICMxNzk2NyAt?=
 =?utf-8?q?_Fix_related_to_regression_on_Windows=2E?=
Message-ID: <3bNpdJ0b45zS9f@mail.python.org>

http://hg.python.org/cpython/rev/e9d0fb934b46
changeset:   84015:e9d0fb934b46
branch:      2.7
parent:      84010:4657d0eebe42
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sun Jun 02 11:59:09 2013 -0700
summary:
  Fix #17967 - Fix related to regression on Windows.

os.path.join(*self.dirs) produces an invalid path on windows.
ftp paths are always forward-slash seperated like this. /pub/dir.

files:
  Lib/urllib.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/urllib.py b/Lib/urllib.py
--- a/Lib/urllib.py
+++ b/Lib/urllib.py
@@ -873,7 +873,8 @@
         self.ftp = ftplib.FTP()
         self.ftp.connect(self.host, self.port, self.timeout)
         self.ftp.login(self.user, self.passwd)
-        self.ftp.cwd(os.path.join(*self.dirs))
+        _target = '/'.join(self.dirs)
+        self.ftp.cwd(_target)
 
     def retrfile(self, file, type):
         import ftplib

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  2 21:00:53 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sun,  2 Jun 2013 21:00:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogRml4ICMxNzk2NyAt?=
 =?utf-8?q?_Fix_related_to_regression_on_Windows=2E?=
Message-ID: <3bNpdK2WWjzRZ6@mail.python.org>

http://hg.python.org/cpython/rev/f5906026a7e9
changeset:   84016:f5906026a7e9
branch:      3.3
parent:      84013:4c8426acd2cf
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sun Jun 02 11:59:47 2013 -0700
summary:
  Fix #17967 - Fix related to regression on Windows.

os.path.join(*self.dirs) produces an invalid path on windows.
ftp paths are always forward-slash seperated like this. /pub/dir.

files:
  Lib/urllib/request.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -2276,7 +2276,8 @@
         self.ftp = ftplib.FTP()
         self.ftp.connect(self.host, self.port, self.timeout)
         self.ftp.login(self.user, self.passwd)
-        self.ftp.cwd(os.path.join(*self.dirs))
+        _target = '/'.join(self.dirs)
+        self.ftp.cwd(_target)
 
     def retrfile(self, file, type):
         import ftplib

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  2 21:00:54 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Sun,  2 Jun 2013 21:00:54 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_merge_from_3=2E3?=
Message-ID: <3bNpdL4MBRz7Ln0@mail.python.org>

http://hg.python.org/cpython/rev/adfec512fb32
changeset:   84017:adfec512fb32
parent:      84014:49bcd1af0f49
parent:      84016:f5906026a7e9
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Sun Jun 02 12:00:45 2013 -0700
summary:
  merge from 3.3

Fix #17967 - Fix related to regression on Windows.

os.path.join(*self.dirs) produces an invalid path on windows.
ftp paths are always forward-slash seperated like this. /pub/dir.

files:
  Lib/urllib/request.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -2306,7 +2306,8 @@
         self.ftp = ftplib.FTP()
         self.ftp.connect(self.host, self.port, self.timeout)
         self.ftp.login(self.user, self.passwd)
-        self.ftp.cwd(os.path.join(*self.dirs))
+        _target = '/'.join(self.dirs)
+        self.ftp.cwd(_target)
 
     def retrfile(self, file, type):
         import ftplib

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Jun  3 05:53:05 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 03 Jun 2013 05:53:05 +0200
Subject: [Python-checkins] Daily reference leaks (adfec512fb32): sum=0
Message-ID: <E1UjLpR-00041A-0e@ap.vmr.nerim.net>

results for adfec512fb32 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/refloglA7A2l', '-x']

From python-checkins at python.org  Mon Jun  3 19:26:03 2013
From: python-checkins at python.org (phillip.eby)
Date: Mon,  3 Jun 2013 19:26:03 +0200 (CEST)
Subject: [Python-checkins] r89000 -
	sandbox/branches/setuptools-0.6/setuptools/ssl_support.py
Message-ID: <3bPNTR1c5NzRbh@mail.python.org>

Author: phillip.eby
Date: Mon Jun  3 19:26:03 2013
New Revision: 89000

Log:
Fix missing import


Modified:
   sandbox/branches/setuptools-0.6/setuptools/ssl_support.py

Modified: sandbox/branches/setuptools-0.6/setuptools/ssl_support.py
==============================================================================
--- sandbox/branches/setuptools-0.6/setuptools/ssl_support.py	(original)
+++ sandbox/branches/setuptools-0.6/setuptools/ssl_support.py	Mon Jun  3 19:26:03 2013
@@ -1,4 +1,4 @@
-import sys, os, socket, urllib2, atexit
+import sys, os, socket, urllib2, atexit, re
 from pkg_resources import ResolutionError, ExtractionError, resource_filename
 
 try:

From python-checkins at python.org  Mon Jun  3 19:26:16 2013
From: python-checkins at python.org (phillip.eby)
Date: Mon,  3 Jun 2013 19:26:16 +0200 (CEST)
Subject: [Python-checkins] r89001 -
	sandbox/trunk/setuptools/setuptools/ssl_support.py
Message-ID: <3bPNTh2jvXzSMx@mail.python.org>

Author: phillip.eby
Date: Mon Jun  3 19:26:16 2013
New Revision: 89001

Log:
Fix missing import


Modified:
   sandbox/trunk/setuptools/setuptools/ssl_support.py

Modified: sandbox/trunk/setuptools/setuptools/ssl_support.py
==============================================================================
--- sandbox/trunk/setuptools/setuptools/ssl_support.py	(original)
+++ sandbox/trunk/setuptools/setuptools/ssl_support.py	Mon Jun  3 19:26:16 2013
@@ -1,4 +1,4 @@
-import sys, os, socket, urllib2, atexit
+import sys, os, socket, urllib2, atexit, re
 from pkg_resources import ResolutionError, ExtractionError, resource_filename
 
 try:

From python-checkins at python.org  Mon Jun  3 22:15:06 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon,  3 Jun 2013 22:15:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_compilater_warnings_on?=
	=?utf-8?q?_Windows_64-bit?=
Message-ID: <3bPSDV45trz7LkJ@mail.python.org>

http://hg.python.org/cpython/rev/46d8fea24490
changeset:   84018:46d8fea24490
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri May 17 00:19:59 2013 +0200
summary:
  Fix compilater warnings on Windows 64-bit

files:
  Modules/_sqlite/util.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_sqlite/util.c b/Modules/_sqlite/util.c
--- a/Modules/_sqlite/util.c
+++ b/Modules/_sqlite/util.c
@@ -132,7 +132,7 @@
     }
 # endif
 #endif
-    return PyLong_FromLong(value);
+    return PyLong_FromLong(Py_SAFE_DOWNCAST(value, sqlite_int64, long));
 }
 
 sqlite_int64

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun  3 22:15:07 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon,  3 Jun 2013 22:15:07 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2UgIzE4MTA5?=
 =?utf-8?q?=3A_os=2Euname=28=29_now_decodes_fields_from_the_locale_encodin?=
 =?utf-8?q?g=2C_and?=
Message-ID: <3bPSDW6PKlz7Lkw@mail.python.org>

http://hg.python.org/cpython/rev/ffdee6b36305
changeset:   84019:ffdee6b36305
branch:      3.3
parent:      84016:f5906026a7e9
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 03 22:07:27 2013 +0200
summary:
  Close #18109: os.uname() now decodes fields from the locale encoding, and
socket.gethostname() now decodes the hostname from the locale encoding, instead
of using the UTF-8 encoding in strict mode.

files:
  Misc/NEWS              |   6 +++++-
  Modules/posixmodule.c  |   2 +-
  Modules/socketmodule.c |  16 ++++++++--------
  3 files changed, 14 insertions(+), 10 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -24,6 +24,10 @@
 Library
 -------
 
+- Issue #18109: os.uname() now decodes fields from the locale encoding, and
+  socket.gethostname() now decodes the hostname from the locale encoding,
+  instead of using the UTF-8 encoding in strict mode.
+
 - Issue #17403: urllib.parse.robotparser normalizes the urls before adding to
   ruleline. This helps in handling certain types invalid urls in a conservative
   manner.
@@ -69,7 +73,7 @@
 
 - Issue #15392: Create a unittest framework for IDLE.
   Initial patch by Rajagopalasarma Jayakrishnan.
-  
+
 - Issue #14146: Highlight source line while debugging on Windows.
 
 - Issue #17532: Always include Options menu for IDLE on OS X.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -4514,7 +4514,7 @@
 
 #define SET(i, field) \
     { \
-    PyObject *o = PyUnicode_DecodeASCII(field, strlen(field), NULL); \
+    PyObject *o = PyUnicode_DecodeFSDefault(field); \
     if (!o) { \
         Py_DECREF(value); \
         return NULL; \
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -1702,7 +1702,7 @@
             return 0;
         }
 #endif
-            
+
 #ifdef PF_SYSTEM
     case PF_SYSTEM:
         switch (s->sock_proto) {
@@ -1710,10 +1710,10 @@
         case SYSPROTO_CONTROL:
         {
             struct sockaddr_ctl *addr;
-            
+
             addr = (struct sockaddr_ctl *)addr_ret;
             addr->sc_family = AF_SYSTEM;
-            addr->ss_sysaddr = AF_SYS_CONTROL; 
+            addr->ss_sysaddr = AF_SYS_CONTROL;
 
             if (PyUnicode_Check(args)) {
                 struct ctl_info info;
@@ -1739,17 +1739,17 @@
                           "cannot find kernel control with provided name");
                     return 0;
                 }
-                
+
                 addr->sc_id = info.ctl_id;
                 addr->sc_unit = 0;
             } else if (!PyArg_ParseTuple(args, "II",
                                          &(addr->sc_id), &(addr->sc_unit))) {
                 PyErr_SetString(PyExc_TypeError, "getsockaddrarg: "
                                 "expected str or tuple of two ints");
-                
+
                 return 0;
             }
-              
+
             *len_ret = sizeof(*addr);
             return 1;
         }
@@ -1866,7 +1866,7 @@
         return 1;
     }
 #endif
-            
+
 #ifdef PF_SYSTEM
     case PF_SYSTEM:
         switch(s->sock_proto) {
@@ -4111,7 +4111,7 @@
     if (res < 0)
         return set_error();
     buf[sizeof buf - 1] = '\0';
-    return PyUnicode_FromString(buf);
+    return PyUnicode_DecodeFSDefault(buf);
 #endif
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun  3 22:15:09 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon,  3 Jun 2013 22:15:09 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogKE1lcmdlIDMuMykgQ2xvc2UgIzE4MTA5OiBvcy51bmFtZSgpIG5vdyBk?=
 =?utf-8?q?ecodes_fields_from_the_locale?=
Message-ID: <3bPSDY1fTgz7LmX@mail.python.org>

http://hg.python.org/cpython/rev/2472603af83e
changeset:   84020:2472603af83e
parent:      84018:46d8fea24490
parent:      84019:ffdee6b36305
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 03 22:09:14 2013 +0200
summary:
  (Merge 3.3) Close #18109: os.uname() now decodes fields from the locale
encoding, and socket.gethostname() now decodes the hostname from the locale
encoding, instead of using the UTF-8 encoding in strict mode.

files:
  Misc/NEWS              |   6 +++++-
  Modules/posixmodule.c  |   2 +-
  Modules/socketmodule.c |  16 ++++++++--------
  3 files changed, 14 insertions(+), 10 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -103,6 +103,10 @@
 Library
 -------
 
+- Issue #18109: os.uname() now decodes fields from the locale encoding, and
+  socket.gethostname() now decodes the hostname from the locale encoding,
+  instead of using the UTF-8 encoding in strict mode.
+
 - Issue #18089: Implement importlib.abc.InspectLoader.load_module.
 
 - Issue #18088: Introduce importlib.abc.Loader.init_module_attrs for setting
@@ -416,7 +420,7 @@
 
 - Issue #15392: Create a unittest framework for IDLE.
   Initial patch by Rajagopalasarma Jayakrishnan.
-  
+
 - Issue #14146: Highlight source line while debugging on Windows.
 
 - Issue #17838: Allow sys.stdin to be reassigned.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -4257,7 +4257,7 @@
 
 #define SET(i, field) \
     { \
-    PyObject *o = PyUnicode_DecodeASCII(field, strlen(field), NULL); \
+    PyObject *o = PyUnicode_DecodeFSDefault(field); \
     if (!o) { \
         Py_DECREF(value); \
         return NULL; \
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -1644,7 +1644,7 @@
             return 0;
         }
 #endif
-            
+
 #ifdef PF_SYSTEM
     case PF_SYSTEM:
         switch (s->sock_proto) {
@@ -1652,10 +1652,10 @@
         case SYSPROTO_CONTROL:
         {
             struct sockaddr_ctl *addr;
-            
+
             addr = (struct sockaddr_ctl *)addr_ret;
             addr->sc_family = AF_SYSTEM;
-            addr->ss_sysaddr = AF_SYS_CONTROL; 
+            addr->ss_sysaddr = AF_SYS_CONTROL;
 
             if (PyUnicode_Check(args)) {
                 struct ctl_info info;
@@ -1681,17 +1681,17 @@
                           "cannot find kernel control with provided name");
                     return 0;
                 }
-                
+
                 addr->sc_id = info.ctl_id;
                 addr->sc_unit = 0;
             } else if (!PyArg_ParseTuple(args, "II",
                                          &(addr->sc_id), &(addr->sc_unit))) {
                 PyErr_SetString(PyExc_TypeError, "getsockaddrarg: "
                                 "expected str or tuple of two ints");
-                
+
                 return 0;
             }
-              
+
             *len_ret = sizeof(*addr);
             return 1;
         }
@@ -1808,7 +1808,7 @@
         return 1;
     }
 #endif
-            
+
 #ifdef PF_SYSTEM
     case PF_SYSTEM:
         switch(s->sock_proto) {
@@ -4048,7 +4048,7 @@
     if (res < 0)
         return set_error();
     buf[sizeof buf - 1] = '\0';
-    return PyUnicode_FromString(buf);
+    return PyUnicode_DecodeFSDefault(buf);
 #endif
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Jun  4 05:51:18 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 04 Jun 2013 05:51:18 +0200
Subject: [Python-checkins] Daily reference leaks (2472603af83e): sum=0
Message-ID: <E1UjiHG-0007tL-Fr@ap.vmr.nerim.net>

results for 2472603af83e on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogIOah9N', '-x']

From python-checkins at python.org  Tue Jun  4 23:03:11 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue,  4 Jun 2013 23:03:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2317932=3A_Fix_an_i?=
 =?utf-8?q?nteger_overflow_issue_on_Windows_64-bit_in_iterators=3A?=
Message-ID: <3bQ5FW4xQ4z7Ljc@mail.python.org>

http://hg.python.org/cpython/rev/757a121a27c2
changeset:   84021:757a121a27c2
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 04 23:02:46 2013 +0200
summary:
  Close #17932: Fix an integer overflow issue on Windows 64-bit in iterators:
change the C type of seqiterobject.it_index from long to Py_ssize_t.

files:
  Misc/NEWS            |  3 +++
  Objects/iterobject.c |  2 +-
  2 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #17932: Fix an integer overflow issue on Windows 64-bit in iterators:
+  change the C type of seqiterobject.it_index from long to Py_ssize_t.
+
 - Issue #18065: Don't set __path__ to the package name for frozen packages.
 
 - Issue #18088: When reloading a module, unconditionally reset all relevant
diff --git a/Objects/iterobject.c b/Objects/iterobject.c
--- a/Objects/iterobject.c
+++ b/Objects/iterobject.c
@@ -4,7 +4,7 @@
 
 typedef struct {
     PyObject_HEAD
-    long      it_index;
+    Py_ssize_t it_index;
     PyObject *it_seq; /* Set to NULL when iterator is exhausted */
 } seqiterobject;
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun  4 23:15:41 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue,  4 Jun 2013 23:15:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Reuse_Py=5FMIN_and_Py=5FMA?=
 =?utf-8?q?X_macros=3A_remove_duplicate_MIN/MAX_macros?=
Message-ID: <3bQ5Wx3t8zzSyS@mail.python.org>

http://hg.python.org/cpython/rev/41a2cbe23349
changeset:   84022:41a2cbe23349
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 04 23:14:37 2013 +0200
summary:
  Reuse Py_MIN and Py_MAX macros: remove duplicate MIN/MAX macros

multiprocessing.h: remove unused MIN and MAX macros

files:
  Modules/_bz2module.c                       |  12 +++-----
  Modules/_cursesmodule.c                    |  14 +++------
  Modules/_multiprocessing/multiprocessing.h |   9 ------
  Modules/md5module.c                        |   6 +---
  Modules/sha1module.c                       |   6 +---
  Modules/socketmodule.c                     |   7 +---
  Objects/floatobject.c                      |  13 ++------
  Objects/frameobject.c                      |  11 ++-----
  Objects/longobject.c                       |  13 ++------
  9 files changed, 25 insertions(+), 66 deletions(-)


diff --git a/Modules/_bz2module.c b/Modules/_bz2module.c
--- a/Modules/_bz2module.c
+++ b/Modules/_bz2module.c
@@ -36,8 +36,6 @@
 #define RELEASE_LOCK(obj)
 #endif
 
-#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y))
-
 
 typedef struct {
     PyObject_HEAD
@@ -157,7 +155,7 @@
         /* On a 64-bit system, len might not fit in avail_in (an unsigned int).
            Do compression in chunks of no more than UINT_MAX bytes each. */
         if (c->bzs.avail_in == 0 && len > 0) {
-            c->bzs.avail_in = MIN(len, UINT_MAX);
+            c->bzs.avail_in = Py_MIN(len, UINT_MAX);
             len -= c->bzs.avail_in;
         }
 
@@ -173,7 +171,7 @@
                 c->bzs.next_out = PyBytes_AS_STRING(result) + data_size;
                 buffer_left = PyBytes_GET_SIZE(result) - data_size;
             }
-            c->bzs.avail_out = MIN(buffer_left, UINT_MAX);
+            c->bzs.avail_out = Py_MIN(buffer_left, UINT_MAX);
         }
 
         Py_BEGIN_ALLOW_THREADS
@@ -370,7 +368,7 @@
     d->bzs.next_in = data;
     /* On a 64-bit system, len might not fit in avail_in (an unsigned int).
        Do decompression in chunks of no more than UINT_MAX bytes each. */
-    d->bzs.avail_in = MIN(len, UINT_MAX);
+    d->bzs.avail_in = Py_MIN(len, UINT_MAX);
     len -= d->bzs.avail_in;
     d->bzs.next_out = PyBytes_AS_STRING(result);
     d->bzs.avail_out = PyBytes_GET_SIZE(result);
@@ -399,7 +397,7 @@
         if (d->bzs.avail_in == 0) {
             if (len == 0)
                 break;
-            d->bzs.avail_in = MIN(len, UINT_MAX);
+            d->bzs.avail_in = Py_MIN(len, UINT_MAX);
             len -= d->bzs.avail_in;
         }
         if (d->bzs.avail_out == 0) {
@@ -410,7 +408,7 @@
                 d->bzs.next_out = PyBytes_AS_STRING(result) + data_size;
                 buffer_left = PyBytes_GET_SIZE(result) - data_size;
             }
-            d->bzs.avail_out = MIN(buffer_left, UINT_MAX);
+            d->bzs.avail_out = Py_MIN(buffer_left, UINT_MAX);
         }
     }
     if (data_size != PyBytes_GET_SIZE(result))
diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c
--- a/Modules/_cursesmodule.c
+++ b/Modules/_cursesmodule.c
@@ -168,10 +168,6 @@
                         "must call start_color() first");       \
         return 0; }
 
-#ifndef MIN
-#define MIN(x,y) ((x) < (y) ? (x) : (y))
-#endif
-
 /* Utility Functions */
 
 /*
@@ -1212,7 +1208,7 @@
         if (!PyArg_ParseTuple(args,"i;n", &n))
             return NULL;
         Py_BEGIN_ALLOW_THREADS
-        rtn2 = wgetnstr(self->win,rtn,MIN(n, 1023));
+        rtn2 = wgetnstr(self->win, rtn, Py_MIN(n, 1023));
         Py_END_ALLOW_THREADS
         break;
     case 2:
@@ -1232,11 +1228,11 @@
 #ifdef STRICT_SYSV_CURSES
         Py_BEGIN_ALLOW_THREADS
         rtn2 = wmove(self->win,y,x)==ERR ? ERR :
-        wgetnstr(self->win, rtn, MIN(n, 1023));
+        wgetnstr(self->win, rtn, Py_MIN(n, 1023));
         Py_END_ALLOW_THREADS
 #else
         Py_BEGIN_ALLOW_THREADS
-        rtn2 = mvwgetnstr(self->win, y, x, rtn, MIN(n, 1023));
+        rtn2 = mvwgetnstr(self->win, y, x, rtn, Py_MIN(n, 1023));
         Py_END_ALLOW_THREADS
 #endif
         break;
@@ -1374,7 +1370,7 @@
     case 1:
         if (!PyArg_ParseTuple(args,"i;n", &n))
             return NULL;
-        rtn2 = winnstr(self->win,rtn,MIN(n,1023));
+        rtn2 = winnstr(self->win, rtn, Py_MIN(n, 1023));
         break;
     case 2:
         if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x))
@@ -1384,7 +1380,7 @@
     case 3:
         if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n))
             return NULL;
-        rtn2 = mvwinnstr(self->win, y, x, rtn, MIN(n,1023));
+        rtn2 = mvwinnstr(self->win, y, x, rtn, Py_MIN(n,1023));
         break;
     default:
         PyErr_SetString(PyExc_TypeError, "instr requires 0 or 3 arguments");
diff --git a/Modules/_multiprocessing/multiprocessing.h b/Modules/_multiprocessing/multiprocessing.h
--- a/Modules/_multiprocessing/multiprocessing.h
+++ b/Modules/_multiprocessing/multiprocessing.h
@@ -99,13 +99,4 @@
 
 extern PyTypeObject _PyMp_SemLockType;
 
-/*
- * Miscellaneous
- */
-
-#ifndef MIN
-#  define MIN(x, y) ((x) < (y) ? x : y)
-#  define MAX(x, y) ((x) > (y) ? x : y)
-#endif
-
 #endif /* MULTIPROCESSING_H */
diff --git a/Modules/md5module.c b/Modules/md5module.c
--- a/Modules/md5module.c
+++ b/Modules/md5module.c
@@ -91,10 +91,6 @@
        (y)[3] = (unsigned char)(((x)>>24)&255); (y)[2] = (unsigned char)(((x)>>16)&255);   \
        (y)[1] = (unsigned char)(((x)>>8)&255); (y)[0] = (unsigned char)((x)&255); }
 
-#ifndef MIN
-   #define MIN(x, y) ( ((x)<(y))?(x):(y) )
-#endif
-
 
 /* MD5 macros */
 
@@ -244,7 +240,7 @@
            in             += MD5_BLOCKSIZE;
            inlen          -= MD5_BLOCKSIZE;
         } else {
-           n = MIN(inlen, (Py_ssize_t)(MD5_BLOCKSIZE - md5->curlen));
+           n = Py_MIN(inlen, (Py_ssize_t)(MD5_BLOCKSIZE - md5->curlen));
            memcpy(md5->buf + md5->curlen, in, (size_t)n);
            md5->curlen    += (MD5_INT32)n;
            in             += n;
diff --git a/Modules/sha1module.c b/Modules/sha1module.c
--- a/Modules/sha1module.c
+++ b/Modules/sha1module.c
@@ -92,10 +92,6 @@
      (y)[4] = (unsigned char)(((x)>>24)&255); (y)[5] = (unsigned char)(((x)>>16)&255);     \
      (y)[6] = (unsigned char)(((x)>>8)&255); (y)[7] = (unsigned char)((x)&255); }
 
-#ifndef MIN
-   #define MIN(x, y) ( ((x)<(y))?(x):(y) )
-#endif
-
 
 /* SHA1 macros */
 
@@ -220,7 +216,7 @@
            in             += SHA1_BLOCKSIZE;
            inlen          -= SHA1_BLOCKSIZE;
         } else {
-           n = MIN(inlen, (Py_ssize_t)(SHA1_BLOCKSIZE - sha1->curlen));
+           n = Py_MIN(inlen, (Py_ssize_t)(SHA1_BLOCKSIZE - sha1->curlen));
            memcpy(sha1->buf + sha1->curlen, in, (size_t)n);
            sha1->curlen   += (SHA1_INT32)n;
            in             += n;
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -95,9 +95,6 @@
 #include "Python.h"
 #include "structmember.h"
 
-#undef MAX
-#define MAX(x, y) ((x) < (y) ? (y) : (x))
-
 /* Socket object documentation */
 PyDoc_STRVAR(sock_doc,
 "socket([family[, type[, proto]]]) -> socket object\n\
@@ -4819,7 +4816,7 @@
     char* ip;
     int retval;
 #ifdef ENABLE_IPV6
-    char packed[MAX(sizeof(struct in_addr), sizeof(struct in6_addr))];
+    char packed[Py_MAX(sizeof(struct in_addr), sizeof(struct in6_addr))];
 #else
     char packed[sizeof(struct in_addr)];
 #endif
@@ -4870,7 +4867,7 @@
     int len;
     const char* retval;
 #ifdef ENABLE_IPV6
-    char ip[MAX(INET_ADDRSTRLEN, INET6_ADDRSTRLEN) + 1];
+    char ip[Py_MAX(INET_ADDRSTRLEN, INET6_ADDRSTRLEN) + 1];
 #else
     char ip[INET_ADDRSTRLEN + 1];
 #endif
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -9,11 +9,6 @@
 #include <ctype.h>
 #include <float.h>
 
-#undef MAX
-#undef MIN
-#define MAX(x, y) ((x) < (y) ? (y) : (x))
-#define MIN(x, y) ((x) < (y) ? (x) : (y))
-
 
 /* Special free list
    free_list is a singly-linked list of available PyFloatObjects, linked
@@ -1131,7 +1126,7 @@
     }
 
     m = frexp(fabs(x), &e);
-    shift = 1 - MAX(DBL_MIN_EXP - e, 0);
+    shift = 1 - Py_MAX(DBL_MIN_EXP - e, 0);
     m = ldexp(m, shift);
     e -= shift;
 
@@ -1285,8 +1280,8 @@
     fdigits = coeff_end - s_store;
     if (ndigits == 0)
         goto parse_error;
-    if (ndigits > MIN(DBL_MIN_EXP - DBL_MANT_DIG - LONG_MIN/2,
-                      LONG_MAX/2 + 1 - DBL_MAX_EXP)/4)
+    if (ndigits > Py_MIN(DBL_MIN_EXP - DBL_MANT_DIG - LONG_MIN/2,
+                         LONG_MAX/2 + 1 - DBL_MAX_EXP)/4)
         goto insane_length_error;
 
     /* [p <exponent>] */
@@ -1342,7 +1337,7 @@
 
     /* lsb = exponent of least significant bit of the *rounded* value.
        This is top_exp - DBL_MANT_DIG unless result is subnormal. */
-    lsb = MAX(top_exp, (long)DBL_MIN_EXP) - DBL_MANT_DIG;
+    lsb = Py_MAX(top_exp, (long)DBL_MIN_EXP) - DBL_MANT_DIG;
 
     x = 0.0;
     if (exp >= lsb) {
diff --git a/Objects/frameobject.c b/Objects/frameobject.c
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -7,11 +7,6 @@
 #include "opcode.h"
 #include "structmember.h"
 
-#undef MIN
-#undef MAX
-#define MIN(a, b) ((a) < (b) ? (a) : (b))
-#define MAX(a, b) ((a) > (b) ? (a) : (b))
-
 #define OFF(x) offsetof(PyFrameObject, x)
 
 static PyMemberDef frame_memberlist[] = {
@@ -160,8 +155,8 @@
 
     /* We're now ready to look at the bytecode. */
     PyBytes_AsStringAndSize(f->f_code->co_code, (char **)&code, &code_len);
-    min_addr = MIN(new_lasti, f->f_lasti);
-    max_addr = MAX(new_lasti, f->f_lasti);
+    min_addr = Py_MIN(new_lasti, f->f_lasti);
+    max_addr = Py_MAX(new_lasti, f->f_lasti);
 
     /* You can't jump onto a line with an 'except' statement on it -
      * they expect to have an exception on the top of the stack, which
@@ -293,7 +288,7 @@
             break;
         }
 
-        min_delta_iblock = MIN(min_delta_iblock, delta_iblock);
+        min_delta_iblock = Py_MIN(min_delta_iblock, delta_iblock);
 
         if (op >= HAVE_ARGUMENT) {
             addr += 2;
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -89,11 +89,6 @@
  */
 #define FIVEARY_CUTOFF 8
 
-#undef MIN
-#undef MAX
-#define MAX(x, y) ((x) < (y) ? (y) : (x))
-#define MIN(x, y) ((x) > (y) ? (y) : (x))
-
 #define SIGCHECK(PyTryBlock)                    \
     do {                                        \
         if (PyErr_CheckSignals()) PyTryBlock    \
@@ -3029,7 +3024,7 @@
     Py_ssize_t size_lo, size_hi;
     const Py_ssize_t size_n = ABS(Py_SIZE(n));
 
-    size_lo = MIN(size_n, size);
+    size_lo = Py_MIN(size_n, size);
     size_hi = size_n - size_lo;
 
     if ((hi = _PyLong_New(size_hi)) == NULL)
@@ -3300,7 +3295,7 @@
     nbdone = 0;
     while (bsize > 0) {
         PyLongObject *product;
-        const Py_ssize_t nbtouse = MIN(bsize, asize);
+        const Py_ssize_t nbtouse = Py_MIN(bsize, asize);
 
         /* Multiply the next slice of b by a. */
         memcpy(bslice->ob_digit, b->ob_digit + nbdone,
@@ -3591,7 +3586,7 @@
         goto underflow_or_zero;
 
     /* Choose value for shift; see comments for step 1 above. */
-    shift = MAX(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2;
+    shift = Py_MAX(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2;
 
     inexact = 0;
 
@@ -3662,7 +3657,7 @@
     x_bits = (x_size-1)*PyLong_SHIFT+bits_in_digit(x->ob_digit[x_size-1]);
 
     /* The number of extra bits that have to be rounded away. */
-    extra_bits = MAX(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG;
+    extra_bits = Py_MAX(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG;
     assert(extra_bits == 2 || extra_bits == 3);
 
     /* Round by directly modifying the low digit of x. */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun  4 23:36:16 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue,  4 Jun 2013 23:36:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Tweak_at_the_suggestion_of?=
 =?utf-8?q?_Ezio_Melotti_for_exception_messages_when?=
Message-ID: <3bQ5zh1q7PzSRc@mail.python.org>

http://hg.python.org/cpython/rev/9e833c1edeb6
changeset:   84023:9e833c1edeb6
parent:      84020:2472603af83e
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 04 17:34:49 2013 -0400
summary:
  Tweak at the suggestion of Ezio Melotti for exception messages when
EOF is hit while trying to read the header of a bytecode file.

files:
  Lib/importlib/_bootstrap.py |     4 +-
  Misc/NEWS                   |     3 +
  Python/importlib.h          |  4610 +++++++++++-----------
  3 files changed, 2311 insertions(+), 2306 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -668,11 +668,11 @@
         _verbose_message(message)
         raise ImportError(message, **exc_details)
     elif len(raw_timestamp) != 4:
-        message = 'incomplete timestamp in {!r}'.format(name)
+        message = 'reached EOF while reading magic number in {!r}'.format(name)
         _verbose_message(message)
         raise EOFError(message)
     elif len(raw_size) != 4:
-        message = 'incomplete size in {!r}'.format(name)
+        message = 'reached EOF while reading size in {!r}'.format(name)
         _verbose_message(message)
         raise EOFError(message)
     if source_stats is not None:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Tweak the exception message when the magic number or size value in a bytecode
+  file is truncated.
+
 - Issue #18065: Don't set __path__ to the package name for frozen packages.
 
 - Issue #18088: When reloading a module, unconditionally reset all relevant
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun  4 23:36:17 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue,  4 Jun 2013 23:36:17 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bQ5zj5ZTRzSVJ@mail.python.org>

http://hg.python.org/cpython/rev/b9af38d33606
changeset:   84024:b9af38d33606
parent:      84023:9e833c1edeb6
parent:      84022:41a2cbe23349
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 04 17:36:07 2013 -0400
summary:
  merge

files:
  Misc/NEWS                                  |   3 ++
  Modules/_bz2module.c                       |  12 +++-----
  Modules/_cursesmodule.c                    |  14 +++------
  Modules/_multiprocessing/multiprocessing.h |   9 ------
  Modules/md5module.c                        |   6 +---
  Modules/sha1module.c                       |   6 +---
  Modules/socketmodule.c                     |   7 +---
  Objects/floatobject.c                      |  13 ++------
  Objects/frameobject.c                      |  11 ++-----
  Objects/iterobject.c                       |   2 +-
  Objects/longobject.c                       |  13 ++------
  11 files changed, 29 insertions(+), 67 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -13,6 +13,9 @@
 - Tweak the exception message when the magic number or size value in a bytecode
   file is truncated.
 
+- Issue #17932: Fix an integer overflow issue on Windows 64-bit in iterators:
+  change the C type of seqiterobject.it_index from long to Py_ssize_t.
+
 - Issue #18065: Don't set __path__ to the package name for frozen packages.
 
 - Issue #18088: When reloading a module, unconditionally reset all relevant
diff --git a/Modules/_bz2module.c b/Modules/_bz2module.c
--- a/Modules/_bz2module.c
+++ b/Modules/_bz2module.c
@@ -36,8 +36,6 @@
 #define RELEASE_LOCK(obj)
 #endif
 
-#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y))
-
 
 typedef struct {
     PyObject_HEAD
@@ -157,7 +155,7 @@
         /* On a 64-bit system, len might not fit in avail_in (an unsigned int).
            Do compression in chunks of no more than UINT_MAX bytes each. */
         if (c->bzs.avail_in == 0 && len > 0) {
-            c->bzs.avail_in = MIN(len, UINT_MAX);
+            c->bzs.avail_in = Py_MIN(len, UINT_MAX);
             len -= c->bzs.avail_in;
         }
 
@@ -173,7 +171,7 @@
                 c->bzs.next_out = PyBytes_AS_STRING(result) + data_size;
                 buffer_left = PyBytes_GET_SIZE(result) - data_size;
             }
-            c->bzs.avail_out = MIN(buffer_left, UINT_MAX);
+            c->bzs.avail_out = Py_MIN(buffer_left, UINT_MAX);
         }
 
         Py_BEGIN_ALLOW_THREADS
@@ -370,7 +368,7 @@
     d->bzs.next_in = data;
     /* On a 64-bit system, len might not fit in avail_in (an unsigned int).
        Do decompression in chunks of no more than UINT_MAX bytes each. */
-    d->bzs.avail_in = MIN(len, UINT_MAX);
+    d->bzs.avail_in = Py_MIN(len, UINT_MAX);
     len -= d->bzs.avail_in;
     d->bzs.next_out = PyBytes_AS_STRING(result);
     d->bzs.avail_out = PyBytes_GET_SIZE(result);
@@ -399,7 +397,7 @@
         if (d->bzs.avail_in == 0) {
             if (len == 0)
                 break;
-            d->bzs.avail_in = MIN(len, UINT_MAX);
+            d->bzs.avail_in = Py_MIN(len, UINT_MAX);
             len -= d->bzs.avail_in;
         }
         if (d->bzs.avail_out == 0) {
@@ -410,7 +408,7 @@
                 d->bzs.next_out = PyBytes_AS_STRING(result) + data_size;
                 buffer_left = PyBytes_GET_SIZE(result) - data_size;
             }
-            d->bzs.avail_out = MIN(buffer_left, UINT_MAX);
+            d->bzs.avail_out = Py_MIN(buffer_left, UINT_MAX);
         }
     }
     if (data_size != PyBytes_GET_SIZE(result))
diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c
--- a/Modules/_cursesmodule.c
+++ b/Modules/_cursesmodule.c
@@ -168,10 +168,6 @@
                         "must call start_color() first");       \
         return 0; }
 
-#ifndef MIN
-#define MIN(x,y) ((x) < (y) ? (x) : (y))
-#endif
-
 /* Utility Functions */
 
 /*
@@ -1212,7 +1208,7 @@
         if (!PyArg_ParseTuple(args,"i;n", &n))
             return NULL;
         Py_BEGIN_ALLOW_THREADS
-        rtn2 = wgetnstr(self->win,rtn,MIN(n, 1023));
+        rtn2 = wgetnstr(self->win, rtn, Py_MIN(n, 1023));
         Py_END_ALLOW_THREADS
         break;
     case 2:
@@ -1232,11 +1228,11 @@
 #ifdef STRICT_SYSV_CURSES
         Py_BEGIN_ALLOW_THREADS
         rtn2 = wmove(self->win,y,x)==ERR ? ERR :
-        wgetnstr(self->win, rtn, MIN(n, 1023));
+        wgetnstr(self->win, rtn, Py_MIN(n, 1023));
         Py_END_ALLOW_THREADS
 #else
         Py_BEGIN_ALLOW_THREADS
-        rtn2 = mvwgetnstr(self->win, y, x, rtn, MIN(n, 1023));
+        rtn2 = mvwgetnstr(self->win, y, x, rtn, Py_MIN(n, 1023));
         Py_END_ALLOW_THREADS
 #endif
         break;
@@ -1374,7 +1370,7 @@
     case 1:
         if (!PyArg_ParseTuple(args,"i;n", &n))
             return NULL;
-        rtn2 = winnstr(self->win,rtn,MIN(n,1023));
+        rtn2 = winnstr(self->win, rtn, Py_MIN(n, 1023));
         break;
     case 2:
         if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x))
@@ -1384,7 +1380,7 @@
     case 3:
         if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n))
             return NULL;
-        rtn2 = mvwinnstr(self->win, y, x, rtn, MIN(n,1023));
+        rtn2 = mvwinnstr(self->win, y, x, rtn, Py_MIN(n,1023));
         break;
     default:
         PyErr_SetString(PyExc_TypeError, "instr requires 0 or 3 arguments");
diff --git a/Modules/_multiprocessing/multiprocessing.h b/Modules/_multiprocessing/multiprocessing.h
--- a/Modules/_multiprocessing/multiprocessing.h
+++ b/Modules/_multiprocessing/multiprocessing.h
@@ -99,13 +99,4 @@
 
 extern PyTypeObject _PyMp_SemLockType;
 
-/*
- * Miscellaneous
- */
-
-#ifndef MIN
-#  define MIN(x, y) ((x) < (y) ? x : y)
-#  define MAX(x, y) ((x) > (y) ? x : y)
-#endif
-
 #endif /* MULTIPROCESSING_H */
diff --git a/Modules/md5module.c b/Modules/md5module.c
--- a/Modules/md5module.c
+++ b/Modules/md5module.c
@@ -91,10 +91,6 @@
        (y)[3] = (unsigned char)(((x)>>24)&255); (y)[2] = (unsigned char)(((x)>>16)&255);   \
        (y)[1] = (unsigned char)(((x)>>8)&255); (y)[0] = (unsigned char)((x)&255); }
 
-#ifndef MIN
-   #define MIN(x, y) ( ((x)<(y))?(x):(y) )
-#endif
-
 
 /* MD5 macros */
 
@@ -244,7 +240,7 @@
            in             += MD5_BLOCKSIZE;
            inlen          -= MD5_BLOCKSIZE;
         } else {
-           n = MIN(inlen, (Py_ssize_t)(MD5_BLOCKSIZE - md5->curlen));
+           n = Py_MIN(inlen, (Py_ssize_t)(MD5_BLOCKSIZE - md5->curlen));
            memcpy(md5->buf + md5->curlen, in, (size_t)n);
            md5->curlen    += (MD5_INT32)n;
            in             += n;
diff --git a/Modules/sha1module.c b/Modules/sha1module.c
--- a/Modules/sha1module.c
+++ b/Modules/sha1module.c
@@ -92,10 +92,6 @@
      (y)[4] = (unsigned char)(((x)>>24)&255); (y)[5] = (unsigned char)(((x)>>16)&255);     \
      (y)[6] = (unsigned char)(((x)>>8)&255); (y)[7] = (unsigned char)((x)&255); }
 
-#ifndef MIN
-   #define MIN(x, y) ( ((x)<(y))?(x):(y) )
-#endif
-
 
 /* SHA1 macros */
 
@@ -220,7 +216,7 @@
            in             += SHA1_BLOCKSIZE;
            inlen          -= SHA1_BLOCKSIZE;
         } else {
-           n = MIN(inlen, (Py_ssize_t)(SHA1_BLOCKSIZE - sha1->curlen));
+           n = Py_MIN(inlen, (Py_ssize_t)(SHA1_BLOCKSIZE - sha1->curlen));
            memcpy(sha1->buf + sha1->curlen, in, (size_t)n);
            sha1->curlen   += (SHA1_INT32)n;
            in             += n;
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -95,9 +95,6 @@
 #include "Python.h"
 #include "structmember.h"
 
-#undef MAX
-#define MAX(x, y) ((x) < (y) ? (y) : (x))
-
 /* Socket object documentation */
 PyDoc_STRVAR(sock_doc,
 "socket([family[, type[, proto]]]) -> socket object\n\
@@ -4819,7 +4816,7 @@
     char* ip;
     int retval;
 #ifdef ENABLE_IPV6
-    char packed[MAX(sizeof(struct in_addr), sizeof(struct in6_addr))];
+    char packed[Py_MAX(sizeof(struct in_addr), sizeof(struct in6_addr))];
 #else
     char packed[sizeof(struct in_addr)];
 #endif
@@ -4870,7 +4867,7 @@
     int len;
     const char* retval;
 #ifdef ENABLE_IPV6
-    char ip[MAX(INET_ADDRSTRLEN, INET6_ADDRSTRLEN) + 1];
+    char ip[Py_MAX(INET_ADDRSTRLEN, INET6_ADDRSTRLEN) + 1];
 #else
     char ip[INET_ADDRSTRLEN + 1];
 #endif
diff --git a/Objects/floatobject.c b/Objects/floatobject.c
--- a/Objects/floatobject.c
+++ b/Objects/floatobject.c
@@ -9,11 +9,6 @@
 #include <ctype.h>
 #include <float.h>
 
-#undef MAX
-#undef MIN
-#define MAX(x, y) ((x) < (y) ? (y) : (x))
-#define MIN(x, y) ((x) < (y) ? (x) : (y))
-
 
 /* Special free list
    free_list is a singly-linked list of available PyFloatObjects, linked
@@ -1131,7 +1126,7 @@
     }
 
     m = frexp(fabs(x), &e);
-    shift = 1 - MAX(DBL_MIN_EXP - e, 0);
+    shift = 1 - Py_MAX(DBL_MIN_EXP - e, 0);
     m = ldexp(m, shift);
     e -= shift;
 
@@ -1285,8 +1280,8 @@
     fdigits = coeff_end - s_store;
     if (ndigits == 0)
         goto parse_error;
-    if (ndigits > MIN(DBL_MIN_EXP - DBL_MANT_DIG - LONG_MIN/2,
-                      LONG_MAX/2 + 1 - DBL_MAX_EXP)/4)
+    if (ndigits > Py_MIN(DBL_MIN_EXP - DBL_MANT_DIG - LONG_MIN/2,
+                         LONG_MAX/2 + 1 - DBL_MAX_EXP)/4)
         goto insane_length_error;
 
     /* [p <exponent>] */
@@ -1342,7 +1337,7 @@
 
     /* lsb = exponent of least significant bit of the *rounded* value.
        This is top_exp - DBL_MANT_DIG unless result is subnormal. */
-    lsb = MAX(top_exp, (long)DBL_MIN_EXP) - DBL_MANT_DIG;
+    lsb = Py_MAX(top_exp, (long)DBL_MIN_EXP) - DBL_MANT_DIG;
 
     x = 0.0;
     if (exp >= lsb) {
diff --git a/Objects/frameobject.c b/Objects/frameobject.c
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -7,11 +7,6 @@
 #include "opcode.h"
 #include "structmember.h"
 
-#undef MIN
-#undef MAX
-#define MIN(a, b) ((a) < (b) ? (a) : (b))
-#define MAX(a, b) ((a) > (b) ? (a) : (b))
-
 #define OFF(x) offsetof(PyFrameObject, x)
 
 static PyMemberDef frame_memberlist[] = {
@@ -160,8 +155,8 @@
 
     /* We're now ready to look at the bytecode. */
     PyBytes_AsStringAndSize(f->f_code->co_code, (char **)&code, &code_len);
-    min_addr = MIN(new_lasti, f->f_lasti);
-    max_addr = MAX(new_lasti, f->f_lasti);
+    min_addr = Py_MIN(new_lasti, f->f_lasti);
+    max_addr = Py_MAX(new_lasti, f->f_lasti);
 
     /* You can't jump onto a line with an 'except' statement on it -
      * they expect to have an exception on the top of the stack, which
@@ -293,7 +288,7 @@
             break;
         }
 
-        min_delta_iblock = MIN(min_delta_iblock, delta_iblock);
+        min_delta_iblock = Py_MIN(min_delta_iblock, delta_iblock);
 
         if (op >= HAVE_ARGUMENT) {
             addr += 2;
diff --git a/Objects/iterobject.c b/Objects/iterobject.c
--- a/Objects/iterobject.c
+++ b/Objects/iterobject.c
@@ -4,7 +4,7 @@
 
 typedef struct {
     PyObject_HEAD
-    long      it_index;
+    Py_ssize_t it_index;
     PyObject *it_seq; /* Set to NULL when iterator is exhausted */
 } seqiterobject;
 
diff --git a/Objects/longobject.c b/Objects/longobject.c
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -89,11 +89,6 @@
  */
 #define FIVEARY_CUTOFF 8
 
-#undef MIN
-#undef MAX
-#define MAX(x, y) ((x) < (y) ? (y) : (x))
-#define MIN(x, y) ((x) > (y) ? (y) : (x))
-
 #define SIGCHECK(PyTryBlock)                    \
     do {                                        \
         if (PyErr_CheckSignals()) PyTryBlock    \
@@ -3029,7 +3024,7 @@
     Py_ssize_t size_lo, size_hi;
     const Py_ssize_t size_n = ABS(Py_SIZE(n));
 
-    size_lo = MIN(size_n, size);
+    size_lo = Py_MIN(size_n, size);
     size_hi = size_n - size_lo;
 
     if ((hi = _PyLong_New(size_hi)) == NULL)
@@ -3300,7 +3295,7 @@
     nbdone = 0;
     while (bsize > 0) {
         PyLongObject *product;
-        const Py_ssize_t nbtouse = MIN(bsize, asize);
+        const Py_ssize_t nbtouse = Py_MIN(bsize, asize);
 
         /* Multiply the next slice of b by a. */
         memcpy(bslice->ob_digit, b->ob_digit + nbdone,
@@ -3591,7 +3586,7 @@
         goto underflow_or_zero;
 
     /* Choose value for shift; see comments for step 1 above. */
-    shift = MAX(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2;
+    shift = Py_MAX(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2;
 
     inexact = 0;
 
@@ -3662,7 +3657,7 @@
     x_bits = (x_size-1)*PyLong_SHIFT+bits_in_digit(x->ob_digit[x_size-1]);
 
     /* The number of extra bits that have to be rounded away. */
-    extra_bits = MAX(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG;
+    extra_bits = Py_MAX(x_bits, DBL_MIN_EXP - shift) - DBL_MANT_DIG;
     assert(extra_bits == 2 || extra_bits == 3);
 
     /* Round by directly modifying the low digit of x. */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun  4 23:57:56 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue,  4 Jun 2013 23:57:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_compi?=
 =?utf-8?q?ler_warning_on_Windows_64-bit_in_=5Fbz2module=2Ec?=
Message-ID: <3bQ6Sh6kNPzSsq@mail.python.org>

http://hg.python.org/cpython/rev/46401ce03547
changeset:   84025:46401ce03547
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 04 23:18:48 2013 +0200
summary:
  Issue #9566: Fix compiler warning on Windows 64-bit in _bz2module.c

files:
  Modules/_bz2module.c |  14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Modules/_bz2module.c b/Modules/_bz2module.c
--- a/Modules/_bz2module.c
+++ b/Modules/_bz2module.c
@@ -147,7 +147,7 @@
     c->bzs.next_in = data;
     c->bzs.avail_in = 0;
     c->bzs.next_out = PyBytes_AS_STRING(result);
-    c->bzs.avail_out = PyBytes_GET_SIZE(result);
+    c->bzs.avail_out = SMALLCHUNK;
     for (;;) {
         char *this_out;
         int bzerror;
@@ -155,7 +155,7 @@
         /* On a 64-bit system, len might not fit in avail_in (an unsigned int).
            Do compression in chunks of no more than UINT_MAX bytes each. */
         if (c->bzs.avail_in == 0 && len > 0) {
-            c->bzs.avail_in = Py_MIN(len, UINT_MAX);
+            c->bzs.avail_in = (unsigned int)Py_MIN(len, UINT_MAX);
             len -= c->bzs.avail_in;
         }
 
@@ -171,7 +171,7 @@
                 c->bzs.next_out = PyBytes_AS_STRING(result) + data_size;
                 buffer_left = PyBytes_GET_SIZE(result) - data_size;
             }
-            c->bzs.avail_out = Py_MIN(buffer_left, UINT_MAX);
+            c->bzs.avail_out = (unsigned int)Py_MIN(buffer_left, UINT_MAX);
         }
 
         Py_BEGIN_ALLOW_THREADS
@@ -368,10 +368,10 @@
     d->bzs.next_in = data;
     /* On a 64-bit system, len might not fit in avail_in (an unsigned int).
        Do decompression in chunks of no more than UINT_MAX bytes each. */
-    d->bzs.avail_in = Py_MIN(len, UINT_MAX);
+    d->bzs.avail_in = (unsigned int)Py_MIN(len, UINT_MAX);
     len -= d->bzs.avail_in;
     d->bzs.next_out = PyBytes_AS_STRING(result);
-    d->bzs.avail_out = PyBytes_GET_SIZE(result);
+    d->bzs.avail_out = SMALLCHUNK;
     for (;;) {
         char *this_out;
         int bzerror;
@@ -397,7 +397,7 @@
         if (d->bzs.avail_in == 0) {
             if (len == 0)
                 break;
-            d->bzs.avail_in = Py_MIN(len, UINT_MAX);
+            d->bzs.avail_in = (unsigned int)Py_MIN(len, UINT_MAX);
             len -= d->bzs.avail_in;
         }
         if (d->bzs.avail_out == 0) {
@@ -408,7 +408,7 @@
                 d->bzs.next_out = PyBytes_AS_STRING(result) + data_size;
                 buffer_left = PyBytes_GET_SIZE(result) - data_size;
             }
-            d->bzs.avail_out = Py_MIN(buffer_left, UINT_MAX);
+            d->bzs.avail_out = (unsigned int)Py_MIN(buffer_left, UINT_MAX);
         }
     }
     if (data_size != PyBytes_GET_SIZE(result))

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun  4 23:57:58 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue,  4 Jun 2013 23:57:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Close_=2317931=3A_Fix_PyLo?=
 =?utf-8?q?ng=5FFromPid=28=29_on_Windows_64-bit=3A_processes_are_identifie?=
 =?utf-8?q?d?=
Message-ID: <3bQ6Sk2fjkzSsq@mail.python.org>

http://hg.python.org/cpython/rev/2298bcba6ec9
changeset:   84026:2298bcba6ec9
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 04 23:56:38 2013 +0200
summary:
  Close #17931: Fix PyLong_FromPid() on Windows 64-bit: processes are identified
by their HANDLE which is a pointer (and not a long, which is smaller).

files:
  Include/pyport.h |  4 ----
  Misc/NEWS        |  4 ++++
  PC/pyconfig.h    |  3 +++
  3 files changed, 7 insertions(+), 4 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -219,10 +219,6 @@
 /* Smallest negative value of type Py_ssize_t. */
 #define PY_SSIZE_T_MIN (-PY_SSIZE_T_MAX-1)
 
-#if SIZEOF_PID_T > SIZEOF_LONG
-#   error "Python doesn't support sizeof(pid_t) > sizeof(long)"
-#endif
-
 /* PY_FORMAT_SIZE_T is a platform-specific modifier for use in a printf
  * format to convert an argument with the width of a size_t or Py_ssize_t.
  * C99 introduced "z" for this purpose, but not all platforms support that;
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,10 @@
 Core and Builtins
 -----------------
 
+- Issue #17931: Fix PyLong_FromPid() on Windows 64-bit: processes are
+  identified by their HANDLE which is a pointer (and not a long, which is
+  smaller).
+
 - Tweak the exception message when the magic number or size value in a bytecode
   file is truncated.
 
diff --git a/PC/pyconfig.h b/PC/pyconfig.h
--- a/PC/pyconfig.h
+++ b/PC/pyconfig.h
@@ -723,6 +723,9 @@
 /* The size of `wchar_t', as computed by sizeof. */
 #define SIZEOF_WCHAR_T 2
 
+/* The size of `pid_t' (HANDLE). */
+#define SIZEOF_PID_T SIZEOF_VOID_P
+
 /* Define if you have the dl library (-ldl).  */
 /* #undef HAVE_LIBDL */
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 00:14:05 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 00:14:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issuse_=2317932=3A_Fix_an_?=
 =?utf-8?q?integer_overflow_issue_on_Windows_64-bit_in_tuple?=
Message-ID: <3bQ6qK2qnSzSTg@mail.python.org>

http://hg.python.org/cpython/rev/52075f60719e
changeset:   84027:52075f60719e
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:11:34 2013 +0200
summary:
  Issuse #17932: Fix an integer overflow issue on Windows 64-bit in tuple
iterators: change the C type of tupleiterobject.it_index from long to
Py_ssize_t.

files:
  Objects/tupleobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c
--- a/Objects/tupleobject.c
+++ b/Objects/tupleobject.c
@@ -930,7 +930,7 @@
 
 typedef struct {
     PyObject_HEAD
-    long it_index;
+    Py_ssize_t it_index;
     PyTupleObject *it_seq; /* Set to NULL when iterator is exhausted */
 } tupleiterobject;
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 00:14:06 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 00:14:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_a_com?=
 =?utf-8?q?piler_warning_on_Windows_64-bit_in_namespace=5Finit=28=29?=
Message-ID: <3bQ6qL52HfzSmY@mail.python.org>

http://hg.python.org/cpython/rev/93f4b32fc95c
changeset:   84028:93f4b32fc95c
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:13:51 2013 +0200
summary:
  Issue #9566: Fix a compiler warning on Windows 64-bit in namespace_init()

The result type is int, return -1 to avoid a compiler warning (cast Py_ssize_t
to int).  PyObject_Size() can only fail with -1, and anyway a constructor
should return -1 on error, not an arbitrary negative number.

files:
  Objects/namespaceobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/namespaceobject.c b/Objects/namespaceobject.c
--- a/Objects/namespaceobject.c
+++ b/Objects/namespaceobject.c
@@ -44,7 +44,7 @@
     if (args != NULL) {
         Py_ssize_t argcount = PyObject_Size(args);
         if (argcount < 0)
-            return argcount;
+            return -1;
         else if (argcount > 0) {
             PyErr_Format(PyExc_TypeError, "no positional arguments expected");
             return -1;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 00:25:15 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 00:25:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_compi?=
 =?utf-8?q?ler_warning_on_Windows_64-bit?=
Message-ID: <3bQ74C3LRjzQWm@mail.python.org>

http://hg.python.org/cpython/rev/5dcbd5d8d004
changeset:   84029:5dcbd5d8d004
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:21:31 2013 +0200
summary:
  Issue #9566: Fix compiler warning on Windows 64-bit

files:
  Objects/unicodeobject.c |  8 +++++---
  1 files changed, 5 insertions(+), 3 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -6725,7 +6725,8 @@
     /* each step cannot decode more than 1 character, but a character can be
        represented as a surrogate pair */
     wchar_t buffer[2], *startout, *out;
-    int insize, outsize;
+    int insize;
+    Py_ssize_t outsize;
     PyObject *errorHandler = NULL;
     PyObject *exc = NULL;
     PyObject *encoding_obj = NULL;
@@ -6995,10 +6996,11 @@
         Py_DECREF(substring);
         return -1;
     }
+    assert(size <= INT_MAX);
 
     /* First get the size of the result */
     outsize = WideCharToMultiByte(code_page, flags,
-                                  p, size,
+                                  p, (int)size,
                                   NULL, 0,
                                   NULL, pusedDefaultChar);
     if (outsize <= 0)
@@ -7035,7 +7037,7 @@
 
     /* Do the conversion */
     outsize = WideCharToMultiByte(code_page, flags,
-                                  p, size,
+                                  p, (int)size,
                                   out, outsize,
                                   NULL, pusedDefaultChar);
     Py_CLEAR(substring);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 00:25:16 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 00:25:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_compi?=
 =?utf-8?q?ler_warning_on_Windows_64-bit?=
Message-ID: <3bQ74D5QVNzQq2@mail.python.org>

http://hg.python.org/cpython/rev/41b8be55b160
changeset:   84030:41b8be55b160
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:22:34 2013 +0200
summary:
  Issue #9566: Fix compiler warning on Windows 64-bit

files:
  PC/getpathp.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/PC/getpathp.c b/PC/getpathp.c
--- a/PC/getpathp.c
+++ b/PC/getpathp.c
@@ -434,7 +434,7 @@
         char * p = fgets(buffer, MAXPATHLEN*2, env_file);
         wchar_t tmpbuffer[MAXPATHLEN*2+1];
         PyObject * decoded;
-        int n;
+        size_t n;
 
         if (p == NULL)
             break;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 00:36:05 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 00:36:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2313772=3A_Fix_comp?=
	=?utf-8?q?iler_warnings_on_Windows?=
Message-ID: <3bQ7Jj4RkFzRBB@mail.python.org>

http://hg.python.org/cpython/rev/f431cd0edd85
changeset:   84031:f431cd0edd85
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:35:54 2013 +0200
summary:
  Issue #13772: Fix compiler warnings on Windows

files:
  Modules/posixmodule.c |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -6765,7 +6765,7 @@
     *ptr = 0;
 }
 
-int _is_absW(WCHAR *path) {
+int _is_absW(const WCHAR *path) {
     /* Is this path absolute? */
 
     return path[0] == L'\\' || path[0] == L'/' || path[1] == L':';
@@ -6781,7 +6781,7 @@
 
 void _joinW(WCHAR *dest_path, const WCHAR *root, const WCHAR *rest) {
     /* join root and rest with a backslash */
-    int root_len;
+    size_t root_len;
 
     if(_is_absW(rest)) {
         wcscpy(dest_path, rest);
@@ -6800,7 +6800,7 @@
 
 void _joinA(char *dest_path, const char *root, const char *rest) {
     /* join root and rest with a backslash */
-    int root_len;
+    size_t root_len;
 
     if(_is_absA(rest)) {
         strcpy(dest_path, rest);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 00:37:26 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 00:37:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2313772=3A_fix_=5Fc?=
 =?utf-8?b?aGVja19kaXJBKCk6IGNhbGwgKkEoKSBmdW5jdGlvbnMsIG5vdCAqVygpIGZ1?=
 =?utf-8?q?nctions?=
Message-ID: <3bQ7LG3q4JzRBB@mail.python.org>

http://hg.python.org/cpython/rev/c351591f1f63
changeset:   84032:c351591f1f63
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:37:12 2013 +0200
summary:
  Issue #13772: fix _check_dirA(): call *A() functions, not *W() functions

files:
  Modules/posixmodule.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -6844,9 +6844,9 @@
 
     /* dest_parent = os.path.dirname(dest) */
     strcpy(dest_parent, dest);
-    _dirnameW(dest_parent);
+    _dirnameA(dest_parent);
     /* src_resolved = os.path.join(dest_parent, src) */
-    _joinW(src_resolved, dest_parent, src);
+    _joinA(src_resolved, dest_parent, src);
     return (
         GetFileAttributesExA(src_resolved, GetFileExInfoStandard, &src_info)
         && src_info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 01:18:32 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 01:18:32 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_compi?=
 =?utf-8?q?ler_warning_on_Windows_64-bit?=
Message-ID: <3bQ8Fh2Xbfz7Ljh@mail.python.org>

http://hg.python.org/cpython/rev/36c35a1893fe
changeset:   84033:36c35a1893fe
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:44:00 2013 +0200
summary:
  Issue #9566: Fix compiler warning on Windows 64-bit

files:
  Parser/tokenizer.c |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -660,7 +660,8 @@
 
 static char *
 translate_newlines(const char *s, int exec_input, struct tok_state *tok) {
-    int skip_next_lf = 0, needed_length = strlen(s) + 2, final_length;
+    int skip_next_lf = 0;
+    size_t needed_length = strlen(s) + 2, final_length;
     char *buf, *current;
     char c = '\0';
     buf = PyMem_MALLOC(needed_length);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 01:18:33 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 01:18:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_compi?=
 =?utf-8?q?ler_warning_on_Windows_64-bit?=
Message-ID: <3bQ8Fj4h0vz7Ljp@mail.python.org>

http://hg.python.org/cpython/rev/88a21c5a97ef
changeset:   84034:88a21c5a97ef
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 00:46:29 2013 +0200
summary:
  Issue #9566: Fix compiler warning on Windows 64-bit

files:
  Python/fileutils.c |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -707,7 +707,8 @@
 _Py_wgetcwd(wchar_t *buf, size_t size)
 {
 #ifdef MS_WINDOWS
-    return _wgetcwd(buf, size);
+    int isize = (int)Py_MIN(size, INT_MAX);
+    return _wgetcwd(buf, isize);
 #else
     char fname[PATH_MAX];
     wchar_t *wname;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 01:18:34 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 01:18:34 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_compi?=
 =?utf-8?q?later_warnings_on_Windows_64-bit?=
Message-ID: <3bQ8Fk6lXYz7LkR@mail.python.org>

http://hg.python.org/cpython/rev/aeebbae8c74c
changeset:   84035:aeebbae8c74c
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 01:18:13 2013 +0200
summary:
  Issue #9566: Fix compilater warnings on Windows 64-bit

files:
  Python/getargs.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Python/getargs.c b/Python/getargs.c
--- a/Python/getargs.c
+++ b/Python/getargs.c
@@ -570,7 +570,7 @@
                 "size does not fit in an int"); \
             return converterr("", arg, msgbuf, bufsize); \
         } \
-        *q=s; \
+        *q = (int)s; \
     }
 #define BUFFER_LEN      ((flags & FLAG_SIZE_T) ? *q2:*q)
 #define RETURN_ERR_OCCURRED return msgbuf

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 01:59:31 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 01:59:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2313772=3A_Fix_a_co?=
	=?utf-8?q?mpiler_warning_on_Windows?=
Message-ID: <3bQ98z0M51zQNg@mail.python.org>

http://hg.python.org/cpython/rev/e024236ea253
changeset:   84036:e024236ea253
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 01:30:25 2013 +0200
summary:
  Issue #13772: Fix a compiler warning on Windows

files:
  Modules/posixmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -6772,7 +6772,7 @@
 
 }
 
-int _is_absA(char *path) {
+int _is_absA(const char *path) {
     /* Is this path absolute? */
 
     return path[0] == '\\' || path[0] == '/' || path[1] == ':';

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 01:59:32 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 01:59:32 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2313772=3A_Mark_hel?=
 =?utf-8?q?per_functions_as_private_=28static=29?=
Message-ID: <3bQ9903l7Qz7Lk5@mail.python.org>

http://hg.python.org/cpython/rev/d9f3ea27f826
changeset:   84037:d9f3ea27f826
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 01:49:17 2013 +0200
summary:
  Issue #13772: Mark helper functions as private (static)

Cleanup also the code to follow the Python coding style (PEP 7).

files:
  Modules/posixmodule.c |  79 ++++++++++++++++--------------
  1 files changed, 43 insertions(+), 36 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -6720,8 +6720,9 @@
 /* Grab CreateSymbolicLinkW dynamically from kernel32 */
 static DWORD (CALLBACK *Py_CreateSymbolicLinkW)(LPWSTR, LPWSTR, DWORD) = NULL;
 static DWORD (CALLBACK *Py_CreateSymbolicLinkA)(LPSTR, LPSTR, DWORD) = NULL;
+
 static int
-check_CreateSymbolicLink()
+check_CreateSymbolicLink(void)
 {
     HINSTANCE hKernel32;
     /* only recheck */
@@ -6735,55 +6736,57 @@
     return (Py_CreateSymbolicLinkW && Py_CreateSymbolicLinkA);
 }
 
-void _dirnameW(WCHAR *path) {
-    /* Remove the last portion of the path */
-
+/* Remove the last portion of the path */
+static void
+_dirnameW(WCHAR *path)
+{
     WCHAR *ptr;
 
     /* walk the path from the end until a backslash is encountered */
-    for(ptr = path + wcslen(path); ptr != path; ptr--)
-    {
-        if(*ptr == *L"\\" || *ptr == *L"/") {
+    for(ptr = path + wcslen(path); ptr != path; ptr--) {
+        if (*ptr == *L"\\" || *ptr == *L"/")
             break;
-        }
     }
     *ptr = 0;
 }
 
-void _dirnameA(char *path) {
-    /* Remove the last portion of the path */
-
+/* Remove the last portion of the path */
+static void
+_dirnameA(char *path)
+{
     char *ptr;
 
     /* walk the path from the end until a backslash is encountered */
-    for(ptr = path + strlen(path); ptr != path; ptr--)
-    {
-        if(*ptr == '\\' || *ptr == '/') {
+    for(ptr = path + strlen(path); ptr != path; ptr--) {
+        if (*ptr == '\\' || *ptr == '/')
             break;
-        }
     }
     *ptr = 0;
 }
 
-int _is_absW(const WCHAR *path) {
-    /* Is this path absolute? */
-
+/* Is this path absolute? */
+static int
+_is_absW(const WCHAR *path)
+{
     return path[0] == L'\\' || path[0] == L'/' || path[1] == L':';
 
 }
 
-int _is_absA(const char *path) {
-    /* Is this path absolute? */
-
+/* Is this path absolute? */
+static int
+_is_absA(const char *path)
+{
     return path[0] == '\\' || path[0] == '/' || path[1] == ':';
 
 }
 
-void _joinW(WCHAR *dest_path, const WCHAR *root, const WCHAR *rest) {
-    /* join root and rest with a backslash */
+/* join root and rest with a backslash */
+static void
+_joinW(WCHAR *dest_path, const WCHAR *root, const WCHAR *rest)
+{
     size_t root_len;
 
-    if(_is_absW(rest)) {
+    if (_is_absW(rest)) {
         wcscpy(dest_path, rest);
         return;
     }
@@ -6792,17 +6795,19 @@
 
     wcscpy(dest_path, root);
     if(root_len) {
-        dest_path[root_len] = *L"\\";
-        root_len += 1;
+        dest_path[root_len] = L'\\';
+        root_len++;
     }
     wcscpy(dest_path+root_len, rest);
 }
 
-void _joinA(char *dest_path, const char *root, const char *rest) {
-    /* join root and rest with a backslash */
+/* join root and rest with a backslash */
+static void
+_joinA(char *dest_path, const char *root, const char *rest)
+{
     size_t root_len;
 
-    if(_is_absA(rest)) {
+    if (_is_absA(rest)) {
         strcpy(dest_path, rest);
         return;
     }
@@ -6812,14 +6817,15 @@
     strcpy(dest_path, root);
     if(root_len) {
         dest_path[root_len] = '\\';
-        root_len += 1;
+        root_len++;
     }
     strcpy(dest_path+root_len, rest);
 }
 
-int _check_dirW(WCHAR *src, WCHAR *dest)
-{
-    /* Return True if the path at src relative to dest is a directory */
+/* Return True if the path at src relative to dest is a directory */
+static int
+_check_dirW(WCHAR *src, WCHAR *dest)
+{
     WIN32_FILE_ATTRIBUTE_DATA src_info;
     WCHAR dest_parent[MAX_PATH];
     WCHAR src_resolved[MAX_PATH] = L"";
@@ -6835,9 +6841,10 @@
     );
 }
 
-int _check_dirA(char *src, char *dest)
-{
-    /* Return True if the path at src relative to dest is a directory */
+/* Return True if the path at src relative to dest is a directory */
+static int
+_check_dirA(char *src, char *dest)
+{
     WIN32_FILE_ATTRIBUTE_DATA src_info;
     char dest_parent[MAX_PATH];
     char src_resolved[MAX_PATH] = "";

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 02:08:02 2013
From: python-checkins at python.org (victor.stinner)
Date: Wed,  5 Jun 2013 02:08:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2313772=3A_Use_synt?=
 =?utf-8?q?ax_for_literal_wchar=5Ft_character?=
Message-ID: <3bQ9Lp24xmzSgY@mail.python.org>

http://hg.python.org/cpython/rev/c8212fca8747
changeset:   84038:c8212fca8747
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Wed Jun 05 02:07:46 2013 +0200
summary:
  Issue #13772: Use syntax for literal wchar_t character

files:
  Modules/posixmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -6744,7 +6744,7 @@
 
     /* walk the path from the end until a backslash is encountered */
     for(ptr = path + wcslen(path); ptr != path; ptr--) {
-        if (*ptr == *L"\\" || *ptr == *L"/")
+        if (*ptr == L'\\' || *ptr == L'/')
             break;
     }
     *ptr = 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 03:29:57 2013
From: python-checkins at python.org (guido.van.rossum)
Date: Wed,  5 Jun 2013 03:29:57 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Accept_PEP_443=2C_making_a_fe?=
 =?utf-8?q?w_minor_edits_to_improve_text_flow=2E?=
Message-ID: <3bQC9K4JXRzQQ2@mail.python.org>

http://hg.python.org/peps/rev/38826efc5fe4
changeset:   4921:38826efc5fe4
user:        Guido van Rossum <guido at python.org>
date:        Tue Jun 04 18:29:49 2013 -0700
summary:
  Accept PEP 443, making a few minor edits to improve text flow.

files:
  pep-0443.txt |  27 +++++++++++++++------------
  1 files changed, 15 insertions(+), 12 deletions(-)


diff --git a/pep-0443.txt b/pep-0443.txt
--- a/pep-0443.txt
+++ b/pep-0443.txt
@@ -4,7 +4,7 @@
 Last-Modified: $Date$
 Author: ?ukasz Langa <lukasz at langa.pl>
 Discussions-To: Python-Dev <python-dev at python.org>
-Status: Draft
+Status: Accepted
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 22-May-2013
@@ -44,11 +44,14 @@
 
 In addition, it is currently a common anti-pattern for Python code to
 inspect the types of received arguments, in order to decide what to do
-with the objects. For example, code may wish to accept either an object
+with the objects.
+
+For example, code may wish to accept either an object
 of some type, or a sequence of objects of that type.
+Currently, the "obvious way" to do this is by type inspection, but this
+is brittle and closed to extension.
 
-Currently, the "obvious way" to do this is by type inspection, but this
-is brittle and closed to extension. Abstract Base Classes make it easier
+Abstract Base Classes make it easier
 to discover present behaviour, but don't help adding new behaviour.
 A developer using an already-written library may be unable to change how
 their objects are treated by such code, especially if the objects they
@@ -63,7 +66,7 @@
 
 To define a generic function, decorate it with the ``@singledispatch``
 decorator. Note that the dispatch happens on the type of the first
-argument, create your function accordingly::
+argument. Create your function accordingly::
 
   >>> from functools import singledispatch
   >>> @singledispatch
@@ -73,7 +76,7 @@
   ...     print(arg)
 
 To add overloaded implementations to the function, use the
-``register()`` attribute of the generic function. It is a decorator,
+``register()`` attribute of the generic function. This is a decorator,
 taking a type parameter and decorating a function implementing the
 operation for that type::
 
@@ -98,7 +101,7 @@
   ...
   >>> fun.register(type(None), nothing)
 
-The ``register()`` attribute returns the undecorated function which
+The ``register()`` attribute returns the undecorated function. This
 enables decorator stacking, pickling, as well as creating unit tests for
 each variant independently::
 
@@ -172,12 +175,12 @@
 reference implementation is available on hg.python.org [#ref-impl]_.
 
 The dispatch type is specified as a decorator argument. An alternative
-form using function annotations has been considered but its inclusion
-has been deferred. As of May 2013, this usage pattern is out of scope
-for the standard library [#pep-0008]_ and the best practices for
+form using function annotations was considered but its inclusion
+has been rejected. As of May 2013, this usage pattern is out of scope
+for the standard library [#pep-0008]_, and the best practices for
 annotation usage are still debated.
 
-Based on the current ``pkgutil.simplegeneric`` implementation and
+Based on the current ``pkgutil.simplegeneric`` implementation, and
 following the convention on registering virtual subclasses on Abstract
 Base Classes, the dispatch registry will not be thread-safe.
 
@@ -278,7 +281,7 @@
 
 This PEP proposes extending behaviour only of functions specifically
 marked as generic. Just as a base class method may be overridden by
-a subclass, so too may a function be overloaded to provide custom
+a subclass, so too a function may be overloaded to provide custom
 functionality for a given type.
 
 Universal overloading does not equal *arbitrary* overloading, in the

-- 
Repository URL: http://hg.python.org/peps

From solipsis at pitrou.net  Wed Jun  5 05:51:46 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 05 Jun 2013 05:51:46 +0200
Subject: [Python-checkins] Daily reference leaks (c8212fca8747): sum=0
Message-ID: <E1Uk4lG-0003PK-EA@ap.vmr.nerim.net>

results for c8212fca8747 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogD2huOK', '-x']

From python-checkins at python.org  Wed Jun  5 12:20:49 2013
From: python-checkins at python.org (lukasz.langa)
Date: Wed,  5 Jun 2013 12:20:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_reference_implementati?=
	=?utf-8?q?on_for_PEP_443?=
Message-ID: <3bQQxs6YX7zRbZ@mail.python.org>

http://hg.python.org/cpython/rev/dfcb64f51f7b
changeset:   84039:dfcb64f51f7b
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Wed Jun 05 12:20:24 2013 +0200
summary:
  Add reference implementation for PEP 443

PEP accepted: http://mail.python.org/pipermail/python-dev/2013-June/126734.html

files:
  Doc/library/functools.rst  |  110 +++++++
  Lib/functools.py           |  128 ++++++++-
  Lib/pkgutil.py             |   52 +---
  Lib/test/test_functools.py |  374 ++++++++++++++++++++++++-
  Misc/NEWS                  |    3 +
  Modules/Setup.dist         |    2 +-
  6 files changed, 614 insertions(+), 55 deletions(-)


diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst
--- a/Doc/library/functools.rst
+++ b/Doc/library/functools.rst
@@ -6,6 +6,7 @@
 .. moduleauthor:: Peter Harris <scav at blueyonder.co.uk>
 .. moduleauthor:: Raymond Hettinger <python at rcn.com>
 .. moduleauthor:: Nick Coghlan <ncoghlan at gmail.com>
+.. moduleauthor:: ?ukasz Langa <lukasz at langa.pl>
 .. sectionauthor:: Peter Harris <scav at blueyonder.co.uk>
 
 **Source code:** :source:`Lib/functools.py`
@@ -186,6 +187,115 @@
    *sequence* contains only one item, the first item is returned.
 
 
+.. decorator:: singledispatch(default)
+
+   Transforms a function into a single-dispatch generic function.  A **generic
+   function** is composed of multiple functions implementing the same operation
+   for different types. Which implementation should be used during a call is
+   determined by the dispatch algorithm.  When the implementation is chosen
+   based on the type of a single argument, this is known as **single
+   dispatch**.
+
+   To define a generic function, decorate it with the ``@singledispatch``
+   decorator. Note that the dispatch happens on the type of the first argument,
+   create your function accordingly::
+
+     >>> from functools import singledispatch
+     >>> @singledispatch
+     ... def fun(arg, verbose=False):
+     ...     if verbose:
+     ...         print("Let me just say,", end=" ")
+     ...     print(arg)
+
+   To add overloaded implementations to the function, use the :func:`register`
+   attribute of the generic function.  It is a decorator, taking a type
+   parameter and decorating a function implementing the operation for that
+   type::
+
+     >>> @fun.register(int)
+     ... def _(arg, verbose=False):
+     ...     if verbose:
+     ...         print("Strength in numbers, eh?", end=" ")
+     ...     print(arg)
+     ...
+     >>> @fun.register(list)
+     ... def _(arg, verbose=False):
+     ...     if verbose:
+     ...         print("Enumerate this:")
+     ...     for i, elem in enumerate(arg):
+     ...         print(i, elem)
+
+   To enable registering lambdas and pre-existing functions, the
+   :func:`register` attribute can be used in a functional form::
+
+     >>> def nothing(arg, verbose=False):
+     ...     print("Nothing.")
+     ...
+     >>> fun.register(type(None), nothing)
+
+   The :func:`register` attribute returns the undecorated function which
+   enables decorator stacking, pickling, as well as creating unit tests for
+   each variant independently::
+
+     >>> @fun.register(float)
+     ... @fun.register(Decimal)
+     ... def fun_num(arg, verbose=False):
+     ...     if verbose:
+     ...         print("Half of your number:", end=" ")
+     ...     print(arg / 2)
+     ...
+     >>> fun_num is fun
+     False
+
+   When called, the generic function dispatches on the type of the first
+   argument::
+
+     >>> fun("Hello, world.")
+     Hello, world.
+     >>> fun("test.", verbose=True)
+     Let me just say, test.
+     >>> fun(42, verbose=True)
+     Strength in numbers, eh? 42
+     >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
+     Enumerate this:
+     0 spam
+     1 spam
+     2 eggs
+     3 spam
+     >>> fun(None)
+     Nothing.
+     >>> fun(1.23)
+     0.615
+
+   Where there is no registered implementation for a specific type, its
+   method resolution order is used to find a more generic implementation.
+   The original function decorated with ``@singledispatch`` is registered
+   for the base ``object`` type, which means it is used if no better
+   implementation is found.
+
+   To check which implementation will the generic function choose for
+   a given type, use the ``dispatch()`` attribute::
+
+     >>> fun.dispatch(float)
+     <function fun_num at 0x1035a2840>
+     >>> fun.dispatch(dict)    # note: default implementation
+     <function fun at 0x103fe0000>
+
+   To access all registered implementations, use the read-only ``registry``
+   attribute::
+
+    >>> fun.registry.keys()
+    dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,
+              <class 'decimal.Decimal'>, <class 'list'>,
+              <class 'float'>])
+    >>> fun.registry[float]
+    <function fun_num at 0x1035a2840>
+    >>> fun.registry[object]
+    <function fun at 0x103fe0000>
+
+   .. versionadded:: 3.4
+
+
 .. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
 
    Update a *wrapper* function to look like the *wrapped* function. The optional
diff --git a/Lib/functools.py b/Lib/functools.py
--- a/Lib/functools.py
+++ b/Lib/functools.py
@@ -3,19 +3,24 @@
 # Python module wrapper for _functools C module
 # to allow utilities written in Python to be added
 # to the functools module.
-# Written by Nick Coghlan <ncoghlan at gmail.com>
-# and Raymond Hettinger <python at rcn.com>
-#   Copyright (C) 2006-2010 Python Software Foundation.
+# Written by Nick Coghlan <ncoghlan at gmail.com>,
+# Raymond Hettinger <python at rcn.com>,
+# and ?ukasz Langa <lukasz at langa.pl>.
+#   Copyright (C) 2006-2013 Python Software Foundation.
 # See C source code for _functools credits/copyright
 
 __all__ = ['update_wrapper', 'wraps', 'WRAPPER_ASSIGNMENTS', 'WRAPPER_UPDATES',
-           'total_ordering', 'cmp_to_key', 'lru_cache', 'reduce', 'partial']
+           'total_ordering', 'cmp_to_key', 'lru_cache', 'reduce', 'partial',
+           'singledispatch']
 
 try:
     from _functools import reduce
 except ImportError:
     pass
+from abc import get_cache_token
 from collections import namedtuple
+from types import MappingProxyType
+from weakref import WeakKeyDictionary
 try:
     from _thread import RLock
 except:
@@ -354,3 +359,118 @@
         return update_wrapper(wrapper, user_function)
 
     return decorating_function
+
+
+################################################################################
+### singledispatch() - single-dispatch generic function decorator
+################################################################################
+
+def _compose_mro(cls, haystack):
+    """Calculates the MRO for a given class `cls`, including relevant abstract
+    base classes from `haystack`.
+
+    """
+    bases = set(cls.__mro__)
+    mro = list(cls.__mro__)
+    for needle in haystack:
+        if (needle in bases or not hasattr(needle, '__mro__')
+                            or not issubclass(cls, needle)):
+            continue   # either present in the __mro__ already or unrelated
+        for index, base in enumerate(mro):
+            if not issubclass(base, needle):
+                break
+        if base in bases and not issubclass(needle, base):
+            # Conflict resolution: put classes present in __mro__ and their
+            # subclasses first. See test_mro_conflicts() in test_functools.py
+            # for examples.
+            index += 1
+        mro.insert(index, needle)
+    return mro
+
+def _find_impl(cls, registry):
+    """Returns the best matching implementation for the given class `cls` in
+    `registry`. Where there is no registered implementation for a specific
+    type, its method resolution order is used to find a more generic
+    implementation.
+
+    Note: if `registry` does not contain an implementation for the base
+    `object` type, this function may return None.
+
+    """
+    mro = _compose_mro(cls, registry.keys())
+    match = None
+    for t in mro:
+        if match is not None:
+            # If `match` is an ABC but there is another unrelated, equally
+            # matching ABC. Refuse the temptation to guess.
+            if (t in registry and not issubclass(match, t)
+                              and match not in cls.__mro__):
+                raise RuntimeError("Ambiguous dispatch: {} or {}".format(
+                    match, t))
+            break
+        if t in registry:
+            match = t
+    return registry.get(match)
+
+def singledispatch(func):
+    """Single-dispatch generic function decorator.
+
+    Transforms a function into a generic function, which can have different
+    behaviours depending upon the type of its first argument. The decorated
+    function acts as the default implementation, and additional
+    implementations can be registered using the 'register()' attribute of
+    the generic function.
+
+    """
+    registry = {}
+    dispatch_cache = WeakKeyDictionary()
+    cache_token = None
+
+    def dispatch(typ):
+        """generic_func.dispatch(type) -> <function implementation>
+
+        Runs the dispatch algorithm to return the best available implementation
+        for the given `type` registered on `generic_func`.
+
+        """
+        nonlocal cache_token
+        if cache_token is not None:
+            current_token = get_cache_token()
+            if cache_token != current_token:
+                dispatch_cache.clear()
+                cache_token = current_token
+        try:
+            impl = dispatch_cache[typ]
+        except KeyError:
+            try:
+                impl = registry[typ]
+            except KeyError:
+                impl = _find_impl(typ, registry)
+            dispatch_cache[typ] = impl
+        return impl
+
+    def register(typ, func=None):
+        """generic_func.register(type, func) -> func
+
+        Registers a new implementation for the given `type` on a `generic_func`.
+
+        """
+        nonlocal cache_token
+        if func is None:
+            return lambda f: register(typ, f)
+        registry[typ] = func
+        if cache_token is None and hasattr(typ, '__abstractmethods__'):
+            cache_token = get_cache_token()
+        dispatch_cache.clear()
+        return func
+
+    def wrapper(*args, **kw):
+        return dispatch(args[0].__class__)(*args, **kw)
+
+    registry[object] = func
+    wrapper.register = register
+    wrapper.dispatch = dispatch
+    wrapper.registry = MappingProxyType(registry)
+    wrapper._clear_cache = dispatch_cache.clear
+    update_wrapper(wrapper, func)
+    return wrapper
diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py
--- a/Lib/pkgutil.py
+++ b/Lib/pkgutil.py
@@ -1,12 +1,13 @@
 """Utilities to support packages."""
 
+from functools import singledispatch as simplegeneric
+import imp
+import importlib
 import os
+import os.path
 import sys
-import importlib
-import imp
-import os.path
+from types import ModuleType
 from warnings import warn
-from types import ModuleType
 
 __all__ = [
     'get_importer', 'iter_importers', 'get_loader', 'find_loader',
@@ -27,46 +28,6 @@
     return marshal.load(stream)
 
 
-def simplegeneric(func):
-    """Make a trivial single-dispatch generic function"""
-    registry = {}
-    def wrapper(*args, **kw):
-        ob = args[0]
-        try:
-            cls = ob.__class__
-        except AttributeError:
-            cls = type(ob)
-        try:
-            mro = cls.__mro__
-        except AttributeError:
-            try:
-                class cls(cls, object):
-                    pass
-                mro = cls.__mro__[1:]
-            except TypeError:
-                mro = object,   # must be an ExtensionClass or some such  :(
-        for t in mro:
-            if t in registry:
-                return registry[t](*args, **kw)
-        else:
-            return func(*args, **kw)
-    try:
-        wrapper.__name__ = func.__name__
-    except (TypeError, AttributeError):
-        pass    # Python 2.3 doesn't allow functions to be renamed
-
-    def register(typ, func=None):
-        if func is None:
-            return lambda f: register(typ, f)
-        registry[typ] = func
-        return func
-
-    wrapper.__dict__ = func.__dict__
-    wrapper.__doc__ = func.__doc__
-    wrapper.register = register
-    return wrapper
-
-
 def walk_packages(path=None, prefix='', onerror=None):
     """Yields (module_loader, name, ispkg) for all modules recursively
     on path, or, if path is None, all accessible modules.
@@ -148,13 +109,12 @@
                 yield i, name, ispkg
 
 
-#@simplegeneric
+ at simplegeneric
 def iter_importer_modules(importer, prefix=''):
     if not hasattr(importer, 'iter_modules'):
         return []
     return importer.iter_modules(prefix)
 
-iter_importer_modules = simplegeneric(iter_importer_modules)
 
 # Implement a file walker for the normal importlib path hook
 def _iter_file_finder_modules(importer, prefix=''):
diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -1,24 +1,30 @@
 import collections
-import sys
-import unittest
-from test import support
-from weakref import proxy
+from itertools import permutations
 import pickle
 from random import choice
+import sys
+from test import support
+import unittest
+from weakref import proxy
 
 import functools
 
 py_functools = support.import_fresh_module('functools', blocked=['_functools'])
 c_functools = support.import_fresh_module('functools', fresh=['_functools'])
 
+decimal = support.import_fresh_module('decimal', fresh=['_decimal'])
+
+
 def capture(*args, **kw):
     """capture all positional and keyword arguments"""
     return args, kw
 
+
 def signature(part):
     """ return the signature of a partial object """
     return (part.func, part.args, part.keywords, part.__dict__)
 
+
 class TestPartial:
 
     def test_basic_examples(self):
@@ -138,6 +144,7 @@
         join = self.partial(''.join)
         self.assertEqual(join(data), '0123456789')
 
+
 @unittest.skipUnless(c_functools, 'requires the C _functools module')
 class TestPartialC(TestPartial, unittest.TestCase):
     if c_functools:
@@ -194,18 +201,22 @@
                 "new style getargs format but argument is not a tuple",
                 f.__setstate__, BadSequence())
 
+
 class TestPartialPy(TestPartial, unittest.TestCase):
     partial = staticmethod(py_functools.partial)
 
+
 if c_functools:
     class PartialSubclass(c_functools.partial):
         pass
 
+
 @unittest.skipUnless(c_functools, 'requires the C _functools module')
 class TestPartialCSubclass(TestPartialC):
     if c_functools:
         partial = PartialSubclass
 
+
 class TestUpdateWrapper(unittest.TestCase):
 
     def check_wrapper(self, wrapper, wrapped,
@@ -312,6 +323,7 @@
         self.assertTrue(wrapper.__doc__.startswith('max('))
         self.assertEqual(wrapper.__annotations__, {})
 
+
 class TestWraps(TestUpdateWrapper):
 
     def _default_update(self):
@@ -372,6 +384,7 @@
         self.assertEqual(wrapper.attr, 'This is a different test')
         self.assertEqual(wrapper.dict_attr, f.dict_attr)
 
+
 class TestReduce(unittest.TestCase):
     func = functools.reduce
 
@@ -452,6 +465,7 @@
         d = {"one": 1, "two": 2, "three": 3}
         self.assertEqual(self.func(add, d), "".join(d.keys()))
 
+
 class TestCmpToKey:
 
     def test_cmp_to_key(self):
@@ -534,14 +548,17 @@
         self.assertRaises(TypeError, hash, k)
         self.assertNotIsInstance(k, collections.Hashable)
 
+
 @unittest.skipUnless(c_functools, 'requires the C _functools module')
 class TestCmpToKeyC(TestCmpToKey, unittest.TestCase):
     if c_functools:
         cmp_to_key = c_functools.cmp_to_key
 
+
 class TestCmpToKeyPy(TestCmpToKey, unittest.TestCase):
     cmp_to_key = staticmethod(py_functools.cmp_to_key)
 
+
 class TestTotalOrdering(unittest.TestCase):
 
     def test_total_ordering_lt(self):
@@ -642,6 +659,7 @@
         with self.assertRaises(TypeError):
             TestTO(8) <= ()
 
+
 class TestLRU(unittest.TestCase):
 
     def test_lru(self):
@@ -834,6 +852,353 @@
                          DoubleEq(2))               # Verify the correct return value
 
 
+class TestSingleDispatch(unittest.TestCase):
+    def test_simple_overloads(self):
+        @functools.singledispatch
+        def g(obj):
+            return "base"
+        def g_int(i):
+            return "integer"
+        g.register(int, g_int)
+        self.assertEqual(g("str"), "base")
+        self.assertEqual(g(1), "integer")
+        self.assertEqual(g([1,2,3]), "base")
+
+    def test_mro(self):
+        @functools.singledispatch
+        def g(obj):
+            return "base"
+        class C:
+            pass
+        class D(C):
+            pass
+        def g_C(c):
+            return "C"
+        g.register(C, g_C)
+        self.assertEqual(g(C()), "C")
+        self.assertEqual(g(D()), "C")
+
+    def test_classic_classes(self):
+        @functools.singledispatch
+        def g(obj):
+            return "base"
+        class C:
+            pass
+        class D(C):
+            pass
+        def g_C(c):
+            return "C"
+        g.register(C, g_C)
+        self.assertEqual(g(C()), "C")
+        self.assertEqual(g(D()), "C")
+
+    def test_register_decorator(self):
+        @functools.singledispatch
+        def g(obj):
+            return "base"
+        @g.register(int)
+        def g_int(i):
+            return "int %s" % (i,)
+        self.assertEqual(g(""), "base")
+        self.assertEqual(g(12), "int 12")
+        self.assertIs(g.dispatch(int), g_int)
+        self.assertIs(g.dispatch(object), g.dispatch(str))
+        # Note: in the assert above this is not g.
+        # @singledispatch returns the wrapper.
+
+    def test_wrapping_attributes(self):
+        @functools.singledispatch
+        def g(obj):
+            "Simple test"
+            return "Test"
+        self.assertEqual(g.__name__, "g")
+        self.assertEqual(g.__doc__, "Simple test")
+
+    @unittest.skipUnless(decimal, 'requires _decimal')
+    @support.cpython_only
+    def test_c_classes(self):
+        @functools.singledispatch
+        def g(obj):
+            return "base"
+        @g.register(decimal.DecimalException)
+        def _(obj):
+            return obj.args
+        subn = decimal.Subnormal("Exponent < Emin")
+        rnd = decimal.Rounded("Number got rounded")
+        self.assertEqual(g(subn), ("Exponent < Emin",))
+        self.assertEqual(g(rnd), ("Number got rounded",))
+        @g.register(decimal.Subnormal)
+        def _(obj):
+            return "Too small to care."
+        self.assertEqual(g(subn), "Too small to care.")
+        self.assertEqual(g(rnd), ("Number got rounded",))
+
+    def test_compose_mro(self):
+        c = collections
+        mro = functools._compose_mro
+        bases = [c.Sequence, c.MutableMapping, c.Mapping, c.Set]
+        for haystack in permutations(bases):
+            m = mro(dict, haystack)
+            self.assertEqual(m, [dict, c.MutableMapping, c.Mapping, object])
+        bases = [c.Container, c.Mapping, c.MutableMapping, c.OrderedDict]
+        for haystack in permutations(bases):
+            m = mro(c.ChainMap, haystack)
+            self.assertEqual(m, [c.ChainMap, c.MutableMapping, c.Mapping,
+                                 c.Sized, c.Iterable, c.Container, object])
+        # Note: The MRO order below depends on haystack ordering.
+        m = mro(c.defaultdict, [c.Sized, c.Container, str])
+        self.assertEqual(m, [c.defaultdict, dict, c.Container, c.Sized, object])
+        m = mro(c.defaultdict, [c.Container, c.Sized, str])
+        self.assertEqual(m, [c.defaultdict, dict, c.Sized, c.Container, object])
+
+    def test_register_abc(self):
+        c = collections
+        d = {"a": "b"}
+        l = [1, 2, 3]
+        s = {object(), None}
+        f = frozenset(s)
+        t = (1, 2, 3)
+        @functools.singledispatch
+        def g(obj):
+            return "base"
+        self.assertEqual(g(d), "base")
+        self.assertEqual(g(l), "base")
+        self.assertEqual(g(s), "base")
+        self.assertEqual(g(f), "base")
+        self.assertEqual(g(t), "base")
+        g.register(c.Sized, lambda obj: "sized")
+        self.assertEqual(g(d), "sized")
+        self.assertEqual(g(l), "sized")
+        self.assertEqual(g(s), "sized")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sized")
+        g.register(c.MutableMapping, lambda obj: "mutablemapping")
+        self.assertEqual(g(d), "mutablemapping")
+        self.assertEqual(g(l), "sized")
+        self.assertEqual(g(s), "sized")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sized")
+        g.register(c.ChainMap, lambda obj: "chainmap")
+        self.assertEqual(g(d), "mutablemapping")  # irrelevant ABCs registered
+        self.assertEqual(g(l), "sized")
+        self.assertEqual(g(s), "sized")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sized")
+        g.register(c.MutableSequence, lambda obj: "mutablesequence")
+        self.assertEqual(g(d), "mutablemapping")
+        self.assertEqual(g(l), "mutablesequence")
+        self.assertEqual(g(s), "sized")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sized")
+        g.register(c.MutableSet, lambda obj: "mutableset")
+        self.assertEqual(g(d), "mutablemapping")
+        self.assertEqual(g(l), "mutablesequence")
+        self.assertEqual(g(s), "mutableset")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sized")
+        g.register(c.Mapping, lambda obj: "mapping")
+        self.assertEqual(g(d), "mutablemapping")  # not specific enough
+        self.assertEqual(g(l), "mutablesequence")
+        self.assertEqual(g(s), "mutableset")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sized")
+        g.register(c.Sequence, lambda obj: "sequence")
+        self.assertEqual(g(d), "mutablemapping")
+        self.assertEqual(g(l), "mutablesequence")
+        self.assertEqual(g(s), "mutableset")
+        self.assertEqual(g(f), "sized")
+        self.assertEqual(g(t), "sequence")
+        g.register(c.Set, lambda obj: "set")
+        self.assertEqual(g(d), "mutablemapping")
+        self.assertEqual(g(l), "mutablesequence")
+        self.assertEqual(g(s), "mutableset")
+        self.assertEqual(g(f), "set")
+        self.assertEqual(g(t), "sequence")
+        g.register(dict, lambda obj: "dict")
+        self.assertEqual(g(d), "dict")
+        self.assertEqual(g(l), "mutablesequence")
+        self.assertEqual(g(s), "mutableset")
+        self.assertEqual(g(f), "set")
+        self.assertEqual(g(t), "sequence")
+        g.register(list, lambda obj: "list")
+        self.assertEqual(g(d), "dict")
+        self.assertEqual(g(l), "list")
+        self.assertEqual(g(s), "mutableset")
+        self.assertEqual(g(f), "set")
+        self.assertEqual(g(t), "sequence")
+        g.register(set, lambda obj: "concrete-set")
+        self.assertEqual(g(d), "dict")
+        self.assertEqual(g(l), "list")
+        self.assertEqual(g(s), "concrete-set")
+        self.assertEqual(g(f), "set")
+        self.assertEqual(g(t), "sequence")
+        g.register(frozenset, lambda obj: "frozen-set")
+        self.assertEqual(g(d), "dict")
+        self.assertEqual(g(l), "list")
+        self.assertEqual(g(s), "concrete-set")
+        self.assertEqual(g(f), "frozen-set")
+        self.assertEqual(g(t), "sequence")
+        g.register(tuple, lambda obj: "tuple")
+        self.assertEqual(g(d), "dict")
+        self.assertEqual(g(l), "list")
+        self.assertEqual(g(s), "concrete-set")
+        self.assertEqual(g(f), "frozen-set")
+        self.assertEqual(g(t), "tuple")
+
+    def test_mro_conflicts(self):
+        c = collections
+
+        @functools.singledispatch
+        def g(arg):
+            return "base"
+
+        class O(c.Sized):
+            def __len__(self):
+                return 0
+
+        o = O()
+        self.assertEqual(g(o), "base")
+        g.register(c.Iterable, lambda arg: "iterable")
+        g.register(c.Container, lambda arg: "container")
+        g.register(c.Sized, lambda arg: "sized")
+        g.register(c.Set, lambda arg: "set")
+        self.assertEqual(g(o), "sized")
+        c.Iterable.register(O)
+        self.assertEqual(g(o), "sized")   # because it's explicitly in __mro__
+        c.Container.register(O)
+        self.assertEqual(g(o), "sized")   # see above: Sized is in __mro__
+
+        class P:
+            pass
+
+        p = P()
+        self.assertEqual(g(p), "base")
+        c.Iterable.register(P)
+        self.assertEqual(g(p), "iterable")
+        c.Container.register(P)
+        with self.assertRaises(RuntimeError) as re:
+            g(p)
+            self.assertEqual(
+                str(re),
+                ("Ambiguous dispatch: <class 'collections.abc.Container'> "
+                    "or <class 'collections.abc.Iterable'>"),
+            )
+
+        class Q(c.Sized):
+            def __len__(self):
+                return 0
+
+        q = Q()
+        self.assertEqual(g(q), "sized")
+        c.Iterable.register(Q)
+        self.assertEqual(g(q), "sized")   # because it's explicitly in __mro__
+        c.Set.register(Q)
+        self.assertEqual(g(q), "set")     # because c.Set is a subclass of
+                                          # c.Sized which is explicitly in
+                                          # __mro__
+
+    def test_cache_invalidation(self):
+        from collections import UserDict
+        class TracingDict(UserDict):
+            def __init__(self, *args, **kwargs):
+                super(TracingDict, self).__init__(*args, **kwargs)
+                self.set_ops = []
+                self.get_ops = []
+            def __getitem__(self, key):
+                result = self.data[key]
+                self.get_ops.append(key)
+                return result
+            def __setitem__(self, key, value):
+                self.set_ops.append(key)
+                self.data[key] = value
+            def clear(self):
+                self.data.clear()
+        _orig_wkd = functools.WeakKeyDictionary
+        td = TracingDict()
+        functools.WeakKeyDictionary = lambda: td
+        c = collections
+        @functools.singledispatch
+        def g(arg):
+            return "base"
+        d = {}
+        l = []
+        self.assertEqual(len(td), 0)
+        self.assertEqual(g(d), "base")
+        self.assertEqual(len(td), 1)
+        self.assertEqual(td.get_ops, [])
+        self.assertEqual(td.set_ops, [dict])
+        self.assertEqual(td.data[dict], g.registry[object])
+        self.assertEqual(g(l), "base")
+        self.assertEqual(len(td), 2)
+        self.assertEqual(td.get_ops, [])
+        self.assertEqual(td.set_ops, [dict, list])
+        self.assertEqual(td.data[dict], g.registry[object])
+        self.assertEqual(td.data[list], g.registry[object])
+        self.assertEqual(td.data[dict], td.data[list])
+        self.assertEqual(g(l), "base")
+        self.assertEqual(g(d), "base")
+        self.assertEqual(td.get_ops, [list, dict])
+        self.assertEqual(td.set_ops, [dict, list])
+        g.register(list, lambda arg: "list")
+        self.assertEqual(td.get_ops, [list, dict])
+        self.assertEqual(len(td), 0)
+        self.assertEqual(g(d), "base")
+        self.assertEqual(len(td), 1)
+        self.assertEqual(td.get_ops, [list, dict])
+        self.assertEqual(td.set_ops, [dict, list, dict])
+        self.assertEqual(td.data[dict],
+                         functools._find_impl(dict, g.registry))
+        self.assertEqual(g(l), "list")
+        self.assertEqual(len(td), 2)
+        self.assertEqual(td.get_ops, [list, dict])
+        self.assertEqual(td.set_ops, [dict, list, dict, list])
+        self.assertEqual(td.data[list],
+                         functools._find_impl(list, g.registry))
+        class X:
+            pass
+        c.MutableMapping.register(X)   # Will not invalidate the cache,
+                                       # not using ABCs yet.
+        self.assertEqual(g(d), "base")
+        self.assertEqual(g(l), "list")
+        self.assertEqual(td.get_ops, [list, dict, dict, list])
+        self.assertEqual(td.set_ops, [dict, list, dict, list])
+        g.register(c.Sized, lambda arg: "sized")
+        self.assertEqual(len(td), 0)
+        self.assertEqual(g(d), "sized")
+        self.assertEqual(len(td), 1)
+        self.assertEqual(td.get_ops, [list, dict, dict, list])
+        self.assertEqual(td.set_ops, [dict, list, dict, list, dict])
+        self.assertEqual(g(l), "list")
+        self.assertEqual(len(td), 2)
+        self.assertEqual(td.get_ops, [list, dict, dict, list])
+        self.assertEqual(td.set_ops, [dict, list, dict, list, dict, list])
+        self.assertEqual(g(l), "list")
+        self.assertEqual(g(d), "sized")
+        self.assertEqual(td.get_ops, [list, dict, dict, list, list, dict])
+        self.assertEqual(td.set_ops, [dict, list, dict, list, dict, list])
+        g.dispatch(list)
+        g.dispatch(dict)
+        self.assertEqual(td.get_ops, [list, dict, dict, list, list, dict,
+                                      list, dict])
+        self.assertEqual(td.set_ops, [dict, list, dict, list, dict, list])
+        c.MutableSet.register(X)       # Will invalidate the cache.
+        self.assertEqual(len(td), 2)   # Stale cache.
+        self.assertEqual(g(l), "list")
+        self.assertEqual(len(td), 1)
+        g.register(c.MutableMapping, lambda arg: "mutablemapping")
+        self.assertEqual(len(td), 0)
+        self.assertEqual(g(d), "mutablemapping")
+        self.assertEqual(len(td), 1)
+        self.assertEqual(g(l), "list")
+        self.assertEqual(len(td), 2)
+        g.register(dict, lambda arg: "dict")
+        self.assertEqual(g(d), "dict")
+        self.assertEqual(g(l), "list")
+        g._clear_cache()
+        self.assertEqual(len(td), 0)
+        functools.WeakKeyDictionary = _orig_wkd
+
+
 def test_main(verbose=None):
     test_classes = (
         TestPartialC,
@@ -846,6 +1211,7 @@
         TestWraps,
         TestReduce,
         TestLRU,
+        TestSingleDispatch,
     )
     support.run_unittest(*test_classes)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -344,6 +344,9 @@
   the default for linking if LDSHARED is not also overriden.  This restores
   Distutils behavior introduced in 3.2.3 and inadvertently dropped in 3.3.0.
 
+- Implement PEP 443 "Single-dispatch generic functions".
+
+
 Tests
 -----
 
diff --git a/Modules/Setup.dist b/Modules/Setup.dist
--- a/Modules/Setup.dist
+++ b/Modules/Setup.dist
@@ -116,6 +116,7 @@
 _operator _operator.c	        # operator.add() and similar goodies
 _collections _collectionsmodule.c # Container types
 itertools itertoolsmodule.c    # Functions creating iterators for efficient looping 
+atexit atexitmodule.c      # Register functions to be run at interpreter-shutdown
 
 # access to ISO C locale support
 _locale _localemodule.c  # -lintl
@@ -170,7 +171,6 @@
 #_weakref _weakref.c	# basic weak reference support
 #_testcapi _testcapimodule.c    # Python C API test module
 #_random _randommodule.c	# Random number generator
-#atexit atexitmodule.c      # Register functions to be run at interpreter-shutdown
 #_elementtree -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI _elementtree.c	# elementtree accelerator
 #_pickle _pickle.c	# pickle accelerator
 #_datetime _datetimemodule.c	# datetime accelerator

-- 
Repository URL: http://hg.python.org/cpython

From brett at python.org  Wed Jun  5 16:31:04 2013
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Jun 2013 10:31:04 -0400
Subject: [Python-checkins] cpython: Add reference implementation for PEP
	443
In-Reply-To: <3bQQxs6YX7zRbZ@mail.python.org>
References: <3bQQxs6YX7zRbZ@mail.python.org>
Message-ID: <CAP1=2W5NNM0V-98FMaXKErdLwtvU-6_mJcKWJ_jm2vnb3BYBrA@mail.gmail.com>

Any chance you could move your definitions for "generic function" and
"single dispatch" to the glossary and just link to them here?


On Wed, Jun 5, 2013 at 6:20 AM, lukasz.langa <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/dfcb64f51f7b
> changeset:   84039:dfcb64f51f7b
> user:        ?ukasz Langa <lukasz at langa.pl>
> date:        Wed Jun 05 12:20:24 2013 +0200
> summary:
>   Add reference implementation for PEP 443
>
> PEP accepted:
> http://mail.python.org/pipermail/python-dev/2013-June/126734.html
>
> files:
>   Doc/library/functools.rst  |  110 +++++++
>   Lib/functools.py           |  128 ++++++++-
>   Lib/pkgutil.py             |   52 +---
>   Lib/test/test_functools.py |  374 ++++++++++++++++++++++++-
>   Misc/NEWS                  |    3 +
>   Modules/Setup.dist         |    2 +-
>   6 files changed, 614 insertions(+), 55 deletions(-)
>
>
> diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst
> --- a/Doc/library/functools.rst
> +++ b/Doc/library/functools.rst
> @@ -6,6 +6,7 @@
>  .. moduleauthor:: Peter Harris <scav at blueyonder.co.uk>
>  .. moduleauthor:: Raymond Hettinger <python at rcn.com>
>  .. moduleauthor:: Nick Coghlan <ncoghlan at gmail.com>
> +.. moduleauthor:: ?ukasz Langa <lukasz at langa.pl>
>  .. sectionauthor:: Peter Harris <scav at blueyonder.co.uk>
>
>  **Source code:** :source:`Lib/functools.py`
> @@ -186,6 +187,115 @@
>     *sequence* contains only one item, the first item is returned.
>
>
> +.. decorator:: singledispatch(default)
> +
> +   Transforms a function into a single-dispatch generic function.  A
> **generic
> +   function** is composed of multiple functions implementing the same
> operation
> +   for different types. Which implementation should be used during a call
> is
> +   determined by the dispatch algorithm.  When the implementation is
> chosen
> +   based on the type of a single argument, this is known as **single
> +   dispatch**.
> +
> +   To define a generic function, decorate it with the ``@singledispatch``
> +   decorator. Note that the dispatch happens on the type of the first
> argument,
> +   create your function accordingly::
> +
> +     >>> from functools import singledispatch
> +     >>> @singledispatch
> +     ... def fun(arg, verbose=False):
> +     ...     if verbose:
> +     ...         print("Let me just say,", end=" ")
> +     ...     print(arg)
> +
> +   To add overloaded implementations to the function, use the
> :func:`register`
> +   attribute of the generic function.  It is a decorator, taking a type
> +   parameter and decorating a function implementing the operation for that
> +   type::
> +
> +     >>> @fun.register(int)
> +     ... def _(arg, verbose=False):
> +     ...     if verbose:
> +     ...         print("Strength in numbers, eh?", end=" ")
> +     ...     print(arg)
> +     ...
> +     >>> @fun.register(list)
> +     ... def _(arg, verbose=False):
> +     ...     if verbose:
> +     ...         print("Enumerate this:")
> +     ...     for i, elem in enumerate(arg):
> +     ...         print(i, elem)
> +
> +   To enable registering lambdas and pre-existing functions, the
> +   :func:`register` attribute can be used in a functional form::
> +
> +     >>> def nothing(arg, verbose=False):
> +     ...     print("Nothing.")
> +     ...
> +     >>> fun.register(type(None), nothing)
> +
> +   The :func:`register` attribute returns the undecorated function which
> +   enables decorator stacking, pickling, as well as creating unit tests
> for
> +   each variant independently::
> +
> +     >>> @fun.register(float)
> +     ... @fun.register(Decimal)
> +     ... def fun_num(arg, verbose=False):
> +     ...     if verbose:
> +     ...         print("Half of your number:", end=" ")
> +     ...     print(arg / 2)
> +     ...
> +     >>> fun_num is fun
> +     False
> +
> +   When called, the generic function dispatches on the type of the first
> +   argument::
> +
> +     >>> fun("Hello, world.")
> +     Hello, world.
> +     >>> fun("test.", verbose=True)
> +     Let me just say, test.
> +     >>> fun(42, verbose=True)
> +     Strength in numbers, eh? 42
> +     >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
> +     Enumerate this:
> +     0 spam
> +     1 spam
> +     2 eggs
> +     3 spam
> +     >>> fun(None)
> +     Nothing.
> +     >>> fun(1.23)
> +     0.615
> +
> +   Where there is no registered implementation for a specific type, its
> +   method resolution order is used to find a more generic implementation.
> +   The original function decorated with ``@singledispatch`` is registered
> +   for the base ``object`` type, which means it is used if no better
> +   implementation is found.
> +
> +   To check which implementation will the generic function choose for
> +   a given type, use the ``dispatch()`` attribute::
> +
> +     >>> fun.dispatch(float)
> +     <function fun_num at 0x1035a2840>
> +     >>> fun.dispatch(dict)    # note: default implementation
> +     <function fun at 0x103fe0000>
> +
> +   To access all registered implementations, use the read-only
> ``registry``
> +   attribute::
> +
> +    >>> fun.registry.keys()
> +    dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,
> +              <class 'decimal.Decimal'>, <class 'list'>,
> +              <class 'float'>])
> +    >>> fun.registry[float]
> +    <function fun_num at 0x1035a2840>
> +    >>> fun.registry[object]
> +    <function fun at 0x103fe0000>
> +
> +   .. versionadded:: 3.4
> +
> +
>  .. function:: update_wrapper(wrapper, wrapped,
> assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
>
>     Update a *wrapper* function to look like the *wrapped* function. The
> optional
> diff --git a/Lib/functools.py b/Lib/functools.py
> --- a/Lib/functools.py
> +++ b/Lib/functools.py
> @@ -3,19 +3,24 @@
>  # Python module wrapper for _functools C module
>  # to allow utilities written in Python to be added
>  # to the functools module.
> -# Written by Nick Coghlan <ncoghlan at gmail.com>
> -# and Raymond Hettinger <python at rcn.com>
> -#   Copyright (C) 2006-2010 Python Software Foundation.
> +# Written by Nick Coghlan <ncoghlan at gmail.com>,
> +# Raymond Hettinger <python at rcn.com>,
> +# and ?ukasz Langa <lukasz at langa.pl>.
> +#   Copyright (C) 2006-2013 Python Software Foundation.
>  # See C source code for _functools credits/copyright
>
>  __all__ = ['update_wrapper', 'wraps', 'WRAPPER_ASSIGNMENTS',
> 'WRAPPER_UPDATES',
> -           'total_ordering', 'cmp_to_key', 'lru_cache', 'reduce',
> 'partial']
> +           'total_ordering', 'cmp_to_key', 'lru_cache', 'reduce',
> 'partial',
> +           'singledispatch']
>
>  try:
>      from _functools import reduce
>  except ImportError:
>      pass
> +from abc import get_cache_token
>  from collections import namedtuple
> +from types import MappingProxyType
> +from weakref import WeakKeyDictionary
>  try:
>      from _thread import RLock
>  except:
> @@ -354,3 +359,118 @@
>          return update_wrapper(wrapper, user_function)
>
>      return decorating_function
> +
> +
>
> +################################################################################
> +### singledispatch() - single-dispatch generic function decorator
>
> +################################################################################
> +
> +def _compose_mro(cls, haystack):
> +    """Calculates the MRO for a given class `cls`, including relevant
> abstract
> +    base classes from `haystack`.
> +
> +    """
> +    bases = set(cls.__mro__)
> +    mro = list(cls.__mro__)
> +    for needle in haystack:
> +        if (needle in bases or not hasattr(needle, '__mro__')
> +                            or not issubclass(cls, needle)):
> +            continue   # either present in the __mro__ already or
> unrelated
> +        for index, base in enumerate(mro):
> +            if not issubclass(base, needle):
> +                break
> +        if base in bases and not issubclass(needle, base):
> +            # Conflict resolution: put classes present in __mro__ and
> their
> +            # subclasses first. See test_mro_conflicts() in
> test_functools.py
> +            # for examples.
> +            index += 1
> +        mro.insert(index, needle)
> +    return mro
> +
> +def _find_impl(cls, registry):
> +    """Returns the best matching implementation for the given class `cls`
> in
> +    `registry`. Where there is no registered implementation for a specific
> +    type, its method resolution order is used to find a more generic
> +    implementation.
> +
> +    Note: if `registry` does not contain an implementation for the base
> +    `object` type, this function may return None.
> +
> +    """
> +    mro = _compose_mro(cls, registry.keys())
> +    match = None
> +    for t in mro:
> +        if match is not None:
> +            # If `match` is an ABC but there is another unrelated, equally
> +            # matching ABC. Refuse the temptation to guess.
> +            if (t in registry and not issubclass(match, t)
> +                              and match not in cls.__mro__):
> +                raise RuntimeError("Ambiguous dispatch: {} or {}".format(
> +                    match, t))
> +            break
> +        if t in registry:
> +            match = t
> +    return registry.get(match)
> +
> +def singledispatch(func):
> +    """Single-dispatch generic function decorator.
> +
> +    Transforms a function into a generic function, which can have
> different
> +    behaviours depending upon the type of its first argument. The
> decorated
> +    function acts as the default implementation, and additional
> +    implementations can be registered using the 'register()' attribute of
> +    the generic function.
> +
> +    """
> +    registry = {}
> +    dispatch_cache = WeakKeyDictionary()
> +    cache_token = None
> +
> +    def dispatch(typ):
> +        """generic_func.dispatch(type) -> <function implementation>
> +
> +        Runs the dispatch algorithm to return the best available
> implementation
> +        for the given `type` registered on `generic_func`.
> +
> +        """
> +        nonlocal cache_token
> +        if cache_token is not None:
> +            current_token = get_cache_token()
> +            if cache_token != current_token:
> +                dispatch_cache.clear()
> +                cache_token = current_token
> +        try:
> +            impl = dispatch_cache[typ]
> +        except KeyError:
> +            try:
> +                impl = registry[typ]
> +            except KeyError:
> +                impl = _find_impl(typ, registry)
> +            dispatch_cache[typ] = impl
> +        return impl
> +
> +    def register(typ, func=None):
> +        """generic_func.register(type, func) -> func
> +
> +        Registers a new implementation for the given `type` on a
> `generic_func`.
> +
> +        """
> +        nonlocal cache_token
> +        if func is None:
> +            return lambda f: register(typ, f)
> +        registry[typ] = func
> +        if cache_token is None and hasattr(typ, '__abstractmethods__'):
> +            cache_token = get_cache_token()
> +        dispatch_cache.clear()
> +        return func
> +
> +    def wrapper(*args, **kw):
> +        return dispatch(args[0].__class__)(*args, **kw)
> +
> +    registry[object] = func
> +    wrapper.register = register
> +    wrapper.dispatch = dispatch
> +    wrapper.registry = MappingProxyType(registry)
> +    wrapper._clear_cache = dispatch_cache.clear
> +    update_wrapper(wrapper, func)
> +    return wrapper
> diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py
> --- a/Lib/pkgutil.py
> +++ b/Lib/pkgutil.py
> @@ -1,12 +1,13 @@
>  """Utilities to support packages."""
>
> +from functools import singledispatch as simplegeneric
> +import imp
> +import importlib
>  import os
> +import os.path
>  import sys
> -import importlib
> -import imp
> -import os.path
> +from types import ModuleType
>  from warnings import warn
> -from types import ModuleType
>
>  __all__ = [
>      'get_importer', 'iter_importers', 'get_loader', 'find_loader',
> @@ -27,46 +28,6 @@
>      return marshal.load(stream)
>
>
> -def simplegeneric(func):
> -    """Make a trivial single-dispatch generic function"""
> -    registry = {}
> -    def wrapper(*args, **kw):
> -        ob = args[0]
> -        try:
> -            cls = ob.__class__
> -        except AttributeError:
> -            cls = type(ob)
> -        try:
> -            mro = cls.__mro__
> -        except AttributeError:
> -            try:
> -                class cls(cls, object):
> -                    pass
> -                mro = cls.__mro__[1:]
> -            except TypeError:
> -                mro = object,   # must be an ExtensionClass or some such
>  :(
> -        for t in mro:
> -            if t in registry:
> -                return registry[t](*args, **kw)
> -        else:
> -            return func(*args, **kw)
> -    try:
> -        wrapper.__name__ = func.__name__
> -    except (TypeError, AttributeError):
> -        pass    # Python 2.3 doesn't allow functions to be renamed
> -
> -    def register(typ, func=None):
> -        if func is None:
> -            return lambda f: register(typ, f)
> -        registry[typ] = func
> -        return func
> -
> -    wrapper.__dict__ = func.__dict__
> -    wrapper.__doc__ = func.__doc__
> -    wrapper.register = register
> -    return wrapper
> -
> -
>  def walk_packages(path=None, prefix='', onerror=None):
>      """Yields (module_loader, name, ispkg) for all modules recursively
>      on path, or, if path is None, all accessible modules.
> @@ -148,13 +109,12 @@
>                  yield i, name, ispkg
>
>
> -#@simplegeneric
> + at simplegeneric
>  def iter_importer_modules(importer, prefix=''):
>      if not hasattr(importer, 'iter_modules'):
>          return []
>      return importer.iter_modules(prefix)
>
> -iter_importer_modules = simplegeneric(iter_importer_modules)
>
>  # Implement a file walker for the normal importlib path hook
>  def _iter_file_finder_modules(importer, prefix=''):
> diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
> --- a/Lib/test/test_functools.py
> +++ b/Lib/test/test_functools.py
> @@ -1,24 +1,30 @@
>  import collections
> -import sys
> -import unittest
> -from test import support
> -from weakref import proxy
> +from itertools import permutations
>  import pickle
>  from random import choice
> +import sys
> +from test import support
> +import unittest
> +from weakref import proxy
>
>  import functools
>
>  py_functools = support.import_fresh_module('functools',
> blocked=['_functools'])
>  c_functools = support.import_fresh_module('functools',
> fresh=['_functools'])
>
> +decimal = support.import_fresh_module('decimal', fresh=['_decimal'])
> +
> +
>  def capture(*args, **kw):
>      """capture all positional and keyword arguments"""
>      return args, kw
>
> +
>  def signature(part):
>      """ return the signature of a partial object """
>      return (part.func, part.args, part.keywords, part.__dict__)
>
> +
>  class TestPartial:
>
>      def test_basic_examples(self):
> @@ -138,6 +144,7 @@
>          join = self.partial(''.join)
>          self.assertEqual(join(data), '0123456789')
>
> +
>  @unittest.skipUnless(c_functools, 'requires the C _functools module')
>  class TestPartialC(TestPartial, unittest.TestCase):
>      if c_functools:
> @@ -194,18 +201,22 @@
>                  "new style getargs format but argument is not a tuple",
>                  f.__setstate__, BadSequence())
>
> +
>  class TestPartialPy(TestPartial, unittest.TestCase):
>      partial = staticmethod(py_functools.partial)
>
> +
>  if c_functools:
>      class PartialSubclass(c_functools.partial):
>          pass
>
> +
>  @unittest.skipUnless(c_functools, 'requires the C _functools module')
>  class TestPartialCSubclass(TestPartialC):
>      if c_functools:
>          partial = PartialSubclass
>
> +
>  class TestUpdateWrapper(unittest.TestCase):
>
>      def check_wrapper(self, wrapper, wrapped,
> @@ -312,6 +323,7 @@
>          self.assertTrue(wrapper.__doc__.startswith('max('))
>          self.assertEqual(wrapper.__annotations__, {})
>
> +
>  class TestWraps(TestUpdateWrapper):
>
>      def _default_update(self):
> @@ -372,6 +384,7 @@
>          self.assertEqual(wrapper.attr, 'This is a different test')
>          self.assertEqual(wrapper.dict_attr, f.dict_attr)
>
> +
>  class TestReduce(unittest.TestCase):
>      func = functools.reduce
>
> @@ -452,6 +465,7 @@
>          d = {"one": 1, "two": 2, "three": 3}
>          self.assertEqual(self.func(add, d), "".join(d.keys()))
>
> +
>  class TestCmpToKey:
>
>      def test_cmp_to_key(self):
> @@ -534,14 +548,17 @@
>          self.assertRaises(TypeError, hash, k)
>          self.assertNotIsInstance(k, collections.Hashable)
>
> +
>  @unittest.skipUnless(c_functools, 'requires the C _functools module')
>  class TestCmpToKeyC(TestCmpToKey, unittest.TestCase):
>      if c_functools:
>          cmp_to_key = c_functools.cmp_to_key
>
> +
>  class TestCmpToKeyPy(TestCmpToKey, unittest.TestCase):
>      cmp_to_key = staticmethod(py_functools.cmp_to_key)
>
> +
>  class TestTotalOrdering(unittest.TestCase):
>
>      def test_total_ordering_lt(self):
> @@ -642,6 +659,7 @@
>          with self.assertRaises(TypeError):
>              TestTO(8) <= ()
>
> +
>  class TestLRU(unittest.TestCase):
>
>      def test_lru(self):
> @@ -834,6 +852,353 @@
>                           DoubleEq(2))               # Verify the correct
> return value
>
>
> +class TestSingleDispatch(unittest.TestCase):
> +    def test_simple_overloads(self):
> +        @functools.singledispatch
> +        def g(obj):
> +            return "base"
> +        def g_int(i):
> +            return "integer"
> +        g.register(int, g_int)
> +        self.assertEqual(g("str"), "base")
> +        self.assertEqual(g(1), "integer")
> +        self.assertEqual(g([1,2,3]), "base")
> +
> +    def test_mro(self):
> +        @functools.singledispatch
> +        def g(obj):
> +            return "base"
> +        class C:
> +            pass
> +        class D(C):
> +            pass
> +        def g_C(c):
> +            return "C"
> +        g.register(C, g_C)
> +        self.assertEqual(g(C()), "C")
> +        self.assertEqual(g(D()), "C")
> +
> +    def test_classic_classes(self):
> +        @functools.singledispatch
> +        def g(obj):
> +            return "base"
> +        class C:
> +            pass
> +        class D(C):
> +            pass
> +        def g_C(c):
> +            return "C"
> +        g.register(C, g_C)
> +        self.assertEqual(g(C()), "C")
> +        self.assertEqual(g(D()), "C")
> +
> +    def test_register_decorator(self):
> +        @functools.singledispatch
> +        def g(obj):
> +            return "base"
> +        @g.register(int)
> +        def g_int(i):
> +            return "int %s" % (i,)
> +        self.assertEqual(g(""), "base")
> +        self.assertEqual(g(12), "int 12")
> +        self.assertIs(g.dispatch(int), g_int)
> +        self.assertIs(g.dispatch(object), g.dispatch(str))
> +        # Note: in the assert above this is not g.
> +        # @singledispatch returns the wrapper.
> +
> +    def test_wrapping_attributes(self):
> +        @functools.singledispatch
> +        def g(obj):
> +            "Simple test"
> +            return "Test"
> +        self.assertEqual(g.__name__, "g")
> +        self.assertEqual(g.__doc__, "Simple test")
> +
> +    @unittest.skipUnless(decimal, 'requires _decimal')
> +    @support.cpython_only
> +    def test_c_classes(self):
> +        @functools.singledispatch
> +        def g(obj):
> +            return "base"
> +        @g.register(decimal.DecimalException)
> +        def _(obj):
> +            return obj.args
> +        subn = decimal.Subnormal("Exponent < Emin")
> +        rnd = decimal.Rounded("Number got rounded")
> +        self.assertEqual(g(subn), ("Exponent < Emin",))
> +        self.assertEqual(g(rnd), ("Number got rounded",))
> +        @g.register(decimal.Subnormal)
> +        def _(obj):
> +            return "Too small to care."
> +        self.assertEqual(g(subn), "Too small to care.")
> +        self.assertEqual(g(rnd), ("Number got rounded",))
> +
> +    def test_compose_mro(self):
> +        c = collections
> +        mro = functools._compose_mro
> +        bases = [c.Sequence, c.MutableMapping, c.Mapping, c.Set]
> +        for haystack in permutations(bases):
> +            m = mro(dict, haystack)
> +            self.assertEqual(m, [dict, c.MutableMapping, c.Mapping,
> object])
> +        bases = [c.Container, c.Mapping, c.MutableMapping, c.OrderedDict]
> +        for haystack in permutations(bases):
> +            m = mro(c.ChainMap, haystack)
> +            self.assertEqual(m, [c.ChainMap, c.MutableMapping, c.Mapping,
> +                                 c.Sized, c.Iterable, c.Container,
> object])
> +        # Note: The MRO order below depends on haystack ordering.
> +        m = mro(c.defaultdict, [c.Sized, c.Container, str])
> +        self.assertEqual(m, [c.defaultdict, dict, c.Container, c.Sized,
> object])
> +        m = mro(c.defaultdict, [c.Container, c.Sized, str])
> +        self.assertEqual(m, [c.defaultdict, dict, c.Sized, c.Container,
> object])
> +
> +    def test_register_abc(self):
> +        c = collections
> +        d = {"a": "b"}
> +        l = [1, 2, 3]
> +        s = {object(), None}
> +        f = frozenset(s)
> +        t = (1, 2, 3)
> +        @functools.singledispatch
> +        def g(obj):
> +            return "base"
> +        self.assertEqual(g(d), "base")
> +        self.assertEqual(g(l), "base")
> +        self.assertEqual(g(s), "base")
> +        self.assertEqual(g(f), "base")
> +        self.assertEqual(g(t), "base")
> +        g.register(c.Sized, lambda obj: "sized")
> +        self.assertEqual(g(d), "sized")
> +        self.assertEqual(g(l), "sized")
> +        self.assertEqual(g(s), "sized")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sized")
> +        g.register(c.MutableMapping, lambda obj: "mutablemapping")
> +        self.assertEqual(g(d), "mutablemapping")
> +        self.assertEqual(g(l), "sized")
> +        self.assertEqual(g(s), "sized")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sized")
> +        g.register(c.ChainMap, lambda obj: "chainmap")
> +        self.assertEqual(g(d), "mutablemapping")  # irrelevant ABCs
> registered
> +        self.assertEqual(g(l), "sized")
> +        self.assertEqual(g(s), "sized")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sized")
> +        g.register(c.MutableSequence, lambda obj: "mutablesequence")
> +        self.assertEqual(g(d), "mutablemapping")
> +        self.assertEqual(g(l), "mutablesequence")
> +        self.assertEqual(g(s), "sized")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sized")
> +        g.register(c.MutableSet, lambda obj: "mutableset")
> +        self.assertEqual(g(d), "mutablemapping")
> +        self.assertEqual(g(l), "mutablesequence")
> +        self.assertEqual(g(s), "mutableset")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sized")
> +        g.register(c.Mapping, lambda obj: "mapping")
> +        self.assertEqual(g(d), "mutablemapping")  # not specific enough
> +        self.assertEqual(g(l), "mutablesequence")
> +        self.assertEqual(g(s), "mutableset")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sized")
> +        g.register(c.Sequence, lambda obj: "sequence")
> +        self.assertEqual(g(d), "mutablemapping")
> +        self.assertEqual(g(l), "mutablesequence")
> +        self.assertEqual(g(s), "mutableset")
> +        self.assertEqual(g(f), "sized")
> +        self.assertEqual(g(t), "sequence")
> +        g.register(c.Set, lambda obj: "set")
> +        self.assertEqual(g(d), "mutablemapping")
> +        self.assertEqual(g(l), "mutablesequence")
> +        self.assertEqual(g(s), "mutableset")
> +        self.assertEqual(g(f), "set")
> +        self.assertEqual(g(t), "sequence")
> +        g.register(dict, lambda obj: "dict")
> +        self.assertEqual(g(d), "dict")
> +        self.assertEqual(g(l), "mutablesequence")
> +        self.assertEqual(g(s), "mutableset")
> +        self.assertEqual(g(f), "set")
> +        self.assertEqual(g(t), "sequence")
> +        g.register(list, lambda obj: "list")
> +        self.assertEqual(g(d), "dict")
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(g(s), "mutableset")
> +        self.assertEqual(g(f), "set")
> +        self.assertEqual(g(t), "sequence")
> +        g.register(set, lambda obj: "concrete-set")
> +        self.assertEqual(g(d), "dict")
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(g(s), "concrete-set")
> +        self.assertEqual(g(f), "set")
> +        self.assertEqual(g(t), "sequence")
> +        g.register(frozenset, lambda obj: "frozen-set")
> +        self.assertEqual(g(d), "dict")
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(g(s), "concrete-set")
> +        self.assertEqual(g(f), "frozen-set")
> +        self.assertEqual(g(t), "sequence")
> +        g.register(tuple, lambda obj: "tuple")
> +        self.assertEqual(g(d), "dict")
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(g(s), "concrete-set")
> +        self.assertEqual(g(f), "frozen-set")
> +        self.assertEqual(g(t), "tuple")
> +
> +    def test_mro_conflicts(self):
> +        c = collections
> +
> +        @functools.singledispatch
> +        def g(arg):
> +            return "base"
> +
> +        class O(c.Sized):
> +            def __len__(self):
> +                return 0
> +
> +        o = O()
> +        self.assertEqual(g(o), "base")
> +        g.register(c.Iterable, lambda arg: "iterable")
> +        g.register(c.Container, lambda arg: "container")
> +        g.register(c.Sized, lambda arg: "sized")
> +        g.register(c.Set, lambda arg: "set")
> +        self.assertEqual(g(o), "sized")
> +        c.Iterable.register(O)
> +        self.assertEqual(g(o), "sized")   # because it's explicitly in
> __mro__
> +        c.Container.register(O)
> +        self.assertEqual(g(o), "sized")   # see above: Sized is in __mro__
> +
> +        class P:
> +            pass
> +
> +        p = P()
> +        self.assertEqual(g(p), "base")
> +        c.Iterable.register(P)
> +        self.assertEqual(g(p), "iterable")
> +        c.Container.register(P)
> +        with self.assertRaises(RuntimeError) as re:
> +            g(p)
> +            self.assertEqual(
> +                str(re),
> +                ("Ambiguous dispatch: <class 'collections.abc.Container'>
> "
> +                    "or <class 'collections.abc.Iterable'>"),
> +            )
> +
> +        class Q(c.Sized):
> +            def __len__(self):
> +                return 0
> +
> +        q = Q()
> +        self.assertEqual(g(q), "sized")
> +        c.Iterable.register(Q)
> +        self.assertEqual(g(q), "sized")   # because it's explicitly in
> __mro__
> +        c.Set.register(Q)
> +        self.assertEqual(g(q), "set")     # because c.Set is a subclass of
> +                                          # c.Sized which is explicitly in
> +                                          # __mro__
> +
> +    def test_cache_invalidation(self):
> +        from collections import UserDict
> +        class TracingDict(UserDict):
> +            def __init__(self, *args, **kwargs):
> +                super(TracingDict, self).__init__(*args, **kwargs)
> +                self.set_ops = []
> +                self.get_ops = []
> +            def __getitem__(self, key):
> +                result = self.data[key]
> +                self.get_ops.append(key)
> +                return result
> +            def __setitem__(self, key, value):
> +                self.set_ops.append(key)
> +                self.data[key] = value
> +            def clear(self):
> +                self.data.clear()
> +        _orig_wkd = functools.WeakKeyDictionary
> +        td = TracingDict()
> +        functools.WeakKeyDictionary = lambda: td
> +        c = collections
> +        @functools.singledispatch
> +        def g(arg):
> +            return "base"
> +        d = {}
> +        l = []
> +        self.assertEqual(len(td), 0)
> +        self.assertEqual(g(d), "base")
> +        self.assertEqual(len(td), 1)
> +        self.assertEqual(td.get_ops, [])
> +        self.assertEqual(td.set_ops, [dict])
> +        self.assertEqual(td.data[dict], g.registry[object])
> +        self.assertEqual(g(l), "base")
> +        self.assertEqual(len(td), 2)
> +        self.assertEqual(td.get_ops, [])
> +        self.assertEqual(td.set_ops, [dict, list])
> +        self.assertEqual(td.data[dict], g.registry[object])
> +        self.assertEqual(td.data[list], g.registry[object])
> +        self.assertEqual(td.data[dict], td.data[list])
> +        self.assertEqual(g(l), "base")
> +        self.assertEqual(g(d), "base")
> +        self.assertEqual(td.get_ops, [list, dict])
> +        self.assertEqual(td.set_ops, [dict, list])
> +        g.register(list, lambda arg: "list")
> +        self.assertEqual(td.get_ops, [list, dict])
> +        self.assertEqual(len(td), 0)
> +        self.assertEqual(g(d), "base")
> +        self.assertEqual(len(td), 1)
> +        self.assertEqual(td.get_ops, [list, dict])
> +        self.assertEqual(td.set_ops, [dict, list, dict])
> +        self.assertEqual(td.data[dict],
> +                         functools._find_impl(dict, g.registry))
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(len(td), 2)
> +        self.assertEqual(td.get_ops, [list, dict])
> +        self.assertEqual(td.set_ops, [dict, list, dict, list])
> +        self.assertEqual(td.data[list],
> +                         functools._find_impl(list, g.registry))
> +        class X:
> +            pass
> +        c.MutableMapping.register(X)   # Will not invalidate the cache,
> +                                       # not using ABCs yet.
> +        self.assertEqual(g(d), "base")
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(td.get_ops, [list, dict, dict, list])
> +        self.assertEqual(td.set_ops, [dict, list, dict, list])
> +        g.register(c.Sized, lambda arg: "sized")
> +        self.assertEqual(len(td), 0)
> +        self.assertEqual(g(d), "sized")
> +        self.assertEqual(len(td), 1)
> +        self.assertEqual(td.get_ops, [list, dict, dict, list])
> +        self.assertEqual(td.set_ops, [dict, list, dict, list, dict])
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(len(td), 2)
> +        self.assertEqual(td.get_ops, [list, dict, dict, list])
> +        self.assertEqual(td.set_ops, [dict, list, dict, list, dict, list])
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(g(d), "sized")
> +        self.assertEqual(td.get_ops, [list, dict, dict, list, list, dict])
> +        self.assertEqual(td.set_ops, [dict, list, dict, list, dict, list])
> +        g.dispatch(list)
> +        g.dispatch(dict)
> +        self.assertEqual(td.get_ops, [list, dict, dict, list, list, dict,
> +                                      list, dict])
> +        self.assertEqual(td.set_ops, [dict, list, dict, list, dict, list])
> +        c.MutableSet.register(X)       # Will invalidate the cache.
> +        self.assertEqual(len(td), 2)   # Stale cache.
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(len(td), 1)
> +        g.register(c.MutableMapping, lambda arg: "mutablemapping")
> +        self.assertEqual(len(td), 0)
> +        self.assertEqual(g(d), "mutablemapping")
> +        self.assertEqual(len(td), 1)
> +        self.assertEqual(g(l), "list")
> +        self.assertEqual(len(td), 2)
> +        g.register(dict, lambda arg: "dict")
> +        self.assertEqual(g(d), "dict")
> +        self.assertEqual(g(l), "list")
> +        g._clear_cache()
> +        self.assertEqual(len(td), 0)
> +        functools.WeakKeyDictionary = _orig_wkd
> +
> +
>  def test_main(verbose=None):
>      test_classes = (
>          TestPartialC,
> @@ -846,6 +1211,7 @@
>          TestWraps,
>          TestReduce,
>          TestLRU,
> +        TestSingleDispatch,
>      )
>      support.run_unittest(*test_classes)
>
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -344,6 +344,9 @@
>    the default for linking if LDSHARED is not also overriden.  This
> restores
>    Distutils behavior introduced in 3.2.3 and inadvertently dropped in
> 3.3.0.
>
> +- Implement PEP 443 "Single-dispatch generic functions".
> +
> +
>  Tests
>  -----
>
> diff --git a/Modules/Setup.dist b/Modules/Setup.dist
> --- a/Modules/Setup.dist
> +++ b/Modules/Setup.dist
> @@ -116,6 +116,7 @@
>  _operator _operator.c          # operator.add() and similar goodies
>  _collections _collectionsmodule.c # Container types
>  itertools itertoolsmodule.c    # Functions creating iterators for
> efficient looping
> +atexit atexitmodule.c      # Register functions to be run at
> interpreter-shutdown
>
>  # access to ISO C locale support
>  _locale _localemodule.c  # -lintl
> @@ -170,7 +171,6 @@
>  #_weakref _weakref.c   # basic weak reference support
>  #_testcapi _testcapimodule.c    # Python C API test module
>  #_random _randommodule.c       # Random number generator
> -#atexit atexitmodule.c      # Register functions to be run at
> interpreter-shutdown
>  #_elementtree -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H
> -DUSE_PYEXPAT_CAPI _elementtree.c        # elementtree accelerator
>  #_pickle _pickle.c     # pickle accelerator
>  #_datetime _datetimemodule.c   # datetime accelerator
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130605/a0c2b1ac/attachment-0001.html>

From lukasz at langa.pl  Wed Jun  5 17:52:56 2013
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Wed, 5 Jun 2013 17:52:56 +0200
Subject: [Python-checkins] [Python-Dev] cpython: Add reference
	implementation for PEP 443
In-Reply-To: <CAP1=2W5NNM0V-98FMaXKErdLwtvU-6_mJcKWJ_jm2vnb3BYBrA@mail.gmail.com>
References: <3bQQxs6YX7zRbZ@mail.python.org>
	<CAP1=2W5NNM0V-98FMaXKErdLwtvU-6_mJcKWJ_jm2vnb3BYBrA@mail.gmail.com>
Message-ID: <0F939F6D-63D1-4D93-9EA7-39C7C09E0F6A@langa.pl>

Dnia 5 cze 2013 o godz. 16:31 Brett Cannon <brett at python.org> napisa?(a):

> Any chance you could move your definitions for "generic function" and "single dispatch" to the glossary and just link to them here?

Sure thing.

-- 
?

From python-checkins at python.org  Wed Jun  5 20:41:11 2013
From: python-checkins at python.org (terry.reedy)
Date: Wed,  5 Jun 2013 20:41:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUxODEzMDog?=
 =?utf-8?q?Test_class_idlelib=2EconfigSectionNameDialog=2EGetCfgSectionNam?=
 =?utf-8?q?eDialog=2E?=
Message-ID: <3bQf3C3gWJz7LkX@mail.python.org>

http://hg.python.org/cpython/rev/db4ecaf852e3
changeset:   84040:db4ecaf852e3
branch:      3.3
parent:      84019:ffdee6b36305
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Wed Jun 05 14:22:26 2013 -0400
summary:
  Issue18130: Test class idlelib.configSectionNameDialog.GetCfgSectionNameDialog.
Fix bug in existing human test and add instructions; fix two bugs in tested
code; remove redundancies, add spaces, and change two internal method names.
Add mock_tk with mocks for tkinter.Variable subclasses and tkinter.messagebox.
Use mocks in test_config_name to unittest methods that are otherwise gui-free.

files:
  Lib/idlelib/configSectionNameDialog.py    |  111 +++++----
  Lib/idlelib/idle_test/mock_tk.py          |   63 +++++
  Lib/idlelib/idle_test/test_config_name.py |   75 ++++++
  3 files changed, 198 insertions(+), 51 deletions(-)


diff --git a/Lib/idlelib/configSectionNameDialog.py b/Lib/idlelib/configSectionNameDialog.py
--- a/Lib/idlelib/configSectionNameDialog.py
+++ b/Lib/idlelib/configSectionNameDialog.py
@@ -1,97 +1,106 @@
 """
 Dialog that allows user to specify a new config file section name.
 Used to get new highlight theme and keybinding set names.
+The 'return value' for the dialog, used two placed in configDialog.py,
+is the .result attribute set in the Ok and Cancel methods.
 """
 from tkinter import *
 import tkinter.messagebox as tkMessageBox
 
 class GetCfgSectionNameDialog(Toplevel):
-    def __init__(self,parent,title,message,usedNames):
+    def __init__(self, parent, title, message, used_names):
         """
         message - string, informational message to display
-        usedNames - list, list of names already in use for validity check
+        used_names - string collection, names already in use for validity check
         """
         Toplevel.__init__(self, parent)
         self.configure(borderwidth=5)
-        self.resizable(height=FALSE,width=FALSE)
+        self.resizable(height=FALSE, width=FALSE)
         self.title(title)
         self.transient(parent)
         self.grab_set()
         self.protocol("WM_DELETE_WINDOW", self.Cancel)
         self.parent = parent
-        self.message=message
-        self.usedNames=usedNames
-        self.result=''
-        self.CreateWidgets()
-        self.withdraw() #hide while setting geometry
+        self.message = message
+        self.used_names = used_names
+        self.create_widgets()
+        self.withdraw()  #hide while setting geometry
         self.update_idletasks()
         #needs to be done here so that the winfo_reqwidth is valid
         self.messageInfo.config(width=self.frameMain.winfo_reqwidth())
-        self.geometry("+%d+%d" %
-            ((parent.winfo_rootx()+((parent.winfo_width()/2)
-                -(self.winfo_reqwidth()/2)),
-              parent.winfo_rooty()+((parent.winfo_height()/2)
-                -(self.winfo_reqheight()/2)) )) ) #centre dialog over parent
-        self.deiconify() #geometry set, unhide
+        self.geometry(
+                "+%d+%d" % (
+                parent.winfo_rootx() +
+                (parent.winfo_width()/2 - self.winfo_reqwidth()/2),
+                parent.winfo_rooty() +
+                (parent.winfo_height()/2 - self.winfo_reqheight()/2)
+                ) )  #centre dialog over parent
+        self.deiconify()  #geometry set, unhide
         self.wait_window()
 
-    def CreateWidgets(self):
-        self.name=StringVar(self)
-        self.fontSize=StringVar(self)
-        self.frameMain = Frame(self,borderwidth=2,relief=SUNKEN)
-        self.frameMain.pack(side=TOP,expand=TRUE,fill=BOTH)
-        self.messageInfo=Message(self.frameMain,anchor=W,justify=LEFT,padx=5,pady=5,
-                text=self.message)#,aspect=200)
-        entryName=Entry(self.frameMain,textvariable=self.name,width=30)
+    def create_widgets(self):
+        self.name = StringVar(self.parent)
+        self.fontSize = StringVar(self.parent)
+        self.frameMain = Frame(self, borderwidth=2, relief=SUNKEN)
+        self.frameMain.pack(side=TOP, expand=TRUE, fill=BOTH)
+        self.messageInfo = Message(self.frameMain, anchor=W, justify=LEFT,
+                    padx=5, pady=5, text=self.message) #,aspect=200)
+        entryName = Entry(self.frameMain, textvariable=self.name, width=30)
         entryName.focus_set()
-        self.messageInfo.pack(padx=5,pady=5)#,expand=TRUE,fill=BOTH)
-        entryName.pack(padx=5,pady=5)
-        frameButtons=Frame(self)
-        frameButtons.pack(side=BOTTOM,fill=X)
-        self.buttonOk = Button(frameButtons,text='Ok',
-                width=8,command=self.Ok)
-        self.buttonOk.grid(row=0,column=0,padx=5,pady=5)
-        self.buttonCancel = Button(frameButtons,text='Cancel',
-                width=8,command=self.Cancel)
-        self.buttonCancel.grid(row=0,column=1,padx=5,pady=5)
+        self.messageInfo.pack(padx=5, pady=5) #, expand=TRUE, fill=BOTH)
+        entryName.pack(padx=5, pady=5)
 
-    def NameOk(self):
-        #simple validity check for a sensible
-        #ConfigParser file section name
-        nameOk=1
-        name=self.name.get()
-        name.strip()
+        frameButtons = Frame(self, pady=2)
+        frameButtons.pack(side=BOTTOM)
+        self.buttonOk = Button(frameButtons, text='Ok',
+                width=8, command=self.Ok)
+        self.buttonOk.pack(side=LEFT, padx=5)
+        self.buttonCancel = Button(frameButtons, text='Cancel',
+                width=8, command=self.Cancel)
+        self.buttonCancel.pack(side=RIGHT, padx=5)
+
+    def name_ok(self):
+        ''' After stripping entered name, check that it is a  sensible
+        ConfigParser file section name. Return it if it is, '' if not.
+        '''
+        name = self.name.get().strip()
         if not name: #no name specified
             tkMessageBox.showerror(title='Name Error',
                     message='No name specified.', parent=self)
-            nameOk=0
         elif len(name)>30: #name too long
             tkMessageBox.showerror(title='Name Error',
                     message='Name too long. It should be no more than '+
                     '30 characters.', parent=self)
-            nameOk=0
-        elif name in self.usedNames:
+            name = ''
+        elif name in self.used_names:
             tkMessageBox.showerror(title='Name Error',
                     message='This name is already in use.', parent=self)
-            nameOk=0
-        return nameOk
+            name = ''
+        return name
 
     def Ok(self, event=None):
-        if self.NameOk():
-            self.result=self.name.get().strip()
+        name = self.name_ok()
+        if name:
+            self.result = name
             self.destroy()
 
     def Cancel(self, event=None):
-        self.result=''
+        self.result = ''
         self.destroy()
 
 if __name__ == '__main__':
-    #test the dialog
-    root=Tk()
+    import unittest
+    unittest.main('idlelib.idle_test.test_config_name', verbosity=2, exit=False)
+    
+    # also human test the dialog
+    root = Tk()
     def run():
-        keySeq=''
         dlg=GetCfgSectionNameDialog(root,'Get Name',
-                'The information here should need to be word wrapped. Test.')
+                "After the text entered with [Ok] is stripped, <nothing>, "
+                "'abc', or more that 30 chars are errors. "
+                "Close with a valid entry (printed), [Cancel], or [X]",
+                {'abc'})
         print(dlg.result)
-    Button(root,text='Dialog',command=run).pack()
+    Message(root, text='').pack()  # will be needed for oher dialog tests
+    Button(root, text='Click to begin dialog test', command=run).pack()
     root.mainloop()
diff --git a/Lib/idlelib/idle_test/mock_tk.py b/Lib/idlelib/idle_test/mock_tk.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/mock_tk.py
@@ -0,0 +1,63 @@
+"""Classes that replace tkinter gui objects used by an object being tested.
+A gui object is anything with a master or parent paramenter, which is typically
+required in spite of what the doc strings say.
+"""
+
+class Var:
+    "Use for String/Int/BooleanVar: incomplete"
+    def __init__(self, master=None, value=None, name=None):
+        self.master = master
+        self.value = value
+        self.name = name
+    def set(self, value):
+        self.value = value
+    def get(self):
+        return self.value
+
+class Mbox_func:
+    """Generic mock for messagebox functions. All have same call signature.
+    Mbox instantiates once for each function. Tester uses attributes.
+    """
+    def __init__(self):
+        self.result = None  # The return for all show funcs
+    def __call__(self, title, message, *args, **kwds):
+        # Save all args for possible examination by tester
+        self.title = title
+        self.message = message
+        self.args = args
+        self.kwds = kwds
+        return self.result  # Set by tester for ask functions
+
+class Mbox:
+    """Mock for tkinter.messagebox with an Mbox_func for each function.
+    This module was 'tkMessageBox' in 2.x; hence the 'import as' in  3.x.
+    Example usage in test_module.py for testing functios in module.py:
+    ---
+from idlelib.idle_test.mock_tk import Mbox
+import module
+
+orig_mbox = module.tkMessageBox
+showerror = Mbox.showerror  # example, for attribute access in test methods
+
+class Test(unittest.TestCase):
+
+    @classmethod
+    def setUpClass(cls):
+        module.tkMessageBox = Mbox
+
+    @classmethod
+    def tearDownClass(cls):
+        module.tkMessageBox = orig_mbox
+    ---
+    When tkMessageBox functions are the only gui making calls in a method,
+    this replacement makes the method gui-free and unit-testable.
+    For 'ask' functions, set func.result return before calling method.
+    """
+    askokcancel = Mbox_func()     # True or False
+    askquestion = Mbox_func()     # 'yes' or 'no'
+    askretrycancel = Mbox_func()  # True or False
+    askyesno = Mbox_func()        # True or False
+    askyesnocancel = Mbox_func()  # True, False, or None
+    showerror = Mbox_func()    # None
+    showinfo = Mbox_func()     # None
+    showwarning = Mbox_func()  # None
diff --git a/Lib/idlelib/idle_test/test_config_name.py b/Lib/idlelib/idle_test/test_config_name.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_config_name.py
@@ -0,0 +1,75 @@
+"""Unit tests for idlelib.configSectionNameDialog"""
+import unittest
+from idlelib.idle_test.mock_tk import Var, Mbox
+from idlelib import configSectionNameDialog as name_dialog_module
+
+name_dialog = name_dialog_module.GetCfgSectionNameDialog
+
+class Dummy_name_dialog:
+    # Mock for testing the following methods of name_dialog
+    name_ok = name_dialog.name_ok
+    Ok = name_dialog.Ok
+    Cancel = name_dialog.Cancel
+    # Attributes, constant or variable, needed for tests
+    used_names = ['used']
+    name = Var()
+    result = None
+    destroyed = False
+    def destroy(self):
+        self.destroyed = True
+
+# name_ok calls Mbox.showerror if name is not ok
+orig_mbox = name_dialog_module.tkMessageBox
+showerror = Mbox.showerror
+
+class TestConfigName(unittest.TestCase):
+    dialog = Dummy_name_dialog()
+   
+    @classmethod
+    def setUpClass(cls):
+        name_dialog_module.tkMessageBox = Mbox
+
+    @classmethod
+    def tearDownClass(cls):
+        name_dialog_module.tkMessageBox = orig_mbox
+
+    def test_blank_name(self):
+        self.dialog.name.set(' ')
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('No', showerror.message)
+        
+    def test_used_name(self):
+        self.dialog.name.set('used')
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('use', showerror.message)
+        
+    def test_long_name(self):
+        self.dialog.name.set('good'*8)
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('too long', showerror.message)
+
+    def test_good_name(self):
+        self.dialog.name.set('  good ')
+        showerror.title = 'No Error'  # should not be called
+        self.assertEqual(self.dialog.name_ok(), 'good')
+        self.assertEqual(showerror.title, 'No Error')
+        
+    def test_ok(self):
+        self.dialog.destroyed = False
+        self.dialog.name.set('good')
+        self.dialog.Ok()
+        self.assertEqual(self.dialog.result, 'good')
+        self.assertTrue(self.dialog.destroyed)
+
+    def test_cancel(self):
+        self.dialog.destroyed = False
+        self.dialog.Cancel()
+        self.assertEqual(self.dialog.result, '')
+        self.assertTrue(self.dialog.destroyed)
+        
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 20:41:12 2013
From: python-checkins at python.org (terry.reedy)
Date: Wed,  5 Jun 2013 20:41:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bQf3D6wLyz7Lmm@mail.python.org>

http://hg.python.org/cpython/rev/367377d800a5
changeset:   84041:367377d800a5
parent:      84039:dfcb64f51f7b
parent:      84040:db4ecaf852e3
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Wed Jun 05 14:23:53 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/idlelib/configSectionNameDialog.py    |  111 +++++----
  Lib/idlelib/idle_test/mock_tk.py          |   63 +++++
  Lib/idlelib/idle_test/test_config_name.py |   75 ++++++
  3 files changed, 198 insertions(+), 51 deletions(-)


diff --git a/Lib/idlelib/configSectionNameDialog.py b/Lib/idlelib/configSectionNameDialog.py
--- a/Lib/idlelib/configSectionNameDialog.py
+++ b/Lib/idlelib/configSectionNameDialog.py
@@ -1,97 +1,106 @@
 """
 Dialog that allows user to specify a new config file section name.
 Used to get new highlight theme and keybinding set names.
+The 'return value' for the dialog, used two placed in configDialog.py,
+is the .result attribute set in the Ok and Cancel methods.
 """
 from tkinter import *
 import tkinter.messagebox as tkMessageBox
 
 class GetCfgSectionNameDialog(Toplevel):
-    def __init__(self,parent,title,message,usedNames):
+    def __init__(self, parent, title, message, used_names):
         """
         message - string, informational message to display
-        usedNames - list, list of names already in use for validity check
+        used_names - string collection, names already in use for validity check
         """
         Toplevel.__init__(self, parent)
         self.configure(borderwidth=5)
-        self.resizable(height=FALSE,width=FALSE)
+        self.resizable(height=FALSE, width=FALSE)
         self.title(title)
         self.transient(parent)
         self.grab_set()
         self.protocol("WM_DELETE_WINDOW", self.Cancel)
         self.parent = parent
-        self.message=message
-        self.usedNames=usedNames
-        self.result=''
-        self.CreateWidgets()
-        self.withdraw() #hide while setting geometry
+        self.message = message
+        self.used_names = used_names
+        self.create_widgets()
+        self.withdraw()  #hide while setting geometry
         self.update_idletasks()
         #needs to be done here so that the winfo_reqwidth is valid
         self.messageInfo.config(width=self.frameMain.winfo_reqwidth())
-        self.geometry("+%d+%d" %
-            ((parent.winfo_rootx()+((parent.winfo_width()/2)
-                -(self.winfo_reqwidth()/2)),
-              parent.winfo_rooty()+((parent.winfo_height()/2)
-                -(self.winfo_reqheight()/2)) )) ) #centre dialog over parent
-        self.deiconify() #geometry set, unhide
+        self.geometry(
+                "+%d+%d" % (
+                parent.winfo_rootx() +
+                (parent.winfo_width()/2 - self.winfo_reqwidth()/2),
+                parent.winfo_rooty() +
+                (parent.winfo_height()/2 - self.winfo_reqheight()/2)
+                ) )  #centre dialog over parent
+        self.deiconify()  #geometry set, unhide
         self.wait_window()
 
-    def CreateWidgets(self):
-        self.name=StringVar(self)
-        self.fontSize=StringVar(self)
-        self.frameMain = Frame(self,borderwidth=2,relief=SUNKEN)
-        self.frameMain.pack(side=TOP,expand=TRUE,fill=BOTH)
-        self.messageInfo=Message(self.frameMain,anchor=W,justify=LEFT,padx=5,pady=5,
-                text=self.message)#,aspect=200)
-        entryName=Entry(self.frameMain,textvariable=self.name,width=30)
+    def create_widgets(self):
+        self.name = StringVar(self.parent)
+        self.fontSize = StringVar(self.parent)
+        self.frameMain = Frame(self, borderwidth=2, relief=SUNKEN)
+        self.frameMain.pack(side=TOP, expand=TRUE, fill=BOTH)
+        self.messageInfo = Message(self.frameMain, anchor=W, justify=LEFT,
+                    padx=5, pady=5, text=self.message) #,aspect=200)
+        entryName = Entry(self.frameMain, textvariable=self.name, width=30)
         entryName.focus_set()
-        self.messageInfo.pack(padx=5,pady=5)#,expand=TRUE,fill=BOTH)
-        entryName.pack(padx=5,pady=5)
-        frameButtons=Frame(self)
-        frameButtons.pack(side=BOTTOM,fill=X)
-        self.buttonOk = Button(frameButtons,text='Ok',
-                width=8,command=self.Ok)
-        self.buttonOk.grid(row=0,column=0,padx=5,pady=5)
-        self.buttonCancel = Button(frameButtons,text='Cancel',
-                width=8,command=self.Cancel)
-        self.buttonCancel.grid(row=0,column=1,padx=5,pady=5)
+        self.messageInfo.pack(padx=5, pady=5) #, expand=TRUE, fill=BOTH)
+        entryName.pack(padx=5, pady=5)
 
-    def NameOk(self):
-        #simple validity check for a sensible
-        #ConfigParser file section name
-        nameOk=1
-        name=self.name.get()
-        name.strip()
+        frameButtons = Frame(self, pady=2)
+        frameButtons.pack(side=BOTTOM)
+        self.buttonOk = Button(frameButtons, text='Ok',
+                width=8, command=self.Ok)
+        self.buttonOk.pack(side=LEFT, padx=5)
+        self.buttonCancel = Button(frameButtons, text='Cancel',
+                width=8, command=self.Cancel)
+        self.buttonCancel.pack(side=RIGHT, padx=5)
+
+    def name_ok(self):
+        ''' After stripping entered name, check that it is a  sensible
+        ConfigParser file section name. Return it if it is, '' if not.
+        '''
+        name = self.name.get().strip()
         if not name: #no name specified
             tkMessageBox.showerror(title='Name Error',
                     message='No name specified.', parent=self)
-            nameOk=0
         elif len(name)>30: #name too long
             tkMessageBox.showerror(title='Name Error',
                     message='Name too long. It should be no more than '+
                     '30 characters.', parent=self)
-            nameOk=0
-        elif name in self.usedNames:
+            name = ''
+        elif name in self.used_names:
             tkMessageBox.showerror(title='Name Error',
                     message='This name is already in use.', parent=self)
-            nameOk=0
-        return nameOk
+            name = ''
+        return name
 
     def Ok(self, event=None):
-        if self.NameOk():
-            self.result=self.name.get().strip()
+        name = self.name_ok()
+        if name:
+            self.result = name
             self.destroy()
 
     def Cancel(self, event=None):
-        self.result=''
+        self.result = ''
         self.destroy()
 
 if __name__ == '__main__':
-    #test the dialog
-    root=Tk()
+    import unittest
+    unittest.main('idlelib.idle_test.test_config_name', verbosity=2, exit=False)
+    
+    # also human test the dialog
+    root = Tk()
     def run():
-        keySeq=''
         dlg=GetCfgSectionNameDialog(root,'Get Name',
-                'The information here should need to be word wrapped. Test.')
+                "After the text entered with [Ok] is stripped, <nothing>, "
+                "'abc', or more that 30 chars are errors. "
+                "Close with a valid entry (printed), [Cancel], or [X]",
+                {'abc'})
         print(dlg.result)
-    Button(root,text='Dialog',command=run).pack()
+    Message(root, text='').pack()  # will be needed for oher dialog tests
+    Button(root, text='Click to begin dialog test', command=run).pack()
     root.mainloop()
diff --git a/Lib/idlelib/idle_test/mock_tk.py b/Lib/idlelib/idle_test/mock_tk.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/mock_tk.py
@@ -0,0 +1,63 @@
+"""Classes that replace tkinter gui objects used by an object being tested.
+A gui object is anything with a master or parent paramenter, which is typically
+required in spite of what the doc strings say.
+"""
+
+class Var:
+    "Use for String/Int/BooleanVar: incomplete"
+    def __init__(self, master=None, value=None, name=None):
+        self.master = master
+        self.value = value
+        self.name = name
+    def set(self, value):
+        self.value = value
+    def get(self):
+        return self.value
+
+class Mbox_func:
+    """Generic mock for messagebox functions. All have same call signature.
+    Mbox instantiates once for each function. Tester uses attributes.
+    """
+    def __init__(self):
+        self.result = None  # The return for all show funcs
+    def __call__(self, title, message, *args, **kwds):
+        # Save all args for possible examination by tester
+        self.title = title
+        self.message = message
+        self.args = args
+        self.kwds = kwds
+        return self.result  # Set by tester for ask functions
+
+class Mbox:
+    """Mock for tkinter.messagebox with an Mbox_func for each function.
+    This module was 'tkMessageBox' in 2.x; hence the 'import as' in  3.x.
+    Example usage in test_module.py for testing functios in module.py:
+    ---
+from idlelib.idle_test.mock_tk import Mbox
+import module
+
+orig_mbox = module.tkMessageBox
+showerror = Mbox.showerror  # example, for attribute access in test methods
+
+class Test(unittest.TestCase):
+
+    @classmethod
+    def setUpClass(cls):
+        module.tkMessageBox = Mbox
+
+    @classmethod
+    def tearDownClass(cls):
+        module.tkMessageBox = orig_mbox
+    ---
+    When tkMessageBox functions are the only gui making calls in a method,
+    this replacement makes the method gui-free and unit-testable.
+    For 'ask' functions, set func.result return before calling method.
+    """
+    askokcancel = Mbox_func()     # True or False
+    askquestion = Mbox_func()     # 'yes' or 'no'
+    askretrycancel = Mbox_func()  # True or False
+    askyesno = Mbox_func()        # True or False
+    askyesnocancel = Mbox_func()  # True, False, or None
+    showerror = Mbox_func()    # None
+    showinfo = Mbox_func()     # None
+    showwarning = Mbox_func()  # None
diff --git a/Lib/idlelib/idle_test/test_config_name.py b/Lib/idlelib/idle_test/test_config_name.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_config_name.py
@@ -0,0 +1,75 @@
+"""Unit tests for idlelib.configSectionNameDialog"""
+import unittest
+from idlelib.idle_test.mock_tk import Var, Mbox
+from idlelib import configSectionNameDialog as name_dialog_module
+
+name_dialog = name_dialog_module.GetCfgSectionNameDialog
+
+class Dummy_name_dialog:
+    # Mock for testing the following methods of name_dialog
+    name_ok = name_dialog.name_ok
+    Ok = name_dialog.Ok
+    Cancel = name_dialog.Cancel
+    # Attributes, constant or variable, needed for tests
+    used_names = ['used']
+    name = Var()
+    result = None
+    destroyed = False
+    def destroy(self):
+        self.destroyed = True
+
+# name_ok calls Mbox.showerror if name is not ok
+orig_mbox = name_dialog_module.tkMessageBox
+showerror = Mbox.showerror
+
+class TestConfigName(unittest.TestCase):
+    dialog = Dummy_name_dialog()
+   
+    @classmethod
+    def setUpClass(cls):
+        name_dialog_module.tkMessageBox = Mbox
+
+    @classmethod
+    def tearDownClass(cls):
+        name_dialog_module.tkMessageBox = orig_mbox
+
+    def test_blank_name(self):
+        self.dialog.name.set(' ')
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('No', showerror.message)
+        
+    def test_used_name(self):
+        self.dialog.name.set('used')
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('use', showerror.message)
+        
+    def test_long_name(self):
+        self.dialog.name.set('good'*8)
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('too long', showerror.message)
+
+    def test_good_name(self):
+        self.dialog.name.set('  good ')
+        showerror.title = 'No Error'  # should not be called
+        self.assertEqual(self.dialog.name_ok(), 'good')
+        self.assertEqual(showerror.title, 'No Error')
+        
+    def test_ok(self):
+        self.dialog.destroyed = False
+        self.dialog.name.set('good')
+        self.dialog.Ok()
+        self.assertEqual(self.dialog.result, 'good')
+        self.assertTrue(self.dialog.destroyed)
+
+    def test_cancel(self):
+        self.dialog.destroyed = False
+        self.dialog.Cancel()
+        self.assertEqual(self.dialog.result, '')
+        self.assertTrue(self.dialog.destroyed)
+        
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 20:41:14 2013
From: python-checkins at python.org (terry.reedy)
Date: Wed,  5 Jun 2013 20:41:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUxODEzMDog?=
 =?utf-8?q?Test_class_idlelib=2EconfigSectionNameDialog=2EGetCfgSectionNam?=
 =?utf-8?q?eDialog=2E?=
Message-ID: <3bQf3G2gllz7LlW@mail.python.org>

http://hg.python.org/cpython/rev/31a67a0194ec
changeset:   84042:31a67a0194ec
branch:      2.7
parent:      84015:e9d0fb934b46
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Wed Jun 05 14:24:42 2013 -0400
summary:
  Issue18130: Test class idlelib.configSectionNameDialog.GetCfgSectionNameDialog.
Fix bug in existing human test and add instructions; fix two bugs in tested
code; remove redundancies, add spaces, and change two internal method names.
Add mock_tk with mocks for tkinter.Variable subclasses and tkinter.messagebox.
Use mocks in test_config_name to unittest methods that are otherwise gui-free.

files:
  Lib/idlelib/configSectionNameDialog.py    |  125 +++++----
  Lib/idlelib/idle_test/mock_tk.py          |   63 +++++
  Lib/idlelib/idle_test/test_config_name.py |   75 ++++++
  3 files changed, 202 insertions(+), 61 deletions(-)


diff --git a/Lib/idlelib/configSectionNameDialog.py b/Lib/idlelib/configSectionNameDialog.py
--- a/Lib/idlelib/configSectionNameDialog.py
+++ b/Lib/idlelib/configSectionNameDialog.py
@@ -1,97 +1,100 @@
 """
 Dialog that allows user to specify a new config file section name.
 Used to get new highlight theme and keybinding set names.
+The 'return value' for the dialog, used two placed in configDialog.py,
+is the .result attribute set in the Ok and Cancel methods.
 """
 from Tkinter import *
 import tkMessageBox
-
 class GetCfgSectionNameDialog(Toplevel):
-    def __init__(self,parent,title,message,usedNames):
+    def __init__(self, parent, title, message, used_names):
         """
         message - string, informational message to display
-        usedNames - list, list of names already in use for validity check
+        used_names - string collection, names already in use for validity check
         """
         Toplevel.__init__(self, parent)
         self.configure(borderwidth=5)
-        self.resizable(height=FALSE,width=FALSE)
+        self.resizable(height=FALSE, width=FALSE)
         self.title(title)
         self.transient(parent)
         self.grab_set()
         self.protocol("WM_DELETE_WINDOW", self.Cancel)
         self.parent = parent
-        self.message=message
-        self.usedNames=usedNames
-        self.result=''
-        self.CreateWidgets()
-        self.withdraw() #hide while setting geometry
+        self.message = message
+        self.used_names = used_names
+        self.create_widgets()
+        self.withdraw()  #hide while setting geometry
         self.update_idletasks()
         #needs to be done here so that the winfo_reqwidth is valid
         self.messageInfo.config(width=self.frameMain.winfo_reqwidth())
-        self.geometry("+%d+%d" %
-            ((parent.winfo_rootx()+((parent.winfo_width()/2)
-                -(self.winfo_reqwidth()/2)),
-              parent.winfo_rooty()+((parent.winfo_height()/2)
-                -(self.winfo_reqheight()/2)) )) ) #centre dialog over parent
-        self.deiconify() #geometry set, unhide
+        self.geometry(
+                "+%d+%d" % (
+                parent.winfo_rootx() +
+                (parent.winfo_width()/2 - self.winfo_reqwidth()/2),
+                parent.winfo_rooty() +
+                (parent.winfo_height()/2 - self.winfo_reqheight()/2)
+                ) )  #centre dialog over parent
+        self.deiconify()  #geometry set, unhide
         self.wait_window()
+    def create_widgets(self):
+        self.name = StringVar(self.parent)
+        self.fontSize = StringVar(self.parent)
+        self.frameMain = Frame(self, borderwidth=2, relief=SUNKEN)
+        self.frameMain.pack(side=TOP, expand=TRUE, fill=BOTH)
+        self.messageInfo = Message(self.frameMain, anchor=W, justify=LEFT,
+                    padx=5, pady=5, text=self.message) #,aspect=200)
+        entryName = Entry(self.frameMain, textvariable=self.name, width=30)
+        entryName.focus_set()
+        self.messageInfo.pack(padx=5, pady=5) #, expand=TRUE, fill=BOTH)
+        entryName.pack(padx=5, pady=5)
+        frameButtons = Frame(self, pady=2)
+        frameButtons.pack(side=BOTTOM)
+        self.buttonOk = Button(frameButtons, text='Ok',
+                width=8, command=self.Ok)
+        self.buttonOk.pack(side=LEFT, padx=5)
+        self.buttonCancel = Button(frameButtons, text='Cancel',
+                width=8, command=self.Cancel)
+        self.buttonCancel.pack(side=RIGHT, padx=5)
 
-    def CreateWidgets(self):
-        self.name=StringVar(self)
-        self.fontSize=StringVar(self)
-        self.frameMain = Frame(self,borderwidth=2,relief=SUNKEN)
-        self.frameMain.pack(side=TOP,expand=TRUE,fill=BOTH)
-        self.messageInfo=Message(self.frameMain,anchor=W,justify=LEFT,padx=5,pady=5,
-                text=self.message)#,aspect=200)
-        entryName=Entry(self.frameMain,textvariable=self.name,width=30)
-        entryName.focus_set()
-        self.messageInfo.pack(padx=5,pady=5)#,expand=TRUE,fill=BOTH)
-        entryName.pack(padx=5,pady=5)
-        frameButtons=Frame(self)
-        frameButtons.pack(side=BOTTOM,fill=X)
-        self.buttonOk = Button(frameButtons,text='Ok',
-                width=8,command=self.Ok)
-        self.buttonOk.grid(row=0,column=0,padx=5,pady=5)
-        self.buttonCancel = Button(frameButtons,text='Cancel',
-                width=8,command=self.Cancel)
-        self.buttonCancel.grid(row=0,column=1,padx=5,pady=5)
-
-    def NameOk(self):
-        #simple validity check for a sensible
-        #ConfigParser file section name
-        nameOk=1
-        name=self.name.get()
-        name.strip()
+    def name_ok(self):
+        ''' After stripping entered name, check that it is a  sensible
+        ConfigParser file section name. Return it if it is, '' if not.
+        '''
+        name = self.name.get().strip()
         if not name: #no name specified
             tkMessageBox.showerror(title='Name Error',
                     message='No name specified.', parent=self)
-            nameOk=0
         elif len(name)>30: #name too long
             tkMessageBox.showerror(title='Name Error',
                     message='Name too long. It should be no more than '+
                     '30 characters.', parent=self)
-            nameOk=0
-        elif name in self.usedNames:
+            name = ''
+        elif name in self.used_names:
             tkMessageBox.showerror(title='Name Error',
                     message='This name is already in use.', parent=self)
-            nameOk=0
-        return nameOk
+            name = ''
+        return name
+    def Ok(self, event=None):
+        name = self.name_ok()
+        if name:
+            self.result = name
+            self.destroy()
+    def Cancel(self, event=None):
+        self.result = ''
+        self.destroy()
+if __name__ == '__main__':
+    import unittest
+    unittest.main('idlelib.idle_test.test_config_name', verbosity=2, exit=False)
 
-    def Ok(self, event=None):
-        if self.NameOk():
-            self.result=self.name.get().strip()
-            self.destroy()
-
-    def Cancel(self, event=None):
-        self.result=''
-        self.destroy()
-
-if __name__ == '__main__':
-    #test the dialog
-    root=Tk()
+    # also human test the dialog
+    root = Tk()
     def run():
-        keySeq=''
         dlg=GetCfgSectionNameDialog(root,'Get Name',
-                'The information here should need to be word wrapped. Test.')
+                "After the text entered with [Ok] is stripped, <nothing>, "
+                "'abc', or more that 30 chars are errors. "
+                "Close with a valid entry (printed), [Cancel], or [X]",
+                {'abc'})
         print dlg.result
-    Button(root,text='Dialog',command=run).pack()
+    Message(root, text='').pack()  # will be needed for oher dialog tests
+    Button(root, text='Click to begin dialog test', command=run).pack()
     root.mainloop()
diff --git a/Lib/idlelib/idle_test/mock_tk.py b/Lib/idlelib/idle_test/mock_tk.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/mock_tk.py
@@ -0,0 +1,63 @@
+"""Classes that replace tkinter gui objects used by an object being tested.
+A gui object is anything with a master or parent paramenter, which is typically
+required in spite of what the doc strings say.
+"""
+
+class Var(object):
+    "Use for String/Int/BooleanVar: incomplete"
+    def __init__(self, master=None, value=None, name=None):
+        self.master = master
+        self.value = value
+        self.name = name
+    def set(self, value):
+        self.value = value
+    def get(self):
+        return self.value
+
+class Mbox_func(object):
+    """Generic mock for messagebox functions. All have same call signature.
+    Mbox instantiates once for each function. Tester uses attributes.
+    """
+    def __init__(self):
+        self.result = None  # The return for all show funcs
+    def __call__(self, title, message, *args, **kwds):
+        # Save all args for possible examination by tester
+        self.title = title
+        self.message = message
+        self.args = args
+        self.kwds = kwds
+        return self.result  # Set by tester for ask functions
+
+class Mbox(object):
+    """Mock for tkinter.messagebox with an Mbox_func for each function.
+    This module was 'tkMessageBox' in 2.x; hence the 'import as' in  3.x.
+    Example usage in test_module.py for testing functios in module.py:
+    ---
+from idlelib.idle_test.mock_tk import Mbox
+import module
+
+orig_mbox = module.tkMessageBox
+showerror = Mbox.showerror  # example, for attribute access in test methods
+
+class Test(unittest.TestCase):
+
+    @classmethod
+    def setUpClass(cls):
+        module.tkMessageBox = Mbox
+
+    @classmethod
+    def tearDownClass(cls):
+        module.tkMessageBox = orig_mbox
+    ---
+    When tkMessageBox functions are the only gui making calls in a method,
+    this replacement makes the method gui-free and unit-testable.
+    For 'ask' functions, set func.result return before calling method.
+    """
+    askokcancel = Mbox_func()     # True or False
+    askquestion = Mbox_func()     # 'yes' or 'no'
+    askretrycancel = Mbox_func()  # True or False
+    askyesno = Mbox_func()        # True or False
+    askyesnocancel = Mbox_func()  # True, False, or None
+    showerror = Mbox_func()    # None
+    showinfo = Mbox_func()     # None
+    showwarning = Mbox_func()  # None
diff --git a/Lib/idlelib/idle_test/test_config_name.py b/Lib/idlelib/idle_test/test_config_name.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_config_name.py
@@ -0,0 +1,75 @@
+"""Unit tests for idlelib.configSectionNameDialog"""
+import unittest
+from idlelib.idle_test.mock_tk import Var, Mbox
+from idlelib import configSectionNameDialog as name_dialog_module
+
+name_dialog = name_dialog_module.GetCfgSectionNameDialog
+
+class Dummy_name_dialog(object):
+    # Mock for testing the following methods of name_dialog
+    name_ok = name_dialog.name_ok.im_func
+    Ok = name_dialog.Ok.im_func
+    Cancel = name_dialog.Cancel.im_func
+    # Attributes, constant or variable, needed for tests
+    used_names = ['used']
+    name = Var()
+    result = None
+    destroyed = False
+    def destroy(self):
+        self.destroyed = True
+
+# name_ok calls Mbox.showerror if name is not ok
+orig_mbox = name_dialog_module.tkMessageBox
+showerror = Mbox.showerror
+
+class TestConfigName(unittest.TestCase):
+    dialog = Dummy_name_dialog()
+
+    @classmethod
+    def setUpClass(cls):
+        name_dialog_module.tkMessageBox = Mbox
+
+    @classmethod
+    def tearDownClass(cls):
+        name_dialog_module.tkMessageBox = orig_mbox
+
+    def test_blank_name(self):
+        self.dialog.name.set(' ')
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('No', showerror.message)
+
+    def test_used_name(self):
+        self.dialog.name.set('used')
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('use', showerror.message)
+
+    def test_long_name(self):
+        self.dialog.name.set('good'*8)
+        self.assertEqual(self.dialog.name_ok(), '')
+        self.assertEqual(showerror.title, 'Name Error')
+        self.assertIn('too long', showerror.message)
+
+    def test_good_name(self):
+        self.dialog.name.set('  good ')
+        showerror.title = 'No Error'  # should not be called
+        self.assertEqual(self.dialog.name_ok(), 'good')
+        self.assertEqual(showerror.title, 'No Error')
+
+    def test_ok(self):
+        self.dialog.destroyed = False
+        self.dialog.name.set('good')
+        self.dialog.Ok()
+        self.assertEqual(self.dialog.result, 'good')
+        self.assertTrue(self.dialog.destroyed)
+
+    def test_cancel(self):
+        self.dialog.destroyed = False
+        self.dialog.Cancel()
+        self.assertEqual(self.dialog.result, '')
+        self.assertTrue(self.dialog.destroyed)
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 20:41:15 2013
From: python-checkins at python.org (terry.reedy)
Date: Wed,  5 Jun 2013 20:41:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgMTgxMzA6?=
	=?utf-8?q?_delete_extra_spaces?=
Message-ID: <3bQf3H4fFYz7Ln0@mail.python.org>

http://hg.python.org/cpython/rev/382f4718e765
changeset:   84043:382f4718e765
branch:      3.3
parent:      84040:db4ecaf852e3
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Wed Jun 05 14:36:33 2013 -0400
summary:
  Issue 18130: delete extra spaces

files:
  Lib/idlelib/configSectionNameDialog.py    |   2 +-
  Lib/idlelib/idle_test/test_config_name.py |  10 +++++-----
  2 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Lib/idlelib/configSectionNameDialog.py b/Lib/idlelib/configSectionNameDialog.py
--- a/Lib/idlelib/configSectionNameDialog.py
+++ b/Lib/idlelib/configSectionNameDialog.py
@@ -91,7 +91,7 @@
 if __name__ == '__main__':
     import unittest
     unittest.main('idlelib.idle_test.test_config_name', verbosity=2, exit=False)
-    
+
     # also human test the dialog
     root = Tk()
     def run():
diff --git a/Lib/idlelib/idle_test/test_config_name.py b/Lib/idlelib/idle_test/test_config_name.py
--- a/Lib/idlelib/idle_test/test_config_name.py
+++ b/Lib/idlelib/idle_test/test_config_name.py
@@ -24,7 +24,7 @@
 
 class TestConfigName(unittest.TestCase):
     dialog = Dummy_name_dialog()
-   
+
     @classmethod
     def setUpClass(cls):
         name_dialog_module.tkMessageBox = Mbox
@@ -38,13 +38,13 @@
         self.assertEqual(self.dialog.name_ok(), '')
         self.assertEqual(showerror.title, 'Name Error')
         self.assertIn('No', showerror.message)
-        
+
     def test_used_name(self):
         self.dialog.name.set('used')
         self.assertEqual(self.dialog.name_ok(), '')
         self.assertEqual(showerror.title, 'Name Error')
         self.assertIn('use', showerror.message)
-        
+
     def test_long_name(self):
         self.dialog.name.set('good'*8)
         self.assertEqual(self.dialog.name_ok(), '')
@@ -56,7 +56,7 @@
         showerror.title = 'No Error'  # should not be called
         self.assertEqual(self.dialog.name_ok(), 'good')
         self.assertEqual(showerror.title, 'No Error')
-        
+
     def test_ok(self):
         self.dialog.destroyed = False
         self.dialog.name.set('good')
@@ -69,7 +69,7 @@
         self.dialog.Cancel()
         self.assertEqual(self.dialog.result, '')
         self.assertTrue(self.dialog.destroyed)
-        
+
 
 if __name__ == '__main__':
     unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 20:41:16 2013
From: python-checkins at python.org (terry.reedy)
Date: Wed,  5 Jun 2013 20:41:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bQf3J6VL2z7Lkn@mail.python.org>

http://hg.python.org/cpython/rev/9c06a9a8bba1
changeset:   84044:9c06a9a8bba1
parent:      84041:367377d800a5
parent:      84043:382f4718e765
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Wed Jun 05 14:36:50 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/idlelib/configSectionNameDialog.py    |   2 +-
  Lib/idlelib/idle_test/test_config_name.py |  10 +++++-----
  2 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/Lib/idlelib/configSectionNameDialog.py b/Lib/idlelib/configSectionNameDialog.py
--- a/Lib/idlelib/configSectionNameDialog.py
+++ b/Lib/idlelib/configSectionNameDialog.py
@@ -91,7 +91,7 @@
 if __name__ == '__main__':
     import unittest
     unittest.main('idlelib.idle_test.test_config_name', verbosity=2, exit=False)
-    
+
     # also human test the dialog
     root = Tk()
     def run():
diff --git a/Lib/idlelib/idle_test/test_config_name.py b/Lib/idlelib/idle_test/test_config_name.py
--- a/Lib/idlelib/idle_test/test_config_name.py
+++ b/Lib/idlelib/idle_test/test_config_name.py
@@ -24,7 +24,7 @@
 
 class TestConfigName(unittest.TestCase):
     dialog = Dummy_name_dialog()
-   
+
     @classmethod
     def setUpClass(cls):
         name_dialog_module.tkMessageBox = Mbox
@@ -38,13 +38,13 @@
         self.assertEqual(self.dialog.name_ok(), '')
         self.assertEqual(showerror.title, 'Name Error')
         self.assertIn('No', showerror.message)
-        
+
     def test_used_name(self):
         self.dialog.name.set('used')
         self.assertEqual(self.dialog.name_ok(), '')
         self.assertEqual(showerror.title, 'Name Error')
         self.assertIn('use', showerror.message)
-        
+
     def test_long_name(self):
         self.dialog.name.set('good'*8)
         self.assertEqual(self.dialog.name_ok(), '')
@@ -56,7 +56,7 @@
         showerror.title = 'No Error'  # should not be called
         self.assertEqual(self.dialog.name_ok(), 'good')
         self.assertEqual(showerror.title, 'No Error')
-        
+
     def test_ok(self):
         self.dialog.destroyed = False
         self.dialog.name.set('good')
@@ -69,7 +69,7 @@
         self.dialog.Cancel()
         self.assertEqual(self.dialog.result, '')
         self.assertTrue(self.dialog.destroyed)
-        
+
 
 if __name__ == '__main__':
     unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun  5 21:03:34 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Wed,  5 Jun 2013 21:03:34 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_obscure_the_email_address_in_?=
	=?utf-8?q?bdfl-delegate=2C_too?=
Message-ID: <3bQfY24CDZz7LnR@mail.python.org>

http://hg.python.org/peps/rev/51170b413ebc
changeset:   4922:51170b413ebc
user:        Benjamin Peterson <benjamin at python.org>
date:        Wed Jun 05 12:03:27 2013 -0700
summary:
  obscure the email address in bdfl-delegate, too

files:
  pep2html.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep2html.py b/pep2html.py
--- a/pep2html.py
+++ b/pep2html.py
@@ -202,7 +202,7 @@
     print >> outfile, '</td></tr></table>'
     print >> outfile, '<div class="header">\n<table border="0">'
     for k, v in header:
-        if k.lower() in ('author', 'discussions-to'):
+        if k.lower() in ('author', 'bdfl-delegate', 'discussions-to'):
             mailtos = []
             for part in re.split(',\s*', v):
                 if '@' in part:

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Thu Jun  6 00:31:41 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Thu,  6 Jun 2013 00:31:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317931=3A_Resolve_?=
 =?utf-8?q?confusion_on_Windows_between_pids_and_process_handles=2E?=
Message-ID: <3bQl996bPdz7LkX@mail.python.org>

http://hg.python.org/cpython/rev/0410bf251e10
changeset:   84045:0410bf251e10
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Wed Jun 05 23:29:30 2013 +0100
summary:
  Issue #17931: Resolve confusion on Windows between pids and process handles.

files:
  Include/longobject.h  |  13 +++++++++++++
  Misc/NEWS             |   5 ++---
  Modules/posixmodule.c |  25 +++++++++----------------
  PC/msvcrtmodule.c     |   5 +++--
  PC/pyconfig.h         |   4 ++--
  5 files changed, 29 insertions(+), 23 deletions(-)


diff --git a/Include/longobject.h b/Include/longobject.h
--- a/Include/longobject.h
+++ b/Include/longobject.h
@@ -52,6 +52,19 @@
 #error "sizeof(pid_t) is neither sizeof(int), sizeof(long) or sizeof(long long)"
 #endif /* SIZEOF_PID_T */
 
+#if SIZEOF_VOID_P == SIZEOF_INT
+#  define _Py_PARSE_INTPTR "i"
+#  define _Py_PARSE_UINTPTR "I"
+#elif SIZEOF_VOID_P == SIZEOF_LONG
+#  define _Py_PARSE_INTPTR "l"
+#  define _Py_PARSE_UINTPTR "k"
+#elif defined(SIZEOF_LONG_LONG) && SIZEOF_VOID_P == SIZEOF_LONG_LONG
+#  define _Py_PARSE_INTPTR "L"
+#  define _Py_PARSE_UINTPTR "K"
+#else
+#  error "void* different in size from int, long and long long"
+#endif /* SIZEOF_VOID_P */
+
 /* Used by Python/mystrtoul.c. */
 #ifndef Py_LIMITED_API
 PyAPI_DATA(unsigned char) _PyLong_DigitValue[256];
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,9 +10,8 @@
 Core and Builtins
 -----------------
 
-- Issue #17931: Fix PyLong_FromPid() on Windows 64-bit: processes are
-  identified by their HANDLE which is a pointer (and not a long, which is
-  smaller).
+- Issue #17931: Resolve confusion on Windows between pids and process
+  handles.
 
 - Tweak the exception message when the magic number or size value in a bytecode
   file is truncated.
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -5014,11 +5014,7 @@
     if (spawnval == -1)
         return posix_error();
     else
-#if SIZEOF_LONG == SIZEOF_VOID_P
-        return Py_BuildValue("l", (long) spawnval);
-#else
-        return Py_BuildValue("L", (PY_LONG_LONG) spawnval);
-#endif
+        return Py_BuildValue(_Py_PARSE_INTPTR, spawnval);
 }
 
 
@@ -5104,11 +5100,7 @@
     if (spawnval == -1)
         (void) posix_error();
     else
-#if SIZEOF_LONG == SIZEOF_VOID_P
-        res = Py_BuildValue("l", (long) spawnval);
-#else
-        res = Py_BuildValue("L", (PY_LONG_LONG) spawnval);
-#endif
+        res = Py_BuildValue(_Py_PARSE_INTPTR, spawnval);
 
     while (--envc >= 0)
         PyMem_DEL(envlist[envc]);
@@ -6178,16 +6170,17 @@
 win32_kill(PyObject *self, PyObject *args)
 {
     PyObject *result;
-    DWORD pid, sig, err;
+    pid_t pid;
+    DWORD sig, err;
     HANDLE handle;
 
-    if (!PyArg_ParseTuple(args, "kk:kill", &pid, &sig))
+    if (!PyArg_ParseTuple(args, _Py_PARSE_PID "k:kill", &pid, &sig))
         return NULL;
 
     /* Console processes which share a common console can be sent CTRL+C or
        CTRL+BREAK events, provided they handle said events. */
     if (sig == CTRL_C_EVENT || sig == CTRL_BREAK_EVENT) {
-        if (GenerateConsoleCtrlEvent(sig, pid) == 0) {
+        if (GenerateConsoleCtrlEvent(sig, (DWORD)pid) == 0) {
             err = GetLastError();
             PyErr_SetFromWindowsErr(err);
         }
@@ -6197,7 +6190,7 @@
 
     /* If the signal is outside of what GenerateConsoleCtrlEvent can use,
        attempt to open and terminate the process. */
-    handle = OpenProcess(PROCESS_ALL_ACCESS, FALSE, pid);
+    handle = OpenProcess(PROCESS_ALL_ACCESS, FALSE, (DWORD)pid);
     if (handle == NULL) {
         err = GetLastError();
         return PyErr_SetFromWindowsErr(err);
@@ -6603,7 +6596,7 @@
     Py_intptr_t pid;
     int status, options;
 
-    if (!PyArg_ParseTuple(args, _Py_PARSE_PID "i:waitpid", &pid, &options))
+    if (!PyArg_ParseTuple(args, _Py_PARSE_INTPTR "i:waitpid", &pid, &options))
         return NULL;
     Py_BEGIN_ALLOW_THREADS
     pid = _cwait(&status, pid, options);
@@ -6612,7 +6605,7 @@
         return posix_error();
 
     /* shift the status left a byte so this is more like the POSIX waitpid */
-    return Py_BuildValue("Ni", PyLong_FromPid(pid), status << 8);
+    return Py_BuildValue(_Py_PARSE_INTPTR "i", pid, status << 8);
 }
 #endif /* HAVE_WAITPID || HAVE_CWAIT */
 
diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c
--- a/PC/msvcrtmodule.c
+++ b/PC/msvcrtmodule.c
@@ -113,11 +113,12 @@
 static PyObject *
 msvcrt_open_osfhandle(PyObject *self, PyObject *args)
 {
-    long handle;
+    Py_intptr_t handle;
     int flags;
     int fd;
 
-    if (!PyArg_ParseTuple(args, "li:open_osfhandle", &handle, &flags))
+    if (!PyArg_ParseTuple(args, _Py_PARSE_INTPTR "i:open_osfhandle",
+                          &handle, &flags))
         return NULL;
 
     fd = _open_osfhandle(handle, flags);
diff --git a/PC/pyconfig.h b/PC/pyconfig.h
--- a/PC/pyconfig.h
+++ b/PC/pyconfig.h
@@ -723,8 +723,8 @@
 /* The size of `wchar_t', as computed by sizeof. */
 #define SIZEOF_WCHAR_T 2
 
-/* The size of `pid_t' (HANDLE). */
-#define SIZEOF_PID_T SIZEOF_VOID_P
+/* The size of `pid_t', as computed by sizeof. */
+#define SIZEOF_PID_T SIZEOF_INT
 
 /* Define if you have the dl library (-ldl).  */
 /* #undef HAVE_LIBDL */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun  6 00:37:55 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu,  6 Jun 2013 00:37:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_tweak_exception_message_?=
	=?utf-8?q?=28again=29?=
Message-ID: <3bQlJM5B3szSYH@mail.python.org>

http://hg.python.org/cpython/rev/6b189f29b1e6
changeset:   84046:6b189f29b1e6
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 05 18:37:50 2013 -0400
summary:
  tweak exception message (again)

files:
  Lib/importlib/_bootstrap.py |     4 +-
  Python/importlib.h          |  4627 +++++++++++-----------
  2 files changed, 2316 insertions(+), 2315 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -668,11 +668,11 @@
         _verbose_message(message)
         raise ImportError(message, **exc_details)
     elif len(raw_timestamp) != 4:
-        message = 'reached EOF while reading magic number in {!r}'.format(name)
+        message = 'reached EOF while reading timestamp in {!r}'.format(name)
         _verbose_message(message)
         raise EOFError(message)
     elif len(raw_size) != 4:
-        message = 'reached EOF while reading size in {!r}'.format(name)
+        message = 'reached EOF while reading size of source in {!r}'.format(name)
         _verbose_message(message)
         raise EOFError(message)
     if source_stats is not None:
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Jun  6 05:51:35 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 06 Jun 2013 05:51:35 +0200
Subject: [Python-checkins] Daily reference leaks (6b189f29b1e6): sum=2779
Message-ID: <E1UkREd-00073Z-Lq@ap.vmr.nerim.net>

results for 6b189f29b1e6 on branch "default"
--------------------------------------------

test_capi leaked [778, 778, 778] references, sum=2334
test_capi leaked [147, 149, 149] memory blocks, sum=445


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogQ0mxId', '-x']

From solipsis at pitrou.net  Fri Jun  7 05:52:37 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 07 Jun 2013 05:52:37 +0200
Subject: [Python-checkins] Daily reference leaks (6b189f29b1e6): sum=2779
Message-ID: <E1UknjB-0006uo-4d@ap.vmr.nerim.net>

results for 6b189f29b1e6 on branch "default"
--------------------------------------------

test_capi leaked [778, 778, 778] references, sum=2334
test_capi leaked [147, 149, 149] memory blocks, sum=445


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogRg7d2Y', '-x']

From python-checkins at python.org  Fri Jun  7 16:21:55 2013
From: python-checkins at python.org (vinay.sajip)
Date: Fri,  7 Jun 2013 16:21:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2311959=3A_SMTPSer?=
 =?utf-8?q?ver_and_SMTPChannel_now_take_an_optional_map=2C_use_of?=
Message-ID: <3bRmC761nzzShX@mail.python.org>

http://hg.python.org/cpython/rev/ed498f477549
changeset:   84047:ed498f477549
user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
date:        Fri Jun 07 15:21:41 2013 +0100
summary:
  Closes #11959: SMTPServer and SMTPChannel now take an optional map, use of which avoids affecting global state.

files:
  Doc/library/smtpd.rst    |  19 +++++++-
  Lib/smtpd.py             |  12 +++--
  Lib/test/test_logging.py |  64 +--------------------------
  Misc/NEWS                |   3 +
  4 files changed, 30 insertions(+), 68 deletions(-)


diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst
--- a/Doc/library/smtpd.rst
+++ b/Doc/library/smtpd.rst
@@ -27,7 +27,8 @@
 ------------------
 
 
-.. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432)
+.. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432,
+                      map=None)
 
    Create a new :class:`SMTPServer` object, which binds to local address
    *localaddr*.  It will treat *remoteaddr* as an upstream SMTP relayer.  It
@@ -38,6 +39,8 @@
    accepted in a ``DATA`` command.  A value of ``None`` or ``0`` means no
    limit.
 
+   A dictionary can be specified in *map* to avoid using a global socket map.
+
    .. method:: process_message(peer, mailfrom, rcpttos, data)
 
       Raise :exc:`NotImplementedError` exception. Override this in subclasses to
@@ -53,6 +56,9 @@
       Override this in subclasses to use a custom :class:`SMTPChannel` for
       managing SMTP clients.
 
+   .. versionchanged:: 3.4
+      The *map* argument was added.
+
 
 DebuggingServer Objects
 -----------------------
@@ -90,11 +96,20 @@
 SMTPChannel Objects
 -------------------
 
-.. class:: SMTPChannel(server, conn, addr)
+.. class:: SMTPChannel(server, conn, addr, data_size_limit=33554432,
+                       map=None))
 
    Create a new :class:`SMTPChannel` object which manages the communication
    between the server and a single SMTP client.
 
+   *conn* and *addr* are as per the instance variables described below.
+
+   *data_size_limit* specifies the maximum number of bytes that will be
+   accepted in a ``DATA`` command.  A value of ``None`` or ``0`` means no
+   limit.
+
+   A dictionary can be specified in *map* to avoid using a global socket map.
+
    To use a custom SMTPChannel implementation you need to override the
    :attr:`SMTPServer.channel_class` of your :class:`SMTPServer`.
 
diff --git a/Lib/smtpd.py b/Lib/smtpd.py
--- a/Lib/smtpd.py
+++ b/Lib/smtpd.py
@@ -121,8 +121,9 @@
         })
     max_command_size_limit = max(command_size_limits.values())
 
-    def __init__(self, server, conn, addr, data_size_limit=DATA_SIZE_DEFAULT):
-        asynchat.async_chat.__init__(self, conn)
+    def __init__(self, server, conn, addr, data_size_limit=DATA_SIZE_DEFAULT,
+                 map=None):
+        asynchat.async_chat.__init__(self, conn, map=map)
         self.smtp_server = server
         self.conn = conn
         self.addr = addr
@@ -576,11 +577,11 @@
     channel_class = SMTPChannel
 
     def __init__(self, localaddr, remoteaddr,
-                 data_size_limit=DATA_SIZE_DEFAULT):
+                 data_size_limit=DATA_SIZE_DEFAULT, map=None):
         self._localaddr = localaddr
         self._remoteaddr = remoteaddr
         self.data_size_limit = data_size_limit
-        asyncore.dispatcher.__init__(self)
+        asyncore.dispatcher.__init__(self, map=map)
         try:
             self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
             # try to re-use a server port if possible
@@ -597,7 +598,8 @@
 
     def handle_accepted(self, conn, addr):
         print('Incoming connection from %s' % repr(addr), file=DEBUGSTREAM)
-        channel = self.channel_class(self, conn, addr, self.data_size_limit)
+        channel = self.channel_class(self, conn, addr, self.data_size_limit,
+                                     self._map)
 
     # API for "doing something useful with the message"
     def process_message(self, peer, mailfrom, rcpttos, data):
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -659,41 +659,6 @@
 # -- if it proves to be of wider utility than just test_logging
 
 if threading:
-    class TestSMTPChannel(smtpd.SMTPChannel):
-        """
-        This derived class has had to be created because smtpd does not
-        support use of custom channel maps, although they are allowed by
-        asyncore's design. Issue #11959 has been raised to address this,
-        and if resolved satisfactorily, some of this code can be removed.
-        """
-        def __init__(self, server, conn, addr, sockmap):
-            asynchat.async_chat.__init__(self, conn, sockmap)
-            self.smtp_server = server
-            self.conn = conn
-            self.addr = addr
-            self.data_size_limit = None
-            self.received_lines = []
-            self.smtp_state = self.COMMAND
-            self.seen_greeting = ''
-            self.mailfrom = None
-            self.rcpttos = []
-            self.received_data = ''
-            self.fqdn = socket.getfqdn()
-            self.num_bytes = 0
-            try:
-                self.peer = conn.getpeername()
-            except OSError as err:
-                # a race condition  may occur if the other end is closing
-                # before we can get the peername
-                self.close()
-                if err.args[0] != errno.ENOTCONN:
-                    raise
-                return
-            self.push('220 %s %s' % (self.fqdn, smtpd.__version__))
-            self.set_terminator(b'\r\n')
-            self.extended_smtp = False
-
-
     class TestSMTPServer(smtpd.SMTPServer):
         """
         This class implements a test SMTP server.
@@ -714,37 +679,14 @@
                         :func:`asyncore.loop`. This avoids changing the
                         :mod:`asyncore` module's global state.
         """
-        channel_class = TestSMTPChannel
 
         def __init__(self, addr, handler, poll_interval, sockmap):
-            self._localaddr = addr
-            self._remoteaddr = None
-            self.data_size_limit = None
-            self.sockmap = sockmap
-            asyncore.dispatcher.__init__(self, map=sockmap)
-            try:
-                sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-                sock.setblocking(0)
-                self.set_socket(sock, map=sockmap)
-                # try to re-use a server port if possible
-                self.set_reuse_addr()
-                self.bind(addr)
-                self.port = sock.getsockname()[1]
-                self.listen(5)
-            except:
-                self.close()
-                raise
+            smtpd.SMTPServer.__init__(self, addr, None, map=sockmap)
+            self.port = self.socket.getsockname()[1]
             self._handler = handler
             self._thread = None
             self.poll_interval = poll_interval
 
-        def handle_accepted(self, conn, addr):
-            """
-            Redefined only because the base class does not pass in a
-            map, forcing use of a global in :mod:`asyncore`.
-            """
-            channel = self.channel_class(self, conn, addr, self.sockmap)
-
         def process_message(self, peer, mailfrom, rcpttos, data):
             """
             Delegates to the handler passed in to the server's constructor.
@@ -775,7 +717,7 @@
                                   :func:`asyncore.loop`.
             """
             try:
-                asyncore.loop(poll_interval, map=self.sockmap)
+                asyncore.loop(poll_interval, map=self._map)
             except OSError:
                 # On FreeBSD 8, closing the server repeatably
                 # raises this error. We swallow it if the
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -112,6 +112,9 @@
 Library
 -------
 
+- Issue #11959: SMTPServer and SMTPChannel now take an optional map, use of
+  which avoids affecting global state.
+
 - Issue #18109: os.uname() now decodes fields from the locale encoding, and
   socket.gethostname() now decodes the hostname from the locale encoding,
   instead of using the UTF-8 encoding in strict mode.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun  7 16:37:50 2013
From: python-checkins at python.org (vinay.sajip)
Date: Fri,  7 Jun 2013 16:37:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317903=3A_Added_pa?=
 =?utf-8?q?th_search_changes_to_launcher=2E?=
Message-ID: <3bRmYV4jHGzS7w@mail.python.org>

http://hg.python.org/cpython/rev/a174d79cef2e
changeset:   84048:a174d79cef2e
user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
date:        Fri Jun 07 15:37:28 2013 +0100
summary:
  Issue #17903: Added path search changes to launcher.

files:
  PC/launcher.c |  56 ++++++++++++++++++++++++++++----------
  1 files changed, 41 insertions(+), 15 deletions(-)


diff --git a/PC/launcher.c b/PC/launcher.c
--- a/PC/launcher.c
+++ b/PC/launcher.c
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2011-2012 Vinay Sajip.
+ * Copyright (C) 2011-2013 Vinay Sajip.
  * Licensed to PSF under a contributor agreement.
  *
  * Based on the work of:
@@ -18,7 +18,7 @@
 
 /* Build options. */
 #define SKIP_PREFIX
-/* #define SEARCH_PATH */
+#define SEARCH_PATH
 
 /* Just for now - static definition */
 
@@ -595,12 +595,17 @@
     }
 }
 
-static wchar_t * builtin_virtual_paths [] = {
-    L"/usr/bin/env python",
-    L"/usr/bin/python",
-    L"/usr/local/bin/python",
-    L"python",
-    NULL
+typedef struct {
+    wchar_t *shebang;
+    BOOL search;
+} SHEBANG;
+
+static SHEBANG builtin_virtual_paths [] = {
+    { L"/usr/bin/env python", TRUE },
+    { L"/usr/bin/python", FALSE },
+    { L"/usr/local/bin/python", FALSE },
+    { L"python", FALSE },
+    { NULL, FALSE },
 };
 
 /* For now, a static array of commands. */
@@ -776,10 +781,10 @@
 
 static BOOL
 parse_shebang(wchar_t * shebang_line, int nchars, wchar_t ** command,
-              wchar_t ** suffix)
+              wchar_t ** suffix, BOOL *search)
 {
     BOOL rc = FALSE;
-    wchar_t ** vpp;
+    SHEBANG * vpp;
     size_t plen;
     wchar_t * p;
     wchar_t zapped;
@@ -789,15 +794,17 @@
 
     *command = NULL;    /* failure return */
     *suffix = NULL;
+    *search = FALSE;
 
     if ((*shebang_line++ == L'#') && (*shebang_line++ == L'!')) {
         shebang_line = skip_whitespace(shebang_line);
         if (*shebang_line) {
             *command = shebang_line;
-            for (vpp = builtin_virtual_paths; *vpp; ++vpp) {
-                plen = wcslen(*vpp);
-                if (wcsncmp(shebang_line, *vpp, plen) == 0) {
+            for (vpp = builtin_virtual_paths; vpp->shebang; ++vpp) {
+                plen = wcslen(vpp->shebang);
+                if (wcsncmp(shebang_line, vpp->shebang, plen) == 0) {
                     rc = TRUE;
+                    *search = vpp->search;
                     /* We can do this because all builtin commands contain
                      * "python".
                      */
@@ -805,7 +812,7 @@
                     break;
                 }
             }
-            if (*vpp == NULL) {
+            if (vpp->shebang == NULL) {
                 /*
                  * Not found in builtins - look in customised commands.
                  *
@@ -1012,8 +1019,10 @@
     int i, j, nchars = 0;
     int header_len;
     BOOL is_virt;
+    BOOL search;
     wchar_t * command;
     wchar_t * suffix;
+    COMMAND *cmd = NULL;
     INSTALLED_PYTHON * ip;
 
     if (rc == 0) {
@@ -1125,7 +1134,7 @@
             if (nchars > 0) {
                 shebang_line[--nchars] = L'\0';
                 is_virt = parse_shebang(shebang_line, nchars, &command,
-                                        &suffix);
+                                        &suffix, &search);
                 if (command != NULL) {
                     debug(L"parse_shebang: found command: %s\n", command);
                     if (!is_virt) {
@@ -1141,6 +1150,23 @@
                             error(RC_BAD_VIRTUAL_PATH, L"Unknown virtual \
 path '%s'", command);
                         command += 6;   /* skip past "python" */
+                        if (search && ((*command == L'\0') || isspace(*command))) {
+                            /* Command is eligible for path search, and there
+                             * is no version specification.
+                             */
+                            debug(L"searching PATH for python executable\n");
+                            cmd = find_on_path(L"python");
+                            debug(L"Python on path: %s\n", cmd ? cmd->value : L"<not found>");
+                            if (cmd) {
+                                debug(L"located python on PATH: %s\n", cmd->value);
+                                invoke_child(cmd->value, suffix, cmdline);
+                                /* Exit here, as we have found the command */
+                                return;
+                            }
+                            /* FALL THROUGH: No python found on PATH, so fall
+                             * back to locating the correct installed python.
+                             */
+                        }
                         if (*command && !validate_version(command))
                             error(RC_BAD_VIRTUAL_PATH, L"Invalid version \
 specification: '%s'.\nIn the first line of the script, 'python' needs to be \

-- 
Repository URL: http://hg.python.org/cpython

From brett at python.org  Fri Jun  7 17:16:05 2013
From: brett at python.org (Brett Cannon)
Date: Fri, 7 Jun 2013 11:16:05 -0400
Subject: [Python-checkins] cpython: Issue #17931: Resolve confusion on
 Windows between pids and process handles.
In-Reply-To: <3bQl996bPdz7LkX@mail.python.org>
References: <3bQl996bPdz7LkX@mail.python.org>
Message-ID: <CAP1=2W5G6gRjW_A7ggPV0_OAwRWUcaSz3QS6YMQoszKsZHn-RQ@mail.gmail.com>

I think this CL introduced a memory leak. The daily leak report went from 0
to not 0 between June 4 and June 5 and this is the only CL that touched C
code.


On Wed, Jun 5, 2013 at 6:31 PM, richard.oudkerk
<python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/0410bf251e10
> changeset:   84045:0410bf251e10
> user:        Richard Oudkerk <shibturn at gmail.com>
> date:        Wed Jun 05 23:29:30 2013 +0100
> summary:
>   Issue #17931: Resolve confusion on Windows between pids and process
> handles.
>
> files:
>   Include/longobject.h  |  13 +++++++++++++
>   Misc/NEWS             |   5 ++---
>   Modules/posixmodule.c |  25 +++++++++----------------
>   PC/msvcrtmodule.c     |   5 +++--
>   PC/pyconfig.h         |   4 ++--
>   5 files changed, 29 insertions(+), 23 deletions(-)
>
>
> diff --git a/Include/longobject.h b/Include/longobject.h
> --- a/Include/longobject.h
> +++ b/Include/longobject.h
> @@ -52,6 +52,19 @@
>  #error "sizeof(pid_t) is neither sizeof(int), sizeof(long) or sizeof(long
> long)"
>  #endif /* SIZEOF_PID_T */
>
> +#if SIZEOF_VOID_P == SIZEOF_INT
> +#  define _Py_PARSE_INTPTR "i"
> +#  define _Py_PARSE_UINTPTR "I"
> +#elif SIZEOF_VOID_P == SIZEOF_LONG
> +#  define _Py_PARSE_INTPTR "l"
> +#  define _Py_PARSE_UINTPTR "k"
> +#elif defined(SIZEOF_LONG_LONG) && SIZEOF_VOID_P == SIZEOF_LONG_LONG
> +#  define _Py_PARSE_INTPTR "L"
> +#  define _Py_PARSE_UINTPTR "K"
> +#else
> +#  error "void* different in size from int, long and long long"
> +#endif /* SIZEOF_VOID_P */
> +
>  /* Used by Python/mystrtoul.c. */
>  #ifndef Py_LIMITED_API
>  PyAPI_DATA(unsigned char) _PyLong_DigitValue[256];
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -10,9 +10,8 @@
>  Core and Builtins
>  -----------------
>
> -- Issue #17931: Fix PyLong_FromPid() on Windows 64-bit: processes are
> -  identified by their HANDLE which is a pointer (and not a long, which is
> -  smaller).
> +- Issue #17931: Resolve confusion on Windows between pids and process
> +  handles.
>
>  - Tweak the exception message when the magic number or size value in a
> bytecode
>    file is truncated.
> diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
> --- a/Modules/posixmodule.c
> +++ b/Modules/posixmodule.c
> @@ -5014,11 +5014,7 @@
>      if (spawnval == -1)
>          return posix_error();
>      else
> -#if SIZEOF_LONG == SIZEOF_VOID_P
> -        return Py_BuildValue("l", (long) spawnval);
> -#else
> -        return Py_BuildValue("L", (PY_LONG_LONG) spawnval);
> -#endif
> +        return Py_BuildValue(_Py_PARSE_INTPTR, spawnval);
>  }
>
>
> @@ -5104,11 +5100,7 @@
>      if (spawnval == -1)
>          (void) posix_error();
>      else
> -#if SIZEOF_LONG == SIZEOF_VOID_P
> -        res = Py_BuildValue("l", (long) spawnval);
> -#else
> -        res = Py_BuildValue("L", (PY_LONG_LONG) spawnval);
> -#endif
> +        res = Py_BuildValue(_Py_PARSE_INTPTR, spawnval);
>
>      while (--envc >= 0)
>          PyMem_DEL(envlist[envc]);
> @@ -6178,16 +6170,17 @@
>  win32_kill(PyObject *self, PyObject *args)
>  {
>      PyObject *result;
> -    DWORD pid, sig, err;
> +    pid_t pid;
> +    DWORD sig, err;
>      HANDLE handle;
>
> -    if (!PyArg_ParseTuple(args, "kk:kill", &pid, &sig))
> +    if (!PyArg_ParseTuple(args, _Py_PARSE_PID "k:kill", &pid, &sig))
>          return NULL;
>
>      /* Console processes which share a common console can be sent CTRL+C
> or
>         CTRL+BREAK events, provided they handle said events. */
>      if (sig == CTRL_C_EVENT || sig == CTRL_BREAK_EVENT) {
> -        if (GenerateConsoleCtrlEvent(sig, pid) == 0) {
> +        if (GenerateConsoleCtrlEvent(sig, (DWORD)pid) == 0) {
>              err = GetLastError();
>              PyErr_SetFromWindowsErr(err);
>          }
> @@ -6197,7 +6190,7 @@
>
>      /* If the signal is outside of what GenerateConsoleCtrlEvent can use,
>         attempt to open and terminate the process. */
> -    handle = OpenProcess(PROCESS_ALL_ACCESS, FALSE, pid);
> +    handle = OpenProcess(PROCESS_ALL_ACCESS, FALSE, (DWORD)pid);
>      if (handle == NULL) {
>          err = GetLastError();
>          return PyErr_SetFromWindowsErr(err);
> @@ -6603,7 +6596,7 @@
>      Py_intptr_t pid;
>      int status, options;
>
> -    if (!PyArg_ParseTuple(args, _Py_PARSE_PID "i:waitpid", &pid,
> &options))
> +    if (!PyArg_ParseTuple(args, _Py_PARSE_INTPTR "i:waitpid", &pid,
> &options))
>          return NULL;
>      Py_BEGIN_ALLOW_THREADS
>      pid = _cwait(&status, pid, options);
> @@ -6612,7 +6605,7 @@
>          return posix_error();
>
>      /* shift the status left a byte so this is more like the POSIX
> waitpid */
> -    return Py_BuildValue("Ni", PyLong_FromPid(pid), status << 8);
> +    return Py_BuildValue(_Py_PARSE_INTPTR "i", pid, status << 8);
>  }
>  #endif /* HAVE_WAITPID || HAVE_CWAIT */
>
> diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c
> --- a/PC/msvcrtmodule.c
> +++ b/PC/msvcrtmodule.c
> @@ -113,11 +113,12 @@
>  static PyObject *
>  msvcrt_open_osfhandle(PyObject *self, PyObject *args)
>  {
> -    long handle;
> +    Py_intptr_t handle;
>      int flags;
>      int fd;
>
> -    if (!PyArg_ParseTuple(args, "li:open_osfhandle", &handle, &flags))
> +    if (!PyArg_ParseTuple(args, _Py_PARSE_INTPTR "i:open_osfhandle",
> +                          &handle, &flags))
>          return NULL;
>
>      fd = _open_osfhandle(handle, flags);
> diff --git a/PC/pyconfig.h b/PC/pyconfig.h
> --- a/PC/pyconfig.h
> +++ b/PC/pyconfig.h
> @@ -723,8 +723,8 @@
>  /* The size of `wchar_t', as computed by sizeof. */
>  #define SIZEOF_WCHAR_T 2
>
> -/* The size of `pid_t' (HANDLE). */
> -#define SIZEOF_PID_T SIZEOF_VOID_P
> +/* The size of `pid_t', as computed by sizeof. */
> +#define SIZEOF_PID_T SIZEOF_INT
>
>  /* Define if you have the dl library (-ldl).  */
>  /* #undef HAVE_LIBDL */
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130607/5b9cf444/attachment-0001.html>

From python-checkins at python.org  Fri Jun  7 17:45:50 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri,  7 Jun 2013 17:45:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317314=3A_Stop_usi?=
 =?utf-8?q?ng_imp_in_multiprocessing=2Eforking_and_move_over?=
Message-ID: <3bRp3y70yJzS7w@mail.python.org>

http://hg.python.org/cpython/rev/97adaa820353
changeset:   84049:97adaa820353
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 07 11:45:41 2013 -0400
summary:
  Issue #17314: Stop using imp in multiprocessing.forking and move over
to importlib.

files:
  Lib/multiprocessing/forking.py |  22 ++++++++++++----------
  Misc/NEWS                      |   2 ++
  2 files changed, 14 insertions(+), 10 deletions(-)


diff --git a/Lib/multiprocessing/forking.py b/Lib/multiprocessing/forking.py
--- a/Lib/multiprocessing/forking.py
+++ b/Lib/multiprocessing/forking.py
@@ -450,6 +450,7 @@
             # Main modules not actually called __main__.py may
             # contain additional code that should still be executed
             import imp
+            import importlib
 
             if main_path is None:
                 dirs = None
@@ -460,16 +461,17 @@
 
             assert main_name not in sys.modules, main_name
             sys.modules.pop('__mp_main__', None)
-            file, path_name, etc = imp.find_module(main_name, dirs)
+            # We should not try to load __main__
+            # since that would execute 'if __name__ == "__main__"'
+            # clauses, potentially causing a psuedo fork bomb.
+            loader = importlib.find_loader(main_name, path=dirs)
+            main_module = imp.new_module(main_name)
             try:
-                # We should not do 'imp.load_module("__main__", ...)'
-                # since that would execute 'if __name__ == "__main__"'
-                # clauses, potentially causing a psuedo fork bomb.
-                main_module = imp.load_module(
-                    '__mp_main__', file, path_name, etc
-                    )
-            finally:
-                if file:
-                    file.close()
+                loader.init_module_attrs(main_module)
+            except AttributeError:  # init_module_attrs is optional
+                pass
+            main_module.__name__ = '__mp_main__'
+            code = loader.get_code(main_name)
+            exec(code, main_module.__dict__)
 
             sys.modules['__main__'] = sys.modules['__mp_main__'] = main_module
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -112,6 +112,8 @@
 Library
 -------
 
+- Issue #17314: Move multiprocessing.forking over to importlib.
+
 - Issue #11959: SMTPServer and SMTPChannel now take an optional map, use of
   which avoids affecting global state.
 

-- 
Repository URL: http://hg.python.org/cpython

From thomas at python.org  Fri Jun  7 17:54:48 2013
From: thomas at python.org (Thomas Wouters)
Date: Fri, 7 Jun 2013 17:54:48 +0200
Subject: [Python-checkins] cpython: Issue #17931: Resolve confusion on
 Windows between pids and process handles.
In-Reply-To: <CAP1=2W5G6gRjW_A7ggPV0_OAwRWUcaSz3QS6YMQoszKsZHn-RQ@mail.gmail.com>
References: <3bQl996bPdz7LkX@mail.python.org>
	<CAP1=2W5G6gRjW_A7ggPV0_OAwRWUcaSz3QS6YMQoszKsZHn-RQ@mail.gmail.com>
Message-ID: <CAPdQG2ofZ+Uq8GOt_4f9ebmp2Ry2NQ7JiZx2DPinV0_UQ8pqyg@mail.gmail.com>

On Fri, Jun 7, 2013 at 5:16 PM, Brett Cannon <brett at python.org> wrote:

> I think this CL introduced a memory leak. The daily leak report went from
> 0 to not 0 between June 4 and June 5 and this is the only CL that touched C
> code.
>

It wasn't introduced by C code :) The refleak report is induced by the PEP
443 implementation, see my message to python-dev.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130607/bc666f27/attachment.html>

From python-checkins at python.org  Fri Jun  7 19:18:46 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri,  7 Jun 2013 19:18:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MDU1?=
 =?utf-8?q?=3A_Move_to_importlib_from_imp_for_IDLE=2E?=
Message-ID: <3bRr7B6Jd7zRFF@mail.python.org>

http://hg.python.org/cpython/rev/a0d8ae880ae6
changeset:   84050:a0d8ae880ae6
branch:      3.3
parent:      84043:382f4718e765
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 07 13:17:48 2013 -0400
summary:
  Issue #18055: Move to importlib from imp for IDLE.

files:
  Lib/idlelib/EditorWindow.py |  57 ++++++++----------------
  Misc/NEWS                   |   2 +
  2 files changed, 21 insertions(+), 38 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -1,5 +1,5 @@
-import imp
 import importlib
+import importlib.abc
 import os
 import re
 import string
@@ -35,34 +35,6 @@
         release += '%s%s' % (level[0], serial)
     return release
 
-def _find_module(fullname, path=None):
-    """Version of imp.find_module() that handles hierarchical module names"""
-
-    file = None
-    for tgt in fullname.split('.'):
-        if file is not None:
-            file.close()            # close intermediate files
-        (file, filename, descr) = imp.find_module(tgt, path)
-        if descr[2] == imp.PY_SOURCE:
-            break                   # find but not load the source file
-        module = imp.load_module(tgt, file, filename, descr)
-        try:
-            path = module.__path__
-        except AttributeError:
-            raise ImportError('No source for module ' + module.__name__)
-    if descr[2] != imp.PY_SOURCE:
-        # If all of the above fails and didn't raise an exception,fallback
-        # to a straight import which can find __init__.py in a package.
-        m = __import__(fullname)
-        try:
-            filename = m.__file__
-        except AttributeError:
-            pass
-        else:
-            file = None
-            descr = os.path.splitext(filename)[1], None, imp.PY_SOURCE
-    return file, filename, descr
-
 
 class HelpDialog(object):
 
@@ -687,20 +659,29 @@
             return
         # XXX Ought to insert current file's directory in front of path
         try:
-            (f, file, (suffix, mode, type)) = _find_module(name)
-        except (NameError, ImportError) as msg:
+            loader = importlib.find_loader(name)
+        except (ValueError, ImportError) as msg:
             tkMessageBox.showerror("Import error", str(msg), parent=self.text)
             return
-        if type != imp.PY_SOURCE:
-            tkMessageBox.showerror("Unsupported type",
-                "%s is not a source module" % name, parent=self.text)
+        if loader is None:
+            tkMessageBox.showerror("Import error", "module not found",
+                                   parent=self.text)
             return
-        if f:
-            f.close()
+        if not isinstance(loader, importlib.abc.SourceLoader):
+            tkMessageBox.showerror("Import error", "not a source-based module",
+                                   parent=self.text)
+            return
+        try:
+            file_path = loader.get_filename(name)
+        except AttributeError:
+            tkMessageBox.showerror("Import error",
+                                   "loader does not support get_filename",
+                                   parent=self.text)
+            return
         if self.flist:
-            self.flist.open(file)
+            self.flist.open(file_path)
         else:
-            self.io.loadfile(file)
+            self.io.loadfile(file_path)
 
     def open_class_browser(self, event=None):
         filename = self.io.filename
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -71,6 +71,8 @@
 IDLE
 ----
 
+- Issue #18055: Move IDLE off of imp and on to importlib.
+
 - Issue #15392: Create a unittest framework for IDLE.
   Initial patch by Rajagopalasarma Jayakrishnan.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun  7 19:18:48 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri,  7 Jun 2013 19:18:48 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_merge_w/_3=2E3_for_issue_=2318055?=
Message-ID: <3bRr7D15clzRMv@mail.python.org>

http://hg.python.org/cpython/rev/3a3ec484ce95
changeset:   84051:3a3ec484ce95
parent:      84049:97adaa820353
parent:      84050:a0d8ae880ae6
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 07 13:18:36 2013 -0400
summary:
  merge w/ 3.3 for issue #18055

files:
  Lib/idlelib/EditorWindow.py |  57 ++++++++----------------
  Misc/NEWS                   |   2 +
  2 files changed, 21 insertions(+), 38 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -1,5 +1,5 @@
-import imp
 import importlib
+import importlib.abc
 import os
 import re
 import string
@@ -35,34 +35,6 @@
         release += '%s%s' % (level[0], serial)
     return release
 
-def _find_module(fullname, path=None):
-    """Version of imp.find_module() that handles hierarchical module names"""
-
-    file = None
-    for tgt in fullname.split('.'):
-        if file is not None:
-            file.close()            # close intermediate files
-        (file, filename, descr) = imp.find_module(tgt, path)
-        if descr[2] == imp.PY_SOURCE:
-            break                   # find but not load the source file
-        module = imp.load_module(tgt, file, filename, descr)
-        try:
-            path = module.__path__
-        except AttributeError:
-            raise ImportError('No source for module ' + module.__name__)
-    if descr[2] != imp.PY_SOURCE:
-        # If all of the above fails and didn't raise an exception,fallback
-        # to a straight import which can find __init__.py in a package.
-        m = __import__(fullname)
-        try:
-            filename = m.__file__
-        except AttributeError:
-            pass
-        else:
-            file = None
-            descr = os.path.splitext(filename)[1], None, imp.PY_SOURCE
-    return file, filename, descr
-
 
 class HelpDialog(object):
 
@@ -687,20 +659,29 @@
             return
         # XXX Ought to insert current file's directory in front of path
         try:
-            (f, file, (suffix, mode, type)) = _find_module(name)
-        except (NameError, ImportError) as msg:
+            loader = importlib.find_loader(name)
+        except (ValueError, ImportError) as msg:
             tkMessageBox.showerror("Import error", str(msg), parent=self.text)
             return
-        if type != imp.PY_SOURCE:
-            tkMessageBox.showerror("Unsupported type",
-                "%s is not a source module" % name, parent=self.text)
+        if loader is None:
+            tkMessageBox.showerror("Import error", "module not found",
+                                   parent=self.text)
             return
-        if f:
-            f.close()
+        if not isinstance(loader, importlib.abc.SourceLoader):
+            tkMessageBox.showerror("Import error", "not a source-based module",
+                                   parent=self.text)
+            return
+        try:
+            file_path = loader.get_filename(name)
+        except AttributeError:
+            tkMessageBox.showerror("Import error",
+                                   "loader does not support get_filename",
+                                   parent=self.text)
+            return
         if self.flist:
-            self.flist.open(file)
+            self.flist.open(file_path)
         else:
-            self.io.loadfile(file)
+            self.io.loadfile(file_path)
 
     def open_class_browser(self, event=None):
         filename = self.io.filename
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -435,6 +435,8 @@
 IDLE
 ----
 
+- Issue #18055: Move IDLE off of imp and on to importlib.
+
 - Issue #15392: Create a unittest framework for IDLE.
   Initial patch by Rajagopalasarma Jayakrishnan.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun  7 19:27:02 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri,  7 Jun 2013 19:27:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=237732=3A_Move_an_i?=
 =?utf-8?q?mp=2Efind=5Fmodule_test_from_test=5Fimport_to?=
Message-ID: <3bRrJk2WF3zRMv@mail.python.org>

http://hg.python.org/cpython/rev/bf882390713c
changeset:   84052:bf882390713c
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 07 13:26:53 2013 -0400
summary:
  Issue #7732: Move an imp.find_module test from test_import to
test_imp.

files:
  Lib/test/test_imp.py    |  11 +++++++++++
  Lib/test/test_import.py |  10 ----------
  2 files changed, 11 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py
--- a/Lib/test/test_imp.py
+++ b/Lib/test/test_imp.py
@@ -273,6 +273,17 @@
             return
         imp.load_module(name, None, *found[1:])
 
+    @unittest.skipIf(sys.dont_write_bytecode,
+        "test meaningful only when writing bytecode")
+    def test_bug7732(self):
+        source = support.TESTFN + '.py'
+        os.mkdir(source)
+        try:
+            self.assertRaisesRegex(ImportError, '^No module',
+                imp.find_module, support.TESTFN, ["."])
+        finally:
+            os.rmdir(source)
+
 
 class ReloadTests(unittest.TestCase):
 
diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -127,16 +127,6 @@
         finally:
             del sys.path[0]
 
-    @skip_if_dont_write_bytecode
-    def test_bug7732(self):
-        source = TESTFN + '.py'
-        os.mkdir(source)
-        try:
-            self.assertRaisesRegex(ImportError, '^No module',
-                imp.find_module, TESTFN, ["."])
-        finally:
-            os.rmdir(source)
-
     def test_module_with_large_stack(self, module='longlist'):
         # Regression test for http://bugs.python.org/issue561858.
         filename = module + '.py'

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun  7 22:26:31 2013
From: python-checkins at python.org (lukasz.langa)
Date: Fri,  7 Jun 2013 22:26:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixed_=2318150=3A_duplicat?=
 =?utf-8?q?e_test_inside_TestSingleDispatch?=
Message-ID: <3bRwHq4TpJzSFM@mail.python.org>

http://hg.python.org/cpython/rev/a16bebe653b1
changeset:   84053:a16bebe653b1
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Fri Jun 07 22:25:27 2013 +0200
summary:
  Fixed #18150: duplicate test inside TestSingleDispatch

Thanks to Vajrasky Kok for the patch

files:
  Lib/test/test_functools.py |  33 +++++++++++--------------
  Misc/ACKS                  |   1 +
  2 files changed, 15 insertions(+), 19 deletions(-)


diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -868,29 +868,24 @@
         @functools.singledispatch
         def g(obj):
             return "base"
-        class C:
+        class A:
             pass
-        class D(C):
+        class C(A):
             pass
-        def g_C(c):
-            return "C"
-        g.register(C, g_C)
-        self.assertEqual(g(C()), "C")
-        self.assertEqual(g(D()), "C")
-
-    def test_classic_classes(self):
-        @functools.singledispatch
-        def g(obj):
-            return "base"
-        class C:
+        class B(A):
             pass
-        class D(C):
+        class D(C, B):
             pass
-        def g_C(c):
-            return "C"
-        g.register(C, g_C)
-        self.assertEqual(g(C()), "C")
-        self.assertEqual(g(D()), "C")
+        def g_A(a):
+            return "A"
+        def g_B(b):
+            return "B"
+        g.register(A, g_A)
+        g.register(B, g_B)
+        self.assertEqual(g(A()), "A")
+        self.assertEqual(g(B()), "B")
+        self.assertEqual(g(C()), "A")
+        self.assertEqual(g(D()), "B")
 
     def test_register_decorator(self):
         @functools.singledispatch
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -666,6 +666,7 @@
 Greg Kochanski
 Damon Kohler
 Marko Kohtala
+Vajrasky Kok
 Guido Kollerie
 Jacek Konieczny
 ???? ?????????

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun  7 22:54:38 2013
From: python-checkins at python.org (lukasz.langa)
Date: Fri,  7 Jun 2013 22:54:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_moved_the_single-dispatch_?=
 =?utf-8?q?generic_function_definitions_to_the_glossary?=
Message-ID: <3bRwwG035Xz7LkB@mail.python.org>

http://hg.python.org/cpython/rev/42519153cb08
changeset:   84054:42519153cb08
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Fri Jun 07 22:54:03 2013 +0200
summary:
  moved the single-dispatch generic function definitions to the glossary

files:
  Doc/glossary.rst          |  13 +++++++++++++
  Doc/library/functools.rst |   8 ++------
  2 files changed, 15 insertions(+), 6 deletions(-)


diff --git a/Doc/glossary.rst b/Doc/glossary.rst
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -302,6 +302,15 @@
          >>> sum(i*i for i in range(10))         # sum of squares 0, 1, 4, ... 81
          285
 
+   generic function
+      A function composed of multiple functions implementing the same operation
+      for different types. Which implementation should be used during a call is
+      determined by the dispatch algorithm.
+
+      See also the :term:`single dispatch` glossary entry, the
+      :func:`functools.singledispatch` decorator, and :pep:`443`.
+
+
    GIL
       See :term:`global interpreter lock`.
 
@@ -745,6 +754,10 @@
       mapping rather than a sequence because the lookups use arbitrary
       :term:`immutable` keys rather than integers.
 
+   single dispatch
+      A form of :term:`generic function` dispatch where the implementation is
+      chosen based on the type of a single argument.
+
    slice
       An object usually containing a portion of a :term:`sequence`.  A slice is
       created using the subscript notation, ``[]`` with colons between numbers
diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst
--- a/Doc/library/functools.rst
+++ b/Doc/library/functools.rst
@@ -189,12 +189,8 @@
 
 .. decorator:: singledispatch(default)
 
-   Transforms a function into a single-dispatch generic function.  A **generic
-   function** is composed of multiple functions implementing the same operation
-   for different types. Which implementation should be used during a call is
-   determined by the dispatch algorithm.  When the implementation is chosen
-   based on the type of a single argument, this is known as **single
-   dispatch**.
+   Transforms a function into a :term:`single-dispatch <single
+   dispatch>` :term:`generic function`.
 
    To define a generic function, decorate it with the ``@singledispatch``
    decorator. Note that the dispatch happens on the type of the first argument,

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Jun  8 05:48:23 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 08 Jun 2013 05:48:23 +0200
Subject: [Python-checkins] Daily reference leaks (42519153cb08): sum=2779
Message-ID: <E1UlA8d-0001jj-3b@ap.vmr.nerim.net>

results for 42519153cb08 on branch "default"
--------------------------------------------

test_capi leaked [778, 778, 778] references, sum=2334
test_capi leaked [147, 149, 149] memory blocks, sum=445


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogOzVZi0', '-x']

From python-checkins at python.org  Sat Jun  8 06:38:11 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat,  8 Jun 2013 06:38:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTUxLCBwYXJ0?=
 =?utf-8?q?_1=3A_Backport_idlelilb_portion_of_Andrew_Svetlov=27s_3=2E4_pat?=
 =?utf-8?q?ch?=
Message-ID: <3bS7C70vDszR2m@mail.python.org>

http://hg.python.org/cpython/rev/2fe64ce5da05
changeset:   84055:2fe64ce5da05
branch:      3.3
parent:      84050:a0d8ae880ae6
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 08 00:22:45 2013 -0400
summary:
  #18151, part 1: Backport idlelilb portion of Andrew Svetlov's 3.4 patch
changing IOError to OSError (#16715).

files:
  Lib/idlelib/EditorWindow.py  |   2 +-
  Lib/idlelib/GrepDialog.py    |   2 +-
  Lib/idlelib/IOBinding.py     |   4 ++--
  Lib/idlelib/OutputWindow.py  |   2 +-
  Lib/idlelib/PyShell.py       |   8 ++++----
  Lib/idlelib/configHandler.py |  18 +++++++-----------
  Lib/idlelib/rpc.py           |   2 +-
  Lib/idlelib/textView.py      |   2 +-
  8 files changed, 18 insertions(+), 22 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -901,7 +901,7 @@
             with open(self.recent_files_path, 'w',
                         encoding='utf_8', errors='replace') as rf_file:
                 rf_file.writelines(rf_list)
-        except IOError as err:
+        except OSError as err:
             if not getattr(self.root, "recentfilelist_error_displayed", False):
                 self.root.recentfilelist_error_displayed = True
                 tkMessageBox.showerror(title='IDLE Error',
diff --git a/Lib/idlelib/GrepDialog.py b/Lib/idlelib/GrepDialog.py
--- a/Lib/idlelib/GrepDialog.py
+++ b/Lib/idlelib/GrepDialog.py
@@ -82,7 +82,7 @@
         for fn in list:
             try:
                 f = open(fn, errors='replace')
-            except IOError as msg:
+            except OSError as msg:
                 print(msg)
                 continue
             lineno = 0
diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py
--- a/Lib/idlelib/IOBinding.py
+++ b/Lib/idlelib/IOBinding.py
@@ -213,7 +213,7 @@
             f.seek(0)
             bytes = f.read()
             f.close()
-        except IOError as msg:
+        except OSError as msg:
             tkMessageBox.showerror("I/O Error", str(msg), master=self.text)
             return False
         chars, converted = self._decode(two_lines, bytes)
@@ -378,7 +378,7 @@
             f.flush()
             f.close()
             return True
-        except IOError as msg:
+        except OSError as msg:
             tkMessageBox.showerror("I/O Error", str(msg),
                                    master=self.text)
             return False
diff --git a/Lib/idlelib/OutputWindow.py b/Lib/idlelib/OutputWindow.py
--- a/Lib/idlelib/OutputWindow.py
+++ b/Lib/idlelib/OutputWindow.py
@@ -106,7 +106,7 @@
                     f = open(filename, "r")
                     f.close()
                     break
-                except IOError:
+                except OSError:
                     continue
         else:
             return None
diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -59,7 +59,7 @@
         try:
             file.write(warnings.formatwarning(message, category, filename,
                                               lineno, line=line))
-        except IOError:
+        except OSError:
             pass  ## file (probably __stderr__) is invalid, warning dropped.
     warnings.showwarning = idle_showwarning
     def idle_formatwarning(message, category, filename, lineno, line=None):
@@ -213,7 +213,7 @@
         try:
             with open(self.breakpointPath, "r") as fp:
                 lines = fp.readlines()
-        except IOError:
+        except OSError:
             lines = []
         try:
             with open(self.breakpointPath, "w") as new_file:
@@ -224,7 +224,7 @@
                 breaks = self.breakpoints
                 if breaks:
                     new_file.write(filename + '=' + str(breaks) + '\n')
-        except IOError as err:
+        except OSError as err:
             if not getattr(self.root, "breakpoint_error_displayed", False):
                 self.root.breakpoint_error_displayed = True
                 tkMessageBox.showerror(title='IDLE Error',
@@ -532,7 +532,7 @@
             return
         try:
             response = clt.pollresponse(self.active_seq, wait=0.05)
-        except (EOFError, IOError, KeyboardInterrupt):
+        except (EOFError, OSError, KeyboardInterrupt):
             # lost connection or subprocess terminated itself, restart
             # [the KBI is from rpc.SocketIO.handle_EOF()]
             if self.tkconsole.closing:
diff --git a/Lib/idlelib/configHandler.py b/Lib/idlelib/configHandler.py
--- a/Lib/idlelib/configHandler.py
+++ b/Lib/idlelib/configHandler.py
@@ -142,7 +142,7 @@
             fname = self.file
             try:
                 cfgFile = open(fname, 'w')
-            except IOError:
+            except OSError:
                 os.unlink(fname)
                 cfgFile = open(fname, 'w')
             with cfgFile:
@@ -207,7 +207,7 @@
                         userDir+',\n but the path does not exist.\n')
                 try:
                     sys.stderr.write(warn)
-                except IOError:
+                except OSError:
                     pass
                 userDir = '~'
         if userDir == "~": # still no path to home!
@@ -217,7 +217,7 @@
         if not os.path.exists(userDir):
             try:
                 os.mkdir(userDir)
-            except (OSError, IOError):
+            except OSError:
                 warn = ('\n Warning: unable to create user config directory\n'+
                         userDir+'\n Check path and permissions.\n Exiting!\n\n')
                 sys.stderr.write(warn)
@@ -251,7 +251,7 @@
                                                      raw=raw)))
             try:
                 sys.stderr.write(warning)
-            except IOError:
+            except OSError:
                 pass
         try:
             if self.defaultCfg[configType].has_option(section,option):
@@ -268,13 +268,11 @@
                        (option, section, default))
             try:
                 sys.stderr.write(warning)
-            except IOError:
+            except OSError:
                 pass
         return default
-
     def SetOption(self, configType, section, option, value):
         """In user's config file, set section's option to value.
-
         """
         self.userCfg[configType].SetOption(section, option, value)
 
@@ -380,7 +378,7 @@
                            (element, themeName, theme[element]))
                 try:
                     sys.stderr.write(warning)
-                except IOError:
+                except OSError:
                     pass
             colour=cfgParser.Get(themeName,element,default=theme[element])
             theme[element]=colour
@@ -637,13 +635,11 @@
                                (event, keySetName, keyBindings[event]))
                     try:
                         sys.stderr.write(warning)
-                    except IOError:
+                    except OSError:
                         pass
         return keyBindings
-
     def GetExtraHelpSourceList(self,configSet):
         """Fetch list of extra help sources from a given configSet.
-
         Valid configSets are 'user' or 'default'.  Return a list of tuples of
         the form (menu_item , path_to_help_file , option), or return the empty
         list.  'option' is the sequence number of the help resource.  'option'
diff --git a/Lib/idlelib/rpc.py b/Lib/idlelib/rpc.py
--- a/Lib/idlelib/rpc.py
+++ b/Lib/idlelib/rpc.py
@@ -339,7 +339,7 @@
                 r, w, x = select.select([], [self.sock], [])
                 n = self.sock.send(s[:BUFSIZE])
             except (AttributeError, TypeError):
-                raise IOError("socket no longer exists")
+                raise OSError("socket no longer exists")
             except socket.error:
                 raise
             else:
diff --git a/Lib/idlelib/textView.py b/Lib/idlelib/textView.py
--- a/Lib/idlelib/textView.py
+++ b/Lib/idlelib/textView.py
@@ -66,7 +66,7 @@
     try:
         with open(filename, 'r', encoding=encoding) as file:
             contents = file.read()
-    except IOError:
+    except OSError:
         import tkinter.messagebox as tkMessageBox
         tkMessageBox.showerror(title='File Load Error',
                                message='Unable to load file %r .' % filename,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  8 06:38:12 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat,  8 Jun 2013 06:38:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=2318151_null_merge_with_3=2E3=2E?=
Message-ID: <3bS7C82qkjzRKx@mail.python.org>

http://hg.python.org/cpython/rev/0be613638523
changeset:   84056:0be613638523
parent:      84054:42519153cb08
parent:      84055:2fe64ce5da05
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 08 00:35:51 2013 -0400
summary:
  #18151 null merge with 3.3.

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  8 17:53:59 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Sat,  8 Jun 2013 17:53:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315528=3A_Delay_im?=
 =?utf-8?q?porting_atexit_until_weakref=2Efinalize=28=29_used=2E?=
Message-ID: <3bSQBv3nlyzNk8@mail.python.org>

http://hg.python.org/cpython/rev/d6ad9d7468f7
changeset:   84057:d6ad9d7468f7
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Sat Jun 08 16:52:29 2013 +0100
summary:
  Issue #15528: Delay importing atexit until weakref.finalize() used.

files:
  Lib/weakref.py |  10 +++++++---
  1 files changed, 7 insertions(+), 3 deletions(-)


diff --git a/Lib/weakref.py b/Lib/weakref.py
--- a/Lib/weakref.py
+++ b/Lib/weakref.py
@@ -23,7 +23,6 @@
 import collections  # Import after _weakref to avoid circular import.
 import sys
 import itertools
-import atexit
 
 ProxyTypes = (ProxyType, CallableProxyType)
 
@@ -464,11 +463,18 @@
     _shutdown = False
     _index_iter = itertools.count()
     _dirty = False
+    _registered_with_atexit = False
 
     class _Info:
         __slots__ = ("weakref", "func", "args", "kwargs", "atexit", "index")
 
     def __init__(self, obj, func, *args, **kwargs):
+        if not self._registered_with_atexit:
+            # We may register the exit function more than once because
+            # of a thread race, but that is harmless
+            import atexit
+            atexit.register(self._exitfunc)
+            finalize._registered_with_atexit = True
         info = self._Info()
         info.weakref = ref(obj, self)
         info.func = func
@@ -569,5 +575,3 @@
             finalize._shutdown = True
             if reenable_gc:
                 gc.enable()
-
-atexit.register(finalize._exitfunc)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  8 23:10:25 2013
From: python-checkins at python.org (ezio.melotti)
Date: Sat,  8 Jun 2013 23:10:25 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE3NjkxOiB0ZXN0?=
 =?utf-8?q?=5Funivnewlines_now_works_with_unittest_test_discovery=2E__Patc?=
 =?utf-8?q?h_by?=
Message-ID: <3bSYD14KsYz7Ljl@mail.python.org>

http://hg.python.org/cpython/rev/a2e093e98d45
changeset:   84058:a2e093e98d45
branch:      3.3
parent:      84055:2fe64ce5da05
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sun Jun 09 00:07:06 2013 +0300
summary:
  #17691: test_univnewlines now works with unittest test discovery.  Patch by Zachary Ware.

files:
  Lib/test/test_univnewlines.py |  37 +++++++++-------------
  Misc/NEWS                     |   3 +
  2 files changed, 19 insertions(+), 21 deletions(-)


diff --git a/Lib/test/test_univnewlines.py b/Lib/test/test_univnewlines.py
--- a/Lib/test/test_univnewlines.py
+++ b/Lib/test/test_univnewlines.py
@@ -30,7 +30,13 @@
 DATA_MIXED = "\n".join(DATA_TEMPLATE) + "\r"
 DATA_SPLIT = [x + "\n" for x in DATA_TEMPLATE]
 
-class TestGenericUnivNewlines(unittest.TestCase):
+class CTest:
+    open = io.open
+
+class PyTest:
+    open = staticmethod(pyio.open)
+
+class TestGenericUnivNewlines:
     # use a class variable DATA to define the data to write to the file
     # and a class variable NEWLINE to set the expected newlines value
     READMODE = 'r'
@@ -85,10 +91,14 @@
 class TestCRNewlines(TestGenericUnivNewlines):
     NEWLINE = '\r'
     DATA = DATA_CR
+class CTestCRNewlines(CTest, TestCRNewlines, unittest.TestCase): pass
+class PyTestCRNewlines(PyTest, TestCRNewlines, unittest.TestCase): pass
 
 class TestLFNewlines(TestGenericUnivNewlines):
     NEWLINE = '\n'
     DATA = DATA_LF
+class CTestLFNewlines(CTest, TestLFNewlines, unittest.TestCase): pass
+class PyTestLFNewlines(PyTest, TestLFNewlines, unittest.TestCase): pass
 
 class TestCRLFNewlines(TestGenericUnivNewlines):
     NEWLINE = '\r\n'
@@ -100,29 +110,14 @@
             data = fp.readline()
             pos = fp.tell()
         self.assertEqual(repr(fp.newlines), repr(self.NEWLINE))
+class CTestCRLFNewlines(CTest, TestCRLFNewlines, unittest.TestCase): pass
+class PyTestCRLFNewlines(PyTest, TestCRLFNewlines, unittest.TestCase): pass
 
 class TestMixedNewlines(TestGenericUnivNewlines):
     NEWLINE = ('\r', '\n')
     DATA = DATA_MIXED
-
-
-def test_main():
-    base_tests = (TestCRNewlines,
-                  TestLFNewlines,
-                  TestCRLFNewlines,
-                  TestMixedNewlines)
-    tests = []
-    # Test the C and Python implementations.
-    for test in base_tests:
-        class CTest(test):
-            open = io.open
-        CTest.__name__ = "C" + test.__name__
-        class PyTest(test):
-            open = staticmethod(pyio.open)
-        PyTest.__name__ = "Py" + test.__name__
-        tests.append(CTest)
-        tests.append(PyTest)
-    support.run_unittest(*tests)
+class CTestMixedNewlines(CTest, TestMixedNewlines, unittest.TestCase): pass
+class PyTestMixedNewlines(PyTest, TestMixedNewlines, unittest.TestCase): pass
 
 if __name__ == '__main__':
-    test_main()
+    unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -84,6 +84,9 @@
 Tests
 -----
 
+- Issue #17691: test_univnewlines now works with unittest test discovery.
+  Patch by Zachary Ware.
+
 - Issue #18094: test_uuid no more reports skipped tests as passed.
 
 - Issue #11995: test_pydoc doesn't import all sys.path modules anymore.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun  8 23:10:26 2013
From: python-checkins at python.org (ezio.melotti)
Date: Sat,  8 Jun 2013 23:10:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogIzE3NjkxOiBtZXJnZSB3aXRoIDMuMy4=?=
Message-ID: <3bSYD26Fy0z7Lkd@mail.python.org>

http://hg.python.org/cpython/rev/d1b5e41acf28
changeset:   84059:d1b5e41acf28
parent:      84057:d6ad9d7468f7
parent:      84058:a2e093e98d45
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sun Jun 09 00:10:04 2013 +0300
summary:
  #17691: merge with 3.3.

files:
  Lib/test/test_univnewlines.py |  37 +++++++++-------------
  Misc/NEWS                     |   3 +
  2 files changed, 19 insertions(+), 21 deletions(-)


diff --git a/Lib/test/test_univnewlines.py b/Lib/test/test_univnewlines.py
--- a/Lib/test/test_univnewlines.py
+++ b/Lib/test/test_univnewlines.py
@@ -30,7 +30,13 @@
 DATA_MIXED = "\n".join(DATA_TEMPLATE) + "\r"
 DATA_SPLIT = [x + "\n" for x in DATA_TEMPLATE]
 
-class TestGenericUnivNewlines(unittest.TestCase):
+class CTest:
+    open = io.open
+
+class PyTest:
+    open = staticmethod(pyio.open)
+
+class TestGenericUnivNewlines:
     # use a class variable DATA to define the data to write to the file
     # and a class variable NEWLINE to set the expected newlines value
     READMODE = 'r'
@@ -85,10 +91,14 @@
 class TestCRNewlines(TestGenericUnivNewlines):
     NEWLINE = '\r'
     DATA = DATA_CR
+class CTestCRNewlines(CTest, TestCRNewlines, unittest.TestCase): pass
+class PyTestCRNewlines(PyTest, TestCRNewlines, unittest.TestCase): pass
 
 class TestLFNewlines(TestGenericUnivNewlines):
     NEWLINE = '\n'
     DATA = DATA_LF
+class CTestLFNewlines(CTest, TestLFNewlines, unittest.TestCase): pass
+class PyTestLFNewlines(PyTest, TestLFNewlines, unittest.TestCase): pass
 
 class TestCRLFNewlines(TestGenericUnivNewlines):
     NEWLINE = '\r\n'
@@ -100,29 +110,14 @@
             data = fp.readline()
             pos = fp.tell()
         self.assertEqual(repr(fp.newlines), repr(self.NEWLINE))
+class CTestCRLFNewlines(CTest, TestCRLFNewlines, unittest.TestCase): pass
+class PyTestCRLFNewlines(PyTest, TestCRLFNewlines, unittest.TestCase): pass
 
 class TestMixedNewlines(TestGenericUnivNewlines):
     NEWLINE = ('\r', '\n')
     DATA = DATA_MIXED
-
-
-def test_main():
-    base_tests = (TestCRNewlines,
-                  TestLFNewlines,
-                  TestCRLFNewlines,
-                  TestMixedNewlines)
-    tests = []
-    # Test the C and Python implementations.
-    for test in base_tests:
-        class CTest(test):
-            open = io.open
-        CTest.__name__ = "C" + test.__name__
-        class PyTest(test):
-            open = staticmethod(pyio.open)
-        PyTest.__name__ = "Py" + test.__name__
-        tests.append(CTest)
-        tests.append(PyTest)
-    support.run_unittest(*tests)
+class CTestMixedNewlines(CTest, TestMixedNewlines, unittest.TestCase): pass
+class PyTestMixedNewlines(PyTest, TestMixedNewlines, unittest.TestCase): pass
 
 if __name__ == '__main__':
-    test_main()
+    unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -379,6 +379,9 @@
 - Issue #12820: add tests for the xml.dom.minicompat module.
   Patch by John Chandler and Phil Connell.
 
+- Issue #17691: test_univnewlines now works with unittest test discovery.
+  Patch by Zachary Ware.
+
 - Issue #17790: test_set now works with unittest test discovery.
   Patch by Zachary Ware.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 00:05:37 2013
From: python-checkins at python.org (ezio.melotti)
Date: Sun,  9 Jun 2013 00:05:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTI2OiB1cGRh?=
 =?utf-8?q?te_NumPy_links_in_the_documentation=2E__Patch_by_Yury_V=2E_Zayt?=
 =?utf-8?q?sev=2E?=
Message-ID: <3bSZRj5hqyz7LlJ@mail.python.org>

http://hg.python.org/cpython/rev/dacd8f430e0e
changeset:   84060:dacd8f430e0e
branch:      2.7
parent:      84042:31a67a0194ec
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sun Jun 09 01:04:21 2013 +0300
summary:
  #18126: update NumPy links in the documentation.  Patch by Yury V. Zaytsev.

files:
  Doc/faq/programming.rst |  2 +-
  Doc/library/array.rst   |  6 ++----
  Misc/ACKS               |  1 +
  3 files changed, 4 insertions(+), 5 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -1204,7 +1204,7 @@
    A = [[None] * w for i in range(h)]
 
 Or, you can use an extension that provides a matrix datatype; `Numeric Python
-<http://numpy.scipy.org/>`_ is the best known.
+<http://www.numpy.org/>`_ is the best known.
 
 
 How do I apply a method to a sequence of objects?
diff --git a/Doc/library/array.rst b/Doc/library/array.rst
--- a/Doc/library/array.rst
+++ b/Doc/library/array.rst
@@ -268,9 +268,7 @@
       Packing and unpacking of External Data Representation (XDR) data as used in some
       remote procedure call systems.
 
-   `The Numerical Python Manual <http://numpy.sourceforge.net/numdoc/HTML/numdoc.htm>`_
+   `The Numerical Python Documentation <http://docs.scipy.org/doc/>`_
       The Numeric Python extension (NumPy) defines another array type; see
-      http://numpy.sourceforge.net/ for further information about Numerical Python.
-      (A PDF version of the NumPy manual is available at
-      http://numpy.sourceforge.net/numdoc/numdoc.pdf).
+      http://www.numpy.org/ for further information about Numerical Python.
 
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1139,6 +1139,7 @@
 Milan Zamazal
 Artur Zaprzala
 Mike Zarnstorff
+Yury V. Zaytsev
 Siebren van der Zee
 Nickolai Zeldovich
 Uwe Zessin

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 00:05:39 2013
From: python-checkins at python.org (ezio.melotti)
Date: Sun,  9 Jun 2013 00:05:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTI2OiB1cGRh?=
 =?utf-8?q?te_NumPy_links_in_the_documentation=2E__Patch_by_Yury_V=2E_Zayt?=
 =?utf-8?q?sev=2E?=
Message-ID: <3bSZRl0RGzz7LlK@mail.python.org>

http://hg.python.org/cpython/rev/aafa11c1dd61
changeset:   84061:aafa11c1dd61
branch:      3.3
parent:      84058:a2e093e98d45
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sun Jun 09 01:04:21 2013 +0300
summary:
  #18126: update NumPy links in the documentation.  Patch by Yury V. Zaytsev.

files:
  Doc/faq/programming.rst |  2 +-
  Doc/library/array.rst   |  6 ++----
  Misc/ACKS               |  1 +
  3 files changed, 4 insertions(+), 5 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -1092,7 +1092,7 @@
    A = [[None] * w for i in range(h)]
 
 Or, you can use an extension that provides a matrix datatype; `Numeric Python
-<http://numpy.scipy.org/>`_ is the best known.
+<http://www.numpy.org/>`_ is the best known.
 
 
 How do I apply a method to a sequence of objects?
diff --git a/Doc/library/array.rst b/Doc/library/array.rst
--- a/Doc/library/array.rst
+++ b/Doc/library/array.rst
@@ -271,9 +271,7 @@
       Packing and unpacking of External Data Representation (XDR) data as used in some
       remote procedure call systems.
 
-   `The Numerical Python Manual <http://numpy.sourceforge.net/numdoc/HTML/numdoc.htm>`_
+   `The Numerical Python Documentation <http://docs.scipy.org/doc/>`_
       The Numeric Python extension (NumPy) defines another array type; see
-      http://numpy.sourceforge.net/ for further information about Numerical Python.
-      (A PDF version of the NumPy manual is available at
-      http://numpy.sourceforge.net/numdoc/numdoc.pdf).
+      http://www.numpy.org/ for further information about Numerical Python.
 
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1361,6 +1361,7 @@
 Milan Zamazal
 Artur Zaprzala
 Mike Zarnstorff
+Yury V. Zaytsev
 Siebren van der Zee
 Nickolai Zeldovich
 Yuxiao Zeng

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 00:05:40 2013
From: python-checkins at python.org (ezio.melotti)
Date: Sun,  9 Jun 2013 00:05:40 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogIzE4MTI2OiBtZXJnZSB3aXRoIDMuMy4=?=
Message-ID: <3bSZRm2XsZz7Lkc@mail.python.org>

http://hg.python.org/cpython/rev/1f0b6462ea3c
changeset:   84062:1f0b6462ea3c
parent:      84059:d1b5e41acf28
parent:      84061:aafa11c1dd61
user:        Ezio Melotti <ezio.melotti at gmail.com>
date:        Sun Jun 09 01:05:16 2013 +0300
summary:
  #18126: merge with 3.3.

files:
  Doc/faq/programming.rst |  2 +-
  Doc/library/array.rst   |  6 ++----
  Misc/ACKS               |  1 +
  3 files changed, 4 insertions(+), 5 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -1092,7 +1092,7 @@
    A = [[None] * w for i in range(h)]
 
 Or, you can use an extension that provides a matrix datatype; `Numeric Python
-<http://numpy.scipy.org/>`_ is the best known.
+<http://www.numpy.org/>`_ is the best known.
 
 
 How do I apply a method to a sequence of objects?
diff --git a/Doc/library/array.rst b/Doc/library/array.rst
--- a/Doc/library/array.rst
+++ b/Doc/library/array.rst
@@ -271,9 +271,7 @@
       Packing and unpacking of External Data Representation (XDR) data as used in some
       remote procedure call systems.
 
-   `The Numerical Python Manual <http://numpy.sourceforge.net/numdoc/HTML/numdoc.htm>`_
+   `The Numerical Python Documentation <http://docs.scipy.org/doc/>`_
       The Numeric Python extension (NumPy) defines another array type; see
-      http://numpy.sourceforge.net/ for further information about Numerical Python.
-      (A PDF version of the NumPy manual is available at
-      http://numpy.sourceforge.net/numdoc/numdoc.pdf).
+      http://www.numpy.org/ for further information about Numerical Python.
 
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1395,6 +1395,7 @@
 Milan Zamazal
 Artur Zaprzala
 Mike Zarnstorff
+Yury V. Zaytsev
 Siebren van der Zee
 Nickolai Zeldovich
 Yuxiao Zeng

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sun Jun  9 05:48:30 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 09 Jun 2013 05:48:30 +0200
Subject: [Python-checkins] Daily reference leaks (1f0b6462ea3c): sum=0
Message-ID: <E1UlWcI-0005CH-AO@ap.vmr.nerim.net>

results for 1f0b6462ea3c on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogQt4MKO', '-x']

From python-checkins at python.org  Sun Jun  9 15:59:01 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 15:59:01 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MDM4?=
 =?utf-8?q?=3A_SyntaxError_raised_during_compilation_sources_with_illegal?=
Message-ID: <3bSzbn5NVSz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/15aa786b723b
changeset:   84063:15aa786b723b
branch:      3.3
parent:      84061:aafa11c1dd61
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 16:51:52 2013 +0300
summary:
  Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.

files:
  Lib/test/test_pep263.py |  18 ++++++++++++++++++
  Misc/NEWS               |   3 +++
  Parser/tokenizer.c      |  14 +++++++-------
  3 files changed, 28 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_pep263.py b/Lib/test/test_pep263.py
--- a/Lib/test/test_pep263.py
+++ b/Lib/test/test_pep263.py
@@ -55,6 +55,24 @@
         # two bytes in common with the UTF-8 BOM
         self.assertRaises(SyntaxError, eval, b'\xef\xbb\x20')
 
+    def test_error_message(self):
+        compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
+        compile(b'\xef\xbb\xbf\n', 'dummy', 'exec')
+        compile(b'\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile(b'# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
+            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+
+
 def test_main():
     support.run_unittest(PEP263Test)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18038: SyntaxError raised during compilation sources with illegal
+  encoding now always contains an encoding name.
+
 - Issue #17644: Fix a crash in str.format when curly braces are used in square
   brackets.
 
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -291,20 +291,20 @@
                     tok->encoding = cs;
                     tok->decoding_state = STATE_NORMAL;
                 }
-                else
+                else {
+                    PyErr_Format(PyExc_SyntaxError,
+                                 "encoding problem: %s", cs);
                     PyMem_FREE(cs);
+                }
             }
         } else {                /* then, compare cs with BOM */
             r = (strcmp(tok->encoding, cs) == 0);
+            if (!r)
+                PyErr_Format(PyExc_SyntaxError,
+                             "encoding problem: %s with BOM", cs);
             PyMem_FREE(cs);
         }
     }
-    if (!r) {
-        cs = tok->encoding;
-        if (!cs)
-            cs = "with BOM";
-        PyErr_Format(PyExc_SyntaxError, "encoding problem: %s", cs);
-    }
     return r;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 15:59:03 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 15:59:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318038=3A_SyntaxError_raised_during_compilation_?=
 =?utf-8?q?sources_with_illegal?=
Message-ID: <3bSzbq0Hgtz7LmY@mail.python.org>

http://hg.python.org/cpython/rev/39e2f0059ee2
changeset:   84064:39e2f0059ee2
parent:      84062:1f0b6462ea3c
parent:      84063:15aa786b723b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 16:53:55 2013 +0300
summary:
  Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.

files:
  Lib/test/test_pep263.py |  18 ++++++++++++++++++
  Misc/NEWS               |   3 +++
  Parser/tokenizer.c      |  14 +++++++-------
  3 files changed, 28 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_pep263.py b/Lib/test/test_pep263.py
--- a/Lib/test/test_pep263.py
+++ b/Lib/test/test_pep263.py
@@ -55,6 +55,24 @@
         # two bytes in common with the UTF-8 BOM
         self.assertRaises(SyntaxError, eval, b'\xef\xbb\x20')
 
+    def test_error_message(self):
+        compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
+        compile(b'\xef\xbb\xbf\n', 'dummy', 'exec')
+        compile(b'\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile(b'# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
+            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+
+
 def test_main():
     support.run_unittest(PEP263Test)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18038: SyntaxError raised during compilation sources with illegal
+  encoding now always contains an encoding name.
+
 - Issue #17931: Resolve confusion on Windows between pids and process
   handles.
 
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -291,20 +291,20 @@
                     tok->encoding = cs;
                     tok->decoding_state = STATE_NORMAL;
                 }
-                else
+                else {
+                    PyErr_Format(PyExc_SyntaxError,
+                                 "encoding problem: %s", cs);
                     PyMem_FREE(cs);
+                }
             }
         } else {                /* then, compare cs with BOM */
             r = (strcmp(tok->encoding, cs) == 0);
+            if (!r)
+                PyErr_Format(PyExc_SyntaxError,
+                             "encoding problem: %s with BOM", cs);
             PyMem_FREE(cs);
         }
     }
-    if (!r) {
-        cs = tok->encoding;
-        if (!cs)
-            cs = "with BOM";
-        PyErr_Format(PyExc_SyntaxError, "encoding problem: %s", cs);
-    }
     return r;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 15:59:04 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 15:59:04 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MDM4?=
 =?utf-8?q?=3A_SyntaxError_raised_during_compilation_sources_with_illegal?=
Message-ID: <3bSzbr2cCMz7LlR@mail.python.org>

http://hg.python.org/cpython/rev/570b5b4040b1
changeset:   84065:570b5b4040b1
branch:      2.7
parent:      84060:dacd8f430e0e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 16:54:56 2013 +0300
summary:
  Issue #18038: SyntaxError raised during compilation sources with illegal
encoding now always contains an encoding name.

files:
  Lib/test/test_pep263.py |  18 ++++++++++++++++++
  Misc/NEWS               |   3 +++
  Parser/tokenizer.c      |  14 +++++++-------
  3 files changed, 28 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_pep263.py b/Lib/test/test_pep263.py
--- a/Lib/test/test_pep263.py
+++ b/Lib/test/test_pep263.py
@@ -41,6 +41,24 @@
         # two bytes in common with the UTF-8 BOM
         self.assertRaises(SyntaxError, eval, '\xef\xbb\x20')
 
+    def test_error_message(self):
+        compile('# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
+        compile('\xef\xbb\xbf\n', 'dummy', 'exec')
+        compile('\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile('# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
+            compile('\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile('\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile('\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile('\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+
+
 def test_main():
     test_support.run_unittest(PEP263Test)
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -9,6 +9,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18038: SyntaxError raised during compilation sources with illegal
+  encoding now always contains an encoding name.
+
 - Issue #18019: Fix crash in the repr of dictionaries containing their own
   views.
 
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -277,8 +277,11 @@
                     tok->encoding = cs;
                     tok->decoding_state = -1;
                 }
-                else
+                else {
+                    PyErr_Format(PyExc_SyntaxError,
+                                 "encoding problem: %s", cs);
                     PyMem_FREE(cs);
+                }
 #else
                 /* Without Unicode support, we cannot
                    process the coding spec. Since there
@@ -289,15 +292,12 @@
             }
         } else {                /* then, compare cs with BOM */
             r = (strcmp(tok->encoding, cs) == 0);
+            if (!r)
+                PyErr_Format(PyExc_SyntaxError,
+                             "encoding problem: %s with BOM", cs);
             PyMem_FREE(cs);
         }
     }
-    if (!r) {
-        cs = tok->encoding;
-        if (!cs)
-            cs = "with BOM";
-        PyErr_Format(PyExc_SyntaxError, "encoding problem: %s", cs);
-    }
     return r;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 16:13:11 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 16:13:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE1MjM5?=
 =?utf-8?q?=3A_Make_mkstringprep=2Epy_work_again_on_Python_3=2E?=
Message-ID: <3bSzw73ZW0z7Ll1@mail.python.org>

http://hg.python.org/cpython/rev/8f95d77443da
changeset:   84066:8f95d77443da
branch:      3.3
parent:      84063:15aa786b723b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 17:08:00 2013 +0300
summary:
  Issue #15239: Make mkstringprep.py work again on Python 3.

files:
  Misc/NEWS                     |   5 ++
  Tools/unicode/mkstringprep.py |  40 +++++++++++++---------
  2 files changed, 28 insertions(+), 17 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -107,6 +107,11 @@
 - Issue #17977: The documentation for the cadefault argument's default value
   in urllib.request.urlopen() is fixed to match the code.
 
+Tools/Demos
+-----------
+
+- Issue #15239: Make mkstringprep.py work again on Python 3.
+
 
 What's New in Python 3.3.2?
 ===========================
diff --git a/Tools/unicode/mkstringprep.py b/Tools/unicode/mkstringprep.py
--- a/Tools/unicode/mkstringprep.py
+++ b/Tools/unicode/mkstringprep.py
@@ -1,4 +1,5 @@
-import re, unicodedata, sys
+import re, sys
+from unicodedata import ucd_3_2_0 as unicodedata
 
 if sys.maxunicode == 65535:
     raise RuntimeError("need UCS-4 Python")
@@ -37,16 +38,20 @@
         tuple.append((prev,prev+span+1))
     else:
         single.append(prev)
-    tuple = " + ".join(["list(range(%d,%d))" % t for t in tuple])
+    if not single and len(tuple) == 1:
+        tuple = "range(%d,%d)" % tuple[0]
+    else:
+        tuple = " + ".join("list(range(%d,%d))" % t for t in tuple)
     if not single:
         return "set(%s)" % tuple
     if not tuple:
-        return "set(%s)" % repr(single)
-    return "set(%s + %s)" % (repr(single),tuple)
+        return "set(%r)" % (single,)
+    return "set(%r + %s)" % (single, tuple)
 
 ############## Read the tables in the RFC #######################
 
-data = open("rfc3454.txt").readlines()
+with open("rfc3454.txt") as f:
+    data = f.readlines()
 
 tables = []
 curname = None
@@ -55,8 +60,7 @@
     if not l:
         continue
     # Skip RFC page breaks
-    if l.startswith("Hoffman & Blanchet") or\
-       l.startswith("RFC 3454"):
+    if l.startswith(("Hoffman & Blanchet", "RFC 3454")):
         continue
     # Find start/end lines
     m = re.match("----- (Start|End) Table ([A-Z](.[0-9])+) -----", l)
@@ -71,6 +75,8 @@
         else:
             if not curname:
                 raise RuntimeError("End without start", l)
+            if curname != m.group(2):
+                raise RuntimeError("Unexpected end", l)
             curname = None
             continue
     if not curname:
@@ -113,10 +119,10 @@
 and mappings, for which a mapping function is provided.
 \"\"\"
 
-import unicodedata
+from unicodedata import ucd_3_2_0 as unicodedata
 """)
 
-print("assert unicodedata.unidata_version == %s" % repr(unicodedata.unidata_version))
+print("assert unicodedata.unidata_version == %r" % (unicodedata.unidata_version,))
 
 # A.1 is the table of unassigned characters
 # XXX Plane 15 PUA is listed as unassigned in Python.
@@ -173,15 +179,15 @@
 b3_exceptions = {}
 
 for k,v in table_b2.items():
-    if map(ord, unichr(k).lower()) != v:
-        b3_exceptions[k] = u"".join(map(unichr,v))
+    if list(map(ord, chr(k).lower())) != v:
+        b3_exceptions[k] = "".join(map(chr,v))
 
 b3 = sorted(b3_exceptions.items())
 
 print("""
 b3_exceptions = {""")
-for i,(k,v) in enumerate(b3):
-    print("0x%x:%s," % (k, repr(v)), end=' ')
+for i, kv in enumerate(b3):
+    print("0x%x:%a," % kv, end=' ')
     if i % 4 == 3:
         print()
 print("}")
@@ -224,7 +230,7 @@
 def map_table_b2(a):
     al = map_table_b3(a)
     b = unicodedata.normalize("NFKC", al)
-    bl = u"".join([map_table_b3(ch) for ch in b])
+    bl = "".join([map_table_b3(ch) for ch in b])
     c = unicodedata.normalize("NFKC", bl)
     if b != c:
         return c
@@ -240,7 +246,7 @@
 
 print("""
 def in_table_c11(code):
-    return code == u" "
+    return code == " "
 """)
 
 # C.1.2 is the rest of all space characters
@@ -249,12 +255,12 @@
 assert name == "C.1.2"
 
 # table = set(table.keys())
-# Zs = set(gen_category(["Zs"])) - set([0x20])
+# Zs = set(gen_category(["Zs"])) - {0x20}
 # assert Zs == table
 
 print("""
 def in_table_c12(code):
-    return unicodedata.category(code) == "Zs" and code != u" "
+    return unicodedata.category(code) == "Zs" and code != " "
 
 def in_table_c11_c12(code):
     return unicodedata.category(code) == "Zs"

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 16:13:12 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 16:13:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2315239=3A_Make_mkstringprep=2Epy_work_again_on_P?=
 =?utf-8?q?ython_3=2E?=
Message-ID: <3bSzw86cWgz7Lm5@mail.python.org>

http://hg.python.org/cpython/rev/4abe61a412be
changeset:   84067:4abe61a412be
parent:      84064:39e2f0059ee2
parent:      84066:8f95d77443da
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 17:11:48 2013 +0300
summary:
  Issue #15239: Make mkstringprep.py work again on Python 3.

files:
  Misc/NEWS                     |   2 +
  Tools/unicode/mkstringprep.py |  40 +++++++++++++---------
  2 files changed, 25 insertions(+), 17 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1787,6 +1787,8 @@
 Tools/Demos
 -----------
 
+- Issue #15239: Make mkstringprep.py work again on Python 3.
+
 - Issue #17028: Allowed Python arguments to be supplied to the Windows
   launcher.
 
diff --git a/Tools/unicode/mkstringprep.py b/Tools/unicode/mkstringprep.py
--- a/Tools/unicode/mkstringprep.py
+++ b/Tools/unicode/mkstringprep.py
@@ -1,4 +1,5 @@
-import re, unicodedata, sys
+import re, sys
+from unicodedata import ucd_3_2_0 as unicodedata
 
 if sys.maxunicode == 65535:
     raise RuntimeError("need UCS-4 Python")
@@ -37,16 +38,20 @@
         tuple.append((prev,prev+span+1))
     else:
         single.append(prev)
-    tuple = " + ".join(["list(range(%d,%d))" % t for t in tuple])
+    if not single and len(tuple) == 1:
+        tuple = "range(%d,%d)" % tuple[0]
+    else:
+        tuple = " + ".join("list(range(%d,%d))" % t for t in tuple)
     if not single:
         return "set(%s)" % tuple
     if not tuple:
-        return "set(%s)" % repr(single)
-    return "set(%s + %s)" % (repr(single),tuple)
+        return "set(%r)" % (single,)
+    return "set(%r + %s)" % (single, tuple)
 
 ############## Read the tables in the RFC #######################
 
-data = open("rfc3454.txt").readlines()
+with open("rfc3454.txt") as f:
+    data = f.readlines()
 
 tables = []
 curname = None
@@ -55,8 +60,7 @@
     if not l:
         continue
     # Skip RFC page breaks
-    if l.startswith("Hoffman & Blanchet") or\
-       l.startswith("RFC 3454"):
+    if l.startswith(("Hoffman & Blanchet", "RFC 3454")):
         continue
     # Find start/end lines
     m = re.match("----- (Start|End) Table ([A-Z](.[0-9])+) -----", l)
@@ -71,6 +75,8 @@
         else:
             if not curname:
                 raise RuntimeError("End without start", l)
+            if curname != m.group(2):
+                raise RuntimeError("Unexpected end", l)
             curname = None
             continue
     if not curname:
@@ -113,10 +119,10 @@
 and mappings, for which a mapping function is provided.
 \"\"\"
 
-import unicodedata
+from unicodedata import ucd_3_2_0 as unicodedata
 """)
 
-print("assert unicodedata.unidata_version == %s" % repr(unicodedata.unidata_version))
+print("assert unicodedata.unidata_version == %r" % (unicodedata.unidata_version,))
 
 # A.1 is the table of unassigned characters
 # XXX Plane 15 PUA is listed as unassigned in Python.
@@ -173,15 +179,15 @@
 b3_exceptions = {}
 
 for k,v in table_b2.items():
-    if map(ord, unichr(k).lower()) != v:
-        b3_exceptions[k] = u"".join(map(unichr,v))
+    if list(map(ord, chr(k).lower())) != v:
+        b3_exceptions[k] = "".join(map(chr,v))
 
 b3 = sorted(b3_exceptions.items())
 
 print("""
 b3_exceptions = {""")
-for i,(k,v) in enumerate(b3):
-    print("0x%x:%s," % (k, repr(v)), end=' ')
+for i, kv in enumerate(b3):
+    print("0x%x:%a," % kv, end=' ')
     if i % 4 == 3:
         print()
 print("}")
@@ -224,7 +230,7 @@
 def map_table_b2(a):
     al = map_table_b3(a)
     b = unicodedata.normalize("NFKC", al)
-    bl = u"".join([map_table_b3(ch) for ch in b])
+    bl = "".join([map_table_b3(ch) for ch in b])
     c = unicodedata.normalize("NFKC", bl)
     if b != c:
         return c
@@ -240,7 +246,7 @@
 
 print("""
 def in_table_c11(code):
-    return code == u" "
+    return code == " "
 """)
 
 # C.1.2 is the rest of all space characters
@@ -249,12 +255,12 @@
 assert name == "C.1.2"
 
 # table = set(table.keys())
-# Zs = set(gen_category(["Zs"])) - set([0x20])
+# Zs = set(gen_category(["Zs"])) - {0x20}
 # assert Zs == table
 
 print("""
 def in_table_c12(code):
-    return unicodedata.category(code) == "Zs" and code != u" "
+    return unicodedata.category(code) == "Zs" and code != " "
 
 def in_table_c11_c12(code):
     return unicodedata.category(code) == "Zs"

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 18:03:10 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun,  9 Jun 2013 18:03:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318143=3A_Implemen?=
 =?utf-8?q?t_ssl=2Eget=5Fdefault=5Fverify=5Fpaths=28=29_in_order_to_debug?=
Message-ID: <3bT2M26dvtzSSY@mail.python.org>

http://hg.python.org/cpython/rev/a4d31e56075d
changeset:   84068:a4d31e56075d
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 09 18:02:55 2013 +0200
summary:
  Issue #18143: Implement ssl.get_default_verify_paths() in order to debug
the default locations for cafile and capath.

files:
  Doc/library/ssl.rst  |  20 ++++++++++++++-
  Lib/ssl.py           |  20 +++++++++++++++
  Lib/test/test_ssl.py |  13 +++++++++
  Misc/NEWS            |   3 ++
  Modules/_ssl.c       |  42 ++++++++++++++++++++++++++++++++
  5 files changed, 97 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -343,6 +343,23 @@
    Given a certificate as an ASCII PEM string, returns a DER-encoded sequence of
    bytes for that same certificate.
 
+.. function:: get_default_verify_paths()
+
+   Returns a named tuple with paths to OpenSSL's default cafile and capath.
+   The paths are the same as used by
+   :meth:`SSLContext.set_default_verify_paths`. The return value is a
+   :term:`named tuple` ``DefaultVerifyPaths``:
+
+   * :attr:`cafile` - resolved path to cafile or None if the file doesn't exist,
+   * :attr:`capath` - resolved path to capath or None if the directory doesn't exist,
+   * :attr:`openssl_cafile_env` - OpenSSL's environment key that points to a cafile,
+   * :attr:`openssl_cafile` - hard coded path to a cafile,
+   * :attr:`openssl_capath_env` - OpenSSL's environment key that points to a capath,
+   * :attr:`openssl_capath` - hard coded path to a capath directory
+
+   .. versionadded:: 3.4
+
+
 Constants
 ^^^^^^^^^
 
@@ -787,7 +804,8 @@
    other peers' certificates when :data:`verify_mode` is other than
    :data:`CERT_NONE`.  At least one of *cafile* or *capath* must be specified.
 
-   The *cafile* string, if present, is the path to a file of concatenated
+   The *cafile* string, if present, is the p
+   ath to a file of concatenated
    CA certificates in PEM format. See the discussion of
    :ref:`ssl-certificates` for more information about how to arrange the
    certificates in this file.
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -89,6 +89,8 @@
 
 import textwrap
 import re
+import os
+import collections
 
 import _ssl             # if we can't import it, let the error propagate
 
@@ -222,6 +224,24 @@
             "subjectAltName fields were found")
 
 
+DefaultVerifyPaths = collections.namedtuple("DefaultVerifyPaths",
+    "cafile capath openssl_cafile_env openssl_cafile openssl_capath_env "
+    "openssl_capath")
+
+def get_default_verify_paths():
+    """Return paths to default cafile and capath.
+    """
+    parts = _ssl.get_default_verify_paths()
+
+    # environment vars shadow paths
+    cafile = os.environ.get(parts[0], parts[1])
+    capath = os.environ.get(parts[2], parts[3])
+
+    return DefaultVerifyPaths(cafile if os.path.isfile(cafile) else None,
+                              capath if os.path.isdir(capath) else None,
+                              *parts)
+
+
 class SSLContext(_SSLContext):
     """An SSLContext holds various SSL-related configuration options and
     data, such as certificates and possibly a private key."""
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -394,6 +394,19 @@
             support.gc_collect()
         self.assertIn(r, str(cm.warning.args[0]))
 
+    def test_get_default_verify_paths(self):
+        paths = ssl.get_default_verify_paths()
+        self.assertEqual(len(paths), 6)
+        self.assertIsInstance(paths, ssl.DefaultVerifyPaths)
+
+        with support.EnvironmentVarGuard() as env:
+            env["SSL_CERT_DIR"] = CAPATH
+            env["SSL_CERT_FILE"] = CERTFILE
+            paths = ssl.get_default_verify_paths()
+            self.assertEqual(paths.cafile, CERTFILE)
+            self.assertEqual(paths.capath, CAPATH)
+
+
 class ContextTests(unittest.TestCase):
 
     @skip_if_broken_ubuntu_ssl
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -115,6 +115,9 @@
 Library
 -------
 
+- Issue #18143: Implement ssl.get_default_verify_paths() in order to debug
+  the default locations for cafile and capath.
+
 - Issue #17314: Move multiprocessing.forking over to importlib.
 
 - Issue #11959: SMTPServer and SMTPChannel now take an optional map, use of
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2761,6 +2761,46 @@
 
 #endif
 
+PyDoc_STRVAR(PySSL_get_default_verify_paths_doc,
+"get_default_verify_paths() -> tuple\n\
+\n\
+Return search paths and environment vars that are used by SSLContext's\n\
+set_default_verify_paths() to load default CAs. The values are\n\
+'cert_file_env', 'cert_file', 'cert_dir_env', 'cert_dir'.");
+
+static PyObject *
+get_default_verify_paths(PyObject *self)
+{
+    PyObject *ofile_env = NULL;
+    PyObject *ofile = NULL;
+    PyObject *odir_env = NULL;
+    PyObject *odir = NULL;
+
+#define convert(info, target) { \
+        const char *tmp = (info); \
+        target = NULL; \
+        if (!tmp) { Py_INCREF(Py_None); target = Py_None; } \
+        else if ((target = PyUnicode_DecodeFSDefault(tmp)) == NULL) { \
+            target = PyBytes_FromString(tmp); } \
+        if (!target) goto error; \
+    } while(0)
+
+    convert(X509_get_default_cert_file_env(), ofile_env);
+    convert(X509_get_default_cert_file(), ofile);
+    convert(X509_get_default_cert_dir_env(), odir_env);
+    convert(X509_get_default_cert_dir(), odir);
+#undef convert
+
+    return Py_BuildValue("(OOOO)", ofile_env, ofile, odir_env, odir);
+
+  error:
+    Py_XDECREF(ofile_env);
+    Py_XDECREF(ofile);
+    Py_XDECREF(odir_env);
+    Py_XDECREF(odir);
+    return NULL;
+}
+
 
 
 /* List of functions exported by this module. */
@@ -2779,6 +2819,8 @@
      PySSL_RAND_egd_doc},
     {"RAND_status",         (PyCFunction)PySSL_RAND_status, METH_NOARGS,
      PySSL_RAND_status_doc},
+    {"get_default_verify_paths", (PyCFunction)get_default_verify_paths,
+     METH_NOARGS, PySSL_get_default_verify_paths_doc},
 #endif
     {NULL,                  NULL}            /* Sentinel */
 };

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 18:07:25 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun,  9 Jun 2013 18:07:25 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_removed_accidental_new_lin?=
	=?utf-8?q?e?=
Message-ID: <3bT2Rx5cH7z7Ljh@mail.python.org>

http://hg.python.org/cpython/rev/716e8b23974a
changeset:   84069:716e8b23974a
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 09 18:07:16 2013 +0200
summary:
  removed accidental new line

files:
  Doc/library/ssl.rst |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -804,8 +804,7 @@
    other peers' certificates when :data:`verify_mode` is other than
    :data:`CERT_NONE`.  At least one of *cafile* or *capath* must be specified.
 
-   The *cafile* string, if present, is the p
-   ath to a file of concatenated
+   The *cafile* string, if present, is the path to a file of concatenated
    CA certificates in PEM format. See the discussion of
    :ref:`ssl-certificates` for more information about how to arrange the
    certificates in this file.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 18:30:03 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun,  9 Jun 2013 18:30:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_get=5Fdefault=5Fverify=5Fp?=
 =?utf-8?q?aths_doesn=27t_belong_inside_the_ifdef_block?=
Message-ID: <3bT2y30qsfzT1s@mail.python.org>

http://hg.python.org/cpython/rev/4fdbe5d05922
changeset:   84070:4fdbe5d05922
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 09 18:29:54 2013 +0200
summary:
  get_default_verify_paths doesn't belong inside the ifdef block

files:
  Modules/_ssl.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2819,9 +2819,9 @@
      PySSL_RAND_egd_doc},
     {"RAND_status",         (PyCFunction)PySSL_RAND_status, METH_NOARGS,
      PySSL_RAND_status_doc},
+#endif
     {"get_default_verify_paths", (PyCFunction)get_default_verify_paths,
      METH_NOARGS, PySSL_get_default_verify_paths_doc},
-#endif
     {NULL,                  NULL}            /* Sentinel */
 };
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 19:03:42 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun,  9 Jun 2013 19:03:42 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317134=3A_Add_ssl?=
 =?utf-8?q?=2Eenum=5Fcert=5Fstore=28=29_as_interface_to_Windows=27_cert_st?=
 =?utf-8?q?ore=2E?=
Message-ID: <3bT3ht1TnTzSRJ@mail.python.org>

http://hg.python.org/cpython/rev/10d325f674f5
changeset:   84071:10d325f674f5
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 09 19:03:31 2013 +0200
summary:
  Issue #17134: Add ssl.enum_cert_store() as interface to Windows' cert store.

files:
  Doc/library/ssl.rst     |   23 ++++
  Lib/ssl.py              |    4 +
  Lib/test/test_ssl.py    |   23 ++++
  Misc/NEWS               |    2 +
  Modules/_ssl.c          |  134 +++++++++++++++++++++++++++-
  PC/VS9.0/_socket.vcproj |   16 +-
  PCbuild/_ssl.vcxproj    |   16 +-
  7 files changed, 201 insertions(+), 17 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -359,6 +359,20 @@
 
    .. versionadded:: 3.4
 
+.. function:: enum_cert_store(store_name, cert_type='certificate')
+
+   Retrieve certificates from Windows' system cert store. *store_name* may be
+   one of ``CA``, ``ROOT`` or ``MY``. Windows may provide additional cert
+   stores, too. *cert_type* is either ``certificate`` for X.509 certificates
+   or ``crl`` for X.509 certificate revocation lists.
+
+   The function returns a list of (bytes, encoding_type) tuples. The
+   encoding_type flag can be interpreted with :const:`X509_ASN_ENCODING` or
+   :const:`PKCS_7_ASN_ENCODING`.
+
+   Availability: Windows.
+
+   .. versionadded:: 3.4
 
 Constants
 ^^^^^^^^^
@@ -598,6 +612,15 @@
 
    .. versionadded:: 3.4
 
+.. data:: X509_ASN_ENCODING
+          PKCS_7_ASN_ENCODING
+
+   Encoding flags for :func:`enum_cert_store`.
+
+   Availability: Windows.
+
+   .. versionadded:: 3.4
+
 
 SSL Sockets
 -----------
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -89,6 +89,7 @@
 
 import textwrap
 import re
+import sys
 import os
 import collections
 
@@ -139,6 +140,9 @@
     _PROTOCOL_NAMES[PROTOCOL_TLSv1_1] = "TLSv1.1"
     _PROTOCOL_NAMES[PROTOCOL_TLSv1_2] = "TLSv1.2"
 
+if sys.platform == "win32":
+    from _ssl import enum_cert_store, X509_ASN_ENCODING, PKCS_7_ASN_ENCODING
+
 from socket import getnameinfo as _getnameinfo
 from socket import socket, AF_INET, SOCK_STREAM, create_connection
 import base64        # for DER-to-PEM translation
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -407,6 +407,29 @@
             self.assertEqual(paths.capath, CAPATH)
 
 
+    @unittest.skipUnless(sys.platform == "win32", "Windows specific")
+    def test_enum_cert_store(self):
+        self.assertEqual(ssl.X509_ASN_ENCODING, 1)
+        self.assertEqual(ssl.PKCS_7_ASN_ENCODING, 0x00010000)
+
+        self.assertEqual(ssl.enum_cert_store("CA"),
+            ssl.enum_cert_store("CA", "certificate"))
+        ssl.enum_cert_store("CA", "crl")
+        self.assertEqual(ssl.enum_cert_store("ROOT"),
+            ssl.enum_cert_store("ROOT", "certificate"))
+        ssl.enum_cert_store("ROOT", "crl")
+
+        self.assertRaises(TypeError, ssl.enum_cert_store)
+        self.assertRaises(WindowsError, ssl.enum_cert_store, "")
+        self.assertRaises(ValueError, ssl.enum_cert_store, "CA", "wrong")
+
+        ca = ssl.enum_cert_store("CA")
+        self.assertIsInstance(ca, list)
+        self.assertIsInstance(ca[0], tuple)
+        self.assertEqual(len(ca[0]), 2)
+        self.assertIsInstance(ca[0][0], bytes)
+        self.assertIsInstance(ca[0][1], int)
+
 class ContextTests(unittest.TestCase):
 
     @skip_if_broken_ubuntu_ssl
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -115,6 +115,8 @@
 Library
 -------
 
+- Issue #17134: Add ssl.enum_cert_store() as interface to Windows' cert store.
+
 - Issue #18143: Implement ssl.get_default_verify_paths() in order to debug
   the default locations for cafile and capath.
 
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2801,7 +2801,129 @@
     return NULL;
 }
 
-
+#ifdef _MSC_VER
+PyDoc_STRVAR(PySSL_enum_cert_store_doc,
+"enum_cert_store(store_name, cert_type='certificate') -> []\n\
+\n\
+Retrieve certificates from Windows' cert store. store_name may be one of\n\
+'CA', 'ROOT' or 'MY'. The system may provide more cert storages, too.\n\
+cert_type must be either 'certificate' or 'crl'.\n\
+The function returns a list of (bytes, encoding_type) tuples. The\n\
+encoding_type flag can be interpreted with X509_ASN_ENCODING or\n\
+PKCS_7_ASN_ENCODING.");
+
+static PyObject *
+PySSL_enum_cert_store(PyObject *self, PyObject *args, PyObject *kwds)
+{
+    char *kwlist[] = {"store_name", "cert_type", NULL};
+    char *store_name;
+    char *cert_type = "certificate";
+    HCERTSTORE hStore = NULL;
+    PyObject *result = NULL;
+    PyObject *tup = NULL, *cert = NULL, *enc = NULL;
+    int ok = 1;
+
+    if (!PyArg_ParseTupleAndKeywords(args, kwds, "s|s:enum_cert_store",
+                                     kwlist, &store_name, &cert_type)) {
+            return NULL;
+    }
+
+    if ((strcmp(cert_type, "certificate") != 0) &&
+        (strcmp(cert_type, "crl") != 0)) {
+        return PyErr_Format(PyExc_ValueError,
+                            "cert_type must be 'certificate' or 'crl', "
+                            "not %.100s", cert_type);
+    }
+
+    if ((result = PyList_New(0)) == NULL) {
+        return NULL;
+    }
+
+    if ((hStore = CertOpenSystemStore(NULL, store_name)) == NULL) {
+        Py_DECREF(result);
+        return PyErr_SetFromWindowsErr(GetLastError());
+    }
+
+    if (strcmp(cert_type, "certificate") == 0) {
+        PCCERT_CONTEXT pCertCtx = NULL;
+        while (pCertCtx = CertEnumCertificatesInStore(hStore, pCertCtx)) {
+            cert = PyBytes_FromStringAndSize((const char*)pCertCtx->pbCertEncoded,
+                                             pCertCtx->cbCertEncoded);
+            if (!cert) {
+                ok = 0;
+                break;
+            }
+            if ((enc = PyLong_FromLong(pCertCtx->dwCertEncodingType)) == NULL) {
+                ok = 0;
+                break;
+            }
+            if ((tup = PyTuple_New(2)) == NULL) {
+                ok = 0;
+                break;
+            }
+            PyTuple_SET_ITEM(tup, 0, cert); cert = NULL;
+            PyTuple_SET_ITEM(tup, 1, enc); enc = NULL;
+
+            if (PyList_Append(result, tup) < 0) {
+                ok = 0;
+                break;
+            }
+            Py_CLEAR(tup);
+        }
+        if (pCertCtx) {
+            /* loop ended with an error, need to clean up context manually */
+            CertFreeCertificateContext(pCertCtx);
+        }
+    } else {
+        PCCRL_CONTEXT pCrlCtx = NULL;
+        while (pCrlCtx = CertEnumCRLsInStore(hStore, pCrlCtx)) {
+            cert = PyBytes_FromStringAndSize((const char*)pCrlCtx->pbCrlEncoded,
+                                             pCrlCtx->cbCrlEncoded);
+            if (!cert) {
+                ok = 0;
+                break;
+            }
+            if ((enc = PyLong_FromLong(pCrlCtx->dwCertEncodingType)) == NULL) {
+                ok = 0;
+                break;
+            }
+            if ((tup = PyTuple_New(2)) == NULL) {
+                ok = 0;
+                break;
+            }
+            PyTuple_SET_ITEM(tup, 0, cert); cert = NULL;
+            PyTuple_SET_ITEM(tup, 1, enc); enc = NULL;
+
+            if (PyList_Append(result, tup) < 0) {
+                ok = 0;
+                break;
+            }
+            Py_CLEAR(tup);
+        }
+        if (pCrlCtx) {
+            /* loop ended with an error, need to clean up context manually */
+            CertFreeCRLContext(pCrlCtx);
+        }
+    }
+
+    /* In error cases cert, enc and tup may not be NULL */
+    Py_XDECREF(cert);
+    Py_XDECREF(enc);
+    Py_XDECREF(tup);
+
+    if (!CertCloseStore(hStore, 0)) {
+        /* This error case might shadow another exception.*/
+        Py_DECREF(result);
+        return PyErr_SetFromWindowsErr(GetLastError());
+    }
+    if (ok) {
+        return result;
+    } else {
+        Py_DECREF(result);
+        return NULL;
+    }
+}
+#endif
 
 /* List of functions exported by this module. */
 
@@ -2822,6 +2944,10 @@
 #endif
     {"get_default_verify_paths", (PyCFunction)get_default_verify_paths,
      METH_NOARGS, PySSL_get_default_verify_paths_doc},
+#ifdef _MSC_VER
+    {"enum_cert_store", (PyCFunction)PySSL_enum_cert_store,
+     METH_VARARGS | METH_KEYWORDS, PySSL_enum_cert_store_doc},
+#endif
     {NULL,                  NULL}            /* Sentinel */
 };
 
@@ -3034,6 +3160,12 @@
     PyModule_AddIntConstant(m, "CERT_REQUIRED",
                             PY_SSL_CERT_REQUIRED);
 
+#ifdef _MSC_VER
+    /* Windows dwCertEncodingType */
+    PyModule_AddIntMacro(m, X509_ASN_ENCODING);
+    PyModule_AddIntMacro(m, PKCS_7_ASN_ENCODING);
+#endif
+
     /* Alert Descriptions from ssl.h */
     /* note RESERVED constants no longer intended for use have been removed */
     /* http://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-6 */
diff --git a/PC/VS9.0/_socket.vcproj b/PC/VS9.0/_socket.vcproj
--- a/PC/VS9.0/_socket.vcproj
+++ b/PC/VS9.0/_socket.vcproj
@@ -54,7 +54,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -115,7 +115,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -176,7 +176,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -238,7 +238,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -299,7 +299,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -361,7 +361,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 				TargetMachine="17"
 			/>
@@ -423,7 +423,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -485,7 +485,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib"
+				AdditionalDependencies="ws2_32.lib;crypt32.lib"
 				BaseAddress="0x1e1D0000"
 				TargetMachine="17"
 			/>
diff --git a/PCbuild/_ssl.vcxproj b/PCbuild/_ssl.vcxproj
--- a/PCbuild/_ssl.vcxproj
+++ b/PCbuild/_ssl.vcxproj
@@ -158,7 +158,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
@@ -173,7 +173,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
@@ -185,7 +185,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
@@ -200,7 +200,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='PGInstrument|Win32'">
@@ -212,7 +212,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='PGInstrument|x64'">
@@ -227,7 +227,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
       <TargetMachine>MachineX64</TargetMachine>
     </Link>
   </ItemDefinitionGroup>
@@ -240,7 +240,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out32\libeay32.lib;$(opensslDir)\out32\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='PGUpdate|x64'">
@@ -255,7 +255,7 @@
       </Command>
     </PreLinkEvent>
     <Link>
-      <AdditionalDependencies>ws2_32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>ws2_32.lib;crypt32.lib;$(opensslDir)\out64\libeay32.lib;$(opensslDir)\out64\ssleay32.lib;%(AdditionalDependencies)</AdditionalDependencies>
       <TargetMachine>MachineX64</TargetMachine>
     </Link>
   </ItemDefinitionGroup>

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 20:10:48 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 20:10:48 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE2MTAy?=
 =?utf-8?q?=3A_Make_uuid=2E=5Fnetbios=5Fgetnode=28=29_work_again_on_Python?=
 =?utf-8?b?IDMu?=
Message-ID: <3bT5BJ1XKXzSnY@mail.python.org>

http://hg.python.org/cpython/rev/27f55ff12f41
changeset:   84072:27f55ff12f41
branch:      3.3
parent:      84066:8f95d77443da
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 21:08:05 2013 +0300
summary:
  Issue #16102: Make uuid._netbios_getnode() work again on Python 3.

files:
  Lib/uuid.py |  2 +-
  Misc/NEWS   |  2 ++
  2 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Lib/uuid.py b/Lib/uuid.py
--- a/Lib/uuid.py
+++ b/Lib/uuid.py
@@ -406,7 +406,7 @@
         if win32wnet.Netbios(ncb) != 0:
             continue
         status._unpack()
-        bytes = map(ord, status.adapter_address)
+        bytes = status.adapter_address
         return ((bytes[0]<<40) + (bytes[1]<<32) + (bytes[2]<<24) +
                 (bytes[3]<<16) + (bytes[4]<<8) + bytes[5])
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -27,6 +27,8 @@
 Library
 -------
 
+- Issue #16102: Make uuid._netbios_getnode() work again on Python 3.
+
 - Issue #18109: os.uname() now decodes fields from the locale encoding, and
   socket.gethostname() now decodes the hostname from the locale encoding,
   instead of using the UTF-8 encoding in strict mode.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun  9 20:10:49 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun,  9 Jun 2013 20:10:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogSXNzdWUgIzE2MTAyOiBNYWtlIHV1aWQuX25ldGJpb3NfZ2V0bm9kZSgp?=
 =?utf-8?q?_work_again_on_Python_3=2E?=
Message-ID: <3bT5BK3XlPzT1s@mail.python.org>

http://hg.python.org/cpython/rev/4a0017722910
changeset:   84073:4a0017722910
parent:      84071:10d325f674f5
parent:      84072:27f55ff12f41
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 09 21:10:13 2013 +0300
summary:
  Issue #16102: Make uuid._netbios_getnode() work again on Python 3.

files:
  Lib/uuid.py |  2 +-
  Misc/NEWS   |  2 ++
  2 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Lib/uuid.py b/Lib/uuid.py
--- a/Lib/uuid.py
+++ b/Lib/uuid.py
@@ -406,7 +406,7 @@
         if win32wnet.Netbios(ncb) != 0:
             continue
         status._unpack()
-        bytes = map(ord, status.adapter_address)
+        bytes = status.adapter_address
         return ((bytes[0]<<40) + (bytes[1]<<32) + (bytes[2]<<24) +
                 (bytes[3]<<16) + (bytes[4]<<8) + bytes[5])
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -115,6 +115,8 @@
 Library
 -------
 
+- Issue #16102: Make uuid._netbios_getnode() work again on Python 3.
+
 - Issue #17134: Add ssl.enum_cert_store() as interface to Windows' cert store.
 
 - Issue #18143: Implement ssl.get_default_verify_paths() in order to debug

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Jun 10 05:51:51 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 10 Jun 2013 05:51:51 +0200
Subject: [Python-checkins] Daily reference leaks (4a0017722910): sum=53
Message-ID: <E1Ult95-0000Gm-BF@ap.vmr.nerim.net>

results for 4a0017722910 on branch "default"
--------------------------------------------

test_unittest leaked [-1, 2, 0] memory blocks, sum=1
test_ssl leaked [8, 8, 8] references, sum=24
test_ssl leaked [8, 10, 10] memory blocks, sum=28


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogyU1vRV', '-x']

From python-checkins at python.org  Mon Jun 10 10:37:28 2013
From: python-checkins at python.org (ronald.oussoren)
Date: Mon, 10 Jun 2013 10:37:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Ensure_that_th?=
 =?utf-8?q?e_fix_for_=2317269_also_works_on_OSX_10=2E4?=
Message-ID: <3bTSQJ3RmKzQct@mail.python.org>

http://hg.python.org/cpython/rev/4d1e4bc6c5b5
changeset:   84074:4d1e4bc6c5b5
branch:      2.7
parent:      84065:570b5b4040b1
user:        Ronald Oussoren <ronaldoussoren at mac.com>
date:        Mon Jun 10 10:35:36 2013 +0200
summary:
  Ensure that the fix for #17269 also works on OSX 10.4

AI_NUMERICSERV isn't defined on OSX 10.4.

files:
  Lib/test/test_socket.py |  3 ++-
  Modules/socketmodule.c  |  2 +-
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -666,7 +666,8 @@
                            socket.AI_PASSIVE)
 
         # Issue 17269
-        socket.getaddrinfo("localhost", None, 0, 0, 0, socket.AI_NUMERICSERV)
+        if hasattr(socket, 'AI_NUMERICSERV'):
+            socket.getaddrinfo("localhost", None, 0, 0, 0, socket.AI_NUMERICSERV)
 
     def check_sendall_interrupted(self, with_timeout):
         # socketpair() is not stricly required, but it makes things easier.
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -4179,7 +4179,7 @@
                         "getaddrinfo() argument 2 must be integer or string");
         goto err;
     }
-#ifdef __APPLE__
+#if defined(__APPLE__) && defined(AI_NUMERICSERV)
     if ((flags & AI_NUMERICSERV) && (pptr == NULL || (pptr[0] == '0' && pptr[1] == 0))) {
         /* On OSX upto at least OSX 10.8 getaddrinfo crashes
 	 * if AI_NUMERICSERV is set and the servname is NULL or "0".

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 10:37:29 2013
From: python-checkins at python.org (ronald.oussoren)
Date: Mon, 10 Jun 2013 10:37:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Ensure_that_th?=
 =?utf-8?q?e_fix_for_=2317269_also_works_on_OSX_10=2E4?=
Message-ID: <3bTSQK5YNrzRfs@mail.python.org>

http://hg.python.org/cpython/rev/ef103e7e7af2
changeset:   84075:ef103e7e7af2
branch:      3.3
parent:      84072:27f55ff12f41
user:        Ronald Oussoren <ronaldoussoren at mac.com>
date:        Mon Jun 10 10:36:28 2013 +0200
summary:
  Ensure that the fix for #17269 also works on OSX 10.4

AI_NUMERICSERV isn't defined on OSX 10.4.

files:
  Lib/test/test_socket.py |  3 ++-
  Modules/socketmodule.c  |  2 +-
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -1167,7 +1167,8 @@
         self.assertRaises(UnicodeEncodeError, socket.getaddrinfo, 'localhost', '\uD800')
 
         # Issue 17269
-        socket.getaddrinfo("localhost", None, 0, 0, 0, socket.AI_NUMERICSERV)
+        if hasattr(socket, 'AI_NUMERICSERV'):
+            socket.getaddrinfo("localhost", None, 0, 0, 0, socket.AI_NUMERICSERV)
 
     def test_getnameinfo(self):
         # only IP addresses are allowed
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -5041,7 +5041,7 @@
         PyErr_SetString(PyExc_OSError, "Int or String expected");
         goto err;
     }
-#ifdef __APPLE__
+#if defined(__APPLE__) && defined(AI_NUMERICSERV)
     if ((flags & AI_NUMERICSERV) && (pptr == NULL || (pptr[0] == '0' && pptr[1] == 0))) {
         /* On OSX upto at least OSX 10.8 getaddrinfo crashes
 	 * if AI_NUMERICSERV is set and the servname is NULL or "0".

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 10:37:31 2013
From: python-checkins at python.org (ronald.oussoren)
Date: Mon, 10 Jun 2013 10:37:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=283=2E3-=3Edefault=29_Ensure_that_the_fix_for_=2317269_?=
 =?utf-8?q?also_works_on_OSX_10=2E4?=
Message-ID: <3bTSQM0S8szSZn@mail.python.org>

http://hg.python.org/cpython/rev/062f1985a5b7
changeset:   84076:062f1985a5b7
parent:      84073:4a0017722910
parent:      84075:ef103e7e7af2
user:        Ronald Oussoren <ronaldoussoren at mac.com>
date:        Mon Jun 10 10:37:12 2013 +0200
summary:
  (3.3->default) Ensure that the fix for #17269 also works on OSX 10.4

AI_NUMERICSERV isn't defined on OSX 10.4.

files:
  Lib/test/test_socket.py |  3 ++-
  Modules/socketmodule.c  |  2 +-
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -1202,7 +1202,8 @@
         self.assertRaises(UnicodeEncodeError, socket.getaddrinfo, 'localhost', '\uD800')
 
         # Issue 17269
-        socket.getaddrinfo("localhost", None, 0, 0, 0, socket.AI_NUMERICSERV)
+        if hasattr(socket, 'AI_NUMERICSERV'):
+            socket.getaddrinfo("localhost", None, 0, 0, 0, socket.AI_NUMERICSERV)
 
     def test_getnameinfo(self):
         # only IP addresses are allowed
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -4975,7 +4975,7 @@
         PyErr_SetString(PyExc_OSError, "Int or String expected");
         goto err;
     }
-#ifdef __APPLE__
+#if defined(__APPLE__) && defined(AI_NUMERICSERV)
     if ((flags & AI_NUMERICSERV) && (pptr == NULL || (pptr[0] == '0' && pptr[1] == 0))) {
         /* On OSX upto at least OSX 10.8 getaddrinfo crashes
 	 * if AI_NUMERICSERV is set and the servname is NULL or "0".

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 10:47:28 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 10 Jun 2013 10:47:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_fixd_refleak?=
Message-ID: <3bTSdr5GRqzRNm@mail.python.org>

http://hg.python.org/cpython/rev/6860263c05b3
changeset:   84077:6860263c05b3
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 10 10:47:22 2013 +0200
summary:
  fixd refleak

files:
  Modules/_ssl.c |  10 +++++++++-
  1 files changed, 9 insertions(+), 1 deletions(-)


diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2775,6 +2775,7 @@
     PyObject *ofile = NULL;
     PyObject *odir_env = NULL;
     PyObject *odir = NULL;
+    PyObject *tup = NULL;
 
 #define convert(info, target) { \
         const char *tmp = (info); \
@@ -2791,7 +2792,14 @@
     convert(X509_get_default_cert_dir(), odir);
 #undef convert
 
-    return Py_BuildValue("(OOOO)", ofile_env, ofile, odir_env, odir);
+    if ((tup = PyTuple_New(4)) == NULL) {
+        goto error;
+    }
+    PyTuple_SET_ITEM(tup, 0, ofile_env);
+    PyTuple_SET_ITEM(tup, 1, ofile);
+    PyTuple_SET_ITEM(tup, 2, odir_env);
+    PyTuple_SET_ITEM(tup, 3, odir);
+    return tup;
 
   error:
     Py_XDECREF(ofile_env);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 16:46:57 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 10 Jun 2013 16:46:57 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTgw?=
 =?utf-8?q?=3A_Fix_ref_leak_in_=5FPyImport=5FGetDynLoadWindows=28=29=2E?=
Message-ID: <3bTccd5hhpzRgM@mail.python.org>

http://hg.python.org/cpython/rev/ec854f76d6b9
changeset:   84078:ec854f76d6b9
branch:      3.3
parent:      84075:ef103e7e7af2
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 10 15:38:54 2013 +0100
summary:
  Issue #18180: Fix ref leak in _PyImport_GetDynLoadWindows().

files:
  Misc/NEWS            |  2 ++
  Python/dynload_win.c |  5 +++--
  2 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,8 @@
 Core and Builtins
 -----------------
 
+- Issue #18180: Fix ref leak in _PyImport_GetDynLoadWindows().
+
 - Issue #18038: SyntaxError raised during compilation sources with illegal
   encoding now always contains an encoding name.
 
diff --git a/Python/dynload_win.c b/Python/dynload_win.c
--- a/Python/dynload_win.c
+++ b/Python/dynload_win.c
@@ -262,8 +262,9 @@
                         theLength));
             }
             if (message != NULL) {
-                PyErr_SetImportError(message, PyUnicode_FromString(shortname),
-                                     pathname);
+                PyObject *shortname_obj = PyUnicode_FromString(shortname);
+                PyErr_SetImportError(message, shortname_obj, pathname);
+                Py_XDECREF(shortname_obj);
                 Py_DECREF(message);
             }
             return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 16:46:59 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 10 Jun 2013 16:46:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Uu?=
Message-ID: <3bTccg1pwczS0l@mail.python.org>

http://hg.python.org/cpython/rev/df79692a1e7e
changeset:   84079:df79692a1e7e
parent:      84077:6860263c05b3
parent:      84078:ec854f76d6b9
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 10 15:45:30 2013 +0100
summary:
  Merge.

files:
  Misc/NEWS            |  2 ++
  Python/dynload_win.c |  5 +++--
  2 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,8 @@
 Core and Builtins
 -----------------
 
+- Issue #18180: Fix ref leak in _PyImport_GetDynLoadWindows().
+
 - Issue #18038: SyntaxError raised during compilation sources with illegal
   encoding now always contains an encoding name.
 
diff --git a/Python/dynload_win.c b/Python/dynload_win.c
--- a/Python/dynload_win.c
+++ b/Python/dynload_win.c
@@ -262,8 +262,9 @@
                         theLength));
             }
             if (message != NULL) {
-                PyErr_SetImportError(message, PyUnicode_FromString(shortname),
-                                     pathname);
+                PyObject *shortname_obj = PyUnicode_FromString(shortname);
+                PyErr_SetImportError(message, shortname_obj, pathname);
+                Py_XDECREF(shortname_obj);
                 Py_DECREF(message);
             }
             return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 17:33:15 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 10 Jun 2013 17:33:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTc0?=
 =?utf-8?q?=3A_Fix_fd_leaks_in_tests=2E?=
Message-ID: <3bTdf32tGBzPkG@mail.python.org>

http://hg.python.org/cpython/rev/a7381fe515e8
changeset:   84080:a7381fe515e8
branch:      2.7
parent:      84074:4d1e4bc6c5b5
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 10 16:27:45 2013 +0100
summary:
  Issue #18174: Fix fd leaks in tests.

files:
  Lib/test/test_openpty.py    |  2 ++
  Lib/test/test_subprocess.py |  3 ++-
  Lib/test/test_uuid.py       |  1 +
  3 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_openpty.py b/Lib/test/test_openpty.py
--- a/Lib/test/test_openpty.py
+++ b/Lib/test/test_openpty.py
@@ -10,6 +10,8 @@
 class OpenptyTest(unittest.TestCase):
     def test(self):
         master, slave = os.openpty()
+        self.addCleanup(os.close, master)
+        self.addCleanup(os.close, slave)
         if not os.isatty(slave):
             self.fail("Slave-end of pty is not a terminal.")
 
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -806,7 +806,8 @@
                         self._testcase.assertNotIn(
                                 fd, (p2cwrite, c2pread, errread))
                 finally:
-                    map(os.close, devzero_fds)
+                    for fd in devzero_fds:
+                        os.close(fd)
 
     @unittest.skipIf(not os.path.exists("/dev/zero"), "/dev/zero required.")
     def test_preexec_errpipe_does_not_double_close_pipes(self):
diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py
--- a/Lib/test/test_uuid.py
+++ b/Lib/test/test_uuid.py
@@ -448,6 +448,7 @@
 
         else:
             os.close(fds[1])
+            self.addCleanup(os.close, fds[0])
             parent_value = uuid.uuid4().hex
             os.waitpid(pid, 0)
             child_value = os.read(fds[0], 100)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 17:33:16 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 10 Jun 2013 17:33:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTc0?=
 =?utf-8?q?=3A_Fix_fd_leaks_in_tests=2E?=
Message-ID: <3bTdf44yl1zRxV@mail.python.org>

http://hg.python.org/cpython/rev/46fe1bb0723c
changeset:   84081:46fe1bb0723c
branch:      3.3
parent:      84078:ec854f76d6b9
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 10 16:29:19 2013 +0100
summary:
  Issue #18174: Fix fd leaks in tests.

files:
  Lib/test/test_openpty.py    |  2 ++
  Lib/test/test_subprocess.py |  3 ++-
  Lib/test/test_uuid.py       |  1 +
  3 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_openpty.py b/Lib/test/test_openpty.py
--- a/Lib/test/test_openpty.py
+++ b/Lib/test/test_openpty.py
@@ -10,6 +10,8 @@
 class OpenptyTest(unittest.TestCase):
     def test(self):
         master, slave = os.openpty()
+        self.addCleanup(os.close, master)
+        self.addCleanup(os.close, slave)
         if not os.isatty(slave):
             self.fail("Slave-end of pty is not a terminal.")
 
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -1263,7 +1263,8 @@
                                      self.stderr.fileno()),
                                 msg="At least one fd was closed early.")
                 finally:
-                    map(os.close, devzero_fds)
+                    for fd in devzero_fds:
+                        os.close(fd)
 
     @unittest.skipIf(not os.path.exists("/dev/zero"), "/dev/zero required.")
     def test_preexec_errpipe_does_not_double_close_pipes(self):
diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py
--- a/Lib/test/test_uuid.py
+++ b/Lib/test/test_uuid.py
@@ -458,6 +458,7 @@
 
         else:
             os.close(fds[1])
+            self.addCleanup(os.close, fds[0])
             parent_value = uuid.uuid4().hex
             os.waitpid(pid, 0)
             child_value = os.read(fds[0], 100).decode('latin-1')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 17:33:18 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 10 Jun 2013 17:33:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Uu?=
Message-ID: <3bTdf60kJwzSNX@mail.python.org>

http://hg.python.org/cpython/rev/69a165d8dc98
changeset:   84082:69a165d8dc98
parent:      84079:df79692a1e7e
parent:      84081:46fe1bb0723c
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 10 16:31:39 2013 +0100
summary:
  Merge.

files:
  Lib/test/test_openpty.py    |  2 ++
  Lib/test/test_subprocess.py |  3 ++-
  Lib/test/test_uuid.py       |  1 +
  3 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_openpty.py b/Lib/test/test_openpty.py
--- a/Lib/test/test_openpty.py
+++ b/Lib/test/test_openpty.py
@@ -10,6 +10,8 @@
 class OpenptyTest(unittest.TestCase):
     def test(self):
         master, slave = os.openpty()
+        self.addCleanup(os.close, master)
+        self.addCleanup(os.close, slave)
         if not os.isatty(slave):
             self.fail("Slave-end of pty is not a terminal.")
 
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -1296,7 +1296,8 @@
                                      self.stderr.fileno()),
                                 msg="At least one fd was closed early.")
                 finally:
-                    map(os.close, devzero_fds)
+                    for fd in devzero_fds:
+                        os.close(fd)
 
     @unittest.skipIf(not os.path.exists("/dev/zero"), "/dev/zero required.")
     def test_preexec_errpipe_does_not_double_close_pipes(self):
diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py
--- a/Lib/test/test_uuid.py
+++ b/Lib/test/test_uuid.py
@@ -458,6 +458,7 @@
 
         else:
             os.close(fds[1])
+            self.addCleanup(os.close, fds[0])
             parent_value = uuid.uuid4().hex
             os.waitpid(pid, 0)
             child_value = os.read(fds[0], 100).decode('latin-1')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 18:24:13 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 10 Jun 2013 18:24:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogcmVtb3ZlIE1BWF9N?=
 =?utf-8?q?AXCHAR_because_it=27s_unsafe_for_computing_maximum_codepoitn_va?=
 =?utf-8?q?lue?=
Message-ID: <3bTfms1YvnzRnq@mail.python.org>

http://hg.python.org/cpython/rev/89b106d298a9
changeset:   84083:89b106d298a9
branch:      3.3
parent:      84081:46fe1bb0723c
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Jun 10 09:19:46 2013 -0700
summary:
  remove MAX_MAXCHAR because it's unsafe for computing maximum codepoitn value (see #18183)

files:
  Lib/test/test_unicode.py |   3 +
  Misc/NEWS                |   3 +
  Objects/unicodeobject.c  |  57 ++++++++++++---------------
  3 files changed, 32 insertions(+), 31 deletions(-)


diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py
+++ b/Lib/test/test_unicode.py
@@ -566,6 +566,9 @@
         self.assertEqual('\U0008fffe'.lower(), '\U0008fffe')
         self.assertEqual('\u2177'.lower(), '\u2177')
 
+        # See issue #18183 for this one.
+        '\U00010000\U00100000'.lower()
+
     def test_casefold(self):
         self.assertEqual('hello'.casefold(), 'hello')
         self.assertEqual('hELlo'.casefold(), 'hello')
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18183: Fix various unicode operations on strings with large unicode
+  codepoints.
+
 - Issue #18180: Fix ref leak in _PyImport_GetDynLoadWindows().
 
 - Issue #18038: SyntaxError raised during compilation sources with illegal
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -112,11 +112,6 @@
 #define _PyUnicode_DATA_ANY(op)                         \
     (((PyUnicodeObject*)(op))->data.any)
 
-/* Optimized version of Py_MAX() to compute the maximum character:
-   use it when your are computing the second argument of PyUnicode_New() */
-#define MAX_MAXCHAR(maxchar1, maxchar2)                 \
-    ((maxchar1) | (maxchar2))
-
 #undef PyUnicode_READY
 #define PyUnicode_READY(op)                             \
     (assert(_PyUnicode_CHECK(op)),                      \
@@ -2495,7 +2490,7 @@
             case 'c':
             {
                 Py_UCS4 ordinal = va_arg(count, int);
-                maxchar = MAX_MAXCHAR(maxchar, ordinal);
+                maxchar = Py_MAX(maxchar, ordinal);
                 n++;
                 break;
             }
@@ -2591,7 +2586,7 @@
                 /* since PyUnicode_DecodeUTF8 returns already flexible
                    unicode objects, there is no need to call ready on them */
                 argmaxchar = PyUnicode_MAX_CHAR_VALUE(str);
-                maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                maxchar = Py_MAX(maxchar, argmaxchar);
                 n += PyUnicode_GET_LENGTH(str);
                 /* Remember the str and switch to the next slot */
                 *callresult++ = str;
@@ -2604,7 +2599,7 @@
                 if (PyUnicode_READY(obj) == -1)
                     goto fail;
                 argmaxchar = PyUnicode_MAX_CHAR_VALUE(obj);
-                maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                maxchar = Py_MAX(maxchar, argmaxchar);
                 n += PyUnicode_GET_LENGTH(obj);
                 break;
             }
@@ -2619,7 +2614,7 @@
                     if (PyUnicode_READY(obj) == -1)
                         goto fail;
                     argmaxchar = PyUnicode_MAX_CHAR_VALUE(obj);
-                    maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                    maxchar = Py_MAX(maxchar, argmaxchar);
                     n += PyUnicode_GET_LENGTH(obj);
                     *callresult++ = NULL;
                 }
@@ -2632,7 +2627,7 @@
                         goto fail;
                     }
                     argmaxchar = PyUnicode_MAX_CHAR_VALUE(str_obj);
-                    maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                    maxchar = Py_MAX(maxchar, argmaxchar);
                     n += PyUnicode_GET_LENGTH(str_obj);
                     *callresult++ = str_obj;
                 }
@@ -2651,7 +2646,7 @@
                     goto fail;
                 }
                 argmaxchar = PyUnicode_MAX_CHAR_VALUE(str);
-                maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                maxchar = Py_MAX(maxchar, argmaxchar);
                 n += PyUnicode_GET_LENGTH(str);
                 /* Remember the str and switch to the next slot */
                 *callresult++ = str;
@@ -2670,7 +2665,7 @@
                     goto fail;
                 }
                 argmaxchar = PyUnicode_MAX_CHAR_VALUE(repr);
-                maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                maxchar = Py_MAX(maxchar, argmaxchar);
                 n += PyUnicode_GET_LENGTH(repr);
                 /* Remember the repr and switch to the next slot */
                 *callresult++ = repr;
@@ -2689,7 +2684,7 @@
                     goto fail;
                 }
                 argmaxchar = PyUnicode_MAX_CHAR_VALUE(ascii);
-                maxchar = MAX_MAXCHAR(maxchar, argmaxchar);
+                maxchar = Py_MAX(maxchar, argmaxchar);
                 n += PyUnicode_GET_LENGTH(ascii);
                 /* Remember the repr and switch to the next slot */
                 *callresult++ = ascii;
@@ -8628,11 +8623,11 @@
             }
             if (fixed != 0) {
                 modified = 1;
-                maxchar = MAX_MAXCHAR(maxchar, fixed);
+                maxchar = Py_MAX(maxchar, fixed);
                 PyUnicode_WRITE(kind, data, i, fixed);
             }
             else
-                maxchar = MAX_MAXCHAR(maxchar, ch);
+                maxchar = Py_MAX(maxchar, ch);
         }
     }
 
@@ -8673,7 +8668,7 @@
             int decimal = Py_UNICODE_TODECIMAL(ch);
             if (decimal >= 0)
                 ch = '0' + decimal;
-            maxchar = MAX_MAXCHAR(maxchar, ch);
+            maxchar = Py_MAX(maxchar, ch);
         }
     }
 
@@ -8914,7 +8909,7 @@
     if (unicode == NULL) {
         *maxchar = 127;
         if (len != n_digits) {
-            *maxchar = MAX_MAXCHAR(*maxchar,
+            *maxchar = Py_MAX(*maxchar,
                                    PyUnicode_MAX_CHAR_VALUE(thousands_sep));
         }
     }
@@ -9309,14 +9304,14 @@
     c = PyUnicode_READ(kind, data, 0);
     n_res = _PyUnicode_ToUpperFull(c, mapped);
     for (j = 0; j < n_res; j++) {
-        *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+        *maxchar = Py_MAX(*maxchar, mapped[j]);
         res[k++] = mapped[j];
     }
     for (i = 1; i < length; i++) {
         c = PyUnicode_READ(kind, data, i);
         n_res = lower_ucs4(kind, data, length, i, c, mapped);
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9341,7 +9336,7 @@
             mapped[0] = c;
         }
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9362,7 +9357,7 @@
         else
             n_res = _PyUnicode_ToUpperFull(c, mapped);
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9391,7 +9386,7 @@
         Py_UCS4 mapped[3];
         int j, n_res = _PyUnicode_ToFoldedFull(c, mapped);
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9416,7 +9411,7 @@
             n_res = _PyUnicode_ToTitleFull(c, mapped);
 
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
 
@@ -9571,7 +9566,7 @@
             goto onError;
         sz += PyUnicode_GET_LENGTH(item);
         item_maxchar = PyUnicode_MAX_CHAR_VALUE(item);
-        maxchar = MAX_MAXCHAR(maxchar, item_maxchar);
+        maxchar = Py_MAX(maxchar, item_maxchar);
         if (i != 0)
             sz += seplen;
         if (sz < old_sz || sz > PY_SSIZE_T_MAX) {
@@ -9747,7 +9742,7 @@
         return NULL;
     }
     maxchar = PyUnicode_MAX_CHAR_VALUE(self);
-    maxchar = MAX_MAXCHAR(maxchar, fill);
+    maxchar = Py_MAX(maxchar, fill);
     u = PyUnicode_New(left + _PyUnicode_LENGTH(self) + right, maxchar);
     if (!u)
         return NULL;
@@ -10061,7 +10056,7 @@
     /* Replacing str1 with str2 may cause a maxchar reduction in the
        result string. */
     mayshrink = (maxchar_str2 < maxchar);
-    maxchar = MAX_MAXCHAR(maxchar, maxchar_str2);
+    maxchar = Py_MAX(maxchar, maxchar_str2);
 
     if (len1 == len2) {
         /* same length */
@@ -10647,7 +10642,7 @@
 
     maxchar = PyUnicode_MAX_CHAR_VALUE(u);
     maxchar2 = PyUnicode_MAX_CHAR_VALUE(v);
-    maxchar = MAX_MAXCHAR(maxchar, maxchar2);
+    maxchar = Py_MAX(maxchar, maxchar2);
 
     /* Concat the two Unicode strings */
     w = PyUnicode_New(new_len, maxchar);
@@ -10734,7 +10729,7 @@
     else {
         maxchar = PyUnicode_MAX_CHAR_VALUE(left);
         maxchar2 = PyUnicode_MAX_CHAR_VALUE(right);
-        maxchar = MAX_MAXCHAR(maxchar, maxchar2);
+        maxchar = Py_MAX(maxchar, maxchar2);
 
         /* Concat the two Unicode strings */
         res = PyUnicode_New(new_len, maxchar);
@@ -13846,15 +13841,15 @@
             if (!(flags & F_LJUST)) {
                 if (sign) {
                     if ((width-1) > len)
-                        bufmaxchar = MAX_MAXCHAR(bufmaxchar, fill);
+                        bufmaxchar = Py_MAX(bufmaxchar, fill);
                 }
                 else {
                     if (width > len)
-                        bufmaxchar = MAX_MAXCHAR(bufmaxchar, fill);
+                        bufmaxchar = Py_MAX(bufmaxchar, fill);
                 }
             }
             maxchar = _PyUnicode_FindMaxChar(temp, 0, pindex+len);
-            bufmaxchar = MAX_MAXCHAR(bufmaxchar, maxchar);
+            bufmaxchar = Py_MAX(bufmaxchar, maxchar);
 
             buflen = width;
             if (sign && len == width)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 10 18:24:14 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Mon, 10 Jun 2013 18:24:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogbWVyZ2UgMy4zICgjMTgxODMp?=
Message-ID: <3bTfmt4zGMzSNX@mail.python.org>

http://hg.python.org/cpython/rev/668aba845fb2
changeset:   84084:668aba845fb2
parent:      84082:69a165d8dc98
parent:      84083:89b106d298a9
user:        Benjamin Peterson <benjamin at python.org>
date:        Mon Jun 10 09:24:01 2013 -0700
summary:
  merge 3.3 (#18183)

files:
  Lib/test/test_unicode.py |   3 +
  Misc/NEWS                |   3 +
  Objects/unicodeobject.c  |  43 ++++++++++++---------------
  3 files changed, 25 insertions(+), 24 deletions(-)


diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py
+++ b/Lib/test/test_unicode.py
@@ -577,6 +577,9 @@
         self.assertEqual('\U0008fffe'.lower(), '\U0008fffe')
         self.assertEqual('\u2177'.lower(), '\u2177')
 
+        # See issue #18183 for this one.
+        '\U00010000\U00100000'.lower()
+
     def test_casefold(self):
         self.assertEqual('hello'.casefold(), 'hello')
         self.assertEqual('hELlo'.casefold(), 'hello')
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18183: Fix various unicode operations on strings with large unicode
+  codepoints.
+
 - Issue #18180: Fix ref leak in _PyImport_GetDynLoadWindows().
 
 - Issue #18038: SyntaxError raised during compilation sources with illegal
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -104,11 +104,6 @@
 #define _PyUnicode_DATA_ANY(op)                         \
     (((PyUnicodeObject*)(op))->data.any)
 
-/* Optimized version of Py_MAX() to compute the maximum character:
-   use it when your are computing the second argument of PyUnicode_New() */
-#define MAX_MAXCHAR(maxchar1, maxchar2)                 \
-    ((maxchar1) | (maxchar2))
-
 #undef PyUnicode_READY
 #define PyUnicode_READY(op)                             \
     (assert(_PyUnicode_CHECK(op)),                      \
@@ -8609,11 +8604,11 @@
             }
             if (fixed != 0) {
                 modified = 1;
-                maxchar = MAX_MAXCHAR(maxchar, fixed);
+                maxchar = Py_MAX(maxchar, fixed);
                 PyUnicode_WRITE(kind, data, i, fixed);
             }
             else
-                maxchar = MAX_MAXCHAR(maxchar, ch);
+                maxchar = Py_MAX(maxchar, ch);
         }
     }
 
@@ -8654,7 +8649,7 @@
             int decimal = Py_UNICODE_TODECIMAL(ch);
             if (decimal >= 0)
                 ch = '0' + decimal;
-            maxchar = MAX_MAXCHAR(maxchar, ch);
+            maxchar = Py_MAX(maxchar, ch);
         }
     }
 
@@ -8895,7 +8890,7 @@
     if (unicode == NULL) {
         *maxchar = 127;
         if (len != n_digits) {
-            *maxchar = MAX_MAXCHAR(*maxchar,
+            *maxchar = Py_MAX(*maxchar,
                                    PyUnicode_MAX_CHAR_VALUE(thousands_sep));
         }
     }
@@ -9289,14 +9284,14 @@
     c = PyUnicode_READ(kind, data, 0);
     n_res = _PyUnicode_ToUpperFull(c, mapped);
     for (j = 0; j < n_res; j++) {
-        *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+        *maxchar = Py_MAX(*maxchar, mapped[j]);
         res[k++] = mapped[j];
     }
     for (i = 1; i < length; i++) {
         c = PyUnicode_READ(kind, data, i);
         n_res = lower_ucs4(kind, data, length, i, c, mapped);
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9321,7 +9316,7 @@
             mapped[0] = c;
         }
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9342,7 +9337,7 @@
         else
             n_res = _PyUnicode_ToUpperFull(c, mapped);
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9371,7 +9366,7 @@
         Py_UCS4 mapped[3];
         int j, n_res = _PyUnicode_ToFoldedFull(c, mapped);
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
     }
@@ -9396,7 +9391,7 @@
             n_res = _PyUnicode_ToTitleFull(c, mapped);
 
         for (j = 0; j < n_res; j++) {
-            *maxchar = MAX_MAXCHAR(*maxchar, mapped[j]);
+            *maxchar = Py_MAX(*maxchar, mapped[j]);
             res[k++] = mapped[j];
         }
 
@@ -9551,7 +9546,7 @@
             goto onError;
         sz += PyUnicode_GET_LENGTH(item);
         item_maxchar = PyUnicode_MAX_CHAR_VALUE(item);
-        maxchar = MAX_MAXCHAR(maxchar, item_maxchar);
+        maxchar = Py_MAX(maxchar, item_maxchar);
         if (i != 0)
             sz += seplen;
         if (sz < old_sz || sz > PY_SSIZE_T_MAX) {
@@ -9735,7 +9730,7 @@
         return NULL;
     }
     maxchar = PyUnicode_MAX_CHAR_VALUE(self);
-    maxchar = MAX_MAXCHAR(maxchar, fill);
+    maxchar = Py_MAX(maxchar, fill);
     u = PyUnicode_New(left + _PyUnicode_LENGTH(self) + right, maxchar);
     if (!u)
         return NULL;
@@ -10075,7 +10070,7 @@
     /* Replacing str1 with str2 may cause a maxchar reduction in the
        result string. */
     mayshrink = (maxchar_str2 < maxchar_str1) && (maxchar == maxchar_str1);
-    maxchar = MAX_MAXCHAR(maxchar, maxchar_str2);
+    maxchar = Py_MAX(maxchar, maxchar_str2);
 
     if (len1 == len2) {
         /* same length */
@@ -10749,7 +10744,7 @@
 
     maxchar = PyUnicode_MAX_CHAR_VALUE(u);
     maxchar2 = PyUnicode_MAX_CHAR_VALUE(v);
-    maxchar = MAX_MAXCHAR(maxchar, maxchar2);
+    maxchar = Py_MAX(maxchar, maxchar2);
 
     /* Concat the two Unicode strings */
     w = PyUnicode_New(new_len, maxchar);
@@ -10831,7 +10826,7 @@
     else {
         maxchar = PyUnicode_MAX_CHAR_VALUE(left);
         maxchar2 = PyUnicode_MAX_CHAR_VALUE(right);
-        maxchar = MAX_MAXCHAR(maxchar, maxchar2);
+        maxchar = Py_MAX(maxchar, maxchar2);
 
         /* Concat the two Unicode strings */
         res = PyUnicode_New(new_len, maxchar);
@@ -12993,7 +12988,7 @@
     }
     newlen = writer->pos + length;
 
-    maxchar = MAX_MAXCHAR(maxchar, writer->min_char);
+    maxchar = Py_MAX(maxchar, writer->min_char);
 
     if (writer->buffer == NULL) {
         assert(!writer->readonly);
@@ -14133,16 +14128,16 @@
     if (!(arg->flags & F_LJUST)) {
         if (arg->sign) {
             if ((arg->width-1) > len)
-                maxchar = MAX_MAXCHAR(maxchar, fill);
+                maxchar = Py_MAX(maxchar, fill);
         }
         else {
             if (arg->width > len)
-                maxchar = MAX_MAXCHAR(maxchar, fill);
+                maxchar = Py_MAX(maxchar, fill);
         }
     }
     if (PyUnicode_MAX_CHAR_VALUE(str) > maxchar) {
         Py_UCS4 strmaxchar = _PyUnicode_FindMaxChar(str, 0, pindex+len);
-        maxchar = MAX_MAXCHAR(maxchar, strmaxchar);
+        maxchar = Py_MAX(maxchar, strmaxchar);
     }
 
     buflen = arg->width;

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Jun 11 05:50:55 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 11 Jun 2013 05:50:55 +0200
Subject: [Python-checkins] Daily reference leaks (668aba845fb2): sum=0
Message-ID: <E1UmFbj-0003tW-Su@ap.vmr.nerim.net>

results for 668aba845fb2 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog2wxycP', '-x']

From python-checkins at python.org  Tue Jun 11 06:08:17 2013
From: python-checkins at python.org (roger.serwy)
Date: Tue, 11 Jun 2013 06:08:17 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE3NTExOiBLZWVw?=
 =?utf-8?q?_IDLE_find_dialog_open_after_clicking_=22Find_Next=22=2E?=
Message-ID: <3bTyPF5GZPzQqp@mail.python.org>

http://hg.python.org/cpython/rev/73de0794ae71
changeset:   84085:73de0794ae71
branch:      2.7
parent:      84080:a7381fe515e8
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Mon Jun 10 23:01:20 2013 -0500
summary:
  #17511: Keep IDLE find dialog open after clicking "Find Next".
Original patch by Sarah K.

files:
  Lib/idlelib/SearchDialog.py |  5 ++---
  Misc/ACKS                   |  1 +
  Misc/NEWS                   |  3 +++
  3 files changed, 6 insertions(+), 3 deletions(-)


diff --git a/Lib/idlelib/SearchDialog.py b/Lib/idlelib/SearchDialog.py
--- a/Lib/idlelib/SearchDialog.py
+++ b/Lib/idlelib/SearchDialog.py
@@ -24,13 +24,12 @@
 
     def create_widgets(self):
         f = SearchDialogBase.create_widgets(self)
-        self.make_button("Find", self.default_command, 1)
+        self.make_button("Find Next", self.default_command, 1)
 
     def default_command(self, event=None):
         if not self.engine.getprog():
             return
-        if self.find_again(self.text):
-            self.close()
+        self.find_again(self.text)
 
     def find_again(self, text):
         if not self.engine.getpat():
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -502,6 +502,7 @@
 Sijin Joseph
 Andreas Jung
 Tattoo Mabonzo K.
+Sarah K.
 Bohuslav Kabrda
 Bob Kahn
 Kurt B. Kaiser
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -50,6 +50,9 @@
 IDLE
 ----
 
+- Issue #17511: Keep IDLE find dialog open after clicking "Find Next".
+  Original patch by Sarah K.
+
 - Issue #15392: Create a unittest framework for IDLE.
   Preliminary patch by Rajagopalasarma Jayakrishnan
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 06:08:19 2013
From: python-checkins at python.org (roger.serwy)
Date: Tue, 11 Jun 2013 06:08:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE3NTExOiBLZWVw?=
 =?utf-8?q?_IDLE_find_dialog_open_after_clicking_=22Find_Next=22=2E?=
Message-ID: <3bTyPH0H78zRg8@mail.python.org>

http://hg.python.org/cpython/rev/dac11f143581
changeset:   84086:dac11f143581
branch:      3.3
parent:      84083:89b106d298a9
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Mon Jun 10 23:01:20 2013 -0500
summary:
  #17511: Keep IDLE find dialog open after clicking "Find Next".
Original patch by Sarah K.

files:
  Lib/idlelib/SearchDialog.py |  5 ++---
  Misc/ACKS                   |  1 +
  Misc/NEWS                   |  3 +++
  3 files changed, 6 insertions(+), 3 deletions(-)


diff --git a/Lib/idlelib/SearchDialog.py b/Lib/idlelib/SearchDialog.py
--- a/Lib/idlelib/SearchDialog.py
+++ b/Lib/idlelib/SearchDialog.py
@@ -24,13 +24,12 @@
 
     def create_widgets(self):
         f = SearchDialogBase.create_widgets(self)
-        self.make_button("Find", self.default_command, 1)
+        self.make_button("Find Next", self.default_command, 1)
 
     def default_command(self, event=None):
         if not self.engine.getprog():
             return
-        if self.find_again(self.text):
-            self.close()
+        self.find_again(self.text)
 
     def find_again(self, text):
         if not self.engine.getpat():
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -606,6 +606,7 @@
 Sijin Joseph
 Andreas Jung
 Tattoo Mabonzo K.
+Sarah K.
 Bohuslav Kabrda
 Bob Kahn
 Kurt B. Kaiser
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -81,6 +81,9 @@
 IDLE
 ----
 
+- Issue #17511: Keep IDLE find dialog open after clicking "Find Next".
+  Original patch by Sarah K.
+
 - Issue #18055: Move IDLE off of imp and on to importlib.
 
 - Issue #15392: Create a unittest framework for IDLE.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 06:08:20 2013
From: python-checkins at python.org (roger.serwy)
Date: Tue, 11 Jun 2013 06:08:20 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogIzE3NTExOiBtZXJnZSB3aXRoIDMuMy4=?=
Message-ID: <3bTyPJ29HqzSVJ@mail.python.org>

http://hg.python.org/cpython/rev/67714c4d9725
changeset:   84087:67714c4d9725
parent:      84084:668aba845fb2
parent:      84086:dac11f143581
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Mon Jun 10 23:02:56 2013 -0500
summary:
  #17511: merge with 3.3.

files:
  Lib/idlelib/SearchDialog.py |  5 ++---
  Misc/ACKS                   |  1 +
  Misc/NEWS                   |  3 +++
  3 files changed, 6 insertions(+), 3 deletions(-)


diff --git a/Lib/idlelib/SearchDialog.py b/Lib/idlelib/SearchDialog.py
--- a/Lib/idlelib/SearchDialog.py
+++ b/Lib/idlelib/SearchDialog.py
@@ -24,13 +24,12 @@
 
     def create_widgets(self):
         f = SearchDialogBase.create_widgets(self)
-        self.make_button("Find", self.default_command, 1)
+        self.make_button("Find Next", self.default_command, 1)
 
     def default_command(self, event=None):
         if not self.engine.getprog():
             return
-        if self.find_again(self.text):
-            self.close()
+        self.find_again(self.text)
 
     def find_again(self, text):
         if not self.engine.getpat():
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -617,6 +617,7 @@
 Sijin Joseph
 Andreas Jung
 Tattoo Mabonzo K.
+Sarah K.
 Bohuslav Kabrda
 Alexey Kachayev
 Bob Kahn
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -453,6 +453,9 @@
 IDLE
 ----
 
+- Issue #17511: Keep IDLE find dialog open after clicking "Find Next".
+  Original patch by Sarah K.
+
 - Issue #18055: Move IDLE off of imp and on to importlib.
 
 - Issue #15392: Create a unittest framework for IDLE.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 23:09:53 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue, 11 Jun 2013 23:09:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318157=3A_stop_usi?=
 =?utf-8?q?ng_imp=2Eload=5Fmodule=28=29_in_imp=2E?=
Message-ID: <3bVP414Gm2z7Ljk@mail.python.org>

http://hg.python.org/cpython/rev/2c747be20d05
changeset:   84088:2c747be20d05
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 11 17:09:36 2013 -0400
summary:
  Issue #18157: stop using imp.load_module() in imp.

files:
  Lib/pydoc.py |  24 ++++++++++++------------
  Misc/NEWS    |   2 ++
  2 files changed, 14 insertions(+), 12 deletions(-)


diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -53,6 +53,7 @@
 
 import builtins
 import imp
+import importlib._bootstrap
 import importlib.machinery
 import inspect
 import io
@@ -269,18 +270,17 @@
     """Import a Python source file or compiled file given its path."""
     magic = imp.get_magic()
     with open(path, 'rb') as file:
-        if file.read(len(magic)) == magic:
-            kind = imp.PY_COMPILED
-        else:
-            kind = imp.PY_SOURCE
-        file.seek(0)
-        filename = os.path.basename(path)
-        name, ext = os.path.splitext(filename)
-        try:
-            module = imp.load_module(name, file, path, (ext, 'r', kind))
-        except:
-            raise ErrorDuringImport(path, sys.exc_info())
-    return module
+        is_bytecode = magic == file.read(len(magic))
+    filename = os.path.basename(path)
+    name, ext = os.path.splitext(filename)
+    if is_bytecode:
+        loader = importlib._bootstrap.SourcelessFileLoader(name, path)
+    else:
+        loader = importlib._bootstrap.SourceFileLoader(name, path)
+    try:
+        return loader.load_module(name)
+    except:
+        raise ErrorDuringImport(path, sys.exc_info())
 
 def safeimport(path, forceload=0, cache={}):
     """Import a module; handle errors; return None if the module isn't found.
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -120,6 +120,8 @@
 Library
 -------
 
+- Issue #18157: Stop using imp.load_module() in pydoc.
+
 - Issue #16102: Make uuid._netbios_getnode() work again on Python 3.
 
 - Issue #17134: Add ssl.enum_cert_store() as interface to Windows' cert store.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 23:12:39 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue, 11 Jun 2013 23:12:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318158=3A_delete_t?=
 =?utf-8?q?est=5Fimporthooks=2E_Redundant_in_the_face_of?=
Message-ID: <3bVP7C6Rjcz7LjS@mail.python.org>

http://hg.python.org/cpython/rev/176fe1f8496f
changeset:   84089:176fe1f8496f
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 11 17:12:30 2013 -0400
summary:
  Issue #18158: delete test_importhooks. Redundant in the face of
test_importlib.

files:
  Lib/test/test_importhooks.py |  250 -----------------------
  1 files changed, 0 insertions(+), 250 deletions(-)


diff --git a/Lib/test/test_importhooks.py b/Lib/test/test_importhooks.py
deleted file mode 100644
--- a/Lib/test/test_importhooks.py
+++ /dev/null
@@ -1,250 +0,0 @@
-import sys
-import imp
-import os
-import unittest
-from test import support
-
-
-test_src = """\
-def get_name():
-    return __name__
-def get_file():
-    return __file__
-"""
-
-absimp = "import sub\n"
-relimp = "from . import sub\n"
-deeprelimp = "from .... import sub\n"
-futimp = "from __future__ import absolute_import\n"
-
-reload_src = test_src+"""\
-reloaded = True
-"""
-
-test_co = compile(test_src, "<???>", "exec")
-reload_co = compile(reload_src, "<???>", "exec")
-
-test2_oldabs_co = compile(absimp + test_src, "<???>", "exec")
-test2_newabs_co = compile(futimp + absimp + test_src, "<???>", "exec")
-test2_newrel_co = compile(relimp + test_src, "<???>", "exec")
-test2_deeprel_co = compile(deeprelimp + test_src, "<???>", "exec")
-test2_futrel_co = compile(futimp + relimp + test_src, "<???>", "exec")
-
-test_path = "!!!_test_!!!"
-
-
-class TestImporter:
-
-    modules = {
-        "hooktestmodule": (False, test_co),
-        "hooktestpackage": (True, test_co),
-        "hooktestpackage.sub": (True, test_co),
-        "hooktestpackage.sub.subber": (True, test_co),
-        "hooktestpackage.oldabs": (False, test2_oldabs_co),
-        "hooktestpackage.newabs": (False, test2_newabs_co),
-        "hooktestpackage.newrel": (False, test2_newrel_co),
-        "hooktestpackage.sub.subber.subest": (True, test2_deeprel_co),
-        "hooktestpackage.futrel": (False, test2_futrel_co),
-        "sub": (False, test_co),
-        "reloadmodule": (False, test_co),
-    }
-
-    def __init__(self, path=test_path):
-        if path != test_path:
-            # if our class is on sys.path_hooks, we must raise
-            # ImportError for any path item that we can't handle.
-            raise ImportError
-        self.path = path
-
-    def _get__path__(self):
-        raise NotImplementedError
-
-    def find_module(self, fullname, path=None):
-        if fullname in self.modules:
-            return self
-        else:
-            return None
-
-    def load_module(self, fullname):
-        ispkg, code = self.modules[fullname]
-        mod = sys.modules.setdefault(fullname,imp.new_module(fullname))
-        mod.__file__ = "<%s>" % self.__class__.__name__
-        mod.__loader__ = self
-        if ispkg:
-            mod.__path__ = self._get__path__()
-        exec(code, mod.__dict__)
-        return mod
-
-
-class MetaImporter(TestImporter):
-    def _get__path__(self):
-        return []
-
-class PathImporter(TestImporter):
-    def _get__path__(self):
-        return [self.path]
-
-
-class ImportBlocker:
-    """Place an ImportBlocker instance on sys.meta_path and you
-    can be sure the modules you specified can't be imported, even
-    if it's a builtin."""
-    def __init__(self, *namestoblock):
-        self.namestoblock = dict.fromkeys(namestoblock)
-    def find_module(self, fullname, path=None):
-        if fullname in self.namestoblock:
-            return self
-        return None
-    def load_module(self, fullname):
-        raise ImportError("I dare you")
-
-
-class ImpWrapper:
-
-    def __init__(self, path=None):
-        if path is not None and not os.path.isdir(path):
-            raise ImportError
-        self.path = path
-
-    def find_module(self, fullname, path=None):
-        subname = fullname.split(".")[-1]
-        if subname != fullname and self.path is None:
-            return None
-        if self.path is None:
-            path = None
-        else:
-            path = [self.path]
-        try:
-            file, filename, stuff = imp.find_module(subname, path)
-        except ImportError:
-            return None
-        return ImpLoader(file, filename, stuff)
-
-
-class ImpLoader:
-
-    def __init__(self, file, filename, stuff):
-        self.file = file
-        self.filename = filename
-        self.stuff = stuff
-
-    def load_module(self, fullname):
-        mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
-        if self.file:
-            self.file.close()
-        mod.__loader__ = self  # for introspection
-        return mod
-
-
-class ImportHooksBaseTestCase(unittest.TestCase):
-
-    def setUp(self):
-        self.path = sys.path[:]
-        self.meta_path = sys.meta_path[:]
-        self.path_hooks = sys.path_hooks[:]
-        sys.path_importer_cache.clear()
-        self.modules_before = support.modules_setup()
-
-    def tearDown(self):
-        sys.path[:] = self.path
-        sys.meta_path[:] = self.meta_path
-        sys.path_hooks[:] = self.path_hooks
-        sys.path_importer_cache.clear()
-        support.modules_cleanup(*self.modules_before)
-
-
-class ImportHooksTestCase(ImportHooksBaseTestCase):
-
-    def doTestImports(self, importer=None):
-        import hooktestmodule
-        import hooktestpackage
-        import hooktestpackage.sub
-        import hooktestpackage.sub.subber
-        self.assertEqual(hooktestmodule.get_name(),
-                         "hooktestmodule")
-        self.assertEqual(hooktestpackage.get_name(),
-                         "hooktestpackage")
-        self.assertEqual(hooktestpackage.sub.get_name(),
-                         "hooktestpackage.sub")
-        self.assertEqual(hooktestpackage.sub.subber.get_name(),
-                         "hooktestpackage.sub.subber")
-        if importer:
-            self.assertEqual(hooktestmodule.__loader__, importer)
-            self.assertEqual(hooktestpackage.__loader__, importer)
-            self.assertEqual(hooktestpackage.sub.__loader__, importer)
-            self.assertEqual(hooktestpackage.sub.subber.__loader__, importer)
-
-        TestImporter.modules['reloadmodule'] = (False, test_co)
-        import reloadmodule
-        self.assertFalse(hasattr(reloadmodule,'reloaded'))
-
-        import hooktestpackage.newrel
-        self.assertEqual(hooktestpackage.newrel.get_name(),
-                         "hooktestpackage.newrel")
-        self.assertEqual(hooktestpackage.newrel.sub,
-                         hooktestpackage.sub)
-
-        import hooktestpackage.sub.subber.subest as subest
-        self.assertEqual(subest.get_name(),
-                         "hooktestpackage.sub.subber.subest")
-        self.assertEqual(subest.sub,
-                         hooktestpackage.sub)
-
-        import hooktestpackage.futrel
-        self.assertEqual(hooktestpackage.futrel.get_name(),
-                         "hooktestpackage.futrel")
-        self.assertEqual(hooktestpackage.futrel.sub,
-                         hooktestpackage.sub)
-
-        import sub
-        self.assertEqual(sub.get_name(), "sub")
-
-        import hooktestpackage.oldabs
-        self.assertEqual(hooktestpackage.oldabs.get_name(),
-                         "hooktestpackage.oldabs")
-        self.assertEqual(hooktestpackage.oldabs.sub, sub)
-
-        import hooktestpackage.newabs
-        self.assertEqual(hooktestpackage.newabs.get_name(),
-                         "hooktestpackage.newabs")
-        self.assertEqual(hooktestpackage.newabs.sub, sub)
-
-    def testMetaPath(self):
-        i = MetaImporter()
-        sys.meta_path.append(i)
-        self.doTestImports(i)
-
-    def testPathHook(self):
-        sys.path_hooks.insert(0, PathImporter)
-        sys.path.append(test_path)
-        self.doTestImports()
-
-    def testBlocker(self):
-        mname = "exceptions"  # an arbitrary harmless builtin module
-        support.unload(mname)
-        sys.meta_path.append(ImportBlocker(mname))
-        self.assertRaises(ImportError, __import__, mname)
-
-    def testImpWrapper(self):
-        i = ImpWrapper()
-        sys.meta_path.append(i)
-        sys.path_hooks.insert(0, ImpWrapper)
-        mnames = (
-            "colorsys", "urllib.parse", "distutils.core", "sys",
-        )
-        for mname in mnames:
-            parent = mname.split(".")[0]
-            for n in list(sys.modules):
-                if n.startswith(parent):
-                    del sys.modules[n]
-        for mname in mnames:
-            m = __import__(mname, globals(), locals(), ["__dummy__"])
-            # to make sure we actually handled the import
-            self.assertTrue(hasattr(m, "__loader__"))
-
-
-def test_main():
-    support.run_unittest(ImportHooksTestCase)
-
-if __name__ == "__main__":
-    test_main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 23:22:46 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue, 11 Jun 2013 23:22:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_typo_fix?=
Message-ID: <3bVPLt3Wg3z7LkH@mail.python.org>

http://hg.python.org/cpython/rev/32527aa9ad16
changeset:   84090:32527aa9ad16
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 11 17:22:39 2013 -0400
summary:
  typo fix

files:
  Lib/importlib/_bootstrap.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -376,7 +376,7 @@
 #                        keyword-only defaults)
 #     Python 3.4a1  3260 (add LOAD_CLASSDEREF; allow locals of class to override
 #                        free vars)
-#     Python 3.4a1  3270 (various tweaks to the __class_ closure)
+#     Python 3.4a1  3270 (various tweaks to the __class__ closure)
 #     Python 3.4a1  3280 (remove implicit class argument)
 #
 # MAGIC must change whenever the bytecode emitted by the compiler may no

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 23:34:10 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue, 11 Jun 2013 23:34:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_explanatory_comment?=
Message-ID: <3bVPc26Jsnz7Lk0@mail.python.org>

http://hg.python.org/cpython/rev/d4ee62a57a75
changeset:   84091:d4ee62a57a75
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 11 17:34:04 2013 -0400
summary:
  explanatory comment

files:
  Lib/importlib/_bootstrap.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -384,7 +384,7 @@
 # due to the addition of new opcodes).
 
 _MAGIC_BYTES = (3280).to_bytes(2, 'little') + b'\r\n'
-_RAW_MAGIC_NUMBER = int.from_bytes(_MAGIC_BYTES, 'little')
+_RAW_MAGIC_NUMBER = int.from_bytes(_MAGIC_BYTES, 'little')  # For import.c
 
 _PYCACHE = '__pycache__'
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 23:40:53 2013
From: python-checkins at python.org (ned.deily)
Date: Tue, 11 Jun 2013 23:40:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTg3?=
 =?utf-8?q?=3A_Fix_broken_link_in_venv_documentation=2E_Patch_by_Berker_Pe?=
 =?utf-8?q?ksag=2E?=
Message-ID: <3bVPln6qmtz7LkD@mail.python.org>

http://hg.python.org/cpython/rev/b1eeda9db91d
changeset:   84092:b1eeda9db91d
branch:      3.3
parent:      84086:dac11f143581
user:        Ned Deily <nad at acm.org>
date:        Tue Jun 11 14:38:39 2013 -0700
summary:
  Issue #18187: Fix broken link in venv documentation. Patch by Berker Peksag.

files:
  Doc/library/venv.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst
--- a/Doc/library/venv.rst
+++ b/Doc/library/venv.rst
@@ -11,7 +11,7 @@
 
 .. versionadded:: 3.3
 
-**Source code:** :source:`Lib/venv.py`
+**Source code:** :source:`Lib/venv`
 
 --------------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 11 23:40:55 2013
From: python-checkins at python.org (ned.deily)
Date: Tue, 11 Jun 2013 23:40:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318187=3A_merge_from_3=2E3?=
Message-ID: <3bVPlq1bXLz7LkP@mail.python.org>

http://hg.python.org/cpython/rev/e6fc120012e5
changeset:   84093:e6fc120012e5
parent:      84091:d4ee62a57a75
parent:      84092:b1eeda9db91d
user:        Ned Deily <nad at acm.org>
date:        Tue Jun 11 14:40:23 2013 -0700
summary:
  Issue #18187: merge from 3.3

files:
  Doc/library/venv.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst
--- a/Doc/library/venv.rst
+++ b/Doc/library/venv.rst
@@ -11,7 +11,7 @@
 
 .. versionadded:: 3.3
 
-**Source code:** :source:`Lib/venv.py`
+**Source code:** :source:`Lib/venv`
 
 --------------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 00:01:19 2013
From: python-checkins at python.org (ned.deily)
Date: Wed, 12 Jun 2013 00:01:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTg2?=
 =?utf-8?q?=3A_remove_obsolete_2=2E2_compatibility_comment=2E?=
Message-ID: <3bVQCM3fSkzN1w@mail.python.org>

http://hg.python.org/cpython/rev/4b2fdd4dd700
changeset:   84094:4b2fdd4dd700
branch:      2.7
parent:      84085:73de0794ae71
user:        Ned Deily <nad at acm.org>
date:        Tue Jun 11 15:00:21 2013 -0700
summary:
  Issue #18186: remove obsolete 2.2 compatibility comment.

files:
  Lib/subprocess.py |  2 --
  1 files changed, 0 insertions(+), 2 deletions(-)


diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -2,8 +2,6 @@
 #
 # For more information about this module, see PEP 324.
 #
-# This module should remain compatible with Python 2.2, see PEP 291.
-#
 # Copyright (c) 2003-2005 by Peter Astrand <astrand at lysator.liu.se>
 #
 # Licensed to PSF under a Contributor Agreement.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 05:13:36 2013
From: python-checkins at python.org (roger.serwy)
Date: Wed, 12 Jun 2013 05:13:36 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzU0OTI6IEF2b2lk?=
 =?utf-8?q?_traceback_when_exiting_IDLE_caused_by_a_race_condition=2E?=
Message-ID: <3bVY7h4VqJzQJ4@mail.python.org>

http://hg.python.org/cpython/rev/ca8e86711403
changeset:   84095:ca8e86711403
branch:      2.7
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Tue Jun 11 22:13:17 2013 -0500
summary:
  #5492: Avoid traceback when exiting IDLE caused by a race condition.

files:
  Lib/idlelib/PyShell.py |  11 +++++------
  Misc/NEWS              |   2 ++
  2 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -370,6 +370,7 @@
         self.port = PORT
         self.original_compiler_flags = self.compile.compiler.flags
 
+    _afterid = None
     rpcclt = None
     rpcpid = None
 
@@ -497,6 +498,8 @@
         threading.Thread(target=self.__request_interrupt).start()
 
     def kill_subprocess(self):
+        if self._afterid is not None:
+            self.tkconsole.text.after_cancel(self._afterid)
         try:
             self.rpcclt.close()
         except AttributeError:  # no socket
@@ -569,8 +572,8 @@
                 pass
         # Reschedule myself
         if not self.tkconsole.closing:
-            self.tkconsole.text.after(self.tkconsole.pollinterval,
-                                      self.poll_subprocess)
+            self._afterid = self.tkconsole.text.after(
+                self.tkconsole.pollinterval, self.poll_subprocess)
 
     debugger = None
 
@@ -987,10 +990,6 @@
         self.stop_readline()
         self.canceled = True
         self.closing = True
-        # Wait for poll_subprocess() rescheduling to stop
-        self.text.after(2 * self.pollinterval, self.close2)
-
-    def close2(self):
         return EditorWindow.close(self)
 
     def _close(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -50,6 +50,8 @@
 IDLE
 ----
 
+- Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.
+
 - Issue #17511: Keep IDLE find dialog open after clicking "Find Next".
   Original patch by Sarah K.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 05:13:38 2013
From: python-checkins at python.org (roger.serwy)
Date: Wed, 12 Jun 2013 05:13:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzU0OTI6IEF2b2lk?=
 =?utf-8?q?_traceback_when_exiting_IDLE_caused_by_a_race_condition=2E?=
Message-ID: <3bVY7k0tLQzQlP@mail.python.org>

http://hg.python.org/cpython/rev/da852f5250af
changeset:   84096:da852f5250af
branch:      3.3
parent:      84092:b1eeda9db91d
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Tue Jun 11 22:13:17 2013 -0500
summary:
  #5492: Avoid traceback when exiting IDLE caused by a race condition.

files:
  Lib/idlelib/PyShell.py |  11 +++++------
  Misc/NEWS              |   2 ++
  2 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -367,6 +367,7 @@
         self.port = PORT
         self.original_compiler_flags = self.compile.compiler.flags
 
+    _afterid = None
     rpcclt = None
     rpcsubproc = None
 
@@ -486,6 +487,8 @@
         threading.Thread(target=self.__request_interrupt).start()
 
     def kill_subprocess(self):
+        if self._afterid is not None:
+            self.tkconsole.text.after_cancel(self._afterid)
         try:
             self.rpcclt.listening_sock.close()
         except AttributeError:  # no socket
@@ -561,8 +564,8 @@
                 pass
         # Reschedule myself
         if not self.tkconsole.closing:
-            self.tkconsole.text.after(self.tkconsole.pollinterval,
-                                      self.poll_subprocess)
+            self._afterid = self.tkconsole.text.after(
+                self.tkconsole.pollinterval, self.poll_subprocess)
 
     debugger = None
 
@@ -973,10 +976,6 @@
         self.stop_readline()
         self.canceled = True
         self.closing = True
-        # Wait for poll_subprocess() rescheduling to stop
-        self.text.after(2 * self.pollinterval, self.close2)
-
-    def close2(self):
         return EditorWindow.close(self)
 
     def _close(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -81,6 +81,8 @@
 IDLE
 ----
 
+- Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.
+
 - Issue #17511: Keep IDLE find dialog open after clicking "Find Next".
   Original patch by Sarah K.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 05:13:39 2013
From: python-checkins at python.org (roger.serwy)
Date: Wed, 12 Jun 2013 05:13:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogIzU0OTI6IG1lcmdlIHdpdGggMy4z?=
Message-ID: <3bVY7l2xTvzQXQ@mail.python.org>

http://hg.python.org/cpython/rev/187da33826eb
changeset:   84097:187da33826eb
parent:      84093:e6fc120012e5
parent:      84096:da852f5250af
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Tue Jun 11 22:13:51 2013 -0500
summary:
  #5492: merge with 3.3

files:
  Lib/idlelib/PyShell.py |  11 +++++------
  Misc/NEWS              |   2 ++
  2 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -367,6 +367,7 @@
         self.port = PORT
         self.original_compiler_flags = self.compile.compiler.flags
 
+    _afterid = None
     rpcclt = None
     rpcsubproc = None
 
@@ -486,6 +487,8 @@
         threading.Thread(target=self.__request_interrupt).start()
 
     def kill_subprocess(self):
+        if self._afterid is not None:
+            self.tkconsole.text.after_cancel(self._afterid)
         try:
             self.rpcclt.listening_sock.close()
         except AttributeError:  # no socket
@@ -561,8 +564,8 @@
                 pass
         # Reschedule myself
         if not self.tkconsole.closing:
-            self.tkconsole.text.after(self.tkconsole.pollinterval,
-                                      self.poll_subprocess)
+            self._afterid = self.tkconsole.text.after(
+                self.tkconsole.pollinterval, self.poll_subprocess)
 
     debugger = None
 
@@ -973,10 +976,6 @@
         self.stop_readline()
         self.canceled = True
         self.closing = True
-        # Wait for poll_subprocess() rescheduling to stop
-        self.text.after(2 * self.pollinterval, self.close2)
-
-    def close2(self):
         return EditorWindow.close(self)
 
     def _close(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -455,6 +455,8 @@
 IDLE
 ----
 
+- Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.
+
 - Issue #17511: Keep IDLE find dialog open after clicking "Find Next".
   Original patch by Sarah K.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 05:24:49 2013
From: python-checkins at python.org (roger.serwy)
Date: Wed, 12 Jun 2013 05:24:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTk2OiBBdm9p?=
 =?utf-8?q?d_displaying_spurious_SystemExit_tracebacks=2E?=
Message-ID: <3bVYNd1bc6zQKf@mail.python.org>

http://hg.python.org/cpython/rev/0e56d4e37777
changeset:   84098:0e56d4e37777
branch:      3.3
parent:      84096:da852f5250af
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Tue Jun 11 22:25:14 2013 -0500
summary:
  #18196: Avoid displaying spurious SystemExit tracebacks.

files:
  Lib/idlelib/run.py |  4 ++++
  Misc/NEWS          |  2 ++
  2 files changed, 6 insertions(+), 0 deletions(-)


diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -339,6 +339,10 @@
                 exec(code, self.locals)
             finally:
                 interruptable = False
+        except SystemExit:
+            # Scripts that raise SystemExit should just
+            # return to the interactive prompt
+            pass
         except:
             self.usr_exc_info = sys.exc_info()
             if quitting:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -81,6 +81,8 @@
 IDLE
 ----
 
+- Issue #18196: Avoid displaying spurious SystemExit tracebacks.
+
 - Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.
 
 - Issue #17511: Keep IDLE find dialog open after clicking "Find Next".

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 05:24:50 2013
From: python-checkins at python.org (roger.serwy)
Date: Wed, 12 Jun 2013 05:24:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=2318196=3A_merge_with_3=2E3?=
Message-ID: <3bVYNf3YRxzQjj@mail.python.org>

http://hg.python.org/cpython/rev/479aad3bb122
changeset:   84099:479aad3bb122
parent:      84097:187da33826eb
parent:      84098:0e56d4e37777
user:        Roger Serwy <roger.serwy at gmail.com>
date:        Tue Jun 11 22:25:34 2013 -0500
summary:
  #18196: merge with 3.3

files:
  Lib/idlelib/run.py |  4 ++++
  Misc/NEWS          |  2 ++
  2 files changed, 6 insertions(+), 0 deletions(-)


diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -339,6 +339,10 @@
                 exec(code, self.locals)
             finally:
                 interruptable = False
+        except SystemExit:
+            # Scripts that raise SystemExit should just
+            # return to the interactive prompt
+            pass
         except:
             self.usr_exc_info = sys.exc_info()
             if quitting:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -455,6 +455,8 @@
 IDLE
 ----
 
+- Issue #18196: Avoid displaying spurious SystemExit tracebacks.
+
 - Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.
 
 - Issue #17511: Keep IDLE find dialog open after clicking "Find Next".

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Wed Jun 12 05:50:51 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 12 Jun 2013 05:50:51 +0200
Subject: [Python-checkins] Daily reference leaks (e6fc120012e5): sum=6
Message-ID: <E1Umc5D-0007Fk-M3@ap.vmr.nerim.net>

results for e6fc120012e5 on branch "default"
--------------------------------------------

test_support leaked [0, -1, 1] references, sum=0
test_support leaked [0, -1, 3] memory blocks, sum=2
test_concurrent_futures leaked [2, 1, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogp6B6A7', '-x']

From python-checkins at python.org  Wed Jun 12 08:29:05 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 12 Jun 2013 08:29:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Add_tests_for_?=
	=?utf-8?q?issue_=2318183=2E?=
Message-ID: <3bVdTF4f2nzNCr@mail.python.org>

http://hg.python.org/cpython/rev/b11507395ce4
changeset:   84100:b11507395ce4
branch:      3.3
parent:      84098:0e56d4e37777
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Jun 12 09:20:44 2013 +0300
summary:
  Add tests for issue #18183.

files:
  Lib/test/test_unicode.py |  18 +++++++++++++++---
  1 files changed, 15 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py
+++ b/Lib/test/test_unicode.py
@@ -566,9 +566,6 @@
         self.assertEqual('\U0008fffe'.lower(), '\U0008fffe')
         self.assertEqual('\u2177'.lower(), '\u2177')
 
-        # See issue #18183 for this one.
-        '\U00010000\U00100000'.lower()
-
     def test_casefold(self):
         self.assertEqual('hello'.casefold(), 'hello')
         self.assertEqual('hELlo'.casefold(), 'hello')
@@ -686,6 +683,17 @@
 
         self.assertRaises(TypeError, "abc".__contains__)
 
+    def test_issue18183(self):
+        '\U00010000\U00100000'.lower()
+        '\U00010000\U00100000'.casefold()
+        '\U00010000\U00100000'.upper()
+        '\U00010000\U00100000'.capitalize()
+        '\U00010000\U00100000'.title()
+        '\U00010000\U00100000'.swapcase()
+        '\U00100000'.center(3, '\U00010000')
+        '\U00100000'.ljust(3, '\U00010000')
+        '\U00100000'.rjust(3, '\U00010000')
+
     def test_format(self):
         self.assertEqual(''.format(), '')
         self.assertEqual('a'.format(), 'a')
@@ -2016,6 +2024,10 @@
         # test "%c"
         self.assertEqual(PyUnicode_FromFormat(b'%c', c_int(0xabcd)), '\uabcd')
         self.assertEqual(PyUnicode_FromFormat(b'%c', c_int(0x10ffff)), '\U0010ffff')
+        # Issue #18183
+        self.assertEqual(
+            PyUnicode_FromFormat(b'%c%c', c_int(0x10000), c_int(0x100000)),
+            '\U00010000\U00100000')
 
         # test "%"
         self.assertEqual(PyUnicode_FromFormat(b'%'), '%')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 08:29:06 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 12 Jun 2013 08:29:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Add_tests_for_issue_=2318183=2E?=
Message-ID: <3bVdTG6gD2zQRq@mail.python.org>

http://hg.python.org/cpython/rev/17c9f1627baf
changeset:   84101:17c9f1627baf
parent:      84099:479aad3bb122
parent:      84100:b11507395ce4
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Jun 12 09:28:20 2013 +0300
summary:
  Add tests for issue #18183.

files:
  Lib/test/test_unicode.py |  17 ++++++++++++++---
  1 files changed, 14 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py
+++ b/Lib/test/test_unicode.py
@@ -577,9 +577,6 @@
         self.assertEqual('\U0008fffe'.lower(), '\U0008fffe')
         self.assertEqual('\u2177'.lower(), '\u2177')
 
-        # See issue #18183 for this one.
-        '\U00010000\U00100000'.lower()
-
     def test_casefold(self):
         self.assertEqual('hello'.casefold(), 'hello')
         self.assertEqual('hELlo'.casefold(), 'hello')
@@ -697,6 +694,17 @@
 
         self.assertRaises(TypeError, "abc".__contains__)
 
+    def test_issue18183(self):
+        '\U00010000\U00100000'.lower()
+        '\U00010000\U00100000'.casefold()
+        '\U00010000\U00100000'.upper()
+        '\U00010000\U00100000'.capitalize()
+        '\U00010000\U00100000'.title()
+        '\U00010000\U00100000'.swapcase()
+        '\U00100000'.center(3, '\U00010000')
+        '\U00100000'.ljust(3, '\U00010000')
+        '\U00100000'.rjust(3, '\U00010000')
+
     def test_format(self):
         self.assertEqual(''.format(), '')
         self.assertEqual('a'.format(), 'a')
@@ -2040,6 +2048,9 @@
                      b'%c', c_int(0xabcd))
         check_format('\U0010ffff',
                      b'%c', c_int(0x10ffff))
+        # Issue #18183
+        check_format('\U00010000\U00100000',
+                     b'%c%c', c_int(0x10000), c_int(0x100000))
 
         # test "%"
         check_format('%',

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 21:57:10 2013
From: python-checkins at python.org (brett.cannon)
Date: Wed, 12 Jun 2013 21:57:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_code_from_test=5Fimpo?=
 =?utf-8?q?rthooks_into_test=5Fzipimport=2E?=
Message-ID: <3bVzPf5t9rzM9b@mail.python.org>

http://hg.python.org/cpython/rev/3e10025346c1
changeset:   84102:3e10025346c1
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 15:57:01 2013 -0400
summary:
  Move code from test_importhooks into test_zipimport.

files:
  Lib/test/test_zipimport.py |  26 +++++++++++++++++++++++++-
  1 files changed, 25 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py
--- a/Lib/test/test_zipimport.py
+++ b/Lib/test/test_zipimport.py
@@ -7,7 +7,6 @@
 import unittest
 
 from test import support
-from test.test_importhooks import ImportHooksBaseTestCase, test_src, test_co
 
 from zipfile import ZipFile, ZipInfo, ZIP_STORED, ZIP_DEFLATED
 
@@ -17,6 +16,14 @@
 import inspect
 import io
 from traceback import extract_tb, extract_stack, print_tb
+
+test_src = """\
+def get_name():
+    return __name__
+def get_file():
+    return __file__
+"""
+test_co = compile(test_src, "<???>", "exec")
 raise_src = 'def do_raise(): raise TypeError\n'
 
 def make_pyc(co, mtime, size):
@@ -46,6 +53,23 @@
 pyc_ext = ('.pyc' if __debug__ else '.pyo')
 
 
+class ImportHooksBaseTestCase(unittest.TestCase):
+
+    def setUp(self):
+        self.path = sys.path[:]
+        self.meta_path = sys.meta_path[:]
+        self.path_hooks = sys.path_hooks[:]
+        sys.path_importer_cache.clear()
+        self.modules_before = support.modules_setup()
+
+    def tearDown(self):
+        sys.path[:] = self.path
+        sys.meta_path[:] = self.meta_path
+        sys.path_hooks[:] = self.path_hooks
+        sys.path_importer_cache.clear()
+        support.modules_cleanup(*self.modules_before)
+
+
 class UncompressedZipImportTestCase(ImportHooksBaseTestCase):
 
     compression = ZIP_STORED

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 12 22:59:56 2013
From: python-checkins at python.org (brett.cannon)
Date: Wed, 12 Jun 2013 22:59:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315767=3A_Introduc?=
 =?utf-8?q?e_ModuleNotFoundError=2C_a_subclass_of?=
Message-ID: <3bW0p40Z8PzSy9@mail.python.org>

http://hg.python.org/cpython/rev/8a0ed9f63c6e
changeset:   84103:8a0ed9f63c6e
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 16:59:46 2013 -0400
summary:
  Issue #15767: Introduce ModuleNotFoundError, a subclass of
ImportError.

The exception is raised by import when a module could not be found.
Technically this is defined as no viable loader could be found for the
specified module. This includes ``from ... import`` statements so that
the module usage is consistent for all situations where import
couldn't find what was requested.

This should allow for the common idiom of::

  try:
    import something
  except ImportError:
    pass

to be updated to using ModuleNotFoundError and not accidentally mask
ImportError messages that should propagate (e.g. issues with a
loader).

This work was driven by the fact that the ``from ... import``
statement needed to be able to tell the difference between an
ImportError that simply couldn't find a module (and thus silence the
exception so that ceval can raise it) and an ImportError that
represented an actual problem.

files:
  Doc/c-api/exceptions.rst                         |    2 +
  Doc/library/exceptions.rst                       |   13 +-
  Doc/whatsnew/3.4.rst                             |    3 +
  Include/pyerrors.h                               |    1 +
  Lib/importlib/_bootstrap.py                      |   15 +-
  Lib/pydoc.py                                     |    2 +-
  Lib/test/exception_hierarchy.txt                 |    1 +
  Lib/test/test_exceptions.py                      |    3 -
  Lib/test/test_import.py                          |   25 +-
  Lib/test/test_importlib/import_/test_api.py      |    4 +
  Lib/test/test_importlib/import_/test_fromlist.py |    8 +-
  Lib/test/test_pydoc.py                           |    2 +-
  Lib/test/test_site.py                            |    2 +-
  Misc/NEWS                                        |    3 +
  Objects/exceptions.c                             |    9 +
  Python/ceval.c                                   |    2 +-
  Python/importlib.h                               |  737 ++++-----
  17 files changed, 424 insertions(+), 408 deletions(-)


diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -686,6 +686,8 @@
 +-----------------------------------------+---------------------------------+----------+
 | :c:data:`PyExc_ImportError`             | :exc:`ImportError`              |          |
 +-----------------------------------------+---------------------------------+----------+
+| :c:data:`PyExc_ModuleNotFoundError`     | :exc:`ModuleNotFoundError`      |          |
++-----------------------------------------+---------------------------------+----------+
 | :c:data:`PyExc_IndexError`              | :exc:`IndexError`               |          |
 +-----------------------------------------+---------------------------------+----------+
 | :c:data:`PyExc_InterruptedError`        | :exc:`InterruptedError`         |          |
diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst
--- a/Doc/library/exceptions.rst
+++ b/Doc/library/exceptions.rst
@@ -169,8 +169,8 @@
 
 .. exception:: ImportError
 
-   Raised when an :keyword:`import` statement fails to find the module definition
-   or when a ``from ... import`` fails to find a name that is to be imported.
+   Raised when the :keyword:`import` statement has troubles trying to load a
+   module.
 
    The :attr:`name` and :attr:`path` attributes can be set using keyword-only
    arguments to the constructor. When set they represent the name of the module
@@ -180,6 +180,15 @@
    .. versionchanged:: 3.3
       Added the :attr:`name` and :attr:`path` attributes.
 
+.. exception:: ModuleNotFoundError
+
+   A subclass of :exc:`ImportError` which is raised by :keyword:`import` when a
+   module could not be located. This includes ``from ... import`` statements as
+   the specific attribute being requested cannot be known a priori to be a module
+   or some other type of object.
+
+   .. versionadded:: 3.4
+
 
 .. exception:: IndexError
 
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -137,6 +137,9 @@
 
 * Unicode database updated to UCD version 6.2.
 
+* Import now raises the new exception :exc:`ModuleNotFoundError` (subclass of
+  :exc:`ImportError`) when it cannot find something.
+
 
 
 New Modules
diff --git a/Include/pyerrors.h b/Include/pyerrors.h
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -152,6 +152,7 @@
 PyAPI_DATA(PyObject *) PyExc_FloatingPointError;
 PyAPI_DATA(PyObject *) PyExc_OSError;
 PyAPI_DATA(PyObject *) PyExc_ImportError;
+PyAPI_DATA(PyObject *) PyExc_ModuleNotFoundError;
 PyAPI_DATA(PyObject *) PyExc_IndexError;
 PyAPI_DATA(PyObject *) PyExc_KeyError;
 PyAPI_DATA(PyObject *) PyExc_KeyboardInterrupt;
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -1553,11 +1553,7 @@
             raise ImportError(msg, name=name)
     loader = _find_module(name, path)
     if loader is None:
-        exc = ImportError(_ERR_MSG.format(name), name=name)
-        # TODO(brett): switch to a proper ModuleNotFound exception in Python
-        # 3.4.
-        exc._not_found = True
-        raise exc
+        raise ModuleNotFoundError(_ERR_MSG.format(name), name=name)
     elif name not in sys.modules:
         # The parent import may have already imported this module.
         loader.load_module(name)
@@ -1643,15 +1639,12 @@
                 from_name = '{}.{}'.format(module.__name__, x)
                 try:
                     _call_with_frames_removed(import_, from_name)
-                except ImportError as exc:
+                except ModuleNotFoundError as exc:
                     # Backwards-compatibility dictates we ignore failed
                     # imports triggered by fromlist for modules that don't
                     # exist.
-                    # TODO(brett): In Python 3.4, have import raise
-                    #   ModuleNotFound and catch that.
-                    if getattr(exc, '_not_found', False):
-                        if exc.name == from_name:
-                            continue
+                    if exc.name == from_name:
+                        continue
                     raise
     return module
 
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -317,7 +317,7 @@
         elif exc is SyntaxError:
             # A SyntaxError occurred before we could execute the module.
             raise ErrorDuringImport(value.filename, info)
-        elif exc is ImportError and value.name == path:
+        elif issubclass(exc, ImportError) and value.name == path:
             # No such module in the path.
             return None
         else:
diff --git a/Lib/test/exception_hierarchy.txt b/Lib/test/exception_hierarchy.txt
--- a/Lib/test/exception_hierarchy.txt
+++ b/Lib/test/exception_hierarchy.txt
@@ -13,6 +13,7 @@
       +-- BufferError
       +-- EOFError
       +-- ImportError
+           +-- ModuleNotFoundError
       +-- LookupError
       |    +-- IndexError
       |    +-- KeyError
diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py
--- a/Lib/test/test_exceptions.py
+++ b/Lib/test/test_exceptions.py
@@ -953,8 +953,5 @@
             self.assertEqual(str(arg), str(exc))
 
 
-def test_main():
-    run_unittest(ExceptionTests, ImportErrorTests)
-
 if __name__ == '__main__':
     unittest.main()
diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -68,7 +68,15 @@
     def tearDown(self):
         unload(TESTFN)
 
-    setUp = tearDown
+    def test_import_raises_ModuleNotFoundError(self):
+        with self.assertRaises(ModuleNotFoundError):
+            import something_that_should_not_exist_anywhere
+
+    def test_from_import_raises_ModuleNotFoundError(self):
+        with self.assertRaises(ModuleNotFoundError):
+            from something_that_should_not_exist_anywhere import blah
+        with self.assertRaises(ModuleNotFoundError):
+            from importlib import something_that_should_not_exist_anywhere
 
     def test_case_sensitivity(self):
         # Brief digression to test that import is case-sensitive:  if we got
@@ -487,7 +495,7 @@
             header = f.read(12)
             code = marshal.load(f)
         constants = list(code.co_consts)
-        foreign_code = test_main.__code__
+        foreign_code = importlib.import_module.__code__
         pos = constants.index(1)
         constants[pos] = foreign_code
         code = type(code)(code.co_argcount, code.co_kwonlyargcount,
@@ -1014,16 +1022,5 @@
             importlib.SourceLoader.load_module = old_load_module
 
 
-def test_main(verbose=None):
-    run_unittest(ImportTests, PycacheTests, FilePermissionTests,
-                 PycRewritingTests, PathsTests, RelativeImportTests,
-                 OverridingImportBuiltinTests,
-                 ImportlibBootstrapTests,
-                 TestSymbolicallyLinkedPackage,
-                 ImportTracebackTests)
-
-
 if __name__ == '__main__':
-    # Test needs to be a package, so we can do relative imports.
-    from test.test_import import test_main
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_importlib/import_/test_api.py b/Lib/test/test_importlib/import_/test_api.py
--- a/Lib/test/test_importlib/import_/test_api.py
+++ b/Lib/test/test_importlib/import_/test_api.py
@@ -22,6 +22,10 @@
     """Test API-specific details for __import__ (e.g. raising the right
     exception when passing in an int for the module name)."""
 
+    def test_raises_ModuleNotFoundError(self):
+        with self.assertRaises(ModuleNotFoundError):
+            util.import_('some module that does not exist')
+
     def test_name_requires_rparition(self):
         # Raise TypeError if a non-string is passed in for the module name.
         with self.assertRaises(TypeError):
diff --git a/Lib/test/test_importlib/import_/test_fromlist.py b/Lib/test/test_importlib/import_/test_fromlist.py
--- a/Lib/test/test_importlib/import_/test_fromlist.py
+++ b/Lib/test/test_importlib/import_/test_fromlist.py
@@ -69,16 +69,16 @@
                 self.assertTrue(hasattr(module, 'module'))
                 self.assertEqual(module.module.__name__, 'pkg.module')
 
-    def test_module_from_package_triggers_ImportError(self):
-        # If a submodule causes an ImportError because it tries to import
-        # a module which doesn't exist, that should let the ImportError
+    def test_module_from_package_triggers_ModuleNotFoundError(self):
+        # If a submodule causes an ModuleNotFoundError because it tries to import
+        # a module which doesn't exist, that should let the ModuleNotFoundError
         # propagate.
         def module_code():
             import i_do_not_exist
         with util.mock_modules('pkg.__init__', 'pkg.mod',
                                module_code={'pkg.mod': module_code}) as importer:
             with util.import_state(meta_path=[importer]):
-                with self.assertRaises(ImportError) as exc:
+                with self.assertRaises(ModuleNotFoundError) as exc:
                     import_util.import_('pkg', fromlist=['mod'])
                 self.assertEqual('i_do_not_exist', exc.exception.name)
 
diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py
--- a/Lib/test/test_pydoc.py
+++ b/Lib/test/test_pydoc.py
@@ -206,7 +206,7 @@
 missing_pattern = "no Python documentation found for '%s'"
 
 # output pattern for module with bad imports
-badimport_pattern = "problem in %s - ImportError: No module named %r"
+badimport_pattern = "problem in %s - ModuleNotFoundError: No module named %r"
 
 def run_pydoc(module_name, *args, **env):
     """
diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py
--- a/Lib/test/test_site.py
+++ b/Lib/test/test_site.py
@@ -131,7 +131,7 @@
             re.escape(os.path.join(pth_dir, pth_fn)))
         # XXX: ditto previous XXX comment.
         self.assertRegex(err_out.getvalue(), 'Traceback')
-        self.assertRegex(err_out.getvalue(), 'ImportError')
+        self.assertRegex(err_out.getvalue(), 'ModuleNotFoundError')
 
     @unittest.skipIf(sys.platform == "win32", "Windows does not raise an "
                       "error for file paths containing null characters")
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #15767: Introduce ModuleNotFoundError which is raised when a module
+  could not be found.
+
 - Issue #18183: Fix various unicode operations on strings with large unicode
   codepoints.
 
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -710,6 +710,13 @@
                         "module.");
 
 /*
+ *    ModuleNotFoundError extends ImportError
+ */
+
+MiddlingExtendsException(PyExc_ImportError, ModuleNotFoundError, ImportError,
+                         "Module not found.");
+
+/*
  *    OSError extends Exception
  */
 
@@ -2395,6 +2402,7 @@
     PRE_INIT(SystemExit)
     PRE_INIT(KeyboardInterrupt)
     PRE_INIT(ImportError)
+    PRE_INIT(ModuleNotFoundError)
     PRE_INIT(OSError)
     PRE_INIT(EOFError)
     PRE_INIT(RuntimeError)
@@ -2465,6 +2473,7 @@
     POST_INIT(SystemExit)
     POST_INIT(KeyboardInterrupt)
     POST_INIT(ImportError)
+    POST_INIT(ModuleNotFoundError)
     POST_INIT(OSError)
     INIT_ALIAS(EnvironmentError, OSError)
     INIT_ALIAS(IOError, OSError)
diff --git a/Python/ceval.c b/Python/ceval.c
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -4588,7 +4588,7 @@
 
     x = PyObject_GetAttr(v, name);
     if (x == NULL && PyErr_ExceptionMatches(PyExc_AttributeError)) {
-        PyErr_Format(PyExc_ImportError, "cannot import name %S", name);
+        PyErr_Format(PyExc_ModuleNotFoundError, "cannot import name %S", name);
     }
     return x;
 }
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 01:57:31 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 01:57:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_test=5Fzipfile_to_uni?=
	=?utf-8?b?dHRlc3QubWFpbigp?=
Message-ID: <3bW4kz5Gf9z7LjN@mail.python.org>

http://hg.python.org/cpython/rev/78b8238d679f
changeset:   84104:78b8238d679f
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 19:57:19 2013 -0400
summary:
  Move test_zipfile to unittest.main()

files:
  Lib/test/test_zipfile.py |  8 +-------
  1 files changed, 1 insertions(+), 7 deletions(-)


diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py
--- a/Lib/test/test_zipfile.py
+++ b/Lib/test/test_zipfile.py
@@ -1911,11 +1911,5 @@
         unlink(TESTFN2)
 
 
-def test_main():
-    run_unittest(TestsWithSourceFile, TestZip64InSmallFiles, OtherTests,
-                 PyZipFileTests, DecryptionTests, TestsWithMultipleOpens,
-                 TestWithDirectory, UniversalNewlineTests,
-                 TestsWithRandomBinaryFiles)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 02:04:28 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 02:04:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Spruce_up_test=5Fxmlrpc_by?=
 =?utf-8?q?_using_ModuleNotFoundError_and_moving_to?=
Message-ID: <3bW4v01lB2zSZL@mail.python.org>

http://hg.python.org/cpython/rev/281857369a78
changeset:   84105:281857369a78
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 20:04:19 2013 -0400
summary:
  Spruce up test_xmlrpc by using ModuleNotFoundError and moving to
unittest.main().

files:
  Lib/test/test_xmlrpc.py |  29 +++++++----------------------
  1 files changed, 7 insertions(+), 22 deletions(-)


diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py
--- a/Lib/test/test_xmlrpc.py
+++ b/Lib/test/test_xmlrpc.py
@@ -14,8 +14,12 @@
 from test import support
 
 try:
+    import gzip
+except ModuleNotFoundError:
+    gzip = None
+try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     threading = None
 
 alist = [{'astring': 'foo at bar.baz.spam',
@@ -785,6 +789,7 @@
 
 #A test case that verifies that gzip encoding works in both directions
 #(for a request and the response)
+ at unittest.skipIf(gzip is None, 'requires gzip')
 class GzipServerTestCase(BaseServerTestCase):
     #a request handler that supports keep-alive and logs requests into a
     #class variable
@@ -1074,25 +1079,5 @@
         self.assertTrue(server.use_builtin_types)
 
 
- at support.reap_threads
-def test_main():
-    xmlrpc_tests = [XMLRPCTestCase, HelperTestCase, DateTimeTestCase,
-         BinaryTestCase, FaultTestCase]
-    xmlrpc_tests.append(UseBuiltinTypesTestCase)
-    xmlrpc_tests.append(SimpleServerTestCase)
-    xmlrpc_tests.append(KeepaliveServerTestCase1)
-    xmlrpc_tests.append(KeepaliveServerTestCase2)
-    try:
-        import gzip
-        xmlrpc_tests.append(GzipServerTestCase)
-    except ImportError:
-        pass #gzip not supported in this build
-    xmlrpc_tests.append(MultiPathServerTestCase)
-    xmlrpc_tests.append(ServerProxyTestCase)
-    xmlrpc_tests.append(FailingServerTestCase)
-    xmlrpc_tests.append(CGIHandlerTestCase)
-
-    support.run_unittest(*xmlrpc_tests)
-
 if __name__ == "__main__":
-    test_main()
+    support.reap_threads(unittest.main)()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 02:12:38 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 02:12:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogTW92ZSB0ZXN0X19fYWxsX18g?=
 =?utf-8?q?over_to_unittest=2Emain=28=29_and_use_ModuleNotFoundError?=
Message-ID: <3bW54Q4xGQzRrw@mail.python.org>

http://hg.python.org/cpython/rev/c4d7228421df
changeset:   84106:c4d7228421df
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 20:12:30 2013 -0400
summary:
  Move test___all__ over to unittest.main() and use ModuleNotFoundError

files:
  Lib/test/regrtest.py     |  16 ++++++++--------
  Lib/test/support.py      |  14 +++++++-------
  Lib/test/test___all__.py |   7 ++-----
  Lib/xmlrpc/server.py     |   2 +-
  Lib/zipfile.py           |   6 +++---
  5 files changed, 21 insertions(+), 24 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -146,11 +146,11 @@
 
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     threading = None
 try:
     import multiprocessing.process
-except ImportError:
+except ModuleNotFoundError:
     multiprocessing = None
 
 
@@ -180,7 +180,7 @@
 if sys.platform == 'darwin':
     try:
         import resource
-    except ImportError:
+    except ModuleNotFoundError:
         pass
     else:
         soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
@@ -571,7 +571,7 @@
     if findleaks:
         try:
             import gc
-        except ImportError:
+        except ModuleNotFoundError:
             print('No GC available, disabling findleaks.')
             findleaks = False
         else:
@@ -692,7 +692,7 @@
     if use_mp:
         try:
             from threading import Thread
-        except ImportError:
+        except ModuleNotFoundError:
             print("Multiprocess option requires thread support")
             sys.exit(2)
         from queue import Queue
@@ -1396,7 +1396,7 @@
     pic = sys.path_importer_cache.copy()
     try:
         import zipimport
-    except ImportError:
+    except ModuleNotFoundError:
         zdc = None # Run unmodified on platforms without zipimport support
     else:
         zdc = zipimport._zip_directory_cache.copy()
@@ -1473,7 +1473,7 @@
     sys.path_importer_cache.update(pic)
     try:
         import zipimport
-    except ImportError:
+    except ModuleNotFoundError:
         pass # Run unmodified on platforms without zipimport support
     else:
         zipimport._zip_directory_cache.clear()
@@ -1510,7 +1510,7 @@
     doctest.master = None
     try:
         import ctypes
-    except ImportError:
+    except ModuleNotFoundError:
         # Don't worry about resetting the cache if ctypes is not supported
         pass
     else:
diff --git a/Lib/test/support.py b/Lib/test/support.py
--- a/Lib/test/support.py
+++ b/Lib/test/support.py
@@ -29,27 +29,27 @@
 
 try:
     import _thread, threading
-except ImportError:
+except ModuleNotFoundError:
     _thread = None
     threading = None
 try:
     import multiprocessing.process
-except ImportError:
+except ModuleNotFoundError:
     multiprocessing = None
 
 try:
     import zlib
-except ImportError:
+except ModuleNotFoundError:
     zlib = None
 
 try:
     import bz2
-except ImportError:
+except ModuleNotFoundError:
     bz2 = None
 
 try:
     import lzma
-except ImportError:
+except ModuleNotFoundError:
     lzma = None
 
 __all__ = [
@@ -116,7 +116,7 @@
     with _ignore_deprecated_imports(deprecated):
         try:
             return importlib.import_module(name)
-        except ImportError as msg:
+        except ModuleNotFoundError as msg:
             if sys.platform.startswith(tuple(required_on)):
                 raise
             raise unittest.SkipTest(str(msg))
@@ -188,7 +188,7 @@
                 if not _save_and_block_module(blocked_name, orig_modules):
                     names_to_remove.append(blocked_name)
             fresh_module = importlib.import_module(name)
-        except ImportError:
+        except ModuleNotFoundError:
             fresh_module = None
         finally:
             for orig_name, module in orig_modules.items():
diff --git a/Lib/test/test___all__.py b/Lib/test/test___all__.py
--- a/Lib/test/test___all__.py
+++ b/Lib/test/test___all__.py
@@ -75,7 +75,7 @@
         try:
             import rlcompleter
             import locale
-        except ImportError:
+        except ModuleNotFoundError:
             pass
         else:
             locale.setlocale(locale.LC_CTYPE, 'C')
@@ -113,8 +113,5 @@
             print('Following modules failed to be imported:', failed_imports)
 
 
-def test_main():
-    support.run_unittest(AllTest)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/xmlrpc/server.py b/Lib/xmlrpc/server.py
--- a/Lib/xmlrpc/server.py
+++ b/Lib/xmlrpc/server.py
@@ -116,7 +116,7 @@
 import traceback
 try:
     import fcntl
-except ImportError:
+except ModuleNotFoundError:
     fcntl = None
 
 def resolve_dotted_attribute(obj, attr, allow_dotted_names=True):
diff --git a/Lib/zipfile.py b/Lib/zipfile.py
--- a/Lib/zipfile.py
+++ b/Lib/zipfile.py
@@ -18,18 +18,18 @@
 try:
     import zlib # We may need its compression method
     crc32 = zlib.crc32
-except ImportError:
+except ModuleNotFoundError:
     zlib = None
     crc32 = binascii.crc32
 
 try:
     import bz2 # We may need its compression method
-except ImportError:
+except ModuleNotFoundError:
     bz2 = None
 
 try:
     import lzma # We may need its compression method
-except ImportError:
+except ModuleNotFoundError:
     lzma = None
 
 __all__ = ["BadZipFile", "BadZipfile", "error",

-- 
Repository URL: http://hg.python.org/cpython

From brett at python.org  Thu Jun 13 02:13:49 2013
From: brett at python.org (Brett Cannon)
Date: Wed, 12 Jun 2013 20:13:49 -0400
Subject: [Python-checkins] cpython: Move test___all__ over to
 unittest.main() and use ModuleNotFoundError
In-Reply-To: <3bW54Q4xGQzRrw@mail.python.org>
References: <3bW54Q4xGQzRrw@mail.python.org>
Message-ID: <CAP1=2W5wGOgRcyk0UQ9whZWKbZS00Vw1sHcPYqC41iM-ETT6vQ@mail.gmail.com>

Obviously the commit message is a little misleading since changes I was
about to stage accidentally went in on this change. Sorry about that. Same
basic concept of the changes, just to more modules.


On Wed, Jun 12, 2013 at 8:12 PM, brett.cannon <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/c4d7228421df
> changeset:   84106:c4d7228421df
> user:        Brett Cannon <brett at python.org>
> date:        Wed Jun 12 20:12:30 2013 -0400
> summary:
>   Move test___all__ over to unittest.main() and use ModuleNotFoundError
>
> files:
>   Lib/test/regrtest.py     |  16 ++++++++--------
>   Lib/test/support.py      |  14 +++++++-------
>   Lib/test/test___all__.py |   7 ++-----
>   Lib/xmlrpc/server.py     |   2 +-
>   Lib/zipfile.py           |   6 +++---
>   5 files changed, 21 insertions(+), 24 deletions(-)
>
>
> diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
> --- a/Lib/test/regrtest.py
> +++ b/Lib/test/regrtest.py
> @@ -146,11 +146,11 @@
>
>  try:
>      import threading
> -except ImportError:
> +except ModuleNotFoundError:
>      threading = None
>  try:
>      import multiprocessing.process
> -except ImportError:
> +except ModuleNotFoundError:
>      multiprocessing = None
>
>
> @@ -180,7 +180,7 @@
>  if sys.platform == 'darwin':
>      try:
>          import resource
> -    except ImportError:
> +    except ModuleNotFoundError:
>          pass
>      else:
>          soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
> @@ -571,7 +571,7 @@
>      if findleaks:
>          try:
>              import gc
> -        except ImportError:
> +        except ModuleNotFoundError:
>              print('No GC available, disabling findleaks.')
>              findleaks = False
>          else:
> @@ -692,7 +692,7 @@
>      if use_mp:
>          try:
>              from threading import Thread
> -        except ImportError:
> +        except ModuleNotFoundError:
>              print("Multiprocess option requires thread support")
>              sys.exit(2)
>          from queue import Queue
> @@ -1396,7 +1396,7 @@
>      pic = sys.path_importer_cache.copy()
>      try:
>          import zipimport
> -    except ImportError:
> +    except ModuleNotFoundError:
>          zdc = None # Run unmodified on platforms without zipimport support
>      else:
>          zdc = zipimport._zip_directory_cache.copy()
> @@ -1473,7 +1473,7 @@
>      sys.path_importer_cache.update(pic)
>      try:
>          import zipimport
> -    except ImportError:
> +    except ModuleNotFoundError:
>          pass # Run unmodified on platforms without zipimport support
>      else:
>          zipimport._zip_directory_cache.clear()
> @@ -1510,7 +1510,7 @@
>      doctest.master = None
>      try:
>          import ctypes
> -    except ImportError:
> +    except ModuleNotFoundError:
>          # Don't worry about resetting the cache if ctypes is not supported
>          pass
>      else:
> diff --git a/Lib/test/support.py b/Lib/test/support.py
> --- a/Lib/test/support.py
> +++ b/Lib/test/support.py
> @@ -29,27 +29,27 @@
>
>  try:
>      import _thread, threading
> -except ImportError:
> +except ModuleNotFoundError:
>      _thread = None
>      threading = None
>  try:
>      import multiprocessing.process
> -except ImportError:
> +except ModuleNotFoundError:
>      multiprocessing = None
>
>  try:
>      import zlib
> -except ImportError:
> +except ModuleNotFoundError:
>      zlib = None
>
>  try:
>      import bz2
> -except ImportError:
> +except ModuleNotFoundError:
>      bz2 = None
>
>  try:
>      import lzma
> -except ImportError:
> +except ModuleNotFoundError:
>      lzma = None
>
>  __all__ = [
> @@ -116,7 +116,7 @@
>      with _ignore_deprecated_imports(deprecated):
>          try:
>              return importlib.import_module(name)
> -        except ImportError as msg:
> +        except ModuleNotFoundError as msg:
>              if sys.platform.startswith(tuple(required_on)):
>                  raise
>              raise unittest.SkipTest(str(msg))
> @@ -188,7 +188,7 @@
>                  if not _save_and_block_module(blocked_name, orig_modules):
>                      names_to_remove.append(blocked_name)
>              fresh_module = importlib.import_module(name)
> -        except ImportError:
> +        except ModuleNotFoundError:
>              fresh_module = None
>          finally:
>              for orig_name, module in orig_modules.items():
> diff --git a/Lib/test/test___all__.py b/Lib/test/test___all__.py
> --- a/Lib/test/test___all__.py
> +++ b/Lib/test/test___all__.py
> @@ -75,7 +75,7 @@
>          try:
>              import rlcompleter
>              import locale
> -        except ImportError:
> +        except ModuleNotFoundError:
>              pass
>          else:
>              locale.setlocale(locale.LC_CTYPE, 'C')
> @@ -113,8 +113,5 @@
>              print('Following modules failed to be imported:',
> failed_imports)
>
>
> -def test_main():
> -    support.run_unittest(AllTest)
> -
>  if __name__ == "__main__":
> -    test_main()
> +    unittest.main()
> diff --git a/Lib/xmlrpc/server.py b/Lib/xmlrpc/server.py
> --- a/Lib/xmlrpc/server.py
> +++ b/Lib/xmlrpc/server.py
> @@ -116,7 +116,7 @@
>  import traceback
>  try:
>      import fcntl
> -except ImportError:
> +except ModuleNotFoundError:
>      fcntl = None
>
>  def resolve_dotted_attribute(obj, attr, allow_dotted_names=True):
> diff --git a/Lib/zipfile.py b/Lib/zipfile.py
> --- a/Lib/zipfile.py
> +++ b/Lib/zipfile.py
> @@ -18,18 +18,18 @@
>  try:
>      import zlib # We may need its compression method
>      crc32 = zlib.crc32
> -except ImportError:
> +except ModuleNotFoundError:
>      zlib = None
>      crc32 = binascii.crc32
>
>  try:
>      import bz2 # We may need its compression method
> -except ImportError:
> +except ModuleNotFoundError:
>      bz2 = None
>
>  try:
>      import lzma # We may need its compression method
> -except ImportError:
> +except ModuleNotFoundError:
>      lzma = None
>
>  __all__ = ["BadZipFile", "BadZipfile", "error",
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130612/28ccaba8/attachment-0001.html>

From python-checkins at python.org  Thu Jun 13 03:26:09 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 03:26:09 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Partially_revert_changeset?=
 =?utf-8?q?_=23281857369a78_to_make_sure_threads_are?=
Message-ID: <3bW6jF1CXrz7LjP@mail.python.org>

http://hg.python.org/cpython/rev/017c431d1798
changeset:   84107:017c431d1798
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 21:25:23 2013 -0400
summary:
  Partially revert changeset #281857369a78 to make sure threads are
reaped in all situations.

files:
  Lib/test/test_xmlrpc.py |  12 +++++++++++-
  1 files changed, 11 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py
--- a/Lib/test/test_xmlrpc.py
+++ b/Lib/test/test_xmlrpc.py
@@ -1079,5 +1079,15 @@
         self.assertTrue(server.use_builtin_types)
 
 
+ at support.reap_threads
+def test_main():
+    support.run_unittest(XMLRPCTestCase, HelperTestCase, DateTimeTestCase,
+            BinaryTestCase, FaultTestCase, UseBuiltinTypesTestCase,
+            SimpleServerTestCase, KeepaliveServerTestCase1,
+            KeepaliveServerTestCase2, GzipServerTestCase,
+            MultiPathServerTestCase, ServerProxyTestCase, FailingServerTestCase,
+            CGIHandlerTestCase)
+
+
 if __name__ == "__main__":
-    support.reap_threads(unittest.main)()
+    test_main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 03:26:10 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 03:26:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_various_test_module?=
 =?utf-8?q?s_to_use_unittest=2Emain=28=29_for_test_discovery?=
Message-ID: <3bW6jG4bXwz7LkJ@mail.python.org>

http://hg.python.org/cpython/rev/05f7883ee92d
changeset:   84108:05f7883ee92d
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 21:25:59 2013 -0400
summary:
  Update various test modules to use unittest.main() for test discovery
instead of manually listing tests for test.support.run_unittest().

files:
  Lib/test/test_ast.py            |   5 +---
  Lib/test/test_asynchat.py       |   6 +----
  Lib/test/test_buffer.py         |   6 +----
  Lib/test/test_codeccallbacks.py |   4 +--
  Lib/test/test_contextlib.py     |   6 +----
  Lib/test/test_faulthandler.py   |   5 +---
  Lib/test/test_fileinput.py      |  18 +-------------
  Lib/test/test_frozen.py         |   4 +--
  Lib/test/test_getargs2.py       |  26 ++++----------------
  Lib/test/test_hashlib.py        |   4 +--
  Lib/test/test_ioctl.py          |   4 +--
  Lib/test/test_runpy.py          |   9 +------
  Lib/test/test_sched.py          |   5 +---
  Lib/test/test_shutil.py         |   6 +----
  Lib/test/test_site.py           |   4 +--
  Lib/test/test_strftime.py       |   4 +--
  Lib/test/test_sundry.py         |   5 +---
  Lib/test/test_urllib2net.py     |  11 +-------
  18 files changed, 24 insertions(+), 108 deletions(-)


diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py
--- a/Lib/test/test_ast.py
+++ b/Lib/test/test_ast.py
@@ -951,9 +951,6 @@
             compile(mod, fn, "exec")
 
 
-def test_main():
-    support.run_unittest(AST_Tests, ASTHelpers_Test, ASTValidatorTests)
-
 def main():
     if __name__ != '__main__':
         return
@@ -966,7 +963,7 @@
             print("]")
         print("main()")
         raise SystemExit
-    test_main()
+    unittest.main()
 
 #### EVERYTHING BELOW IS GENERATED #####
 exec_results = [
diff --git a/Lib/test/test_asynchat.py b/Lib/test/test_asynchat.py
--- a/Lib/test/test_asynchat.py
+++ b/Lib/test/test_asynchat.py
@@ -283,9 +283,5 @@
         self.assertEqual(f.pop(), (0, None))
 
 
-def test_main(verbose=None):
-    support.run_unittest(TestAsynchat, TestAsynchat_WithPoll,
-                              TestHelperFunctions, TestFifo)
-
 if __name__ == "__main__":
-    test_main(verbose=True)
+    unittest.main()
diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py
--- a/Lib/test/test_buffer.py
+++ b/Lib/test/test_buffer.py
@@ -4283,9 +4283,5 @@
         self.assertRaises(BufferError, memoryview, x)
 
 
-def test_main():
-    support.run_unittest(TestBufferProtocol)
-
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_codeccallbacks.py b/Lib/test/test_codeccallbacks.py
--- a/Lib/test/test_codeccallbacks.py
+++ b/Lib/test/test_codeccallbacks.py
@@ -875,8 +875,6 @@
                 with self.assertRaises(TypeError):
                     data.decode(encoding, "test.replacing")
 
-def test_main():
-    test.support.run_unittest(CodecCallbackTest)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_contextlib.py b/Lib/test/test_contextlib.py
--- a/Lib/test/test_contextlib.py
+++ b/Lib/test/test_contextlib.py
@@ -617,9 +617,5 @@
             'Hello'[50]
 
 
-# This is needed to make the test actually run under regrtest.py!
-def test_main():
-    support.run_unittest(__name__)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -588,8 +588,5 @@
         self.check_register(chain=True)
 
 
-def test_main():
-    support.run_unittest(FaultHandlerTests)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py
--- a/Lib/test/test_fileinput.py
+++ b/Lib/test/test_fileinput.py
@@ -835,22 +835,6 @@
         self.assertIs(kwargs.pop('encoding'), encoding)
         self.assertFalse(kwargs)
 
-def test_main():
-    run_unittest(
-        BufferSizesTests,
-        FileInputTests,
-        Test_fileinput_input,
-        Test_fileinput_close,
-        Test_fileinput_nextfile,
-        Test_fileinput_filename,
-        Test_fileinput_lineno,
-        Test_fileinput_filelineno,
-        Test_fileinput_fileno,
-        Test_fileinput_isfirstline,
-        Test_fileinput_isstdin,
-        Test_hook_compressed,
-        Test_hook_encoded,
-    )
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_frozen.py b/Lib/test/test_frozen.py
--- a/Lib/test/test_frozen.py
+++ b/Lib/test/test_frozen.py
@@ -72,8 +72,6 @@
         del sys.modules['__phello__']
         del sys.modules['__phello__.spam']
 
-def test_main():
-    run_unittest(FrozenTests)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py
--- a/Lib/test/test_getargs2.py
+++ b/Lib/test/test_getargs2.py
@@ -1,6 +1,10 @@
 import unittest
 from test import support
 from _testcapi import getargs_keywords, getargs_keyword_only
+try:
+    from _testcapi import getargs_L, getargs_K
+except ImportError:
+    getargs_L = None # PY_LONG_LONG not available
 
 # > How about the following counterproposal. This also changes some of
 # > the other format codes to be a little more regular.
@@ -182,6 +186,7 @@
         self.assertRaises(OverflowError, getargs_n, VERY_LARGE)
 
 
+ at unittest.skipIf(getargs_L is None, 'PY_LONG_LONG is not available')
 class LongLong_TestCase(unittest.TestCase):
     def test_L(self):
         from _testcapi import getargs_L
@@ -534,24 +539,5 @@
         self.assertIsNone(getargs_Z_hash(None))
 
 
-def test_main():
-    tests = [
-        Signed_TestCase,
-        Unsigned_TestCase,
-        Boolean_TestCase,
-        Tuple_TestCase,
-        Keywords_TestCase,
-        KeywordOnly_TestCase,
-        Bytes_TestCase,
-        Unicode_TestCase,
-    ]
-    try:
-        from _testcapi import getargs_L, getargs_K
-    except ImportError:
-        pass # PY_LONG_LONG not available
-    else:
-        tests.append(LongLong_TestCase)
-    support.run_unittest(*tests)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py
--- a/Lib/test/test_hashlib.py
+++ b/Lib/test/test_hashlib.py
@@ -494,8 +494,6 @@
 
         self.assertEqual(expected_hash, hasher.hexdigest())
 
-def test_main():
-    support.run_unittest(HashLibTestCase)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_ioctl.py b/Lib/test/test_ioctl.py
--- a/Lib/test/test_ioctl.py
+++ b/Lib/test/test_ioctl.py
@@ -86,8 +86,6 @@
             os.close(mfd)
             os.close(sfd)
 
-def test_main():
-    run_unittest(IoctlTests)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py
--- a/Lib/test/test_runpy.py
+++ b/Lib/test/test_runpy.py
@@ -575,12 +575,5 @@
             self.assertEqual(result['s'], "non-ASCII: h\xe9")
 
 
-def test_main():
-    run_unittest(
-                 ExecutionLayerTestCase,
-                 RunModuleTestCase,
-                 RunPathTestCase
-                 )
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_sched.py b/Lib/test/test_sched.py
--- a/Lib/test/test_sched.py
+++ b/Lib/test/test_sched.py
@@ -197,8 +197,5 @@
         self.assertEqual(l, [])
 
 
-def test_main():
-    support.run_unittest(TestCase)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py
--- a/Lib/test/test_shutil.py
+++ b/Lib/test/test_shutil.py
@@ -1719,9 +1719,5 @@
         self.assertEqual(expected, actual)
 
 
-def test_main():
-    support.run_unittest(TestShutil, TestMove, TestCopyFile,
-                         TermsizeTests, TestWhich)
-
 if __name__ == '__main__':
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py
--- a/Lib/test/test_site.py
+++ b/Lib/test/test_site.py
@@ -401,8 +401,6 @@
             else:
                 self.fail("sitecustomize not imported automatically")
 
-def test_main():
-    run_unittest(HelperFunctionsTests, ImportSideEffectTests)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_strftime.py b/Lib/test/test_strftime.py
--- a/Lib/test/test_strftime.py
+++ b/Lib/test/test_strftime.py
@@ -176,8 +176,6 @@
                            (e[0], e[2]))
                     print("  Expected %s, but got %s" % (e[1], result))
 
-def test_main():
-    support.run_unittest(StrftimeTest)
 
 if __name__ == '__main__':
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_sundry.py b/Lib/test/test_sundry.py
--- a/Lib/test/test_sundry.py
+++ b/Lib/test/test_sundry.py
@@ -54,8 +54,5 @@
                     print("skipping tty")
 
 
-def test_main():
-    support.run_unittest(TestUntestedModules)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Lib/test/test_urllib2net.py b/Lib/test/test_urllib2net.py
--- a/Lib/test/test_urllib2net.py
+++ b/Lib/test/test_urllib2net.py
@@ -338,13 +338,6 @@
             self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60)
 
 
-def test_main():
+if __name__ == "__main__":
     support.requires("network")
-    support.run_unittest(AuthTests,
-                         OtherNetworkTests,
-                         CloseSocketTest,
-                         TimeoutTest,
-                         )
-
-if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 05:29:29 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 05:29:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315767=3A_Touch_up?=
 =?utf-8?q?_ModuleNotFoundError_usage_by_import=2E?=
Message-ID: <3bW9RY1VTpzS4T@mail.python.org>

http://hg.python.org/cpython/rev/3a50025f1900
changeset:   84109:3a50025f1900
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 23:29:18 2013 -0400
summary:
  Issue #15767: Touch up ModuleNotFoundError usage by import.

Forgot to raise ModuleNotFoundError when None is found in sys.modules.
This led to introducing the C function PyErr_SetImportErrorSubclass()
to make setting ModuleNotFoundError easier.

Also updated the reference docs to mention ModuleNotFoundError
appropriately. Updated the docs for ModuleNotFoundError to mention the
None in sys.modules case.

Lastly, it was noticed that PyErr_SetImportError() was not setting an
exception when returning None in one case. That issue is now fixed.

files:
  Doc/c-api/exceptions.rst    |   7 +++++++
  Doc/library/exceptions.rst  |   3 ++-
  Doc/reference/import.rst    |  12 ++++++------
  Doc/whatsnew/3.4.rst        |   3 +++
  Include/pyerrors.h          |   3 +++
  Lib/importlib/_bootstrap.py |   2 +-
  Misc/NEWS                   |   4 ++++
  Python/errors.c             |  25 ++++++++++++++++++++++---
  Python/import.c             |   3 ++-
  Python/importlib.h          |   4 ++--
  10 files changed, 52 insertions(+), 14 deletions(-)


diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -292,6 +292,13 @@
 
    .. versionadded:: 3.3
 
+.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *msg, PyObject *name, PyObject *path)
+
+   Much like :c:func:`PyErr_SetImportError` but this function allows for
+   specifying a subclass of :exc:`ImportError` to raise.
+
+   .. versionadded:: 3.4
+
 
 .. c:function:: void PyErr_SyntaxLocationEx(char *filename, int lineno, int col_offset)
 
diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst
--- a/Doc/library/exceptions.rst
+++ b/Doc/library/exceptions.rst
@@ -185,7 +185,8 @@
    A subclass of :exc:`ImportError` which is raised by :keyword:`import` when a
    module could not be located. This includes ``from ... import`` statements as
    the specific attribute being requested cannot be known a priori to be a module
-   or some other type of object.
+   or some other type of object. It is also raised when ``None`` is found in
+   :data:`sys.modules`.
 
    .. versionadded:: 3.4
 
diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst
--- a/Doc/reference/import.rst
+++ b/Doc/reference/import.rst
@@ -37,7 +37,7 @@
 
 When a module is first imported, Python searches for the module and if found,
 it creates a module object [#fnmo]_, initializing it.  If the named module
-cannot be found, an :exc:`ImportError` is raised.  Python implements various
+cannot be found, an :exc:`ModuleNotFoundError` is raised.  Python implements various
 strategies to search for the named module when the import machinery is
 invoked.  These strategies can be modified and extended by using various hooks
 described in the sections below.
@@ -168,7 +168,7 @@
 This name will be used in various phases of the import search, and it may be
 the dotted path to a submodule, e.g. ``foo.bar.baz``.  In this case, Python
 first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``.
-If any of the intermediate imports fail, an :exc:`ImportError` is raised.
+If any of the intermediate imports fail, an :exc:`ModuleNotFoundError` is raised.
 
 
 The module cache
@@ -187,7 +187,7 @@
 During import, the module name is looked up in :data:`sys.modules` and if
 present, the associated value is the module satisfying the import, and the
 process completes.  However, if the value is ``None``, then an
-:exc:`ImportError` is raised.  If the module name is missing, Python will
+:exc:`ModuleNotFoundError` is raised.  If the module name is missing, Python will
 continue searching for the module.
 
 :data:`sys.modules` is writable.  Deleting a key may not destroy the
@@ -195,7 +195,7 @@
 but it will invalidate the cache entry for the named module, causing
 Python to search anew for the named module upon its next
 import. The key can also be assigned to ``None``, forcing the next import
-of the module to result in an :exc:`ImportError`.
+of the module to result in an :exc:`ModuleNotFoundError`.
 
 Beware though, as if you keep a reference to the module object,
 invalidate its cache entry in :data:`sys.modules`, and then re-import the
@@ -284,7 +284,7 @@
 If the meta path finder knows how to handle the named module, it returns a
 loader object.  If it cannot handle the named module, it returns ``None``.  If
 :data:`sys.meta_path` processing reaches the end of its list without returning
-a loader, then an :exc:`ImportError` is raised.  Any other exceptions raised
+a loader, then an :exc:`ModuleNotFoundError` is raised.  Any other exceptions raised
 are simply propagated up, aborting the import process.
 
 The :meth:`find_module()` method of meta path finders is called with two
@@ -647,7 +647,7 @@
 
 To selectively prevent import of some modules from a hook early on the
 meta path (rather than disabling the standard import system entirely),
-it is sufficient to raise :exc:`ImportError` directly from
+it is sufficient to raise :exc:`ModuleNotFoundError` directly from
 :meth:`find_module` instead of returning ``None``. The latter indicates
 that the meta path search should continue. while raising an exception
 terminates it immediately.
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -269,3 +269,6 @@
 * Frozen packages no longer set ``__path__`` to a list containg the package name
   but an empty list instead. Determing if a module is a package should be done
   using ``hasattr(module, '__path__')``.
+
+* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
+  argument is not set. Previously only ``NULL`` was returned.
diff --git a/Include/pyerrors.h b/Include/pyerrors.h
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -268,6 +268,9 @@
 
 PyAPI_FUNC(PyObject *) PyErr_SetExcWithArgsKwargs(PyObject *, PyObject *,
     PyObject *);
+
+PyAPI_FUNC(PyObject *) PyErr_SetImportErrorSubclass(PyObject *, PyObject *,
+    PyObject *, PyObject *);
 PyAPI_FUNC(PyObject *) PyErr_SetImportError(PyObject *, PyObject *,
     PyObject *);
 
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -1614,7 +1614,7 @@
         _imp.release_lock()
         message = ("import of {} halted; "
                     "None in sys.modules".format(name))
-        raise ImportError(message, name=name)
+        raise ModuleNotFoundError(message, name=name)
     _lock_unlock_module(name)
     return module
 
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -443,6 +443,10 @@
 C-API
 -----
 
+- Issue #15767: Added PyErr_SetImportErrorSubclass().
+
+- PyErr_SetImportError() now sets TypeError when its msg argument is set.
+
 - Issue #9369: The types of `char*` arguments of PyObject_CallFunction() and
   PyObject_CallMethod() now changed to `const char*`.  Based on patches by
   J?rg M?ller and Lars Buitinck.
diff --git a/Python/errors.c b/Python/errors.c
--- a/Python/errors.c
+++ b/Python/errors.c
@@ -619,12 +619,25 @@
 #endif /* MS_WINDOWS */
 
 PyObject *
-PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path)
+PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg,
+    PyObject *name, PyObject *path)
 {
+    int issubclass;
     PyObject *args, *kwargs, *error;
 
-    if (msg == NULL)
+    issubclass = PyObject_IsSubclass(exception, PyExc_ImportError);
+    if (issubclass < 0) {
         return NULL;
+    }
+    else if (!issubclass) {
+        PyErr_SetString(PyExc_TypeError, "expected a subclass of ImportError");
+        return NULL;
+    }
+
+    if (msg == NULL) {
+        PyErr_SetString(PyExc_TypeError, "expected a message argument");
+        return NULL;
+    }
 
     args = PyTuple_New(1);
     if (args == NULL)
@@ -649,7 +662,7 @@
     PyDict_SetItemString(kwargs, "name", name);
     PyDict_SetItemString(kwargs, "path", path);
 
-    error = PyObject_Call(PyExc_ImportError, args, kwargs);
+    error = PyObject_Call(exception, args, kwargs);
     if (error != NULL) {
         PyErr_SetObject((PyObject *)Py_TYPE(error), error);
         Py_DECREF(error);
@@ -661,6 +674,12 @@
     return NULL;
 }
 
+PyObject *
+PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path)
+{
+    return PyErr_SetImportErrorSubclass(PyExc_ImportError, msg, name, path);
+}
+
 void
 _PyErr_BadInternalCall(const char *filename, int lineno)
 {
diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -1436,7 +1436,8 @@
         PyObject *msg = PyUnicode_FromFormat("import of %R halted; "
                                              "None in sys.modules", abs_name);
         if (msg != NULL) {
-            PyErr_SetImportError(msg, abs_name, NULL);
+            PyErr_SetImportErrorSubclass(PyExc_ModuleNotFoundError, msg,
+                    abs_name, NULL);
             Py_DECREF(msg);
         }
         mod = NULL;
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 05:38:58 2013
From: python-checkins at python.org (brett.cannon)
Date: Thu, 13 Jun 2013 05:38:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315767=3A_Add_an_e?=
 =?utf-8?q?xplicit_test_for_raising_ModuleNotFoundError?=
Message-ID: <3bW9fV3JZ1zSBf@mail.python.org>

http://hg.python.org/cpython/rev/c6c0faaf65d7
changeset:   84110:c6c0faaf65d7
user:        Brett Cannon <brett at python.org>
date:        Wed Jun 12 23:38:50 2013 -0400
summary:
  Issue #15767: Add an explicit test for raising ModuleNotFoundError
when None in sys.modules.

files:
  Lib/test/test_importlib/import_/test_api.py |  7 +++++++
  1 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_importlib/import_/test_api.py b/Lib/test/test_importlib/import_/test_api.py
--- a/Lib/test/test_importlib/import_/test_api.py
+++ b/Lib/test/test_importlib/import_/test_api.py
@@ -26,6 +26,13 @@
         with self.assertRaises(ModuleNotFoundError):
             util.import_('some module that does not exist')
 
+    def test_raises_ModuleNotFoundError_for_None(self):
+        # None in sys.modules should raise ModuleNotFoundError.
+        with importlib_test_util.uncache('not_here'):
+            sys.modules['not_here'] = None
+            with self.assertRaises(ModuleNotFoundError):
+                util.import_('not_here')
+
     def test_name_requires_rparition(self):
         # Raise TypeError if a non-string is passed in for the module name.
         with self.assertRaises(TypeError):

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Jun 13 05:50:15 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 13 Jun 2013 05:50:15 +0200
Subject: [Python-checkins] Daily reference leaks (05f7883ee92d): sum=0
Message-ID: <E1UmyYB-0002K3-Ke@ap.vmr.nerim.net>

results for 05f7883ee92d on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogUFNR7J', '-x']

From python-checkins at python.org  Thu Jun 13 09:12:55 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 13 Jun 2013 09:12:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318048=3A_Rename_t?=
 =?utf-8?q?est=5Fpep263=2Epy_to_test=5Fsource=5Fencoding=2Epy=2E?=
Message-ID: <3bWGPM2nBVzRhf@mail.python.org>

http://hg.python.org/cpython/rev/00a199c265c3
changeset:   84111:00a199c265c3
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Jun 13 09:48:15 2013 +0300
summary:
  Issue #18048: Rename test_pep263.py to test_source_encoding.py.

files:
  Lib/test/test_pep263.py |  0 
  1 files changed, 0 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_pep263.py b/Lib/test/test_source_encoding.py
rename from Lib/test/test_pep263.py
rename to Lib/test/test_source_encoding.py

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 09:12:56 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 13 Jun 2013 09:12:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318048=3A_Rename_t?=
 =?utf-8?q?est=5Fcoding=2Epy_to_test=5Fsource=5Fencoding=2Epy=2E?=
Message-ID: <3bWGPN4Zm0zSdC@mail.python.org>

http://hg.python.org/cpython/rev/3b906421245d
changeset:   84112:3b906421245d
parent:      84110:c6c0faaf65d7
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Jun 13 09:50:42 2013 +0300
summary:
  Issue #18048: Rename test_coding.py to test_source_encoding.py.

files:
  Lib/test/test_coding.py |  0 
  1 files changed, 0 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_coding.py b/Lib/test/test_source_encoding.py
rename from Lib/test/test_coding.py
rename to Lib/test/test_source_encoding.py

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 13 09:12:58 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Thu, 13 Jun 2013 09:12:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318048=3A_Merge_test=5Fpep263=2Epy_and_test=5Fco?=
 =?utf-8?q?ding=2Epy_into?=
Message-ID: <3bWGPQ0NWFz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/464e8fd7300d
changeset:   84113:464e8fd7300d
parent:      84112:3b906421245d
parent:      84111:00a199c265c3
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Thu Jun 13 10:08:00 2013 +0300
summary:
  Issue #18048: Merge test_pep263.py and test_coding.py into test_source_encoding.py.

files:
  Lib/test/test_pep263.py          |  80 -------------------
  Lib/test/test_source_encoding.py |  83 ++++++++++++++++++-
  2 files changed, 76 insertions(+), 87 deletions(-)


diff --git a/Lib/test/test_pep263.py b/Lib/test/test_pep263.py
deleted file mode 100644
--- a/Lib/test/test_pep263.py
+++ /dev/null
@@ -1,80 +0,0 @@
-# -*- coding: koi8-r -*-
-
-import unittest
-from test import support
-
-class PEP263Test(unittest.TestCase):
-
-    def test_pep263(self):
-        self.assertEqual(
-            "?????".encode("utf-8"),
-            b'\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'
-        )
-        self.assertEqual(
-            "\?".encode("utf-8"),
-            b'\\\xd0\x9f'
-        )
-
-    def test_compilestring(self):
-        # see #1882
-        c = compile(b"\n# coding: utf-8\nu = '\xc3\xb3'\n", "dummy", "exec")
-        d = {}
-        exec(c, d)
-        self.assertEqual(d['u'], '\xf3')
-
-    def test_issue2301(self):
-        try:
-            compile(b"# coding: cp932\nprint '\x94\x4e'", "dummy", "exec")
-        except SyntaxError as v:
-            self.assertEqual(v.text, "print '\u5e74'\n")
-        else:
-            self.fail()
-
-    def test_issue4626(self):
-        c = compile("# coding=latin-1\n\u00c6 = '\u00c6'", "dummy", "exec")
-        d = {}
-        exec(c, d)
-        self.assertEqual(d['\xc6'], '\xc6')
-
-    def test_issue3297(self):
-        c = compile("a, b = '\U0001010F', '\\U0001010F'", "dummy", "exec")
-        d = {}
-        exec(c, d)
-        self.assertEqual(d['a'], d['b'])
-        self.assertEqual(len(d['a']), len(d['b']))
-        self.assertEqual(ascii(d['a']), ascii(d['b']))
-
-    def test_issue7820(self):
-        # Ensure that check_bom() restores all bytes in the right order if
-        # check_bom() fails in pydebug mode: a buffer starts with the first
-        # byte of a valid BOM, but next bytes are different
-
-        # one byte in common with the UTF-16-LE BOM
-        self.assertRaises(SyntaxError, eval, b'\xff\x20')
-
-        # two bytes in common with the UTF-8 BOM
-        self.assertRaises(SyntaxError, eval, b'\xef\xbb\x20')
-
-    def test_error_message(self):
-        compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
-        compile(b'\xef\xbb\xbf\n', 'dummy', 'exec')
-        compile(b'\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'fake'):
-            compile(b'# -*- coding: fake -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
-            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
-                    'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
-            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
-                    'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'fake'):
-            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
-            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
-
-
-def test_main():
-    support.run_unittest(PEP263Test)
-
-if __name__=="__main__":
-    test_main()
diff --git a/Lib/test/test_source_encoding.py b/Lib/test/test_source_encoding.py
--- a/Lib/test/test_source_encoding.py
+++ b/Lib/test/test_source_encoding.py
@@ -1,8 +1,80 @@
-import test.support, unittest
+# -*- coding: koi8-r -*-
+
+import unittest
 from test.support import TESTFN, unlink, unload
-import importlib, os, sys
+import importlib
+import os
+import sys
 
-class CodingTest(unittest.TestCase):
+class SourceEncodingTest(unittest.TestCase):
+
+    def test_pep263(self):
+        self.assertEqual(
+            "?????".encode("utf-8"),
+            b'\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd'
+        )
+        self.assertEqual(
+            "\?".encode("utf-8"),
+            b'\\\xd0\x9f'
+        )
+
+    def test_compilestring(self):
+        # see #1882
+        c = compile(b"\n# coding: utf-8\nu = '\xc3\xb3'\n", "dummy", "exec")
+        d = {}
+        exec(c, d)
+        self.assertEqual(d['u'], '\xf3')
+
+    def test_issue2301(self):
+        try:
+            compile(b"# coding: cp932\nprint '\x94\x4e'", "dummy", "exec")
+        except SyntaxError as v:
+            self.assertEqual(v.text, "print '\u5e74'\n")
+        else:
+            self.fail()
+
+    def test_issue4626(self):
+        c = compile("# coding=latin-1\n\u00c6 = '\u00c6'", "dummy", "exec")
+        d = {}
+        exec(c, d)
+        self.assertEqual(d['\xc6'], '\xc6')
+
+    def test_issue3297(self):
+        c = compile("a, b = '\U0001010F', '\\U0001010F'", "dummy", "exec")
+        d = {}
+        exec(c, d)
+        self.assertEqual(d['a'], d['b'])
+        self.assertEqual(len(d['a']), len(d['b']))
+        self.assertEqual(ascii(d['a']), ascii(d['b']))
+
+    def test_issue7820(self):
+        # Ensure that check_bom() restores all bytes in the right order if
+        # check_bom() fails in pydebug mode: a buffer starts with the first
+        # byte of a valid BOM, but next bytes are different
+
+        # one byte in common with the UTF-16-LE BOM
+        self.assertRaises(SyntaxError, eval, b'\xff\x20')
+
+        # two bytes in common with the UTF-8 BOM
+        self.assertRaises(SyntaxError, eval, b'\xef\xbb\x20')
+
+    def test_error_message(self):
+        compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
+        compile(b'\xef\xbb\xbf\n', 'dummy', 'exec')
+        compile(b'\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile(b'# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
+            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
+                    'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+            compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
+
     def test_bad_coding(self):
         module_name = 'bad_coding'
         self.verify_bad_module(module_name)
@@ -58,8 +130,5 @@
         self.assertTrue(c.exception.args[0].startswith(expected))
 
 
-def test_main():
-    test.support.run_unittest(CodingTest)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 02:57:40 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri, 14 Jun 2013 02:57:40 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318200=3A_Update_t?=
 =?utf-8?q?he_stdlib_=28except_tests=29_to_use?=
Message-ID: <3bWk1w4xwxzRhj@mail.python.org>

http://hg.python.org/cpython/rev/8d28d44f3a9a
changeset:   84114:8d28d44f3a9a
user:        Brett Cannon <brett at python.org>
date:        Thu Jun 13 20:57:26 2013 -0400
summary:
  Issue #18200: Update the stdlib (except tests) to use
ModuleNotFoundError.

files:
  Lib/_dummy_thread.py              |   2 +-
  Lib/_osx_support.py               |   2 +-
  Lib/_pyio.py                      |   4 ++--
  Lib/_strptime.py                  |   2 +-
  Lib/bisect.py                     |   2 +-
  Lib/bz2.py                        |   2 +-
  Lib/cmd.py                        |   4 ++--
  Lib/code.py                       |   2 +-
  Lib/collections/__init__.py       |   2 +-
  Lib/copy.py                       |   2 +-
  Lib/datetime.py                   |   2 +-
  Lib/decimal.py                    |   8 ++++----
  Lib/distutils/archive_util.py     |   2 +-
  Lib/distutils/ccompiler.py        |   7 +++----
  Lib/distutils/dist.py             |   9 ++++-----
  Lib/distutils/msvccompiler.py     |   4 ++--
  Lib/distutils/util.py             |   2 +-
  Lib/encodings/__init__.py         |   7 +++----
  Lib/ftplib.py                     |   2 +-
  Lib/functools.py                  |   6 +++---
  Lib/getopt.py                     |   2 +-
  Lib/getpass.py                    |   2 +-
  Lib/hashlib.py                    |   4 ++--
  Lib/heapq.py                      |   2 +-
  Lib/http/client.py                |   2 +-
  Lib/http/cookiejar.py             |   2 +-
  Lib/http/server.py                |   2 +-
  Lib/imaplib.py                    |   2 +-
  Lib/imp.py                        |   2 +-
  Lib/importlib/__init__.py         |   2 +-
  Lib/importlib/abc.py              |   2 +-
  Lib/inspect.py                    |   2 +-
  Lib/json/decoder.py               |   2 +-
  Lib/json/encoder.py               |   4 ++--
  Lib/json/scanner.py               |   2 +-
  Lib/lib2to3/refactor.py           |   2 +-
  Lib/locale.py                     |   2 +-
  Lib/logging/__init__.py           |   2 +-
  Lib/logging/config.py             |   2 +-
  Lib/logging/handlers.py           |   4 ++--
  Lib/macpath.py                    |   2 +-
  Lib/mailbox.py                    |   2 +-
  Lib/mimetypes.py                  |   2 +-
  Lib/multiprocessing/connection.py |   2 +-
  Lib/multiprocessing/forking.py    |   2 +-
  Lib/nntplib.py                    |   2 +-
  Lib/ntpath.py                     |   4 ++--
  Lib/operator.py                   |   2 +-
  Lib/optparse.py                   |   2 +-
  Lib/os.py                         |  14 +++++++-------
  Lib/pdb.py                        |   2 +-
  Lib/pickle.py                     |   4 ++--
  Lib/platform.py                   |  14 +++++++-------
  Lib/poplib.py                     |   2 +-
  Lib/pstats.py                     |   2 +-
  Lib/pty.py                        |   2 +-
  Lib/pydoc.py                      |   4 ++--
  Lib/queue.py                      |   4 ++--
  Lib/quopri.py                     |   2 +-
  Lib/reprlib.py                    |   2 +-
  Lib/rlcompleter.py                |   2 +-
  Lib/sched.py                      |   4 ++--
  Lib/shutil.py                     |  10 +++++-----
  Lib/site.py                       |   6 +++---
  Lib/smtpd.py                      |   2 +-
  Lib/smtplib.py                    |   2 +-
  Lib/socket.py                     |   2 +-
  Lib/socketserver.py               |   2 +-
  Lib/sqlite3/test/dbapi.py         |   2 +-
  Lib/sqlite3/test/types.py         |   2 +-
  Lib/sre_compile.py                |   2 +-
  Lib/ssl.py                        |   4 ++--
  Lib/subprocess.py                 |   2 +-
  Lib/tarfile.py                    |  12 ++++++------
  Lib/tempfile.py                   |   4 ++--
  Lib/threading.py                  |   6 +++---
  Lib/trace.py                      |   4 ++--
  Lib/urllib/request.py             |   6 +++---
  Lib/venv/__init__.py              |   2 +-
  Lib/warnings.py                   |   6 +++---
  Lib/xml/etree/ElementTree.py      |  10 +++++-----
  Lib/xml/sax/expatreader.py        |  17 +++--------------
  Lib/xmlrpc/client.py              |   2 +-
  83 files changed, 144 insertions(+), 158 deletions(-)


diff --git a/Lib/_dummy_thread.py b/Lib/_dummy_thread.py
--- a/Lib/_dummy_thread.py
+++ b/Lib/_dummy_thread.py
@@ -7,7 +7,7 @@
 
     try:
         import _thread
-    except ImportError:
+    except ModuleNotFoundError:
         import _dummy_thread as _thread
 
 """
diff --git a/Lib/_osx_support.py b/Lib/_osx_support.py
--- a/Lib/_osx_support.py
+++ b/Lib/_osx_support.py
@@ -62,7 +62,7 @@
     try:
         import tempfile
         fp = tempfile.NamedTemporaryFile()
-    except ImportError:
+    except ModuleNotFoundError:
         fp = open("/tmp/_osx_support.%s"%(
             os.getpid(),), "w+b")
 
diff --git a/Lib/_pyio.py b/Lib/_pyio.py
--- a/Lib/_pyio.py
+++ b/Lib/_pyio.py
@@ -9,7 +9,7 @@
 # Import _thread instead of threading to reduce startup cost
 try:
     from _thread import allocate_lock as Lock
-except ImportError:
+except ModuleNotFoundError:
     from _dummy_thread import allocate_lock as Lock
 
 import io
@@ -1486,7 +1486,7 @@
             if encoding is None:
                 try:
                     import locale
-                except ImportError:
+                except ModuleNotFoundError:
                     # Importing locale may fail if Python is being built
                     encoding = "ascii"
                 else:
diff --git a/Lib/_strptime.py b/Lib/_strptime.py
--- a/Lib/_strptime.py
+++ b/Lib/_strptime.py
@@ -21,7 +21,7 @@
                       timezone as datetime_timezone)
 try:
     from _thread import allocate_lock as _thread_allocate_lock
-except ImportError:
+except ModuleNotFoundError:
     from _dummy_thread import allocate_lock as _thread_allocate_lock
 
 __all__ = []
diff --git a/Lib/bisect.py b/Lib/bisect.py
--- a/Lib/bisect.py
+++ b/Lib/bisect.py
@@ -88,5 +88,5 @@
 # Overwrite above definitions with a fast C implementation
 try:
     from _bisect import *
-except ImportError:
+except ModuleNotFoundError:
     pass
diff --git a/Lib/bz2.py b/Lib/bz2.py
--- a/Lib/bz2.py
+++ b/Lib/bz2.py
@@ -14,7 +14,7 @@
 
 try:
     from threading import RLock
-except ImportError:
+except ModuleNotFoundError:
     from dummy_threading import RLock
 
 from _bz2 import BZ2Compressor, BZ2Decompressor
diff --git a/Lib/cmd.py b/Lib/cmd.py
--- a/Lib/cmd.py
+++ b/Lib/cmd.py
@@ -109,7 +109,7 @@
                 self.old_completer = readline.get_completer()
                 readline.set_completer(self.complete)
                 readline.parse_and_bind(self.completekey+": complete")
-            except ImportError:
+            except ModuleNotFoundError:
                 pass
         try:
             if intro is not None:
@@ -143,7 +143,7 @@
                 try:
                     import readline
                     readline.set_completer(self.old_completer)
-                except ImportError:
+                except ModuleNotFoundError:
                     pass
 
 
diff --git a/Lib/code.py b/Lib/code.py
--- a/Lib/code.py
+++ b/Lib/code.py
@@ -293,7 +293,7 @@
     else:
         try:
             import readline
-        except ImportError:
+        except ModuleNotFoundError:
             pass
     console.interact(banner)
 
diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py
--- a/Lib/collections/__init__.py
+++ b/Lib/collections/__init__.py
@@ -395,7 +395,7 @@
 
 try:                                    # Load C helper function if available
     from _collections import _count_elements
-except ImportError:
+except ModuleNotFoundError:
     pass
 
 class Counter(dict):
diff --git a/Lib/copy.py b/Lib/copy.py
--- a/Lib/copy.py
+++ b/Lib/copy.py
@@ -59,7 +59,7 @@
 
 try:
     from org.python.core import PyStringMap
-except ImportError:
+except ModuleNotFoundError:
     PyStringMap = None
 
 __all__ = ["Error", "copy", "deepcopy"]
diff --git a/Lib/datetime.py b/Lib/datetime.py
--- a/Lib/datetime.py
+++ b/Lib/datetime.py
@@ -2116,7 +2116,7 @@
 
 try:
     from _datetime import *
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     # Clean up unused names
diff --git a/Lib/decimal.py b/Lib/decimal.py
--- a/Lib/decimal.py
+++ b/Lib/decimal.py
@@ -149,7 +149,7 @@
 try:
     from collections import namedtuple as _namedtuple
     DecimalTuple = _namedtuple('DecimalTuple', 'sign digits exponent')
-except ImportError:
+except ModuleNotFoundError:
     DecimalTuple = lambda *args: args
 
 # Rounding
@@ -430,7 +430,7 @@
 
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     # Python was compiled without threads; create a mock object instead
     class MockThreading(object):
         def local(self, sys=sys):
@@ -6147,7 +6147,7 @@
 # don't care too much if locale isn't present.
 try:
     import locale as _locale
-except ImportError:
+except ModuleNotFoundError:
     pass
 
 def _parse_format_specifier(format_spec, _localeconv=None):
@@ -6391,7 +6391,7 @@
 
 try:
     import _decimal
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     s1 = set(dir())
diff --git a/Lib/distutils/archive_util.py b/Lib/distutils/archive_util.py
--- a/Lib/distutils/archive_util.py
+++ b/Lib/distutils/archive_util.py
@@ -9,7 +9,7 @@
 
 try:
     import zipfile
-except ImportError:
+except ModuleNotFoundError:
     zipfile = None
 
 
diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py
--- a/Lib/distutils/ccompiler.py
+++ b/Lib/distutils/ccompiler.py
@@ -3,7 +3,7 @@
 Contains CCompiler, an abstract base class that defines the interface
 for the Distutils compiler abstraction model."""
 
-import sys, os, re
+import importlib, sys, os, re
 from distutils.errors import *
 from distutils.spawn import spawn
 from distutils.file_util import move_file
@@ -1013,10 +1013,9 @@
 
     try:
         module_name = "distutils." + module_name
-        __import__ (module_name)
-        module = sys.modules[module_name]
+        module = importlib.import_module(module_name)
         klass = vars(module)[class_name]
-    except ImportError:
+    except ModuleNotFoundError:
         raise DistutilsModuleError(
               "can't compile C/C++ code: unable to load module '%s'" % \
               module_name)
diff --git a/Lib/distutils/dist.py b/Lib/distutils/dist.py
--- a/Lib/distutils/dist.py
+++ b/Lib/distutils/dist.py
@@ -4,11 +4,11 @@
 being built/installed/distributed.
 """
 
-import sys, os, re
+import importlib, sys, os, re
 
 try:
     import warnings
-except ImportError:
+except ModuleNotFoundError:
     warnings = None
 
 from distutils.errors import *
@@ -788,9 +788,8 @@
             klass_name = command
 
             try:
-                __import__ (module_name)
-                module = sys.modules[module_name]
-            except ImportError:
+                module = importlib.import_module(module_name)
+            except ModuleNotFoundError:
                 continue
 
             try:
diff --git a/Lib/distutils/msvccompiler.py b/Lib/distutils/msvccompiler.py
--- a/Lib/distutils/msvccompiler.py
+++ b/Lib/distutils/msvccompiler.py
@@ -28,7 +28,7 @@
     RegEnumValue = winreg.EnumValue
     RegError = winreg.error
 
-except ImportError:
+except ModuleNotFoundError:
     try:
         import win32api
         import win32con
@@ -39,7 +39,7 @@
         RegEnumKey = win32api.RegEnumKey
         RegEnumValue = win32api.RegEnumValue
         RegError = win32api.error
-    except ImportError:
+    except ModuleNotFoundError:
         log.info("Warning: Can't read registry to find the "
                  "necessary compiler setting\n"
                  "Make sure that Python modules winreg, "
diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py
--- a/Lib/distutils/util.py
+++ b/Lib/distutils/util.py
@@ -388,7 +388,7 @@
         try:
             from tempfile import mkstemp
             (script_fd, script_name) = mkstemp(".py")
-        except ImportError:
+        except ModuleNotFoundError:
             from tempfile import mktemp
             (script_fd, script_name) = None, mktemp(".py")
         log.info("writing byte-compilation script '%s'", script_name)
diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py
--- a/Lib/encodings/__init__.py
+++ b/Lib/encodings/__init__.py
@@ -29,11 +29,11 @@
 """#"
 
 import codecs
+import importlib
 from . import aliases
 
 _cache = {}
 _unknown = '--unknown--'
-_import_tail = ['*']
 _aliases = aliases.aliases
 
 class CodecRegistryError(LookupError, SystemError):
@@ -94,9 +94,8 @@
         try:
             # Import is absolute to prevent the possibly malicious import of a
             # module with side-effects that is not in the 'encodings' package.
-            mod = __import__('encodings.' + modname, fromlist=_import_tail,
-                             level=0)
-        except ImportError:
+            mod = importlib.import_module('encodings.' + modname)
+        except ModuleNotFoundError:
             pass
         else:
             break
diff --git a/Lib/ftplib.py b/Lib/ftplib.py
--- a/Lib/ftplib.py
+++ b/Lib/ftplib.py
@@ -667,7 +667,7 @@
 
 try:
     import ssl
-except ImportError:
+except ModuleNotFoundError:
     _SSLSocket = None
 else:
     _SSLSocket = ssl.SSLSocket
diff --git a/Lib/functools.py b/Lib/functools.py
--- a/Lib/functools.py
+++ b/Lib/functools.py
@@ -15,7 +15,7 @@
 
 try:
     from _functools import reduce
-except ImportError:
+except ModuleNotFoundError:
     pass
 from abc import get_cache_token
 from collections import namedtuple
@@ -143,7 +143,7 @@
 
 try:
     from _functools import cmp_to_key
-except ImportError:
+except ModuleNotFoundError:
     pass
 
 
@@ -166,7 +166,7 @@
 
 try:
     from _functools import partial
-except ImportError:
+except ModuleNotFoundError:
     pass
 
 
diff --git a/Lib/getopt.py b/Lib/getopt.py
--- a/Lib/getopt.py
+++ b/Lib/getopt.py
@@ -36,7 +36,7 @@
 import os
 try:
     from gettext import gettext as _
-except ImportError:
+except ModuleNotFoundError:
     # Bootstrapping Python: gettext's dependencies not built yet
     def _(s): return s
 
diff --git a/Lib/getpass.py b/Lib/getpass.py
--- a/Lib/getpass.py
+++ b/Lib/getpass.py
@@ -164,7 +164,7 @@
 except (ImportError, AttributeError):
     try:
         import msvcrt
-    except ImportError:
+    except ModuleNotFoundError:
         getpass = fallback_getpass
     else:
         getpass = win_getpass
diff --git a/Lib/hashlib.py b/Lib/hashlib.py
--- a/Lib/hashlib.py
+++ b/Lib/hashlib.py
@@ -98,7 +98,7 @@
                 return _sha3.sha3_384
             elif bs == '512':
                 return _sha3.sha3_512
-    except ImportError:
+    except ModuleNotFoundError:
         pass  # no extension module, this hash is unsupported.
 
     raise ValueError('unsupported hash type ' + name)
@@ -143,7 +143,7 @@
     __get_hash = __get_openssl_constructor
     algorithms_available = algorithms_available.union(
             _hashlib.openssl_md_meth_names)
-except ImportError:
+except ModuleNotFoundError:
     new = __py_new
     __get_hash = __get_builtin_constructor
 
diff --git a/Lib/heapq.py b/Lib/heapq.py
--- a/Lib/heapq.py
+++ b/Lib/heapq.py
@@ -343,7 +343,7 @@
 # If available, use C implementation
 try:
     from _heapq import *
-except ImportError:
+except ModuleNotFoundError:
     pass
 
 def merge(*iterables):
diff --git a/Lib/http/client.py b/Lib/http/client.py
--- a/Lib/http/client.py
+++ b/Lib/http/client.py
@@ -1156,7 +1156,7 @@
 
 try:
     import ssl
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     class HTTPSConnection(HTTPConnection):
diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py
--- a/Lib/http/cookiejar.py
+++ b/Lib/http/cookiejar.py
@@ -35,7 +35,7 @@
 import urllib.parse, urllib.request
 try:
     import threading as _threading
-except ImportError:
+except ModuleNotFoundError:
     import dummy_threading as _threading
 import http.client  # only for the default HTTP port
 from calendar import timegm
diff --git a/Lib/http/server.py b/Lib/http/server.py
--- a/Lib/http/server.py
+++ b/Lib/http/server.py
@@ -904,7 +904,7 @@
         return nobody
     try:
         import pwd
-    except ImportError:
+    except ModuleNotFoundError:
         return -1
     try:
         nobody = pwd.getpwnam('nobody')[2]
diff --git a/Lib/imaplib.py b/Lib/imaplib.py
--- a/Lib/imaplib.py
+++ b/Lib/imaplib.py
@@ -29,7 +29,7 @@
 try:
     import ssl
     HAVE_SSL = True
-except ImportError:
+except ModuleNotFoundError:
     HAVE_SSL = False
 
 __all__ = ["IMAP4", "IMAP4_stream", "Internaldate2tuple",
diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -12,7 +12,7 @@
                   _fix_co_filename)
 try:
     from _imp import load_dynamic
-except ImportError:
+except ModuleNotFoundError:
     # Platform doesn't support dynamic loading.
     load_dynamic = None
 
diff --git a/Lib/importlib/__init__.py b/Lib/importlib/__init__.py
--- a/Lib/importlib/__init__.py
+++ b/Lib/importlib/__init__.py
@@ -14,7 +14,7 @@
 
 try:
     import _frozen_importlib as _bootstrap
-except ImportError:
+except ModuleNotFoundError:
     from . import _bootstrap
     _bootstrap._setup(sys, _imp)
 else:
diff --git a/Lib/importlib/abc.py b/Lib/importlib/abc.py
--- a/Lib/importlib/abc.py
+++ b/Lib/importlib/abc.py
@@ -3,7 +3,7 @@
 from . import machinery
 try:
     import _frozen_importlib
-except ImportError as exc:
+except ModuleNotFoundError as exc:
     if exc.name != '_frozen_importlib':
         raise
     _frozen_importlib = None
diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -51,7 +51,7 @@
 # back to hardcording so the dependency is optional
 try:
     from dis import COMPILER_FLAG_NAMES as _flag_names
-except ImportError:
+except ModuleNotFoundError:
     CO_OPTIMIZED, CO_NEWLOCALS = 0x1, 0x2
     CO_VARARGS, CO_VARKEYWORDS = 0x4, 0x8
     CO_NESTED, CO_GENERATOR, CO_NOFREE = 0x10, 0x20, 0x40
diff --git a/Lib/json/decoder.py b/Lib/json/decoder.py
--- a/Lib/json/decoder.py
+++ b/Lib/json/decoder.py
@@ -5,7 +5,7 @@
 from json import scanner
 try:
     from _json import scanstring as c_scanstring
-except ImportError:
+except ModuleNotFoundError:
     c_scanstring = None
 
 __all__ = ['JSONDecoder']
diff --git a/Lib/json/encoder.py b/Lib/json/encoder.py
--- a/Lib/json/encoder.py
+++ b/Lib/json/encoder.py
@@ -4,11 +4,11 @@
 
 try:
     from _json import encode_basestring_ascii as c_encode_basestring_ascii
-except ImportError:
+except ModuleNotFoundError:
     c_encode_basestring_ascii = None
 try:
     from _json import make_encoder as c_make_encoder
-except ImportError:
+except ModuleNotFoundError:
     c_make_encoder = None
 
 ESCAPE = re.compile(r'[\x00-\x1f\\"\b\f\n\r\t]')
diff --git a/Lib/json/scanner.py b/Lib/json/scanner.py
--- a/Lib/json/scanner.py
+++ b/Lib/json/scanner.py
@@ -3,7 +3,7 @@
 import re
 try:
     from _json import make_scanner as c_make_scanner
-except ImportError:
+except ModuleNotFoundError:
     c_make_scanner = None
 
 __all__ = ['make_scanner']
diff --git a/Lib/lib2to3/refactor.py b/Lib/lib2to3/refactor.py
--- a/Lib/lib2to3/refactor.py
+++ b/Lib/lib2to3/refactor.py
@@ -706,7 +706,7 @@
                 items, write, doctests_only)
         try:
             import multiprocessing
-        except ImportError:
+        except ModuleNotFoundError:
             raise MultiprocessingUnsupported
         if self.queue is not None:
             raise RuntimeError("already doing multiple processes")
diff --git a/Lib/locale.py b/Lib/locale.py
--- a/Lib/locale.py
+++ b/Lib/locale.py
@@ -47,7 +47,7 @@
 
     from _locale import *
 
-except ImportError:
+except ModuleNotFoundError:
 
     # Locale emulation
 
diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -37,7 +37,7 @@
 
 try:
     import threading
-except ImportError: #pragma: no cover
+except ModuleNotFoundError: #pragma: no cover
     threading = None
 
 __author__  = "Vinay Sajip <vinay_sajip at red-dove.com>"
diff --git a/Lib/logging/config.py b/Lib/logging/config.py
--- a/Lib/logging/config.py
+++ b/Lib/logging/config.py
@@ -30,7 +30,7 @@
 try:
     import _thread as thread
     import threading
-except ImportError: #pragma: no cover
+except ModuleNotFoundError: #pragma: no cover
     thread = None
 
 from socketserver import ThreadingTCPServer, StreamRequestHandler
diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py
--- a/Lib/logging/handlers.py
+++ b/Lib/logging/handlers.py
@@ -29,7 +29,7 @@
 import queue
 try:
     import threading
-except ImportError: #pragma: no cover
+except ModuleNotFoundError: #pragma: no cover
     threading = None
 
 #
@@ -995,7 +995,7 @@
                 logging.ERROR   : win32evtlog.EVENTLOG_ERROR_TYPE,
                 logging.CRITICAL: win32evtlog.EVENTLOG_ERROR_TYPE,
          }
-        except ImportError:
+        except ModuleNotFoundError:
             print("The Python Win32 extensions for NT (service, event "\
                         "logging) appear not to be available.")
             self._welu = None
diff --git a/Lib/macpath.py b/Lib/macpath.py
--- a/Lib/macpath.py
+++ b/Lib/macpath.py
@@ -187,7 +187,7 @@
     path = abspath(path)
     try:
         import Carbon.File
-    except ImportError:
+    except ModuleNotFoundError:
         return path
     if not path:
         return path
diff --git a/Lib/mailbox.py b/Lib/mailbox.py
--- a/Lib/mailbox.py
+++ b/Lib/mailbox.py
@@ -23,7 +23,7 @@
 import contextlib
 try:
     import fcntl
-except ImportError:
+except ModuleNotFoundError:
     fcntl = None
 
 __all__ = [ 'Mailbox', 'Maildir', 'mbox', 'MH', 'Babyl', 'MMDF',
diff --git a/Lib/mimetypes.py b/Lib/mimetypes.py
--- a/Lib/mimetypes.py
+++ b/Lib/mimetypes.py
@@ -29,7 +29,7 @@
 import urllib.parse
 try:
     import winreg as _winreg
-except ImportError:
+except ModuleNotFoundError:
     _winreg = None
 
 __all__ = [
diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py
--- a/Lib/multiprocessing/connection.py
+++ b/Lib/multiprocessing/connection.py
@@ -27,7 +27,7 @@
 try:
     import _winapi
     from _winapi import WAIT_OBJECT_0, WAIT_TIMEOUT, INFINITE
-except ImportError:
+except ModuleNotFoundError:
     if sys.platform == 'win32':
         raise
     _winapi = None
diff --git a/Lib/multiprocessing/forking.py b/Lib/multiprocessing/forking.py
--- a/Lib/multiprocessing/forking.py
+++ b/Lib/multiprocessing/forking.py
@@ -73,7 +73,7 @@
 
 try:
     from functools import partial
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     def _reduce_partial(p):
diff --git a/Lib/nntplib.py b/Lib/nntplib.py
--- a/Lib/nntplib.py
+++ b/Lib/nntplib.py
@@ -71,7 +71,7 @@
 
 try:
     import ssl
-except ImportError:
+except ModuleNotFoundError:
     _have_ssl = False
 else:
     _have_ssl = True
diff --git a/Lib/ntpath.py b/Lib/ntpath.py
--- a/Lib/ntpath.py
+++ b/Lib/ntpath.py
@@ -566,7 +566,7 @@
 try:
     from nt import _getfullpathname
 
-except ImportError: # not running on Windows - mock up something sensible
+except ModuleNotFoundError: # not running on Windows - mock up something sensible
     def abspath(path):
         """Return the absolute version of a path."""
         if not isabs(path):
@@ -659,6 +659,6 @@
     # This is overkill on Windows - just pass the path to GetFileAttributes
     # and check the attribute from there.
     from nt import _isdir as isdir
-except ImportError:
+except ModuleNotFoundError:
     # Use genericpath.isdir as imported above.
     pass
diff --git a/Lib/operator.py b/Lib/operator.py
--- a/Lib/operator.py
+++ b/Lib/operator.py
@@ -360,7 +360,7 @@
 
 try:
     from _operator import *
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     from _operator import __doc__
diff --git a/Lib/optparse.py b/Lib/optparse.py
--- a/Lib/optparse.py
+++ b/Lib/optparse.py
@@ -87,7 +87,7 @@
 
 try:
     from gettext import gettext, ngettext
-except ImportError:
+except ModuleNotFoundError:
     def gettext(message):
         return message
 
diff --git a/Lib/os.py b/Lib/os.py
--- a/Lib/os.py
+++ b/Lib/os.py
@@ -52,13 +52,13 @@
     try:
         from posix import _exit
         __all__.append('_exit')
-    except ImportError:
+    except ModuleNotFoundError:
         pass
     import posixpath as path
 
     try:
         from posix import _have_functions
-    except ImportError:
+    except ModuleNotFoundError:
         pass
 
 elif 'nt' in _names:
@@ -68,7 +68,7 @@
     try:
         from nt import _exit
         __all__.append('_exit')
-    except ImportError:
+    except ModuleNotFoundError:
         pass
     import ntpath as path
 
@@ -78,7 +78,7 @@
 
     try:
         from nt import _have_functions
-    except ImportError:
+    except ModuleNotFoundError:
         pass
 
 elif 'ce' in _names:
@@ -88,7 +88,7 @@
     try:
         from ce import _exit
         __all__.append('_exit')
-    except ImportError:
+    except ModuleNotFoundError:
         pass
     # We can use the standard Windows path.
     import ntpath as path
@@ -99,11 +99,11 @@
 
     try:
         from ce import _have_functions
-    except ImportError:
+    except ModuleNotFoundError:
         pass
 
 else:
-    raise ImportError('no os specific module found')
+    raise ModuleNotFoundError('no os specific module found')
 
 sys.modules['os.path'] = path
 from os.path import (curdir, pardir, sep, pathsep, defpath, extsep, altsep,
diff --git a/Lib/pdb.py b/Lib/pdb.py
--- a/Lib/pdb.py
+++ b/Lib/pdb.py
@@ -158,7 +158,7 @@
             import readline
             # remove some common file name delimiters
             readline.set_completer_delims(' \t\n`@#$%^&*()=+[{]}\\|;:\'",<>?')
-        except ImportError:
+        except ModuleNotFoundError:
             pass
         self.allow_kbdint = False
         self.nosigint = nosigint
diff --git a/Lib/pickle.py b/Lib/pickle.py
--- a/Lib/pickle.py
+++ b/Lib/pickle.py
@@ -90,7 +90,7 @@
 # Jython has PyStringMap; it's a dict subclass with string keys
 try:
     from org.python.core import PyStringMap
-except ImportError:
+except ModuleNotFoundError:
     PyStringMap = None
 
 # Pickle opcodes.  See pickletools.py for extensive docs.  The listing
@@ -1296,7 +1296,7 @@
 # Use the faster _pickle if possible
 try:
     from _pickle import *
-except ImportError:
+except ModuleNotFoundError:
     Pickler, Unpickler = _Pickler, _Unpickler
 
 # Doctest
diff --git a/Lib/platform.py b/Lib/platform.py
--- a/Lib/platform.py
+++ b/Lib/platform.py
@@ -460,7 +460,7 @@
     try:
         # Use win32api if available
         from win32api import RegQueryValueEx
-    except ImportError:
+    except ModuleNotFoundError:
         # On Python 2.0 and later, emulate using winreg
         import winreg
         RegQueryValueEx = winreg.QueryValueEx
@@ -503,7 +503,7 @@
              RegCloseKey, GetVersionEx
         from win32con import HKEY_LOCAL_MACHINE, VER_PLATFORM_WIN32_NT, \
              VER_PLATFORM_WIN32_WINDOWS, VER_NT_WORKSTATION
-    except ImportError:
+    except ModuleNotFoundError:
         # Emulate the win32api module using Python APIs
         try:
             sys.getwindowsversion
@@ -661,7 +661,7 @@
     # Check whether the version info module is available
     try:
         import _gestalt
-    except ImportError:
+    except ModuleNotFoundError:
         return None
     # Get the infos
     sysv, sysa = _mac_ver_lookup(('sysv','sysa'))
@@ -697,7 +697,7 @@
 
     try:
         import plistlib
-    except ImportError:
+    except ModuleNotFoundError:
         return None
 
     pl = plistlib.readPlist(fn)
@@ -762,7 +762,7 @@
     # Import the needed APIs
     try:
         import java.lang
-    except ImportError:
+    except ModuleNotFoundError:
         return release,vendor,vminfo,osinfo
 
     vendor = _java_getprop('java.vendor', vendor)
@@ -874,7 +874,7 @@
     """
     try:
         import socket
-    except ImportError:
+    except ModuleNotFoundError:
         # No sockets...
         return default
     try:
@@ -1138,7 +1138,7 @@
         # Get processor information
         try:
             import vms_lib
-        except ImportError:
+        except ModuleNotFoundError:
             pass
         else:
             csid, cpu_number = vms_lib.getsyi('SYI$_CPU',0)
diff --git a/Lib/poplib.py b/Lib/poplib.py
--- a/Lib/poplib.py
+++ b/Lib/poplib.py
@@ -20,7 +20,7 @@
 try:
     import ssl
     HAVE_SSL = True
-except ImportError:
+except ModuleNotFoundError:
     HAVE_SSL = False
 
 __all__ = ["POP3","error_proto"]
diff --git a/Lib/pstats.py b/Lib/pstats.py
--- a/Lib/pstats.py
+++ b/Lib/pstats.py
@@ -528,7 +528,7 @@
     import cmd
     try:
         import readline
-    except ImportError:
+    except ModuleNotFoundError:
         pass
 
     class ProfileBrowser(cmd.Cmd):
diff --git a/Lib/pty.py b/Lib/pty.py
--- a/Lib/pty.py
+++ b/Lib/pty.py
@@ -67,7 +67,7 @@
     result = os.open(tty_name, os.O_RDWR)
     try:
         from fcntl import ioctl, I_PUSH
-    except ImportError:
+    except ModuleNotFoundError:
         return result
     try:
         ioctl(result, I_PUSH, "ptem")
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -1893,7 +1893,7 @@
     def showtopic(self, topic, more_xrefs=''):
         try:
             import pydoc_data.topics
-        except ImportError:
+        except ModuleNotFoundError:
             self.output.write('''
 Sorry, topic and keyword documentation is not available because the
 module "pydoc_data.topics" could not be found.
@@ -1933,7 +1933,7 @@
         """
         try:
             import pydoc_data.topics
-        except ImportError:
+        except ModuleNotFoundError:
             return('''
 Sorry, topic and keyword documentation is not available because the
 module "pydoc_data.topics" could not be found.
diff --git a/Lib/queue.py b/Lib/queue.py
--- a/Lib/queue.py
+++ b/Lib/queue.py
@@ -2,13 +2,13 @@
 
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     import dummy_threading as threading
 from collections import deque
 from heapq import heappush, heappop
 try:
     from time import monotonic as time
-except ImportError:
+except ModuleNotFoundError:
     from time import time
 
 __all__ = ['Empty', 'Full', 'Queue', 'PriorityQueue', 'LifoQueue']
diff --git a/Lib/quopri.py b/Lib/quopri.py
--- a/Lib/quopri.py
+++ b/Lib/quopri.py
@@ -13,7 +13,7 @@
 
 try:
     from binascii import a2b_qp, b2a_qp
-except ImportError:
+except ModuleNotFoundError:
     a2b_qp = None
     b2a_qp = None
 
diff --git a/Lib/reprlib.py b/Lib/reprlib.py
--- a/Lib/reprlib.py
+++ b/Lib/reprlib.py
@@ -6,7 +6,7 @@
 from itertools import islice
 try:
     from _thread import get_ident
-except ImportError:
+except ModuleNotFoundError:
     from _dummy_thread import get_ident
 
 def recursive_repr(fillvalue='...'):
diff --git a/Lib/rlcompleter.py b/Lib/rlcompleter.py
--- a/Lib/rlcompleter.py
+++ b/Lib/rlcompleter.py
@@ -154,7 +154,7 @@
 
 try:
     import readline
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     readline.set_completer(Completer().complete)
diff --git a/Lib/sched.py b/Lib/sched.py
--- a/Lib/sched.py
+++ b/Lib/sched.py
@@ -33,11 +33,11 @@
 from collections import namedtuple
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     import dummy_threading as threading
 try:
     from time import monotonic as _time
-except ImportError:
+except ModuleNotFoundError:
     from time import time as _time
 
 __all__ = ["scheduler"]
diff --git a/Lib/shutil.py b/Lib/shutil.py
--- a/Lib/shutil.py
+++ b/Lib/shutil.py
@@ -17,17 +17,17 @@
     import bz2
     del bz2
     _BZ2_SUPPORTED = True
-except ImportError:
+except ModuleNotFoundError:
     _BZ2_SUPPORTED = False
 
 try:
     from pwd import getpwnam
-except ImportError:
+except ModuleNotFoundError:
     getpwnam = None
 
 try:
     from grp import getgrnam
-except ImportError:
+except ModuleNotFoundError:
     getgrnam = None
 
 __all__ = ["copyfileobj", "copyfile", "copymode", "copystat", "copy", "copy2",
@@ -668,7 +668,7 @@
     # command.
     try:
         import zipfile
-    except ImportError:
+    except ModuleNotFoundError:
         zipfile = None
 
     if zipfile is None:
@@ -858,7 +858,7 @@
     """
     try:
         import zipfile
-    except ImportError:
+    except ModuleNotFoundError:
         raise ReadError('zlib not supported, cannot unpack this archive.')
 
     if not zipfile.is_zipfile(filename):
diff --git a/Lib/site.py b/Lib/site.py
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -469,7 +469,7 @@
         try:
             import readline
             import rlcompleter
-        except ImportError:
+        except ModuleNotFoundError:
             return
 
         # Reading the initialization (config) file may not be enough to set a
@@ -570,7 +570,7 @@
     """Run custom site specific code, if available."""
     try:
         import sitecustomize
-    except ImportError:
+    except ModuleNotFoundError:
         pass
     except Exception as err:
         if os.environ.get("PYTHONVERBOSE"):
@@ -586,7 +586,7 @@
     """Run custom user specific code, if available."""
     try:
         import usercustomize
-    except ImportError:
+    except ModuleNotFoundError:
         pass
     except Exception as err:
         if os.environ.get("PYTHONVERBOSE"):
diff --git a/Lib/smtpd.py b/Lib/smtpd.py
--- a/Lib/smtpd.py
+++ b/Lib/smtpd.py
@@ -846,7 +846,7 @@
     if options.setuid:
         try:
             import pwd
-        except ImportError:
+        except ModuleNotFoundError:
             print('Cannot import module "pwd"; try running with -n option.', file=sys.stderr)
             sys.exit(1)
         nobody = pwd.getpwnam('nobody')[2]
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -171,7 +171,7 @@
 
 try:
     import ssl
-except ImportError:
+except ModuleNotFoundError:
     _have_ssl = False
 else:
     _have_ssl = True
diff --git a/Lib/socket.py b/Lib/socket.py
--- a/Lib/socket.py
+++ b/Lib/socket.py
@@ -51,7 +51,7 @@
 
 try:
     import errno
-except ImportError:
+except ModuleNotFoundError:
     errno = None
 EBADF = getattr(errno, 'EBADF', 9)
 EAGAIN = getattr(errno, 'EAGAIN', 11)
diff --git a/Lib/socketserver.py b/Lib/socketserver.py
--- a/Lib/socketserver.py
+++ b/Lib/socketserver.py
@@ -136,7 +136,7 @@
 import errno
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     import dummy_threading as threading
 
 __all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer",
diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py
--- a/Lib/sqlite3/test/dbapi.py
+++ b/Lib/sqlite3/test/dbapi.py
@@ -25,7 +25,7 @@
 import sqlite3 as sqlite
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     threading = None
 
 from test.support import TESTFN, unlink
diff --git a/Lib/sqlite3/test/types.py b/Lib/sqlite3/test/types.py
--- a/Lib/sqlite3/test/types.py
+++ b/Lib/sqlite3/test/types.py
@@ -26,7 +26,7 @@
 import sqlite3 as sqlite
 try:
     import zlib
-except ImportError:
+except ModuleNotFoundError:
     zlib = None
 
 
diff --git a/Lib/sre_compile.py b/Lib/sre_compile.py
--- a/Lib/sre_compile.py
+++ b/Lib/sre_compile.py
@@ -295,7 +295,7 @@
 def _optimize_unicode(charset, fixup):
     try:
         import array
-    except ImportError:
+    except ModuleNotFoundError:
         return charset
     charmap = [0]*65536
     negate = 0
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -127,14 +127,14 @@
 try:
     from _ssl import PROTOCOL_SSLv2
     _SSLv2_IF_EXISTS = PROTOCOL_SSLv2
-except ImportError:
+except ModuleNotFoundError:
     _SSLv2_IF_EXISTS = None
 else:
     _PROTOCOL_NAMES[PROTOCOL_SSLv2] = "SSLv2"
 
 try:
     from _ssl import PROTOCOL_TLSv1_1, PROTOCOL_TLSv1_2
-except ImportError:
+except ModuleNotFoundError:
     pass
 else:
     _PROTOCOL_NAMES[PROTOCOL_TLSv1_1] = "TLSv1.1"
diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -353,7 +353,7 @@
 import errno
 try:
     from time import monotonic as _time
-except ImportError:
+except ModuleNotFoundError:
     from time import time as _time
 
 # Exception classes used by this module.
diff --git a/Lib/tarfile.py b/Lib/tarfile.py
--- a/Lib/tarfile.py
+++ b/Lib/tarfile.py
@@ -50,7 +50,7 @@
 
 try:
     import grp, pwd
-except ImportError:
+except ModuleNotFoundError:
     grp = pwd = None
 
 # os.symlink on Windows prior to 6.0 raises NotImplementedError
@@ -381,7 +381,7 @@
             if comptype == "gz":
                 try:
                     import zlib
-                except ImportError:
+                except ModuleNotFoundError:
                     raise CompressionError("zlib module is not available")
                 self.zlib = zlib
                 self.crc = zlib.crc32(b"")
@@ -394,7 +394,7 @@
             elif comptype == "bz2":
                 try:
                     import bz2
-                except ImportError:
+                except ModuleNotFoundError:
                     raise CompressionError("bz2 module is not available")
                 if mode == "r":
                     self.dbuf = b""
@@ -406,7 +406,7 @@
             elif comptype == "xz":
                 try:
                     import lzma
-                except ImportError:
+                except ModuleNotFoundError:
                     raise CompressionError("lzma module is not available")
                 if mode == "r":
                     self.dbuf = b""
@@ -1654,7 +1654,7 @@
 
         try:
             import bz2
-        except ImportError:
+        except ModuleNotFoundError:
             raise CompressionError("bz2 module is not available")
 
         fileobj = bz2.BZ2File(fileobj or name, mode,
@@ -1678,7 +1678,7 @@
 
         try:
             import lzma
-        except ImportError:
+        except ModuleNotFoundError:
             raise CompressionError("lzma module is not available")
 
         fileobj = lzma.LZMAFile(fileobj or name, mode, preset=preset)
diff --git a/Lib/tempfile.py b/Lib/tempfile.py
--- a/Lib/tempfile.py
+++ b/Lib/tempfile.py
@@ -36,7 +36,7 @@
 
 try:
     import fcntl as _fcntl
-except ImportError:
+except ModuleNotFoundError:
     def _set_cloexec(fd):
         pass
 else:
@@ -53,7 +53,7 @@
 
 try:
     import _thread
-except ImportError:
+except ModuleNotFoundError:
     import _dummy_thread as _thread
 _allocate_lock = _thread.allocate_lock
 
diff --git a/Lib/threading.py b/Lib/threading.py
--- a/Lib/threading.py
+++ b/Lib/threading.py
@@ -6,14 +6,14 @@
 from time import sleep as _sleep
 try:
     from time import monotonic as _time
-except ImportError:
+except ModuleNotFoundError:
     from time import time as _time
 from traceback import format_exc as _format_exc
 from _weakrefset import WeakSet
 from itertools import islice as _islice
 try:
     from _collections import deque as _deque
-except ImportError:
+except ModuleNotFoundError:
     from collections import deque as _deque
 
 # Note regarding PEP 8 compliant names
@@ -922,7 +922,7 @@
 
 try:
     from _thread import _local as local
-except ImportError:
+except ModuleNotFoundError:
     from _threading_local import local
 
 
diff --git a/Lib/trace.py b/Lib/trace.py
--- a/Lib/trace.py
+++ b/Lib/trace.py
@@ -61,12 +61,12 @@
 from warnings import warn as _warn
 try:
     from time import monotonic as _time
-except ImportError:
+except ModuleNotFoundError:
     from time import time as _time
 
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     _settrace = sys.settrace
 
     def _unsettrace():
diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -110,7 +110,7 @@
 # check for SSL
 try:
     import ssl
-except ImportError:
+except ModuleNotFoundError:
     _have_ssl = False
 else:
     _have_ssl = True
@@ -2512,7 +2512,7 @@
         proxies = {}
         try:
             import winreg
-        except ImportError:
+        except ModuleNotFoundError:
             # Std module, so should be around - but you never know!
             return proxies
         try:
@@ -2560,7 +2560,7 @@
     def proxy_bypass_registry(host):
         try:
             import winreg
-        except ImportError:
+        except ModuleNotFoundError:
             # Std modules, so should be around - but you never know!
             return 0
         try:
diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py
--- a/Lib/venv/__init__.py
+++ b/Lib/venv/__init__.py
@@ -35,7 +35,7 @@
 import sysconfig
 try:
     import threading
-except ImportError:
+except ModuleNotFoundError:
     threading = None
 
 logger = logging.getLogger(__name__)
diff --git a/Lib/warnings.py b/Lib/warnings.py
--- a/Lib/warnings.py
+++ b/Lib/warnings.py
@@ -144,8 +144,8 @@
         module = category[:i]
         klass = category[i+1:]
         try:
-            m = __import__(module, None, None, [klass])
-        except ImportError:
+            m = __import__(module, fromlist[klass])
+        except ModuleNotFoundError:
             raise _OptionError("invalid module name: %r" % (module,))
         try:
             cat = getattr(m, klass)
@@ -362,7 +362,7 @@
     defaultaction = _defaultaction
     onceregistry = _onceregistry
     _warnings_defaults = True
-except ImportError:
+except ModuleNotFoundError:
     filters = []
     defaultaction = "default"
     onceregistry = {}
diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py
--- a/Lib/xml/etree/ElementTree.py
+++ b/Lib/xml/etree/ElementTree.py
@@ -1439,13 +1439,13 @@
     def __init__(self, html=0, target=None, encoding=None):
         try:
             from xml.parsers import expat
-        except ImportError:
+        except ModuleNotFoundError:
             try:
                 import pyexpat as expat
-            except ImportError:
-                raise ImportError(
-                    "No module named expat; use SimpleXMLTreeBuilder instead"
-                    )
+            except ModuleNotFoundError:
+                raise ModuleNotFoundError(
+                    "No module named expat; use SimpleXMLTreeBuilder instead",
+                    name='expat')
         parser = expat.ParserCreate(encoding, "}")
         if target is None:
             target = TreeBuilder()
diff --git a/Lib/xml/sax/expatreader.py b/Lib/xml/sax/expatreader.py
--- a/Lib/xml/sax/expatreader.py
+++ b/Lib/xml/sax/expatreader.py
@@ -20,7 +20,7 @@
 
 try:
     from xml.parsers import expat
-except ImportError:
+except ModuleNotFoundError:
     raise SAXReaderNotAvailable("expat not supported", None)
 else:
     if not hasattr(expat, "ParserCreate"):
@@ -30,18 +30,7 @@
 AttributesImpl = xmlreader.AttributesImpl
 AttributesNSImpl = xmlreader.AttributesNSImpl
 
-# If we're using a sufficiently recent version of Python, we can use
-# weak references to avoid cycles between the parser and content
-# handler, otherwise we'll just have to pretend.
-try:
-    import _weakref
-except ImportError:
-    def _mkproxy(o):
-        return o
-else:
-    import weakref
-    _mkproxy = weakref.proxy
-    del weakref, _weakref
+import weakref
 
 # --- ExpatLocator
 
@@ -52,7 +41,7 @@
     a circular reference between the parser and the content handler.
     """
     def __init__(self, parser):
-        self._ref = _mkproxy(parser)
+        self._ref = weakref.proxy(parser)
 
     def getColumnNumber(self):
         parser = self._ref
diff --git a/Lib/xmlrpc/client.py b/Lib/xmlrpc/client.py
--- a/Lib/xmlrpc/client.py
+++ b/Lib/xmlrpc/client.py
@@ -139,7 +139,7 @@
 from io import BytesIO
 try:
     import gzip
-except ImportError:
+except ModuleNotFoundError:
     gzip = None #python can be built without zlib/gzip support
 
 # --------------------------------------------------------------------

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 03:15:28 2013
From: python-checkins at python.org (richard.jones)
Date: Fri, 14 Jun 2013 03:15:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_clarify_some_wording_based_on?=
 =?utf-8?q?_feedback_from_Marcus_Smith=3B_note_resolution_of?=
Message-ID: <3bWkQS54yFzP9C@mail.python.org>

http://hg.python.org/peps/rev/85b25c4fb26d
changeset:   4923:85b25c4fb26d
user:        Richard Jones <richard at mechanicalcat.net>
date:        Fri Jun 14 11:15:22 2013 +1000
summary:
  clarify some wording based on feedback from Marcus Smith; note resolution of Fedora issue

files:
  pep-0439.txt |  47 ++++++++++++++++++++++-----------------
  1 files changed, 26 insertions(+), 21 deletions(-)


diff --git a/pep-0439.txt b/pep-0439.txt
--- a/pep-0439.txt
+++ b/pep-0439.txt
@@ -63,14 +63,15 @@
 -----------------
 
 The Python installation includes an executable called "pip3" (see PEP 394 for
-naming rationale etc.) that attempts to import pip machinery.  If it can
-then the pip command proceeds as normal.  If it cannot it will bootstrap pip by
-downloading the pip implementation wheel file.  Once installed, the pip command
-proceeds as normal.
+naming rationale etc.) that attempts to import pip machinery.  If it can then
+the pip command proceeds as normal.  If it cannot it will bootstrap pip by
+downloading the pip implementation wheel file.  Once installed, the pip
+command proceeds as normal. Once the bootstrap process is complete the "pip3"
+command is no longer the bootstrap but rather the full pip command.
 
-A boostrap is used in the place of a the full pip code so that we
-don't have to bundle pip and also the install tool is upgradeable
-outside of the regular Python upgrade timeframe and processes.
+A boostrap is used in the place of a the full pip code so that we don't have
+to bundle pip and also pip is upgradeable outside of the regular Python
+upgrade timeframe and processes.
 
 To avoid issues with sudo we will have the bootstrap default to
 installing the pip implementation to the per-user site-packages
@@ -114,13 +115,12 @@
    it) and display further instructions for downloading and installing
    the file manually.
 
-Manual installation of the pip implementation will be supported
-through the manual download of the wheel file and "pip3 install
-<downloaded wheel file>".
-
-This installation will not perform standard pip installation steps of
-saving the file to a cache directory or updating any local database of
-installed files.
+Some users may have no Internet access suitable for fetching the pip
+implementation file. Manual installation of the pip implementation will be
+supported through the manual download of the wheel file and "pip3 install
+<downloaded wheel file>". This installation - since it uses only the bootstrap
+code - will not perform standard pip installation steps of saving the file to
+a cache directory or updating any local database of installed files.
 
 The download of the pip implementation install file should be performed
 securely.  The transport from pypi.python.org will be done over HTTPS but the CA
@@ -195,16 +195,16 @@
 Risks
 =====
 
-The Fedora variant of Linux has had a separate program called "pip" (a
-Perl package installer) available for install for some time.  The
-current Python "pip" program is installed as "pip-python".  It is
-hoped that the Fedora community will resolve this issue by renaming
-the Perl installer.
-
 The key that is used to sign the pip implementation download might be
 compromised and this PEP currently proposes no mechanism for key
 revocation.
 
+There is a Perl package installer also named "pip". It is quite rare and not
+commonly used. The Fedora variant of Linux has historically named Python's
+"pip" as "python-pip" and Perl's "pip" as "perl-pip".  This policy has been
+altered[3] so that future Fedora installations will use the name "pip" for
+Python's "pip". Existing installations will still have the old name for the
+Python "pip", though the potential for confusion is now much reduced.
 
 
 References
@@ -216,6 +216,9 @@
 .. [2] pip issue tracking work needed for this PEP
        https://github.com/pypa/pip/issues/863
 
+.. [3] Fedora's python-pip package does not provide /usr/bin/pip
+       https://bugzilla.redhat.com/show_bug.cgi?id=958377
+
 
 Acknowledgments
 ===============
@@ -223,7 +226,9 @@
 Nick Coghlan for his thoughts on the proposal and dealing with the Red
 Hat issue.
 
-Jannis Leidel and Carl Meyer for their thoughts.
+Jannis Leidel and Carl Meyer for their thoughts. Marcus Smith for feedback.
+
+Marcela Ma?l??ov? for resolving the Fedora issue.
 
 
 Copyright

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Fri Jun 14 03:18:51 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri, 14 Jun 2013 03:18:51 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_test=5Fpep352_over_to?=
	=?utf-8?q?_unittest=2Emain=28=29?=
Message-ID: <3bWkVM0wM9zP5m@mail.python.org>

http://hg.python.org/cpython/rev/af27c661d4fb
changeset:   84115:af27c661d4fb
user:        Brett Cannon <brett at python.org>
date:        Thu Jun 13 21:18:43 2013 -0400
summary:
  Move test_pep352 over to unittest.main()

files:
  Lib/test/test_pep352.py |  4 +---
  1 files changed, 1 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_pep352.py b/Lib/test/test_pep352.py
--- a/Lib/test/test_pep352.py
+++ b/Lib/test/test_pep352.py
@@ -180,8 +180,6 @@
         # Catching a string is bad.
         self.catch_fails("spam")
 
-def test_main():
-    run_unittest(ExceptionClassTests, UsageTests)
 
 if __name__ == '__main__':
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 03:36:26 2013
From: python-checkins at python.org (richard.jones)
Date: Fri, 14 Jun 2013 03:36:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_slight_clarification?=
Message-ID: <3bWktf50KRzMfc@mail.python.org>

http://hg.python.org/peps/rev/0a051a14592b
changeset:   4924:0a051a14592b
user:        Richard Jones <richard at mechanicalcat.net>
date:        Fri Jun 14 11:36:16 2013 +1000
summary:
  slight clarification

files:
  pep-0439.txt |  7 ++++---
  1 files changed, 4 insertions(+), 3 deletions(-)


diff --git a/pep-0439.txt b/pep-0439.txt
--- a/pep-0439.txt
+++ b/pep-0439.txt
@@ -202,9 +202,10 @@
 There is a Perl package installer also named "pip". It is quite rare and not
 commonly used. The Fedora variant of Linux has historically named Python's
 "pip" as "python-pip" and Perl's "pip" as "perl-pip".  This policy has been
-altered[3] so that future Fedora installations will use the name "pip" for
-Python's "pip". Existing installations will still have the old name for the
-Python "pip", though the potential for confusion is now much reduced.
+altered[3] so that future and upgraded Fedora installations will use the name
+"pip" for Python's "pip". Existing (non-upgraded) installations will still
+have the old name for the Python "pip", though the potential for confusion is
+now much reduced.
 
 
 References

-- 
Repository URL: http://hg.python.org/peps

From solipsis at pitrou.net  Fri Jun 14 05:49:05 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 14 Jun 2013 05:49:05 +0200
Subject: [Python-checkins] Daily reference leaks (af27c661d4fb): sum=0
Message-ID: <E1UnL0b-0005W4-3g@ap.vmr.nerim.net>

results for af27c661d4fb on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogMnOdYG', '-x']

From python-checkins at python.org  Fri Jun 14 07:06:49 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Fri, 14 Jun 2013 07:06:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_comment_bl?=
 =?utf-8?q?ocks=2E__Adjust_blocksize_to_a_power-of-two_for_better_divmod?=
Message-ID: <3bWqYP3sqgzQdN@mail.python.org>

http://hg.python.org/cpython/rev/5accb0ac8bfb
changeset:   84116:5accb0ac8bfb
branch:      2.7
parent:      84095:ca8e86711403
user:        Raymond Hettinger <python at rcn.com>
date:        Fri Jun 14 01:06:33 2013 -0400
summary:
  Fix comment blocks.  Adjust blocksize to a power-of-two for better divmod computations.

files:
  Lib/test/test_deque.py       |   2 +-
  Modules/_collectionsmodule.c |  18 +++++++-----------
  2 files changed, 8 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py
--- a/Lib/test/test_deque.py
+++ b/Lib/test/test_deque.py
@@ -522,7 +522,7 @@
 
     @test_support.cpython_only
     def test_sizeof(self):
-        BLOCKLEN = 62
+        BLOCKLEN = 64
         basesize = test_support.calcobjsize('2P4PlP')
         blocksize = struct.calcsize('2P%dP' % BLOCKLEN)
         self.assertEqual(object.__sizeof__(deque()), basesize)
diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -8,12 +8,13 @@
 */
 
 /* The block length may be set to any number over 1.  Larger numbers
- * reduce the number of calls to the memory allocator but take more
- * memory.  Ideally, BLOCKLEN should be set with an eye to the
- * length of a cache line.
+ * reduce the number of calls to the memory allocator, give faster
+ * indexing and rotation, and reduce the link::data overhead ratio.
+ * Ideally, the block length should be a power-of-two for faster
+ * division/modulo computations during indexing.
  */
 
-#define BLOCKLEN 62
+#define BLOCKLEN 64
 #define CENTER ((BLOCKLEN - 1) / 2)
 
 /* A `dequeobject` is composed of a doubly-linked list of `block` nodes.
@@ -58,13 +59,8 @@
 static block *
 newblock(block *leftlink, block *rightlink, Py_ssize_t len) {
     block *b;
-    /* To prevent len from overflowing PY_SSIZE_T_MAX on 64-bit machines, we
-     * refuse to allocate new blocks if the current len is dangerously
-     * close.  There is some extra margin to prevent spurious arithmetic
-     * overflows at various places.  The following check ensures that
-     * the blocks allocated to the deque, in the worst case, can only
-     * have PY_SSIZE_T_MAX-2 entries in total.
-     */
+    /* To prevent len from overflowing PY_SSIZE_T_MAX on 32-bit machines, we
+     * refuse to allocate new blocks if the current len is nearing overflow. */
     if (len >= PY_SSIZE_T_MAX - 2*BLOCKLEN) {
         PyErr_SetString(PyExc_OverflowError,
                         "cannot add more blocks to the deque");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 09:31:14 2013
From: python-checkins at python.org (ethan.furman)
Date: Fri, 14 Jun 2013 09:31:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_issue_17947=2E__Add?=
 =?utf-8?q?s_PEP-0435_=28Enum=2C_IntEnum=29_to_the_stdlib=2E?=
Message-ID: <3bWtm22z9dz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/fae92309c3be
changeset:   84117:fae92309c3be
parent:      84115:af27c661d4fb
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Fri Jun 14 00:30:27 2013 -0700
summary:
  Closes issue 17947.  Adds PEP-0435 (Enum, IntEnum) to the stdlib.

files:
  Doc/library/datatypes.rst |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Doc/library/datatypes.rst b/Doc/library/datatypes.rst
--- a/Doc/library/datatypes.rst
+++ b/Doc/library/datatypes.rst
@@ -30,3 +30,4 @@
    copy.rst
    pprint.rst
    reprlib.rst
+   enum.rst

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 15:01:33 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 14 Jun 2013 15:01:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_comparing_with_http=3A//hg?=
	=?utf-8?q?=2Epython=2Eorg/cpython/?=
Message-ID: <3bX2590lktz7LkH@mail.python.org>

http://hg.python.org/cpython/rev/4751f1b54540
changeset:   84118:4751f1b54540
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 14 15:01:03 2013 +0200
summary:
  comparing with http://hg.python.org/cpython/
searching for changes
changeset:   84118:98343392fd81
tag:         tip
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 14 14:54:48 2013 +0200
files:       PC/VS9.0/_socket.vcproj PC/VS9.0/_ssl.vcproj
description:
Fix compilation of Python with VS 2008
Contrary to VS 2010 the compiler doesn't like semicolon seperated
dependency names

files:
  PC/VS9.0/_socket.vcproj |  16 ++++++++--------
  PC/VS9.0/_ssl.vcproj    |  18 +++++++++---------
  2 files changed, 17 insertions(+), 17 deletions(-)


diff --git a/PC/VS9.0/_socket.vcproj b/PC/VS9.0/_socket.vcproj
--- a/PC/VS9.0/_socket.vcproj
+++ b/PC/VS9.0/_socket.vcproj
@@ -54,7 +54,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -115,7 +115,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -176,7 +176,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -238,7 +238,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -299,7 +299,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -361,7 +361,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 				TargetMachine="17"
 			/>
@@ -423,7 +423,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 			/>
 			<Tool
@@ -485,7 +485,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib;crypt32.lib"
+				AdditionalDependencies="ws2_32.lib"
 				BaseAddress="0x1e1D0000"
 				TargetMachine="17"
 			/>
diff --git a/PC/VS9.0/_ssl.vcproj b/PC/VS9.0/_ssl.vcproj
--- a/PC/VS9.0/_ssl.vcproj
+++ b/PC/VS9.0/_ssl.vcproj
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="Windows-1252"?>
 <VisualStudioProject
 	ProjectType="Visual C++"
-	Version="9.00"
+	Version="9,00"
 	Name="_ssl"
 	ProjectGUID="{C6E20F84-3247-4AD6-B051-B073268F73BA}"
 	RootNamespace="_ssl"
@@ -56,7 +56,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
 			/>
 			<Tool
 				Name="VCALinkTool"
@@ -118,7 +118,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
 			/>
 			<Tool
 				Name="VCALinkTool"
@@ -180,7 +180,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
 			/>
 			<Tool
 				Name="VCALinkTool"
@@ -243,7 +243,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
 			/>
 			<Tool
 				Name="VCALinkTool"
@@ -305,7 +305,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
 			/>
 			<Tool
 				Name="VCALinkTool"
@@ -368,7 +368,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
 				TargetMachine="17"
 			/>
 			<Tool
@@ -431,7 +431,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out32\libeay32.lib $(opensslDir)\out32\ssleay32.lib"
 			/>
 			<Tool
 				Name="VCALinkTool"
@@ -494,7 +494,7 @@
 			/>
 			<Tool
 				Name="VCLinkerTool"
-				AdditionalDependencies="ws2_32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
+				AdditionalDependencies="ws2_32.lib crypt32.lib $(opensslDir)\out64\libeay32.lib $(opensslDir)\out64\ssleay32.lib"
 				TargetMachine="17"
 			/>
 			<Tool

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 15:14:38 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 14 Jun 2013 15:14:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Simplify_return_value_of_s?=
 =?utf-8?q?sl=2Eget=5Fdefault=5Fverify=5Fpaths?=
Message-ID: <3bX2NG0d4kz7LjW@mail.python.org>

http://hg.python.org/cpython/rev/5f7e12eb65b8
changeset:   84119:5f7e12eb65b8
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 14 15:14:29 2013 +0200
summary:
  Simplify return value of ssl.get_default_verify_paths
prefix function with PySSL_, too. Other module level functions have a prefix, too.

files:
  Modules/_ssl.c |  14 +++-----------
  1 files changed, 3 insertions(+), 11 deletions(-)


diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -2769,13 +2769,12 @@
 'cert_file_env', 'cert_file', 'cert_dir_env', 'cert_dir'.");
 
 static PyObject *
-get_default_verify_paths(PyObject *self)
+PySSL_get_default_verify_paths(PyObject *self)
 {
     PyObject *ofile_env = NULL;
     PyObject *ofile = NULL;
     PyObject *odir_env = NULL;
     PyObject *odir = NULL;
-    PyObject *tup = NULL;
 
 #define convert(info, target) { \
         const char *tmp = (info); \
@@ -2792,14 +2791,7 @@
     convert(X509_get_default_cert_dir(), odir);
 #undef convert
 
-    if ((tup = PyTuple_New(4)) == NULL) {
-        goto error;
-    }
-    PyTuple_SET_ITEM(tup, 0, ofile_env);
-    PyTuple_SET_ITEM(tup, 1, ofile);
-    PyTuple_SET_ITEM(tup, 2, odir_env);
-    PyTuple_SET_ITEM(tup, 3, odir);
-    return tup;
+    return Py_BuildValue("NNNN", ofile_env, ofile, odir_env, odir);
 
   error:
     Py_XDECREF(ofile_env);
@@ -2950,7 +2942,7 @@
     {"RAND_status",         (PyCFunction)PySSL_RAND_status, METH_NOARGS,
      PySSL_RAND_status_doc},
 #endif
-    {"get_default_verify_paths", (PyCFunction)get_default_verify_paths,
+    {"get_default_verify_paths", (PyCFunction)PySSL_get_default_verify_paths,
      METH_NOARGS, PySSL_get_default_verify_paths_doc},
 #ifdef _MSC_VER
     {"enum_cert_store", (PyCFunction)PySSL_enum_cert_store,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 15:48:33 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 14 Jun 2013 15:48:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE1MTcy?=
 =?utf-8?q?=3A_Document_NASM_2=2E10+_as_requirement_for_building_OpenSSL_1?=
 =?utf-8?q?=2E0=2E1_on?=
Message-ID: <3bX37P5S0dz7Lk9@mail.python.org>

http://hg.python.org/cpython/rev/ee3952965934
changeset:   84120:ee3952965934
branch:      3.3
parent:      84100:b11507395ce4
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 14 15:40:28 2013 +0200
summary:
  Issue #15172: Document NASM 2.10+ as requirement for building OpenSSL 1.0.1 on Windows

files:
  Misc/NEWS          |  5 +++++
  PCbuild/readme.txt |  7 +++++--
  2 files changed, 10 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -126,6 +126,11 @@
 
 - Issue #15239: Make mkstringprep.py work again on Python 3.
 
+Build
+-----
+
+- Issue #15172: Document NASM 2.10+ as requirement for building OpenSSL 1.0.1
+  on Windows.
 
 What's New in Python 3.3.2?
 ===========================
diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt
--- a/PCbuild/readme.txt
+++ b/PCbuild/readme.txt
@@ -152,9 +152,12 @@
     You can (theoretically) use any version of OpenSSL you like - the
     build process will automatically select the latest version.
 
-    You must install the NASM assembler from
+    You must install the NASM assembler 2.10 or newer from
         http://nasm.sf.net
-    for x86 builds.  Put nasmw.exe anywhere in your PATH.
+    for x86 builds.  Put nasmw.exe anywhere in your PATH. More recent
+    versions of OpenSSL may need a later version of NASM. If OpenSSL's self
+    tests don't pass, you should first try to update NASM and do a full
+    rebuild of OpenSSL.
     Note: recent releases of nasm only have nasm.exe. Just rename it to 
     nasmw.exe.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 15:48:35 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 14 Jun 2013 15:48:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2315172=3A_Document_NASM_2=2E10+_as_requirement_f?=
 =?utf-8?q?or_building_OpenSSL_1=2E0=2E1_on?=
Message-ID: <3bX37R0gTQz7LkK@mail.python.org>

http://hg.python.org/cpython/rev/1b2cbdc9c1d4
changeset:   84121:1b2cbdc9c1d4
parent:      84119:5f7e12eb65b8
parent:      84120:ee3952965934
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 14 15:48:16 2013 +0200
summary:
  Issue #15172: Document NASM 2.10+ as requirement for building OpenSSL 1.0.1 on Windows

files:
  Misc/NEWS          |  2 ++
  PCbuild/readme.txt |  7 +++++--
  2 files changed, 7 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -493,6 +493,8 @@
 - Issue #17547: In configure, explicitly pass -Wformat for the benefit for GCC
   4.8.
 
+- Issue #15172: Document NASM 2.10+ as requirement for building OpenSSL 1.0.1
+  on Windows.
 
 What's New in Python 3.3.1 release candidate 1?
 ===============================================
diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt
--- a/PCbuild/readme.txt
+++ b/PCbuild/readme.txt
@@ -152,9 +152,12 @@
     You can (theoretically) use any version of OpenSSL you like - the
     build process will automatically select the latest version.
 
-    You must install the NASM assembler from
+    You must install the NASM assembler 2.10 or newer from
         http://nasm.sf.net
-    for x86 builds.  Put nasmw.exe anywhere in your PATH.
+    for x86 builds.  Put nasmw.exe anywhere in your PATH. More recent
+    versions of OpenSSL may need a later version of NASM. If OpenSSL's self
+    tests don't pass, you should first try to update NASM and do a full
+    rebuild of OpenSSL.
     Note: recent releases of nasm only have nasm.exe. Just rename it to 
     nasmw.exe.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 16:42:55 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri, 14 Jun 2013 16:42:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_a_dead_import_line?=
	=?utf-8?q?=2E?=
Message-ID: <3bX4L75Lz0z7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/e61d7da84680
changeset:   84122:e61d7da84680
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 10:42:48 2013 -0400
summary:
  Remove a dead import line.

Noticed by Serhly Storchaka.

files:
  Lib/test/test_pep352.py |  1 -
  1 files changed, 0 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_pep352.py b/Lib/test/test_pep352.py
--- a/Lib/test/test_pep352.py
+++ b/Lib/test/test_pep352.py
@@ -1,7 +1,6 @@
 import unittest
 import builtins
 import warnings
-from test.support import run_unittest
 import os
 from platform import system as platform_system
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 14 21:04:37 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri, 14 Jun 2013 21:04:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318193=3A_Add_impo?=
 =?utf-8?q?rtlib=2Ereload=28=29=2C_documenting_=28but_not?=
Message-ID: <3bXB8500BTz7Lkr@mail.python.org>

http://hg.python.org/cpython/rev/01da7bf11ca1
changeset:   84123:01da7bf11ca1
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 15:04:26 2013 -0400
summary:
  Issue #18193: Add importlib.reload(), documenting (but not
implementing in code) the deprecation of imp.reload().

Thanks to Berker Peksag for the patch.

files:
  Doc/library/imp.rst                 |   3 +
  Doc/library/importlib.rst           |  67 +++++++++++++++++
  Lib/imp.py                          |  28 +-----
  Lib/importlib/__init__.py           |  34 ++++++++-
  Lib/test/test_importlib/test_api.py |  12 +++
  Misc/NEWS                           |   2 +
  6 files changed, 122 insertions(+), 24 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -171,6 +171,9 @@
    the class does not affect the method definitions of the instances --- they
    continue to use the old class definition.  The same is true for derived classes.
 
+   .. deprecated:: 3.4
+      Use :func:`importlib.reload` instead.
+
 
 The following functions are conveniences for handling :pep:`3147` byte-compiled
 file paths.
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -115,6 +115,73 @@
 
    .. versionadded:: 3.3
 
+.. function:: reload(module)
+
+   Reload a previously imported *module*.  The argument must be a module object,
+   so it must have been successfully imported before.  This is useful if you
+   have edited the module source file using an external editor and want to try
+   out the new version without leaving the Python interpreter.  The return value
+   is the module object (the same as the *module* argument).
+
+   When :func:`.reload` is executed:
+
+   * Python modules' code is recompiled and the module-level code re-executed,
+     defining a new set of objects which are bound to names in the module's
+     dictionary by reusing the :term:`loader` which originally loaded the
+     module.  The ``init`` function of extension modules is not called a second
+     time.
+
+   * As with all other objects in Python the old objects are only reclaimed
+     after their reference counts drop to zero.
+
+   * The names in the module namespace are updated to point to any new or
+     changed objects.
+
+   * Other references to the old objects (such as names external to the module) are
+     not rebound to refer to the new objects and must be updated in each namespace
+     where they occur if that is desired.
+
+   There are a number of other caveats:
+
+   If a module is syntactically correct but its initialization fails, the first
+   :keyword:`import` statement for it does not bind its name locally, but does
+   store a (partially initialized) module object in ``sys.modules``.  To reload
+   the module you must first :keyword:`import` it again (this will bind the name
+   to the partially initialized module object) before you can :func:`reload` it.
+
+   When a module is reloaded, its dictionary (containing the module's global
+   variables) is retained.  Redefinitions of names will override the old
+   definitions, so this is generally not a problem.  If the new version of a
+   module does not define a name that was defined by the old version, the old
+   definition remains.  This feature can be used to the module's advantage if it
+   maintains a global table or cache of objects --- with a :keyword:`try`
+   statement it can test for the table's presence and skip its initialization if
+   desired::
+
+      try:
+          cache
+      except NameError:
+          cache = {}
+
+   It is legal though generally not very useful to reload built-in or
+   dynamically loaded modules (this is not true for e.g. :mod:`sys`,
+   :mod:`__main__`, :mod:`__builtin__` and other key modules where reloading is
+   frowned upon). In many cases, however, extension modules are not designed to
+   be initialized more than once, and may fail in arbitrary ways when reloaded.
+
+   If a module imports objects from another module using :keyword:`from` ...
+   :keyword:`import` ..., calling :func:`reload` for the other module does not
+   redefine the objects imported from it --- one way around this is to
+   re-execute the :keyword:`from` statement, another is to use :keyword:`import`
+   and qualified names (*module.name*) instead.
+
+   If a module instantiates instances of a class, reloading the module that
+   defines the class does not affect the method definitions of the instances ---
+   they continue to use the old class definition.  The same is true for derived
+   classes.
+
+   .. versionadded:: 3.4
+
 
 :mod:`importlib.abc` -- Abstract base classes related to import
 ---------------------------------------------------------------
diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -23,6 +23,7 @@
 
 from importlib import _bootstrap
 from importlib import machinery
+import importlib
 import os
 import sys
 import tokenize
@@ -246,31 +247,12 @@
     return file, file_path, (suffix, mode, type_)
 
 
-_RELOADING = {}
+def reload(module):
+    """**DEPRECATED**
 
-def reload(module):
-    """Reload the module and return it.
+    Reload the module and return it.
 
     The module must have been successfully imported before.
 
     """
-    if not module or type(module) != type(sys):
-        raise TypeError("reload() argument must be module")
-    name = module.__name__
-    if name not in sys.modules:
-        msg = "module {} not in sys.modules"
-        raise ImportError(msg.format(name), name=name)
-    if name in _RELOADING:
-        return _RELOADING[name]
-    _RELOADING[name] = module
-    try:
-        parent_name = name.rpartition('.')[0]
-        if parent_name and parent_name not in sys.modules:
-            msg = "parent {!r} not in sys.modules"
-            raise ImportError(msg.format(parentname), name=parent_name)
-        return module.__loader__.load_module(name)
-    finally:
-        try:
-            del _RELOADING[name]
-        except KeyError:
-            pass
+    return importlib.reload(module)
diff --git a/Lib/importlib/__init__.py b/Lib/importlib/__init__.py
--- a/Lib/importlib/__init__.py
+++ b/Lib/importlib/__init__.py
@@ -1,5 +1,5 @@
 """A pure Python implementation of import."""
-__all__ = ['__import__', 'import_module', 'invalidate_caches']
+__all__ = ['__import__', 'import_module', 'invalidate_caches', 'reload']
 
 # Bootstrap help #####################################################
 
@@ -11,6 +11,7 @@
 # initialised below if the frozen one is not available).
 import _imp  # Just the builtin component, NOT the full Python module
 import sys
+import types
 
 try:
     import _frozen_importlib as _bootstrap
@@ -90,3 +91,34 @@
                 break
             level += 1
     return _bootstrap._gcd_import(name[level:], package, level)
+
+
+_RELOADING = {}
+
+
+def reload(module):
+    """Reload the module and return it.
+
+    The module must have been successfully imported before.
+
+    """
+    if not module or not isinstance(module, types.ModuleType):
+        raise TypeError("reload() argument must be module")
+    name = module.__name__
+    if name not in sys.modules:
+        msg = "module {} not in sys.modules"
+        raise ImportError(msg.format(name), name=name)
+    if name in _RELOADING:
+        return _RELOADING[name]
+    _RELOADING[name] = module
+    try:
+        parent_name = name.rpartition('.')[0]
+        if parent_name and parent_name not in sys.modules:
+            msg = "parent {!r} not in sys.modules"
+            raise ImportError(msg.format(parentname), name=parent_name)
+        return module.__loader__.load_module(name)
+    finally:
+        try:
+            del _RELOADING[name]
+        except KeyError:
+            pass
diff --git a/Lib/test/test_importlib/test_api.py b/Lib/test/test_importlib/test_api.py
--- a/Lib/test/test_importlib/test_api.py
+++ b/Lib/test/test_importlib/test_api.py
@@ -151,6 +151,18 @@
         self.assertIsNone(importlib.find_loader('nevergoingtofindthismodule'))
 
 
+class ReloadTests(unittest.TestCase):
+
+    """Test module reloading for builtin and extension modules."""
+
+    def test_reload_modules(self):
+        for mod in ('tokenize', 'time', 'marshal'):
+            with self.subTest(module=mod):
+                with support.CleanImport(mod):
+                    module = importlib.import_module(mod)
+                    importlib.reload(module)
+
+
 class InvalidateCacheTests(unittest.TestCase):
 
     def test_method_called(self):
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #18193: Add importlib.reload().
+
 - Issue #18157: Stop using imp.load_module() in pydoc.
 
 - Issue #16102: Make uuid._netbios_getnode() work again on Python 3.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 00:20:24 2013
From: python-checkins at python.org (ned.deily)
Date: Sat, 15 Jun 2013 00:20:24 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318149=3A_Add_file?=
 =?utf-8?q?cmp=2Eclear=5Fcache=28=29_to_manually_clear_the_filecmp_cache?=
 =?utf-8?q?=2E?=
Message-ID: <3bXGV01trMz7Ljp@mail.python.org>

http://hg.python.org/cpython/rev/bfd53dcb02ff
changeset:   84124:bfd53dcb02ff
user:        Ned Deily <nad at acm.org>
date:        Fri Jun 14 15:19:11 2013 -0700
summary:
  Issue #18149: Add filecmp.clear_cache() to manually clear the filecmp cache.
Patch by Mark Levitt

files:
  Doc/library/filecmp.rst  |  13 +++++++++++++
  Lib/filecmp.py           |  11 ++++++++---
  Lib/test/test_filecmp.py |   7 +++++++
  Misc/ACKS                |   1 +
  Misc/NEWS                |   3 +++
  5 files changed, 32 insertions(+), 3 deletions(-)


diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst
--- a/Doc/library/filecmp.rst
+++ b/Doc/library/filecmp.rst
@@ -27,6 +27,10 @@
    Note that no external programs are called from this function, giving it
    portability and efficiency.
 
+   This function uses a cache for past comparisons and the results,
+   with a cache invalidation mechanism relying on stale signatures
+   or by explicitly calling :func:`clear_cache`.
+
 
 .. function:: cmpfiles(dir1, dir2, common, shallow=True)
 
@@ -48,6 +52,15 @@
    one of the three returned lists.
 
 
+.. function:: clear_cache()
+
+   .. versionadded:: 3.4
+
+   Clear the filecmp cache. This may be useful if a file is compared so quickly
+   after it is modified that it is within the mtime resolution of
+   the underlying filesystem.
+
+
 .. _dircmp-objects:
 
 The :class:`dircmp` class
diff --git a/Lib/filecmp.py b/Lib/filecmp.py
--- a/Lib/filecmp.py
+++ b/Lib/filecmp.py
@@ -6,6 +6,7 @@
 Functions:
     cmp(f1, f2, shallow=True) -> int
     cmpfiles(a, b, common) -> ([], [], [])
+    clear_cache()
 
 """
 
@@ -13,7 +14,7 @@
 import stat
 from itertools import filterfalse
 
-__all__ = ['cmp', 'dircmp', 'cmpfiles', 'DEFAULT_IGNORES']
+__all__ = ['clear_cache', 'cmp', 'dircmp', 'cmpfiles', 'DEFAULT_IGNORES']
 
 _cache = {}
 BUFSIZE = 8*1024
@@ -21,6 +22,9 @@
 DEFAULT_IGNORES = [
     'RCS', 'CVS', 'tags', '.git', '.hg', '.bzr', '_darcs', '__pycache__']
 
+def clear_cache():
+    """Clear the filecmp cache."""
+    _cache.clear()
 
 def cmp(f1, f2, shallow=True):
     """Compare two files.
@@ -39,7 +43,8 @@
     True if the files are the same, False otherwise.
 
     This function uses a cache for past comparisons and the results,
-    with a cache invalidation mechanism relying on stale signatures.
+    with a cache invalidation mechanism relying on stale signatures
+    or by explicitly calling clear_cache().
 
     """
 
@@ -56,7 +61,7 @@
     if outcome is None:
         outcome = _do_cmp(f1, f2)
         if len(_cache) > 100:      # limit the maximum size of the cache
-            _cache.clear()
+            clear_cache()
         _cache[f1, f2, s1, s2] = outcome
     return outcome
 
diff --git a/Lib/test/test_filecmp.py b/Lib/test/test_filecmp.py
--- a/Lib/test/test_filecmp.py
+++ b/Lib/test/test_filecmp.py
@@ -39,6 +39,13 @@
         self.assertFalse(filecmp.cmp(self.name, self.dir),
                     "File and directory compare as equal")
 
+    def test_cache_clear(self):
+        first_compare = filecmp.cmp(self.name, self.name_same, shallow=False)
+        second_compare = filecmp.cmp(self.name, self.name_diff, shallow=False)
+        filecmp.clear_cache()
+        self.assertTrue(len(filecmp._cache) == 0,
+                        "Cache not cleared after calling clear_cache")
+
 class DirCompareTestCase(unittest.TestCase):
     def setUp(self):
         tmpdir = tempfile.gettempdir()
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -738,6 +738,7 @@
 Christopher Tur Lesniewski-Laas
 Alain Leufroy
 Mark Levinson
+Mark Levitt
 William Lewis
 Akira Li
 Xuanji Li
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- Issue #18149: Add filecmp.clear_cache() to manually clear the filecmp cache.
+  Patch by Mark Levitt
+
 - Issue #18193: Add importlib.reload().
 
 - Issue #18157: Stop using imp.load_module() in pydoc.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 00:33:30 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 00:33:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317222=3A_Raise_Fi?=
 =?utf-8?q?leExistsError_when_py=5Fcompile=2Ecompile_would?=
Message-ID: <3bXGn61cGRz7LkN@mail.python.org>

http://hg.python.org/cpython/rev/46ef1d2af352
changeset:   84125:46ef1d2af352
parent:      84123:01da7bf11ca1
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 18:33:00 2013 -0400
summary:
  Issue #17222: Raise FileExistsError when py_compile.compile would
overwrite a symlink or non-regular file with a regular file.

files:
  Doc/library/py_compile.rst  |  11 ++++++++++-
  Doc/whatsnew/3.4.rst        |   5 +++++
  Lib/py_compile.py           |  14 ++++++++++++++
  Lib/test/test_py_compile.py |  20 ++++++++++++++++++++
  Misc/NEWS                   |   3 +++
  5 files changed, 52 insertions(+), 1 deletions(-)


diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst
--- a/Doc/library/py_compile.rst
+++ b/Doc/library/py_compile.rst
@@ -41,6 +41,13 @@
    is raised.  This function returns the path to byte-compiled file, i.e.
    whatever *cfile* value was used.
 
+   If the path that *cfile* becomes (either explicitly specified or computed)
+   is a symlink or non-regular file, :exc:`FileExistsError` will be raised.
+   This is to act as a warning that import will turn those paths into regular
+   files if it is allowed to write byte-compiled files to those paths. This is
+   a side-effect of import using file renaming to place the final byte-compiled
+   file into place to prevent concurrent file writing issues.
+
    *optimize* controls the optimization level and is passed to the built-in
    :func:`compile` function.  The default of ``-1`` selects the optimization
    level of the current interpreter.
@@ -53,7 +60,9 @@
    .. versionchanged:: 3.4
       Changed code to use :mod:`importlib` for the byte-code cache file writing.
       This means file creation/writing semantics now match what :mod:`importlib`
-      does, e.g. permissions, write-and-move semantics, etc.
+      does, e.g. permissions, write-and-move semantics, etc. Also added the
+      caveat that :exc:`FileExistsError` is raised if *cfile* is a symlink or
+      non-regular file.
 
 
 .. function:: main(args=None)
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -272,3 +272,8 @@
 
 * :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
   argument is not set. Previously only ``NULL`` was returned.
+
+* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path
+  it would write to is a symlink or a non-regular file. This is to act as a
+  warning that import will overwrite those files with a regular file regardless
+  of what type of file path they were originally.
diff --git a/Lib/py_compile.py b/Lib/py_compile.py
--- a/Lib/py_compile.py
+++ b/Lib/py_compile.py
@@ -7,6 +7,7 @@
 import importlib._bootstrap
 import importlib.machinery
 import os
+import os.path
 import sys
 import traceback
 
@@ -96,12 +97,25 @@
     See compileall.py for a script/module that uses this module to
     byte-compile all installed files (or all files in selected
     directories).
+
+    Do note that FileExistsError is raised if cfile ends up pointing at a
+    non-regular file or symlink. Because the compilation uses a file renaming,
+    the resulting file would be regular and thus not the same type of file as
+    it was previously.
     """
     if cfile is None:
         if optimize >= 0:
             cfile = imp.cache_from_source(file, debug_override=not optimize)
         else:
             cfile = imp.cache_from_source(file)
+    if os.path.islink(cfile):
+        msg = ('{} is a symlink and will be changed into a regular file if '
+               'import writes a byte-compiled file to it')
+        raise FileExistsError(msg.format(file, cfile))
+    elif os.path.exists(cfile) and not os.path.isfile(cfile):
+        msg = ('{} is a non-regular file and will be changed into a regular '
+               'one if import writes a byte-compiled file to it')
+        raise FileExistsError(msg.format(file, cfile))
     loader = importlib.machinery.SourceFileLoader('<py_compile>', file)
     source_bytes = loader.get_data(file)
     try:
diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py
--- a/Lib/test/test_py_compile.py
+++ b/Lib/test/test_py_compile.py
@@ -36,6 +36,26 @@
         self.assertTrue(os.path.exists(self.pyc_path))
         self.assertFalse(os.path.exists(self.cache_path))
 
+    def test_do_not_overwrite_symlinks(self):
+        # In the face of a cfile argument being a symlink, bail out.
+        # Issue #17222
+        try:
+            os.symlink(self.pyc_path + '.actual', self.pyc_path)
+        except OSError:
+            self.skipTest('need to be able to create a symlink for a file')
+        else:
+            assert os.path.islink(self.pyc_path)
+            with self.assertRaises(FileExistsError):
+                py_compile.compile(self.source_path, self.pyc_path)
+
+    @unittest.skipIf(not os.path.exists(os.devnull) or os.path.isfile(os.devnull),
+                     'requires os.devnull and for it to be a non-regular file')
+    def test_do_not_overwrite_nonregular_files(self):
+        # In the face of a cfile argument being a non-regular file, bail out.
+        # Issue #17222
+        with self.assertRaises(FileExistsError):
+            py_compile.compile(self.source_path, os.devnull)
+
     def test_cache_path(self):
         py_compile.compile(self.source_path)
         self.assertTrue(os.path.exists(self.cache_path))
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1005,6 +1005,9 @@
   Python file.  Patch by Ben Morgan.
 
 - Have py_compile use importlib as much as possible to avoid code duplication.
+  Code now raises FileExistsError if the file path to be used for the
+  byte-compiled file is a symlink or non-regular file as a warning that import
+  will not keep the file path type if it writes to that path.
 
 - Issue #180022: Have site.addpackage() consider already known paths even when
   none are explicitly passed in. Bug report and fix by Kirill.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 00:33:31 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 00:33:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bXGn74Vnzz7Lk4@mail.python.org>

http://hg.python.org/cpython/rev/8ee9f50b0538
changeset:   84126:8ee9f50b0538
parent:      84125:46ef1d2af352
parent:      84124:bfd53dcb02ff
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 18:33:21 2013 -0400
summary:
  merge

files:
  Doc/library/filecmp.rst  |  13 +++++++++++++
  Lib/filecmp.py           |  11 ++++++++---
  Lib/test/test_filecmp.py |   7 +++++++
  Misc/ACKS                |   1 +
  Misc/NEWS                |   3 +++
  5 files changed, 32 insertions(+), 3 deletions(-)


diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst
--- a/Doc/library/filecmp.rst
+++ b/Doc/library/filecmp.rst
@@ -27,6 +27,10 @@
    Note that no external programs are called from this function, giving it
    portability and efficiency.
 
+   This function uses a cache for past comparisons and the results,
+   with a cache invalidation mechanism relying on stale signatures
+   or by explicitly calling :func:`clear_cache`.
+
 
 .. function:: cmpfiles(dir1, dir2, common, shallow=True)
 
@@ -48,6 +52,15 @@
    one of the three returned lists.
 
 
+.. function:: clear_cache()
+
+   .. versionadded:: 3.4
+
+   Clear the filecmp cache. This may be useful if a file is compared so quickly
+   after it is modified that it is within the mtime resolution of
+   the underlying filesystem.
+
+
 .. _dircmp-objects:
 
 The :class:`dircmp` class
diff --git a/Lib/filecmp.py b/Lib/filecmp.py
--- a/Lib/filecmp.py
+++ b/Lib/filecmp.py
@@ -6,6 +6,7 @@
 Functions:
     cmp(f1, f2, shallow=True) -> int
     cmpfiles(a, b, common) -> ([], [], [])
+    clear_cache()
 
 """
 
@@ -13,7 +14,7 @@
 import stat
 from itertools import filterfalse
 
-__all__ = ['cmp', 'dircmp', 'cmpfiles', 'DEFAULT_IGNORES']
+__all__ = ['clear_cache', 'cmp', 'dircmp', 'cmpfiles', 'DEFAULT_IGNORES']
 
 _cache = {}
 BUFSIZE = 8*1024
@@ -21,6 +22,9 @@
 DEFAULT_IGNORES = [
     'RCS', 'CVS', 'tags', '.git', '.hg', '.bzr', '_darcs', '__pycache__']
 
+def clear_cache():
+    """Clear the filecmp cache."""
+    _cache.clear()
 
 def cmp(f1, f2, shallow=True):
     """Compare two files.
@@ -39,7 +43,8 @@
     True if the files are the same, False otherwise.
 
     This function uses a cache for past comparisons and the results,
-    with a cache invalidation mechanism relying on stale signatures.
+    with a cache invalidation mechanism relying on stale signatures
+    or by explicitly calling clear_cache().
 
     """
 
@@ -56,7 +61,7 @@
     if outcome is None:
         outcome = _do_cmp(f1, f2)
         if len(_cache) > 100:      # limit the maximum size of the cache
-            _cache.clear()
+            clear_cache()
         _cache[f1, f2, s1, s2] = outcome
     return outcome
 
diff --git a/Lib/test/test_filecmp.py b/Lib/test/test_filecmp.py
--- a/Lib/test/test_filecmp.py
+++ b/Lib/test/test_filecmp.py
@@ -39,6 +39,13 @@
         self.assertFalse(filecmp.cmp(self.name, self.dir),
                     "File and directory compare as equal")
 
+    def test_cache_clear(self):
+        first_compare = filecmp.cmp(self.name, self.name_same, shallow=False)
+        second_compare = filecmp.cmp(self.name, self.name_diff, shallow=False)
+        filecmp.clear_cache()
+        self.assertTrue(len(filecmp._cache) == 0,
+                        "Cache not cleared after calling clear_cache")
+
 class DirCompareTestCase(unittest.TestCase):
     def setUp(self):
         tmpdir = tempfile.gettempdir()
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -738,6 +738,7 @@
 Christopher Tur Lesniewski-Laas
 Alain Leufroy
 Mark Levinson
+Mark Levitt
 William Lewis
 Akira Li
 Xuanji Li
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- Issue #18149: Add filecmp.clear_cache() to manually clear the filecmp cache.
+  Patch by Mark Levitt
+
 - Issue #18193: Add importlib.reload().
 
 - Issue #18157: Stop using imp.load_module() in pydoc.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 00:44:11 2013
From: python-checkins at python.org (victor.stinner)
Date: Sat, 15 Jun 2013 00:44:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=233329=3A_Add_new_A?=
 =?utf-8?q?PIs_to_customize_memory_allocators?=
Message-ID: <3bXH1R0dqGz7LkT@mail.python.org>

http://hg.python.org/cpython/rev/6661a8154eb3
changeset:   84127:6661a8154eb3
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sat Jun 15 00:37:46 2013 +0200
summary:
  Issue #3329: Add new APIs to customize memory allocators

* Add a new PyMemAllocators structure
* New functions:

  - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory
    allocator functions
  - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()
  - PyMem_GetAllocators(), PyMem_SetAllocators()
  - PyMem_SetupDebugHooks()
  - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()

* Add unit test for PyMem_Malloc(0) and PyObject_Malloc(0)
* Add unit test for new get/set allocators functions
* PyObject_Malloc() now falls back on PyMem_Malloc() instead of malloc() if
  size is bigger than SMALL_REQUEST_THRESHOLD, and PyObject_Realloc() falls
  back on PyMem_Realloc() instead of realloc()
* PyMem_Malloc() and PyMem_Realloc() now always call malloc() and realloc(),
  instead of calling PyObject_Malloc() and PyObject_Realloc() in debug mode

files:
  Doc/c-api/memory.rst      |  121 ++++++-
  Include/objimpl.h         |   67 +-
  Include/pymem.h           |   91 +++-
  Modules/_testcapimodule.c |  178 +++++++++
  Objects/object.c          |   20 -
  Objects/obmalloc.c        |  501 ++++++++++++++++++-------
  6 files changed, 769 insertions(+), 209 deletions(-)


diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst
--- a/Doc/c-api/memory.rst
+++ b/Doc/c-api/memory.rst
@@ -84,6 +84,46 @@
 for the I/O buffer escapes completely the Python memory manager.
 
 
+Raw Memory Interface
+====================
+
+The following function are wrappers to system allocators: :c:func:`malloc`,
+:c:func:`realloc`, :c:func:`free`. These functions are thread-safe, the
+:term:`GIL <global interpreter lock>` does not need to be held to use these
+functions.
+
+The behaviour of requesting zero bytes is not defined: return *NULL* or a
+distinct non-*NULL* pointer depending on the platform. Use
+:c:func:`PyMem_Malloc` and :c:func:`PyMem_Realloc` to have a well defined
+behaviour.
+
+.. versionadded:: 3.4
+
+.. c:function:: void* PyMem_RawMalloc(size_t n)
+
+   Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the
+   allocated memory, or *NULL* if the request fails. The memory
+   will not have been initialized in any way.
+
+
+.. c:function:: void* PyMem_RawRealloc(void *p, size_t n)
+
+   Resizes the memory block pointed to by *p* to *n* bytes. The contents will
+   be unchanged to the minimum of the old and the new sizes. If *p* is *NULL*,
+   the call is equivalent to ``PyMem_RawMalloc(n)``. Unless *p* is *NULL*, it
+   must have been returned by a previous call to :c:func:`PyMem_RawMalloc` or
+   :c:func:`PyMem_RawRealloc`. If the request fails, :c:func:`PyMem_RawRealloc`
+   returns *NULL* and *p* remains a valid pointer to the previous memory area.
+
+
+.. c:function:: void PyMem_RawFree(void *p)
+
+   Frees the memory block pointed to by *p*, which must have been returned by a
+   previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`.
+   Otherwise, or if ``PyMem_Free(p)`` has been called before, undefined
+   behavior occurs. If *p* is *NULL*, no operation is performed.
+
+
 .. _memoryinterface:
 
 Memory Interface
@@ -91,8 +131,12 @@
 
 The following function sets, modeled after the ANSI C standard, but specifying
 behavior when requesting zero bytes, are available for allocating and releasing
-memory from the Python heap:
+memory from the Python heap.
 
+.. warning::
+
+   The :term:`GIL <global interpreter lock>` must be held when using these
+   functions.
 
 .. c:function:: void* PyMem_Malloc(size_t n)
 
@@ -155,6 +199,81 @@
 :c:func:`PyMem_NEW`, :c:func:`PyMem_RESIZE`, :c:func:`PyMem_DEL`.
 
 
+Customize Memory Allocators
+===========================
+
+.. versionadded:: 3.4
+
+.. c:type:: PyMemAllocators
+
+   Structure used to describe memory allocator.  This structure has
+   four fields:
+
+   +----------------------------------------------------------+-----------------+
+   | Field                                                    | Meaning         |
+   +==========================================================+=================+
+   | ``void *ctx``                                            | user data       |
+   +----------------------------------------------------------+-----------------+
+   | ``void* malloc(void *ctx, size_t size)``                 | allocate memory |
+   +----------------------------------------------------------+-----------------+
+   | ``void* realloc(void *ctx, void *ptr, size_t new_size)`` | allocate memory |
+   |                                                          | or resize a     |
+   |                                                          | memory block    |
+   +----------------------------------------------------------+-----------------+
+   | ``void free(void *ctx, void *ptr)``                      | release memory  |
+   +----------------------------------------------------------+-----------------+
+
+.. c:function:: void PyMem_GetRawAllocators(PyMemAllocators *allocators)
+
+   Get internal functions of :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`
+   and :c:func:`PyMem_RawFree`.
+
+.. c:function:: void PyMem_SetRawAllocators(PyMemAllocators *allocators)
+
+   Set internal functions of :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`
+   and :c:func:`PyMem_RawFree`.
+
+   :c:func:`PyMem_SetupDebugHooks` should be called to reinstall debug hooks if
+   new functions do no call original functions anymore.
+
+.. c:function:: void PyMem_GetAllocators(PyMemAllocators *allocators)
+
+   Get internal functions of :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`
+   and :c:func:`PyMem_Free`.
+
+.. c:function:: void PyMem_SetAllocators(PyMemAllocators *allocators)
+
+   Set internal functions of :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`
+   and :c:func:`PyMem_Free`.
+
+   ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*: it
+   would be treated as an error.
+
+   :c:func:`PyMem_SetupDebugHooks` should be called to reinstall debug hooks if
+   new functions do no call original functions anymore.
+
+.. c:function:: void PyMem_SetupDebugHooks(void)
+
+   Setup hooks to detect bugs in the following Python memory allocator
+   functions:
+
+   - :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`,
+     :c:func:`PyMem_RawFree`
+   - :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, :c:func:`PyMem_Free`
+   - :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`,
+     :c:func:`PyObject_Free`
+
+   Newly allocated memory is filled with the byte ``0xCB``, freed memory is
+   filled with the byte ``0xDB``. Additionnal checks:
+
+   - detect API violations, ex: :c:func:`PyObject_Free` called on a buffer
+     allocated by :c:func:`PyMem_Malloc`
+   - detect write before the start of the buffer (buffer underflow)
+   - detect write after the end of the buffer (buffer overflow)
+
+   The function does nothing if Python is not compiled is debug mode.
+
+
 .. _memoryexamples:
 
 Examples
diff --git a/Include/objimpl.h b/Include/objimpl.h
--- a/Include/objimpl.h
+++ b/Include/objimpl.h
@@ -94,9 +94,9 @@
    the object gets initialized via PyObject_{Init, InitVar} after obtaining
    the raw memory.
 */
-PyAPI_FUNC(void *) PyObject_Malloc(size_t);
-PyAPI_FUNC(void *) PyObject_Realloc(void *, size_t);
-PyAPI_FUNC(void) PyObject_Free(void *);
+PyAPI_FUNC(void *) PyObject_Malloc(size_t size);
+PyAPI_FUNC(void *) PyObject_Realloc(void *ptr, size_t new_size);
+PyAPI_FUNC(void) PyObject_Free(void *ptr);
 
 /* This function returns the number of allocated memory blocks, regardless of size */
 PyAPI_FUNC(Py_ssize_t) _Py_GetAllocatedBlocks(void);
@@ -106,41 +106,46 @@
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(void) _PyObject_DebugMallocStats(FILE *out);
 #endif /* #ifndef Py_LIMITED_API */
-#ifdef PYMALLOC_DEBUG   /* WITH_PYMALLOC && PYMALLOC_DEBUG */
-PyAPI_FUNC(void *) _PyObject_DebugMalloc(size_t nbytes);
-PyAPI_FUNC(void *) _PyObject_DebugRealloc(void *p, size_t nbytes);
-PyAPI_FUNC(void) _PyObject_DebugFree(void *p);
-PyAPI_FUNC(void) _PyObject_DebugDumpAddress(const void *p);
-PyAPI_FUNC(void) _PyObject_DebugCheckAddress(const void *p);
-PyAPI_FUNC(void *) _PyObject_DebugMallocApi(char api, size_t nbytes);
-PyAPI_FUNC(void *) _PyObject_DebugReallocApi(char api, void *p, size_t nbytes);
-PyAPI_FUNC(void) _PyObject_DebugFreeApi(char api, void *p);
-PyAPI_FUNC(void) _PyObject_DebugCheckAddressApi(char api, const void *p);
-PyAPI_FUNC(void *) _PyMem_DebugMalloc(size_t nbytes);
-PyAPI_FUNC(void *) _PyMem_DebugRealloc(void *p, size_t nbytes);
-PyAPI_FUNC(void) _PyMem_DebugFree(void *p);
-#define PyObject_MALLOC         _PyObject_DebugMalloc
-#define PyObject_Malloc         _PyObject_DebugMalloc
-#define PyObject_REALLOC        _PyObject_DebugRealloc
-#define PyObject_Realloc        _PyObject_DebugRealloc
-#define PyObject_FREE           _PyObject_DebugFree
-#define PyObject_Free           _PyObject_DebugFree
+#endif
 
-#else   /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */
+/* Macros */
 #define PyObject_MALLOC         PyObject_Malloc
 #define PyObject_REALLOC        PyObject_Realloc
 #define PyObject_FREE           PyObject_Free
-#endif
+#define PyObject_Del            PyObject_Free
+#define PyObject_DEL            PyObject_Free
 
-#else   /* ! WITH_PYMALLOC */
-#define PyObject_MALLOC         PyMem_MALLOC
-#define PyObject_REALLOC        PyMem_REALLOC
-#define PyObject_FREE           PyMem_FREE
+/* Get internal functions of PyObject_Malloc(), PyObject_Realloc() and
+   PyObject_Free(). *ctx_p is an arbitrary user value. */
+PyAPI_FUNC(void) PyObject_GetAllocators(PyMemAllocators *allocators);
 
-#endif  /* WITH_PYMALLOC */
+/* Set internal functions of PyObject_Malloc(), PyObject_Realloc() and PyObject_Free().
+   ctx is an arbitrary user value.
 
-#define PyObject_Del            PyObject_Free
-#define PyObject_DEL            PyObject_FREE
+   malloc(ctx, 0) and realloc(ctx, ptr, 0) must not return NULL: it would be
+   treated as an error.
+
+   PyMem_SetupDebugHooks() should be called to reinstall debug hooks if new
+   functions do no call original functions anymore. */
+PyAPI_FUNC(void) PyObject_SetAllocators(PyMemAllocators *allocators);
+
+/* Get internal functions allocating and deallocating arenas for
+   PyObject_Malloc(), PyObject_Realloc() and PyObject_Free().
+   *ctx_p is an arbitrary user value. */
+PyAPI_FUNC(void) _PyObject_GetArenaAllocators(
+    void **ctx_p,
+    void* (**malloc_p) (void *ctx, size_t size),
+    void (**free_p) (void *ctx, void *ptr, size_t size)
+    );
+
+/* Get internal functions allocating and deallocating arenas for
+   PyObject_Malloc(), PyObject_Realloc() and PyObject_Free().
+   ctx is an arbitrary user value. */
+PyAPI_FUNC(void) _PyObject_SetArenaAllocators(
+    void *ctx,
+    void* (*malloc) (void *ctx, size_t size),
+    void (*free) (void *ctx, void *ptr, size_t size)
+    );
 
 /*
  * Generic object allocator interface
diff --git a/Include/pymem.h b/Include/pymem.h
--- a/Include/pymem.h
+++ b/Include/pymem.h
@@ -11,6 +11,40 @@
 extern "C" {
 #endif
 
+typedef struct {
+    /* user context passed as the first argument to the 3 functions */
+    void *ctx;
+
+    /* allocate memory */
+    void* (*malloc) (void *ctx, size_t size);
+
+    /* allocate memory or resize a memory buffer */
+    void* (*realloc) (void *ctx, void *ptr, size_t new_size);
+
+    /* release memory */
+    void (*free) (void *ctx, void *ptr);
+} PyMemAllocators;
+
+/* Raw memory allocators, system functions: malloc(), realloc(), free().
+
+   These functions are thread-safe, the GIL does not need to be held. */
+
+/* Get internal functions of PyMem_RawMalloc(), PyMem_RawRealloc() and
+   PyMem_RawFree(). *ctx_p is an arbitrary user value. */
+PyAPI_FUNC(void) PyMem_GetRawAllocators(PyMemAllocators *allocators);
+
+/* Set internal functions of PyMem_RawMalloc(), PyMem_RawRealloc() and
+   PyMem_RawFree(). ctx is an arbitrary user value.
+
+   PyMem_SetupDebugHooks() should be called to reinstall debug hooks if new
+   functions do no call original functions anymore. */
+PyAPI_FUNC(void) PyMem_SetRawAllocators(PyMemAllocators *allocators);
+
+PyAPI_FUNC(void *) PyMem_RawMalloc(size_t size);
+PyAPI_FUNC(void *) PyMem_RawRealloc(void *ptr, size_t new_size);
+PyAPI_FUNC(void) PyMem_RawFree(void *ptr);
+
+
 /* BEWARE:
 
    Each interface exports both functions and macros.  Extension modules should
@@ -49,21 +83,11 @@
    performed on failure (no exception is set, no warning is printed, etc).
 */
 
-PyAPI_FUNC(void *) PyMem_Malloc(size_t);
-PyAPI_FUNC(void *) PyMem_Realloc(void *, size_t);
-PyAPI_FUNC(void) PyMem_Free(void *);
-
-/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
-   no longer supported. They used to call PyErr_NoMemory() on failure. */
+PyAPI_FUNC(void *) PyMem_Malloc(size_t size);
+PyAPI_FUNC(void *) PyMem_Realloc(void *ptr, size_t new_size);
+PyAPI_FUNC(void) PyMem_Free(void *ptr);
 
 /* Macros. */
-#ifdef PYMALLOC_DEBUG
-/* Redirect all memory operations to Python's debugging allocator. */
-#define PyMem_MALLOC		_PyMem_DebugMalloc
-#define PyMem_REALLOC		_PyMem_DebugRealloc
-#define PyMem_FREE		_PyMem_DebugFree
-
-#else	/* ! PYMALLOC_DEBUG */
 
 /* PyMem_MALLOC(0) means malloc(1). Some systems would return NULL
    for malloc(0), which would be treated as an error. Some platforms
@@ -71,13 +95,9 @@
    pymalloc. To solve these problems, allocate an extra byte. */
 /* Returns NULL to indicate error if a negative size or size larger than
    Py_ssize_t can represent is supplied.  Helps prevents security holes. */
-#define PyMem_MALLOC(n)		((size_t)(n) > (size_t)PY_SSIZE_T_MAX ? NULL \
-				: malloc((n) ? (n) : 1))
-#define PyMem_REALLOC(p, n)	((size_t)(n) > (size_t)PY_SSIZE_T_MAX  ? NULL \
-				: realloc((p), (n) ? (n) : 1))
-#define PyMem_FREE		free
-
-#endif	/* PYMALLOC_DEBUG */
+#define PyMem_MALLOC(n)         PyMem_Malloc(n)
+#define PyMem_REALLOC(p, n)     PyMem_Realloc(p, n)
+#define PyMem_FREE(p)           PyMem_Free(p)
 
 /*
  * Type-oriented memory interface
@@ -115,6 +135,37 @@
 #define PyMem_Del		PyMem_Free
 #define PyMem_DEL		PyMem_FREE
 
+/* Get internal functions of PyMem_Malloc(), PyMem_Realloc()
+   and PyMem_Free() */
+PyAPI_FUNC(void) PyMem_GetAllocators(PyMemAllocators *allocators);
+
+/* Set internal functions of PyMem_Malloc(), PyMem_Realloc() and PyMem_Free().
+
+   malloc(ctx, 0) and realloc(ctx, ptr, 0) must not return NULL: it would be
+   treated as an error.
+
+   PyMem_SetupDebugHooks() should be called to reinstall debug hooks if new
+   functions do no call original functions anymore. */
+PyAPI_FUNC(void) PyMem_SetAllocators(PyMemAllocators *allocators);
+
+/* Setup hooks to detect bugs in the following Python memory allocator
+   functions:
+
+   - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
+   - PyMem_Malloc(), PyMem_Realloc(), PyMem_Free()
+   - PyObject_Malloc(), PyObject_Realloc() and PyObject_Free()
+
+   Newly allocated memory is filled with the byte 0xCB, freed memory is filled
+   with the byte 0xDB. Additionnal checks:
+
+   - detect API violations, ex: PyObject_Free() called on a buffer allocated
+     by PyMem_Malloc()
+   - detect write before the start of the buffer (buffer underflow)
+   - detect write after the end of the buffer (buffer overflow)
+
+   The function does nothing if Python is not compiled is debug mode. */
+PyAPI_FUNC(void) PyMem_SetupDebugHooks(void);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2511,6 +2511,176 @@
     Py_RETURN_NONE;
 }
 
+static PyObject *
+test_pymem_alloc0(PyObject *self)
+{
+    void *ptr;
+
+    ptr = PyMem_Malloc(0);
+    if (ptr == NULL) {
+        PyErr_SetString(PyExc_RuntimeError, "PyMem_Malloc(0) returns NULL");
+        return NULL;
+    }
+    PyMem_Free(ptr);
+
+    ptr = PyObject_Malloc(0);
+    if (ptr == NULL) {
+        PyErr_SetString(PyExc_RuntimeError, "PyObject_Malloc(0) returns NULL");
+        return NULL;
+    }
+    PyObject_Free(ptr);
+
+    Py_RETURN_NONE;
+}
+
+typedef struct {
+    PyMemAllocators alloc;
+
+    size_t malloc_size;
+    void *realloc_ptr;
+    size_t realloc_new_size;
+    void *free_ptr;
+} alloc_hook_t;
+
+static void* hook_malloc (void* ctx, size_t size)
+{
+    alloc_hook_t *hook = (alloc_hook_t *)ctx;
+    hook->malloc_size = size;
+    return hook->alloc.malloc(hook->alloc.ctx, size);
+}
+
+static void* hook_realloc (void* ctx, void* ptr, size_t new_size)
+{
+    alloc_hook_t *hook = (alloc_hook_t *)ctx;
+    hook->realloc_ptr = ptr;
+    hook->realloc_new_size = new_size;
+    return hook->alloc.realloc(hook->alloc.ctx, ptr, new_size);
+}
+
+static void hook_free (void *ctx, void *ptr)
+{
+    alloc_hook_t *hook = (alloc_hook_t *)ctx;
+    printf("HOOK\n");
+    hook->free_ptr = ptr;
+    hook->alloc.free(hook->alloc.ctx, ptr);
+}
+
+static PyObject *
+test_setallocators(char api)
+{
+    PyObject *res = NULL;
+    const char *error_msg;
+    alloc_hook_t hook;
+    PyMemAllocators alloc;
+    size_t size, size2;
+    void *ptr, *ptr2;
+
+    hook.malloc_size = 0;
+    hook.realloc_ptr = NULL;
+    hook.realloc_new_size = 0;
+    hook.free_ptr = NULL;
+
+    alloc.ctx = &hook;
+    alloc.malloc = &hook_malloc;
+    alloc.realloc = &hook_realloc;
+    alloc.free = &hook_free;
+    if (api == 'o') {
+        PyObject_GetAllocators(&hook.alloc);
+        PyObject_SetAllocators(&alloc);
+    }
+    else if (api == 'r') {
+        PyMem_GetRawAllocators(&hook.alloc);
+        PyMem_SetRawAllocators(&alloc);
+    }
+    else {
+        PyMem_GetAllocators(&hook.alloc);
+        PyMem_SetAllocators(&alloc);
+    }
+
+    size = 42;
+    if (api == 'o')
+        ptr = PyObject_Malloc(size);
+    else if (api == 'r')
+        ptr = PyMem_RawMalloc(size);
+    else
+        ptr = PyMem_Malloc(size);
+    if (ptr == NULL) {
+        error_msg = "malloc failed";
+        goto fail;
+    }
+
+    if (hook.malloc_size != size) {
+        error_msg = "malloc invalid size";
+        goto fail;
+    }
+
+    size2 = 200;
+    if (api == 'o')
+        ptr2 = PyObject_Realloc(ptr, size2);
+    else if (api == 'r')
+        ptr2 = PyMem_RawRealloc(ptr, size2);
+    else
+        ptr2 = PyMem_Realloc(ptr, size2);
+    if (ptr2 == NULL) {
+        error_msg = "realloc failed";
+        goto fail;
+    }
+
+    if (hook.realloc_ptr != ptr
+        || hook.realloc_new_size != size2) {
+        error_msg = "realloc invalid parameters";
+        goto fail;
+    }
+
+    if (api == 'o')
+        PyObject_Free(ptr2);
+    else if (api == 'r')
+        PyMem_RawFree(ptr2);
+    else {
+        printf("PyMem_Free\n");
+        PyMem_Free(ptr2);
+    }
+
+    if (hook.free_ptr != ptr2) {
+        error_msg = "free invalid pointer";
+        goto fail;
+    }
+
+    Py_INCREF(Py_None);
+    res = Py_None;
+    goto finally;
+
+fail:
+    PyErr_SetString(PyExc_RuntimeError, error_msg);
+
+finally:
+    if (api == 'o')
+        PyObject_SetAllocators(&hook.alloc);
+    else if (api == 'r')
+        PyMem_SetRawAllocators(&hook.alloc);
+    else
+        PyMem_SetAllocators(&hook.alloc);
+    return res;
+}
+
+static PyObject *
+test_pymem_setrawallocators(PyObject *self)
+{
+    return test_setallocators('r');
+}
+
+static PyObject *
+test_pymem_setallocators(PyObject *self)
+{
+    return test_setallocators('m');
+}
+
+static PyObject *
+test_pyobject_setallocators(PyObject *self)
+{
+    return test_setallocators('o');
+}
+
 static PyMethodDef TestMethods[] = {
     {"raise_exception",         raise_exception,                 METH_VARARGS},
     {"raise_memoryerror",   (PyCFunction)raise_memoryerror,  METH_NOARGS},
@@ -2611,6 +2781,14 @@
     {"pytime_object_to_time_t", test_pytime_object_to_time_t,  METH_VARARGS},
     {"pytime_object_to_timeval", test_pytime_object_to_timeval,  METH_VARARGS},
     {"pytime_object_to_timespec", test_pytime_object_to_timespec,  METH_VARARGS},
+    {"test_pymem",
+     (PyCFunction)test_pymem_alloc0, METH_NOARGS},
+    {"test_pymem_alloc0",
+     (PyCFunction)test_pymem_setrawallocators, METH_NOARGS},
+    {"test_pymem_setallocators",
+     (PyCFunction)test_pymem_setallocators, METH_NOARGS},
+    {"test_pyobject_setallocators",
+     (PyCFunction)test_pyobject_setallocators, METH_NOARGS},
     {NULL, NULL} /* sentinel */
 };
 
diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -1859,26 +1859,6 @@
 Py_ssize_t (*_Py_abstract_hack)(PyObject *) = PyObject_Size;
 
 
-/* Python's malloc wrappers (see pymem.h) */
-
-void *
-PyMem_Malloc(size_t nbytes)
-{
-    return PyMem_MALLOC(nbytes);
-}
-
-void *
-PyMem_Realloc(void *p, size_t nbytes)
-{
-    return PyMem_REALLOC(p, nbytes);
-}
-
-void
-PyMem_Free(void *p)
-{
-    PyMem_FREE(p);
-}
-
 void
 _PyObject_DebugTypeStats(FILE *out)
 {
diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c
--- a/Objects/obmalloc.c
+++ b/Objects/obmalloc.c
@@ -1,5 +1,327 @@
 #include "Python.h"
 
+/* Python's malloc wrappers (see pymem.h) */
+
+/* Forward declaration */
+
+#ifdef PYMALLOC_DEBUG   /* WITH_PYMALLOC && PYMALLOC_DEBUG */
+static void* _PyMem_DebugMalloc(void *ctx, size_t size);
+static void _PyMem_DebugFree(void *ctx, void *p);
+static void* _PyMem_DebugRealloc(void *ctx, void *ptr, size_t size);
+
+static void _PyObject_DebugDumpAddress(const void *p);
+static void _PyMem_DebugCheckAddress(char api_id, const void *p);
+#endif
+
+#ifdef WITH_PYMALLOC
+static void* _PyObject_Malloc(void *ctx, size_t size);
+static void _PyObject_Free(void *ctx, void *p);
+static void* _PyObject_Realloc(void *ctx, void *ptr, size_t size);
+#endif
+
+
+static void *
+_PyMem_RawMalloc(void *ctx, size_t size)
+{
+    return malloc(size);
+}
+
+static void *
+_PyMem_RawRealloc(void *ctx, void *ptr, size_t size)
+{
+    return realloc(ptr, size);
+}
+
+static void
+_PyMem_RawFree(void *ctx, void *ptr)
+{
+    return free(ptr);
+}
+
+static void *
+_PyMem_Malloc(void *ctx, size_t size)
+{
+    /* PyMem_Malloc(0) means malloc(1). Some systems would return NULL
+       for malloc(0), which would be treated as an error. Some platforms would
+       return a pointer with no memory behind it, which would break pymalloc.
+       To solve these problems, allocate an extra byte. */
+    if (size == 0)
+        size = 1;
+    return malloc(size);
+}
+
+static void *
+_PyMem_Realloc(void *ctx, void *ptr, size_t size)
+{
+    if (size == 0)
+        size = 1;
+    return realloc(ptr, size);
+}
+
+#ifdef ARENAS_USE_MMAP
+static void *
+_PyObject_ArenaMmap(void *ctx, size_t size)
+{
+    void *ptr;
+    ptr = mmap(NULL, size, PROT_READ|PROT_WRITE,
+               MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
+    if (ptr == MAP_FAILED)
+        return NULL;
+    assert(ptr != NULL);
+    return ptr;
+}
+
+static void
+_PyObject_ArenaMunmap(void *ctx, void *ptr, size_t size)
+{
+    return munmap(ptr, size);
+}
+#else
+static void *
+_PyObject_ArenaMalloc(void *ctx, size_t size)
+{
+    return malloc(size);
+}
+
+static void
+_PyObject_ArenaFree(void *ctx, void *ptr, size_t size)
+{
+    free(ptr);
+}
+#endif
+
+#define PYRAW_FUNCS _PyMem_RawMalloc, _PyMem_RawRealloc, _PyMem_RawFree
+#define PYMEM_FUNCS _PyMem_Malloc, _PyMem_Realloc, _PyMem_RawFree
+#ifdef WITH_PYMALLOC
+#define PYOBJECT_FUNCS _PyObject_Malloc, _PyObject_Realloc, _PyObject_Free
+#else
+#define PYOBJECT_FUNCS PYMEM_FUNCS
+#endif
+
+#ifdef PYMALLOC_DEBUG
+typedef struct {
+    /* We tag each block with an API ID in order to tag API violations */
+    char api_id;
+    PyMemAllocators alloc;
+} debug_alloc_api_t;
+static struct {
+    debug_alloc_api_t raw;
+    debug_alloc_api_t mem;
+    debug_alloc_api_t obj;
+} _PyMem_Debug = {
+    {'r', {NULL, PYRAW_FUNCS}},
+    {'m', {NULL, PYMEM_FUNCS}},
+    {'o', {NULL, PYOBJECT_FUNCS}}
+    };
+
+#define PYDEBUG_FUNCS _PyMem_DebugMalloc, _PyMem_DebugRealloc, _PyMem_DebugFree
+#endif
+
+static PyMemAllocators _PyMem_Raw = {
+#ifdef PYMALLOC_DEBUG
+    &_PyMem_Debug.raw, PYDEBUG_FUNCS
+#else
+    NULL, PYMEM_FUNCS
+#endif
+    };
+
+static PyMemAllocators _PyMem = {
+#ifdef PYMALLOC_DEBUG
+    &_PyMem_Debug.mem, PYDEBUG_FUNCS
+#else
+    NULL, PYMEM_FUNCS
+#endif
+    };
+
+static PyMemAllocators _PyObject = {
+#ifdef PYMALLOC_DEBUG
+    &_PyMem_Debug.obj, PYDEBUG_FUNCS
+#else
+    NULL, PYOBJECT_FUNCS
+#endif
+    };
+
+#undef PYRAW_FUNCS
+#undef PYMEM_FUNCS
+#undef PYOBJECT_FUNCS
+#undef PYDEBUG_FUNCS
+
+static struct {
+    void *ctx;
+    void* (*malloc) (void*, size_t);
+    void (*free) (void*, void*, size_t);
+} _PyObject_Arena = {NULL,
+#ifdef ARENAS_USE_MMAP
+    _PyObject_ArenaMmap, _PyObject_ArenaMunmap
+#else
+    _PyObject_ArenaMalloc, _PyObject_ArenaFree
+#endif
+    };
+
+void
+PyMem_SetupDebugHooks(void)
+{
+#ifdef PYMALLOC_DEBUG
+    PyMemAllocators alloc;
+
+    alloc.malloc = _PyMem_DebugMalloc;
+    alloc.realloc = _PyMem_DebugRealloc;
+    alloc.free = _PyMem_DebugFree;
+
+    if (_PyMem_Raw.malloc != _PyMem_DebugMalloc) {
+        alloc.ctx = &_PyMem_Debug.raw;
+        PyMem_GetAllocators(&_PyMem_Debug.raw.alloc);
+        PyMem_SetAllocators(&alloc);
+    }
+
+    if (_PyMem.malloc != _PyMem_DebugMalloc) {
+        alloc.ctx = &_PyMem_Debug.mem;
+        PyMem_GetAllocators(&_PyMem_Debug.mem.alloc);
+        PyMem_SetAllocators(&alloc);
+    }
+
+    if (_PyObject.malloc != _PyMem_DebugMalloc) {
+        alloc.ctx = &_PyMem_Debug.obj;
+        PyObject_GetAllocators(&_PyMem_Debug.obj.alloc);
+        PyObject_SetAllocators(&alloc);
+    }
+#endif
+}
+
+void
+PyMem_GetRawAllocators(PyMemAllocators *allocators)
+{
+    *allocators = _PyMem_Raw;
+}
+
+void
+PyMem_SetRawAllocators(PyMemAllocators *allocators)
+{
+    _PyMem_Raw = *allocators;
+}
+
+void
+PyMem_GetAllocators(PyMemAllocators *allocators)
+{
+    *allocators = _PyMem;
+}
+
+void
+PyMem_SetAllocators(PyMemAllocators *allocators)
+{
+    _PyMem = *allocators;
+}
+
+void
+PyObject_GetAllocators(PyMemAllocators *allocators)
+{
+    *allocators = _PyObject;
+}
+
+void
+PyObject_SetAllocators(PyMemAllocators *allocators)
+{
+    _PyObject = *allocators;
+}
+
+void
+_PyObject_GetArenaAllocators(void **ctx_p,
+                             void* (**malloc_p) (void *ctx, size_t size),
+                             void (**free_p) (void *ctx, void *ptr, size_t size))
+{
+    *malloc_p = _PyObject_Arena.malloc;
+    *free_p = _PyObject_Arena.free;
+    *ctx_p = _PyObject_Arena.ctx;
+}
+
+void
+_PyObject_SetArenaAllocators(void *ctx,
+                             void* (*malloc) (void *ctx, size_t size),
+                             void (*free) (void *ctx, void *ptr, size_t size))
+{
+    _PyObject_Arena.malloc = malloc;
+    _PyObject_Arena.free = free;
+    _PyObject_Arena.ctx = ctx;
+}
+
+void *
+PyMem_RawMalloc(size_t size)
+{
+    return _PyMem_Raw.malloc(_PyMem_Raw.ctx, size);
+}
+
+void*
+PyMem_RawRealloc(void *ptr, size_t new_size)
+{
+    return _PyMem_Raw.realloc(_PyMem_Raw.ctx, ptr, new_size);
+}
+
+void PyMem_RawFree(void *ptr)
+{
+    _PyMem_Raw.free(_PyMem_Raw.ctx, ptr);
+}
+
+void *
+PyMem_Malloc(size_t size)
+{
+    /*
+     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
+     * Most python internals blindly use a signed Py_ssize_t to track
+     * things without checking for overflows or negatives.
+     * As size_t is unsigned, checking for size < 0 is not required.
+     */
+    if (size > (size_t)PY_SSIZE_T_MAX)
+        return NULL;
+
+    return _PyMem.malloc(_PyMem.ctx, size);
+}
+
+void *
+PyMem_Realloc(void *ptr, size_t new_size)
+{
+    if (new_size > (size_t)PY_SSIZE_T_MAX)
+        return NULL;
+
+    return _PyMem.realloc(_PyMem.ctx, ptr, new_size);
+}
+
+void
+PyMem_Free(void *ptr)
+{
+    _PyMem.free(_PyMem.ctx, ptr);
+}
+
+void *
+PyObject_Malloc(size_t size)
+{
+    /*
+     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
+     * Most python internals blindly use a signed Py_ssize_t to track
+     * things without checking for overflows or negatives.
+     * As size_t is unsigned, checking for size < 0 is not required.
+     */
+    if (size > (size_t)PY_SSIZE_T_MAX)
+        return NULL;
+
+    return _PyObject.malloc(_PyObject.ctx, size);
+}
+
+void *
+PyObject_Realloc(void *ptr, size_t new_size)
+{
+    if (new_size > (size_t)PY_SSIZE_T_MAX)
+        return NULL;
+
+    return _PyObject.realloc(_PyObject.ctx, ptr, new_size);
+}
+
+void
+PyObject_Free(void *ptr)
+{
+    _PyObject.free(_PyObject.ctx, ptr);
+}
+
+
 #ifdef WITH_PYMALLOC
 
 #ifdef HAVE_MMAP
@@ -545,7 +867,6 @@
     struct arena_object* arenaobj;
     uint excess;        /* number of bytes above pool alignment */
     void *address;
-    int err;
 
 #ifdef PYMALLOC_DEBUG
     if (Py_GETENV("PYTHONMALLOCSTATS"))
@@ -567,11 +888,12 @@
             return NULL;                /* overflow */
 #endif
         nbytes = numarenas * sizeof(*arenas);
-        arenaobj = (struct arena_object *)realloc(arenas, nbytes);
+        arenaobj = (struct arena_object *)PyMem_Realloc(arenas, nbytes);
         if (arenaobj == NULL)
             return NULL;
         arenas = arenaobj;
 
+
         /* We might need to fix pointers that were copied.  However,
          * new_arena only gets called when all the pages in the
          * previous arenas are full.  Thus, there are *no* pointers
@@ -598,15 +920,8 @@
     arenaobj = unused_arena_objects;
     unused_arena_objects = arenaobj->nextarena;
     assert(arenaobj->address == 0);
-#ifdef ARENAS_USE_MMAP
-    address = mmap(NULL, ARENA_SIZE, PROT_READ|PROT_WRITE,
-                                   MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
-    err = (address == MAP_FAILED);
-#else
-    address = malloc(ARENA_SIZE);
-    err = (address == 0);
-#endif
-    if (err) {
+    address = _PyObject_Arena.malloc(_PyObject_Arena.ctx, ARENA_SIZE);
+    if (address == NULL) {
         /* The allocation failed: return NULL after putting the
          * arenaobj back.
          */
@@ -769,9 +1084,8 @@
  * Unless the optimizer reorders everything, being too smart...
  */
 
-#undef PyObject_Malloc
-void *
-PyObject_Malloc(size_t nbytes)
+static void *
+_PyObject_Malloc(void *ctx, size_t nbytes)
 {
     block *bp;
     poolp pool;
@@ -788,17 +1102,6 @@
 #endif
 
     /*
-     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
-     * Most python internals blindly use a signed Py_ssize_t to track
-     * things without checking for overflows or negatives.
-     * As size_t is unsigned, checking for nbytes < 0 is not required.
-     */
-    if (nbytes > PY_SSIZE_T_MAX) {
-        _Py_AllocatedBlocks--;
-        return NULL;
-    }
-
-    /*
      * This implicitly redirects malloc(0).
      */
     if ((nbytes - 1) < SMALL_REQUEST_THRESHOLD) {
@@ -970,10 +1273,8 @@
      * last chance to serve the request) or when the max memory limit
      * has been reached.
      */
-    if (nbytes == 0)
-        nbytes = 1;
     {
-        void *result = malloc(nbytes);
+        void *result = PyMem_Malloc(nbytes);
         if (!result)
             _Py_AllocatedBlocks--;
         return result;
@@ -982,9 +1283,8 @@
 
 /* free */
 
-#undef PyObject_Free
-void
-PyObject_Free(void *p)
+static void
+_PyObject_Free(void *ctx, void *p)
 {
     poolp pool;
     block *lastfree;
@@ -1093,11 +1393,8 @@
                 unused_arena_objects = ao;
 
                 /* Free the entire arena. */
-#ifdef ARENAS_USE_MMAP
-                munmap((void *)ao->address, ARENA_SIZE);
-#else
-                free((void *)ao->address);
-#endif
+                _PyObject_Arena.free(_PyObject_Arena.ctx,
+                                     (void *)ao->address, ARENA_SIZE);
                 ao->address = 0;                        /* mark unassociated */
                 --narenas_currently_allocated;
 
@@ -1206,7 +1503,7 @@
 redirect:
 #endif
     /* We didn't allocate this address. */
-    free(p);
+    PyMem_Free(p);
 }
 
 /* realloc.  If p is NULL, this acts like malloc(nbytes).  Else if nbytes==0,
@@ -1214,9 +1511,8 @@
  * return a non-NULL result.
  */
 
-#undef PyObject_Realloc
-void *
-PyObject_Realloc(void *p, size_t nbytes)
+static void *
+_PyObject_Realloc(void *ctx, void *p, size_t nbytes)
 {
     void *bp;
     poolp pool;
@@ -1226,16 +1522,7 @@
 #endif
 
     if (p == NULL)
-        return PyObject_Malloc(nbytes);
-
-    /*
-     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
-     * Most python internals blindly use a signed Py_ssize_t to track
-     * things without checking for overflows or negatives.
-     * As size_t is unsigned, checking for nbytes < 0 is not required.
-     */
-    if (nbytes > PY_SSIZE_T_MAX)
-        return NULL;
+        return _PyObject_Malloc(ctx, nbytes);
 
 #ifdef WITH_VALGRIND
     /* Treat running_on_valgrind == -1 the same as 0 */
@@ -1263,10 +1550,10 @@
             }
             size = nbytes;
         }
-        bp = PyObject_Malloc(nbytes);
+        bp = _PyObject_Malloc(ctx, nbytes);
         if (bp != NULL) {
             memcpy(bp, p, size);
-            PyObject_Free(p);
+            _PyObject_Free(ctx, p);
         }
         return bp;
     }
@@ -1284,14 +1571,14 @@
      * at p.  Instead we punt:  let C continue to manage this block.
      */
     if (nbytes)
-        return realloc(p, nbytes);
+        return PyMem_Realloc(p, nbytes);
     /* C doesn't define the result of realloc(p, 0) (it may or may not
      * return NULL then), but Python's docs promise that nbytes==0 never
      * returns NULL.  We don't pass 0 to realloc(), to avoid that endcase
      * to begin with.  Even then, we can't be sure that realloc() won't
      * return NULL.
      */
-    bp = realloc(p, 1);
+    bp = PyMem_Realloc(p, 1);
     return bp ? bp : p;
 }
 
@@ -1301,24 +1588,6 @@
 /* pymalloc not enabled:  Redirect the entry points to malloc.  These will
  * only be used by extensions that are compiled with pymalloc enabled. */
 
-void *
-PyObject_Malloc(size_t n)
-{
-    return PyMem_MALLOC(n);
-}
-
-void *
-PyObject_Realloc(void *p, size_t n)
-{
-    return PyMem_REALLOC(p, n);
-}
-
-void
-PyObject_Free(void *p)
-{
-    PyMem_FREE(p);
-}
-
 Py_ssize_t
 _Py_GetAllocatedBlocks(void)
 {
@@ -1344,10 +1613,6 @@
 #define DEADBYTE       0xDB    /* dead (newly freed) memory */
 #define FORBIDDENBYTE  0xFB    /* untouchable bytes at each end of a block */
 
-/* We tag each block with an API ID in order to tag API violations */
-#define _PYMALLOC_MEM_ID 'm'   /* the PyMem_Malloc() API */
-#define _PYMALLOC_OBJ_ID 'o'   /* The PyObject_Malloc() API */
-
 static size_t serialno = 0;     /* incremented on each debug {m,re}alloc */
 
 /* serialno is always incremented via calling this routine.  The point is
@@ -1430,58 +1695,18 @@
 p[2*S+n: 2*S+n+S]
     Copies of FORBIDDENBYTE.  Used to catch over- writes and reads.
 p[2*S+n+S: 2*S+n+2*S]
-    A serial number, incremented by 1 on each call to _PyObject_DebugMalloc
-    and _PyObject_DebugRealloc.
+    A serial number, incremented by 1 on each call to _PyMem_DebugMalloc
+    and _PyMem_DebugRealloc.
     This is a big-endian size_t.
     If "bad memory" is detected later, the serial number gives an
     excellent way to set a breakpoint on the next run, to capture the
     instant at which this block was passed out.
 */
 
-/* debug replacements for the PyMem_* memory API */
-void *
-_PyMem_DebugMalloc(size_t nbytes)
+static void *
+_PyMem_DebugMalloc(void *ctx, size_t nbytes)
 {
-    return _PyObject_DebugMallocApi(_PYMALLOC_MEM_ID, nbytes);
-}
-void *
-_PyMem_DebugRealloc(void *p, size_t nbytes)
-{
-    return _PyObject_DebugReallocApi(_PYMALLOC_MEM_ID, p, nbytes);
-}
-void
-_PyMem_DebugFree(void *p)
-{
-    _PyObject_DebugFreeApi(_PYMALLOC_MEM_ID, p);
-}
-
-/* debug replacements for the PyObject_* memory API */
-void *
-_PyObject_DebugMalloc(size_t nbytes)
-{
-    return _PyObject_DebugMallocApi(_PYMALLOC_OBJ_ID, nbytes);
-}
-void *
-_PyObject_DebugRealloc(void *p, size_t nbytes)
-{
-    return _PyObject_DebugReallocApi(_PYMALLOC_OBJ_ID, p, nbytes);
-}
-void
-_PyObject_DebugFree(void *p)
-{
-    _PyObject_DebugFreeApi(_PYMALLOC_OBJ_ID, p);
-}
-void
-_PyObject_DebugCheckAddress(const void *p)
-{
-    _PyObject_DebugCheckAddressApi(_PYMALLOC_OBJ_ID, p);
-}
-
-
-/* generic debug memory api, with an "id" to identify the API in use */
-void *
-_PyObject_DebugMallocApi(char id, size_t nbytes)
-{
+    debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
     uchar *p;           /* base address of malloc'ed block */
     uchar *tail;        /* p + 2*SST + nbytes == pointer to tail pad bytes */
     size_t total;       /* nbytes + 4*SST */
@@ -1492,14 +1717,14 @@
         /* overflow:  can't represent total as a size_t */
         return NULL;
 
-    p = (uchar *)PyObject_Malloc(total);
+    p = (uchar *)api->alloc.malloc(api->alloc.ctx, total);
     if (p == NULL)
         return NULL;
 
     /* at p, write size (SST bytes), id (1 byte), pad (SST-1 bytes) */
     write_size_t(p, nbytes);
-    p[SST] = (uchar)id;
-    memset(p + SST + 1 , FORBIDDENBYTE, SST-1);
+    p[SST] = (uchar)api->api_id;
+    memset(p + SST + 1, FORBIDDENBYTE, SST-1);
 
     if (nbytes > 0)
         memset(p + 2*SST, CLEANBYTE, nbytes);
@@ -1517,25 +1742,27 @@
    Then fills the original bytes with DEADBYTE.
    Then calls the underlying free.
 */
-void
-_PyObject_DebugFreeApi(char api, void *p)
+static void
+_PyMem_DebugFree(void *ctx, void *p)
 {
+    debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
     uchar *q = (uchar *)p - 2*SST;  /* address returned from malloc */
     size_t nbytes;
 
     if (p == NULL)
         return;
-    _PyObject_DebugCheckAddressApi(api, p);
+    _PyMem_DebugCheckAddress(api->api_id, p);
     nbytes = read_size_t(q);
     nbytes += 4*SST;
     if (nbytes > 0)
         memset(q, DEADBYTE, nbytes);
-    PyObject_Free(q);
+    api->alloc.free(api->alloc.ctx, q);
 }
 
-void *
-_PyObject_DebugReallocApi(char api, void *p, size_t nbytes)
+static void *
+_PyMem_DebugRealloc(void *ctx, void *p, size_t nbytes)
 {
+    debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
     uchar *q = (uchar *)p;
     uchar *tail;
     size_t total;       /* nbytes + 4*SST */
@@ -1543,9 +1770,9 @@
     int i;
 
     if (p == NULL)
-        return _PyObject_DebugMallocApi(api, nbytes);
+        return _PyMem_DebugMalloc(ctx, nbytes);
 
-    _PyObject_DebugCheckAddressApi(api, p);
+    _PyMem_DebugCheckAddress(api->api_id, p);
     bumpserialno();
     original_nbytes = read_size_t(q - 2*SST);
     total = nbytes + 4*SST;
@@ -1562,12 +1789,12 @@
      * case we didn't get the chance to mark the old memory with DEADBYTE,
      * but we live with that.
      */
-    q = (uchar *)PyObject_Realloc(q - 2*SST, total);
+    q = (uchar *)api->alloc.realloc(api->alloc.ctx, q - 2*SST, total);
     if (q == NULL)
         return NULL;
 
     write_size_t(q, nbytes);
-    assert(q[SST] == (uchar)api);
+    assert(q[SST] == (uchar)api->api_id);
     for (i = 1; i < SST; ++i)
         assert(q[SST + i] == FORBIDDENBYTE);
     q += 2*SST;
@@ -1589,8 +1816,8 @@
  * and call Py_FatalError to kill the program.
  * The API id, is also checked.
  */
- void
-_PyObject_DebugCheckAddressApi(char api, const void *p)
+static void
+_PyMem_DebugCheckAddress(char api, const void *p)
 {
     const uchar *q = (const uchar *)p;
     char msgbuf[64];
@@ -1642,7 +1869,7 @@
 }
 
 /* Display info to stderr about the memory block at p. */
-void
+static void
 _PyObject_DebugDumpAddress(const void *p)
 {
     const uchar *q = (const uchar *)p;

-- 
Repository URL: http://hg.python.org/cpython

From brett at python.org  Sat Jun 15 00:57:01 2013
From: brett at python.org (Brett Cannon)
Date: Fri, 14 Jun 2013 18:57:01 -0400
Subject: [Python-checkins] cpython: Closes issue 17947. Adds PEP-0435
 (Enum, IntEnum) to the stdlib.
In-Reply-To: <3bWtm22z9dz7Ljj@mail.python.org>
References: <3bWtm22z9dz7Ljj@mail.python.org>
Message-ID: <CAP1=2W7EtxgQDrmjoKjY8kM649xQ8NYrBSfZFOaFaXSkcCK4Qw@mail.gmail.com>

Ethan, did you forget to run ``hg add`` before committing? If not then why
the heck did we argue over enums for so long if this was all it took to
make everyone happy? =)


On Fri, Jun 14, 2013 at 3:31 AM, ethan.furman <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/fae92309c3be
> changeset:   84117:fae92309c3be
> parent:      84115:af27c661d4fb
> user:        Ethan Furman <ethan at stoneleaf.us>
> date:        Fri Jun 14 00:30:27 2013 -0700
> summary:
>   Closes issue 17947.  Adds PEP-0435 (Enum, IntEnum) to the stdlib.
>
> files:
>   Doc/library/datatypes.rst |  1 +
>   1 files changed, 1 insertions(+), 0 deletions(-)
>
>
> diff --git a/Doc/library/datatypes.rst b/Doc/library/datatypes.rst
> --- a/Doc/library/datatypes.rst
> +++ b/Doc/library/datatypes.rst
> @@ -30,3 +30,4 @@
>     copy.rst
>     pprint.rst
>     reprlib.rst
> +   enum.rst
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130614/f5ad9abd/attachment.html>

From python-checkins at python.org  Sat Jun 15 01:02:43 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 01:02:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318192=3A_Introduc?=
 =?utf-8?q?e_importlib=2Eutil=2EMAGIC=5FNUMBER_and_document_the?=
Message-ID: <3bXHQq4Bbgz7Ljs@mail.python.org>

http://hg.python.org/cpython/rev/5619bc2d8207
changeset:   84128:5619bc2d8207
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 19:02:34 2013 -0400
summary:
  Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the
deprecation of imp.get_magic().

files:
  Doc/library/imp.rst                  |   3 +++
  Doc/library/importlib.rst            |   7 +++++++
  Lib/imp.py                           |   8 ++++++--
  Lib/importlib/_bootstrap.py          |   8 ++++----
  Lib/importlib/util.py                |   1 +
  Lib/test/test_importlib/test_util.py |  11 +++++++++++
  Misc/NEWS                            |   3 +++
  Python/importlib.h                   |   4 ++--
  8 files changed, 37 insertions(+), 8 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -21,6 +21,9 @@
    Return the magic string value used to recognize byte-compiled code files
    (:file:`.pyc` files).  (This value may be different for each Python version.)
 
+   .. deprecated:: 3.4
+       Use :attr:`importlib.util.MAGIC_NUMBER` instead.
+
 
 .. function:: get_suffixes()
 
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -879,6 +879,13 @@
 This module contains the various objects that help in the construction of
 an :term:`importer`.
 
+.. attribute:: MAGIC_NUMBER
+
+   The bytes which represent the bytecode version number. If you need help with
+   loading/writing bytecode then consider :class:`importlib.abc.SourceLoader`.
+
+   .. versionadded:: 3.4
+
 .. function:: resolve_name(name, package)
 
    Resolve a relative module name to an absolute one.
diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -23,6 +23,7 @@
 
 from importlib import _bootstrap
 from importlib import machinery
+from importlib import util
 import importlib
 import os
 import sys
@@ -44,8 +45,11 @@
 
 
 def get_magic():
-    """Return the magic number for .pyc or .pyo files."""
-    return _bootstrap._MAGIC_BYTES
+    """**DEPRECATED**
+
+    Return the magic number for .pyc or .pyo files.
+    """
+    return util.MAGIC_NUMBER
 
 
 def get_tag():
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -383,8 +383,8 @@
 # longer be understood by older implementations of the eval loop (usually
 # due to the addition of new opcodes).
 
-_MAGIC_BYTES = (3280).to_bytes(2, 'little') + b'\r\n'
-_RAW_MAGIC_NUMBER = int.from_bytes(_MAGIC_BYTES, 'little')  # For import.c
+MAGIC_NUMBER = (3280).to_bytes(2, 'little') + b'\r\n'
+_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little')  # For import.c
 
 _PYCACHE = '__pycache__'
 
@@ -663,7 +663,7 @@
     magic = data[:4]
     raw_timestamp = data[4:8]
     raw_size = data[8:12]
-    if magic != _MAGIC_BYTES:
+    if magic != MAGIC_NUMBER:
         message = 'bad magic number in {!r}: {!r}'.format(name, magic)
         _verbose_message(message)
         raise ImportError(message, **exc_details)
@@ -711,7 +711,7 @@
 def _code_to_bytecode(code, mtime=0, source_size=0):
     """Compile a code object into bytecode for writing out to a byte-compiled
     file."""
-    data = bytearray(_MAGIC_BYTES)
+    data = bytearray(MAGIC_NUMBER)
     data.extend(_w_long(mtime))
     data.extend(_w_long(source_size))
     data.extend(marshal.dumps(code))
diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py
--- a/Lib/importlib/util.py
+++ b/Lib/importlib/util.py
@@ -1,5 +1,6 @@
 """Utility code for constructing importers, etc."""
 
+from ._bootstrap import MAGIC_NUMBER
 from ._bootstrap import module_to_load
 from ._bootstrap import set_loader
 from ._bootstrap import set_package
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -313,5 +313,16 @@
             util.resolve_name('..bacon', 'spam')
 
 
+class MagicNumberTests(unittest.TestCase):
+
+    def test_length(self):
+        # Should be 4 bytes.
+        self.assertEqual(len(util.MAGIC_NUMBER), 4)
+
+    def test_incorporates_rn(self):
+        # The magic number uses \r\n to come out wrong when splitting on lines.
+        self.assertTrue(util.MAGIC_NUMBER.endswith(b'\r\n'))
+
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document as deprecated
+  imp.get_magic().
+
 - Issue #18149: Add filecmp.clear_cache() to manually clear the filecmp cache.
   Patch by Mark Levitt
 
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 01:20:06 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 01:20:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317907=3A_Document?=
 =?utf-8?q?_types=2EModuleType=27s_constructor_and_attributes=2C?=
Message-ID: <3bXHpt3K6Kz7Ljs@mail.python.org>

http://hg.python.org/cpython/rev/be50f1403f4d
changeset:   84129:be50f1403f4d
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 19:19:57 2013 -0400
summary:
  Issue #17907: Document types.ModuleType's constructor and attributes,
allowing for documenting imp.new_module() as deprecated.

files:
  Doc/library/imp.rst   |   3 +++
  Doc/library/types.rst |  30 ++++++++++++++++++++++++++++--
  Misc/NEWS             |   3 +++
  3 files changed, 34 insertions(+), 2 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -112,6 +112,9 @@
    Return a new empty module object called *name*.  This object is *not* inserted
    in ``sys.modules``.
 
+   .. deprecated:: 3.4
+      Use :class:`types.ModuleType` instead.
+
 
 .. function:: reload(module)
 
diff --git a/Doc/library/types.rst b/Doc/library/types.rst
--- a/Doc/library/types.rst
+++ b/Doc/library/types.rst
@@ -107,9 +107,35 @@
    C".)
 
 
-.. data:: ModuleType
+.. class:: ModuleType(name, doc=None)
 
-   The type of modules.
+   The type of :term:`modules <module>`. Constructor takes the name of the
+   module to be created and optionally its :term:`docstring`.
+
+   .. attribute:: __doc__
+
+      The :term:`docstring` of the module. Defaults to ``None``.
+
+   .. attribute:: __loader__
+
+      The :term:`loader` which loaded the module. Defaults to ``None``.
+
+      .. versionchanged:: 3.4
+         Defaults to ``None``. Previously the attribute was optional.
+
+   .. attribute:: __name__
+
+      The name of the module.
+
+   .. attribute:: __package__
+
+      Which :term:`package` a module belongs to. If the module is top-level
+      (i.e. not a part of any specific package) then the attribute should be set
+      to ``''``, else it should be set to the name of the package (which can be
+      :attr:`__name__` if the module is a package itself). Defaults to ``None``.
+
+      .. versionchanged:: 3.4
+         Defaults to ``None``. Previously the attribute was optional.
 
 
 .. data:: TracebackType
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- Issue #17907: Document imp.new_module() as deprecated in favour of
+  types.ModuleType.
+
 - Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document as deprecated
   imp.get_magic().
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 01:56:12 2013
From: python-checkins at python.org (ethan.furman)
Date: Sat, 15 Jun 2013 01:56:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_issue_17947=2E__Add?=
 =?utf-8?q?s_PEP-0435_=28Adding_an_Enum_type_to_the_Python_standard?=
Message-ID: <3bXJcX1qwVz7Lkc@mail.python.org>

http://hg.python.org/cpython/rev/e7a01c7f69fe
changeset:   84130:e7a01c7f69fe
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Fri Jun 14 16:55:46 2013 -0700
summary:
  Closes issue 17947.  Adds PEP-0435 (Adding an Enum type to the Python standard library).

Missing files added.
News entry added.

files:
  Doc/library/enum.rst  |  542 +++++++++++++++++
  Lib/enum.py           |  465 +++++++++++++++
  Lib/test/test_enum.py |  921 ++++++++++++++++++++++++++++++
  Misc/NEWS             |    2 +
  4 files changed, 1930 insertions(+), 0 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
new file mode 100644
--- /dev/null
+++ b/Doc/library/enum.rst
@@ -0,0 +1,542 @@
+:mod:`enum` --- Support for enumerations
+========================================
+
+.. module:: enum
+.. :synopsis: enumerations are sets of symbolic names bound to unique, constant
+  values.
+.. :moduleauthor:: Ethan Furman <ethan at stoneleaf.us>
+.. :sectionauthor:: Barry Warsaw <barry at python.org>,
+.. :sectionauthor:: Eli Bendersky <eliben at gmail.com>,
+.. :sectionauthor:: Ethan Furman <ethan at stoneleaf.us>
+
+**Source code:** :source:`Lib/enum.py`
+
+----------------
+
+An enumeration is a set of symbolic names (members) bound to unique, constant
+values.  Within an enumeration, the members can be compared by identity, and
+the enumeration itself can be iterated over.
+
+This module defines two enumeration classes that can be used to define unique
+sets of names and values: :class:`Enum` and :class:`IntEnum`.
+
+Creating an Enum
+----------------
+
+Enumerations are created using the :keyword:`class` syntax, which makes them
+easy to read and write.  An alternative creation method is described in
+`Functional API`_.  To define an enumeration, subclass :class:`Enum` as
+follows::
+
+    >>> from enum import Enum
+    >>> class Color(Enum):
+    ...     red = 1
+    ...     green = 2
+    ...     blue = 3
+
+**A note on nomenclature**: we call :class:`Color` an *enumeration* (or *enum*)
+and :attr:`Color.red`, :attr:`Color.green` are *enumeration members* (or
+*enum members*).  Enumeration members also have *values* (the value of
+:attr:`Color.red` is ``1``, etc.)
+
+Enumeration members have human readable string representations::
+
+    >>> print(Color.red)
+    Color.red
+
+...while their ``repr`` has more information::
+
+    >>> print(repr(Color.red))
+    <Color.red: 1>
+
+The *type* of an enumeration member is the enumeration it belongs to::
+
+    >>> type(Color.red)
+    <enum 'Color'>
+    >>> isinstance(Color.green, Color)
+    True
+    >>>
+
+Enum members also have a property that contains just their item name::
+
+    >>> print(Color.red.name)
+    red
+
+Enumerations support iteration, in definition order::
+
+    >>> class Shake(Enum):
+    ...   vanilla = 7
+    ...   chocolate = 4
+    ...   cookies = 9
+    ...   mint = 3
+    ...
+    >>> for shake in Shake:
+    ...   print(shake)
+    ...
+    Shake.vanilla
+    Shake.chocolate
+    Shake.cookies
+    Shake.mint
+
+Enumeration members are hashable, so they can be used in dictionaries and sets::
+
+    >>> apples = {}
+    >>> apples[Color.red] = 'red delicious'
+    >>> apples[Color.green] = 'granny smith'
+    >>> apples == {Color.red: 'red delicious', Color.green: 'granny smith'}
+    True
+
+
+Programmatic access to enumeration members
+------------------------------------------
+
+Sometimes it's useful to access members in enumerations programmatically (i.e.
+situations where ``Color.red`` won't do because the exact color is not known
+at program-writing time).  ``Enum`` allows such access::
+
+    >>> Color(1)
+    <Color.red: 1>
+    >>> Color(3)
+    <Color.blue: 3>
+
+If you want to access enum members by *name*, use item access::
+
+    >>> Color['red']
+    <Color.red: 1>
+    >>> Color['green']
+    <Color.green: 2>
+
+
+Duplicating enum members and values
+-----------------------------------
+
+Having two enum members with the same name is invalid::
+
+    >>> class Shape(Enum):
+    ...   square = 2
+    ...   square = 3
+    ...
+    Traceback (most recent call last):
+    ...
+    TypeError: Attempted to reuse key: 'square'
+
+However, two enum members are allowed to have the same value.  Given two members
+A and B with the same value (and A defined first), B is an alias to A.  By-value
+lookup of the value of A and B will return A.  By-name lookup of B will also
+return A::
+
+    >>> class Shape(Enum):
+    ...   square = 2
+    ...   diamond = 1
+    ...   circle = 3
+    ...   alias_for_square = 2
+    ...
+    >>> Shape.square
+    <Shape.square: 2>
+    >>> Shape.alias_for_square
+    <Shape.square: 2>
+    >>> Shape(2)
+    <Shape.square: 2>
+
+Iterating over the members of an enum does not provide the aliases::
+
+    >>> list(Shape)
+    [<Shape.square: 2>, <Shape.diamond: 1>, <Shape.circle: 3>]
+
+The special attribute ``__members__`` is an ordered dictionary mapping names
+to members.  It includes all names defined in the enumeration, including the
+aliases::
+
+    >>> for name, member in Shape.__members__.items():
+    ...   name, member
+    ...
+    ('square', <Shape.square: 2>)
+    ('diamond', <Shape.diamond: 1>)
+    ('circle', <Shape.circle: 3>)
+    ('alias_for_square', <Shape.square: 2>)
+
+The ``__members__`` attribute can be used for detailed programmatic access to
+the enumeration members.  For example, finding all the aliases::
+
+    >>> [name for name, member in Shape.__members__.items() if member.name != name]
+    ['alias_for_square']
+
+Comparisons
+-----------
+
+Enumeration members are compared by identity::
+
+    >>> Color.red is Color.red
+    True
+    >>> Color.red is Color.blue
+    False
+    >>> Color.red is not Color.blue
+    True
+
+Ordered comparisons between enumeration values are *not* supported.  Enum
+members are not integers (but see `IntEnum`_ below)::
+
+    >>> Color.red < Color.blue
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in <module>
+    TypeError: unorderable types: Color() < Color()
+
+Equality comparisons are defined though::
+
+    >>> Color.blue == Color.red
+    False
+    >>> Color.blue != Color.red
+    True
+    >>> Color.blue == Color.blue
+    True
+
+Comparisons against non-enumeration values will always compare not equal
+(again, class:`IntEnum` was explicitly designed to behave differently, see
+below)::
+
+    >>> Color.blue == 2
+    False
+
+
+Allowed members and attributes of enumerations
+----------------------------------------------
+
+The examples above use integers for enumeration values.  Using integers is
+short and handy (and provided by default by the `Functional API`_), but not
+strictly enforced.  In the vast majority of use-cases, one doesn't care what
+the actual value of an enumeration is.  But if the value *is* important,
+enumerations can have arbitrary values.
+
+Enumerations are Python classes, and can have methods and special methods as
+usual.  If we have this enumeration::
+
+    >>> class Mood(Enum):
+    ...   funky = 1
+    ...   happy = 3
+    ...
+    ...   def describe(self):
+    ...     # self is the member here
+    ...     return self.name, self.value
+    ...
+    ...   def __str__(self):
+    ...     return 'my custom str! {0}'.format(self.value)
+    ...
+    ...   @classmethod
+    ...   def favorite_mood(cls):
+    ...     # cls here is the enumeration
+    ...     return cls.happy
+
+Then::
+
+    >>> Mood.favorite_mood()
+    <Mood.happy: 3>
+    >>> Mood.happy.describe()
+    ('happy', 3)
+    >>> str(Mood.funky)
+    'my custom str! 1'
+
+The rules for what is allowed are as follows: _sunder_ names (starting and
+ending with a single underscore) are reserved by enum and cannot be used;
+all other attributes defined within an enumeration will become members of this
+enumeration, with the exception of *__dunder__* names and descriptors (methods
+are also descriptors).
+
+Note:  if your enumeration defines :meth:`__new__` and/or :meth:`__init__` then
+whatever value(s) were given to the enum member will be passed into those
+methods.  See `Planet`_ for an example.
+
+
+Restricted subclassing of enumerations
+--------------------------------------
+
+Subclassing an enumeration is allowed only if the enumeration does not define
+any members.  So this is forbidden::
+
+    >>> class MoreColor(Color):
+    ...   pink = 17
+    Traceback (most recent call last):
+    ...
+    TypeError: Cannot extend enumerations
+
+But this is allowed::
+
+    >>> class Foo(Enum):
+    ...   def some_behavior(self):
+    ...     pass
+    ...
+    >>> class Bar(Foo):
+    ...   happy = 1
+    ...   sad = 2
+    ...
+
+Allowing subclassing of enums that define members would lead to a violation of
+some important invariants of types and instances.  On the other hand, it makes
+sense to allow sharing some common behavior between a group of enumerations.
+(See `OrderedEnum`_ for an example.)
+
+
+Pickling
+--------
+
+Enumerations can be pickled and unpickled::
+
+    >>> from test.test_enum import Fruit
+    >>> from pickle import dumps, loads
+    >>> Fruit.tomato is loads(dumps(Fruit.tomato))
+    True
+
+The usual restrictions for pickling apply: picklable enums must be defined in
+the top level of a module, since unpickling requires them to be importable
+from that module.
+
+.. warning::
+
+    In order to support the singleton nature of enumeration members, pickle
+    protocol version 2 or higher must be used.
+
+
+Functional API
+--------------
+
+The :class:`Enum` class is callable, providing the following functional API::
+
+    >>> Animal = Enum('Animal', 'ant bee cat dog')
+    >>> Animal
+    <enum 'Animal'>
+    >>> Animal.ant
+    <Animal.ant: 1>
+    >>> Animal.ant.value
+    1
+    >>> list(Animal)
+    [<Animal.ant: 1>, <Animal.bee: 2>, <Animal.cat: 3>, <Animal.dog: 4>]
+
+The semantics of this API resemble :class:`namedtuple`. The first argument
+of the call to :class:`Enum` is the name of the enumeration.
+
+The second argument is the *source* of enumeration member names.  It can be a
+whitespace-separated string of names, a sequence of names, a sequence of
+2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to
+values.  The last two options enable assigning arbitrary values to
+enumerations; the others auto-assign increasing integers starting with 1.  A
+new class derived from :class:`Enum` is returned.  In other words, the above
+assignment to :class:`Animal` is equivalent to::
+
+    >>> class Animals(Enum):
+    ...   ant = 1
+    ...   bee = 2
+    ...   cat = 3
+    ...   dog = 4
+
+Pickling enums created with the functional API can be tricky as frame stack
+implementation details are used to try and figure out which module the
+enumeration is being created in (e.g. it will fail if you use a utility
+function in separate module, and also may not work on IronPython or Jython).
+The solution is to specify the module name explicitly as follows::
+
+    >>> Animals = Enum('Animals', 'ant bee cat dog', module=__name__)
+
+Derived Enumerations
+====================
+
+IntEnum
+-------
+
+A variation of :class:`Enum` is provided which is also a subclass of
+:class:`int`.  Members of an :class:`IntEnum` can be compared to integers;
+by extension, integer enumerations of different types can also be compared
+to each other::
+
+    >>> from enum import IntEnum
+    >>> class Shape(IntEnum):
+    ...   circle = 1
+    ...   square = 2
+    ...
+    >>> class Request(IntEnum):
+    ...   post = 1
+    ...   get = 2
+    ...
+    >>> Shape == 1
+    False
+    >>> Shape.circle == 1
+    True
+    >>> Shape.circle == Request.post
+    True
+
+However, they still can't be compared to standard :class:`Enum` enumerations::
+
+    >>> class Shape(IntEnum):
+    ...   circle = 1
+    ...   square = 2
+    ...
+    >>> class Color(Enum):
+    ...   red = 1
+    ...   green = 2
+    ...
+    >>> Shape.circle == Color.red
+    False
+
+:class:`IntEnum` values behave like integers in other ways you'd expect::
+
+    >>> int(Shape.circle)
+    1
+    >>> ['a', 'b', 'c'][Shape.circle]
+    'b'
+    >>> [i for i in range(Shape.square)]
+    [0, 1]
+
+For the vast majority of code, :class:`Enum` is strongly recommended,
+since :class:`IntEnum` breaks some semantic promises of an enumeration (by
+being comparable to integers, and thus by transitivity to other
+unrelated enumerations).  It should be used only in special cases where
+there's no other choice; for example, when integer constants are
+replaced with enumerations and backwards compatibility is required with code
+that still expects integers.
+
+
+Others
+------
+
+While :class:`IntEnum` is part of the :mod:`enum` module, it would be very
+simple to implement independently::
+
+    class IntEnum(int, Enum):
+        pass
+
+This demonstrates how similar derived enumerations can be defined; for example
+a :class:`StrEnum` that mixes in :class:`str` instead of :class:`int`.
+
+Some rules:
+
+1. When subclassing :class:`Enum`, mix-in types must appear before
+   :class:`Enum` itself in the sequence of bases, as in the :class:`IntEnum`
+   example above.
+2. While :class:`Enum` can have members of any type, once you mix in an
+   additional type, all the members must have values of that type, e.g.
+   :class:`int` above.  This restriction does not apply to mix-ins which only
+   add methods and don't specify another data type such as :class:`int` or
+   :class:`str`.
+3. When another data type is mixed in, the :attr:`value` attribute is *not the
+   same* as the enum member itself, although it is equivalant and will compare
+   equal.
+
+
+Interesting examples
+====================
+
+While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of
+use-cases, they cannot cover them all.  Here are recipes for some different
+types of enumerations that can be used directly, or as examples for creating
+one's own.
+
+
+AutoNumber
+----------
+
+Avoids having to specify the value for each enumeration member::
+
+    >>> class AutoNumber(Enum):
+    ...     def __new__(cls):
+    ...         value = len(cls.__members__) + 1
+    ...         obj = object.__new__(cls)
+    ...         obj._value = value
+    ...         return obj
+    ...
+    >>> class Color(AutoNumber):
+    ...     red = ()
+    ...     green = ()
+    ...     blue = ()
+    ...
+    >>> Color.green.value == 2
+    True
+
+
+UniqueEnum
+----------
+
+Raises an error if a duplicate member name is found instead of creating an
+alias::
+
+    >>> class UniqueEnum(Enum):
+    ...     def __init__(self, *args):
+    ...         cls = self.__class__
+    ...         if any(self.value == e.value for e in cls):
+    ...             a = self.name
+    ...             e = cls(self.value).name
+    ...             raise ValueError(
+    ...                     "aliases not allowed in UniqueEnum:  %r --> %r"
+    ...                     % (a, e))
+    ...
+    >>> class Color(UniqueEnum):
+    ...     red = 1
+    ...     green = 2
+    ...     blue = 3
+    ...     grene = 2
+    Traceback (most recent call last):
+    ...
+    ValueError: aliases not allowed in UniqueEnum:  'grene' --> 'green'
+
+
+OrderedEnum
+-----------
+
+An ordered enumeration that is not based on :class:`IntEnum` and so maintains
+the normal :class:`Enum` invariants (such as not being comparable to other
+enumerations)::
+
+    >>> class OrderedEnum(Enum):
+    ...     def __ge__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value >= other._value
+    ...         return NotImplemented
+    ...     def __gt__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value > other._value
+    ...         return NotImplemented
+    ...     def __le__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value <= other._value
+    ...         return NotImplemented
+    ...     def __lt__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value < other._value
+    ...         return NotImplemented
+    ...
+    >>> class Grade(OrderedEnum):
+    ...     A = 5
+    ...     B = 4
+    ...     C = 3
+    ...     D = 2
+    ...     F = 1
+    ...
+    >>> Grade.C < Grade.A
+    True
+
+
+Planet
+------
+
+If :meth:`__new__` or :meth:`__init__` is defined the value of the enum member
+will be passed to those methods::
+
+    >>> class Planet(Enum):
+    ...     MERCURY = (3.303e+23, 2.4397e6)
+    ...     VENUS   = (4.869e+24, 6.0518e6)
+    ...     EARTH   = (5.976e+24, 6.37814e6)
+    ...     MARS    = (6.421e+23, 3.3972e6)
+    ...     JUPITER = (1.9e+27,   7.1492e7)
+    ...     SATURN  = (5.688e+26, 6.0268e7)
+    ...     URANUS  = (8.686e+25, 2.5559e7)
+    ...     NEPTUNE = (1.024e+26, 2.4746e7)
+    ...     def __init__(self, mass, radius):
+    ...         self.mass = mass       # in kilograms
+    ...         self.radius = radius   # in meters
+    ...     @property
+    ...     def surface_gravity(self):
+    ...         # universal gravitational constant  (m3 kg-1 s-2)
+    ...         G = 6.67300E-11
+    ...         return G * self.mass / (self.radius * self.radius)
+    ...
+    >>> Planet.EARTH.value
+    (5.976e+24, 6378140.0)
+    >>> Planet.EARTH.surface_gravity
+    9.802652743337129
diff --git a/Lib/enum.py b/Lib/enum.py
new file mode 100644
--- /dev/null
+++ b/Lib/enum.py
@@ -0,0 +1,465 @@
+"""Python Enumerations"""
+
+import sys
+from collections import OrderedDict
+from types import MappingProxyType
+
+__all__ = ['Enum', 'IntEnum']
+
+
+class _RouteClassAttributeToGetattr:
+    """Route attribute access on a class to __getattr__.
+
+    This is a descriptor, used to define attributes that act differently when
+    accessed through an instance and through a class.  Instance access remains
+    normal, but access to an attribute through a class will be routed to the
+    class's __getattr__ method; this is done by raising AttributeError.
+
+    """
+    def __init__(self, fget=None):
+        self.fget = fget
+
+    def __get__(self, instance, ownerclass=None):
+        if instance is None:
+            raise AttributeError()
+        return self.fget(instance)
+
+    def __set__(self, instance, value):
+        raise AttributeError("can't set attribute")
+
+    def __delete__(self, instance):
+        raise AttributeError("can't delete attribute")
+
+
+def _is_dunder(name):
+    """Returns True if a __dunder__ name, False otherwise."""
+    return (name[:2] == name[-2:] == '__' and
+            name[2:3] != '_' and
+            name[-3:-2] != '_')
+
+
+def _is_sunder(name):
+    """Returns True if a _sunder_ name, False otherwise."""
+    return (name[0] == name[-1] == '_' and
+            name[1:2] != '_' and
+            name[-2:-1] != '_')
+
+
+def _make_class_unpicklable(cls):
+    """Make the given class un-picklable."""
+    def _break_on_call_reduce(self):
+        raise TypeError('%r cannot be pickled' % self)
+    cls.__reduce__ = _break_on_call_reduce
+    cls.__module__ = '<unknown>'
+
+
+class _EnumDict(dict):
+    """Keeps track of definition order of the enum items.
+
+    EnumMeta will use the names found in self._member_names as the
+    enumeration member names.
+
+    """
+    def __init__(self):
+        super().__init__()
+        self._member_names = []
+
+    def __setitem__(self, key, value):
+        """Changes anything not dundered or that doesn't have __get__.
+
+        If a descriptor is added with the same name as an enum member, the name
+        is removed from _member_names (this may leave a hole in the numerical
+        sequence of values).
+
+        If an enum member name is used twice, an error is raised; duplicate
+        values are not checked for.
+
+        Single underscore (sunder) names are reserved.
+
+        """
+        if _is_sunder(key):
+            raise ValueError('_names_ are reserved for future Enum use')
+        elif _is_dunder(key) or hasattr(value, '__get__'):
+            if key in self._member_names:
+                # overwriting an enum with a method?  then remove the name from
+                # _member_names or it will become an enum anyway when the class
+                # is created
+                self._member_names.remove(key)
+        else:
+            if key in self._member_names:
+                raise TypeError('Attempted to reuse key: %r' % key)
+            self._member_names.append(key)
+        super().__setitem__(key, value)
+
+
+# Dummy value for Enum as EnumMeta explicity checks for it, but of course until
+# EnumMeta finishes running the first time the Enum class doesn't exist.  This
+# is also why there are checks in EnumMeta like `if Enum is not None`
+Enum = None
+
+
+class EnumMeta(type):
+    """Metaclass for Enum"""
+    @classmethod
+    def __prepare__(metacls, cls, bases):
+        return _EnumDict()
+
+    def __new__(metacls, cls, bases, classdict):
+        # an Enum class is final once enumeration items have been defined; it
+        # cannot be mixed with other types (int, float, etc.) if it has an
+        # inherited __new__ unless a new __new__ is defined (or the resulting
+        # class will fail).
+        member_type, first_enum = metacls._get_mixins_(bases)
+        __new__, save_new, use_args = metacls._find_new_(classdict, member_type,
+                                                        first_enum)
+
+        # save enum items into separate mapping so they don't get baked into
+        # the new class
+        members = {k: classdict[k] for k in classdict._member_names}
+        for name in classdict._member_names:
+            del classdict[name]
+
+        # check for illegal enum names (any others?)
+        invalid_names = set(members) & {'mro', }
+        if invalid_names:
+            raise ValueError('Invalid enum member name: {0}'.format(
+                ','.join(invalid_names)))
+
+        # create our new Enum type
+        enum_class = super().__new__(metacls, cls, bases, classdict)
+        enum_class._member_names = []               # names in definition order
+        enum_class._member_map = OrderedDict()      # name->value map
+
+        # Reverse value->name map for hashable values.
+        enum_class._value2member_map = {}
+
+        # check for a __getnewargs__, and if not present sabotage
+        # pickling, since it won't work anyway
+        if (member_type is not object and
+            member_type.__dict__.get('__getnewargs__') is None
+            ):
+            _make_class_unpicklable(enum_class)
+
+        # instantiate them, checking for duplicates as we go
+        # we instantiate first instead of checking for duplicates first in case
+        # a custom __new__ is doing something funky with the values -- such as
+        # auto-numbering ;)
+        for member_name in classdict._member_names:
+            value = members[member_name]
+            if not isinstance(value, tuple):
+                args = (value, )
+            else:
+                args = value
+            if member_type is tuple:   # special case for tuple enums
+                args = (args, )     # wrap it one more time
+            if not use_args:
+                enum_member = __new__(enum_class)
+                enum_member._value = value
+            else:
+                enum_member = __new__(enum_class, *args)
+                if not hasattr(enum_member, '_value'):
+                    enum_member._value = member_type(*args)
+            enum_member._member_type = member_type
+            enum_member._name = member_name
+            enum_member.__init__(*args)
+            # If another member with the same value was already defined, the
+            # new member becomes an alias to the existing one.
+            for name, canonical_member in enum_class._member_map.items():
+                if canonical_member.value == enum_member._value:
+                    enum_member = canonical_member
+                    break
+            else:
+                # Aliases don't appear in member names (only in __members__).
+                enum_class._member_names.append(member_name)
+            enum_class._member_map[member_name] = enum_member
+            try:
+                # This may fail if value is not hashable. We can't add the value
+                # to the map, and by-value lookups for this value will be
+                # linear.
+                enum_class._value2member_map[value] = enum_member
+            except TypeError:
+                pass
+
+        # double check that repr and friends are not the mixin's or various
+        # things break (such as pickle)
+        for name in ('__repr__', '__str__', '__getnewargs__'):
+            class_method = getattr(enum_class, name)
+            obj_method = getattr(member_type, name, None)
+            enum_method = getattr(first_enum, name, None)
+            if obj_method is not None and obj_method is class_method:
+                setattr(enum_class, name, enum_method)
+
+        # replace any other __new__ with our own (as long as Enum is not None,
+        # anyway) -- again, this is to support pickle
+        if Enum is not None:
+            # if the user defined their own __new__, save it before it gets
+            # clobbered in case they subclass later
+            if save_new:
+                enum_class.__new_member__ = __new__
+            enum_class.__new__ = Enum.__new__
+        return enum_class
+
+    def __call__(cls, value, names=None, *, module=None, type=None):
+        """Either returns an existing member, or creates a new enum class.
+
+        This method is used both when an enum class is given a value to match
+        to an enumeration member (i.e. Color(3)) and for the functional API
+        (i.e. Color = Enum('Color', names='red green blue')).
+
+        When used for the functional API: `module`, if set, will be stored in
+        the new class' __module__ attribute; `type`, if set, will be mixed in
+        as the first base class.
+
+        Note: if `module` is not set this routine will attempt to discover the
+        calling module by walking the frame stack; if this is unsuccessful
+        the resulting class will not be pickleable.
+
+        """
+        if names is None:  # simple value lookup
+            return cls.__new__(cls, value)
+        # otherwise, functional API: we're creating a new Enum type
+        return cls._create_(value, names, module=module, type=type)
+
+    def __contains__(cls, member):
+        return isinstance(member, cls) and member.name in cls._member_map
+
+    def __dir__(self):
+        return ['__class__', '__doc__', '__members__'] + self._member_names
+
+    @property
+    def __members__(cls):
+        """Returns a mapping of member name->value.
+
+        This mapping lists all enum members, including aliases. Note that this
+        is a read-only view of the internal mapping.
+
+        """
+        return MappingProxyType(cls._member_map)
+
+    def __getattr__(cls, name):
+        """Return the enum member matching `name`
+
+        We use __getattr__ instead of descriptors or inserting into the enum
+        class' __dict__ in order to support `name` and `value` being both
+        properties for enum members (which live in the class' __dict__) and
+        enum members themselves.
+
+        """
+        if _is_dunder(name):
+            raise AttributeError(name)
+        try:
+            return cls._member_map[name]
+        except KeyError:
+            raise AttributeError(name) from None
+
+    def __getitem__(cls, name):
+        return cls._member_map[name]
+
+    def __iter__(cls):
+        return (cls._member_map[name] for name in cls._member_names)
+
+    def __len__(cls):
+        return len(cls._member_names)
+
+    def __repr__(cls):
+        return "<enum %r>" % cls.__name__
+
+    def _create_(cls, class_name, names=None, *, module=None, type=None):
+        """Convenience method to create a new Enum class.
+
+        `names` can be:
+
+        * A string containing member names, separated either with spaces or
+          commas.  Values are auto-numbered from 1.
+        * An iterable of member names.  Values are auto-numbered from 1.
+        * An iterable of (member name, value) pairs.
+        * A mapping of member name -> value.
+
+        """
+        metacls = cls.__class__
+        bases = (cls, ) if type is None else (type, cls)
+        classdict = metacls.__prepare__(class_name, bases)
+
+        # special processing needed for names?
+        if isinstance(names, str):
+            names = names.replace(',', ' ').split()
+        if isinstance(names, (tuple, list)) and isinstance(names[0], str):
+            names = [(e, i) for (i, e) in enumerate(names, 1)]
+
+        # Here, names is either an iterable of (name, value) or a mapping.
+        for item in names:
+            if isinstance(item, str):
+                member_name, member_value = item, names[item]
+            else:
+                member_name, member_value = item
+            classdict[member_name] = member_value
+        enum_class = metacls.__new__(metacls, class_name, bases, classdict)
+
+        # TODO: replace the frame hack if a blessed way to know the calling
+        # module is ever developed
+        if module is None:
+            try:
+                module = sys._getframe(2).f_globals['__name__']
+            except (AttributeError, ValueError) as exc:
+                pass
+        if module is None:
+            _make_class_unpicklable(enum_class)
+        else:
+            enum_class.__module__ = module
+
+        return enum_class
+
+    @staticmethod
+    def _get_mixins_(bases):
+        """Returns the type for creating enum members, and the first inherited
+        enum class.
+
+        bases: the tuple of bases that was given to __new__
+
+        """
+        if not bases:
+            return object, Enum
+
+        # double check that we are not subclassing a class with existing
+        # enumeration members; while we're at it, see if any other data
+        # type has been mixed in so we can use the correct __new__
+        member_type = first_enum = None
+        for base in bases:
+            if  (base is not Enum and
+                    issubclass(base, Enum) and
+                    base._member_names):
+                raise TypeError("Cannot extend enumerations")
+        # base is now the last base in bases
+        if not issubclass(base, Enum):
+            raise TypeError("new enumerations must be created as "
+                    "`ClassName([mixin_type,] enum_type)`")
+
+        # get correct mix-in type (either mix-in type of Enum subclass, or
+        # first base if last base is Enum)
+        if not issubclass(bases[0], Enum):
+            member_type = bases[0]     # first data type
+            first_enum = bases[-1]  # enum type
+        else:
+            for base in bases[0].__mro__:
+                # most common: (IntEnum, int, Enum, object)
+                # possible:    (<Enum 'AutoIntEnum'>, <Enum 'IntEnum'>,
+                #               <class 'int'>, <Enum 'Enum'>,
+                #               <class 'object'>)
+                if issubclass(base, Enum):
+                    if first_enum is None:
+                        first_enum = base
+                else:
+                    if member_type is None:
+                        member_type = base
+
+        return member_type, first_enum
+
+    @staticmethod
+    def _find_new_(classdict, member_type, first_enum):
+        """Returns the __new__ to be used for creating the enum members.
+
+        classdict: the class dictionary given to __new__
+        member_type: the data type whose __new__ will be used by default
+        first_enum: enumeration to check for an overriding __new__
+
+        """
+        # now find the correct __new__, checking to see of one was defined
+        # by the user; also check earlier enum classes in case a __new__ was
+        # saved as __new_member__
+        __new__ = classdict.get('__new__', None)
+
+        # should __new__ be saved as __new_member__ later?
+        save_new = __new__ is not None
+
+        if __new__ is None:
+            # check all possibles for __new_member__ before falling back to
+            # __new__
+            for method in ('__new_member__', '__new__'):
+                for possible in (member_type, first_enum):
+                    target = getattr(possible, method, None)
+                    if target not in {
+                            None,
+                            None.__new__,
+                            object.__new__,
+                            Enum.__new__,
+                            }:
+                        __new__ = target
+                        break
+                if __new__ is not None:
+                    break
+            else:
+                __new__ = object.__new__
+
+        # if a non-object.__new__ is used then whatever value/tuple was
+        # assigned to the enum member name will be passed to __new__ and to the
+        # new enum member's __init__
+        if __new__ is object.__new__:
+            use_args = False
+        else:
+            use_args = True
+
+        return __new__, save_new, use_args
+
+
+class Enum(metaclass=EnumMeta):
+    """Generic enumeration.
+
+    Derive from this class to define new enumerations.
+
+    """
+    def __new__(cls, value):
+        # all enum instances are actually created during class construction
+        # without calling this method; this method is called by the metaclass'
+        # __call__ (i.e. Color(3) ), and by pickle
+        if type(value) is cls:
+            # For lookups like Color(Color.red)
+            return value
+        # by-value search for a matching enum member
+        # see if it's in the reverse mapping (for hashable values)
+        if value in cls._value2member_map:
+            return cls._value2member_map[value]
+        # not there, now do long search -- O(n) behavior
+        for member in cls._member_map.values():
+            if member.value == value:
+                return member
+        raise ValueError("%s is not a valid %s" % (value, cls.__name__))
+
+    def __repr__(self):
+        return "<%s.%s: %r>" % (
+                self.__class__.__name__, self._name, self._value)
+
+    def __str__(self):
+        return "%s.%s" % (self.__class__.__name__, self._name)
+
+    def __dir__(self):
+        return (['__class__', '__doc__', 'name', 'value'])
+
+    def __eq__(self, other):
+        if type(other) is self.__class__:
+            return self is other
+        return NotImplemented
+
+    def __getnewargs__(self):
+        return (self._value, )
+
+    def __hash__(self):
+        return hash(self._name)
+
+    # _RouteClassAttributeToGetattr is used to provide access to the `name`
+    # and `value` properties of enum members while keeping some measure of
+    # protection from modification, while still allowing for an enumeration
+    # to have members named `name` and `value`.  This works because enumeration
+    # members are not set directly on the enum class -- __getattr__ is
+    # used to look them up.
+
+    @_RouteClassAttributeToGetattr
+    def name(self):
+        return self._name
+
+    @_RouteClassAttributeToGetattr
+    def value(self):
+        return self._value
+
+
+class IntEnum(int, Enum):
+    """Enum where members are also (and must be) ints"""
diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py
new file mode 100644
--- /dev/null
+++ b/Lib/test/test_enum.py
@@ -0,0 +1,921 @@
+import enum
+import unittest
+from collections import OrderedDict
+from pickle import dumps, loads, PicklingError
+from enum import Enum, IntEnum
+
+# for pickle tests
+try:
+    class Stooges(Enum):
+        LARRY = 1
+        CURLY = 2
+        MOE = 3
+except Exception as exc:
+    Stooges = exc
+
+try:
+    class IntStooges(int, Enum):
+        LARRY = 1
+        CURLY = 2
+        MOE = 3
+except Exception as exc:
+    IntStooges = exc
+
+try:
+    class FloatStooges(float, Enum):
+        LARRY = 1.39
+        CURLY = 2.72
+        MOE = 3.142596
+except Exception as exc:
+    FloatStooges = exc
+
+# for pickle test and subclass tests
+try:
+    class StrEnum(str, Enum):
+        'accepts only string values'
+    class Name(StrEnum):
+        BDFL = 'Guido van Rossum'
+        FLUFL = 'Barry Warsaw'
+except Exception as exc:
+    Name = exc
+
+try:
+    Question = Enum('Question', 'who what when where why', module=__name__)
+except Exception as exc:
+    Question = exc
+
+try:
+    Answer = Enum('Answer', 'him this then there because')
+except Exception as exc:
+    Answer = exc
+
+# for doctests
+try:
+    class Fruit(Enum):
+        tomato = 1
+        banana = 2
+        cherry = 3
+except Exception:
+    pass
+
+class TestEnum(unittest.TestCase):
+    def setUp(self):
+        class Season(Enum):
+            SPRING = 1
+            SUMMER = 2
+            AUTUMN = 3
+            WINTER = 4
+        self.Season = Season
+
+    def test_enum_in_enum_out(self):
+        Season = self.Season
+        self.assertIs(Season(Season.WINTER), Season.WINTER)
+
+    def test_enum_value(self):
+        Season = self.Season
+        self.assertEqual(Season.SPRING.value, 1)
+
+    def test_intenum_value(self):
+        self.assertEqual(IntStooges.CURLY.value, 2)
+
+    def test_dir_on_class(self):
+        Season = self.Season
+        self.assertEqual(
+            set(dir(Season)),
+            set(['__class__', '__doc__', '__members__',
+                'SPRING', 'SUMMER', 'AUTUMN', 'WINTER']),
+            )
+
+    def test_dir_on_item(self):
+        Season = self.Season
+        self.assertEqual(
+            set(dir(Season.WINTER)),
+            set(['__class__', '__doc__', 'name', 'value']),
+            )
+
+    def test_enum(self):
+        Season = self.Season
+        lst = list(Season)
+        self.assertEqual(len(lst), len(Season))
+        self.assertEqual(len(Season), 4, Season)
+        self.assertEqual(
+            [Season.SPRING, Season.SUMMER, Season.AUTUMN, Season.WINTER], lst)
+
+        for i, season in enumerate('SPRING SUMMER AUTUMN WINTER'.split(), 1):
+            e = Season(i)
+            self.assertEqual(e, getattr(Season, season))
+            self.assertEqual(e.value, i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, season)
+            self.assertIn(e, Season)
+            self.assertIs(type(e), Season)
+            self.assertIsInstance(e, Season)
+            self.assertEqual(str(e), 'Season.' + season)
+            self.assertEqual(
+                    repr(e),
+                    '<Season.{0}: {1}>'.format(season, i),
+                    )
+
+    def test_value_name(self):
+        Season = self.Season
+        self.assertEqual(Season.SPRING.name, 'SPRING')
+        self.assertEqual(Season.SPRING.value, 1)
+        with self.assertRaises(AttributeError):
+            Season.SPRING.name = 'invierno'
+        with self.assertRaises(AttributeError):
+            Season.SPRING.value = 2
+
+    def test_invalid_names(self):
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                mro = 9
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _create_= 11
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _get_mixins_ = 9
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _find_new_ = 1
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _any_name_ = 9
+
+    def test_contains(self):
+        Season = self.Season
+        self.assertIn(Season.AUTUMN, Season)
+        self.assertNotIn(3, Season)
+
+        val = Season(3)
+        self.assertIn(val, Season)
+
+        class OtherEnum(Enum):
+            one = 1; two = 2
+        self.assertNotIn(OtherEnum.two, Season)
+
+    def test_comparisons(self):
+        Season = self.Season
+        with self.assertRaises(TypeError):
+            Season.SPRING < Season.WINTER
+        with self.assertRaises(TypeError):
+            Season.SPRING > 4
+
+        self.assertNotEqual(Season.SPRING, 1)
+
+        class Part(Enum):
+            SPRING = 1
+            CLIP = 2
+            BARREL = 3
+
+        self.assertNotEqual(Season.SPRING, Part.SPRING)
+        with self.assertRaises(TypeError):
+            Season.SPRING < Part.CLIP
+
+    def test_enum_duplicates(self):
+        class Season(Enum):
+            SPRING = 1
+            SUMMER = 2
+            AUTUMN = FALL = 3
+            WINTER = 4
+            ANOTHER_SPRING = 1
+        lst = list(Season)
+        self.assertEqual(
+            lst,
+            [Season.SPRING, Season.SUMMER,
+             Season.AUTUMN, Season.WINTER,
+            ])
+        self.assertIs(Season.FALL, Season.AUTUMN)
+        self.assertEqual(Season.FALL.value, 3)
+        self.assertEqual(Season.AUTUMN.value, 3)
+        self.assertIs(Season(3), Season.AUTUMN)
+        self.assertIs(Season(1), Season.SPRING)
+        self.assertEqual(Season.FALL.name, 'AUTUMN')
+        self.assertEqual(
+                [k for k,v in Season.__members__.items() if v.name != k],
+                ['FALL', 'ANOTHER_SPRING'],
+                )
+
+    def test_enum_with_value_name(self):
+        class Huh(Enum):
+            name = 1
+            value = 2
+        self.assertEqual(
+            list(Huh),
+            [Huh.name, Huh.value],
+            )
+        self.assertIs(type(Huh.name), Huh)
+        self.assertEqual(Huh.name.name, 'name')
+        self.assertEqual(Huh.name.value, 1)
+    def test_hash(self):
+        Season = self.Season
+        dates = {}
+        dates[Season.WINTER] = '1225'
+        dates[Season.SPRING] = '0315'
+        dates[Season.SUMMER] = '0704'
+        dates[Season.AUTUMN] = '1031'
+        self.assertEqual(dates[Season.AUTUMN], '1031')
+
+    def test_intenum_from_scratch(self):
+        class phy(int, Enum):
+            pi = 3
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_intenum_inherited(self):
+        class IntEnum(int, Enum):
+            pass
+        class phy(IntEnum):
+            pi = 3
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_floatenum_from_scratch(self):
+        class phy(float, Enum):
+            pi = 3.141596
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_floatenum_inherited(self):
+        class FloatEnum(float, Enum):
+            pass
+        class phy(FloatEnum):
+            pi = 3.141596
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_strenum_from_scratch(self):
+        class phy(str, Enum):
+            pi = 'Pi'
+            tau = 'Tau'
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_strenum_inherited(self):
+        class StrEnum(str, Enum):
+            pass
+        class phy(StrEnum):
+            pi = 'Pi'
+            tau = 'Tau'
+        self.assertTrue(phy.pi < phy.tau)
+
+
+    def test_intenum(self):
+        class WeekDay(IntEnum):
+            SUNDAY = 1
+            MONDAY = 2
+            TUESDAY = 3
+            WEDNESDAY = 4
+            THURSDAY = 5
+            FRIDAY = 6
+            SATURDAY = 7
+
+        self.assertEqual(['a', 'b', 'c'][WeekDay.MONDAY], 'c')
+        self.assertEqual([i for i in range(WeekDay.TUESDAY)], [0, 1, 2])
+
+        lst = list(WeekDay)
+        self.assertEqual(len(lst), len(WeekDay))
+        self.assertEqual(len(WeekDay), 7)
+        target = 'SUNDAY MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY'
+        target = target.split()
+        for i, weekday in enumerate(target, 1):
+            e = WeekDay(i)
+            self.assertEqual(e, i)
+            self.assertEqual(int(e), i)
+            self.assertEqual(e.name, weekday)
+            self.assertIn(e, WeekDay)
+            self.assertEqual(lst.index(e)+1, i)
+            self.assertTrue(0 < e < 8)
+            self.assertIs(type(e), WeekDay)
+            self.assertIsInstance(e, int)
+            self.assertIsInstance(e, Enum)
+
+    def test_intenum_duplicates(self):
+        class WeekDay(IntEnum):
+            SUNDAY = 1
+            MONDAY = 2
+            TUESDAY = TEUSDAY = 3
+            WEDNESDAY = 4
+            THURSDAY = 5
+            FRIDAY = 6
+            SATURDAY = 7
+        self.assertIs(WeekDay.TEUSDAY, WeekDay.TUESDAY)
+        self.assertEqual(WeekDay(3).name, 'TUESDAY')
+        self.assertEqual([k for k,v in WeekDay.__members__.items()
+                if v.name != k], ['TEUSDAY', ])
+
+    def test_pickle_enum(self):
+        if isinstance(Stooges, Exception):
+            raise Stooges
+        self.assertIs(Stooges.CURLY, loads(dumps(Stooges.CURLY)))
+        self.assertIs(Stooges, loads(dumps(Stooges)))
+
+    def test_pickle_int(self):
+        if isinstance(IntStooges, Exception):
+            raise IntStooges
+        self.assertIs(IntStooges.CURLY, loads(dumps(IntStooges.CURLY)))
+        self.assertIs(IntStooges, loads(dumps(IntStooges)))
+
+    def test_pickle_float(self):
+        if isinstance(FloatStooges, Exception):
+            raise FloatStooges
+        self.assertIs(FloatStooges.CURLY, loads(dumps(FloatStooges.CURLY)))
+        self.assertIs(FloatStooges, loads(dumps(FloatStooges)))
+
+    def test_pickle_enum_function(self):
+        if isinstance(Answer, Exception):
+            raise Answer
+        self.assertIs(Answer.him, loads(dumps(Answer.him)))
+        self.assertIs(Answer, loads(dumps(Answer)))
+
+    def test_pickle_enum_function_with_module(self):
+        if isinstance(Question, Exception):
+            raise Question
+        self.assertIs(Question.who, loads(dumps(Question.who)))
+        self.assertIs(Question, loads(dumps(Question)))
+
+    def test_exploding_pickle(self):
+        BadPickle = Enum('BadPickle', 'dill sweet bread-n-butter')
+        enum._make_class_unpicklable(BadPickle)
+        globals()['BadPickle'] = BadPickle
+        with self.assertRaises(TypeError):
+            dumps(BadPickle.dill)
+        with self.assertRaises(PicklingError):
+            dumps(BadPickle)
+
+    def test_string_enum(self):
+        class SkillLevel(str, Enum):
+            master = 'what is the sound of one hand clapping?'
+            journeyman = 'why did the chicken cross the road?'
+            apprentice = 'knock, knock!'
+        self.assertEqual(SkillLevel.apprentice, 'knock, knock!')
+
+    def test_getattr_getitem(self):
+        class Period(Enum):
+            morning = 1
+            noon = 2
+            evening = 3
+            night = 4
+        self.assertIs(Period(2), Period.noon)
+        self.assertIs(getattr(Period, 'night'), Period.night)
+        self.assertIs(Period['morning'], Period.morning)
+
+    def test_getattr_dunder(self):
+        Season = self.Season
+        self.assertTrue(getattr(Season, '__eq__'))
+
+    def test_iteration_order(self):
+        class Season(Enum):
+            SUMMER = 2
+            WINTER = 4
+            AUTUMN = 3
+            SPRING = 1
+        self.assertEqual(
+                list(Season),
+                [Season.SUMMER, Season.WINTER, Season.AUTUMN, Season.SPRING],
+                )
+
+    def test_programatic_function_string(self):
+        SummerMonth = Enum('SummerMonth', 'june july august')
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_string_list(self):
+        SummerMonth = Enum('SummerMonth', ['june', 'july', 'august'])
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_iterable(self):
+        SummerMonth = Enum(
+                'SummerMonth',
+                (('june', 1), ('july', 2), ('august', 3))
+                )
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_from_dict(self):
+        SummerMonth = Enum(
+                'SummerMonth',
+                OrderedDict((('june', 1), ('july', 2), ('august', 3)))
+                )
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_type(self):
+        SummerMonth = Enum('SummerMonth', 'june july august', type=int)
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_type_from_subclass(self):
+        SummerMonth = IntEnum('SummerMonth', 'june july august')
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_subclassing(self):
+        if isinstance(Name, Exception):
+            raise Name
+        self.assertEqual(Name.BDFL, 'Guido van Rossum')
+        self.assertTrue(Name.BDFL, Name('Guido van Rossum'))
+        self.assertIs(Name.BDFL, getattr(Name, 'BDFL'))
+        self.assertIs(Name.BDFL, loads(dumps(Name.BDFL)))
+
+    def test_extending(self):
+        class Color(Enum):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(TypeError):
+            class MoreColor(Color):
+                cyan = 4
+                magenta = 5
+                yellow = 6
+
+    def test_exclude_methods(self):
+        class whatever(Enum):
+            this = 'that'
+            these = 'those'
+            def really(self):
+                return 'no, not %s' % self.value
+        self.assertIsNot(type(whatever.really), whatever)
+        self.assertEqual(whatever.this.really(), 'no, not that')
+
+    def test_overwrite_enums(self):
+        class Why(Enum):
+            question = 1
+            answer = 2
+            propisition = 3
+            def question(self):
+                print(42)
+        self.assertIsNot(type(Why.question), Why)
+        self.assertNotIn(Why.question, Why._member_names)
+        self.assertNotIn(Why.question, Why)
+
+    def test_wrong_inheritance_order(self):
+        with self.assertRaises(TypeError):
+            class Wrong(Enum, str):
+                NotHere = 'error before this point'
+
+    def test_intenum_transitivity(self):
+        class number(IntEnum):
+            one = 1
+            two = 2
+            three = 3
+        class numero(IntEnum):
+            uno = 1
+            dos = 2
+            tres = 3
+        self.assertEqual(number.one, numero.uno)
+        self.assertEqual(number.two, numero.dos)
+        self.assertEqual(number.three, numero.tres)
+
+    def test_wrong_enum_in_call(self):
+        class Monochrome(Enum):
+            black = 0
+            white = 1
+        class Gender(Enum):
+            male = 0
+            female = 1
+        self.assertRaises(ValueError, Monochrome, Gender.male)
+
+    def test_wrong_enum_in_mixed_call(self):
+        class Monochrome(IntEnum):
+            black = 0
+            white = 1
+        class Gender(Enum):
+            male = 0
+            female = 1
+        self.assertRaises(ValueError, Monochrome, Gender.male)
+
+    def test_mixed_enum_in_call_1(self):
+        class Monochrome(IntEnum):
+            black = 0
+            white = 1
+        class Gender(IntEnum):
+            male = 0
+            female = 1
+        self.assertIs(Monochrome(Gender.female), Monochrome.white)
+
+    def test_mixed_enum_in_call_2(self):
+        class Monochrome(Enum):
+            black = 0
+            white = 1
+        class Gender(IntEnum):
+            male = 0
+            female = 1
+        self.assertIs(Monochrome(Gender.male), Monochrome.black)
+
+    def test_flufl_enum(self):
+        class Fluflnum(Enum):
+            def __int__(self):
+                return int(self.value)
+        class MailManOptions(Fluflnum):
+            option1 = 1
+            option2 = 2
+            option3 = 3
+        self.assertEqual(int(MailManOptions.option1), 1)
+
+    def test_no_such_enum_member(self):
+        class Color(Enum):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(ValueError):
+            Color(4)
+        with self.assertRaises(KeyError):
+            Color['chartreuse']
+
+    def test_new_repr(self):
+        class Color(Enum):
+            red = 1
+            green = 2
+            blue = 3
+            def __repr__(self):
+                return "don't you just love shades of %s?" % self.name
+        self.assertEqual(
+                repr(Color.blue),
+                "don't you just love shades of blue?",
+                )
+
+    def test_inherited_repr(self):
+        class MyEnum(Enum):
+            def __repr__(self):
+                return "My name is %s." % self.name
+        class MyIntEnum(int, MyEnum):
+            this = 1
+            that = 2
+            theother = 3
+        self.assertEqual(repr(MyIntEnum.that), "My name is that.")
+
+    def test_multiple_mixin_mro(self):
+        class auto_enum(type(Enum)):
+            def __new__(metacls, cls, bases, classdict):
+                temp = type(classdict)()
+                names = set(classdict._member_names)
+                i = 0
+                for k in classdict._member_names:
+                    v = classdict[k]
+                    if v is Ellipsis:
+                        v = i
+                    else:
+                        i = v
+                    i += 1
+                    temp[k] = v
+                for k, v in classdict.items():
+                    if k not in names:
+                        temp[k] = v
+                return super(auto_enum, metacls).__new__(
+                        metacls, cls, bases, temp)
+
+        class AutoNumberedEnum(Enum, metaclass=auto_enum):
+            pass
+
+        class AutoIntEnum(IntEnum, metaclass=auto_enum):
+            pass
+
+        class TestAutoNumber(AutoNumberedEnum):
+            a = ...
+            b = 3
+            c = ...
+
+        class TestAutoInt(AutoIntEnum):
+            a = ...
+            b = 3
+            c = ...
+
+    def test_subclasses_with_getnewargs(self):
+        class NamedInt(int):
+            def __new__(cls, *args):
+                _args = args
+                name, *args = args
+                if len(args) == 0:
+                    raise TypeError("name and value must be specified")
+                self = int.__new__(cls, *args)
+                self._intname = name
+                self._args = _args
+                return self
+            def __getnewargs__(self):
+                return self._args
+            @property
+            def __name__(self):
+                return self._intname
+            def __repr__(self):
+                # repr() is updated to include the name and type info
+                return "{}({!r}, {})".format(type(self).__name__,
+                                             self.__name__,
+                                             int.__repr__(self))
+            def __str__(self):
+                # str() is unchanged, even if it relies on the repr() fallback
+                base = int
+                base_str = base.__str__
+                if base_str.__objclass__ is object:
+                    return base.__repr__(self)
+                return base_str(self)
+            # for simplicity, we only define one operator that
+            # propagates expressions
+            def __add__(self, other):
+                temp = int(self) + int( other)
+                if isinstance(self, NamedInt) and isinstance(other, NamedInt):
+                    return NamedInt(
+                        '({0} + {1})'.format(self.__name__, other.__name__),
+                        temp )
+                else:
+                    return temp
+
+        class NEI(NamedInt, Enum):
+            x = ('the-x', 1)
+            y = ('the-y', 2)
+
+        self.assertIs(NEI.__new__, Enum.__new__)
+        self.assertEqual(repr(NEI.x + NEI.y), "NamedInt('(the-x + the-y)', 3)")
+        globals()['NamedInt'] = NamedInt
+        globals()['NEI'] = NEI
+        NI5 = NamedInt('test', 5)
+        self.assertEqual(NI5, 5)
+        self.assertEqual(loads(dumps(NI5)), 5)
+        self.assertEqual(NEI.y.value, 2)
+        self.assertIs(loads(dumps(NEI.y)), NEI.y)
+
+    def test_subclasses_without_getnewargs(self):
+        class NamedInt(int):
+            def __new__(cls, *args):
+                _args = args
+                name, *args = args
+                if len(args) == 0:
+                    raise TypeError("name and value must be specified")
+                self = int.__new__(cls, *args)
+                self._intname = name
+                self._args = _args
+                return self
+            @property
+            def __name__(self):
+                return self._intname
+            def __repr__(self):
+                # repr() is updated to include the name and type info
+                return "{}({!r}, {})".format(type(self).__name__,
+                                             self.__name__,
+                                             int.__repr__(self))
+            def __str__(self):
+                # str() is unchanged, even if it relies on the repr() fallback
+                base = int
+                base_str = base.__str__
+                if base_str.__objclass__ is object:
+                    return base.__repr__(self)
+                return base_str(self)
+            # for simplicity, we only define one operator that
+            # propagates expressions
+            def __add__(self, other):
+                temp = int(self) + int( other)
+                if isinstance(self, NamedInt) and isinstance(other, NamedInt):
+                    return NamedInt(
+                        '({0} + {1})'.format(self.__name__, other.__name__),
+                        temp )
+                else:
+                    return temp
+
+        class NEI(NamedInt, Enum):
+            x = ('the-x', 1)
+            y = ('the-y', 2)
+
+        self.assertIs(NEI.__new__, Enum.__new__)
+        self.assertEqual(repr(NEI.x + NEI.y), "NamedInt('(the-x + the-y)', 3)")
+        globals()['NamedInt'] = NamedInt
+        globals()['NEI'] = NEI
+        NI5 = NamedInt('test', 5)
+        self.assertEqual(NI5, 5)
+        self.assertEqual(NEI.y.value, 2)
+        with self.assertRaises(TypeError):
+            dumps(NEI.x)
+        with self.assertRaises(PicklingError):
+            dumps(NEI)
+
+    def test_tuple_subclass(self):
+        class SomeTuple(tuple, Enum):
+            first = (1, 'for the money')
+            second = (2, 'for the show')
+            third = (3, 'for the music')
+        self.assertIs(type(SomeTuple.first), SomeTuple)
+        self.assertIsInstance(SomeTuple.second, tuple)
+        self.assertEqual(SomeTuple.third, (3, 'for the music'))
+        globals()['SomeTuple'] = SomeTuple
+        self.assertIs(loads(dumps(SomeTuple.first)), SomeTuple.first)
+
+    def test_duplicate_values_give_unique_enum_items(self):
+        class AutoNumber(Enum):
+            first = ()
+            second = ()
+            third = ()
+            def __new__(cls):
+                value = len(cls.__members__) + 1
+                obj = object.__new__(cls)
+                obj._value = value
+                return obj
+            def __int__(self):
+                return int(self._value)
+        self.assertEqual(
+                list(AutoNumber),
+                [AutoNumber.first, AutoNumber.second, AutoNumber.third],
+                )
+        self.assertEqual(int(AutoNumber.second), 2)
+        self.assertIs(AutoNumber(1), AutoNumber.first)
+
+    def test_inherited_new_from_enhanced_enum(self):
+        class AutoNumber(Enum):
+            def __new__(cls):
+                value = len(cls.__members__) + 1
+                obj = object.__new__(cls)
+                obj._value = value
+                return obj
+            def __int__(self):
+                return int(self._value)
+        class Color(AutoNumber):
+            red = ()
+            green = ()
+            blue = ()
+        self.assertEqual(list(Color), [Color.red, Color.green, Color.blue])
+        self.assertEqual(list(map(int, Color)), [1, 2, 3])
+
+    def test_inherited_new_from_mixed_enum(self):
+        class AutoNumber(IntEnum):
+            def __new__(cls):
+                value = len(cls.__members__) + 1
+                obj = int.__new__(cls, value)
+                obj._value = value
+                return obj
+        class Color(AutoNumber):
+            red = ()
+            green = ()
+            blue = ()
+        self.assertEqual(list(Color), [Color.red, Color.green, Color.blue])
+        self.assertEqual(list(map(int, Color)), [1, 2, 3])
+
+    def test_ordered_mixin(self):
+        class OrderedEnum(Enum):
+            def __ge__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value >= other._value
+                return NotImplemented
+            def __gt__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value > other._value
+                return NotImplemented
+            def __le__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value <= other._value
+                return NotImplemented
+            def __lt__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value < other._value
+                return NotImplemented
+        class Grade(OrderedEnum):
+            A = 5
+            B = 4
+            C = 3
+            D = 2
+            F = 1
+        self.assertGreater(Grade.A, Grade.B)
+        self.assertLessEqual(Grade.F, Grade.C)
+        self.assertLess(Grade.D, Grade.A)
+        self.assertGreaterEqual(Grade.B, Grade.B)
+    def test_extending2(self):
+        class Shade(Enum):
+            def shade(self):
+                print(self.name)
+        class Color(Shade):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(TypeError):
+            class MoreColor(Color):
+                cyan = 4
+                magenta = 5
+                yellow = 6
+
+    def test_extending3(self):
+        class Shade(Enum):
+            def shade(self):
+                return self.name
+        class Color(Shade):
+            def hex(self):
+                return '%s hexlified!' % self.value
+        class MoreColor(Color):
+            cyan = 4
+            magenta = 5
+            yellow = 6
+        self.assertEqual(MoreColor.magenta.hex(), '5 hexlified!')
+
+
+    def test_no_duplicates(self):
+        class UniqueEnum(Enum):
+            def __init__(self, *args):
+                cls = self.__class__
+                if any(self.value == e.value for e in cls):
+                    a = self.name
+                    e = cls(self.value).name
+                    raise ValueError(
+                            "aliases not allowed in UniqueEnum:  %r --> %r"
+                            % (a, e)
+                            )
+        class Color(UniqueEnum):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(ValueError):
+            class Color(UniqueEnum):
+                red = 1
+                green = 2
+                blue = 3
+                grene = 2
+
+    def test_init(self):
+        class Planet(Enum):
+            MERCURY = (3.303e+23, 2.4397e6)
+            VENUS   = (4.869e+24, 6.0518e6)
+            EARTH   = (5.976e+24, 6.37814e6)
+            MARS    = (6.421e+23, 3.3972e6)
+            JUPITER = (1.9e+27,   7.1492e7)
+            SATURN  = (5.688e+26, 6.0268e7)
+            URANUS  = (8.686e+25, 2.5559e7)
+            NEPTUNE = (1.024e+26, 2.4746e7)
+            def __init__(self, mass, radius):
+                self.mass = mass       # in kilograms
+                self.radius = radius   # in meters
+            @property
+            def surface_gravity(self):
+                # universal gravitational constant  (m3 kg-1 s-2)
+                G = 6.67300E-11
+                return G * self.mass / (self.radius * self.radius)
+        self.assertEqual(round(Planet.EARTH.surface_gravity, 2), 9.80)
+        self.assertEqual(Planet.EARTH.value, (5.976e+24, 6.37814e6))
+
+
+if __name__ == '__main__':
+    unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -381,6 +381,8 @@
 
 - Implement PEP 443 "Single-dispatch generic functions".
 
+- Implement PEP 435 "Adding an Enum type to the Python standard library".
+
 
 Tests
 -----

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 03:38:03 2013
From: python-checkins at python.org (victor.stinner)
Date: Sat, 15 Jun 2013 03:38:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Revert_changeset_6661a8154?=
 =?utf-8?q?eb3=3A_Issue_=233329=3A_Add_new_APIs_to_customize_memory?=
Message-ID: <3bXLt349w4z7Lkf@mail.python.org>

http://hg.python.org/cpython/rev/b1455dd08000
changeset:   84131:b1455dd08000
parent:      84128:5619bc2d8207
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sat Jun 15 03:37:01 2013 +0200
summary:
  Revert changeset 6661a8154eb3: Issue #3329: Add new APIs to customize memory allocators

The new API require more discussion.

files:
  Doc/c-api/memory.rst      |  121 +------
  Include/objimpl.h         |   71 +-
  Include/pymem.h           |   91 +---
  Modules/_testcapimodule.c |  178 ---------
  Objects/object.c          |   20 +
  Objects/obmalloc.c        |  501 +++++++------------------
  6 files changed, 211 insertions(+), 771 deletions(-)


diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst
--- a/Doc/c-api/memory.rst
+++ b/Doc/c-api/memory.rst
@@ -84,46 +84,6 @@
 for the I/O buffer escapes completely the Python memory manager.
 
 
-Raw Memory Interface
-====================
-
-The following function are wrappers to system allocators: :c:func:`malloc`,
-:c:func:`realloc`, :c:func:`free`. These functions are thread-safe, the
-:term:`GIL <global interpreter lock>` does not need to be held to use these
-functions.
-
-The behaviour of requesting zero bytes is not defined: return *NULL* or a
-distinct non-*NULL* pointer depending on the platform. Use
-:c:func:`PyMem_Malloc` and :c:func:`PyMem_Realloc` to have a well defined
-behaviour.
-
-.. versionadded:: 3.4
-
-.. c:function:: void* PyMem_RawMalloc(size_t n)
-
-   Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the
-   allocated memory, or *NULL* if the request fails. The memory
-   will not have been initialized in any way.
-
-
-.. c:function:: void* PyMem_RawRealloc(void *p, size_t n)
-
-   Resizes the memory block pointed to by *p* to *n* bytes. The contents will
-   be unchanged to the minimum of the old and the new sizes. If *p* is *NULL*,
-   the call is equivalent to ``PyMem_RawMalloc(n)``. Unless *p* is *NULL*, it
-   must have been returned by a previous call to :c:func:`PyMem_RawMalloc` or
-   :c:func:`PyMem_RawRealloc`. If the request fails, :c:func:`PyMem_RawRealloc`
-   returns *NULL* and *p* remains a valid pointer to the previous memory area.
-
-
-.. c:function:: void PyMem_RawFree(void *p)
-
-   Frees the memory block pointed to by *p*, which must have been returned by a
-   previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`.
-   Otherwise, or if ``PyMem_Free(p)`` has been called before, undefined
-   behavior occurs. If *p* is *NULL*, no operation is performed.
-
-
 .. _memoryinterface:
 
 Memory Interface
@@ -131,12 +91,8 @@
 
 The following function sets, modeled after the ANSI C standard, but specifying
 behavior when requesting zero bytes, are available for allocating and releasing
-memory from the Python heap.
+memory from the Python heap:
 
-.. warning::
-
-   The :term:`GIL <global interpreter lock>` must be held when using these
-   functions.
 
 .. c:function:: void* PyMem_Malloc(size_t n)
 
@@ -199,81 +155,6 @@
 :c:func:`PyMem_NEW`, :c:func:`PyMem_RESIZE`, :c:func:`PyMem_DEL`.
 
 
-Customize Memory Allocators
-===========================
-
-.. versionadded:: 3.4
-
-.. c:type:: PyMemAllocators
-
-   Structure used to describe memory allocator.  This structure has
-   four fields:
-
-   +----------------------------------------------------------+-----------------+
-   | Field                                                    | Meaning         |
-   +==========================================================+=================+
-   | ``void *ctx``                                            | user data       |
-   +----------------------------------------------------------+-----------------+
-   | ``void* malloc(void *ctx, size_t size)``                 | allocate memory |
-   +----------------------------------------------------------+-----------------+
-   | ``void* realloc(void *ctx, void *ptr, size_t new_size)`` | allocate memory |
-   |                                                          | or resize a     |
-   |                                                          | memory block    |
-   +----------------------------------------------------------+-----------------+
-   | ``void free(void *ctx, void *ptr)``                      | release memory  |
-   +----------------------------------------------------------+-----------------+
-
-.. c:function:: void PyMem_GetRawAllocators(PyMemAllocators *allocators)
-
-   Get internal functions of :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`
-   and :c:func:`PyMem_RawFree`.
-
-.. c:function:: void PyMem_SetRawAllocators(PyMemAllocators *allocators)
-
-   Set internal functions of :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`
-   and :c:func:`PyMem_RawFree`.
-
-   :c:func:`PyMem_SetupDebugHooks` should be called to reinstall debug hooks if
-   new functions do no call original functions anymore.
-
-.. c:function:: void PyMem_GetAllocators(PyMemAllocators *allocators)
-
-   Get internal functions of :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`
-   and :c:func:`PyMem_Free`.
-
-.. c:function:: void PyMem_SetAllocators(PyMemAllocators *allocators)
-
-   Set internal functions of :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`
-   and :c:func:`PyMem_Free`.
-
-   ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*: it
-   would be treated as an error.
-
-   :c:func:`PyMem_SetupDebugHooks` should be called to reinstall debug hooks if
-   new functions do no call original functions anymore.
-
-.. c:function:: void PyMem_SetupDebugHooks(void)
-
-   Setup hooks to detect bugs in the following Python memory allocator
-   functions:
-
-   - :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`,
-     :c:func:`PyMem_RawFree`
-   - :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, :c:func:`PyMem_Free`
-   - :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`,
-     :c:func:`PyObject_Free`
-
-   Newly allocated memory is filled with the byte ``0xCB``, freed memory is
-   filled with the byte ``0xDB``. Additionnal checks:
-
-   - detect API violations, ex: :c:func:`PyObject_Free` called on a buffer
-     allocated by :c:func:`PyMem_Malloc`
-   - detect write before the start of the buffer (buffer underflow)
-   - detect write after the end of the buffer (buffer overflow)
-
-   The function does nothing if Python is not compiled is debug mode.
-
-
 .. _memoryexamples:
 
 Examples
diff --git a/Include/objimpl.h b/Include/objimpl.h
--- a/Include/objimpl.h
+++ b/Include/objimpl.h
@@ -94,9 +94,9 @@
    the object gets initialized via PyObject_{Init, InitVar} after obtaining
    the raw memory.
 */
-PyAPI_FUNC(void *) PyObject_Malloc(size_t size);
-PyAPI_FUNC(void *) PyObject_Realloc(void *ptr, size_t new_size);
-PyAPI_FUNC(void) PyObject_Free(void *ptr);
+PyAPI_FUNC(void *) PyObject_Malloc(size_t);
+PyAPI_FUNC(void *) PyObject_Realloc(void *, size_t);
+PyAPI_FUNC(void) PyObject_Free(void *);
 
 /* This function returns the number of allocated memory blocks, regardless of size */
 PyAPI_FUNC(Py_ssize_t) _Py_GetAllocatedBlocks(void);
@@ -106,46 +106,41 @@
 #ifndef Py_LIMITED_API
 PyAPI_FUNC(void) _PyObject_DebugMallocStats(FILE *out);
 #endif /* #ifndef Py_LIMITED_API */
-#endif
+#ifdef PYMALLOC_DEBUG   /* WITH_PYMALLOC && PYMALLOC_DEBUG */
+PyAPI_FUNC(void *) _PyObject_DebugMalloc(size_t nbytes);
+PyAPI_FUNC(void *) _PyObject_DebugRealloc(void *p, size_t nbytes);
+PyAPI_FUNC(void) _PyObject_DebugFree(void *p);
+PyAPI_FUNC(void) _PyObject_DebugDumpAddress(const void *p);
+PyAPI_FUNC(void) _PyObject_DebugCheckAddress(const void *p);
+PyAPI_FUNC(void *) _PyObject_DebugMallocApi(char api, size_t nbytes);
+PyAPI_FUNC(void *) _PyObject_DebugReallocApi(char api, void *p, size_t nbytes);
+PyAPI_FUNC(void) _PyObject_DebugFreeApi(char api, void *p);
+PyAPI_FUNC(void) _PyObject_DebugCheckAddressApi(char api, const void *p);
+PyAPI_FUNC(void *) _PyMem_DebugMalloc(size_t nbytes);
+PyAPI_FUNC(void *) _PyMem_DebugRealloc(void *p, size_t nbytes);
+PyAPI_FUNC(void) _PyMem_DebugFree(void *p);
+#define PyObject_MALLOC         _PyObject_DebugMalloc
+#define PyObject_Malloc         _PyObject_DebugMalloc
+#define PyObject_REALLOC        _PyObject_DebugRealloc
+#define PyObject_Realloc        _PyObject_DebugRealloc
+#define PyObject_FREE           _PyObject_DebugFree
+#define PyObject_Free           _PyObject_DebugFree
 
-/* Macros */
+#else   /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */
 #define PyObject_MALLOC         PyObject_Malloc
 #define PyObject_REALLOC        PyObject_Realloc
 #define PyObject_FREE           PyObject_Free
+#endif
+
+#else   /* ! WITH_PYMALLOC */
+#define PyObject_MALLOC         PyMem_MALLOC
+#define PyObject_REALLOC        PyMem_REALLOC
+#define PyObject_FREE           PyMem_FREE
+
+#endif  /* WITH_PYMALLOC */
+
 #define PyObject_Del            PyObject_Free
-#define PyObject_DEL            PyObject_Free
-
-/* Get internal functions of PyObject_Malloc(), PyObject_Realloc() and
-   PyObject_Free(). *ctx_p is an arbitrary user value. */
-PyAPI_FUNC(void) PyObject_GetAllocators(PyMemAllocators *allocators);
-
-/* Set internal functions of PyObject_Malloc(), PyObject_Realloc() and PyObject_Free().
-   ctx is an arbitrary user value.
-
-   malloc(ctx, 0) and realloc(ctx, ptr, 0) must not return NULL: it would be
-   treated as an error.
-
-   PyMem_SetupDebugHooks() should be called to reinstall debug hooks if new
-   functions do no call original functions anymore. */
-PyAPI_FUNC(void) PyObject_SetAllocators(PyMemAllocators *allocators);
-
-/* Get internal functions allocating and deallocating arenas for
-   PyObject_Malloc(), PyObject_Realloc() and PyObject_Free().
-   *ctx_p is an arbitrary user value. */
-PyAPI_FUNC(void) _PyObject_GetArenaAllocators(
-    void **ctx_p,
-    void* (**malloc_p) (void *ctx, size_t size),
-    void (**free_p) (void *ctx, void *ptr, size_t size)
-    );
-
-/* Get internal functions allocating and deallocating arenas for
-   PyObject_Malloc(), PyObject_Realloc() and PyObject_Free().
-   ctx is an arbitrary user value. */
-PyAPI_FUNC(void) _PyObject_SetArenaAllocators(
-    void *ctx,
-    void* (*malloc) (void *ctx, size_t size),
-    void (*free) (void *ctx, void *ptr, size_t size)
-    );
+#define PyObject_DEL            PyObject_FREE
 
 /*
  * Generic object allocator interface
diff --git a/Include/pymem.h b/Include/pymem.h
--- a/Include/pymem.h
+++ b/Include/pymem.h
@@ -11,40 +11,6 @@
 extern "C" {
 #endif
 
-typedef struct {
-    /* user context passed as the first argument to the 3 functions */
-    void *ctx;
-
-    /* allocate memory */
-    void* (*malloc) (void *ctx, size_t size);
-
-    /* allocate memory or resize a memory buffer */
-    void* (*realloc) (void *ctx, void *ptr, size_t new_size);
-
-    /* release memory */
-    void (*free) (void *ctx, void *ptr);
-} PyMemAllocators;
-
-/* Raw memory allocators, system functions: malloc(), realloc(), free().
-
-   These functions are thread-safe, the GIL does not need to be held. */
-
-/* Get internal functions of PyMem_RawMalloc(), PyMem_RawRealloc() and
-   PyMem_RawFree(). *ctx_p is an arbitrary user value. */
-PyAPI_FUNC(void) PyMem_GetRawAllocators(PyMemAllocators *allocators);
-
-/* Set internal functions of PyMem_RawMalloc(), PyMem_RawRealloc() and
-   PyMem_RawFree(). ctx is an arbitrary user value.
-
-   PyMem_SetupDebugHooks() should be called to reinstall debug hooks if new
-   functions do no call original functions anymore. */
-PyAPI_FUNC(void) PyMem_SetRawAllocators(PyMemAllocators *allocators);
-
-PyAPI_FUNC(void *) PyMem_RawMalloc(size_t size);
-PyAPI_FUNC(void *) PyMem_RawRealloc(void *ptr, size_t new_size);
-PyAPI_FUNC(void) PyMem_RawFree(void *ptr);
-
-
 /* BEWARE:
 
    Each interface exports both functions and macros.  Extension modules should
@@ -83,11 +49,21 @@
    performed on failure (no exception is set, no warning is printed, etc).
 */
 
-PyAPI_FUNC(void *) PyMem_Malloc(size_t size);
-PyAPI_FUNC(void *) PyMem_Realloc(void *ptr, size_t new_size);
-PyAPI_FUNC(void) PyMem_Free(void *ptr);
+PyAPI_FUNC(void *) PyMem_Malloc(size_t);
+PyAPI_FUNC(void *) PyMem_Realloc(void *, size_t);
+PyAPI_FUNC(void) PyMem_Free(void *);
+
+/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
+   no longer supported. They used to call PyErr_NoMemory() on failure. */
 
 /* Macros. */
+#ifdef PYMALLOC_DEBUG
+/* Redirect all memory operations to Python's debugging allocator. */
+#define PyMem_MALLOC		_PyMem_DebugMalloc
+#define PyMem_REALLOC		_PyMem_DebugRealloc
+#define PyMem_FREE		_PyMem_DebugFree
+
+#else	/* ! PYMALLOC_DEBUG */
 
 /* PyMem_MALLOC(0) means malloc(1). Some systems would return NULL
    for malloc(0), which would be treated as an error. Some platforms
@@ -95,9 +71,13 @@
    pymalloc. To solve these problems, allocate an extra byte. */
 /* Returns NULL to indicate error if a negative size or size larger than
    Py_ssize_t can represent is supplied.  Helps prevents security holes. */
-#define PyMem_MALLOC(n)         PyMem_Malloc(n)
-#define PyMem_REALLOC(p, n)     PyMem_Realloc(p, n)
-#define PyMem_FREE(p)           PyMem_Free(p)
+#define PyMem_MALLOC(n)		((size_t)(n) > (size_t)PY_SSIZE_T_MAX ? NULL \
+				: malloc((n) ? (n) : 1))
+#define PyMem_REALLOC(p, n)	((size_t)(n) > (size_t)PY_SSIZE_T_MAX  ? NULL \
+				: realloc((p), (n) ? (n) : 1))
+#define PyMem_FREE		free
+
+#endif	/* PYMALLOC_DEBUG */
 
 /*
  * Type-oriented memory interface
@@ -135,37 +115,6 @@
 #define PyMem_Del		PyMem_Free
 #define PyMem_DEL		PyMem_FREE
 
-/* Get internal functions of PyMem_Malloc(), PyMem_Realloc()
-   and PyMem_Free() */
-PyAPI_FUNC(void) PyMem_GetAllocators(PyMemAllocators *allocators);
-
-/* Set internal functions of PyMem_Malloc(), PyMem_Realloc() and PyMem_Free().
-
-   malloc(ctx, 0) and realloc(ctx, ptr, 0) must not return NULL: it would be
-   treated as an error.
-
-   PyMem_SetupDebugHooks() should be called to reinstall debug hooks if new
-   functions do no call original functions anymore. */
-PyAPI_FUNC(void) PyMem_SetAllocators(PyMemAllocators *allocators);
-
-/* Setup hooks to detect bugs in the following Python memory allocator
-   functions:
-
-   - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
-   - PyMem_Malloc(), PyMem_Realloc(), PyMem_Free()
-   - PyObject_Malloc(), PyObject_Realloc() and PyObject_Free()
-
-   Newly allocated memory is filled with the byte 0xCB, freed memory is filled
-   with the byte 0xDB. Additionnal checks:
-
-   - detect API violations, ex: PyObject_Free() called on a buffer allocated
-     by PyMem_Malloc()
-   - detect write before the start of the buffer (buffer underflow)
-   - detect write after the end of the buffer (buffer overflow)
-
-   The function does nothing if Python is not compiled is debug mode. */
-PyAPI_FUNC(void) PyMem_SetupDebugHooks(void);
-
 #ifdef __cplusplus
 }
 #endif
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2511,176 +2511,6 @@
     Py_RETURN_NONE;
 }
 
-static PyObject *
-test_pymem_alloc0(PyObject *self)
-{
-    void *ptr;
-
-    ptr = PyMem_Malloc(0);
-    if (ptr == NULL) {
-        PyErr_SetString(PyExc_RuntimeError, "PyMem_Malloc(0) returns NULL");
-        return NULL;
-    }
-    PyMem_Free(ptr);
-
-    ptr = PyObject_Malloc(0);
-    if (ptr == NULL) {
-        PyErr_SetString(PyExc_RuntimeError, "PyObject_Malloc(0) returns NULL");
-        return NULL;
-    }
-    PyObject_Free(ptr);
-
-    Py_RETURN_NONE;
-}
-
-typedef struct {
-    PyMemAllocators alloc;
-
-    size_t malloc_size;
-    void *realloc_ptr;
-    size_t realloc_new_size;
-    void *free_ptr;
-} alloc_hook_t;
-
-static void* hook_malloc (void* ctx, size_t size)
-{
-    alloc_hook_t *hook = (alloc_hook_t *)ctx;
-    hook->malloc_size = size;
-    return hook->alloc.malloc(hook->alloc.ctx, size);
-}
-
-static void* hook_realloc (void* ctx, void* ptr, size_t new_size)
-{
-    alloc_hook_t *hook = (alloc_hook_t *)ctx;
-    hook->realloc_ptr = ptr;
-    hook->realloc_new_size = new_size;
-    return hook->alloc.realloc(hook->alloc.ctx, ptr, new_size);
-}
-
-static void hook_free (void *ctx, void *ptr)
-{
-    alloc_hook_t *hook = (alloc_hook_t *)ctx;
-    printf("HOOK\n");
-    hook->free_ptr = ptr;
-    hook->alloc.free(hook->alloc.ctx, ptr);
-}
-
-static PyObject *
-test_setallocators(char api)
-{
-    PyObject *res = NULL;
-    const char *error_msg;
-    alloc_hook_t hook;
-    PyMemAllocators alloc;
-    size_t size, size2;
-    void *ptr, *ptr2;
-
-    hook.malloc_size = 0;
-    hook.realloc_ptr = NULL;
-    hook.realloc_new_size = 0;
-    hook.free_ptr = NULL;
-
-    alloc.ctx = &hook;
-    alloc.malloc = &hook_malloc;
-    alloc.realloc = &hook_realloc;
-    alloc.free = &hook_free;
-    if (api == 'o') {
-        PyObject_GetAllocators(&hook.alloc);
-        PyObject_SetAllocators(&alloc);
-    }
-    else if (api == 'r') {
-        PyMem_GetRawAllocators(&hook.alloc);
-        PyMem_SetRawAllocators(&alloc);
-    }
-    else {
-        PyMem_GetAllocators(&hook.alloc);
-        PyMem_SetAllocators(&alloc);
-    }
-
-    size = 42;
-    if (api == 'o')
-        ptr = PyObject_Malloc(size);
-    else if (api == 'r')
-        ptr = PyMem_RawMalloc(size);
-    else
-        ptr = PyMem_Malloc(size);
-    if (ptr == NULL) {
-        error_msg = "malloc failed";
-        goto fail;
-    }
-
-    if (hook.malloc_size != size) {
-        error_msg = "malloc invalid size";
-        goto fail;
-    }
-
-    size2 = 200;
-    if (api == 'o')
-        ptr2 = PyObject_Realloc(ptr, size2);
-    else if (api == 'r')
-        ptr2 = PyMem_RawRealloc(ptr, size2);
-    else
-        ptr2 = PyMem_Realloc(ptr, size2);
-    if (ptr2 == NULL) {
-        error_msg = "realloc failed";
-        goto fail;
-    }
-
-    if (hook.realloc_ptr != ptr
-        || hook.realloc_new_size != size2) {
-        error_msg = "realloc invalid parameters";
-        goto fail;
-    }
-
-    if (api == 'o')
-        PyObject_Free(ptr2);
-    else if (api == 'r')
-        PyMem_RawFree(ptr2);
-    else {
-        printf("PyMem_Free\n");
-        PyMem_Free(ptr2);
-    }
-
-    if (hook.free_ptr != ptr2) {
-        error_msg = "free invalid pointer";
-        goto fail;
-    }
-
-    Py_INCREF(Py_None);
-    res = Py_None;
-    goto finally;
-
-fail:
-    PyErr_SetString(PyExc_RuntimeError, error_msg);
-
-finally:
-    if (api == 'o')
-        PyObject_SetAllocators(&hook.alloc);
-    else if (api == 'r')
-        PyMem_SetRawAllocators(&hook.alloc);
-    else
-        PyMem_SetAllocators(&hook.alloc);
-    return res;
-}
-
-static PyObject *
-test_pymem_setrawallocators(PyObject *self)
-{
-    return test_setallocators('r');
-}
-
-static PyObject *
-test_pymem_setallocators(PyObject *self)
-{
-    return test_setallocators('m');
-}
-
-static PyObject *
-test_pyobject_setallocators(PyObject *self)
-{
-    return test_setallocators('o');
-}
-
 static PyMethodDef TestMethods[] = {
     {"raise_exception",         raise_exception,                 METH_VARARGS},
     {"raise_memoryerror",   (PyCFunction)raise_memoryerror,  METH_NOARGS},
@@ -2781,14 +2611,6 @@
     {"pytime_object_to_time_t", test_pytime_object_to_time_t,  METH_VARARGS},
     {"pytime_object_to_timeval", test_pytime_object_to_timeval,  METH_VARARGS},
     {"pytime_object_to_timespec", test_pytime_object_to_timespec,  METH_VARARGS},
-    {"test_pymem",
-     (PyCFunction)test_pymem_alloc0, METH_NOARGS},
-    {"test_pymem_alloc0",
-     (PyCFunction)test_pymem_setrawallocators, METH_NOARGS},
-    {"test_pymem_setallocators",
-     (PyCFunction)test_pymem_setallocators, METH_NOARGS},
-    {"test_pyobject_setallocators",
-     (PyCFunction)test_pyobject_setallocators, METH_NOARGS},
     {NULL, NULL} /* sentinel */
 };
 
diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -1859,6 +1859,26 @@
 Py_ssize_t (*_Py_abstract_hack)(PyObject *) = PyObject_Size;
 
 
+/* Python's malloc wrappers (see pymem.h) */
+
+void *
+PyMem_Malloc(size_t nbytes)
+{
+    return PyMem_MALLOC(nbytes);
+}
+
+void *
+PyMem_Realloc(void *p, size_t nbytes)
+{
+    return PyMem_REALLOC(p, nbytes);
+}
+
+void
+PyMem_Free(void *p)
+{
+    PyMem_FREE(p);
+}
+
 void
 _PyObject_DebugTypeStats(FILE *out)
 {
diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c
--- a/Objects/obmalloc.c
+++ b/Objects/obmalloc.c
@@ -1,327 +1,5 @@
 #include "Python.h"
 
-/* Python's malloc wrappers (see pymem.h) */
-
-/* Forward declaration */
-
-#ifdef PYMALLOC_DEBUG   /* WITH_PYMALLOC && PYMALLOC_DEBUG */
-static void* _PyMem_DebugMalloc(void *ctx, size_t size);
-static void _PyMem_DebugFree(void *ctx, void *p);
-static void* _PyMem_DebugRealloc(void *ctx, void *ptr, size_t size);
-
-static void _PyObject_DebugDumpAddress(const void *p);
-static void _PyMem_DebugCheckAddress(char api_id, const void *p);
-#endif
-
-#ifdef WITH_PYMALLOC
-static void* _PyObject_Malloc(void *ctx, size_t size);
-static void _PyObject_Free(void *ctx, void *p);
-static void* _PyObject_Realloc(void *ctx, void *ptr, size_t size);
-#endif
-
-
-static void *
-_PyMem_RawMalloc(void *ctx, size_t size)
-{
-    return malloc(size);
-}
-
-static void *
-_PyMem_RawRealloc(void *ctx, void *ptr, size_t size)
-{
-    return realloc(ptr, size);
-}
-
-static void
-_PyMem_RawFree(void *ctx, void *ptr)
-{
-    return free(ptr);
-}
-
-static void *
-_PyMem_Malloc(void *ctx, size_t size)
-{
-    /* PyMem_Malloc(0) means malloc(1). Some systems would return NULL
-       for malloc(0), which would be treated as an error. Some platforms would
-       return a pointer with no memory behind it, which would break pymalloc.
-       To solve these problems, allocate an extra byte. */
-    if (size == 0)
-        size = 1;
-    return malloc(size);
-}
-
-static void *
-_PyMem_Realloc(void *ctx, void *ptr, size_t size)
-{
-    if (size == 0)
-        size = 1;
-    return realloc(ptr, size);
-}
-
-#ifdef ARENAS_USE_MMAP
-static void *
-_PyObject_ArenaMmap(void *ctx, size_t size)
-{
-    void *ptr;
-    ptr = mmap(NULL, size, PROT_READ|PROT_WRITE,
-               MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
-    if (ptr == MAP_FAILED)
-        return NULL;
-    assert(ptr != NULL);
-    return ptr;
-}
-
-static void
-_PyObject_ArenaMunmap(void *ctx, void *ptr, size_t size)
-{
-    return munmap(ptr, size);
-}
-#else
-static void *
-_PyObject_ArenaMalloc(void *ctx, size_t size)
-{
-    return malloc(size);
-}
-
-static void
-_PyObject_ArenaFree(void *ctx, void *ptr, size_t size)
-{
-    free(ptr);
-}
-#endif
-
-#define PYRAW_FUNCS _PyMem_RawMalloc, _PyMem_RawRealloc, _PyMem_RawFree
-#define PYMEM_FUNCS _PyMem_Malloc, _PyMem_Realloc, _PyMem_RawFree
-#ifdef WITH_PYMALLOC
-#define PYOBJECT_FUNCS _PyObject_Malloc, _PyObject_Realloc, _PyObject_Free
-#else
-#define PYOBJECT_FUNCS PYMEM_FUNCS
-#endif
-
-#ifdef PYMALLOC_DEBUG
-typedef struct {
-    /* We tag each block with an API ID in order to tag API violations */
-    char api_id;
-    PyMemAllocators alloc;
-} debug_alloc_api_t;
-static struct {
-    debug_alloc_api_t raw;
-    debug_alloc_api_t mem;
-    debug_alloc_api_t obj;
-} _PyMem_Debug = {
-    {'r', {NULL, PYRAW_FUNCS}},
-    {'m', {NULL, PYMEM_FUNCS}},
-    {'o', {NULL, PYOBJECT_FUNCS}}
-    };
-
-#define PYDEBUG_FUNCS _PyMem_DebugMalloc, _PyMem_DebugRealloc, _PyMem_DebugFree
-#endif
-
-static PyMemAllocators _PyMem_Raw = {
-#ifdef PYMALLOC_DEBUG
-    &_PyMem_Debug.raw, PYDEBUG_FUNCS
-#else
-    NULL, PYMEM_FUNCS
-#endif
-    };
-
-static PyMemAllocators _PyMem = {
-#ifdef PYMALLOC_DEBUG
-    &_PyMem_Debug.mem, PYDEBUG_FUNCS
-#else
-    NULL, PYMEM_FUNCS
-#endif
-    };
-
-static PyMemAllocators _PyObject = {
-#ifdef PYMALLOC_DEBUG
-    &_PyMem_Debug.obj, PYDEBUG_FUNCS
-#else
-    NULL, PYOBJECT_FUNCS
-#endif
-    };
-
-#undef PYRAW_FUNCS
-#undef PYMEM_FUNCS
-#undef PYOBJECT_FUNCS
-#undef PYDEBUG_FUNCS
-
-static struct {
-    void *ctx;
-    void* (*malloc) (void*, size_t);
-    void (*free) (void*, void*, size_t);
-} _PyObject_Arena = {NULL,
-#ifdef ARENAS_USE_MMAP
-    _PyObject_ArenaMmap, _PyObject_ArenaMunmap
-#else
-    _PyObject_ArenaMalloc, _PyObject_ArenaFree
-#endif
-    };
-
-void
-PyMem_SetupDebugHooks(void)
-{
-#ifdef PYMALLOC_DEBUG
-    PyMemAllocators alloc;
-
-    alloc.malloc = _PyMem_DebugMalloc;
-    alloc.realloc = _PyMem_DebugRealloc;
-    alloc.free = _PyMem_DebugFree;
-
-    if (_PyMem_Raw.malloc != _PyMem_DebugMalloc) {
-        alloc.ctx = &_PyMem_Debug.raw;
-        PyMem_GetAllocators(&_PyMem_Debug.raw.alloc);
-        PyMem_SetAllocators(&alloc);
-    }
-
-    if (_PyMem.malloc != _PyMem_DebugMalloc) {
-        alloc.ctx = &_PyMem_Debug.mem;
-        PyMem_GetAllocators(&_PyMem_Debug.mem.alloc);
-        PyMem_SetAllocators(&alloc);
-    }
-
-    if (_PyObject.malloc != _PyMem_DebugMalloc) {
-        alloc.ctx = &_PyMem_Debug.obj;
-        PyObject_GetAllocators(&_PyMem_Debug.obj.alloc);
-        PyObject_SetAllocators(&alloc);
-    }
-#endif
-}
-
-void
-PyMem_GetRawAllocators(PyMemAllocators *allocators)
-{
-    *allocators = _PyMem_Raw;
-}
-
-void
-PyMem_SetRawAllocators(PyMemAllocators *allocators)
-{
-    _PyMem_Raw = *allocators;
-}
-
-void
-PyMem_GetAllocators(PyMemAllocators *allocators)
-{
-    *allocators = _PyMem;
-}
-
-void
-PyMem_SetAllocators(PyMemAllocators *allocators)
-{
-    _PyMem = *allocators;
-}
-
-void
-PyObject_GetAllocators(PyMemAllocators *allocators)
-{
-    *allocators = _PyObject;
-}
-
-void
-PyObject_SetAllocators(PyMemAllocators *allocators)
-{
-    _PyObject = *allocators;
-}
-
-void
-_PyObject_GetArenaAllocators(void **ctx_p,
-                             void* (**malloc_p) (void *ctx, size_t size),
-                             void (**free_p) (void *ctx, void *ptr, size_t size))
-{
-    *malloc_p = _PyObject_Arena.malloc;
-    *free_p = _PyObject_Arena.free;
-    *ctx_p = _PyObject_Arena.ctx;
-}
-
-void
-_PyObject_SetArenaAllocators(void *ctx,
-                             void* (*malloc) (void *ctx, size_t size),
-                             void (*free) (void *ctx, void *ptr, size_t size))
-{
-    _PyObject_Arena.malloc = malloc;
-    _PyObject_Arena.free = free;
-    _PyObject_Arena.ctx = ctx;
-}
-
-void *
-PyMem_RawMalloc(size_t size)
-{
-    return _PyMem_Raw.malloc(_PyMem_Raw.ctx, size);
-}
-
-void*
-PyMem_RawRealloc(void *ptr, size_t new_size)
-{
-    return _PyMem_Raw.realloc(_PyMem_Raw.ctx, ptr, new_size);
-}
-
-void PyMem_RawFree(void *ptr)
-{
-    _PyMem_Raw.free(_PyMem_Raw.ctx, ptr);
-}
-
-void *
-PyMem_Malloc(size_t size)
-{
-    /*
-     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
-     * Most python internals blindly use a signed Py_ssize_t to track
-     * things without checking for overflows or negatives.
-     * As size_t is unsigned, checking for size < 0 is not required.
-     */
-    if (size > (size_t)PY_SSIZE_T_MAX)
-        return NULL;
-
-    return _PyMem.malloc(_PyMem.ctx, size);
-}
-
-void *
-PyMem_Realloc(void *ptr, size_t new_size)
-{
-    if (new_size > (size_t)PY_SSIZE_T_MAX)
-        return NULL;
-
-    return _PyMem.realloc(_PyMem.ctx, ptr, new_size);
-}
-
-void
-PyMem_Free(void *ptr)
-{
-    _PyMem.free(_PyMem.ctx, ptr);
-}
-
-void *
-PyObject_Malloc(size_t size)
-{
-    /*
-     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
-     * Most python internals blindly use a signed Py_ssize_t to track
-     * things without checking for overflows or negatives.
-     * As size_t is unsigned, checking for size < 0 is not required.
-     */
-    if (size > (size_t)PY_SSIZE_T_MAX)
-        return NULL;
-
-    return _PyObject.malloc(_PyObject.ctx, size);
-}
-
-void *
-PyObject_Realloc(void *ptr, size_t new_size)
-{
-    if (new_size > (size_t)PY_SSIZE_T_MAX)
-        return NULL;
-
-    return _PyObject.realloc(_PyObject.ctx, ptr, new_size);
-}
-
-void
-PyObject_Free(void *ptr)
-{
-    _PyObject.free(_PyObject.ctx, ptr);
-}
-
-
 #ifdef WITH_PYMALLOC
 
 #ifdef HAVE_MMAP
@@ -867,6 +545,7 @@
     struct arena_object* arenaobj;
     uint excess;        /* number of bytes above pool alignment */
     void *address;
+    int err;
 
 #ifdef PYMALLOC_DEBUG
     if (Py_GETENV("PYTHONMALLOCSTATS"))
@@ -888,12 +567,11 @@
             return NULL;                /* overflow */
 #endif
         nbytes = numarenas * sizeof(*arenas);
-        arenaobj = (struct arena_object *)PyMem_Realloc(arenas, nbytes);
+        arenaobj = (struct arena_object *)realloc(arenas, nbytes);
         if (arenaobj == NULL)
             return NULL;
         arenas = arenaobj;
 
-
         /* We might need to fix pointers that were copied.  However,
          * new_arena only gets called when all the pages in the
          * previous arenas are full.  Thus, there are *no* pointers
@@ -920,8 +598,15 @@
     arenaobj = unused_arena_objects;
     unused_arena_objects = arenaobj->nextarena;
     assert(arenaobj->address == 0);
-    address = _PyObject_Arena.malloc(_PyObject_Arena.ctx, ARENA_SIZE);
-    if (address == NULL) {
+#ifdef ARENAS_USE_MMAP
+    address = mmap(NULL, ARENA_SIZE, PROT_READ|PROT_WRITE,
+                                   MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
+    err = (address == MAP_FAILED);
+#else
+    address = malloc(ARENA_SIZE);
+    err = (address == 0);
+#endif
+    if (err) {
         /* The allocation failed: return NULL after putting the
          * arenaobj back.
          */
@@ -1084,8 +769,9 @@
  * Unless the optimizer reorders everything, being too smart...
  */
 
-static void *
-_PyObject_Malloc(void *ctx, size_t nbytes)
+#undef PyObject_Malloc
+void *
+PyObject_Malloc(size_t nbytes)
 {
     block *bp;
     poolp pool;
@@ -1102,6 +788,17 @@
 #endif
 
     /*
+     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
+     * Most python internals blindly use a signed Py_ssize_t to track
+     * things without checking for overflows or negatives.
+     * As size_t is unsigned, checking for nbytes < 0 is not required.
+     */
+    if (nbytes > PY_SSIZE_T_MAX) {
+        _Py_AllocatedBlocks--;
+        return NULL;
+    }
+
+    /*
      * This implicitly redirects malloc(0).
      */
     if ((nbytes - 1) < SMALL_REQUEST_THRESHOLD) {
@@ -1273,8 +970,10 @@
      * last chance to serve the request) or when the max memory limit
      * has been reached.
      */
+    if (nbytes == 0)
+        nbytes = 1;
     {
-        void *result = PyMem_Malloc(nbytes);
+        void *result = malloc(nbytes);
         if (!result)
             _Py_AllocatedBlocks--;
         return result;
@@ -1283,8 +982,9 @@
 
 /* free */
 
-static void
-_PyObject_Free(void *ctx, void *p)
+#undef PyObject_Free
+void
+PyObject_Free(void *p)
 {
     poolp pool;
     block *lastfree;
@@ -1393,8 +1093,11 @@
                 unused_arena_objects = ao;
 
                 /* Free the entire arena. */
-                _PyObject_Arena.free(_PyObject_Arena.ctx,
-                                     (void *)ao->address, ARENA_SIZE);
+#ifdef ARENAS_USE_MMAP
+                munmap((void *)ao->address, ARENA_SIZE);
+#else
+                free((void *)ao->address);
+#endif
                 ao->address = 0;                        /* mark unassociated */
                 --narenas_currently_allocated;
 
@@ -1503,7 +1206,7 @@
 redirect:
 #endif
     /* We didn't allocate this address. */
-    PyMem_Free(p);
+    free(p);
 }
 
 /* realloc.  If p is NULL, this acts like malloc(nbytes).  Else if nbytes==0,
@@ -1511,8 +1214,9 @@
  * return a non-NULL result.
  */
 
-static void *
-_PyObject_Realloc(void *ctx, void *p, size_t nbytes)
+#undef PyObject_Realloc
+void *
+PyObject_Realloc(void *p, size_t nbytes)
 {
     void *bp;
     poolp pool;
@@ -1522,7 +1226,16 @@
 #endif
 
     if (p == NULL)
-        return _PyObject_Malloc(ctx, nbytes);
+        return PyObject_Malloc(nbytes);
+
+    /*
+     * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
+     * Most python internals blindly use a signed Py_ssize_t to track
+     * things without checking for overflows or negatives.
+     * As size_t is unsigned, checking for nbytes < 0 is not required.
+     */
+    if (nbytes > PY_SSIZE_T_MAX)
+        return NULL;
 
 #ifdef WITH_VALGRIND
     /* Treat running_on_valgrind == -1 the same as 0 */
@@ -1550,10 +1263,10 @@
             }
             size = nbytes;
         }
-        bp = _PyObject_Malloc(ctx, nbytes);
+        bp = PyObject_Malloc(nbytes);
         if (bp != NULL) {
             memcpy(bp, p, size);
-            _PyObject_Free(ctx, p);
+            PyObject_Free(p);
         }
         return bp;
     }
@@ -1571,14 +1284,14 @@
      * at p.  Instead we punt:  let C continue to manage this block.
      */
     if (nbytes)
-        return PyMem_Realloc(p, nbytes);
+        return realloc(p, nbytes);
     /* C doesn't define the result of realloc(p, 0) (it may or may not
      * return NULL then), but Python's docs promise that nbytes==0 never
      * returns NULL.  We don't pass 0 to realloc(), to avoid that endcase
      * to begin with.  Even then, we can't be sure that realloc() won't
      * return NULL.
      */
-    bp = PyMem_Realloc(p, 1);
+    bp = realloc(p, 1);
     return bp ? bp : p;
 }
 
@@ -1588,6 +1301,24 @@
 /* pymalloc not enabled:  Redirect the entry points to malloc.  These will
  * only be used by extensions that are compiled with pymalloc enabled. */
 
+void *
+PyObject_Malloc(size_t n)
+{
+    return PyMem_MALLOC(n);
+}
+
+void *
+PyObject_Realloc(void *p, size_t n)
+{
+    return PyMem_REALLOC(p, n);
+}
+
+void
+PyObject_Free(void *p)
+{
+    PyMem_FREE(p);
+}
+
 Py_ssize_t
 _Py_GetAllocatedBlocks(void)
 {
@@ -1613,6 +1344,10 @@
 #define DEADBYTE       0xDB    /* dead (newly freed) memory */
 #define FORBIDDENBYTE  0xFB    /* untouchable bytes at each end of a block */
 
+/* We tag each block with an API ID in order to tag API violations */
+#define _PYMALLOC_MEM_ID 'm'   /* the PyMem_Malloc() API */
+#define _PYMALLOC_OBJ_ID 'o'   /* The PyObject_Malloc() API */
+
 static size_t serialno = 0;     /* incremented on each debug {m,re}alloc */
 
 /* serialno is always incremented via calling this routine.  The point is
@@ -1695,18 +1430,58 @@
 p[2*S+n: 2*S+n+S]
     Copies of FORBIDDENBYTE.  Used to catch over- writes and reads.
 p[2*S+n+S: 2*S+n+2*S]
-    A serial number, incremented by 1 on each call to _PyMem_DebugMalloc
-    and _PyMem_DebugRealloc.
+    A serial number, incremented by 1 on each call to _PyObject_DebugMalloc
+    and _PyObject_DebugRealloc.
     This is a big-endian size_t.
     If "bad memory" is detected later, the serial number gives an
     excellent way to set a breakpoint on the next run, to capture the
     instant at which this block was passed out.
 */
 
-static void *
-_PyMem_DebugMalloc(void *ctx, size_t nbytes)
+/* debug replacements for the PyMem_* memory API */
+void *
+_PyMem_DebugMalloc(size_t nbytes)
 {
-    debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
+    return _PyObject_DebugMallocApi(_PYMALLOC_MEM_ID, nbytes);
+}
+void *
+_PyMem_DebugRealloc(void *p, size_t nbytes)
+{
+    return _PyObject_DebugReallocApi(_PYMALLOC_MEM_ID, p, nbytes);
+}
+void
+_PyMem_DebugFree(void *p)
+{
+    _PyObject_DebugFreeApi(_PYMALLOC_MEM_ID, p);
+}
+
+/* debug replacements for the PyObject_* memory API */
+void *
+_PyObject_DebugMalloc(size_t nbytes)
+{
+    return _PyObject_DebugMallocApi(_PYMALLOC_OBJ_ID, nbytes);
+}
+void *
+_PyObject_DebugRealloc(void *p, size_t nbytes)
+{
+    return _PyObject_DebugReallocApi(_PYMALLOC_OBJ_ID, p, nbytes);
+}
+void
+_PyObject_DebugFree(void *p)
+{
+    _PyObject_DebugFreeApi(_PYMALLOC_OBJ_ID, p);
+}
+void
+_PyObject_DebugCheckAddress(const void *p)
+{
+    _PyObject_DebugCheckAddressApi(_PYMALLOC_OBJ_ID, p);
+}
+
+
+/* generic debug memory api, with an "id" to identify the API in use */
+void *
+_PyObject_DebugMallocApi(char id, size_t nbytes)
+{
     uchar *p;           /* base address of malloc'ed block */
     uchar *tail;        /* p + 2*SST + nbytes == pointer to tail pad bytes */
     size_t total;       /* nbytes + 4*SST */
@@ -1717,14 +1492,14 @@
         /* overflow:  can't represent total as a size_t */
         return NULL;
 
-    p = (uchar *)api->alloc.malloc(api->alloc.ctx, total);
+    p = (uchar *)PyObject_Malloc(total);
     if (p == NULL)
         return NULL;
 
     /* at p, write size (SST bytes), id (1 byte), pad (SST-1 bytes) */
     write_size_t(p, nbytes);
-    p[SST] = (uchar)api->api_id;
-    memset(p + SST + 1, FORBIDDENBYTE, SST-1);
+    p[SST] = (uchar)id;
+    memset(p + SST + 1 , FORBIDDENBYTE, SST-1);
 
     if (nbytes > 0)
         memset(p + 2*SST, CLEANBYTE, nbytes);
@@ -1742,27 +1517,25 @@
    Then fills the original bytes with DEADBYTE.
    Then calls the underlying free.
 */
-static void
-_PyMem_DebugFree(void *ctx, void *p)
+void
+_PyObject_DebugFreeApi(char api, void *p)
 {
-    debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
     uchar *q = (uchar *)p - 2*SST;  /* address returned from malloc */
     size_t nbytes;
 
     if (p == NULL)
         return;
-    _PyMem_DebugCheckAddress(api->api_id, p);
+    _PyObject_DebugCheckAddressApi(api, p);
     nbytes = read_size_t(q);
     nbytes += 4*SST;
     if (nbytes > 0)
         memset(q, DEADBYTE, nbytes);
-    api->alloc.free(api->alloc.ctx, q);
+    PyObject_Free(q);
 }
 
-static void *
-_PyMem_DebugRealloc(void *ctx, void *p, size_t nbytes)
+void *
+_PyObject_DebugReallocApi(char api, void *p, size_t nbytes)
 {
-    debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
     uchar *q = (uchar *)p;
     uchar *tail;
     size_t total;       /* nbytes + 4*SST */
@@ -1770,9 +1543,9 @@
     int i;
 
     if (p == NULL)
-        return _PyMem_DebugMalloc(ctx, nbytes);
+        return _PyObject_DebugMallocApi(api, nbytes);
 
-    _PyMem_DebugCheckAddress(api->api_id, p);
+    _PyObject_DebugCheckAddressApi(api, p);
     bumpserialno();
     original_nbytes = read_size_t(q - 2*SST);
     total = nbytes + 4*SST;
@@ -1789,12 +1562,12 @@
      * case we didn't get the chance to mark the old memory with DEADBYTE,
      * but we live with that.
      */
-    q = (uchar *)api->alloc.realloc(api->alloc.ctx, q - 2*SST, total);
+    q = (uchar *)PyObject_Realloc(q - 2*SST, total);
     if (q == NULL)
         return NULL;
 
     write_size_t(q, nbytes);
-    assert(q[SST] == (uchar)api->api_id);
+    assert(q[SST] == (uchar)api);
     for (i = 1; i < SST; ++i)
         assert(q[SST + i] == FORBIDDENBYTE);
     q += 2*SST;
@@ -1816,8 +1589,8 @@
  * and call Py_FatalError to kill the program.
  * The API id, is also checked.
  */
-static void
-_PyMem_DebugCheckAddress(char api, const void *p)
+ void
+_PyObject_DebugCheckAddressApi(char api, const void *p)
 {
     const uchar *q = (const uchar *)p;
     char msgbuf[64];
@@ -1869,7 +1642,7 @@
 }
 
 /* Display info to stderr about the memory block at p. */
-static void
+void
 _PyObject_DebugDumpAddress(const void *p)
 {
     const uchar *q = (const uchar *)p;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 03:38:05 2013
From: python-checkins at python.org (victor.stinner)
Date: Sat, 15 Jun 2013 03:38:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_heads?=
Message-ID: <3bXLt53bdGz7LkY@mail.python.org>

http://hg.python.org/cpython/rev/3cced3170f98
changeset:   84132:3cced3170f98
parent:      84131:b1455dd08000
parent:      84130:e7a01c7f69fe
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sat Jun 15 03:37:45 2013 +0200
summary:
  Merge heads

files:
  Doc/library/enum.rst  |  542 +++++++++++++++++
  Doc/library/imp.rst   |    3 +
  Doc/library/types.rst |   30 +-
  Lib/enum.py           |  465 +++++++++++++++
  Lib/test/test_enum.py |  921 ++++++++++++++++++++++++++++++
  Misc/NEWS             |    5 +
  6 files changed, 1964 insertions(+), 2 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
new file mode 100644
--- /dev/null
+++ b/Doc/library/enum.rst
@@ -0,0 +1,542 @@
+:mod:`enum` --- Support for enumerations
+========================================
+
+.. module:: enum
+.. :synopsis: enumerations are sets of symbolic names bound to unique, constant
+  values.
+.. :moduleauthor:: Ethan Furman <ethan at stoneleaf.us>
+.. :sectionauthor:: Barry Warsaw <barry at python.org>,
+.. :sectionauthor:: Eli Bendersky <eliben at gmail.com>,
+.. :sectionauthor:: Ethan Furman <ethan at stoneleaf.us>
+
+**Source code:** :source:`Lib/enum.py`
+
+----------------
+
+An enumeration is a set of symbolic names (members) bound to unique, constant
+values.  Within an enumeration, the members can be compared by identity, and
+the enumeration itself can be iterated over.
+
+This module defines two enumeration classes that can be used to define unique
+sets of names and values: :class:`Enum` and :class:`IntEnum`.
+
+Creating an Enum
+----------------
+
+Enumerations are created using the :keyword:`class` syntax, which makes them
+easy to read and write.  An alternative creation method is described in
+`Functional API`_.  To define an enumeration, subclass :class:`Enum` as
+follows::
+
+    >>> from enum import Enum
+    >>> class Color(Enum):
+    ...     red = 1
+    ...     green = 2
+    ...     blue = 3
+
+**A note on nomenclature**: we call :class:`Color` an *enumeration* (or *enum*)
+and :attr:`Color.red`, :attr:`Color.green` are *enumeration members* (or
+*enum members*).  Enumeration members also have *values* (the value of
+:attr:`Color.red` is ``1``, etc.)
+
+Enumeration members have human readable string representations::
+
+    >>> print(Color.red)
+    Color.red
+
+...while their ``repr`` has more information::
+
+    >>> print(repr(Color.red))
+    <Color.red: 1>
+
+The *type* of an enumeration member is the enumeration it belongs to::
+
+    >>> type(Color.red)
+    <enum 'Color'>
+    >>> isinstance(Color.green, Color)
+    True
+    >>>
+
+Enum members also have a property that contains just their item name::
+
+    >>> print(Color.red.name)
+    red
+
+Enumerations support iteration, in definition order::
+
+    >>> class Shake(Enum):
+    ...   vanilla = 7
+    ...   chocolate = 4
+    ...   cookies = 9
+    ...   mint = 3
+    ...
+    >>> for shake in Shake:
+    ...   print(shake)
+    ...
+    Shake.vanilla
+    Shake.chocolate
+    Shake.cookies
+    Shake.mint
+
+Enumeration members are hashable, so they can be used in dictionaries and sets::
+
+    >>> apples = {}
+    >>> apples[Color.red] = 'red delicious'
+    >>> apples[Color.green] = 'granny smith'
+    >>> apples == {Color.red: 'red delicious', Color.green: 'granny smith'}
+    True
+
+
+Programmatic access to enumeration members
+------------------------------------------
+
+Sometimes it's useful to access members in enumerations programmatically (i.e.
+situations where ``Color.red`` won't do because the exact color is not known
+at program-writing time).  ``Enum`` allows such access::
+
+    >>> Color(1)
+    <Color.red: 1>
+    >>> Color(3)
+    <Color.blue: 3>
+
+If you want to access enum members by *name*, use item access::
+
+    >>> Color['red']
+    <Color.red: 1>
+    >>> Color['green']
+    <Color.green: 2>
+
+
+Duplicating enum members and values
+-----------------------------------
+
+Having two enum members with the same name is invalid::
+
+    >>> class Shape(Enum):
+    ...   square = 2
+    ...   square = 3
+    ...
+    Traceback (most recent call last):
+    ...
+    TypeError: Attempted to reuse key: 'square'
+
+However, two enum members are allowed to have the same value.  Given two members
+A and B with the same value (and A defined first), B is an alias to A.  By-value
+lookup of the value of A and B will return A.  By-name lookup of B will also
+return A::
+
+    >>> class Shape(Enum):
+    ...   square = 2
+    ...   diamond = 1
+    ...   circle = 3
+    ...   alias_for_square = 2
+    ...
+    >>> Shape.square
+    <Shape.square: 2>
+    >>> Shape.alias_for_square
+    <Shape.square: 2>
+    >>> Shape(2)
+    <Shape.square: 2>
+
+Iterating over the members of an enum does not provide the aliases::
+
+    >>> list(Shape)
+    [<Shape.square: 2>, <Shape.diamond: 1>, <Shape.circle: 3>]
+
+The special attribute ``__members__`` is an ordered dictionary mapping names
+to members.  It includes all names defined in the enumeration, including the
+aliases::
+
+    >>> for name, member in Shape.__members__.items():
+    ...   name, member
+    ...
+    ('square', <Shape.square: 2>)
+    ('diamond', <Shape.diamond: 1>)
+    ('circle', <Shape.circle: 3>)
+    ('alias_for_square', <Shape.square: 2>)
+
+The ``__members__`` attribute can be used for detailed programmatic access to
+the enumeration members.  For example, finding all the aliases::
+
+    >>> [name for name, member in Shape.__members__.items() if member.name != name]
+    ['alias_for_square']
+
+Comparisons
+-----------
+
+Enumeration members are compared by identity::
+
+    >>> Color.red is Color.red
+    True
+    >>> Color.red is Color.blue
+    False
+    >>> Color.red is not Color.blue
+    True
+
+Ordered comparisons between enumeration values are *not* supported.  Enum
+members are not integers (but see `IntEnum`_ below)::
+
+    >>> Color.red < Color.blue
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in <module>
+    TypeError: unorderable types: Color() < Color()
+
+Equality comparisons are defined though::
+
+    >>> Color.blue == Color.red
+    False
+    >>> Color.blue != Color.red
+    True
+    >>> Color.blue == Color.blue
+    True
+
+Comparisons against non-enumeration values will always compare not equal
+(again, class:`IntEnum` was explicitly designed to behave differently, see
+below)::
+
+    >>> Color.blue == 2
+    False
+
+
+Allowed members and attributes of enumerations
+----------------------------------------------
+
+The examples above use integers for enumeration values.  Using integers is
+short and handy (and provided by default by the `Functional API`_), but not
+strictly enforced.  In the vast majority of use-cases, one doesn't care what
+the actual value of an enumeration is.  But if the value *is* important,
+enumerations can have arbitrary values.
+
+Enumerations are Python classes, and can have methods and special methods as
+usual.  If we have this enumeration::
+
+    >>> class Mood(Enum):
+    ...   funky = 1
+    ...   happy = 3
+    ...
+    ...   def describe(self):
+    ...     # self is the member here
+    ...     return self.name, self.value
+    ...
+    ...   def __str__(self):
+    ...     return 'my custom str! {0}'.format(self.value)
+    ...
+    ...   @classmethod
+    ...   def favorite_mood(cls):
+    ...     # cls here is the enumeration
+    ...     return cls.happy
+
+Then::
+
+    >>> Mood.favorite_mood()
+    <Mood.happy: 3>
+    >>> Mood.happy.describe()
+    ('happy', 3)
+    >>> str(Mood.funky)
+    'my custom str! 1'
+
+The rules for what is allowed are as follows: _sunder_ names (starting and
+ending with a single underscore) are reserved by enum and cannot be used;
+all other attributes defined within an enumeration will become members of this
+enumeration, with the exception of *__dunder__* names and descriptors (methods
+are also descriptors).
+
+Note:  if your enumeration defines :meth:`__new__` and/or :meth:`__init__` then
+whatever value(s) were given to the enum member will be passed into those
+methods.  See `Planet`_ for an example.
+
+
+Restricted subclassing of enumerations
+--------------------------------------
+
+Subclassing an enumeration is allowed only if the enumeration does not define
+any members.  So this is forbidden::
+
+    >>> class MoreColor(Color):
+    ...   pink = 17
+    Traceback (most recent call last):
+    ...
+    TypeError: Cannot extend enumerations
+
+But this is allowed::
+
+    >>> class Foo(Enum):
+    ...   def some_behavior(self):
+    ...     pass
+    ...
+    >>> class Bar(Foo):
+    ...   happy = 1
+    ...   sad = 2
+    ...
+
+Allowing subclassing of enums that define members would lead to a violation of
+some important invariants of types and instances.  On the other hand, it makes
+sense to allow sharing some common behavior between a group of enumerations.
+(See `OrderedEnum`_ for an example.)
+
+
+Pickling
+--------
+
+Enumerations can be pickled and unpickled::
+
+    >>> from test.test_enum import Fruit
+    >>> from pickle import dumps, loads
+    >>> Fruit.tomato is loads(dumps(Fruit.tomato))
+    True
+
+The usual restrictions for pickling apply: picklable enums must be defined in
+the top level of a module, since unpickling requires them to be importable
+from that module.
+
+.. warning::
+
+    In order to support the singleton nature of enumeration members, pickle
+    protocol version 2 or higher must be used.
+
+
+Functional API
+--------------
+
+The :class:`Enum` class is callable, providing the following functional API::
+
+    >>> Animal = Enum('Animal', 'ant bee cat dog')
+    >>> Animal
+    <enum 'Animal'>
+    >>> Animal.ant
+    <Animal.ant: 1>
+    >>> Animal.ant.value
+    1
+    >>> list(Animal)
+    [<Animal.ant: 1>, <Animal.bee: 2>, <Animal.cat: 3>, <Animal.dog: 4>]
+
+The semantics of this API resemble :class:`namedtuple`. The first argument
+of the call to :class:`Enum` is the name of the enumeration.
+
+The second argument is the *source* of enumeration member names.  It can be a
+whitespace-separated string of names, a sequence of names, a sequence of
+2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to
+values.  The last two options enable assigning arbitrary values to
+enumerations; the others auto-assign increasing integers starting with 1.  A
+new class derived from :class:`Enum` is returned.  In other words, the above
+assignment to :class:`Animal` is equivalent to::
+
+    >>> class Animals(Enum):
+    ...   ant = 1
+    ...   bee = 2
+    ...   cat = 3
+    ...   dog = 4
+
+Pickling enums created with the functional API can be tricky as frame stack
+implementation details are used to try and figure out which module the
+enumeration is being created in (e.g. it will fail if you use a utility
+function in separate module, and also may not work on IronPython or Jython).
+The solution is to specify the module name explicitly as follows::
+
+    >>> Animals = Enum('Animals', 'ant bee cat dog', module=__name__)
+
+Derived Enumerations
+====================
+
+IntEnum
+-------
+
+A variation of :class:`Enum` is provided which is also a subclass of
+:class:`int`.  Members of an :class:`IntEnum` can be compared to integers;
+by extension, integer enumerations of different types can also be compared
+to each other::
+
+    >>> from enum import IntEnum
+    >>> class Shape(IntEnum):
+    ...   circle = 1
+    ...   square = 2
+    ...
+    >>> class Request(IntEnum):
+    ...   post = 1
+    ...   get = 2
+    ...
+    >>> Shape == 1
+    False
+    >>> Shape.circle == 1
+    True
+    >>> Shape.circle == Request.post
+    True
+
+However, they still can't be compared to standard :class:`Enum` enumerations::
+
+    >>> class Shape(IntEnum):
+    ...   circle = 1
+    ...   square = 2
+    ...
+    >>> class Color(Enum):
+    ...   red = 1
+    ...   green = 2
+    ...
+    >>> Shape.circle == Color.red
+    False
+
+:class:`IntEnum` values behave like integers in other ways you'd expect::
+
+    >>> int(Shape.circle)
+    1
+    >>> ['a', 'b', 'c'][Shape.circle]
+    'b'
+    >>> [i for i in range(Shape.square)]
+    [0, 1]
+
+For the vast majority of code, :class:`Enum` is strongly recommended,
+since :class:`IntEnum` breaks some semantic promises of an enumeration (by
+being comparable to integers, and thus by transitivity to other
+unrelated enumerations).  It should be used only in special cases where
+there's no other choice; for example, when integer constants are
+replaced with enumerations and backwards compatibility is required with code
+that still expects integers.
+
+
+Others
+------
+
+While :class:`IntEnum` is part of the :mod:`enum` module, it would be very
+simple to implement independently::
+
+    class IntEnum(int, Enum):
+        pass
+
+This demonstrates how similar derived enumerations can be defined; for example
+a :class:`StrEnum` that mixes in :class:`str` instead of :class:`int`.
+
+Some rules:
+
+1. When subclassing :class:`Enum`, mix-in types must appear before
+   :class:`Enum` itself in the sequence of bases, as in the :class:`IntEnum`
+   example above.
+2. While :class:`Enum` can have members of any type, once you mix in an
+   additional type, all the members must have values of that type, e.g.
+   :class:`int` above.  This restriction does not apply to mix-ins which only
+   add methods and don't specify another data type such as :class:`int` or
+   :class:`str`.
+3. When another data type is mixed in, the :attr:`value` attribute is *not the
+   same* as the enum member itself, although it is equivalant and will compare
+   equal.
+
+
+Interesting examples
+====================
+
+While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of
+use-cases, they cannot cover them all.  Here are recipes for some different
+types of enumerations that can be used directly, or as examples for creating
+one's own.
+
+
+AutoNumber
+----------
+
+Avoids having to specify the value for each enumeration member::
+
+    >>> class AutoNumber(Enum):
+    ...     def __new__(cls):
+    ...         value = len(cls.__members__) + 1
+    ...         obj = object.__new__(cls)
+    ...         obj._value = value
+    ...         return obj
+    ...
+    >>> class Color(AutoNumber):
+    ...     red = ()
+    ...     green = ()
+    ...     blue = ()
+    ...
+    >>> Color.green.value == 2
+    True
+
+
+UniqueEnum
+----------
+
+Raises an error if a duplicate member name is found instead of creating an
+alias::
+
+    >>> class UniqueEnum(Enum):
+    ...     def __init__(self, *args):
+    ...         cls = self.__class__
+    ...         if any(self.value == e.value for e in cls):
+    ...             a = self.name
+    ...             e = cls(self.value).name
+    ...             raise ValueError(
+    ...                     "aliases not allowed in UniqueEnum:  %r --> %r"
+    ...                     % (a, e))
+    ...
+    >>> class Color(UniqueEnum):
+    ...     red = 1
+    ...     green = 2
+    ...     blue = 3
+    ...     grene = 2
+    Traceback (most recent call last):
+    ...
+    ValueError: aliases not allowed in UniqueEnum:  'grene' --> 'green'
+
+
+OrderedEnum
+-----------
+
+An ordered enumeration that is not based on :class:`IntEnum` and so maintains
+the normal :class:`Enum` invariants (such as not being comparable to other
+enumerations)::
+
+    >>> class OrderedEnum(Enum):
+    ...     def __ge__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value >= other._value
+    ...         return NotImplemented
+    ...     def __gt__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value > other._value
+    ...         return NotImplemented
+    ...     def __le__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value <= other._value
+    ...         return NotImplemented
+    ...     def __lt__(self, other):
+    ...         if self.__class__ is other.__class__:
+    ...             return self._value < other._value
+    ...         return NotImplemented
+    ...
+    >>> class Grade(OrderedEnum):
+    ...     A = 5
+    ...     B = 4
+    ...     C = 3
+    ...     D = 2
+    ...     F = 1
+    ...
+    >>> Grade.C < Grade.A
+    True
+
+
+Planet
+------
+
+If :meth:`__new__` or :meth:`__init__` is defined the value of the enum member
+will be passed to those methods::
+
+    >>> class Planet(Enum):
+    ...     MERCURY = (3.303e+23, 2.4397e6)
+    ...     VENUS   = (4.869e+24, 6.0518e6)
+    ...     EARTH   = (5.976e+24, 6.37814e6)
+    ...     MARS    = (6.421e+23, 3.3972e6)
+    ...     JUPITER = (1.9e+27,   7.1492e7)
+    ...     SATURN  = (5.688e+26, 6.0268e7)
+    ...     URANUS  = (8.686e+25, 2.5559e7)
+    ...     NEPTUNE = (1.024e+26, 2.4746e7)
+    ...     def __init__(self, mass, radius):
+    ...         self.mass = mass       # in kilograms
+    ...         self.radius = radius   # in meters
+    ...     @property
+    ...     def surface_gravity(self):
+    ...         # universal gravitational constant  (m3 kg-1 s-2)
+    ...         G = 6.67300E-11
+    ...         return G * self.mass / (self.radius * self.radius)
+    ...
+    >>> Planet.EARTH.value
+    (5.976e+24, 6378140.0)
+    >>> Planet.EARTH.surface_gravity
+    9.802652743337129
diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -112,6 +112,9 @@
    Return a new empty module object called *name*.  This object is *not* inserted
    in ``sys.modules``.
 
+   .. deprecated:: 3.4
+      Use :class:`types.ModuleType` instead.
+
 
 .. function:: reload(module)
 
diff --git a/Doc/library/types.rst b/Doc/library/types.rst
--- a/Doc/library/types.rst
+++ b/Doc/library/types.rst
@@ -107,9 +107,35 @@
    C".)
 
 
-.. data:: ModuleType
+.. class:: ModuleType(name, doc=None)
 
-   The type of modules.
+   The type of :term:`modules <module>`. Constructor takes the name of the
+   module to be created and optionally its :term:`docstring`.
+
+   .. attribute:: __doc__
+
+      The :term:`docstring` of the module. Defaults to ``None``.
+
+   .. attribute:: __loader__
+
+      The :term:`loader` which loaded the module. Defaults to ``None``.
+
+      .. versionchanged:: 3.4
+         Defaults to ``None``. Previously the attribute was optional.
+
+   .. attribute:: __name__
+
+      The name of the module.
+
+   .. attribute:: __package__
+
+      Which :term:`package` a module belongs to. If the module is top-level
+      (i.e. not a part of any specific package) then the attribute should be set
+      to ``''``, else it should be set to the name of the package (which can be
+      :attr:`__name__` if the module is a package itself). Defaults to ``None``.
+
+      .. versionchanged:: 3.4
+         Defaults to ``None``. Previously the attribute was optional.
 
 
 .. data:: TracebackType
diff --git a/Lib/enum.py b/Lib/enum.py
new file mode 100644
--- /dev/null
+++ b/Lib/enum.py
@@ -0,0 +1,465 @@
+"""Python Enumerations"""
+
+import sys
+from collections import OrderedDict
+from types import MappingProxyType
+
+__all__ = ['Enum', 'IntEnum']
+
+
+class _RouteClassAttributeToGetattr:
+    """Route attribute access on a class to __getattr__.
+
+    This is a descriptor, used to define attributes that act differently when
+    accessed through an instance and through a class.  Instance access remains
+    normal, but access to an attribute through a class will be routed to the
+    class's __getattr__ method; this is done by raising AttributeError.
+
+    """
+    def __init__(self, fget=None):
+        self.fget = fget
+
+    def __get__(self, instance, ownerclass=None):
+        if instance is None:
+            raise AttributeError()
+        return self.fget(instance)
+
+    def __set__(self, instance, value):
+        raise AttributeError("can't set attribute")
+
+    def __delete__(self, instance):
+        raise AttributeError("can't delete attribute")
+
+
+def _is_dunder(name):
+    """Returns True if a __dunder__ name, False otherwise."""
+    return (name[:2] == name[-2:] == '__' and
+            name[2:3] != '_' and
+            name[-3:-2] != '_')
+
+
+def _is_sunder(name):
+    """Returns True if a _sunder_ name, False otherwise."""
+    return (name[0] == name[-1] == '_' and
+            name[1:2] != '_' and
+            name[-2:-1] != '_')
+
+
+def _make_class_unpicklable(cls):
+    """Make the given class un-picklable."""
+    def _break_on_call_reduce(self):
+        raise TypeError('%r cannot be pickled' % self)
+    cls.__reduce__ = _break_on_call_reduce
+    cls.__module__ = '<unknown>'
+
+
+class _EnumDict(dict):
+    """Keeps track of definition order of the enum items.
+
+    EnumMeta will use the names found in self._member_names as the
+    enumeration member names.
+
+    """
+    def __init__(self):
+        super().__init__()
+        self._member_names = []
+
+    def __setitem__(self, key, value):
+        """Changes anything not dundered or that doesn't have __get__.
+
+        If a descriptor is added with the same name as an enum member, the name
+        is removed from _member_names (this may leave a hole in the numerical
+        sequence of values).
+
+        If an enum member name is used twice, an error is raised; duplicate
+        values are not checked for.
+
+        Single underscore (sunder) names are reserved.
+
+        """
+        if _is_sunder(key):
+            raise ValueError('_names_ are reserved for future Enum use')
+        elif _is_dunder(key) or hasattr(value, '__get__'):
+            if key in self._member_names:
+                # overwriting an enum with a method?  then remove the name from
+                # _member_names or it will become an enum anyway when the class
+                # is created
+                self._member_names.remove(key)
+        else:
+            if key in self._member_names:
+                raise TypeError('Attempted to reuse key: %r' % key)
+            self._member_names.append(key)
+        super().__setitem__(key, value)
+
+
+# Dummy value for Enum as EnumMeta explicity checks for it, but of course until
+# EnumMeta finishes running the first time the Enum class doesn't exist.  This
+# is also why there are checks in EnumMeta like `if Enum is not None`
+Enum = None
+
+
+class EnumMeta(type):
+    """Metaclass for Enum"""
+    @classmethod
+    def __prepare__(metacls, cls, bases):
+        return _EnumDict()
+
+    def __new__(metacls, cls, bases, classdict):
+        # an Enum class is final once enumeration items have been defined; it
+        # cannot be mixed with other types (int, float, etc.) if it has an
+        # inherited __new__ unless a new __new__ is defined (or the resulting
+        # class will fail).
+        member_type, first_enum = metacls._get_mixins_(bases)
+        __new__, save_new, use_args = metacls._find_new_(classdict, member_type,
+                                                        first_enum)
+
+        # save enum items into separate mapping so they don't get baked into
+        # the new class
+        members = {k: classdict[k] for k in classdict._member_names}
+        for name in classdict._member_names:
+            del classdict[name]
+
+        # check for illegal enum names (any others?)
+        invalid_names = set(members) & {'mro', }
+        if invalid_names:
+            raise ValueError('Invalid enum member name: {0}'.format(
+                ','.join(invalid_names)))
+
+        # create our new Enum type
+        enum_class = super().__new__(metacls, cls, bases, classdict)
+        enum_class._member_names = []               # names in definition order
+        enum_class._member_map = OrderedDict()      # name->value map
+
+        # Reverse value->name map for hashable values.
+        enum_class._value2member_map = {}
+
+        # check for a __getnewargs__, and if not present sabotage
+        # pickling, since it won't work anyway
+        if (member_type is not object and
+            member_type.__dict__.get('__getnewargs__') is None
+            ):
+            _make_class_unpicklable(enum_class)
+
+        # instantiate them, checking for duplicates as we go
+        # we instantiate first instead of checking for duplicates first in case
+        # a custom __new__ is doing something funky with the values -- such as
+        # auto-numbering ;)
+        for member_name in classdict._member_names:
+            value = members[member_name]
+            if not isinstance(value, tuple):
+                args = (value, )
+            else:
+                args = value
+            if member_type is tuple:   # special case for tuple enums
+                args = (args, )     # wrap it one more time
+            if not use_args:
+                enum_member = __new__(enum_class)
+                enum_member._value = value
+            else:
+                enum_member = __new__(enum_class, *args)
+                if not hasattr(enum_member, '_value'):
+                    enum_member._value = member_type(*args)
+            enum_member._member_type = member_type
+            enum_member._name = member_name
+            enum_member.__init__(*args)
+            # If another member with the same value was already defined, the
+            # new member becomes an alias to the existing one.
+            for name, canonical_member in enum_class._member_map.items():
+                if canonical_member.value == enum_member._value:
+                    enum_member = canonical_member
+                    break
+            else:
+                # Aliases don't appear in member names (only in __members__).
+                enum_class._member_names.append(member_name)
+            enum_class._member_map[member_name] = enum_member
+            try:
+                # This may fail if value is not hashable. We can't add the value
+                # to the map, and by-value lookups for this value will be
+                # linear.
+                enum_class._value2member_map[value] = enum_member
+            except TypeError:
+                pass
+
+        # double check that repr and friends are not the mixin's or various
+        # things break (such as pickle)
+        for name in ('__repr__', '__str__', '__getnewargs__'):
+            class_method = getattr(enum_class, name)
+            obj_method = getattr(member_type, name, None)
+            enum_method = getattr(first_enum, name, None)
+            if obj_method is not None and obj_method is class_method:
+                setattr(enum_class, name, enum_method)
+
+        # replace any other __new__ with our own (as long as Enum is not None,
+        # anyway) -- again, this is to support pickle
+        if Enum is not None:
+            # if the user defined their own __new__, save it before it gets
+            # clobbered in case they subclass later
+            if save_new:
+                enum_class.__new_member__ = __new__
+            enum_class.__new__ = Enum.__new__
+        return enum_class
+
+    def __call__(cls, value, names=None, *, module=None, type=None):
+        """Either returns an existing member, or creates a new enum class.
+
+        This method is used both when an enum class is given a value to match
+        to an enumeration member (i.e. Color(3)) and for the functional API
+        (i.e. Color = Enum('Color', names='red green blue')).
+
+        When used for the functional API: `module`, if set, will be stored in
+        the new class' __module__ attribute; `type`, if set, will be mixed in
+        as the first base class.
+
+        Note: if `module` is not set this routine will attempt to discover the
+        calling module by walking the frame stack; if this is unsuccessful
+        the resulting class will not be pickleable.
+
+        """
+        if names is None:  # simple value lookup
+            return cls.__new__(cls, value)
+        # otherwise, functional API: we're creating a new Enum type
+        return cls._create_(value, names, module=module, type=type)
+
+    def __contains__(cls, member):
+        return isinstance(member, cls) and member.name in cls._member_map
+
+    def __dir__(self):
+        return ['__class__', '__doc__', '__members__'] + self._member_names
+
+    @property
+    def __members__(cls):
+        """Returns a mapping of member name->value.
+
+        This mapping lists all enum members, including aliases. Note that this
+        is a read-only view of the internal mapping.
+
+        """
+        return MappingProxyType(cls._member_map)
+
+    def __getattr__(cls, name):
+        """Return the enum member matching `name`
+
+        We use __getattr__ instead of descriptors or inserting into the enum
+        class' __dict__ in order to support `name` and `value` being both
+        properties for enum members (which live in the class' __dict__) and
+        enum members themselves.
+
+        """
+        if _is_dunder(name):
+            raise AttributeError(name)
+        try:
+            return cls._member_map[name]
+        except KeyError:
+            raise AttributeError(name) from None
+
+    def __getitem__(cls, name):
+        return cls._member_map[name]
+
+    def __iter__(cls):
+        return (cls._member_map[name] for name in cls._member_names)
+
+    def __len__(cls):
+        return len(cls._member_names)
+
+    def __repr__(cls):
+        return "<enum %r>" % cls.__name__
+
+    def _create_(cls, class_name, names=None, *, module=None, type=None):
+        """Convenience method to create a new Enum class.
+
+        `names` can be:
+
+        * A string containing member names, separated either with spaces or
+          commas.  Values are auto-numbered from 1.
+        * An iterable of member names.  Values are auto-numbered from 1.
+        * An iterable of (member name, value) pairs.
+        * A mapping of member name -> value.
+
+        """
+        metacls = cls.__class__
+        bases = (cls, ) if type is None else (type, cls)
+        classdict = metacls.__prepare__(class_name, bases)
+
+        # special processing needed for names?
+        if isinstance(names, str):
+            names = names.replace(',', ' ').split()
+        if isinstance(names, (tuple, list)) and isinstance(names[0], str):
+            names = [(e, i) for (i, e) in enumerate(names, 1)]
+
+        # Here, names is either an iterable of (name, value) or a mapping.
+        for item in names:
+            if isinstance(item, str):
+                member_name, member_value = item, names[item]
+            else:
+                member_name, member_value = item
+            classdict[member_name] = member_value
+        enum_class = metacls.__new__(metacls, class_name, bases, classdict)
+
+        # TODO: replace the frame hack if a blessed way to know the calling
+        # module is ever developed
+        if module is None:
+            try:
+                module = sys._getframe(2).f_globals['__name__']
+            except (AttributeError, ValueError) as exc:
+                pass
+        if module is None:
+            _make_class_unpicklable(enum_class)
+        else:
+            enum_class.__module__ = module
+
+        return enum_class
+
+    @staticmethod
+    def _get_mixins_(bases):
+        """Returns the type for creating enum members, and the first inherited
+        enum class.
+
+        bases: the tuple of bases that was given to __new__
+
+        """
+        if not bases:
+            return object, Enum
+
+        # double check that we are not subclassing a class with existing
+        # enumeration members; while we're at it, see if any other data
+        # type has been mixed in so we can use the correct __new__
+        member_type = first_enum = None
+        for base in bases:
+            if  (base is not Enum and
+                    issubclass(base, Enum) and
+                    base._member_names):
+                raise TypeError("Cannot extend enumerations")
+        # base is now the last base in bases
+        if not issubclass(base, Enum):
+            raise TypeError("new enumerations must be created as "
+                    "`ClassName([mixin_type,] enum_type)`")
+
+        # get correct mix-in type (either mix-in type of Enum subclass, or
+        # first base if last base is Enum)
+        if not issubclass(bases[0], Enum):
+            member_type = bases[0]     # first data type
+            first_enum = bases[-1]  # enum type
+        else:
+            for base in bases[0].__mro__:
+                # most common: (IntEnum, int, Enum, object)
+                # possible:    (<Enum 'AutoIntEnum'>, <Enum 'IntEnum'>,
+                #               <class 'int'>, <Enum 'Enum'>,
+                #               <class 'object'>)
+                if issubclass(base, Enum):
+                    if first_enum is None:
+                        first_enum = base
+                else:
+                    if member_type is None:
+                        member_type = base
+
+        return member_type, first_enum
+
+    @staticmethod
+    def _find_new_(classdict, member_type, first_enum):
+        """Returns the __new__ to be used for creating the enum members.
+
+        classdict: the class dictionary given to __new__
+        member_type: the data type whose __new__ will be used by default
+        first_enum: enumeration to check for an overriding __new__
+
+        """
+        # now find the correct __new__, checking to see of one was defined
+        # by the user; also check earlier enum classes in case a __new__ was
+        # saved as __new_member__
+        __new__ = classdict.get('__new__', None)
+
+        # should __new__ be saved as __new_member__ later?
+        save_new = __new__ is not None
+
+        if __new__ is None:
+            # check all possibles for __new_member__ before falling back to
+            # __new__
+            for method in ('__new_member__', '__new__'):
+                for possible in (member_type, first_enum):
+                    target = getattr(possible, method, None)
+                    if target not in {
+                            None,
+                            None.__new__,
+                            object.__new__,
+                            Enum.__new__,
+                            }:
+                        __new__ = target
+                        break
+                if __new__ is not None:
+                    break
+            else:
+                __new__ = object.__new__
+
+        # if a non-object.__new__ is used then whatever value/tuple was
+        # assigned to the enum member name will be passed to __new__ and to the
+        # new enum member's __init__
+        if __new__ is object.__new__:
+            use_args = False
+        else:
+            use_args = True
+
+        return __new__, save_new, use_args
+
+
+class Enum(metaclass=EnumMeta):
+    """Generic enumeration.
+
+    Derive from this class to define new enumerations.
+
+    """
+    def __new__(cls, value):
+        # all enum instances are actually created during class construction
+        # without calling this method; this method is called by the metaclass'
+        # __call__ (i.e. Color(3) ), and by pickle
+        if type(value) is cls:
+            # For lookups like Color(Color.red)
+            return value
+        # by-value search for a matching enum member
+        # see if it's in the reverse mapping (for hashable values)
+        if value in cls._value2member_map:
+            return cls._value2member_map[value]
+        # not there, now do long search -- O(n) behavior
+        for member in cls._member_map.values():
+            if member.value == value:
+                return member
+        raise ValueError("%s is not a valid %s" % (value, cls.__name__))
+
+    def __repr__(self):
+        return "<%s.%s: %r>" % (
+                self.__class__.__name__, self._name, self._value)
+
+    def __str__(self):
+        return "%s.%s" % (self.__class__.__name__, self._name)
+
+    def __dir__(self):
+        return (['__class__', '__doc__', 'name', 'value'])
+
+    def __eq__(self, other):
+        if type(other) is self.__class__:
+            return self is other
+        return NotImplemented
+
+    def __getnewargs__(self):
+        return (self._value, )
+
+    def __hash__(self):
+        return hash(self._name)
+
+    # _RouteClassAttributeToGetattr is used to provide access to the `name`
+    # and `value` properties of enum members while keeping some measure of
+    # protection from modification, while still allowing for an enumeration
+    # to have members named `name` and `value`.  This works because enumeration
+    # members are not set directly on the enum class -- __getattr__ is
+    # used to look them up.
+
+    @_RouteClassAttributeToGetattr
+    def name(self):
+        return self._name
+
+    @_RouteClassAttributeToGetattr
+    def value(self):
+        return self._value
+
+
+class IntEnum(int, Enum):
+    """Enum where members are also (and must be) ints"""
diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py
new file mode 100644
--- /dev/null
+++ b/Lib/test/test_enum.py
@@ -0,0 +1,921 @@
+import enum
+import unittest
+from collections import OrderedDict
+from pickle import dumps, loads, PicklingError
+from enum import Enum, IntEnum
+
+# for pickle tests
+try:
+    class Stooges(Enum):
+        LARRY = 1
+        CURLY = 2
+        MOE = 3
+except Exception as exc:
+    Stooges = exc
+
+try:
+    class IntStooges(int, Enum):
+        LARRY = 1
+        CURLY = 2
+        MOE = 3
+except Exception as exc:
+    IntStooges = exc
+
+try:
+    class FloatStooges(float, Enum):
+        LARRY = 1.39
+        CURLY = 2.72
+        MOE = 3.142596
+except Exception as exc:
+    FloatStooges = exc
+
+# for pickle test and subclass tests
+try:
+    class StrEnum(str, Enum):
+        'accepts only string values'
+    class Name(StrEnum):
+        BDFL = 'Guido van Rossum'
+        FLUFL = 'Barry Warsaw'
+except Exception as exc:
+    Name = exc
+
+try:
+    Question = Enum('Question', 'who what when where why', module=__name__)
+except Exception as exc:
+    Question = exc
+
+try:
+    Answer = Enum('Answer', 'him this then there because')
+except Exception as exc:
+    Answer = exc
+
+# for doctests
+try:
+    class Fruit(Enum):
+        tomato = 1
+        banana = 2
+        cherry = 3
+except Exception:
+    pass
+
+class TestEnum(unittest.TestCase):
+    def setUp(self):
+        class Season(Enum):
+            SPRING = 1
+            SUMMER = 2
+            AUTUMN = 3
+            WINTER = 4
+        self.Season = Season
+
+    def test_enum_in_enum_out(self):
+        Season = self.Season
+        self.assertIs(Season(Season.WINTER), Season.WINTER)
+
+    def test_enum_value(self):
+        Season = self.Season
+        self.assertEqual(Season.SPRING.value, 1)
+
+    def test_intenum_value(self):
+        self.assertEqual(IntStooges.CURLY.value, 2)
+
+    def test_dir_on_class(self):
+        Season = self.Season
+        self.assertEqual(
+            set(dir(Season)),
+            set(['__class__', '__doc__', '__members__',
+                'SPRING', 'SUMMER', 'AUTUMN', 'WINTER']),
+            )
+
+    def test_dir_on_item(self):
+        Season = self.Season
+        self.assertEqual(
+            set(dir(Season.WINTER)),
+            set(['__class__', '__doc__', 'name', 'value']),
+            )
+
+    def test_enum(self):
+        Season = self.Season
+        lst = list(Season)
+        self.assertEqual(len(lst), len(Season))
+        self.assertEqual(len(Season), 4, Season)
+        self.assertEqual(
+            [Season.SPRING, Season.SUMMER, Season.AUTUMN, Season.WINTER], lst)
+
+        for i, season in enumerate('SPRING SUMMER AUTUMN WINTER'.split(), 1):
+            e = Season(i)
+            self.assertEqual(e, getattr(Season, season))
+            self.assertEqual(e.value, i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, season)
+            self.assertIn(e, Season)
+            self.assertIs(type(e), Season)
+            self.assertIsInstance(e, Season)
+            self.assertEqual(str(e), 'Season.' + season)
+            self.assertEqual(
+                    repr(e),
+                    '<Season.{0}: {1}>'.format(season, i),
+                    )
+
+    def test_value_name(self):
+        Season = self.Season
+        self.assertEqual(Season.SPRING.name, 'SPRING')
+        self.assertEqual(Season.SPRING.value, 1)
+        with self.assertRaises(AttributeError):
+            Season.SPRING.name = 'invierno'
+        with self.assertRaises(AttributeError):
+            Season.SPRING.value = 2
+
+    def test_invalid_names(self):
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                mro = 9
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _create_= 11
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _get_mixins_ = 9
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _find_new_ = 1
+        with self.assertRaises(ValueError):
+            class Wrong(Enum):
+                _any_name_ = 9
+
+    def test_contains(self):
+        Season = self.Season
+        self.assertIn(Season.AUTUMN, Season)
+        self.assertNotIn(3, Season)
+
+        val = Season(3)
+        self.assertIn(val, Season)
+
+        class OtherEnum(Enum):
+            one = 1; two = 2
+        self.assertNotIn(OtherEnum.two, Season)
+
+    def test_comparisons(self):
+        Season = self.Season
+        with self.assertRaises(TypeError):
+            Season.SPRING < Season.WINTER
+        with self.assertRaises(TypeError):
+            Season.SPRING > 4
+
+        self.assertNotEqual(Season.SPRING, 1)
+
+        class Part(Enum):
+            SPRING = 1
+            CLIP = 2
+            BARREL = 3
+
+        self.assertNotEqual(Season.SPRING, Part.SPRING)
+        with self.assertRaises(TypeError):
+            Season.SPRING < Part.CLIP
+
+    def test_enum_duplicates(self):
+        class Season(Enum):
+            SPRING = 1
+            SUMMER = 2
+            AUTUMN = FALL = 3
+            WINTER = 4
+            ANOTHER_SPRING = 1
+        lst = list(Season)
+        self.assertEqual(
+            lst,
+            [Season.SPRING, Season.SUMMER,
+             Season.AUTUMN, Season.WINTER,
+            ])
+        self.assertIs(Season.FALL, Season.AUTUMN)
+        self.assertEqual(Season.FALL.value, 3)
+        self.assertEqual(Season.AUTUMN.value, 3)
+        self.assertIs(Season(3), Season.AUTUMN)
+        self.assertIs(Season(1), Season.SPRING)
+        self.assertEqual(Season.FALL.name, 'AUTUMN')
+        self.assertEqual(
+                [k for k,v in Season.__members__.items() if v.name != k],
+                ['FALL', 'ANOTHER_SPRING'],
+                )
+
+    def test_enum_with_value_name(self):
+        class Huh(Enum):
+            name = 1
+            value = 2
+        self.assertEqual(
+            list(Huh),
+            [Huh.name, Huh.value],
+            )
+        self.assertIs(type(Huh.name), Huh)
+        self.assertEqual(Huh.name.name, 'name')
+        self.assertEqual(Huh.name.value, 1)
+    def test_hash(self):
+        Season = self.Season
+        dates = {}
+        dates[Season.WINTER] = '1225'
+        dates[Season.SPRING] = '0315'
+        dates[Season.SUMMER] = '0704'
+        dates[Season.AUTUMN] = '1031'
+        self.assertEqual(dates[Season.AUTUMN], '1031')
+
+    def test_intenum_from_scratch(self):
+        class phy(int, Enum):
+            pi = 3
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_intenum_inherited(self):
+        class IntEnum(int, Enum):
+            pass
+        class phy(IntEnum):
+            pi = 3
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_floatenum_from_scratch(self):
+        class phy(float, Enum):
+            pi = 3.141596
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_floatenum_inherited(self):
+        class FloatEnum(float, Enum):
+            pass
+        class phy(FloatEnum):
+            pi = 3.141596
+            tau = 2 * pi
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_strenum_from_scratch(self):
+        class phy(str, Enum):
+            pi = 'Pi'
+            tau = 'Tau'
+        self.assertTrue(phy.pi < phy.tau)
+
+    def test_strenum_inherited(self):
+        class StrEnum(str, Enum):
+            pass
+        class phy(StrEnum):
+            pi = 'Pi'
+            tau = 'Tau'
+        self.assertTrue(phy.pi < phy.tau)
+
+
+    def test_intenum(self):
+        class WeekDay(IntEnum):
+            SUNDAY = 1
+            MONDAY = 2
+            TUESDAY = 3
+            WEDNESDAY = 4
+            THURSDAY = 5
+            FRIDAY = 6
+            SATURDAY = 7
+
+        self.assertEqual(['a', 'b', 'c'][WeekDay.MONDAY], 'c')
+        self.assertEqual([i for i in range(WeekDay.TUESDAY)], [0, 1, 2])
+
+        lst = list(WeekDay)
+        self.assertEqual(len(lst), len(WeekDay))
+        self.assertEqual(len(WeekDay), 7)
+        target = 'SUNDAY MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY'
+        target = target.split()
+        for i, weekday in enumerate(target, 1):
+            e = WeekDay(i)
+            self.assertEqual(e, i)
+            self.assertEqual(int(e), i)
+            self.assertEqual(e.name, weekday)
+            self.assertIn(e, WeekDay)
+            self.assertEqual(lst.index(e)+1, i)
+            self.assertTrue(0 < e < 8)
+            self.assertIs(type(e), WeekDay)
+            self.assertIsInstance(e, int)
+            self.assertIsInstance(e, Enum)
+
+    def test_intenum_duplicates(self):
+        class WeekDay(IntEnum):
+            SUNDAY = 1
+            MONDAY = 2
+            TUESDAY = TEUSDAY = 3
+            WEDNESDAY = 4
+            THURSDAY = 5
+            FRIDAY = 6
+            SATURDAY = 7
+        self.assertIs(WeekDay.TEUSDAY, WeekDay.TUESDAY)
+        self.assertEqual(WeekDay(3).name, 'TUESDAY')
+        self.assertEqual([k for k,v in WeekDay.__members__.items()
+                if v.name != k], ['TEUSDAY', ])
+
+    def test_pickle_enum(self):
+        if isinstance(Stooges, Exception):
+            raise Stooges
+        self.assertIs(Stooges.CURLY, loads(dumps(Stooges.CURLY)))
+        self.assertIs(Stooges, loads(dumps(Stooges)))
+
+    def test_pickle_int(self):
+        if isinstance(IntStooges, Exception):
+            raise IntStooges
+        self.assertIs(IntStooges.CURLY, loads(dumps(IntStooges.CURLY)))
+        self.assertIs(IntStooges, loads(dumps(IntStooges)))
+
+    def test_pickle_float(self):
+        if isinstance(FloatStooges, Exception):
+            raise FloatStooges
+        self.assertIs(FloatStooges.CURLY, loads(dumps(FloatStooges.CURLY)))
+        self.assertIs(FloatStooges, loads(dumps(FloatStooges)))
+
+    def test_pickle_enum_function(self):
+        if isinstance(Answer, Exception):
+            raise Answer
+        self.assertIs(Answer.him, loads(dumps(Answer.him)))
+        self.assertIs(Answer, loads(dumps(Answer)))
+
+    def test_pickle_enum_function_with_module(self):
+        if isinstance(Question, Exception):
+            raise Question
+        self.assertIs(Question.who, loads(dumps(Question.who)))
+        self.assertIs(Question, loads(dumps(Question)))
+
+    def test_exploding_pickle(self):
+        BadPickle = Enum('BadPickle', 'dill sweet bread-n-butter')
+        enum._make_class_unpicklable(BadPickle)
+        globals()['BadPickle'] = BadPickle
+        with self.assertRaises(TypeError):
+            dumps(BadPickle.dill)
+        with self.assertRaises(PicklingError):
+            dumps(BadPickle)
+
+    def test_string_enum(self):
+        class SkillLevel(str, Enum):
+            master = 'what is the sound of one hand clapping?'
+            journeyman = 'why did the chicken cross the road?'
+            apprentice = 'knock, knock!'
+        self.assertEqual(SkillLevel.apprentice, 'knock, knock!')
+
+    def test_getattr_getitem(self):
+        class Period(Enum):
+            morning = 1
+            noon = 2
+            evening = 3
+            night = 4
+        self.assertIs(Period(2), Period.noon)
+        self.assertIs(getattr(Period, 'night'), Period.night)
+        self.assertIs(Period['morning'], Period.morning)
+
+    def test_getattr_dunder(self):
+        Season = self.Season
+        self.assertTrue(getattr(Season, '__eq__'))
+
+    def test_iteration_order(self):
+        class Season(Enum):
+            SUMMER = 2
+            WINTER = 4
+            AUTUMN = 3
+            SPRING = 1
+        self.assertEqual(
+                list(Season),
+                [Season.SUMMER, Season.WINTER, Season.AUTUMN, Season.SPRING],
+                )
+
+    def test_programatic_function_string(self):
+        SummerMonth = Enum('SummerMonth', 'june july august')
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_string_list(self):
+        SummerMonth = Enum('SummerMonth', ['june', 'july', 'august'])
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_iterable(self):
+        SummerMonth = Enum(
+                'SummerMonth',
+                (('june', 1), ('july', 2), ('august', 3))
+                )
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_from_dict(self):
+        SummerMonth = Enum(
+                'SummerMonth',
+                OrderedDict((('june', 1), ('july', 2), ('august', 3)))
+                )
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(int(e.value), i)
+            self.assertNotEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_type(self):
+        SummerMonth = Enum('SummerMonth', 'june july august', type=int)
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_programatic_function_type_from_subclass(self):
+        SummerMonth = IntEnum('SummerMonth', 'june july august')
+        lst = list(SummerMonth)
+        self.assertEqual(len(lst), len(SummerMonth))
+        self.assertEqual(len(SummerMonth), 3, SummerMonth)
+        self.assertEqual(
+                [SummerMonth.june, SummerMonth.july, SummerMonth.august],
+                lst,
+                )
+        for i, month in enumerate('june july august'.split(), 1):
+            e = SummerMonth(i)
+            self.assertEqual(e, i)
+            self.assertEqual(e.name, month)
+            self.assertIn(e, SummerMonth)
+            self.assertIs(type(e), SummerMonth)
+
+    def test_subclassing(self):
+        if isinstance(Name, Exception):
+            raise Name
+        self.assertEqual(Name.BDFL, 'Guido van Rossum')
+        self.assertTrue(Name.BDFL, Name('Guido van Rossum'))
+        self.assertIs(Name.BDFL, getattr(Name, 'BDFL'))
+        self.assertIs(Name.BDFL, loads(dumps(Name.BDFL)))
+
+    def test_extending(self):
+        class Color(Enum):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(TypeError):
+            class MoreColor(Color):
+                cyan = 4
+                magenta = 5
+                yellow = 6
+
+    def test_exclude_methods(self):
+        class whatever(Enum):
+            this = 'that'
+            these = 'those'
+            def really(self):
+                return 'no, not %s' % self.value
+        self.assertIsNot(type(whatever.really), whatever)
+        self.assertEqual(whatever.this.really(), 'no, not that')
+
+    def test_overwrite_enums(self):
+        class Why(Enum):
+            question = 1
+            answer = 2
+            propisition = 3
+            def question(self):
+                print(42)
+        self.assertIsNot(type(Why.question), Why)
+        self.assertNotIn(Why.question, Why._member_names)
+        self.assertNotIn(Why.question, Why)
+
+    def test_wrong_inheritance_order(self):
+        with self.assertRaises(TypeError):
+            class Wrong(Enum, str):
+                NotHere = 'error before this point'
+
+    def test_intenum_transitivity(self):
+        class number(IntEnum):
+            one = 1
+            two = 2
+            three = 3
+        class numero(IntEnum):
+            uno = 1
+            dos = 2
+            tres = 3
+        self.assertEqual(number.one, numero.uno)
+        self.assertEqual(number.two, numero.dos)
+        self.assertEqual(number.three, numero.tres)
+
+    def test_wrong_enum_in_call(self):
+        class Monochrome(Enum):
+            black = 0
+            white = 1
+        class Gender(Enum):
+            male = 0
+            female = 1
+        self.assertRaises(ValueError, Monochrome, Gender.male)
+
+    def test_wrong_enum_in_mixed_call(self):
+        class Monochrome(IntEnum):
+            black = 0
+            white = 1
+        class Gender(Enum):
+            male = 0
+            female = 1
+        self.assertRaises(ValueError, Monochrome, Gender.male)
+
+    def test_mixed_enum_in_call_1(self):
+        class Monochrome(IntEnum):
+            black = 0
+            white = 1
+        class Gender(IntEnum):
+            male = 0
+            female = 1
+        self.assertIs(Monochrome(Gender.female), Monochrome.white)
+
+    def test_mixed_enum_in_call_2(self):
+        class Monochrome(Enum):
+            black = 0
+            white = 1
+        class Gender(IntEnum):
+            male = 0
+            female = 1
+        self.assertIs(Monochrome(Gender.male), Monochrome.black)
+
+    def test_flufl_enum(self):
+        class Fluflnum(Enum):
+            def __int__(self):
+                return int(self.value)
+        class MailManOptions(Fluflnum):
+            option1 = 1
+            option2 = 2
+            option3 = 3
+        self.assertEqual(int(MailManOptions.option1), 1)
+
+    def test_no_such_enum_member(self):
+        class Color(Enum):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(ValueError):
+            Color(4)
+        with self.assertRaises(KeyError):
+            Color['chartreuse']
+
+    def test_new_repr(self):
+        class Color(Enum):
+            red = 1
+            green = 2
+            blue = 3
+            def __repr__(self):
+                return "don't you just love shades of %s?" % self.name
+        self.assertEqual(
+                repr(Color.blue),
+                "don't you just love shades of blue?",
+                )
+
+    def test_inherited_repr(self):
+        class MyEnum(Enum):
+            def __repr__(self):
+                return "My name is %s." % self.name
+        class MyIntEnum(int, MyEnum):
+            this = 1
+            that = 2
+            theother = 3
+        self.assertEqual(repr(MyIntEnum.that), "My name is that.")
+
+    def test_multiple_mixin_mro(self):
+        class auto_enum(type(Enum)):
+            def __new__(metacls, cls, bases, classdict):
+                temp = type(classdict)()
+                names = set(classdict._member_names)
+                i = 0
+                for k in classdict._member_names:
+                    v = classdict[k]
+                    if v is Ellipsis:
+                        v = i
+                    else:
+                        i = v
+                    i += 1
+                    temp[k] = v
+                for k, v in classdict.items():
+                    if k not in names:
+                        temp[k] = v
+                return super(auto_enum, metacls).__new__(
+                        metacls, cls, bases, temp)
+
+        class AutoNumberedEnum(Enum, metaclass=auto_enum):
+            pass
+
+        class AutoIntEnum(IntEnum, metaclass=auto_enum):
+            pass
+
+        class TestAutoNumber(AutoNumberedEnum):
+            a = ...
+            b = 3
+            c = ...
+
+        class TestAutoInt(AutoIntEnum):
+            a = ...
+            b = 3
+            c = ...
+
+    def test_subclasses_with_getnewargs(self):
+        class NamedInt(int):
+            def __new__(cls, *args):
+                _args = args
+                name, *args = args
+                if len(args) == 0:
+                    raise TypeError("name and value must be specified")
+                self = int.__new__(cls, *args)
+                self._intname = name
+                self._args = _args
+                return self
+            def __getnewargs__(self):
+                return self._args
+            @property
+            def __name__(self):
+                return self._intname
+            def __repr__(self):
+                # repr() is updated to include the name and type info
+                return "{}({!r}, {})".format(type(self).__name__,
+                                             self.__name__,
+                                             int.__repr__(self))
+            def __str__(self):
+                # str() is unchanged, even if it relies on the repr() fallback
+                base = int
+                base_str = base.__str__
+                if base_str.__objclass__ is object:
+                    return base.__repr__(self)
+                return base_str(self)
+            # for simplicity, we only define one operator that
+            # propagates expressions
+            def __add__(self, other):
+                temp = int(self) + int( other)
+                if isinstance(self, NamedInt) and isinstance(other, NamedInt):
+                    return NamedInt(
+                        '({0} + {1})'.format(self.__name__, other.__name__),
+                        temp )
+                else:
+                    return temp
+
+        class NEI(NamedInt, Enum):
+            x = ('the-x', 1)
+            y = ('the-y', 2)
+
+        self.assertIs(NEI.__new__, Enum.__new__)
+        self.assertEqual(repr(NEI.x + NEI.y), "NamedInt('(the-x + the-y)', 3)")
+        globals()['NamedInt'] = NamedInt
+        globals()['NEI'] = NEI
+        NI5 = NamedInt('test', 5)
+        self.assertEqual(NI5, 5)
+        self.assertEqual(loads(dumps(NI5)), 5)
+        self.assertEqual(NEI.y.value, 2)
+        self.assertIs(loads(dumps(NEI.y)), NEI.y)
+
+    def test_subclasses_without_getnewargs(self):
+        class NamedInt(int):
+            def __new__(cls, *args):
+                _args = args
+                name, *args = args
+                if len(args) == 0:
+                    raise TypeError("name and value must be specified")
+                self = int.__new__(cls, *args)
+                self._intname = name
+                self._args = _args
+                return self
+            @property
+            def __name__(self):
+                return self._intname
+            def __repr__(self):
+                # repr() is updated to include the name and type info
+                return "{}({!r}, {})".format(type(self).__name__,
+                                             self.__name__,
+                                             int.__repr__(self))
+            def __str__(self):
+                # str() is unchanged, even if it relies on the repr() fallback
+                base = int
+                base_str = base.__str__
+                if base_str.__objclass__ is object:
+                    return base.__repr__(self)
+                return base_str(self)
+            # for simplicity, we only define one operator that
+            # propagates expressions
+            def __add__(self, other):
+                temp = int(self) + int( other)
+                if isinstance(self, NamedInt) and isinstance(other, NamedInt):
+                    return NamedInt(
+                        '({0} + {1})'.format(self.__name__, other.__name__),
+                        temp )
+                else:
+                    return temp
+
+        class NEI(NamedInt, Enum):
+            x = ('the-x', 1)
+            y = ('the-y', 2)
+
+        self.assertIs(NEI.__new__, Enum.__new__)
+        self.assertEqual(repr(NEI.x + NEI.y), "NamedInt('(the-x + the-y)', 3)")
+        globals()['NamedInt'] = NamedInt
+        globals()['NEI'] = NEI
+        NI5 = NamedInt('test', 5)
+        self.assertEqual(NI5, 5)
+        self.assertEqual(NEI.y.value, 2)
+        with self.assertRaises(TypeError):
+            dumps(NEI.x)
+        with self.assertRaises(PicklingError):
+            dumps(NEI)
+
+    def test_tuple_subclass(self):
+        class SomeTuple(tuple, Enum):
+            first = (1, 'for the money')
+            second = (2, 'for the show')
+            third = (3, 'for the music')
+        self.assertIs(type(SomeTuple.first), SomeTuple)
+        self.assertIsInstance(SomeTuple.second, tuple)
+        self.assertEqual(SomeTuple.third, (3, 'for the music'))
+        globals()['SomeTuple'] = SomeTuple
+        self.assertIs(loads(dumps(SomeTuple.first)), SomeTuple.first)
+
+    def test_duplicate_values_give_unique_enum_items(self):
+        class AutoNumber(Enum):
+            first = ()
+            second = ()
+            third = ()
+            def __new__(cls):
+                value = len(cls.__members__) + 1
+                obj = object.__new__(cls)
+                obj._value = value
+                return obj
+            def __int__(self):
+                return int(self._value)
+        self.assertEqual(
+                list(AutoNumber),
+                [AutoNumber.first, AutoNumber.second, AutoNumber.third],
+                )
+        self.assertEqual(int(AutoNumber.second), 2)
+        self.assertIs(AutoNumber(1), AutoNumber.first)
+
+    def test_inherited_new_from_enhanced_enum(self):
+        class AutoNumber(Enum):
+            def __new__(cls):
+                value = len(cls.__members__) + 1
+                obj = object.__new__(cls)
+                obj._value = value
+                return obj
+            def __int__(self):
+                return int(self._value)
+        class Color(AutoNumber):
+            red = ()
+            green = ()
+            blue = ()
+        self.assertEqual(list(Color), [Color.red, Color.green, Color.blue])
+        self.assertEqual(list(map(int, Color)), [1, 2, 3])
+
+    def test_inherited_new_from_mixed_enum(self):
+        class AutoNumber(IntEnum):
+            def __new__(cls):
+                value = len(cls.__members__) + 1
+                obj = int.__new__(cls, value)
+                obj._value = value
+                return obj
+        class Color(AutoNumber):
+            red = ()
+            green = ()
+            blue = ()
+        self.assertEqual(list(Color), [Color.red, Color.green, Color.blue])
+        self.assertEqual(list(map(int, Color)), [1, 2, 3])
+
+    def test_ordered_mixin(self):
+        class OrderedEnum(Enum):
+            def __ge__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value >= other._value
+                return NotImplemented
+            def __gt__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value > other._value
+                return NotImplemented
+            def __le__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value <= other._value
+                return NotImplemented
+            def __lt__(self, other):
+                if self.__class__ is other.__class__:
+                    return self._value < other._value
+                return NotImplemented
+        class Grade(OrderedEnum):
+            A = 5
+            B = 4
+            C = 3
+            D = 2
+            F = 1
+        self.assertGreater(Grade.A, Grade.B)
+        self.assertLessEqual(Grade.F, Grade.C)
+        self.assertLess(Grade.D, Grade.A)
+        self.assertGreaterEqual(Grade.B, Grade.B)
+    def test_extending2(self):
+        class Shade(Enum):
+            def shade(self):
+                print(self.name)
+        class Color(Shade):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(TypeError):
+            class MoreColor(Color):
+                cyan = 4
+                magenta = 5
+                yellow = 6
+
+    def test_extending3(self):
+        class Shade(Enum):
+            def shade(self):
+                return self.name
+        class Color(Shade):
+            def hex(self):
+                return '%s hexlified!' % self.value
+        class MoreColor(Color):
+            cyan = 4
+            magenta = 5
+            yellow = 6
+        self.assertEqual(MoreColor.magenta.hex(), '5 hexlified!')
+
+
+    def test_no_duplicates(self):
+        class UniqueEnum(Enum):
+            def __init__(self, *args):
+                cls = self.__class__
+                if any(self.value == e.value for e in cls):
+                    a = self.name
+                    e = cls(self.value).name
+                    raise ValueError(
+                            "aliases not allowed in UniqueEnum:  %r --> %r"
+                            % (a, e)
+                            )
+        class Color(UniqueEnum):
+            red = 1
+            green = 2
+            blue = 3
+        with self.assertRaises(ValueError):
+            class Color(UniqueEnum):
+                red = 1
+                green = 2
+                blue = 3
+                grene = 2
+
+    def test_init(self):
+        class Planet(Enum):
+            MERCURY = (3.303e+23, 2.4397e6)
+            VENUS   = (4.869e+24, 6.0518e6)
+            EARTH   = (5.976e+24, 6.37814e6)
+            MARS    = (6.421e+23, 3.3972e6)
+            JUPITER = (1.9e+27,   7.1492e7)
+            SATURN  = (5.688e+26, 6.0268e7)
+            URANUS  = (8.686e+25, 2.5559e7)
+            NEPTUNE = (1.024e+26, 2.4746e7)
+            def __init__(self, mass, radius):
+                self.mass = mass       # in kilograms
+                self.radius = radius   # in meters
+            @property
+            def surface_gravity(self):
+                # universal gravitational constant  (m3 kg-1 s-2)
+                G = 6.67300E-11
+                return G * self.mass / (self.radius * self.radius)
+        self.assertEqual(round(Planet.EARTH.surface_gravity, 2), 9.80)
+        self.assertEqual(Planet.EARTH.value, (5.976e+24, 6.37814e6))
+
+
+if __name__ == '__main__':
+    unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- Issue #17907: Document imp.new_module() as deprecated in favour of
+  types.ModuleType.
+
 - Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document as deprecated
   imp.get_magic().
 
@@ -378,6 +381,8 @@
 
 - Implement PEP 443 "Single-dispatch generic functions".
 
+- Implement PEP 435 "Adding an Enum type to the Python standard library".
+
 
 Tests
 -----

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 03:59:23 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 03:59:23 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_markup_of_the_synopsis?=
 =?utf-8?q?_along_with_changing_to_state_what_is_in_the?=
Message-ID: <3bXMLg1wRlzPqc@mail.python.org>

http://hg.python.org/cpython/rev/704a05135efe
changeset:   84133:704a05135efe
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 21:59:16 2013 -0400
summary:
  Fix markup of the synopsis along with changing to state what is in the
module and not as a definition of what an enumeration is.

files:
  Doc/library/enum.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
--- a/Doc/library/enum.rst
+++ b/Doc/library/enum.rst
@@ -2,8 +2,8 @@
 ========================================
 
 .. module:: enum
-.. :synopsis: enumerations are sets of symbolic names bound to unique, constant
-  values.
+   :synopsis: Implementation of an enumeration class.
+
 .. :moduleauthor:: Ethan Furman <ethan at stoneleaf.us>
 .. :sectionauthor:: Barry Warsaw <barry at python.org>,
 .. :sectionauthor:: Eli Bendersky <eliben at gmail.com>,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 04:24:38 2013
From: python-checkins at python.org (victor.stinner)
Date: Sat, 15 Jun 2013 04:24:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_Add_new_APIs_to_cu?=
	=?utf-8?q?stomize_memory_allocators?=
Message-ID: <3bXMvp596rzRMx@mail.python.org>

http://hg.python.org/peps/rev/2c07212981f7
changeset:   4925:2c07212981f7
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sat Jun 15 04:24:25 2013 +0200
summary:
  PEP 445: Add new APIs to customize memory allocators

files:
  pep-0445.txt |  174 +++++++++++++++++++++++++++++++++++++++
  1 files changed, 174 insertions(+), 0 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
new file mode 100644
--- /dev/null
+++ b/pep-0445.txt
@@ -0,0 +1,174 @@
+PEP: 445
+Title: Add new APIs to customize memory allocators
+Version: $Revision$
+Last-Modified: $Date$
+Author: Victor Stinner <victor.stinner at gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 15-june-2013
+Python-Version: 3.4
+
+Abstract
+========
+
+Add new APIs to customize memory allocators
+
+
+Rationale
+=========
+
+Use cases:
+
+* Application embedding Python wanting to use a custom memory allocator
+  to allocate all Python memory somewhere else or with a different algorithm
+* Python running on embedded devices with low memory and slow CPU.
+  A custom memory allocator may be required to use efficiently the memory
+  and/or to be able to use all memory of the device.
+* Debug tool to track memory leaks
+* Debug tool to detect buffer underflow, buffer overflow and misuse
+  of Python allocator APIs
+* Debug tool to inject bugs, simulate out of memory for example
+
+API:
+
+* Setup a custom memory allocator for all memory allocated by Python
+* Hook memory allocator functions to call extra code before and/or after
+  the underlying allocator function
+
+
+Proposal
+========
+
+* Add a new ``PyMemAllocators`` structure
+
+* Add new GIL-free memory allocator functions:
+
+  - ``PyMem_RawMalloc()``
+  - ``PyMem_RawRealloc()``
+  - ``PyMem_RawFree()``
+
+* Add new functions to get and set memory allocators:
+
+  - ``PyMem_GetRawAllocators()``, ``PyMem_SetRawAllocators()``
+  - ``PyMem_GetAllocators()``, ``PyMem_SetAllocators()``
+  - ``PyObject_GetAllocators()``, ``PyObject_SetAllocators()``
+  - ``_PyObject_GetArenaAllocators()``, ``_PyObject_SetArenaAllocators()``
+
+* Add a new function to setup debug hooks after memory allocators were
+  replaced:
+
+  - ``PyMem_SetupDebugHooks()``
+
+* ``PyObject_Malloc()`` now falls back on ``PyMem_Malloc()`` instead of
+  ``malloc()`` if size is bigger than ``SMALL_REQUEST_THRESHOLD``, and
+  ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of
+  ``realloc()``
+
+* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call ``malloc()`` and
+  ``realloc()``, instead of calling ``PyObject_Malloc()`` and
+  ``PyObject_Realloc()`` in debug mode
+
+
+Performances
+============
+
+The Python benchmarks suite (-b 2n3): some tests are 1.04x faster, some tests
+are 1.04 slower, significant is between 115 and -191. I don't understand these
+output, but I guess that the overhead cannot be seen with such test.
+
+pybench: "+0.1%" (diff between -4.9% and +5.6%).
+
+Full output attached to the issue #3329.
+
+
+Alternatives
+============
+
+Only one get and one set function
+---------------------------------
+
+Replace the 6 functions:
+
+* ``PyMem_GetRawAllocators()``
+* ``PyMem_GetAllocators()``
+* ``PyObject_GetAllocators()``
+* ``PyMem_SetRawAllocators(allocators)``
+* ``PyMem_SetAllocators(allocators)``
+, ``PyObject_SetAllocators(allocators)``
+
+with 2 functions with an additional *domain* argument:
+
+* ``PyMem_GetAllocators(domain)``
+* ``PyMem_SetAllocators(domain, allocators)``
+
+
+Setup Builtin Debug Hooks
+-------------------------
+
+To be able to use Python debug functions (like ``_PyMem_DebugMalloc()``) even
+when a custom memory allocator is set, an environment variable
+``PYDEBUGMALLOC`` can be added to set these debug function hooks, instead of
+the new function ``PyMem_SetupDebugHooks()``.
+
+
+Use macros to get customizable allocators
+-----------------------------------------
+
+To have no overhead in the default configuration, customizable allocators would
+be an optional feature enabled by a configuration option or by macros.
+
+
+Pass the C filename and line number
+-----------------------------------
+
+Use C macros using ``__FILE__`` and ``__LINE__`` to get the C filename
+and line number of a memory allocation.
+
+
+No context argument
+-------------------
+
+Simplify the signature of allocator functions, remove the context argument:
+
+* ``void* malloc(size_t size)``
+* ``void* realloc(void *ptr, size_t new_size)``
+* ``void free(void *ptr)``
+
+The context is a convinient way to reuse the same allocator for different APIs
+(ex: PyMem and PyObject).
+
+
+PyMem_Malloc() GIL-free
+-----------------------
+
+There is no real reason to require the GIL when calling PyMem_Malloc().
+
+
+CCP API
+-------
+
+XXX To be done (Kristj?n Valur J?nsson) XXX
+
+
+Links
+=====
+
+Memory allocators:
+
+* `Issue #3329: Add new APIs to customize memory allocators
+  <http://bugs.python.org/issue3329>`_
+* `pytracemalloc
+  <https://pypi.python.org/pypi/pytracemalloc>`_
+* `Meliae: Python Memory Usage Analyzer
+  <https://pypi.python.org/pypi/meliae>`_
+* `Guppy-PE: umbrella package combining Heapy and GSL
+  <http://guppy-pe.sourceforge.net/>`_
+* `PySizer (developed for Python 2.4)
+  <http://pysizer.8325.org/>`_
+
+Other:
+
+* `Python benchmark suite
+  <http://hg.python.org/benchmarks>`_
+

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 15 04:25:53 2013
From: python-checkins at python.org (victor.stinner)
Date: Sat, 15 Jun 2013 04:25:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_fix_typo?=
Message-ID: <3bXMxF64sCz7Ljw@mail.python.org>

http://hg.python.org/peps/rev/2b7d3c9206b5
changeset:   4926:2b7d3c9206b5
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sat Jun 15 04:25:42 2013 +0200
summary:
  PEP 445: fix typo

files:
  pep-0445.txt |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -95,7 +95,7 @@
 * ``PyObject_GetAllocators()``
 * ``PyMem_SetRawAllocators(allocators)``
 * ``PyMem_SetAllocators(allocators)``
-, ``PyObject_SetAllocators(allocators)``
+* ``PyObject_SetAllocators(allocators)``
 
 with 2 functions with an additional *domain* argument:
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 15 04:26:39 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 04:26:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317907=3A_touch_up?=
 =?utf-8?q?_the_code_for_imp=2Enew=5Fmodule=28=29=2E?=
Message-ID: <3bXMy76LPDzMwL@mail.python.org>

http://hg.python.org/cpython/rev/9cacdb9d0c59
changeset:   84134:9cacdb9d0c59
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 22:26:30 2013 -0400
summary:
  Issue #17907: touch up the code for imp.new_module().

files:
  Doc/library/imp.rst                  |     6 +
  Doc/library/importlib.rst            |    30 +
  Lib/imp.py                           |    13 +-
  Lib/importlib/_bootstrap.py          |    11 +-
  Lib/importlib/util.py                |     2 +
  Lib/test/test_import.py              |     1 -
  Lib/test/test_importlib/test_util.py |   116 +
  Python/importlib.h                   |  7077 ++++++-------
  8 files changed, 3699 insertions(+), 3557 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -205,6 +205,9 @@
       If :attr:`sys.implementation.cache_tag` is ``None``, then
       :exc:`NotImplementedError` is raised.
 
+   .. deprecated:: 3.4
+      Use :func:`importlib.util.cache_from_source` instead.
+
 
 .. function:: source_from_cache(path)
 
@@ -220,6 +223,9 @@
       Raise :exc:`NotImplementedError` when
       :attr:`sys.implementation.cache_tag` is not defined.
 
+   .. deprecated:: 3.4
+      Use :func:`importlib.util.source_from_cache` instead.
+
 
 .. function:: get_tag()
 
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -886,6 +886,36 @@
 
    .. versionadded:: 3.4
 
+.. function:: cache_from_source(path, debug_override=None)
+
+   Return the :pep:`3147` path to the byte-compiled file associated with the
+   source *path*.  For example, if *path* is ``/foo/bar/baz.py`` the return
+   value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
+   The ``cpython-32`` string comes from the current magic tag (see
+   :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
+   :exc:`NotImplementedError` will be raised).  The returned path will end in
+   ``.pyc`` when ``__debug__`` is True or ``.pyo`` for an optimized Python
+   (i.e. ``__debug__`` is False).  By passing in True or False for
+   *debug_override* you can override the system's value for ``__debug__`` for
+   extension selection.
+
+   *path* need not exist.
+
+   .. versionadded:: 3.4
+
+
+.. function:: source_from_cache(path)
+
+   Given the *path* to a :pep:`3147` file name, return the associated source code
+   file path.  For example, if *path* is
+   ``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be
+   ``/foo/bar/baz.py``.  *path* need not exist, however if it does not conform
+   to :pep:`3147` format, a ``ValueError`` is raised. If
+   :attr:`sys.implementation.cache_tag` is not defined,
+   :exc:`NotImplementedError` is raised.
+
+   .. versionadded:: 3.4
+
 .. function:: resolve_name(name, package)
 
    Resolve a relative module name to an absolute one.
diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -17,7 +17,6 @@
     load_dynamic = None
 
 # Directly exposed by this module
-from importlib._bootstrap import new_module
 from importlib._bootstrap import cache_from_source, source_from_cache
 
 
@@ -28,6 +27,7 @@
 import os
 import sys
 import tokenize
+import types
 import warnings
 
 
@@ -44,6 +44,17 @@
 IMP_HOOK = 9
 
 
+def new_module(name):
+    """**DEPRECATED**
+
+    Create a new module.
+
+    The module is not entered into sys.modules.
+
+    """
+    return types.ModuleType(name)
+
+
 def get_magic():
     """**DEPRECATED**
 
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -121,15 +121,6 @@
 _code_type = type(_wrap.__code__)
 
 
-def new_module(name):
-    """Create a new module.
-
-    The module is not entered into sys.modules.
-
-    """
-    return type(_io)(name)
-
-
 # Module-level locking ########################################################
 
 # A dict mapping module names to weakrefs of _ModuleLock instances
@@ -509,7 +500,7 @@
             # This must be done before open() is called as the 'io' module
             # implicitly imports 'locale' and would otherwise trigger an
             # infinite loop.
-            self._module = new_module(self._name)
+            self._module = type(_io)(self._name)
             # This must be done before putting the module in sys.modules
             # (otherwise an optimization shortcut in import.c becomes wrong)
             self._module.__initializing__ = True
diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py
--- a/Lib/importlib/util.py
+++ b/Lib/importlib/util.py
@@ -1,9 +1,11 @@
 """Utility code for constructing importers, etc."""
 
 from ._bootstrap import MAGIC_NUMBER
+from ._bootstrap import cache_from_source
 from ._bootstrap import module_to_load
 from ._bootstrap import set_loader
 from ._bootstrap import set_package
+from ._bootstrap import source_from_cache
 from ._bootstrap import _resolve_name
 
 import functools
diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -859,7 +859,6 @@
         from importlib import machinery
         mod = sys.modules['_frozen_importlib']
         self.assertIs(machinery.FileFinder, mod.FileFinder)
-        self.assertIs(imp.new_module, mod.new_module)
 
 
 class ImportTracebackTests(unittest.TestCase):
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -1,6 +1,7 @@
 from importlib import util
 from . import util as test_util
 import imp
+import os
 import sys
 from test import support
 import types
@@ -324,5 +325,120 @@
         self.assertTrue(util.MAGIC_NUMBER.endswith(b'\r\n'))
 
 
+class PEP3147Tests(unittest.TestCase):
+    """Tests of PEP 3147-related functions:
+    cache_from_source and source_from_cache.
+
+    """
+
+    tag = imp.get_tag()
+
+    @unittest.skipUnless(sys.implementation.cache_tag is not None,
+                         'requires sys.implementation.cache_tag not be None')
+    def test_cache_from_source(self):
+        # Given the path to a .py file, return the path to its PEP 3147
+        # defined .pyc file (i.e. under __pycache__).
+        path = os.path.join('foo', 'bar', 'baz', 'qux.py')
+        expect = os.path.join('foo', 'bar', 'baz', '__pycache__',
+                              'qux.{}.pyc'.format(self.tag))
+        self.assertEqual(util.cache_from_source(path, True), expect)
+
+    def test_cache_from_source_no_cache_tag(self):
+        # No cache tag means NotImplementedError.
+        with support.swap_attr(sys.implementation, 'cache_tag', None):
+            with self.assertRaises(NotImplementedError):
+                util.cache_from_source('whatever.py')
+
+    def test_cache_from_source_no_dot(self):
+        # Directory with a dot, filename without dot.
+        path = os.path.join('foo.bar', 'file')
+        expect = os.path.join('foo.bar', '__pycache__',
+                              'file{}.pyc'.format(self.tag))
+        self.assertEqual(util.cache_from_source(path, True), expect)
+
+    def test_cache_from_source_optimized(self):
+        # Given the path to a .py file, return the path to its PEP 3147
+        # defined .pyo file (i.e. under __pycache__).
+        path = os.path.join('foo', 'bar', 'baz', 'qux.py')
+        expect = os.path.join('foo', 'bar', 'baz', '__pycache__',
+                              'qux.{}.pyo'.format(self.tag))
+        self.assertEqual(util.cache_from_source(path, False), expect)
+
+    def test_cache_from_source_cwd(self):
+        path = 'foo.py'
+        expect = os.path.join('__pycache__', 'foo.{}.pyc'.format(self.tag))
+        self.assertEqual(util.cache_from_source(path, True), expect)
+
+    def test_cache_from_source_override(self):
+        # When debug_override is not None, it can be any true-ish or false-ish
+        # value.
+        path = os.path.join('foo', 'bar', 'baz.py')
+        partial_expect = os.path.join('foo', 'bar', '__pycache__',
+                                      'baz.{}.py'.format(self.tag))
+        self.assertEqual(util.cache_from_source(path, []), partial_expect + 'o')
+        self.assertEqual(util.cache_from_source(path, [17]),
+                         partial_expect + 'c')
+        # However if the bool-ishness can't be determined, the exception
+        # propagates.
+        class Bearish:
+            def __bool__(self): raise RuntimeError
+        with self.assertRaises(RuntimeError):
+            util.cache_from_source('/foo/bar/baz.py', Bearish())
+
+    @unittest.skipUnless(os.sep == '\\' and os.altsep == '/',
+                     'test meaningful only where os.altsep is defined')
+    def test_sep_altsep_and_sep_cache_from_source(self):
+        # Windows path and PEP 3147 where sep is right of altsep.
+        self.assertEqual(
+            util.cache_from_source('\\foo\\bar\\baz/qux.py', True),
+            '\\foo\\bar\\baz\\__pycache__\\qux.{}.pyc'.format(self.tag))
+
+    @unittest.skipUnless(sys.implementation.cache_tag is not None,
+                         'requires sys.implementation.cache_tag to not be '
+                         'None')
+    def test_source_from_cache(self):
+        # Given the path to a PEP 3147 defined .pyc file, return the path to
+        # its source.  This tests the good path.
+        path = os.path.join('foo', 'bar', 'baz', '__pycache__',
+                            'qux.{}.pyc'.format(self.tag))
+        expect = os.path.join('foo', 'bar', 'baz', 'qux.py')
+        self.assertEqual(util.source_from_cache(path), expect)
+
+    def test_source_from_cache_no_cache_tag(self):
+        # If sys.implementation.cache_tag is None, raise NotImplementedError.
+        path = os.path.join('blah', '__pycache__', 'whatever.pyc')
+        with support.swap_attr(sys.implementation, 'cache_tag', None):
+            with self.assertRaises(NotImplementedError):
+                util.source_from_cache(path)
+
+    def test_source_from_cache_bad_path(self):
+        # When the path to a pyc file is not in PEP 3147 format, a ValueError
+        # is raised.
+        self.assertRaises(
+            ValueError, util.source_from_cache, '/foo/bar/bazqux.pyc')
+
+    def test_source_from_cache_no_slash(self):
+        # No slashes at all in path -> ValueError
+        self.assertRaises(
+            ValueError, util.source_from_cache, 'foo.cpython-32.pyc')
+
+    def test_source_from_cache_too_few_dots(self):
+        # Too few dots in final path component -> ValueError
+        self.assertRaises(
+            ValueError, util.source_from_cache, '__pycache__/foo.pyc')
+
+    def test_source_from_cache_too_many_dots(self):
+        # Too many dots in final path component -> ValueError
+        self.assertRaises(
+            ValueError, util.source_from_cache,
+            '__pycache__/foo.cpython-32.foo.pyc')
+
+    def test_source_from_cache_no__pycache__(self):
+        # Another problem with the path -> ValueError
+        self.assertRaises(
+            ValueError, util.source_from_cache,
+            '/foo/bar/foo.cpython-32.foo.pyc')
+
+
 if __name__ == '__main__':
     unittest.main()
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 04:30:07 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 04:30:07 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_it_more_obvious_what_?=
 =?utf-8?q?things_used_in_imp_are_snuck_in_through_private_APIs?=
Message-ID: <3bXN274XGGzPlS@mail.python.org>

http://hg.python.org/cpython/rev/7bb54aa762e1
changeset:   84135:7bb54aa762e1
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 22:29:58 2013 -0400
summary:
  Make it more obvious what things used in imp are snuck in through private APIs

files:
  Lib/imp.py |  19 ++++++++-----------
  1 files changed, 8 insertions(+), 11 deletions(-)


diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -16,11 +16,9 @@
     # Platform doesn't support dynamic loading.
     load_dynamic = None
 
-# Directly exposed by this module
-from importlib._bootstrap import cache_from_source, source_from_cache
+from importlib._bootstrap import (cache_from_source, source_from_cache,
+                                  SourcelessFileLoader, _ERR_MSG)
 
-
-from importlib import _bootstrap
 from importlib import machinery
 from importlib import util
 import importlib
@@ -117,7 +115,7 @@
             return super().get_data(path)
 
 
-class _LoadSourceCompatibility(_HackedGetData, _bootstrap.SourceFileLoader):
+class _LoadSourceCompatibility(_HackedGetData, machinery.SourceFileLoader):
 
     """Compatibility support for implementing load_source()."""
 
@@ -131,12 +129,11 @@
     module = sys.modules[name]
     # To allow reloading to potentially work, use a non-hacked loader which
     # won't rely on a now-closed file object.
-    module.__loader__ = _bootstrap.SourceFileLoader(name, pathname)
+    module.__loader__ = machinery.SourceFileLoader(name, pathname)
     return module
 
 
-class _LoadCompiledCompatibility(_HackedGetData,
-        _bootstrap.SourcelessFileLoader):
+class _LoadCompiledCompatibility(_HackedGetData, SourcelessFileLoader):
 
     """Compatibility support for implementing load_compiled()."""
 
@@ -150,7 +147,7 @@
     module = sys.modules[name]
     # To allow reloading to potentially work, use a non-hacked loader which
     # won't rely on a now-closed file object.
-    module.__loader__ = _bootstrap.SourcelessFileLoader(name, pathname)
+    module.__loader__ = SourcelessFileLoader(name, pathname)
     return module
 
 
@@ -168,7 +165,7 @@
                 break
         else:
             raise ValueError('{!r} is not a package'.format(path))
-    return _bootstrap.SourceFileLoader(name, path).load_module(name)
+    return machinery.SourceFileLoader(name, path).load_module(name)
 
 
 def load_module(name, file, filename, details):
@@ -252,7 +249,7 @@
                 continue
             break  # Break out of outer loop when breaking out of inner loop.
     else:
-        raise ImportError(_bootstrap._ERR_MSG.format(name), name=name)
+        raise ImportError(_ERR_MSG.format(name), name=name)
 
     encoding = None
     if mode == 'U':

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 04:35:49 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 04:35:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318194=3A_Introduc?=
 =?utf-8?q?e_importlib=2Eutil=2Ecache=5Ffrom=5Fsource=28=29_and?=
Message-ID: <3bXN8j0xzqzQHY@mail.python.org>

http://hg.python.org/cpython/rev/32067804942e
changeset:   84136:32067804942e
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 22:35:40 2013 -0400
summary:
  Issue #18194: Introduce importlib.util.cache_from_source() and
source_from_cache(), finishing the work introduced in changset
4134:9cacdb9d0c59.

files:
  Lib/imp.py |  35 +++++++++++++++++++++++++++++++++--
  Misc/NEWS  |   4 ++++
  2 files changed, 37 insertions(+), 2 deletions(-)


diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -16,8 +16,7 @@
     # Platform doesn't support dynamic loading.
     load_dynamic = None
 
-from importlib._bootstrap import (cache_from_source, source_from_cache,
-                                  SourcelessFileLoader, _ERR_MSG)
+from importlib._bootstrap import SourcelessFileLoader, _ERR_MSG
 
 from importlib import machinery
 from importlib import util
@@ -66,6 +65,38 @@
     return sys.implementation.cache_tag
 
 
+def cache_from_source(path, debug_override=None):
+    """**DEPRECATED**
+
+    Given the path to a .py file, return the path to its .pyc/.pyo file.
+
+    The .py file does not need to exist; this simply returns the path to the
+    .pyc/.pyo file calculated as if the .py file were imported.  The extension
+    will be .pyc unless sys.flags.optimize is non-zero, then it will be .pyo.
+
+    If debug_override is not None, then it must be a boolean and is used in
+    place of sys.flags.optimize.
+
+    If sys.implementation.cache_tag is None then NotImplementedError is raised.
+
+    """
+    return util.cache_from_source(path, debug_override)
+
+
+def source_from_cache(path):
+    """**DEPRECATED**
+
+    Given the path to a .pyc./.pyo file, return the path to its .py file.
+
+    The .pyc/.pyo file does not need to exist; this simply returns the path to
+    the .py file calculated to correspond to the .pyc/.pyo file.  If path does
+    not conform to PEP 3147 format, ValueError will be raised. If
+    sys.implementation.cache_tag is None then NotImplementedError is raised.
+
+    """
+    return util.source_from_cache(path)
+
+
 def get_suffixes():
     warnings.warn('imp.get_suffixes() is deprecated; use the constants '
                   'defined on importlib.machinery instead',
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,10 @@
 Library
 -------
 
+- Issue #18194: Introduce importlib.util.cache_from_source() and
+  source_from_cache() while documenting the equivalent functions in imp as
+  deprecated.
+
 - Issue #17907: Document imp.new_module() as deprecated in favour of
   types.ModuleType.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 04:37:18 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 04:37:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_something_to_the_righ?=
	=?utf-8?q?t_section_of_What=27s_New?=
Message-ID: <3bXNBQ2DGNzPtN@mail.python.org>

http://hg.python.org/cpython/rev/68f8f9d0cebf
changeset:   84137:68f8f9d0cebf
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 22:37:11 2013 -0400
summary:
  Move something to the right section of What's New

files:
  Doc/whatsnew/3.4.rst |  10 ++++++----
  1 files changed, 6 insertions(+), 4 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -225,6 +225,11 @@
   :meth:`difflib.SequenceMatcher.isbpopulur`: use ``x in sm.bjunk`` and
   ``x in sm.bpopular``, where sm is a :class:`~difflib.SequenceMatcher` object.
 
+* :func:`importlib.util.module_for_loader` is pending deprecation. Using
+  :func:`importlib.util.module_to_load` and
+  :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
+  to more easily customize module loading.
+
 
 Deprecated functions and types of the C API
 -------------------------------------------
@@ -235,10 +240,7 @@
 Deprecated features
 -------------------
 
-* :func:`importlib.util.module_for_loader` is pending deprecation. Using
-  :func:`importlib.util.module_to_load` and
-  :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
-  to more easily customize module loading.
+* None yet.
 
 
 Porting to Python 3.4

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 04:49:10 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 04:49:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Update_t?=
 =?utf-8?q?he_programming_FAQ_to_use_importlib?=
Message-ID: <3bXNS60zCYzQHY@mail.python.org>

http://hg.python.org/cpython/rev/3d3b9d456eb8
changeset:   84138:3d3b9d456eb8
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 22:49:00 2013 -0400
summary:
  Issue #17177: Update the programming FAQ to use importlib

files:
  Doc/faq/programming.rst |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -1738,12 +1738,12 @@
 For reasons of efficiency as well as consistency, Python only reads the module
 file on the first time a module is imported.  If it didn't, in a program
 consisting of many modules where each one imports the same basic module, the
-basic module would be parsed and re-parsed many times.  To force rereading of a
+basic module would be parsed and re-parsed many times.  To force re-reading of a
 changed module, do this::
 
-   import imp
+   import importlib
    import modname
-   imp.reload(modname)
+   importlib.reload(modname)
 
 Warning: this technique is not 100% fool-proof.  In particular, modules
 containing statements like ::
@@ -1755,10 +1755,10 @@
 updated to use the new class definition.  This can result in the following
 paradoxical behaviour:
 
-   >>> import imp
+   >>> import importlib
    >>> import cls
    >>> c = cls.C()                # Create an instance of C
-   >>> imp.reload(cls)
+   >>> importlib.reload(cls)
    <module 'cls' from 'cls.py'>
    >>> isinstance(c, cls.C)       # isinstance is false?!?
    False

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 04:51:10 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 04:51:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_make_test_more_robust_unde?=
	=?utf-8?q?r_Windows?=
Message-ID: <3bXNVQ5WqBzQ1G@mail.python.org>

http://hg.python.org/cpython/rev/54ad15b3cb6a
changeset:   84139:54ad15b3cb6a
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 22:50:57 2013 -0400
summary:
  make test more robust under Windows

files:
  Lib/test/test_py_compile.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py
--- a/Lib/test/test_py_compile.py
+++ b/Lib/test/test_py_compile.py
@@ -41,7 +41,7 @@
         # Issue #17222
         try:
             os.symlink(self.pyc_path + '.actual', self.pyc_path)
-        except OSError:
+        except (NotImplementedError, OSError):
             self.skipTest('need to be able to create a symlink for a file')
         else:
             assert os.path.islink(self.pyc_path)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 05:04:12 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 05:04:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_stop_usi?=
	=?utf-8?q?ng_imp_for_compileall=2E?=
Message-ID: <3bXNnS4HQzzQr3@mail.python.org>

http://hg.python.org/cpython/rev/cc27d50bd91a
changeset:   84140:cc27d50bd91a
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 14 23:04:02 2013 -0400
summary:
  Issue #17177: stop using imp for compileall.

files:
  Lib/compileall.py           |  11 +++--
  Lib/test/test_compileall.py |  50 +++++++++++-------------
  2 files changed, 29 insertions(+), 32 deletions(-)


diff --git a/Lib/compileall.py b/Lib/compileall.py
--- a/Lib/compileall.py
+++ b/Lib/compileall.py
@@ -13,7 +13,7 @@
 import os
 import sys
 import errno
-import imp
+import importlib.util
 import py_compile
 import struct
 
@@ -91,17 +91,18 @@
             cfile = fullname + ('c' if __debug__ else 'o')
         else:
             if optimize >= 0:
-                cfile = imp.cache_from_source(fullname,
-                                              debug_override=not optimize)
+                cfile = importlib.util.cache_from_source(
+                                fullname, debug_override=not optimize)
             else:
-                cfile = imp.cache_from_source(fullname)
+                cfile = importlib.util.cache_from_source(fullname)
             cache_dir = os.path.dirname(cfile)
         head, tail = name[:-3], name[-3:]
         if tail == '.py':
             if not force:
                 try:
                     mtime = int(os.stat(fullname).st_mtime)
-                    expect = struct.pack('<4sl', imp.get_magic(), mtime)
+                    expect = struct.pack('<4sl', importlib.util.MAGIC_NUMBER,
+                                         mtime)
                     with open(cfile, 'rb') as chandle:
                         actual = chandle.read(8)
                     if expect == actual:
diff --git a/Lib/test/test_compileall.py b/Lib/test/test_compileall.py
--- a/Lib/test/test_compileall.py
+++ b/Lib/test/test_compileall.py
@@ -1,11 +1,12 @@
 import sys
 import compileall
-import imp
+import importlib.util
 import os
 import py_compile
 import shutil
 import struct
 import subprocess
+import sys
 import tempfile
 import time
 import unittest
@@ -18,11 +19,11 @@
     def setUp(self):
         self.directory = tempfile.mkdtemp()
         self.source_path = os.path.join(self.directory, '_test.py')
-        self.bc_path = imp.cache_from_source(self.source_path)
+        self.bc_path = importlib.util.cache_from_source(self.source_path)
         with open(self.source_path, 'w') as file:
             file.write('x = 123\n')
         self.source_path2 = os.path.join(self.directory, '_test2.py')
-        self.bc_path2 = imp.cache_from_source(self.source_path2)
+        self.bc_path2 = importlib.util.cache_from_source(self.source_path2)
         shutil.copyfile(self.source_path, self.source_path2)
         self.subdirectory = os.path.join(self.directory, '_subdir')
         os.mkdir(self.subdirectory)
@@ -36,7 +37,7 @@
         with open(self.bc_path, 'rb') as file:
             data = file.read(8)
         mtime = int(os.stat(self.source_path).st_mtime)
-        compare = struct.pack('<4sl', imp.get_magic(), mtime)
+        compare = struct.pack('<4sl', importlib.util.MAGIC_NUMBER, mtime)
         return data, compare
 
     def recreation_check(self, metadata):
@@ -57,7 +58,8 @@
 
     def test_mtime(self):
         # Test a change in mtime leads to a new .pyc.
-        self.recreation_check(struct.pack('<4sl', imp.get_magic(), 1))
+        self.recreation_check(struct.pack('<4sl', importlib.util.MAGIC_NUMBER,
+                                          1))
 
     def test_magic_number(self):
         # Test a change in mtime leads to a new .pyc.
@@ -97,14 +99,14 @@
         # interpreter's creates the correct file names
         optimize = 1 if __debug__ else 0
         compileall.compile_dir(self.directory, quiet=True, optimize=optimize)
-        cached = imp.cache_from_source(self.source_path,
-                                       debug_override=not optimize)
+        cached = importlib.util.cache_from_source(self.source_path,
+                                                  debug_override=not optimize)
         self.assertTrue(os.path.isfile(cached))
-        cached2 = imp.cache_from_source(self.source_path2,
-                                       debug_override=not optimize)
+        cached2 = importlib.util.cache_from_source(self.source_path2,
+                                                   debug_override=not optimize)
         self.assertTrue(os.path.isfile(cached2))
-        cached3 = imp.cache_from_source(self.source_path3,
-                                       debug_override=not optimize)
+        cached3 = importlib.util.cache_from_source(self.source_path3,
+                                                   debug_override=not optimize)
         self.assertTrue(os.path.isfile(cached3))
 
 
@@ -152,10 +154,12 @@
         return rc, out, err
 
     def assertCompiled(self, fn):
-        self.assertTrue(os.path.exists(imp.cache_from_source(fn)))
+        path = importlib.util.cache_from_source(fn)
+        self.assertTrue(os.path.exists(path))
 
     def assertNotCompiled(self, fn):
-        self.assertFalse(os.path.exists(imp.cache_from_source(fn)))
+        path = importlib.util.cache_from_source(fn)
+        self.assertFalse(os.path.exists(path))
 
     def setUp(self):
         self.addCleanup(self._cleanup)
@@ -190,8 +194,8 @@
                 ['-m', 'compileall', '-q', self.pkgdir]))
             # Verify the __pycache__ directory contents.
             self.assertTrue(os.path.exists(self.pkgdir_cachedir))
-            expected = sorted(base.format(imp.get_tag(), ext) for base in
-                              ('__init__.{}.{}', 'bar.{}.{}'))
+            expected = sorted(base.format(sys.implementation.cache_tag, ext)
+                              for base in ('__init__.{}.{}', 'bar.{}.{}'))
             self.assertEqual(sorted(os.listdir(self.pkgdir_cachedir)), expected)
             # Make sure there are no .pyc files in the source directory.
             self.assertFalse([fn for fn in os.listdir(self.pkgdir)
@@ -224,7 +228,7 @@
 
     def test_force(self):
         self.assertRunOK('-q', self.pkgdir)
-        pycpath = imp.cache_from_source(self.barfn)
+        pycpath = importlib.util.cache_from_source(self.barfn)
         # set atime/mtime backward to avoid file timestamp resolution issues
         os.utime(pycpath, (time.time()-60,)*2)
         mtime = os.stat(pycpath).st_mtime
@@ -288,7 +292,7 @@
         bazfn = script_helper.make_script(self.pkgdir, 'baz', 'raise Exception')
         self.assertRunOK('-q', '-d', 'dinsdale', self.pkgdir)
         fn = script_helper.make_script(self.pkgdir, 'bing', 'import baz')
-        pyc = imp.cache_from_source(bazfn)
+        pyc = importlib.util.cache_from_source(bazfn)
         os.rename(pyc, os.path.join(self.pkgdir, 'baz.pyc'))
         os.remove(bazfn)
         rc, out, err = script_helper.assert_python_failure(fn)
@@ -299,7 +303,7 @@
             '-i', os.path.join(self.directory, 'nosuchfile'), self.pkgdir)
         self.assertRegex(out, b'rror.*nosuchfile')
         self.assertNotRegex(err, b'Traceback')
-        self.assertFalse(os.path.exists(imp.cache_from_source(
+        self.assertFalse(os.path.exists(importlib.util.cache_from_source(
                                             self.pkgdir_cachedir)))
 
     def test_include_file_with_arg(self):
@@ -356,13 +360,5 @@
         self.assertRegex(out, b"Can't list 'badfilename'")
 
 
-def test_main():
-    support.run_unittest(
-        CommandLineTests,
-        CompileallTests,
-        EncodingTest,
-        )
-
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Jun 15 05:54:45 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 15 Jun 2013 05:54:45 +0200
Subject: [Python-checkins] Daily reference leaks (e7a01c7f69fe): sum=5
Message-ID: <E1UnhZd-0000lE-97@ap.vmr.nerim.net>

results for e7a01c7f69fe on branch "default"
--------------------------------------------

test_support leaked [1, 0, 0] references, sum=1
test_support leaked [1, 2, 1] memory blocks, sum=4


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogtG_3Ow', '-x']

From python-checkins at python.org  Sat Jun 15 07:31:09 2013
From: python-checkins at python.org (ethan.furman)
Date: Sat, 15 Jun 2013 07:31:09 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Marked_PEP-0435_as_Final=2E?=
Message-ID: <3bXS311vPRz7Ljw@mail.python.org>

http://hg.python.org/peps/rev/09ac124d966a
changeset:   4927:09ac124d966a
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Fri Jun 14 22:31:01 2013 -0700
summary:
  Marked PEP-0435 as Final.

files:
  pep-0435.txt |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep-0435.txt b/pep-0435.txt
--- a/pep-0435.txt
+++ b/pep-0435.txt
@@ -5,7 +5,7 @@
 Author: Barry Warsaw <barry at python.org>,
         Eli Bendersky <eliben at gmail.com>,
         Ethan Furman <ethan at stoneleaf.us>
-Status: Accepted
+Status: Final
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 2013-02-23

-- 
Repository URL: http://hg.python.org/peps

From ethan at stoneleaf.us  Sat Jun 15 07:33:10 2013
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 14 Jun 2013 22:33:10 -0700
Subject: [Python-checkins] cpython: Closes issue 17947. Adds PEP-0435
 (Enum, IntEnum) to the stdlib.
In-Reply-To: <CAP1=2W7EtxgQDrmjoKjY8kM649xQ8NYrBSfZFOaFaXSkcCK4Qw@mail.gmail.com>
References: <3bWtm22z9dz7Ljj@mail.python.org>
	<CAP1=2W7EtxgQDrmjoKjY8kM649xQ8NYrBSfZFOaFaXSkcCK4Qw@mail.gmail.com>
Message-ID: <51BBFC96.1000509@stoneleaf.us>

On 06/14/2013 03:57 PM, Brett Cannon wrote:
> Ethan, did you forget to run ``hg add`` before committing? If not then why the heck did we argue over enums for so long
> if this was all it took to make everyone happy? =)

Oops.  ;)

--
~Ethan~

From python-checkins at python.org  Sat Jun 15 16:52:18 2013
From: python-checkins at python.org (lukasz.langa)
Date: Sat, 15 Jun 2013 16:52:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Marked_PEP_443_as_Final=2E?=
Message-ID: <3bXhVV2sxZzNs3@mail.python.org>

http://hg.python.org/peps/rev/61c0d5961dfd
changeset:   4928:61c0d5961dfd
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Sat Jun 15 16:52:08 2013 +0200
summary:
  Marked PEP 443 as Final.

files:
  pep-0443.txt |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep-0443.txt b/pep-0443.txt
--- a/pep-0443.txt
+++ b/pep-0443.txt
@@ -4,7 +4,7 @@
 Last-Modified: $Date$
 Author: ?ukasz Langa <lukasz at langa.pl>
 Discussions-To: Python-Dev <python-dev at python.org>
-Status: Accepted
+Status: Final
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 22-May-2013

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 15 18:28:33 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 18:28:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_fix_spelling?=
Message-ID: <3bXkdY6PdLzPTs@mail.python.org>

http://hg.python.org/peps/rev/957c02c37244
changeset:   4929:957c02c37244
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 12:28:29 2013 -0400
summary:
  fix spelling

files:
  pep-0445.txt |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -135,7 +135,7 @@
 * ``void* realloc(void *ptr, size_t new_size)``
 * ``void free(void *ptr)``
 
-The context is a convinient way to reuse the same allocator for different APIs
+The context is a convenient way to reuse the same allocator for different APIs
 (ex: PyMem and PyObject).
 
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 15 19:00:02 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 19:00:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_in_distutils?=
Message-ID: <3bXlKt6DDqzPTs@mail.python.org>

http://hg.python.org/cpython/rev/30aa032c4bd0
changeset:   84141:30aa032c4bd0
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 12:59:53 2013 -0400
summary:
  Issue #17177: Stop using imp in distutils

files:
  Lib/distutils/command/build_py.py       |  10 +++++-----
  Lib/distutils/command/install_lib.py    |   6 +++---
  Lib/distutils/tests/test_bdist_dumb.py  |   3 +--
  Lib/distutils/tests/test_build_py.py    |  10 ++++++----
  Lib/distutils/tests/test_install.py     |   4 ++--
  Lib/distutils/tests/test_install_lib.py |   8 +++++---
  Lib/distutils/util.py                   |   7 ++++---
  7 files changed, 26 insertions(+), 22 deletions(-)


diff --git a/Lib/distutils/command/build_py.py b/Lib/distutils/command/build_py.py
--- a/Lib/distutils/command/build_py.py
+++ b/Lib/distutils/command/build_py.py
@@ -3,7 +3,7 @@
 Implements the Distutils 'build_py' command."""
 
 import os
-import imp
+import importlib.util
 import sys
 from glob import glob
 
@@ -312,11 +312,11 @@
             outputs.append(filename)
             if include_bytecode:
                 if self.compile:
-                    outputs.append(imp.cache_from_source(filename,
-                                                         debug_override=True))
+                    outputs.append(importlib.util.cache_from_source(
+                        filename, debug_override=True))
                 if self.optimize > 0:
-                    outputs.append(imp.cache_from_source(filename,
-                                                         debug_override=False))
+                    outputs.append(importlib.util.cache_from_source(
+                        filename, debug_override=False))
 
         outputs += [
             os.path.join(build_dir, filename)
diff --git a/Lib/distutils/command/install_lib.py b/Lib/distutils/command/install_lib.py
--- a/Lib/distutils/command/install_lib.py
+++ b/Lib/distutils/command/install_lib.py
@@ -4,7 +4,7 @@
 (install all Python modules)."""
 
 import os
-import imp
+import importlib.util
 import sys
 
 from distutils.core import Command
@@ -165,10 +165,10 @@
             if ext != PYTHON_SOURCE_EXTENSION:
                 continue
             if self.compile:
-                bytecode_files.append(imp.cache_from_source(
+                bytecode_files.append(importlib.util.cache_from_source(
                     py_file, debug_override=True))
             if self.optimize > 0:
-                bytecode_files.append(imp.cache_from_source(
+                bytecode_files.append(importlib.util.cache_from_source(
                     py_file, debug_override=False))
 
         return bytecode_files
diff --git a/Lib/distutils/tests/test_bdist_dumb.py b/Lib/distutils/tests/test_bdist_dumb.py
--- a/Lib/distutils/tests/test_bdist_dumb.py
+++ b/Lib/distutils/tests/test_bdist_dumb.py
@@ -1,7 +1,6 @@
 """Tests for distutils.command.bdist_dumb."""
 
 import os
-import imp
 import sys
 import zipfile
 import unittest
@@ -88,7 +87,7 @@
         contents = sorted(os.path.basename(fn) for fn in contents)
         wanted = ['foo-0.1-py%s.%s.egg-info' % sys.version_info[:2], 'foo.py']
         if not sys.dont_write_bytecode:
-            wanted.append('foo.%s.pyc' % imp.get_tag())
+            wanted.append('foo.%s.pyc' % sys.implementation.cache_tag)
         self.assertEqual(contents, sorted(wanted))
 
 def test_suite():
diff --git a/Lib/distutils/tests/test_build_py.py b/Lib/distutils/tests/test_build_py.py
--- a/Lib/distutils/tests/test_build_py.py
+++ b/Lib/distutils/tests/test_build_py.py
@@ -2,7 +2,6 @@
 
 import os
 import sys
-import imp
 import unittest
 
 from distutils.command.build_py import build_py
@@ -63,7 +62,8 @@
             self.assertFalse(os.path.exists(pycache_dir))
         else:
             pyc_files = os.listdir(pycache_dir)
-            self.assertIn("__init__.%s.pyc" % imp.get_tag(), pyc_files)
+            self.assertIn("__init__.%s.pyc" % sys.implementation.cache_tag,
+                          pyc_files)
 
     def test_empty_package_dir(self):
         # See bugs #1668596/#1720897
@@ -102,7 +102,8 @@
         found = os.listdir(cmd.build_lib)
         self.assertEqual(sorted(found), ['__pycache__', 'boiledeggs.py'])
         found = os.listdir(os.path.join(cmd.build_lib, '__pycache__'))
-        self.assertEqual(found, ['boiledeggs.%s.pyc' % imp.get_tag()])
+        self.assertEqual(found,
+                         ['boiledeggs.%s.pyc' % sys.implementation.cache_tag])
 
     @unittest.skipIf(sys.dont_write_bytecode, 'byte-compile disabled')
     def test_byte_compile_optimized(self):
@@ -119,7 +120,8 @@
         found = os.listdir(cmd.build_lib)
         self.assertEqual(sorted(found), ['__pycache__', 'boiledeggs.py'])
         found = os.listdir(os.path.join(cmd.build_lib, '__pycache__'))
-        self.assertEqual(sorted(found), ['boiledeggs.%s.pyo' % imp.get_tag()])
+        self.assertEqual(sorted(found),
+                         ['boiledeggs.%s.pyo' % sys.implementation.cache_tag])
 
     def test_dont_write_bytecode(self):
         # makes sure byte_compile is not used
diff --git a/Lib/distutils/tests/test_install.py b/Lib/distutils/tests/test_install.py
--- a/Lib/distutils/tests/test_install.py
+++ b/Lib/distutils/tests/test_install.py
@@ -1,7 +1,6 @@
 """Tests for distutils.command.install."""
 
 import os
-import imp
 import sys
 import unittest
 import site
@@ -193,7 +192,8 @@
             f.close()
 
         found = [os.path.basename(line) for line in content.splitlines()]
-        expected = ['hello.py', 'hello.%s.pyc' % imp.get_tag(), 'sayhi',
+        expected = ['hello.py', 'hello.%s.pyc' % sys.implementation.cache_tag,
+                    'sayhi',
                     'UNKNOWN-0.0.0-py%s.%s.egg-info' % sys.version_info[:2]]
         self.assertEqual(found, expected)
 
diff --git a/Lib/distutils/tests/test_install_lib.py b/Lib/distutils/tests/test_install_lib.py
--- a/Lib/distutils/tests/test_install_lib.py
+++ b/Lib/distutils/tests/test_install_lib.py
@@ -1,7 +1,7 @@
 """Tests for distutils.command.install_data."""
 import sys
 import os
-import imp
+import importlib.util
 import unittest
 
 from distutils.command.install_lib import install_lib
@@ -44,8 +44,10 @@
         f = os.path.join(project_dir, 'foo.py')
         self.write_file(f, '# python file')
         cmd.byte_compile([f])
-        pyc_file = imp.cache_from_source('foo.py', debug_override=True)
-        pyo_file = imp.cache_from_source('foo.py', debug_override=False)
+        pyc_file = importlib.util.cache_from_source('foo.py',
+                                                    debug_override=True)
+        pyo_file = importlib.util.cache_from_source('foo.py',
+                                                    debug_override=False)
         self.assertTrue(os.path.exists(pyc_file))
         self.assertTrue(os.path.exists(pyo_file))
 
diff --git a/Lib/distutils/util.py b/Lib/distutils/util.py
--- a/Lib/distutils/util.py
+++ b/Lib/distutils/util.py
@@ -6,7 +6,7 @@
 
 import os
 import re
-import imp
+import importlib.util
 import sys
 import string
 from distutils.errors import DistutilsPlatformError
@@ -453,9 +453,10 @@
             #   cfile - byte-compiled file
             #   dfile - purported source filename (same as 'file' by default)
             if optimize >= 0:
-                cfile = imp.cache_from_source(file, debug_override=not optimize)
+                cfile = importlib.util.cache_from_source(
+                    file, debug_override=not optimize)
             else:
-                cfile = imp.cache_from_source(file)
+                cfile = importlib.util.cache_from_source(file)
             dfile = file
             if prefix:
                 if file[:len(prefix)] != prefix:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 19:23:09 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 19:23:09 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_in_multiprocessing?=
Message-ID: <3bXlrY5ZtGzQCV@mail.python.org>

http://hg.python.org/cpython/rev/0b96a16656b7
changeset:   84142:0b96a16656b7
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 13:23:01 2013 -0400
summary:
  Issue #17177: Stop using imp in multiprocessing

files:
  Lib/multiprocessing/forking.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/multiprocessing/forking.py b/Lib/multiprocessing/forking.py
--- a/Lib/multiprocessing/forking.py
+++ b/Lib/multiprocessing/forking.py
@@ -449,8 +449,8 @@
         elif main_name != 'ipython':
             # Main modules not actually called __main__.py may
             # contain additional code that should still be executed
-            import imp
             import importlib
+            import types
 
             if main_path is None:
                 dirs = None
@@ -465,7 +465,7 @@
             # since that would execute 'if __name__ == "__main__"'
             # clauses, potentially causing a psuedo fork bomb.
             loader = importlib.find_loader(main_name, path=dirs)
-            main_module = imp.new_module(main_name)
+            main_module = types.ModuleType(main_name)
             try:
                 loader.init_module_attrs(main_module)
             except AttributeError:  # init_module_attrs is optional

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 19:29:19 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 15 Jun 2013 19:29:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_typo=3B_clarify_that_t?=
 =?utf-8?q?he_methods_were_removed_entirely?=
Message-ID: <3bXlzg2SJGzQ3P@mail.python.org>

http://hg.python.org/cpython/rev/5d57f2deffba
changeset:   84143:5d57f2deffba
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 15 13:29:09 2013 -0400
summary:
  Fix typo; clarify that the methods were removed entirely

files:
  Doc/whatsnew/3.4.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -222,8 +222,8 @@
 ------------------------------------------------
 
 * :meth:`difflib.SequenceMatcher.isbjunk` and
-  :meth:`difflib.SequenceMatcher.isbpopulur`: use ``x in sm.bjunk`` and
-  ``x in sm.bpopular``, where sm is a :class:`~difflib.SequenceMatcher` object.
+  :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and
+  ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object.
 
 * :func:`importlib.util.module_for_loader` is pending deprecation. Using
   :func:`importlib.util.module_to_load` and

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 19:38:14 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 19:38:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Drop_some_dead?=
	=?utf-8?q?_imports_of_imp?=
Message-ID: <3bXm9y4RXxzSrd@mail.python.org>

http://hg.python.org/cpython/rev/61a1aa69e83e
changeset:   84144:61a1aa69e83e
branch:      3.3
parent:      84120:ee3952965934
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 13:37:12 2013 -0400
summary:
  Drop some dead imports of imp

files:
  Lib/idlelib/PathBrowser.py |  1 -
  Lib/idlelib/TreeWidget.py  |  1 -
  Misc/NEWS                  |  2 ++
  3 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/idlelib/PathBrowser.py b/Lib/idlelib/PathBrowser.py
--- a/Lib/idlelib/PathBrowser.py
+++ b/Lib/idlelib/PathBrowser.py
@@ -1,6 +1,5 @@
 import os
 import sys
-import imp
 import importlib.machinery
 
 from idlelib.TreeWidget import TreeItem
diff --git a/Lib/idlelib/TreeWidget.py b/Lib/idlelib/TreeWidget.py
--- a/Lib/idlelib/TreeWidget.py
+++ b/Lib/idlelib/TreeWidget.py
@@ -16,7 +16,6 @@
 
 import os
 from tkinter import *
-import imp
 
 from idlelib import ZoomHeight
 from idlelib.configHandler import idleConf
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -81,6 +81,8 @@
 IDLE
 ----
 
+- Remove dead imports of imp.
+
 - Issue #18196: Avoid displaying spurious SystemExit tracebacks.
 
 - Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 19:38:15 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 19:38:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?b?KTogbWVyZ2Ugdy8gMy4z?=
Message-ID: <3bXm9z6LcHz7LjS@mail.python.org>

http://hg.python.org/cpython/rev/48b79f11571e
changeset:   84145:48b79f11571e
parent:      84142:0b96a16656b7
parent:      84144:61a1aa69e83e
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 13:37:38 2013 -0400
summary:
  merge w/ 3.3

files:
  Lib/idlelib/PathBrowser.py |  1 -
  Lib/idlelib/TreeWidget.py  |  1 -
  Misc/NEWS                  |  2 ++
  3 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/idlelib/PathBrowser.py b/Lib/idlelib/PathBrowser.py
--- a/Lib/idlelib/PathBrowser.py
+++ b/Lib/idlelib/PathBrowser.py
@@ -1,6 +1,5 @@
 import os
 import sys
-import imp
 import importlib.machinery
 
 from idlelib.TreeWidget import TreeItem
diff --git a/Lib/idlelib/TreeWidget.py b/Lib/idlelib/TreeWidget.py
--- a/Lib/idlelib/TreeWidget.py
+++ b/Lib/idlelib/TreeWidget.py
@@ -16,7 +16,6 @@
 
 import os
 from tkinter import *
-import imp
 
 from idlelib import ZoomHeight
 from idlelib.configHandler import idleConf
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -479,6 +479,8 @@
 IDLE
 ----
 
+- Remove dead imports of imp.
+
 - Issue #18196: Avoid displaying spurious SystemExit tracebacks.
 
 - Issue #5492: Avoid traceback when exiting IDLE caused by a race condition.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 19:38:17 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 19:38:17 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bXmB10v5Jz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/acca81736b69
changeset:   84146:acca81736b69
parent:      84145:48b79f11571e
parent:      84143:5d57f2deffba
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 13:38:07 2013 -0400
summary:
  merge

files:
  Doc/whatsnew/3.4.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -222,8 +222,8 @@
 ------------------------------------------------
 
 * :meth:`difflib.SequenceMatcher.isbjunk` and
-  :meth:`difflib.SequenceMatcher.isbpopulur`: use ``x in sm.bjunk`` and
-  ``x in sm.bpopular``, where sm is a :class:`~difflib.SequenceMatcher` object.
+  :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and
+  ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object.
 
 * :func:`importlib.util.module_for_loader` is pending deprecation. Using
   :func:`importlib.util.module_to_load` and

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 20:01:15 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 15 Jun 2013 20:01:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_An_object=27s_finalizer_will_?=
 =?utf-8?q?only_ever_be_called_once=2C_even_if_resurrected=2E?=
Message-ID: <3bXmhW2THQz7Ljs@mail.python.org>

http://hg.python.org/peps/rev/b9c80ae6a17d
changeset:   4930:b9c80ae6a17d
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Jun 15 19:59:55 2013 +0200
summary:
  An object's finalizer will only ever be called once, even if resurrected.

files:
  pep-0442.txt |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/pep-0442.txt b/pep-0442.txt
--- a/pep-0442.txt
+++ b/pep-0442.txt
@@ -201,8 +201,7 @@
 --------------
 
 Following this scheme, an object's finalizer is always called exactly
-once.  The only exception is if an object is resurrected: the finalizer
-will be called again when the object becomes unreachable again.
+once, even if it was resurrected afterwards.
 
 For CI objects, the order in which finalizers are called (step 2 above)
 is undefined.

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 15 20:01:16 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 15 Jun 2013 20:01:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Mark_PEP_442_as_accepted?=
Message-ID: <3bXmhX45bNzPtG@mail.python.org>

http://hg.python.org/peps/rev/7428ee7e90f2
changeset:   4931:7428ee7e90f2
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Jun 15 20:01:08 2013 +0200
summary:
  Mark PEP 442 as accepted

files:
  pep-0442.txt |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/pep-0442.txt b/pep-0442.txt
--- a/pep-0442.txt
+++ b/pep-0442.txt
@@ -4,13 +4,13 @@
 Last-Modified: $Date$
 Author: Antoine Pitrou <solipsis at pitrou.net>
 BDFL-Delegate: Benjamin Peterson <benjamin at python.org>
-Status: Draft
+Status: Accepted
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 2013-05-18
 Python-Version: 3.4
 Post-History: 2013-05-18
-Resolution: TBD
+Resolution: http://mail.python.org/pipermail/python-dev/2013-June/126746.html
 
 
 Abstract

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 15 20:25:13 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 20:25:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_with_py=5Fcompile?=
Message-ID: <3bXnD92Dwwz7Ljp@mail.python.org>

http://hg.python.org/cpython/rev/91467f342977
changeset:   84147:91467f342977
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 14:07:21 2013 -0400
summary:
  Issue #17177: Stop using imp with py_compile

files:
  Lib/py_compile.py           |  7 ++++---
  Lib/test/test_py_compile.py |  4 ++--
  2 files changed, 6 insertions(+), 5 deletions(-)


diff --git a/Lib/py_compile.py b/Lib/py_compile.py
--- a/Lib/py_compile.py
+++ b/Lib/py_compile.py
@@ -3,9 +3,9 @@
 This module has intimate knowledge of the format of .pyc files.
 """
 
-import imp
 import importlib._bootstrap
 import importlib.machinery
+import importlib.util
 import os
 import os.path
 import sys
@@ -105,9 +105,10 @@
     """
     if cfile is None:
         if optimize >= 0:
-            cfile = imp.cache_from_source(file, debug_override=not optimize)
+            cfile = importlib.util.cache_from_source(file,
+                                                     debug_override=not optimize)
         else:
-            cfile = imp.cache_from_source(file)
+            cfile = importlib.util.cache_from_source(file)
     if os.path.islink(cfile):
         msg = ('{} is a symlink and will be changed into a regular file if '
                'import writes a byte-compiled file to it')
diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py
--- a/Lib/test/test_py_compile.py
+++ b/Lib/test/test_py_compile.py
@@ -1,4 +1,4 @@
-import imp
+import importlib.util
 import os
 import py_compile
 import shutil
@@ -14,7 +14,7 @@
         self.directory = tempfile.mkdtemp()
         self.source_path = os.path.join(self.directory, '_test.py')
         self.pyc_path = self.source_path + 'c'
-        self.cache_path = imp.cache_from_source(self.source_path)
+        self.cache_path = importlib.util.cache_from_source(self.source_path)
         self.cwd_drive = os.path.splitdrive(os.getcwd())[0]
         # In these tests we compute relative paths.  When using Windows, the
         # current working directory path and the 'self.source_path' might be

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 20:25:14 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 20:25:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_in_pydoc?=
Message-ID: <3bXnDB46L8z7LjS@mail.python.org>

http://hg.python.org/cpython/rev/81cf3d6e6756
changeset:   84148:81cf3d6e6756
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 14:25:04 2013 -0400
summary:
  Issue #17177: Stop using imp in pydoc

files:
  Lib/pydoc.py |  9 ++++-----
  1 files changed, 4 insertions(+), 5 deletions(-)


diff --git a/Lib/pydoc.py b/Lib/pydoc.py
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -44,17 +44,16 @@
 """
 
 # Known bugs that can't be fixed here:
-#   - imp.load_module() cannot be prevented from clobbering existing
-#     loaded modules, so calling synopsis() on a binary module file
-#     changes the contents of any existing module with the same name.
+#   - synopsis() cannot be prevented from clobbering existing
+#     loaded modules.
 #   - If the __file__ attribute on a module is a relative path and
 #     the current directory is changed with os.chdir(), an incorrect
 #     path will be displayed.
 
 import builtins
-import imp
 import importlib._bootstrap
 import importlib.machinery
+import importlib.util
 import inspect
 import io
 import os
@@ -268,7 +267,7 @@
 
 def importfile(path):
     """Import a Python source file or compiled file given its path."""
-    magic = imp.get_magic()
+    magic = importlib.util.MAGIC_NUMBER
     with open(path, 'rb') as file:
         is_bytecode = magic == file.read(len(magic))
     filename = os.path.basename(path)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 20:27:30 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 20:27:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_switch_f?=
 =?utf-8?q?rom_imp=2Enew=5Fmodule_to_types=2EModuleType_for_runpy?=
Message-ID: <3bXnGp1T1BzP98@mail.python.org>

http://hg.python.org/cpython/rev/8fff5554125d
changeset:   84149:8fff5554125d
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 14:27:21 2013 -0400
summary:
  Issue #17177: switch from imp.new_module to types.ModuleType for runpy

files:
  Lib/runpy.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/runpy.py b/Lib/runpy.py
--- a/Lib/runpy.py
+++ b/Lib/runpy.py
@@ -14,6 +14,7 @@
 import sys
 import importlib.machinery # importlib first so we can test #15386 via -m
 import imp
+import types
 from pkgutil import read_code, get_loader, get_importer
 
 __all__ = [
@@ -24,7 +25,7 @@
     """Temporarily replace a module in sys.modules with an empty namespace"""
     def __init__(self, mod_name):
         self.mod_name = mod_name
-        self.module = imp.new_module(mod_name)
+        self.module = types.ModuleType(mod_name)
         self._saved_module = []
 
     def __enter__(self):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 20:32:21 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 20:32:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_in_sysconfig?=
Message-ID: <3bXnNP0HWmz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/1e141c2cc0d7
changeset:   84150:1e141c2cc0d7
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 14:32:11 2013 -0400
summary:
  Issue #17177: Stop using imp in sysconfig

files:
  Lib/sysconfig.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -383,8 +383,8 @@
     # get_platform() succeeds.
     name = '_sysconfigdata'
     if 'darwin' in sys.platform:
-        import imp
-        module = imp.new_module(name)
+        import types
+        module = types.ModuleType(name)
         module.build_time_vars = vars
         sys.modules[name] = module
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 20:48:16 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 15 Jun 2013 20:48:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTEzOiBPYmpl?=
 =?utf-8?q?cts_associated_to_a_curses=2Epanel_object_with_set=5Fuserptr=28?=
 =?utf-8?q?=29_were?=
Message-ID: <3bXnkm4CrzzRvc@mail.python.org>

http://hg.python.org/cpython/rev/aff0bdab358e
changeset:   84151:aff0bdab358e
branch:      2.7
parent:      84116:5accb0ac8bfb
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 15 13:53:10 2013 -0400
summary:
  #18113: Objects associated to a curses.panel object with set_userptr() were leaked.

Reported by Atsuo Ishimoto.

files:
  Lib/test/test_curses.py |  13 +++++++++++++
  Misc/NEWS               |   3 +++
  Modules/_curses_panel.c |   4 ++++
  3 files changed, 20 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -250,6 +250,18 @@
     except curses.panel.error:
         pass
 
+def test_userptr_memory_leak(stdscr):
+    w = curses.newwin(10, 10)
+    p = curses.panel.new_panel(w)
+    obj = object()
+    nrefs = sys.getrefcount(obj)
+    for i in range(100):
+        p.set_userptr(obj)
+
+    p.set_userptr(None)
+    if sys.getrefcount(obj) != nrefs:
+        raise RuntimeError, "set_userptr leaked references"
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -268,6 +280,7 @@
         module_funcs(stdscr)
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
+        test_userptr_memory_leak(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
     finally:
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -40,6 +40,9 @@
   the default for linking if LDSHARED is not also overriden.  This restores
   Distutils behavior introduced in 2.7.3 and inadvertently dropped in 2.7.4.
 
+- Issue #18113: Fixed a refcount leak in the curses.panel module's
+  set_userptr() method.  Reported by Atsuo Ishimoto.
+
 Build
 -----
 
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -293,6 +293,10 @@
 static PyObject *
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
+    PyObject *oldobj;
+    PyCursesInitialised;
+    oldobj = (PyObject *) panel_userptr(self->pan);
+    Py_XDECREF(oldobj);
     Py_INCREF(obj);
     return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
                             "set_panel_userptr");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 21:03:32 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 15 Jun 2013 21:03:32 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTEzOiBPYmpl?=
 =?utf-8?q?cts_associated_to_a_curses=2Epanel_object_with_set=5Fuserptr=28?=
 =?utf-8?q?=29_were?=
Message-ID: <3bXp4N3XvdzQ3P@mail.python.org>

http://hg.python.org/cpython/rev/3d75bd69e5a9
changeset:   84152:3d75bd69e5a9
branch:      3.3
parent:      84144:61a1aa69e83e
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 15 14:04:04 2013 -0400
summary:
  #18113: Objects associated to a curses.panel object with set_userptr() were leaked.

Reported by Atsuo Ishimoto.

files:
  Lib/test/test_curses.py |  13 +++++++++++++
  Misc/NEWS               |   3 +++
  Modules/_curses_panel.c |   4 ++++
  3 files changed, 20 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -252,6 +252,18 @@
     except curses.panel.error:
         pass
 
+def test_userptr_memory_leak(stdscr):
+    w = curses.newwin(10, 10)
+    p = curses.panel.new_panel(w)
+    obj = object()
+    nrefs = sys.getrefcount(obj)
+    for i in range(100):
+        p.set_userptr(obj)
+
+    p.set_userptr(None)
+    if sys.getrefcount(obj) != nrefs:
+        raise RuntimeError("set_userptr leaked references")
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -317,6 +329,7 @@
         module_funcs(stdscr)
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
+        test_userptr_memory_leak(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
         test_unget_wch(stdscr)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -78,6 +78,9 @@
   the default for linking if LDSHARED is not also overriden.  This restores
   Distutils behavior introduced in 3.2.3 and inadvertently dropped in 3.3.0.
 
+- Issue #18113: Fixed a refcount leak in the curses.panel module's
+  set_userptr() method.  Reported by Atsuo Ishimoto.
+
 IDLE
 ----
 
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -322,6 +322,10 @@
 static PyObject *
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
+    PyObject *oldobj;
+    PyCursesInitialised;
+    oldobj = (PyObject *) panel_userptr(self->pan);
+    Py_XDECREF(oldobj);
     Py_INCREF(obj);
     return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
                             "set_panel_userptr");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 21:11:57 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 15 Jun 2013 21:11:57 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bXpG50DKYz7Ljv@mail.python.org>

http://hg.python.org/cpython/rev/fcb686e720ff
changeset:   84153:fcb686e720ff
parent:      84150:1e141c2cc0d7
parent:      84152:3d75bd69e5a9
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 15 15:10:08 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/test/test_curses.py |  13 +++++++++++++
  Misc/NEWS               |   3 +++
  Modules/_curses_panel.c |   4 ++++
  3 files changed, 20 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -252,6 +252,18 @@
     except curses.panel.error:
         pass
 
+def test_userptr_memory_leak(stdscr):
+    w = curses.newwin(10, 10)
+    p = curses.panel.new_panel(w)
+    obj = object()
+    nrefs = sys.getrefcount(obj)
+    for i in range(100):
+        p.set_userptr(obj)
+
+    p.set_userptr(None)
+    if sys.getrefcount(obj) != nrefs:
+        raise RuntimeError("set_userptr leaked references")
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -317,6 +329,7 @@
         module_funcs(stdscr)
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
+        test_userptr_memory_leak(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
         test_unget_wch(stdscr)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -383,6 +383,9 @@
   the default for linking if LDSHARED is not also overriden.  This restores
   Distutils behavior introduced in 3.2.3 and inadvertently dropped in 3.3.0.
 
+- Issue #18113: Fixed a refcount leak in the curses.panel module's
+  set_userptr() method.  Reported by Atsuo Ishimoto.
+
 - Implement PEP 443 "Single-dispatch generic functions".
 
 - Implement PEP 435 "Adding an Enum type to the Python standard library".
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -322,6 +322,10 @@
 static PyObject *
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
+    PyObject *oldobj;
+    PyCursesInitialised;
+    oldobj = (PyObject *) panel_userptr(self->pan);
+    Py_XDECREF(oldobj);
     Py_INCREF(obj);
     return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
                             "set_panel_userptr");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 23:11:35 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 23:11:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_in_a_bunch_of_tests?=
Message-ID: <3bXrw70WyczRF7@mail.python.org>

http://hg.python.org/cpython/rev/bc18b5d4920e
changeset:   84154:bc18b5d4920e
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 17:11:25 2013 -0400
summary:
  Issue #17177: Stop using imp in a bunch of tests

files:
  Lib/test/script_helper.py  |  2 +-
  Lib/test/support.py        |  8 ++++----
  Lib/test/test_pdb.py       |  4 ++--
  Lib/test/test_pkgimport.py |  2 +-
  Lib/test/test_pkgutil.py   |  4 ++--
  Lib/test/test_reprlib.py   |  5 +++--
  Lib/test/test_zipimport.py |  7 ++++---
  7 files changed, 17 insertions(+), 15 deletions(-)


diff --git a/Lib/test/script_helper.py b/Lib/test/script_helper.py
--- a/Lib/test/script_helper.py
+++ b/Lib/test/script_helper.py
@@ -12,7 +12,7 @@
 import shutil
 import zipfile
 
-from imp import source_from_cache
+from importlib.util import source_from_cache
 from test.support import make_legacy_pyc, strip_python_stderr
 
 # Executing the interpreter in a subprocess
diff --git a/Lib/test/support.py b/Lib/test/support.py
--- a/Lib/test/support.py
+++ b/Lib/test/support.py
@@ -15,10 +15,10 @@
 import warnings
 import unittest
 import importlib
+import importlib.util
 import collections.abc
 import re
 import subprocess
-import imp
 import time
 import sysconfig
 import fnmatch
@@ -316,7 +316,7 @@
         does not need to exist, however the PEP 3147 pyc file must exist.
     :return: The file system path to the legacy pyc file.
     """
-    pyc_file = imp.cache_from_source(source)
+    pyc_file = importlib.util.cache_from_source(source)
     up_one = os.path.dirname(os.path.abspath(source))
     legacy_pyc = os.path.join(up_one, source + ('c' if __debug__ else 'o'))
     os.rename(pyc_file, legacy_pyc)
@@ -335,8 +335,8 @@
         # combinations of PEP 3147 and legacy pyc and pyo files.
         unlink(source + 'c')
         unlink(source + 'o')
-        unlink(imp.cache_from_source(source, debug_override=True))
-        unlink(imp.cache_from_source(source, debug_override=False))
+        unlink(importlib.util.cache_from_source(source, debug_override=True))
+        unlink(importlib.util.cache_from_source(source, debug_override=False))
 
 # On some platforms, should not run gui test even if it is allowed
 # in `use_resources'.
diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -1,9 +1,9 @@
 # A test suite for pdb; not very comprehensive at the moment.
 
 import doctest
-import imp
 import pdb
 import sys
+import types
 import unittest
 import subprocess
 import textwrap
@@ -464,7 +464,7 @@
 
 
 # Module for testing skipping of module that makes a callback
-mod = imp.new_module('module_to_skip')
+mod = types.ModuleType('module_to_skip')
 exec('def foo_pony(callback): x = 1; callback(); return None', mod.__dict__)
 
 
diff --git a/Lib/test/test_pkgimport.py b/Lib/test/test_pkgimport.py
--- a/Lib/test/test_pkgimport.py
+++ b/Lib/test/test_pkgimport.py
@@ -6,7 +6,7 @@
 import tempfile
 import unittest
 
-from imp import cache_from_source
+from importlib.util import cache_from_source
 from test.support import run_unittest, create_empty_file
 
 class TestImport(unittest.TestCase):
diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py
--- a/Lib/test/test_pkgutil.py
+++ b/Lib/test/test_pkgutil.py
@@ -1,12 +1,12 @@
 from test.support import run_unittest, unload, check_warnings
 import unittest
 import sys
-import imp
 import importlib
 import pkgutil
 import os
 import os.path
 import tempfile
+import types
 import shutil
 import zipfile
 
@@ -105,7 +105,7 @@
     class MyTestLoader(object):
         def load_module(self, fullname):
             # Create an empty module
-            mod = sys.modules.setdefault(fullname, imp.new_module(fullname))
+            mod = sys.modules.setdefault(fullname, types.ModuleType(fullname))
             mod.__file__ = "<%s>" % self.__class__.__name__
             mod.__loader__ = self
             # Make it a package
diff --git a/Lib/test/test_reprlib.py b/Lib/test/test_reprlib.py
--- a/Lib/test/test_reprlib.py
+++ b/Lib/test/test_reprlib.py
@@ -3,11 +3,11 @@
   Nick Mathewson
 """
 
-import imp
 import sys
 import os
 import shutil
 import importlib
+import importlib.util
 import unittest
 
 from test.support import run_unittest, create_empty_file, verbose
@@ -241,7 +241,8 @@
         source_path_len += 2 * (len(self.longname) + 1)
         # a path separator + `module_name` + ".py"
         source_path_len += len(module_name) + 1 + len(".py")
-        cached_path_len = source_path_len + len(imp.cache_from_source("x.py")) - len("x.py")
+        cached_path_len = (source_path_len +
+            len(importlib.util.cache_from_source("x.py")) - len("x.py"))
         if os.name == 'nt' and cached_path_len >= 258:
             # Under Windows, the max path len is 260 including C's terminating
             # NUL character.
diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py
--- a/Lib/test/test_zipimport.py
+++ b/Lib/test/test_zipimport.py
@@ -1,7 +1,7 @@
 import sys
 import os
 import marshal
-import imp
+import importlib.util
 import struct
 import time
 import unittest
@@ -34,7 +34,8 @@
             mtime = int(mtime)
         else:
             mtime = int(-0x100000000 + int(mtime))
-    pyc = imp.get_magic() + struct.pack("<ii", int(mtime), size & 0xFFFFFFFF) + data
+    pyc = (importlib.util.MAGIC_NUMBER +
+        struct.pack("<ii", int(mtime), size & 0xFFFFFFFF) + data)
     return pyc
 
 def module_path_to_dotted_name(path):
@@ -49,7 +50,7 @@
 TESTPACK2 = "ziptestpackage2"
 TEMP_ZIP = os.path.abspath("junk95142.zip")
 
-pyc_file = imp.cache_from_source(TESTMOD + '.py')
+pyc_file = importlib.util.cache_from_source(TESTMOD + '.py')
 pyc_ext = ('.pyc' if __debug__ else '.pyo')
 
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 23:33:38 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 23:33:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Stop_usi?=
	=?utf-8?q?ng_imp_in_zipfile?=
Message-ID: <3bXsPZ0CSjz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/f96eb1dc335f
changeset:   84155:f96eb1dc335f
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 17:32:30 2013 -0400
summary:
  Issue #17177: Stop using imp in zipfile

files:
  Lib/test/test_zipfile.py |  4 ++--
  Lib/zipfile.py           |  6 +++---
  2 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py
--- a/Lib/test/test_zipfile.py
+++ b/Lib/test/test_zipfile.py
@@ -1,7 +1,7 @@
 import io
 import os
 import sys
-import imp
+import importlib.util
 import time
 import shutil
 import struct
@@ -869,7 +869,7 @@
                 if os.altsep is not None:
                     path_split.extend(fn.split(os.altsep))
                 if '__pycache__' in path_split:
-                    fn = imp.source_from_cache(fn)
+                    fn = importlib.util.source_from_cache(fn)
                 else:
                     fn = fn[:-1]
 
diff --git a/Lib/zipfile.py b/Lib/zipfile.py
--- a/Lib/zipfile.py
+++ b/Lib/zipfile.py
@@ -6,7 +6,7 @@
 import io
 import os
 import re
-import imp
+import importlib.util
 import sys
 import time
 import stat
@@ -1645,8 +1645,8 @@
         file_py  = pathname + ".py"
         file_pyc = pathname + ".pyc"
         file_pyo = pathname + ".pyo"
-        pycache_pyc = imp.cache_from_source(file_py, True)
-        pycache_pyo = imp.cache_from_source(file_py, False)
+        pycache_pyc = importlib.util.cache_from_source(file_py, True)
+        pycache_pyo = importlib.util.cache_from_source(file_py, False)
         if self._optimize == -1:
             # legacy mode: use whatever file is present
             if (os.path.isfile(file_pyo) and

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 23:33:39 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 23:33:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=23_17177=3A_Stop_us?=
	=?utf-8?q?ing_imp_in_turtledemo?=
Message-ID: <3bXsPb1ytnz7LkJ@mail.python.org>

http://hg.python.org/cpython/rev/b2b2bdc4cf6f
changeset:   84156:b2b2bdc4cf6f
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 17:33:27 2013 -0400
summary:
  Issue # 17177: Stop using imp in turtledemo

files:
  Lib/turtledemo/__main__.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/turtledemo/__main__.py b/Lib/turtledemo/__main__.py
--- a/Lib/turtledemo/__main__.py
+++ b/Lib/turtledemo/__main__.py
@@ -6,7 +6,7 @@
 from idlelib.Percolator import Percolator
 from idlelib.ColorDelegator import ColorDelegator
 from idlelib.textView import view_file # TextViewer
-from imp import reload
+from importlib import reload
 
 import turtle
 import time

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 15 23:53:08 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 15 Jun 2013 23:53:08 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=23_17177=3A_Stop_us?=
	=?utf-8?q?ing_imp_in_setup=2Epy?=
Message-ID: <3bXsr44P39z7Lkf@mail.python.org>

http://hg.python.org/cpython/rev/4a1161eaed99
changeset:   84157:4a1161eaed99
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 17:52:59 2013 -0400
summary:
  Issue # 17177: Stop using imp in setup.py

files:
  setup.py |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -1,7 +1,7 @@
 # Autodetecting setup.py script for building the Python extensions
 #
 
-import sys, os, imp, re, optparse
+import sys, os, importlib.machinery, re, optparse
 from glob import glob
 import sysconfig
 
@@ -325,8 +325,9 @@
         if cross_compiling:
             return
 
+        loader = importlib.machinery.ExtensionFileLoader(ext.name, ext_filename)
         try:
-            imp.load_dynamic(ext.name, ext_filename)
+            loader.load_module()
         except ImportError as why:
             self.failed.append(ext.name)
             self.announce('*** WARNING: renaming "%s" since importing it'

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 00:10:26 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 00:10:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_update_c?=
	=?utf-8?q?heckpyc_to_stop_using_imp?=
Message-ID: <3bXtD21qf8z7Ljr@mail.python.org>

http://hg.python.org/cpython/rev/ca3bdac1f88a
changeset:   84158:ca3bdac1f88a
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 18:10:18 2013 -0400
summary:
  Issue #17177: update checkpyc to stop using imp

files:
  Tools/scripts/checkpyc.py |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Tools/scripts/checkpyc.py b/Tools/scripts/checkpyc.py
--- a/Tools/scripts/checkpyc.py
+++ b/Tools/scripts/checkpyc.py
@@ -5,11 +5,11 @@
 import sys
 import os
 from stat import ST_MTIME
-import imp
+import importlib.util
 
 # PEP 3147 compatibility (PYC Repository Directories)
-cache_from_source = (imp.cache_from_source if hasattr(imp, 'get_tag') else
-                     lambda path: path + 'c')
+cache_from_source = (importlib.util.cache_from_source if sys.implementation.cache_tag
+                     else lambda path: path + 'c')
 
 
 def main():
@@ -18,7 +18,7 @@
         silent = (sys.argv[1] == '-s')
     else:
         verbose = silent = False
-    MAGIC = imp.get_magic()
+    MAGIC = importlib.util.MAGIC_NUMBER
     if not silent:
         print('Using MAGIC word', repr(MAGIC))
     for dirname in sys.path:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 00:39:30 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 00:39:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_stop_usi?=
	=?utf-8?q?ng_imp_in_test=5Fimportlib?=
Message-ID: <3bXtsZ6Hn2z7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/5e8b377942f7
changeset:   84159:5e8b377942f7
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 18:39:21 2013 -0400
summary:
  Issue #17177: stop using imp in test_importlib

files:
  Lib/test/test_importlib/extension/test_case_sensitivity.py |   9 +-
  Lib/test/test_importlib/extension/test_path_hook.py        |   1 -
  Lib/test/test_importlib/extension/util.py                  |   1 -
  Lib/test/test_importlib/frozen/test_loader.py              |  10 +-
  Lib/test/test_importlib/import_/test___loader__.py         |   6 +-
  Lib/test/test_importlib/import_/test_api.py                |   5 +-
  Lib/test/test_importlib/import_/test_fromlist.py           |   1 -
  Lib/test/test_importlib/source/test_case_sensitivity.py    |   6 +-
  Lib/test/test_importlib/source/test_file_loader.py         |  19 ++-
  Lib/test/test_importlib/source/test_finder.py              |   1 -
  Lib/test/test_importlib/source/test_path_hook.py           |   1 -
  Lib/test/test_importlib/source/util.py                     |   1 -
  Lib/test/test_importlib/test_abc.py                        |  49 +++++----
  Lib/test/test_importlib/test_api.py                        |  10 +-
  Lib/test/test_importlib/test_util.py                       |  30 +++---
  Lib/test/test_importlib/util.py                            |   4 +-
  16 files changed, 78 insertions(+), 76 deletions(-)


diff --git a/Lib/test/test_importlib/extension/test_case_sensitivity.py b/Lib/test/test_importlib/extension/test_case_sensitivity.py
--- a/Lib/test/test_importlib/extension/test_case_sensitivity.py
+++ b/Lib/test/test_importlib/extension/test_case_sensitivity.py
@@ -1,8 +1,9 @@
-import imp
 import sys
 from test import support
 import unittest
+
 from importlib import _bootstrap
+from importlib import machinery
 from .. import util
 from . import util as ext_util
 
@@ -14,9 +15,9 @@
         good_name = ext_util.NAME
         bad_name = good_name.upper()
         assert good_name != bad_name
-        finder = _bootstrap.FileFinder(ext_util.PATH,
-                                        (_bootstrap.ExtensionFileLoader,
-                                         _bootstrap.EXTENSION_SUFFIXES))
+        finder = machinery.FileFinder(ext_util.PATH,
+                                        (machinery.ExtensionFileLoader,
+                                         machinery.EXTENSION_SUFFIXES))
         return finder.find_module(bad_name)
 
     def test_case_sensitive(self):
diff --git a/Lib/test/test_importlib/extension/test_path_hook.py b/Lib/test/test_importlib/extension/test_path_hook.py
--- a/Lib/test/test_importlib/extension/test_path_hook.py
+++ b/Lib/test/test_importlib/extension/test_path_hook.py
@@ -2,7 +2,6 @@
 from . import util
 
 import collections
-import imp
 import sys
 import unittest
 
diff --git a/Lib/test/test_importlib/extension/util.py b/Lib/test/test_importlib/extension/util.py
--- a/Lib/test/test_importlib/extension/util.py
+++ b/Lib/test/test_importlib/extension/util.py
@@ -1,4 +1,3 @@
-import imp
 from importlib import machinery
 import os
 import sys
diff --git a/Lib/test/test_importlib/frozen/test_loader.py b/Lib/test/test_importlib/frozen/test_loader.py
--- a/Lib/test/test_importlib/frozen/test_loader.py
+++ b/Lib/test/test_importlib/frozen/test_loader.py
@@ -1,9 +1,11 @@
-from importlib import machinery
-import imp
-import unittest
 from .. import abc
 from .. import util
+
+from importlib import machinery
+import unittest
 from test.support import captured_stdout
+import types
+
 
 class LoaderTests(abc.LoaderTests):
 
@@ -85,7 +87,7 @@
         name = '__hello__'
         with captured_stdout() as stdout:
             code = machinery.FrozenImporter.get_code(name)
-            mod = imp.new_module(name)
+            mod = types.ModuleType(name)
             exec(code, mod.__dict__)
             self.assertTrue(hasattr(mod, 'initialized'))
             self.assertEqual(stdout.getvalue(), 'Hello world!\n')
diff --git a/Lib/test/test_importlib/import_/test___loader__.py b/Lib/test/test_importlib/import_/test___loader__.py
--- a/Lib/test/test_importlib/import_/test___loader__.py
+++ b/Lib/test/test_importlib/import_/test___loader__.py
@@ -1,5 +1,5 @@
-import imp
 import sys
+import types
 import unittest
 
 from .. import util
@@ -19,7 +19,7 @@
 class LoaderAttributeTests(unittest.TestCase):
 
     def test___loader___missing(self):
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         try:
             del module.__loader__
         except AttributeError:
@@ -31,7 +31,7 @@
         self.assertEqual(loader, module.__loader__)
 
     def test___loader___is_None(self):
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         module.__loader__ = None
         loader = LoaderMock()
         loader.module = module
diff --git a/Lib/test/test_importlib/import_/test_api.py b/Lib/test/test_importlib/import_/test_api.py
--- a/Lib/test/test_importlib/import_/test_api.py
+++ b/Lib/test/test_importlib/import_/test_api.py
@@ -2,6 +2,7 @@
 from . import util
 import imp
 import sys
+import types
 import unittest
 
 
@@ -48,7 +49,7 @@
     def test_nonexistent_fromlist_entry(self):
         # If something in fromlist doesn't exist, that's okay.
         # issue15715
-        mod = imp.new_module('fine')
+        mod = types.ModuleType('fine')
         mod.__path__ = ['XXX']
         with importlib_test_util.import_state(meta_path=[BadLoaderFinder]):
             with importlib_test_util.uncache('fine'):
@@ -59,7 +60,7 @@
         # If something in fromlist triggers an exception not related to not
         # existing, let that exception propagate.
         # issue15316
-        mod = imp.new_module('fine')
+        mod = types.ModuleType('fine')
         mod.__path__ = ['XXX']
         with importlib_test_util.import_state(meta_path=[BadLoaderFinder]):
             with importlib_test_util.uncache('fine'):
diff --git a/Lib/test/test_importlib/import_/test_fromlist.py b/Lib/test/test_importlib/import_/test_fromlist.py
--- a/Lib/test/test_importlib/import_/test_fromlist.py
+++ b/Lib/test/test_importlib/import_/test_fromlist.py
@@ -1,7 +1,6 @@
 """Test that the semantics relating to the 'fromlist' argument are correct."""
 from .. import util
 from . import util as import_util
-import imp
 import unittest
 
 class ReturnValue(unittest.TestCase):
diff --git a/Lib/test/test_importlib/source/test_case_sensitivity.py b/Lib/test/test_importlib/source/test_case_sensitivity.py
--- a/Lib/test/test_importlib/source/test_case_sensitivity.py
+++ b/Lib/test/test_importlib/source/test_case_sensitivity.py
@@ -1,9 +1,9 @@
 """Test case-sensitivity (PEP 235)."""
+from .. import util
+from . import util as source_util
+
 from importlib import _bootstrap
 from importlib import machinery
-from .. import util
-from . import util as source_util
-import imp
 import os
 import sys
 from test import support as test_support
diff --git a/Lib/test/test_importlib/source/test_file_loader.py b/Lib/test/test_importlib/source/test_file_loader.py
--- a/Lib/test/test_importlib/source/test_file_loader.py
+++ b/Lib/test/test_importlib/source/test_file_loader.py
@@ -1,6 +1,7 @@
 from importlib import machinery
 import importlib
 import importlib.abc
+import importlib.util
 from .. import abc
 from .. import util
 from . import util as source_util
@@ -13,6 +14,7 @@
 import shutil
 import stat
 import sys
+import types
 import unittest
 
 from test.support import make_legacy_pyc, unload
@@ -112,7 +114,7 @@
         value = '<test>'
         name = '_temp'
         with source_util.create_modules(name) as mapping:
-            orig_module = imp.new_module(name)
+            orig_module = types.ModuleType(name)
             for attr in attributes:
                 setattr(orig_module, attr, value)
             with open(mapping[name], 'w') as file:
@@ -144,11 +146,11 @@
                 loader = machinery.SourceFileLoader('_temp', file_path)
                 mod = loader.load_module('_temp')
                 self.assertEqual(file_path, mod.__file__)
-                self.assertEqual(imp.cache_from_source(file_path),
+                self.assertEqual(importlib.util.cache_from_source(file_path),
                                  mod.__cached__)
         finally:
             os.unlink(file_path)
-            pycache = os.path.dirname(imp.cache_from_source(file_path))
+            pycache = os.path.dirname(importlib.util.cache_from_source(file_path))
             if os.path.exists(pycache):
                 shutil.rmtree(pycache)
 
@@ -157,7 +159,7 @@
         # truncated rather than raise an OverflowError.
         with source_util.create_modules('_temp') as mapping:
             source = mapping['_temp']
-            compiled = imp.cache_from_source(source)
+            compiled = importlib.util.cache_from_source(source)
             with open(source, 'w') as f:
                 f.write("x = 5")
             try:
@@ -194,7 +196,7 @@
             pass
         py_compile.compile(mapping[name])
         if not del_source:
-            bytecode_path = imp.cache_from_source(mapping[name])
+            bytecode_path = importlib.util.cache_from_source(mapping[name])
         else:
             os.unlink(mapping[name])
             bytecode_path = make_legacy_pyc(mapping[name])
@@ -322,7 +324,8 @@
         def test(name, mapping, bytecode_path):
             self.import_(mapping[name], name)
             with open(bytecode_path, 'rb') as bytecode_file:
-                self.assertEqual(bytecode_file.read(4), imp.get_magic())
+                self.assertEqual(bytecode_file.read(4),
+                                 importlib.util.MAGIC_NUMBER)
 
         self._test_bad_magic(test)
 
@@ -372,7 +375,7 @@
         zeros = b'\x00\x00\x00\x00'
         with source_util.create_modules('_temp') as mapping:
             py_compile.compile(mapping['_temp'])
-            bytecode_path = imp.cache_from_source(mapping['_temp'])
+            bytecode_path = importlib.util.cache_from_source(mapping['_temp'])
             with open(bytecode_path, 'r+b') as bytecode_file:
                 bytecode_file.seek(4)
                 bytecode_file.write(zeros)
@@ -390,7 +393,7 @@
         with source_util.create_modules('_temp') as mapping:
             # Create bytecode that will need to be re-created.
             py_compile.compile(mapping['_temp'])
-            bytecode_path = imp.cache_from_source(mapping['_temp'])
+            bytecode_path = importlib.util.cache_from_source(mapping['_temp'])
             with open(bytecode_path, 'r+b') as bytecode_file:
                 bytecode_file.seek(0)
                 bytecode_file.write(b'\x00\x00\x00\x00')
diff --git a/Lib/test/test_importlib/source/test_finder.py b/Lib/test/test_importlib/source/test_finder.py
--- a/Lib/test/test_importlib/source/test_finder.py
+++ b/Lib/test/test_importlib/source/test_finder.py
@@ -3,7 +3,6 @@
 
 from importlib import machinery
 import errno
-import imp
 import os
 import py_compile
 import stat
diff --git a/Lib/test/test_importlib/source/test_path_hook.py b/Lib/test/test_importlib/source/test_path_hook.py
--- a/Lib/test/test_importlib/source/test_path_hook.py
+++ b/Lib/test/test_importlib/source/test_path_hook.py
@@ -1,7 +1,6 @@
 from . import util as source_util
 
 from importlib import machinery
-import imp
 import unittest
 
 
diff --git a/Lib/test/test_importlib/source/util.py b/Lib/test/test_importlib/source/util.py
--- a/Lib/test/test_importlib/source/util.py
+++ b/Lib/test/test_importlib/source/util.py
@@ -2,7 +2,6 @@
 import contextlib
 import errno
 import functools
-import imp
 import os
 import os.path
 import sys
diff --git a/Lib/test/test_importlib/test_abc.py b/Lib/test/test_importlib/test_abc.py
--- a/Lib/test/test_importlib/test_abc.py
+++ b/Lib/test/test_importlib/test_abc.py
@@ -1,15 +1,16 @@
 import importlib
+import importlib.util
 from importlib import abc
 from importlib import machinery
 
 import contextlib
-import imp
 import inspect
 import io
 import marshal
 import os
 import sys
 from test import support
+import types
 import unittest
 from unittest import mock
 
@@ -140,7 +141,7 @@
             self.ins.load_module('something')
 
     def test_module_repr(self):
-        mod = imp.new_module('blah')
+        mod = types.ModuleType('blah')
         with self.assertRaises(NotImplementedError):
             self.ins.module_repr(mod)
         original_repr = repr(mod)
@@ -205,7 +206,7 @@
 
     def test_init_module_attrs(self):
         loader = LoaderSubclass()
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         loader.init_module_attrs(module)
         self.assertEqual(module.__loader__, loader)
 
@@ -215,7 +216,7 @@
 
     def source_to_module(self, data, path=None):
         """Help with source_to_code() tests."""
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         loader = InspectLoaderSubclass()
         if path is None:
             code = loader.source_to_code(data)
@@ -257,7 +258,7 @@
 
     def test_get_code(self):
         # Test success.
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         with mock.patch.object(InspectLoaderSubclass, 'get_source') as mocked:
             mocked.return_value = 'attr = 42'
             loader = InspectLoaderSubclass()
@@ -289,7 +290,7 @@
 
     def init_module_attrs(self, name):
         loader = InspectLoaderSubclass()
-        module = imp.new_module(name)
+        module = types.ModuleType(name)
         loader.init_module_attrs(module)
         self.assertEqual(module.__loader__, loader)
         return module
@@ -390,7 +391,7 @@
             loader = ExecutionLoaderSubclass()
             code = loader.get_code('blah')
         self.assertEqual(code.co_filename, path)
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         exec(code, module.__dict__)
         self.assertEqual(module.attr, 42)
 
@@ -420,7 +421,7 @@
             loader = ExecutionLoaderSubclass()
             code = loader.get_code('blah')
         self.assertEqual(code.co_filename, '<string>')
-        module = imp.new_module('blah')
+        module = types.ModuleType('blah')
         exec(code, module.__dict__)
         self.assertEqual(module.attr, 42)
 
@@ -444,7 +445,7 @@
         path = os.path.join('some', 'path', '{}.py'.format(name))
         with self.mock_methods(False, path):
             loader = ExecutionLoaderSubclass()
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             loader.init_module_attrs(module)
         self.assertIs(module.__loader__, loader)
         self.assertEqual(module.__file__, path)
@@ -457,7 +458,7 @@
         path = os.path.join('some', 'pkg', '__init__.py')
         with self.mock_methods(True, path):
             loader = ExecutionLoaderSubclass()
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             loader.init_module_attrs(module)
         self.assertIs(module.__loader__, loader)
         self.assertEqual(module.__file__, path)
@@ -471,7 +472,7 @@
         path = os.path.join('some', 'pkg', 'submodule.py')
         with self.mock_methods(False, path):
             loader = ExecutionLoaderSubclass()
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             loader.init_module_attrs(module)
         self.assertEqual(module.__package__, 'pkg')
         self.assertEqual(module.__file__, path)
@@ -484,7 +485,7 @@
         with self.mock_methods(False, path) as mocked_methods:
             mocked_methods['get_filename'].side_effect = ImportError
             loader = ExecutionLoaderSubclass()
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             loader.init_module_attrs(module)
         self.assertFalse(hasattr(module, '__file__'))
 
@@ -515,9 +516,9 @@
 
     source_mtime = 1
 
-    def __init__(self, path, magic=imp.get_magic()):
+    def __init__(self, path, magic=importlib.util.MAGIC_NUMBER):
         super().__init__(path)
-        self.bytecode_path = imp.cache_from_source(self.path)
+        self.bytecode_path = importlib.util.cache_from_source(self.path)
         self.source_size = len(self.source)
         data = bytearray(magic)
         data.extend(importlib._w_long(self.source_mtime))
@@ -557,7 +558,7 @@
             module_name = 'mod'
             self.path = os.path.join(self.package, '.'.join(['mod', 'py']))
             self.name = '.'.join([self.package, module_name])
-        self.cached = imp.cache_from_source(self.path)
+        self.cached = importlib.util.cache_from_source(self.path)
         self.loader = self.loader_mock(self.path, **kwargs)
 
     def verify_module(self, module):
@@ -574,7 +575,7 @@
         self.assertEqual(values[4], repr(self.loader))
 
     def verify_code(self, code_object):
-        module = imp.new_module(self.name)
+        module = types.ModuleType(self.name)
         module.__file__ = self.path
         module.__cached__ = self.cached
         module.__package__ = self.package
@@ -673,7 +674,7 @@
         super().verify_code(code_object)
         if bytecode_written:
             self.assertIn(self.cached, self.loader.written)
-            data = bytearray(imp.get_magic())
+            data = bytearray(importlib.util.MAGIC_NUMBER)
             data.extend(importlib._w_long(self.loader.source_mtime))
             data.extend(importlib._w_long(self.loader.source_size))
             data.extend(marshal.dumps(code_object))
@@ -689,7 +690,7 @@
         self.loader.bytecode_path = "<does not exist>"
         # Sanity check
         with self.assertRaises(OSError):
-            bytecode_path = imp.cache_from_source(self.path)
+            bytecode_path = importlib.util.cache_from_source(self.path)
             self.loader.get_data(bytecode_path)
         code_object = self.loader.get_code(self.name)
         self.verify_code(code_object, bytecode_written=True)
@@ -787,26 +788,26 @@
     """Tests for importlib.abc.SourceLoader.init_module_attrs()."""
 
     def test_init_module_attrs(self):
-        # If __file__ set, __cached__ == imp.cached_from_source(__file__).
+        # If __file__ set, __cached__ == importlib.util.cached_from_source(__file__).
         name = 'blah'
         path = 'blah.py'
         loader = SourceOnlyLoaderMock(path)
-        module = imp.new_module(name)
+        module = types.ModuleType(name)
         loader.init_module_attrs(module)
         self.assertEqual(module.__loader__, loader)
         self.assertEqual(module.__package__, '')
         self.assertEqual(module.__file__, path)
-        self.assertEqual(module.__cached__, imp.cache_from_source(path))
+        self.assertEqual(module.__cached__, importlib.util.cache_from_source(path))
 
     @mock.patch('importlib._bootstrap.cache_from_source')
     def test_cache_from_source_NotImplementedError(self, mock_cache_from_source):
-        # If imp.cache_from_source() raises NotImplementedError don't set
+        # If importlib.util.cache_from_source() raises NotImplementedError don't set
         # __cached__.
         mock_cache_from_source.side_effect = NotImplementedError
         name = 'blah'
         path = 'blah.py'
         loader = SourceOnlyLoaderMock(path)
-        module = imp.new_module(name)
+        module = types.ModuleType(name)
         loader.init_module_attrs(module)
         self.assertEqual(module.__file__, path)
         self.assertFalse(hasattr(module, '__cached__'))
@@ -817,7 +818,7 @@
             mocked.side_effect = ImportError
             name = 'blah'
             loader = SourceOnlyLoaderMock('blah.py')
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             loader.init_module_attrs(module)
         self.assertFalse(hasattr(module, '__file__'))
         self.assertFalse(hasattr(module, '__cached__'))
diff --git a/Lib/test/test_importlib/test_api.py b/Lib/test/test_importlib/test_api.py
--- a/Lib/test/test_importlib/test_api.py
+++ b/Lib/test/test_importlib/test_api.py
@@ -1,5 +1,5 @@
 from . import util
-import imp
+
 import importlib
 from importlib import _bootstrap
 from importlib import machinery
@@ -99,7 +99,7 @@
         # If a module with __loader__ is in sys.modules, then return it.
         name = 'some_mod'
         with util.uncache(name):
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             loader = 'a loader!'
             module.__loader__ = loader
             sys.modules[name] = module
@@ -110,7 +110,7 @@
         # If sys.modules[name].__loader__ is None, raise ValueError.
         name = 'some_mod'
         with util.uncache(name):
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             module.__loader__ = None
             sys.modules[name] = module
             with self.assertRaises(ValueError):
@@ -121,7 +121,7 @@
         # Issue #17099
         name = 'some_mod'
         with util.uncache(name):
-            module = imp.new_module(name)
+            module = types.ModuleType(name)
             try:
                 del module.__loader__
             except AttributeError:
@@ -189,7 +189,7 @@
     def test_method_lacking(self):
         # There should be no issues if the method is not defined.
         key = 'gobbledeegook'
-        sys.path_importer_cache[key] = imp.NullImporter('abc')
+        sys.path_importer_cache[key] = None
         self.addCleanup(lambda: sys.path_importer_cache.__delitem__(key))
         importlib.invalidate_caches()  # Shouldn't trigger an exception.
 
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -1,6 +1,6 @@
 from importlib import util
 from . import util as test_util
-import imp
+
 import os
 import sys
 from test import support
@@ -40,14 +40,14 @@
 
     def test_reload(self):
         # Test that the same module is in sys.modules.
-        created_module = imp.new_module(self.module_name)
+        created_module = types.ModuleType(self.module_name)
         sys.modules[self.module_name] = created_module
         with util.module_to_load(self.module_name) as module:
             self.assertIs(module, created_module)
 
     def test_reload_failed(self):
         # Test that the module was left in sys.modules.
-        created_module = imp.new_module(self.module_name)
+        created_module = types.ModuleType(self.module_name)
         sys.modules[self.module_name] = created_module
         try:
             with util.module_to_load(self.module_name) as module:
@@ -60,7 +60,7 @@
     def test_reset_name(self):
         # If reset_name is true then module.__name__ = name, else leave it be.
         odd_name = 'not your typical name'
-        created_module = imp.new_module(self.module_name)
+        created_module = types.ModuleType(self.module_name)
         created_module.__name__ = odd_name
         sys.modules[self.module_name] = created_module
         with util.module_to_load(self.module_name) as module:
@@ -119,7 +119,7 @@
             def load_module(self, module):
                 return module
         name = 'a.b.c'
-        module = imp.new_module('a.b.c')
+        module = types.ModuleType('a.b.c')
         module.__loader__ = 42
         module.__package__ = 42
         with test_util.uncache(name):
@@ -141,7 +141,7 @@
     def test_reload_failure(self):
         # Test that a failure on reload leaves the module in-place.
         name = 'a.b.c'
-        module = imp.new_module(name)
+        module = types.ModuleType(name)
         with test_util.uncache(name):
             sys.modules[name] = module
             self.raise_exception(name)
@@ -212,26 +212,26 @@
     def test_top_level(self):
         # __package__ should be set to the empty string if a top-level module.
         # Implicitly tests when package is set to None.
-        module = imp.new_module('module')
+        module = types.ModuleType('module')
         module.__package__ = None
         self.verify(module, '')
 
     def test_package(self):
         # Test setting __package__ for a package.
-        module = imp.new_module('pkg')
+        module = types.ModuleType('pkg')
         module.__path__ = ['<path>']
         module.__package__ = None
         self.verify(module, 'pkg')
 
     def test_submodule(self):
         # Test __package__ for a module in a package.
-        module = imp.new_module('pkg.mod')
+        module = types.ModuleType('pkg.mod')
         module.__package__ = None
         self.verify(module, 'pkg')
 
     def test_setting_if_missing(self):
         # __package__ should be set if it is missing.
-        module = imp.new_module('mod')
+        module = types.ModuleType('mod')
         if hasattr(module, '__package__'):
             delattr(module, '__package__')
         self.verify(module, '')
@@ -239,7 +239,7 @@
     def test_leaving_alone(self):
         # If __package__ is set and not None then leave it alone.
         for value in (True, False):
-            module = imp.new_module('mod')
+            module = types.ModuleType('mod')
             module.__package__ = value
             self.verify(module, value)
 
@@ -261,7 +261,7 @@
 
     def test_no_attribute(self):
         loader = self.DummyLoader()
-        loader.module = imp.new_module('blah')
+        loader.module = types.ModuleType('blah')
         try:
             del loader.module.__loader__
         except AttributeError:
@@ -270,13 +270,13 @@
 
     def test_attribute_is_None(self):
         loader = self.DummyLoader()
-        loader.module = imp.new_module('blah')
+        loader.module = types.ModuleType('blah')
         loader.module.__loader__ = None
         self.assertEqual(loader, loader.load_module('blah').__loader__)
 
     def test_not_reset(self):
         loader = self.DummyLoader()
-        loader.module = imp.new_module('blah')
+        loader.module = types.ModuleType('blah')
         loader.module.__loader__ = 42
         self.assertEqual(42, loader.load_module('blah').__loader__)
 
@@ -331,7 +331,7 @@
 
     """
 
-    tag = imp.get_tag()
+    tag = sys.implementation.cache_tag
 
     @unittest.skipUnless(sys.implementation.cache_tag is not None,
                          'requires sys.implementation.cache_tag not be None')
diff --git a/Lib/test/test_importlib/util.py b/Lib/test/test_importlib/util.py
--- a/Lib/test/test_importlib/util.py
+++ b/Lib/test/test_importlib/util.py
@@ -1,9 +1,9 @@
 from contextlib import contextmanager
-import imp
 import os.path
 from test import support
 import unittest
 import sys
+import types
 
 
 CASE_INSENSITIVE_FS = True
@@ -98,7 +98,7 @@
                 package = name.rsplit('.', 1)[0]
             else:
                 package = import_name
-            module = imp.new_module(import_name)
+            module = types.ModuleType(import_name)
             module.__loader__ = self
             module.__file__ = '<mock __file__>'
             module.__package__ = package

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 03:15:10 2013
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 16 Jun 2013 03:15:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Prevent_a_poss?=
 =?utf-8?q?ible_double_close_of_parent_pipe_fds_when_the_subprocess?=
Message-ID: <3bXyKB25fNz7Lkh@mail.python.org>

http://hg.python.org/cpython/rev/be08c8b8c574
changeset:   84160:be08c8b8c574
branch:      3.3
parent:      84152:3d75bd69e5a9
user:        Gregory P. Smith <greg at krypto.org>
date:        Sat Jun 15 18:04:26 2013 -0700
summary:
  Prevent a possible double close of parent pipe fds when the subprocess
exec runs into an error.  Prevent a regular multi-close of the /dev/null
fd when any of stdin, stdout and stderr was set to DEVNULL.

files:
  Lib/subprocess.py |  43 ++++++++++++++++++++--------------
  1 files changed, 25 insertions(+), 18 deletions(-)


diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -810,6 +810,7 @@
             if universal_newlines:
                 self.stderr = io.TextIOWrapper(self.stderr)
 
+        self._closed_child_pipe_fds = False
         try:
             self._execute_child(args, executable, preexec_fn, close_fds,
                                 pass_fds, cwd, env,
@@ -826,19 +827,21 @@
                 except EnvironmentError:
                     pass  # Ignore EBADF or other errors.
 
-            # Make sure the child pipes are closed as well.
-            to_close = []
-            if stdin == PIPE:
-                to_close.append(p2cread)
-            if stdout == PIPE:
-                to_close.append(c2pwrite)
-            if stderr == PIPE:
-                to_close.append(errwrite)
-            for fd in to_close:
-                try:
-                    os.close(fd)
-                except EnvironmentError:
-                    pass
+            if not self._closed_child_pipe_fds:
+                to_close = []
+                if stdin == PIPE:
+                    to_close.append(p2cread)
+                if stdout == PIPE:
+                    to_close.append(c2pwrite)
+                if stderr == PIPE:
+                    to_close.append(errwrite)
+                if hasattr(self, '_devnull'):
+                    to_close.append(self._devnull)
+                for fd in to_close:
+                    try:
+                        os.close(fd)
+                    except EnvironmentError:
+                        pass
 
             raise
 
@@ -1383,14 +1386,18 @@
                     # be sure the FD is closed no matter what
                     os.close(errpipe_write)
 
-                if p2cread != -1 and p2cwrite != -1:
+                # self._devnull is not always defined.
+                devnull_fd = getattr(self, '_devnull', None)
+                if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                     os.close(p2cread)
-                if c2pwrite != -1 and c2pread != -1:
+                if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                     os.close(c2pwrite)
-                if errwrite != -1 and errread != -1:
+                if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                     os.close(errwrite)
-                if hasattr(self, '_devnull'):
-                    os.close(self._devnull)
+                if devnull_fd is not None:
+                    os.close(devnull_fd)
+                # Prevent a double close of these fds from __init__ on error.
+                self._closed_child_pipe_fds = True
 
                 # Wait for exec to fail or succeed; possibly raising an
                 # exception (limited in size)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 03:15:11 2013
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 16 Jun 2013 03:15:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_news_entry_for?=
 =?utf-8?q?_subprocess_double_close_fix=2E?=
Message-ID: <3bXyKC3y15z7Lk6@mail.python.org>

http://hg.python.org/cpython/rev/7e06a99bb821
changeset:   84161:7e06a99bb821
branch:      3.3
user:        Gregory P. Smith <greg at krypto.org>
date:        Sat Jun 15 18:05:17 2013 -0700
summary:
  news entry for subprocess double close fix.

files:
  Misc/NEWS |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -32,6 +32,10 @@
 Library
 -------
 
+- subprocess: Prevent a possible double close of parent pipe fds when the
+  subprocess exec runs into an error.  Prevent a regular multi-close of the
+  /dev/null fd when any of stdin, stdout and stderr was set to DEVNULL.
+
 - Issue #16102: Make uuid._netbios_getnode() work again on Python 3.
 
 - Issue #18109: os.uname() now decodes fields from the locale encoding, and

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 03:15:13 2013
From: python-checkins at python.org (gregory.p.smith)
Date: Sun, 16 Jun 2013 03:15:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Prevent_a_possible_double_close_of_parent_pipe_fds_when_?=
 =?utf-8?q?the_subprocess?=
Message-ID: <3bXyKF028pz7Lkj@mail.python.org>

http://hg.python.org/cpython/rev/7dee56b6ff34
changeset:   84162:7dee56b6ff34
parent:      84159:5e8b377942f7
parent:      84161:7e06a99bb821
user:        Gregory P. Smith <greg at krypto.org>
date:        Sat Jun 15 18:14:56 2013 -0700
summary:
  Prevent a possible double close of parent pipe fds when the subprocess
exec runs into an error.  Prevent a regular multi-close of the /dev/null
fd when any of stdin, stdout and stderr was set to DEVNULL.

files:
  Lib/subprocess.py |  43 ++++++++++++++++++++--------------
  Misc/NEWS         |   4 +++
  2 files changed, 29 insertions(+), 18 deletions(-)


diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -828,6 +828,7 @@
             if universal_newlines:
                 self.stderr = io.TextIOWrapper(self.stderr)
 
+        self._closed_child_pipe_fds = False
         try:
             self._execute_child(args, executable, preexec_fn, close_fds,
                                 pass_fds, cwd, env,
@@ -844,19 +845,21 @@
                 except OSError:
                     pass  # Ignore EBADF or other errors.
 
-            # Make sure the child pipes are closed as well.
-            to_close = []
-            if stdin == PIPE:
-                to_close.append(p2cread)
-            if stdout == PIPE:
-                to_close.append(c2pwrite)
-            if stderr == PIPE:
-                to_close.append(errwrite)
-            for fd in to_close:
-                try:
-                    os.close(fd)
-                except OSError:
-                    pass
+            if not self._closed_child_pipe_fds:
+                to_close = []
+                if stdin == PIPE:
+                    to_close.append(p2cread)
+                if stdout == PIPE:
+                    to_close.append(c2pwrite)
+                if stderr == PIPE:
+                    to_close.append(errwrite)
+                if hasattr(self, '_devnull'):
+                    to_close.append(self._devnull)
+                for fd in to_close:
+                    try:
+                        os.close(fd)
+                    except OSError:
+                        pass
 
             raise
 
@@ -1363,14 +1366,18 @@
                     # be sure the FD is closed no matter what
                     os.close(errpipe_write)
 
-                if p2cread != -1 and p2cwrite != -1:
+                # self._devnull is not always defined.
+                devnull_fd = getattr(self, '_devnull', None)
+                if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                     os.close(p2cread)
-                if c2pwrite != -1 and c2pread != -1:
+                if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                     os.close(c2pwrite)
-                if errwrite != -1 and errread != -1:
+                if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                     os.close(errwrite)
-                if hasattr(self, '_devnull'):
-                    os.close(self._devnull)
+                if devnull_fd is not None:
+                    os.close(devnull_fd)
+                # Prevent a double close of these fds from __init__ on error.
+                self._closed_child_pipe_fds = True
 
                 # Wait for exec to fail or succeed; possibly raising an
                 # exception (limited in size)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,10 @@
 Library
 -------
 
+- subprocess: Prevent a possible double close of parent pipe fds when the
+  subprocess exec runs into an error.  Prevent a regular multi-close of the
+  /dev/null fd when any of stdin, stdout and stderr was set to DEVNULL.
+
 - Issue #18194: Introduce importlib.util.cache_from_source() and
   source_from_cache() while documenting the equivalent functions in imp as
   deprecated.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 03:49:42 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 16 Jun 2013 03:49:42 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_more_background?=
Message-ID: <3bXz524Vnhz7LjS@mail.python.org>

http://hg.python.org/peps/rev/5e26a6276514
changeset:   4932:5e26a6276514
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 16 03:49:29 2013 +0200
summary:
  PEP 445: more background

files:
  pep-0445.txt |  59 +++++++++++++++++++++++++++++++++++++++-
  1 files changed, 58 insertions(+), 1 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -151,13 +151,65 @@
 XXX To be done (Kristj?n Valur J?nsson) XXX
 
 
+External libraries
+==================
+
+* glib: `g_mem_set_vtable()
+  <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
+
+
+Memory allocators
+=================
+
+The C standard library provides the well known ``malloc()`` function. Its
+implementation depends on the platform and of the C library. The GNU C library
+uses a modified ptmalloc2, based on "Doug Lea's Malloc" (dlmalloc). FreeBSD
+uses `jemalloc <http://www.canonware.com/jemalloc/>`_. Google provides
+tcmalloc which is part of `gperftools <http://code.google.com/p/gperftools/>`_.
+
+``malloc()`` uses two kinds of memory: heap and memory mappings. Memory
+mappings are usually used for large allocations (ex: larger than 256 KB),
+whereas the heap is used for small allocations.
+
+The heap is handled by ``brk()`` and ``sbrk()`` system calls on Linux, and is
+contiguous.  Memory mappings are handled by ``mmap()`` on UNIX and
+``VirtualAlloc()`` on Windows, they are discontiguous. Releasing a memory
+mapping gives back the memory immediatly to the system. For the heap, memory is
+only gave back to the system if it is at the end of the heap. Otherwise, the
+memory will only gave back to the system when all the memory located after the
+released memory are also released. This limitation causes an issue called the
+"memory fragmentation": the memory usage seen by the system may be much higher
+than real usage.
+
+Windows provides a `Low-fragmentation Heap
+<http://msdn.microsoft.com/en-us/library/windows/desktop/aa366750%28v=vs.85%29.aspx>`_.
+
+The Linux kernel uses `slab allocation
+<http://en.wikipedia.org/wiki/Slab_allocation>`_.
+
+The glib library has a `Memory Slice API
+<https://developer.gnome.org/glib/unstable/glib-Memory-Slices.html>`_:
+efficient way to allocate groups of equal-sized chunks of memory
+
+
 Links
 =====
 
-Memory allocators:
+CPython issues related to memory allocation:
 
 * `Issue #3329: Add new APIs to customize memory allocators
   <http://bugs.python.org/issue3329>`_
+* `Issue #13483: Use VirtualAlloc to allocate memory arenas
+  <http://bugs.python.org/issue13483>`_
+* `Issue #16742: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which
+  isn't thread safe <http://bugs.python.org/issue16742>`_
+* `Issue #18203: Replace calls to malloc() with PyMem_Malloc()
+  <http://bugs.python.org/issue18203>`_
+* `Issue #18227: Use Python memory allocators in external libraries like zlib
+  or OpenSSL <http://bugs.python.org/issue18227>`_
+
+Projects analyzing the memory usage of Python applications:
+
 * `pytracemalloc
   <https://pypi.python.org/pypi/pytracemalloc>`_
 * `Meliae: Python Memory Usage Analyzer
@@ -167,6 +219,11 @@
 * `PySizer (developed for Python 2.4)
   <http://pysizer.8325.org/>`_
 
+APIs to set a custom memory allocator and/or hook memory allocators:
+
+* `GNU libc: Memory Allocation Hooks
+  <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_
+
 Other:
 
 * `Python benchmark suite

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sun Jun 16 04:01:12 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 16 Jun 2013 04:01:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bXzLJ3rDzzQWx@mail.python.org>

http://hg.python.org/peps/rev/add8a583c05d
changeset:   4933:add8a583c05d
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 16 04:01:00 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  10 ++++++++--
  1 files changed, 8 insertions(+), 2 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -99,8 +99,14 @@
 
 with 2 functions with an additional *domain* argument:
 
-* ``PyMem_GetAllocators(domain)``
-* ``PyMem_SetAllocators(domain, allocators)``
+* ``Py_GetAllocators(domain)``
+* ``Py_SetAllocators(domain, allocators)``
+
+where domain is one of these values:
+
+* ``PYALLOC_PYMEM``
+* ``PYALLOC_PYMEM_RAW``
+* ``PYALLOC_PYOBJECT``
 
 
 Setup Builtin Debug Hooks

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sun Jun 16 04:03:28 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 16 Jun 2013 04:03:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_reorder?=
Message-ID: <3bXzNw4pGyzStl@mail.python.org>

http://hg.python.org/peps/rev/e3e07f064d7f
changeset:   4934:e3e07f064d7f
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 16 04:03:15 2013 +0200
summary:
  PEP 445: reorder

files:
  pep-0445.txt |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -60,15 +60,15 @@
 
   - ``PyMem_SetupDebugHooks()``
 
+* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call ``malloc()`` and
+  ``realloc()``, instead of calling ``PyObject_Malloc()`` and
+  ``PyObject_Realloc()`` in debug mode
+
 * ``PyObject_Malloc()`` now falls back on ``PyMem_Malloc()`` instead of
   ``malloc()`` if size is bigger than ``SMALL_REQUEST_THRESHOLD``, and
   ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of
   ``realloc()``
 
-* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call ``malloc()`` and
-  ``realloc()``, instead of calling ``PyObject_Malloc()`` and
-  ``PyObject_Realloc()`` in debug mode
-
 
 Performances
 ============

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sun Jun 16 05:24:19 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 05:24:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_test=5Fbuiltin_work_w?=
	=?utf-8?q?hen_executed_directly?=
Message-ID: <3bY1BC39KzzSrd@mail.python.org>

http://hg.python.org/cpython/rev/5b90da280515
changeset:   84163:5b90da280515
user:        Brett Cannon <brett at python.org>
date:        Sat Jun 15 23:24:11 2013 -0400
summary:
  Make test_builtin work when executed directly

files:
  Lib/test/test_builtin.py |  5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py
--- a/Lib/test/test_builtin.py
+++ b/Lib/test/test_builtin.py
@@ -584,7 +584,10 @@
                 raise frozendict_error("frozendict is readonly")
 
         # read-only builtins
-        frozen_builtins = frozendict(__builtins__)
+        if isinstance(__builtins__, types.ModuleType):
+            frozen_builtins = frozendict(__builtins__.__dict__)
+        else:
+            frozen_builtins = frozendict(__builtins__)
         code = compile("__builtins__['superglobal']=2; print(superglobal)", "test", "exec")
         self.assertRaises(frozendict_error,
                           exec, code, {'__builtins__': frozen_builtins})

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sun Jun 16 05:52:13 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 16 Jun 2013 05:52:13 +0200
Subject: [Python-checkins] Daily reference leaks (7dee56b6ff34): sum=2
Message-ID: <E1Uo40j-0004ni-0G@ap.vmr.nerim.net>

results for 7dee56b6ff34 on branch "default"
--------------------------------------------

test_support leaked [0, -1, 1] references, sum=0
test_support leaked [0, -1, 3] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogtBhb00', '-x']

From python-checkins at python.org  Sun Jun 16 17:38:30 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 17:38:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Stop_using_the_deprecated_?=
 =?utf-8?q?unittest=2ETestCase=2EassertRaisesRegexp=28=29?=
Message-ID: <3bYKTL0ZYJz7LkX@mail.python.org>

http://hg.python.org/cpython/rev/4f7c25ab2ed2
changeset:   84164:4f7c25ab2ed2
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 11:37:57 2013 -0400
summary:
  Stop using the deprecated unittest.TestCase.assertRaisesRegexp()

files:
  Lib/test/test_source_encoding.py |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_source_encoding.py b/Lib/test/test_source_encoding.py
--- a/Lib/test/test_source_encoding.py
+++ b/Lib/test/test_source_encoding.py
@@ -62,17 +62,17 @@
         compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
         compile(b'\xef\xbb\xbf\n', 'dummy', 'exec')
         compile(b'\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+        with self.assertRaisesRegex(SyntaxError, 'fake'):
             compile(b'# -*- coding: fake -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
+        with self.assertRaisesRegex(SyntaxError, 'iso-8859-15'):
             compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
                     'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+        with self.assertRaisesRegex(SyntaxError, 'BOM'):
             compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
                     'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+        with self.assertRaisesRegex(SyntaxError, 'fake'):
             compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+        with self.assertRaisesRegex(SyntaxError, 'BOM'):
             compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
 
     def test_bad_coding(self):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 19:00:54 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sun, 16 Jun 2013 19:00:54 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogRGVzY3JpYmUgJ3N1?=
 =?utf-8?q?rrogateescape=27_in_the_documentation=2E?=
Message-ID: <3bYMJQ2lJtz7Ljb@mail.python.org>

http://hg.python.org/cpython/rev/55f611f55952
changeset:   84165:55f611f55952
branch:      3.3
parent:      84161:7e06a99bb821
user:        Andrew Kuchling <amk at amk.ca>
date:        Sun Jun 16 12:58:48 2013 -0400
summary:
  Describe 'surrogateescape' in the documentation.

Also, improve some docstring descriptions of the 'errors' parameter.

Closes #14015.

files:
  Doc/library/codecs.rst    |   6 +++-
  Doc/library/functions.rst |  40 ++++++++++++++++++++------
  Lib/codecs.py             |   1 +
  Modules/_io/_iomodule.c   |   4 +-
  Modules/_io/textio.c      |   5 ++-
  5 files changed, 41 insertions(+), 15 deletions(-)


diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -78,7 +78,11 @@
      reference (for encoding only)
    * ``'backslashreplace'``: replace with backslashed escape sequences (for
      encoding only)
-   * ``'surrogateescape'``: replace with surrogate U+DCxx, see :pep:`383`
+   * ``'surrogateescape'``: on decoding, replace with code points in the Unicode
+     Private Use Area ranging from U+DC80 to U+DCFF.  These private code
+     points will then be turned back into the same bytes when the
+     ``surrogateescape`` error handler is used when encoding the data.
+     (See :pep:`383` for more.)
 
    as well as any other error handling name defined via :func:`register_error`.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -895,16 +895,36 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.  Pass
-   ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding
-   error (the default of ``None`` has the same effect), or pass ``'ignore'`` to
-   ignore errors.  (Note that ignoring encoding errors can lead to data loss.)
-   ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
-   where there is malformed data.  When writing, ``'xmlcharrefreplace'``
-   (replace with the appropriate XML character reference) or
-   ``'backslashreplace'`` (replace with backslashed escape sequences) can be
-   used.  Any other error handling name that has been registered with
-   :func:`codecs.register_error` is also valid.
+   errors are to be handled--this cannot be used in binary mode.
+   A variety of standard error handlers are available, though any
+   error handling name that has been registered with
+   :func:`codecs.register_error` is also valid.  The standard names
+   are:
+
+   * ``'strict'`` to raise a :exc:`ValueError` exception if there is
+     an encoding error.  The default value of ``None`` has the same
+     effect.
+
+   * ``'ignore'`` ignores errors.  Note that ignoring encoding errors
+     can lead to data loss.
+
+   * ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
+     where there is malformed data.
+
+   * ``'surrogateescape'`` will represent any incorrect bytes as code
+     points in the Unicode Private Use Area ranging from U+DC80 to
+     U+DCFF.  These private code points will then be turned back into
+     the same bytes when the ``surrogateescape`` error handler is used
+     when writing data.  This is useful for processing files in an
+     unknown encoding.
+
+   * ``'xmlcharrefreplace'`` is only supported when writing to a file.
+     Characters not supported by the encoding are replaced with the
+     appropriate XML character reference ``&#nnn;``.
+
+   * ``'backslashreplace'`` (also only supported when writing)
+     replaces unsupported characters with Python's backslashed escape
+     sequences.
 
    .. index::
       single: universal newlines; open() built-in function
diff --git a/Lib/codecs.py b/Lib/codecs.py
--- a/Lib/codecs.py
+++ b/Lib/codecs.py
@@ -105,6 +105,7 @@
                     Python will use the official U+FFFD REPLACEMENT
                     CHARACTER for the builtin Unicode codecs on
                     decoding and '?' on encoding.
+         'surrogateescape' - replace with private codepoints U+DCnn.
          'xmlcharrefreplace' - Replace with the appropriate XML
                                character reference (only for encoding).
          'backslashreplace'  - Replace with backslashed escape sequences
diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
--- a/Modules/_io/_iomodule.c
+++ b/Modules/_io/_iomodule.c
@@ -168,8 +168,8 @@
 "'strict' to raise a ValueError exception if there is an encoding error\n"
 "(the default of None has the same effect), or pass 'ignore' to ignore\n"
 "errors. (Note that ignoring encoding errors can lead to data loss.)\n"
-"See the documentation for codecs.register for a list of the permitted\n"
-"encoding error strings.\n"
+"See the documentation for codecs.register or run 'help(codecs.Codec)'\n"
+"for a list of the permitted encoding error strings.\n"
 "\n"
 "newline controls how universal newlines works (it only applies to text\n"
 "mode). It can be None, '', '\\n', '\\r', and '\\r\\n'.  It works as\n"
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -642,8 +642,9 @@
     "encoding gives the name of the encoding that the stream will be\n"
     "decoded or encoded with. It defaults to locale.getpreferredencoding(False).\n"
     "\n"
-    "errors determines the strictness of encoding and decoding (see the\n"
-    "codecs.register) and defaults to \"strict\".\n"
+    "errors determines the strictness of encoding and decoding (see\n"
+    "help(codecs.Codec) or the documentation for codecs.register) and\n"
+    "defaults to \"strict\".\n"
     "\n"
     "newline controls how line endings are handled. It can be None, '',\n"
     "'\\n', '\\r', and '\\r\\n'.  It works as follows:\n"

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 19:04:47 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sun, 16 Jun 2013 19:04:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bYMNv5QQQz7Ljb@mail.python.org>

http://hg.python.org/cpython/rev/81648329b37a
changeset:   84166:81648329b37a
parent:      84164:4f7c25ab2ed2
parent:      84165:55f611f55952
user:        Andrew Kuchling <amk at amk.ca>
date:        Sun Jun 16 13:02:55 2013 -0400
summary:
  Merge with 3.3

files:
  Doc/library/codecs.rst    |   6 +++-
  Doc/library/functions.rst |  40 ++++++++++++++++++++------
  Lib/codecs.py             |   1 +
  Modules/_io/_iomodule.c   |   4 +-
  Modules/_io/textio.c      |   5 ++-
  5 files changed, 41 insertions(+), 15 deletions(-)


diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -78,7 +78,11 @@
      reference (for encoding only)
    * ``'backslashreplace'``: replace with backslashed escape sequences (for
      encoding only)
-   * ``'surrogateescape'``: replace with surrogate U+DCxx, see :pep:`383`
+   * ``'surrogateescape'``: on decoding, replace with code points in the Unicode
+     Private Use Area ranging from U+DC80 to U+DCFF.  These private code
+     points will then be turned back into the same bytes when the
+     ``surrogateescape`` error handler is used when encoding the data.
+     (See :pep:`383` for more.)
 
    as well as any other error handling name defined via :func:`register_error`.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -905,16 +905,36 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.  Pass
-   ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding
-   error (the default of ``None`` has the same effect), or pass ``'ignore'`` to
-   ignore errors.  (Note that ignoring encoding errors can lead to data loss.)
-   ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
-   where there is malformed data.  When writing, ``'xmlcharrefreplace'``
-   (replace with the appropriate XML character reference) or
-   ``'backslashreplace'`` (replace with backslashed escape sequences) can be
-   used.  Any other error handling name that has been registered with
-   :func:`codecs.register_error` is also valid.
+   errors are to be handled--this cannot be used in binary mode.
+   A variety of standard error handlers are available, though any
+   error handling name that has been registered with
+   :func:`codecs.register_error` is also valid.  The standard names
+   are:
+
+   * ``'strict'`` to raise a :exc:`ValueError` exception if there is
+     an encoding error.  The default value of ``None`` has the same
+     effect.
+
+   * ``'ignore'`` ignores errors.  Note that ignoring encoding errors
+     can lead to data loss.
+
+   * ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
+     where there is malformed data.
+
+   * ``'surrogateescape'`` will represent any incorrect bytes as code
+     points in the Unicode Private Use Area ranging from U+DC80 to
+     U+DCFF.  These private code points will then be turned back into
+     the same bytes when the ``surrogateescape`` error handler is used
+     when writing data.  This is useful for processing files in an
+     unknown encoding.
+
+   * ``'xmlcharrefreplace'`` is only supported when writing to a file.
+     Characters not supported by the encoding are replaced with the
+     appropriate XML character reference ``&#nnn;``.
+
+   * ``'backslashreplace'`` (also only supported when writing)
+     replaces unsupported characters with Python's backslashed escape
+     sequences.
 
    .. index::
       single: universal newlines; open() built-in function
diff --git a/Lib/codecs.py b/Lib/codecs.py
--- a/Lib/codecs.py
+++ b/Lib/codecs.py
@@ -105,6 +105,7 @@
                     Python will use the official U+FFFD REPLACEMENT
                     CHARACTER for the builtin Unicode codecs on
                     decoding and '?' on encoding.
+         'surrogateescape' - replace with private codepoints U+DCnn.
          'xmlcharrefreplace' - Replace with the appropriate XML
                                character reference (only for encoding).
          'backslashreplace'  - Replace with backslashed escape sequences
diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
--- a/Modules/_io/_iomodule.c
+++ b/Modules/_io/_iomodule.c
@@ -168,8 +168,8 @@
 "'strict' to raise a ValueError exception if there is an encoding error\n"
 "(the default of None has the same effect), or pass 'ignore' to ignore\n"
 "errors. (Note that ignoring encoding errors can lead to data loss.)\n"
-"See the documentation for codecs.register for a list of the permitted\n"
-"encoding error strings.\n"
+"See the documentation for codecs.register or run 'help(codecs.Codec)'\n"
+"for a list of the permitted encoding error strings.\n"
 "\n"
 "newline controls how universal newlines works (it only applies to text\n"
 "mode). It can be None, '', '\\n', '\\r', and '\\r\\n'.  It works as\n"
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -642,8 +642,9 @@
     "encoding gives the name of the encoding that the stream will be\n"
     "decoded or encoded with. It defaults to locale.getpreferredencoding(False).\n"
     "\n"
-    "errors determines the strictness of encoding and decoding (see the\n"
-    "codecs.register) and defaults to \"strict\".\n"
+    "errors determines the strictness of encoding and decoding (see\n"
+    "help(codecs.Codec) or the documentation for codecs.register) and\n"
+    "defaults to \"strict\".\n"
     "\n"
     "newline controls how line endings are handled. It can be None, '',\n"
     "'\\n', '\\r', and '\\r\\n'.  It works as follows:\n"

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 19:14:15 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 19:14:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_The_imp_?=
 =?utf-8?q?module_is_pending_deprecation=2E?=
Message-ID: <3bYMbq4xFSz7Ljd@mail.python.org>

http://hg.python.org/cpython/rev/1b8f08c4efd5
changeset:   84167:1b8f08c4efd5
parent:      84164:4f7c25ab2ed2
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 13:13:40 2013 -0400
summary:
  Issue #17177: The imp module is pending deprecation.

To make sure there is no issue with code that is both Python 2 and 3
compatible, there are no plans to remove the module any sooner than
Python 4 (unless the community moves to Python 3 solidly before then).

files:
  Doc/library/imp.rst                                |   7 +-
  Doc/whatsnew/3.4.rst                               |   3 +
  Lib/imp.py                                         |  93 ++++-----
  Lib/inspect.py                                     |   4 +-
  Lib/modulefinder.py                                |   5 +-
  Lib/pkgutil.py                                     |  22 +-
  Lib/runpy.py                                       |   8 +-
  Lib/test/test_fork1.py                             |   2 +-
  Lib/test/test_imp.py                               |   4 +-
  Lib/test/test_import.py                            |  32 +-
  Lib/test/test_importlib/import_/test_api.py        |   1 -
  Lib/test/test_importlib/source/test_file_loader.py |   1 -
  Lib/test/test_socketserver.py                      |   2 +-
  Lib/test/test_threaded_import.py                   |   2 +-
  Misc/NEWS                                          |   2 +
  15 files changed, 103 insertions(+), 85 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -2,7 +2,7 @@
 ================================================================
 
 .. deprecated:: 3.4
-   The :mod:`imp` package has been deprecated in favor of :mod:`importlib`.
+   The :mod:`imp` package is pending deprecation in favor of :mod:`importlib`.
 
 .. module:: imp
    :synopsis: Access the implementation of the import statement.
@@ -232,7 +232,7 @@
    Return the :pep:`3147` magic tag string matching this version of Python's
    magic number, as returned by :func:`get_magic`.
 
-   .. note::
+   .. deprecated:: 3.4
       You may use :attr:`sys.implementation.cache_tag` directly starting
       in Python 3.3.
 
@@ -355,6 +355,9 @@
       ``None`` is inserted into ``sys.path_importer_cache`` instead of an
       instance of :class:`NullImporter`.
 
+   .. deprecated:: 3.4
+      Insert ``None`` into ``sys.path_importer_cache`` instead.
+
 
 .. _examples-imp:
 
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -230,6 +230,9 @@
   :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
   to more easily customize module loading.
 
+* The :mod:`imp` module is pending deprecation. To keep compatibility with
+  Python 2/3 code bases, the module's removal is currently not scheduled.
+
 
 Deprecated functions and types of the C API
 -------------------------------------------
diff --git a/Lib/imp.py b/Lib/imp.py
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -27,6 +27,9 @@
 import types
 import warnings
 
+warnings.warn("the imp module is deprecated in favour of importlib; "
+              "see the module's documentation for alternative uses",
+              PendingDeprecationWarning)
 
 # DEPRECATED
 SEARCH_ERROR = 0
@@ -98,9 +101,7 @@
 
 
 def get_suffixes():
-    warnings.warn('imp.get_suffixes() is deprecated; use the constants '
-                  'defined on importlib.machinery instead',
-                  DeprecationWarning, 2)
+    """**DEPRECATED**"""
     extensions = [(s, 'rb', C_EXTENSION) for s in machinery.EXTENSION_SUFFIXES]
     source = [(s, 'U', PY_SOURCE) for s in machinery.SOURCE_SUFFIXES]
     bytecode = [(s, 'rb', PY_COMPILED) for s in machinery.BYTECODE_SUFFIXES]
@@ -110,7 +111,11 @@
 
 class NullImporter:
 
-    """Null import object."""
+    """**DEPRECATED**
+
+    Null import object.
+
+    """
 
     def __init__(self, path):
         if path == '':
@@ -152,10 +157,6 @@
 
 
 def load_source(name, pathname, file=None):
-    msg = ('imp.load_source() is deprecated; use '
-           'importlib.machinery.SourceFileLoader(name, pathname).load_module()'
-           ' instead')
-    warnings.warn(msg, DeprecationWarning, 2)
     _LoadSourceCompatibility(name, pathname, file).load_module(name)
     module = sys.modules[name]
     # To allow reloading to potentially work, use a non-hacked loader which
@@ -170,10 +171,7 @@
 
 
 def load_compiled(name, pathname, file=None):
-    msg = ('imp.load_compiled() is deprecated; use '
-           'importlib.machinery.SourcelessFileLoader(name, pathname).'
-           'load_module() instead ')
-    warnings.warn(msg, DeprecationWarning, 2)
+    """**DEPRECATED**"""
     _LoadCompiledCompatibility(name, pathname, file).load_module(name)
     module = sys.modules[name]
     # To allow reloading to potentially work, use a non-hacked loader which
@@ -183,10 +181,7 @@
 
 
 def load_package(name, path):
-    msg = ('imp.load_package() is deprecated; use either '
-           'importlib.machinery.SourceFileLoader() or '
-           'importlib.machinery.SourcelessFileLoader() instead')
-    warnings.warn(msg, DeprecationWarning, 2)
+    """**DEPRECATED**"""
     if os.path.isdir(path):
         extensions = (machinery.SOURCE_SUFFIXES[:] +
                       machinery.BYTECODE_SUFFIXES[:])
@@ -208,32 +203,30 @@
 
     """
     suffix, mode, type_ = details
-    with warnings.catch_warnings():
-        warnings.simplefilter('ignore')
-        if mode and (not mode.startswith(('r', 'U')) or '+' in mode):
-            raise ValueError('invalid file open mode {!r}'.format(mode))
-        elif file is None and type_ in {PY_SOURCE, PY_COMPILED}:
-            msg = 'file object required for import (type code {})'.format(type_)
-            raise ValueError(msg)
-        elif type_ == PY_SOURCE:
-            return load_source(name, filename, file)
-        elif type_ == PY_COMPILED:
-            return load_compiled(name, filename, file)
-        elif type_ == C_EXTENSION and load_dynamic is not None:
-            if file is None:
-                with open(filename, 'rb') as opened_file:
-                    return load_dynamic(name, filename, opened_file)
-            else:
-                return load_dynamic(name, filename, file)
-        elif type_ == PKG_DIRECTORY:
-            return load_package(name, filename)
-        elif type_ == C_BUILTIN:
-            return init_builtin(name)
-        elif type_ == PY_FROZEN:
-            return init_frozen(name)
+    if mode and (not mode.startswith(('r', 'U')) or '+' in mode):
+        raise ValueError('invalid file open mode {!r}'.format(mode))
+    elif file is None and type_ in {PY_SOURCE, PY_COMPILED}:
+        msg = 'file object required for import (type code {})'.format(type_)
+        raise ValueError(msg)
+    elif type_ == PY_SOURCE:
+        return load_source(name, filename, file)
+    elif type_ == PY_COMPILED:
+        return load_compiled(name, filename, file)
+    elif type_ == C_EXTENSION and load_dynamic is not None:
+        if file is None:
+            with open(filename, 'rb') as opened_file:
+                return load_dynamic(name, filename, opened_file)
         else:
-            msg =  "Don't know how to import {} (type code {})".format(name, type_)
-            raise ImportError(msg, name=name)
+            return load_dynamic(name, filename, file)
+    elif type_ == PKG_DIRECTORY:
+        return load_package(name, filename)
+    elif type_ == C_BUILTIN:
+        return init_builtin(name)
+    elif type_ == PY_FROZEN:
+        return init_frozen(name)
+    else:
+        msg =  "Don't know how to import {} (type code {})".format(name, type_)
+        raise ImportError(msg, name=name)
 
 
 def find_module(name, path=None):
@@ -269,16 +262,14 @@
             file_path = os.path.join(package_directory, package_file_name)
             if os.path.isfile(file_path):
                 return None, package_directory, ('', '', PKG_DIRECTORY)
-        with warnings.catch_warnings():
-            warnings.simplefilter('ignore')
-            for suffix, mode, type_ in get_suffixes():
-                file_name = name + suffix
-                file_path = os.path.join(entry, file_name)
-                if os.path.isfile(file_path):
-                    break
-            else:
-                continue
-            break  # Break out of outer loop when breaking out of inner loop.
+        for suffix, mode, type_ in get_suffixes():
+            file_name = name + suffix
+            file_path = os.path.join(entry, file_name)
+            if os.path.isfile(file_path):
+                break
+        else:
+            continue
+        break  # Break out of outer loop when breaking out of inner loop.
     else:
         raise ImportError(_ERR_MSG.format(name), name=name)
 
diff --git a/Lib/inspect.py b/Lib/inspect.py
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -31,7 +31,6 @@
 __author__ = ('Ka-Ping Yee <ping at lfw.org>',
               'Yury Selivanov <yselivanov at sprymix.com>')
 
-import imp
 import importlib.machinery
 import itertools
 import linecache
@@ -440,6 +439,9 @@
     """Get the module name, suffix, mode, and module type for a given file."""
     warnings.warn('inspect.getmoduleinfo() is deprecated', DeprecationWarning,
                   2)
+    with warnings.catch_warnings():
+        warnings.simplefilter('ignore', PendingDeprecationWarning)
+        import imp
     filename = os.path.basename(path)
     suffixes = [(-len(suffix), suffix, mode, mtype)
                     for suffix, mode, mtype in imp.get_suffixes()]
diff --git a/Lib/modulefinder.py b/Lib/modulefinder.py
--- a/Lib/modulefinder.py
+++ b/Lib/modulefinder.py
@@ -1,13 +1,16 @@
 """Find modules used by a script, using introspection."""
 
 import dis
-import imp
 import importlib.machinery
 import marshal
 import os
 import sys
 import types
 import struct
+import warnings
+with warnings.catch_warnings():
+    warnings.simplefilter('ignore', PendingDeprecationWarning)
+    import imp
 
 # XXX Clean up once str8's cstor matches bytes.
 LOAD_CONST = bytes([dis.opname.index('LOAD_CONST')])
diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py
--- a/Lib/pkgutil.py
+++ b/Lib/pkgutil.py
@@ -1,13 +1,13 @@
 """Utilities to support packages."""
 
 from functools import singledispatch as simplegeneric
-import imp
 import importlib
+import importlib.util
 import os
 import os.path
 import sys
 from types import ModuleType
-from warnings import warn
+import warnings
 
 __all__ = [
     'get_importer', 'iter_importers', 'get_loader', 'find_loader',
@@ -21,7 +21,7 @@
     import marshal
 
     magic = stream.read(4)
-    if magic != imp.get_magic():
+    if magic != importlib.util.MAGIC_NUMBER:
         return None
 
     stream.read(8) # Skip timestamp and size
@@ -160,6 +160,13 @@
 iter_importer_modules.register(
     importlib.machinery.FileFinder, _iter_file_finder_modules)
 
+
+def _import_imp():
+    global imp
+    with warnings.catch_warnings():
+        warnings.simplefilter('ignore', PendingDeprecationWarning)
+        imp = importlib.import_module('imp')
+
 class ImpImporter:
     """PEP 302 Importer that wraps Python's "classic" import algorithm
 
@@ -172,8 +179,10 @@
     """
 
     def __init__(self, path=None):
-        warn("This emulation is deprecated, use 'importlib' instead",
+        global imp
+        warnings.warn("This emulation is deprecated, use 'importlib' instead",
              DeprecationWarning)
+        _import_imp()
         self.path = path
 
     def find_module(self, fullname, path=None):
@@ -238,8 +247,9 @@
     code = source = None
 
     def __init__(self, fullname, file, filename, etc):
-        warn("This emulation is deprecated, use 'importlib' instead",
-             DeprecationWarning)
+        warnings.warn("This emulation is deprecated, use 'importlib' instead",
+                      DeprecationWarning)
+        _import_imp()
         self.file = file
         self.filename = filename
         self.fullname = fullname
diff --git a/Lib/runpy.py b/Lib/runpy.py
--- a/Lib/runpy.py
+++ b/Lib/runpy.py
@@ -13,7 +13,6 @@
 import os
 import sys
 import importlib.machinery # importlib first so we can test #15386 via -m
-import imp
 import types
 from pkgutil import read_code, get_loader, get_importer
 
@@ -224,7 +223,12 @@
         run_name = "<run_path>"
     pkg_name = run_name.rpartition(".")[0]
     importer = get_importer(path_name)
-    if isinstance(importer, (type(None), imp.NullImporter)):
+    # Trying to avoid importing imp so as to not consume the deprecation warning.
+    is_NullImporter = False
+    if type(importer).__module__ == 'imp':
+        if type(importer).__name__ == 'NullImporter':
+            is_NullImporter = True
+    if isinstance(importer, type(None)) or is_NullImporter:
         # Not a valid sys.path entry, so run the code directly
         # execfile() doesn't help as we want to allow compiled files
         code, mod_loader = _get_code_from_file(run_name, path_name)
diff --git a/Lib/test/test_fork1.py b/Lib/test/test_fork1.py
--- a/Lib/test/test_fork1.py
+++ b/Lib/test/test_fork1.py
@@ -1,7 +1,7 @@
 """This test checks for correct fork() behavior.
 """
 
-import imp
+import _imp as imp
 import os
 import signal
 import sys
diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py
--- a/Lib/test/test_imp.py
+++ b/Lib/test/test_imp.py
@@ -2,7 +2,6 @@
     import _thread
 except ImportError:
     _thread = None
-import imp
 import importlib
 import os
 import os.path
@@ -11,6 +10,9 @@
 from test import support
 import unittest
 import warnings
+with warnings.catch_warnings():
+    warnings.simplefilter('ignore', PendingDeprecationWarning)
+    import imp
 
 
 def requires_load_dynamic(meth):
diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py
--- a/Lib/test/test_import.py
+++ b/Lib/test/test_import.py
@@ -1,7 +1,7 @@
 # We import importlib *ASAP* in order to test #15386
 import importlib
+import importlib.util
 import builtins
-import imp
 from test.test_importlib.import_ import util as importlib_util
 import marshal
 import os
@@ -221,7 +221,7 @@
             with open(source, "w") as f:
                 f.write("a = 10\nb=20//0\n")
 
-            self.assertRaises(ZeroDivisionError, imp.reload, mod)
+            self.assertRaises(ZeroDivisionError, importlib.reload, mod)
             # But we still expect the module to be in sys.modules.
             mod = sys.modules.get(TESTFN)
             self.assertIsNot(mod, None, "expected module to be in sys.modules")
@@ -287,7 +287,7 @@
             import sys
             class C:
                def __del__(self):
-                  import imp
+                  import importlib
             sys.argv.insert(0, C())
             """))
         script_helper.assert_python_ok(testfn)
@@ -298,7 +298,7 @@
         sys.path.insert(0, os.curdir)
         try:
             source = TESTFN + ".py"
-            compiled = imp.cache_from_source(source)
+            compiled = importlib.util.cache_from_source(source)
             with open(source, 'w') as f:
                 pass
             try:
@@ -339,7 +339,7 @@
     def test_creation_mode(self):
         mask = 0o022
         with temp_umask(mask), _ready_to_import() as (name, path):
-            cached_path = imp.cache_from_source(path)
+            cached_path = importlib.util.cache_from_source(path)
             module = __import__(name)
             if not os.path.exists(cached_path):
                 self.fail("__import__ did not result in creation of "
@@ -357,7 +357,7 @@
         # permissions of .pyc should match those of .py, regardless of mask
         mode = 0o600
         with temp_umask(0o022), _ready_to_import() as (name, path):
-            cached_path = imp.cache_from_source(path)
+            cached_path = importlib.util.cache_from_source(path)
             os.chmod(path, mode)
             __import__(name)
             if not os.path.exists(cached_path):
@@ -372,7 +372,7 @@
     def test_cached_readonly(self):
         mode = 0o400
         with temp_umask(0o022), _ready_to_import() as (name, path):
-            cached_path = imp.cache_from_source(path)
+            cached_path = importlib.util.cache_from_source(path)
             os.chmod(path, mode)
             __import__(name)
             if not os.path.exists(cached_path):
@@ -412,7 +412,7 @@
                 bytecode_only = path + "c"
             else:
                 bytecode_only = path + "o"
-            os.rename(imp.cache_from_source(path), bytecode_only)
+            os.rename(importlib.util.cache_from_source(path), bytecode_only)
             m = __import__(name)
             self.assertEqual(m.x, 'rewritten')
 
@@ -434,7 +434,7 @@
 """
     dir_name = os.path.abspath(TESTFN)
     file_name = os.path.join(dir_name, module_name) + os.extsep + "py"
-    compiled_name = imp.cache_from_source(file_name)
+    compiled_name = importlib.util.cache_from_source(file_name)
 
     def setUp(self):
         self.sys_path = sys.path[:]
@@ -637,7 +637,7 @@
 class PycacheTests(unittest.TestCase):
     # Test the various PEP 3147 related behaviors.
 
-    tag = imp.get_tag()
+    tag = sys.implementation.cache_tag
 
     def _clean(self):
         forget(TESTFN)
@@ -685,7 +685,7 @@
         # With PEP 3147 cache layout, removing the source but leaving the pyc
         # file does not satisfy the import.
         __import__(TESTFN)
-        pyc_file = imp.cache_from_source(self.source)
+        pyc_file = importlib.util.cache_from_source(self.source)
         self.assertTrue(os.path.exists(pyc_file))
         os.remove(self.source)
         forget(TESTFN)
@@ -710,7 +710,7 @@
     def test___cached__(self):
         # Modules now also have an __cached__ that points to the pyc file.
         m = __import__(TESTFN)
-        pyc_file = imp.cache_from_source(TESTFN + '.py')
+        pyc_file = importlib.util.cache_from_source(TESTFN + '.py')
         self.assertEqual(m.__cached__, os.path.join(os.curdir, pyc_file))
 
     @skip_if_dont_write_bytecode
@@ -745,10 +745,10 @@
             pass
         importlib.invalidate_caches()
         m = __import__('pep3147.foo')
-        init_pyc = imp.cache_from_source(
+        init_pyc = importlib.util.cache_from_source(
             os.path.join('pep3147', '__init__.py'))
         self.assertEqual(m.__cached__, os.path.join(os.curdir, init_pyc))
-        foo_pyc = imp.cache_from_source(os.path.join('pep3147', 'foo.py'))
+        foo_pyc = importlib.util.cache_from_source(os.path.join('pep3147', 'foo.py'))
         self.assertEqual(sys.modules['pep3147.foo'].__cached__,
                          os.path.join(os.curdir, foo_pyc))
 
@@ -772,10 +772,10 @@
         unload('pep3147')
         importlib.invalidate_caches()
         m = __import__('pep3147.foo')
-        init_pyc = imp.cache_from_source(
+        init_pyc = importlib.util.cache_from_source(
             os.path.join('pep3147', '__init__.py'))
         self.assertEqual(m.__cached__, os.path.join(os.curdir, init_pyc))
-        foo_pyc = imp.cache_from_source(os.path.join('pep3147', 'foo.py'))
+        foo_pyc = importlib.util.cache_from_source(os.path.join('pep3147', 'foo.py'))
         self.assertEqual(sys.modules['pep3147.foo'].__cached__,
                          os.path.join(os.curdir, foo_pyc))
 
diff --git a/Lib/test/test_importlib/import_/test_api.py b/Lib/test/test_importlib/import_/test_api.py
--- a/Lib/test/test_importlib/import_/test_api.py
+++ b/Lib/test/test_importlib/import_/test_api.py
@@ -1,6 +1,5 @@
 from .. import util as importlib_test_util
 from . import util
-import imp
 import sys
 import types
 import unittest
diff --git a/Lib/test/test_importlib/source/test_file_loader.py b/Lib/test/test_importlib/source/test_file_loader.py
--- a/Lib/test/test_importlib/source/test_file_loader.py
+++ b/Lib/test/test_importlib/source/test_file_loader.py
@@ -7,7 +7,6 @@
 from . import util as source_util
 
 import errno
-import imp
 import marshal
 import os
 import py_compile
diff --git a/Lib/test/test_socketserver.py b/Lib/test/test_socketserver.py
--- a/Lib/test/test_socketserver.py
+++ b/Lib/test/test_socketserver.py
@@ -2,8 +2,8 @@
 Test suite for socketserver.
 """
 
+import _imp as imp
 import contextlib
-import imp
 import os
 import select
 import signal
diff --git a/Lib/test/test_threaded_import.py b/Lib/test/test_threaded_import.py
--- a/Lib/test/test_threaded_import.py
+++ b/Lib/test/test_threaded_import.py
@@ -5,8 +5,8 @@
 # complains several times about module random having no attribute
 # randrange, and then Python hangs.
 
+import _imp as imp
 import os
-import imp
 import importlib
 import sys
 import time
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #17177: The imp module is pending deprecation.
+
 - subprocess: Prevent a possible double close of parent pipe fds when the
   subprocess exec runs into an error.  Prevent a regular multi-close of the
   /dev/null fd when any of stdin, stdout and stderr was set to DEVNULL.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 19:14:17 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 19:14:17 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bYMbs0sl3z7Ljx@mail.python.org>

http://hg.python.org/cpython/rev/7ca3e27e3b1a
changeset:   84168:7ca3e27e3b1a
parent:      84167:1b8f08c4efd5
parent:      84166:81648329b37a
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 13:14:06 2013 -0400
summary:
  merge

files:
  Doc/library/codecs.rst    |   6 +++-
  Doc/library/functions.rst |  40 ++++++++++++++++++++------
  Lib/codecs.py             |   1 +
  Modules/_io/_iomodule.c   |   4 +-
  Modules/_io/textio.c      |   5 ++-
  5 files changed, 41 insertions(+), 15 deletions(-)


diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -78,7 +78,11 @@
      reference (for encoding only)
    * ``'backslashreplace'``: replace with backslashed escape sequences (for
      encoding only)
-   * ``'surrogateescape'``: replace with surrogate U+DCxx, see :pep:`383`
+   * ``'surrogateescape'``: on decoding, replace with code points in the Unicode
+     Private Use Area ranging from U+DC80 to U+DCFF.  These private code
+     points will then be turned back into the same bytes when the
+     ``surrogateescape`` error handler is used when encoding the data.
+     (See :pep:`383` for more.)
 
    as well as any other error handling name defined via :func:`register_error`.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -905,16 +905,36 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.  Pass
-   ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding
-   error (the default of ``None`` has the same effect), or pass ``'ignore'`` to
-   ignore errors.  (Note that ignoring encoding errors can lead to data loss.)
-   ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
-   where there is malformed data.  When writing, ``'xmlcharrefreplace'``
-   (replace with the appropriate XML character reference) or
-   ``'backslashreplace'`` (replace with backslashed escape sequences) can be
-   used.  Any other error handling name that has been registered with
-   :func:`codecs.register_error` is also valid.
+   errors are to be handled--this cannot be used in binary mode.
+   A variety of standard error handlers are available, though any
+   error handling name that has been registered with
+   :func:`codecs.register_error` is also valid.  The standard names
+   are:
+
+   * ``'strict'`` to raise a :exc:`ValueError` exception if there is
+     an encoding error.  The default value of ``None`` has the same
+     effect.
+
+   * ``'ignore'`` ignores errors.  Note that ignoring encoding errors
+     can lead to data loss.
+
+   * ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
+     where there is malformed data.
+
+   * ``'surrogateescape'`` will represent any incorrect bytes as code
+     points in the Unicode Private Use Area ranging from U+DC80 to
+     U+DCFF.  These private code points will then be turned back into
+     the same bytes when the ``surrogateescape`` error handler is used
+     when writing data.  This is useful for processing files in an
+     unknown encoding.
+
+   * ``'xmlcharrefreplace'`` is only supported when writing to a file.
+     Characters not supported by the encoding are replaced with the
+     appropriate XML character reference ``&#nnn;``.
+
+   * ``'backslashreplace'`` (also only supported when writing)
+     replaces unsupported characters with Python's backslashed escape
+     sequences.
 
    .. index::
       single: universal newlines; open() built-in function
diff --git a/Lib/codecs.py b/Lib/codecs.py
--- a/Lib/codecs.py
+++ b/Lib/codecs.py
@@ -105,6 +105,7 @@
                     Python will use the official U+FFFD REPLACEMENT
                     CHARACTER for the builtin Unicode codecs on
                     decoding and '?' on encoding.
+         'surrogateescape' - replace with private codepoints U+DCnn.
          'xmlcharrefreplace' - Replace with the appropriate XML
                                character reference (only for encoding).
          'backslashreplace'  - Replace with backslashed escape sequences
diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
--- a/Modules/_io/_iomodule.c
+++ b/Modules/_io/_iomodule.c
@@ -168,8 +168,8 @@
 "'strict' to raise a ValueError exception if there is an encoding error\n"
 "(the default of None has the same effect), or pass 'ignore' to ignore\n"
 "errors. (Note that ignoring encoding errors can lead to data loss.)\n"
-"See the documentation for codecs.register for a list of the permitted\n"
-"encoding error strings.\n"
+"See the documentation for codecs.register or run 'help(codecs.Codec)'\n"
+"for a list of the permitted encoding error strings.\n"
 "\n"
 "newline controls how universal newlines works (it only applies to text\n"
 "mode). It can be None, '', '\\n', '\\r', and '\\r\\n'.  It works as\n"
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -642,8 +642,9 @@
     "encoding gives the name of the encoding that the stream will be\n"
     "decoded or encoded with. It defaults to locale.getpreferredencoding(False).\n"
     "\n"
-    "errors determines the strictness of encoding and decoding (see the\n"
-    "codecs.register) and defaults to \"strict\".\n"
+    "errors determines the strictness of encoding and decoding (see\n"
+    "help(codecs.Codec) or the documentation for codecs.register) and\n"
+    "defaults to \"strict\".\n"
     "\n"
     "newline controls how line endings are handled. It can be None, '',\n"
     "'\\n', '\\r', and '\\r\\n'.  It works as follows:\n"

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 20:57:07 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 20:57:07 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issues_=2318058=2C_18057?=
 =?utf-8?q?=3A_Make_importlib=2E=5Fbootstrap=2ENamespaceLoader?=
Message-ID: <3bYPtW3C1Fz7Ll7@mail.python.org>

http://hg.python.org/cpython/rev/ebec625b13f9
changeset:   84169:ebec625b13f9
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 14:56:58 2013 -0400
summary:
  Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
conform the the InspectLoader ABC. Perk of this is that runpy/-m can
now work with namespace packages.

files:
  Lib/importlib/_bootstrap.py     |    15 +-
  Lib/importlib/abc.py            |     2 +-
  Lib/test/test_namespace_pkgs.py |    29 +-
  Misc/NEWS                       |     4 +
  Python/importlib.h              |  2185 +++++++++---------
  5 files changed, 1156 insertions(+), 1079 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -1238,12 +1238,25 @@
     def module_repr(cls, module):
         return "<module '{}' (namespace)>".format(module.__name__)
 
+    def is_package(self, fullname):
+        return True
+
+    def get_source(self, fullname):
+        return ''
+
+    def get_code(self, fullname):
+        return compile('', '<string>', 'exec', dont_inherit=True)
+
+    def init_module_attrs(self, module):
+        module.__loader__ = self
+        module.__package__ = module.__name__
+
     def load_module(self, fullname):
         """Load a namespace module."""
         _verbose_message('namespace module loaded with path {!r}', self._path)
         with module_to_load(fullname) as module:
+            self.init_module_attrs(module)
             module.__path__ = self._path
-            module.__package__ = fullname
             return module
 
 
diff --git a/Lib/importlib/abc.py b/Lib/importlib/abc.py
--- a/Lib/importlib/abc.py
+++ b/Lib/importlib/abc.py
@@ -188,7 +188,7 @@
     load_module = _bootstrap._LoaderBasics.load_module
 
 _register(InspectLoader, machinery.BuiltinImporter, machinery.FrozenImporter,
-            machinery.ExtensionFileLoader)
+            machinery.ExtensionFileLoader, _bootstrap.NamespaceLoader)
 
 
 class ExecutionLoader(InspectLoader):
diff --git a/Lib/test/test_namespace_pkgs.py b/Lib/test/test_namespace_pkgs.py
--- a/Lib/test/test_namespace_pkgs.py
+++ b/Lib/test/test_namespace_pkgs.py
@@ -1,7 +1,11 @@
+import contextlib
+from importlib._bootstrap import NamespaceLoader
+import importlib.abc
+import importlib.machinery
+import os
 import sys
-import contextlib
+import types
 import unittest
-import os
 
 from test.test_importlib import util
 from test.support import run_unittest
@@ -286,9 +290,24 @@
         self.assertEqual(a_test.attr, 'in module')
 
 
-def test_main():
-    run_unittest(*NamespacePackageTest.__subclasses__())
+class ABCTests(unittest.TestCase):
+
+    def setUp(self):
+        self.loader = NamespaceLoader('foo', ['pkg'],
+                                      importlib.machinery.PathFinder)
+
+    def test_is_package(self):
+        self.assertTrue(self.loader.is_package('foo'))
+
+    def test_get_code(self):
+        self.assertTrue(isinstance(self.loader.get_code('foo'), types.CodeType))
+
+    def test_get_source(self):
+        self.assertEqual(self.loader.get_source('foo'), '')
+
+    def test_abc_isinstance(self):
+        self.assertTrue(isinstance(self.loader, importlib.abc.InspectLoader))
 
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,10 @@
 Library
 -------
 
+- Issue #18058, 18057: Make the namespace package loader meet the
+  importlib.abc.InspectLoader ABC, allowing for namespace packages to work with
+  runpy.
+
 - Issue #17177: The imp module is pending deprecation.
 
 - subprocess: Prevent a possible double close of parent pipe fds when the
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 21:20:55 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 21:20:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Check_that_warnings=2Eshow?=
 =?utf-8?q?warning=28=29_is_not_changed=2E?=
Message-ID: <3bYQPz0TmRzSpD@mail.python.org>

http://hg.python.org/cpython/rev/59a11c81dd3c
changeset:   84170:59a11c81dd3c
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 15:20:48 2013 -0400
summary:
  Check that warnings.showwarning() is not changed.

files:
  Lib/test/regrtest.py |  7 ++++++-
  1 files changed, 6 insertions(+), 1 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1062,7 +1062,7 @@
                  'sys.warnoptions', 'threading._dangling',
                  'multiprocessing.process._dangling',
                  'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES',
-                 'support.TESTFN', 'locale',
+                 'support.TESTFN', 'locale', 'warnings.showwarning',
                 )
 
     def get_sys_argv(self):
@@ -1244,6 +1244,11 @@
         for lc, setting in saved:
             locale.setlocale(lc, setting)
 
+    def get_warnings_showwarning(self):
+        return warnings.showwarning
+    def restore_warnings_showwarning(self, fxn):
+        warnings.showwarning = fxn
+
     def resource_info(self):
         for name in self.resources:
             method_suffix = name.replace('.', '_')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 16 23:23:16 2013
From: python-checkins at python.org (brett.cannon)
Date: Sun, 16 Jun 2013 23:23:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318115=3A_Abstract?=
 =?utf-8?q?_out_managing_the_cleanup_of_modules_to_use_in?=
Message-ID: <3bYT785bTXz7Lkd@mail.python.org>

http://hg.python.org/cpython/rev/205aa49e5cd5
changeset:   84171:205aa49e5cd5
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 17:23:06 2013 -0400
summary:
  Issue #18115: Abstract out managing the cleanup of modules to use in
loaders where C code provides the loaded module.

files:
  Lib/importlib/_bootstrap.py |    47 +-
  Python/importlib.h          |  4919 +++++++++++-----------
  2 files changed, 2498 insertions(+), 2468 deletions(-)


diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -475,8 +475,24 @@
         print(message.format(*args), file=sys.stderr)
 
 
+class _ManageReload:
+
+    def __init__(self, name):
+        self._name = name
+
+    def __enter__(self):
+        self._is_reload = self._name in sys.modules
+
+    def __exit__(self, *args):
+        if any(arg is not None for arg in args) and not self._is_reload:
+            try:
+                del sys.modules[self._name]
+            except KeyError:
+                pass
+
+
 # Written as a class only because contextlib is not available.
-class _ModuleManager:
+class _ModuleManager(_ManageReload):
 
     """Context manager which returns the module to be loaded.
 
@@ -490,12 +506,12 @@
         The reset_name argument specifies whether to unconditionally reset
         the __name__ attribute if the module is found to be a reload.
         """
-        self._name = name
+        super().__init__(name)
         self._reset_name = reset_name
 
     def __enter__(self):
+        super().__enter__()
         self._module = sys.modules.get(self._name)
-        self._is_reload = self._module is not None
         if not self._is_reload:
             # This must be done before open() is called as the 'io' module
             # implicitly imports 'locale' and would otherwise trigger an
@@ -510,14 +526,12 @@
                 self._module.__name__ = self._name
             except AttributeError:
                 pass
-
         return self._module
 
     def __exit__(self, *args):
         self._module.__initializing__ = False
         del self._module
-        if any(arg is not None for arg in args) and not self._is_reload:
-            del sys.modules[self._name]
+        super().__exit__(*args)
 
 
 def module_to_load(name, *, reset_name=True):
@@ -741,13 +755,8 @@
     @_requires_builtin
     def load_module(cls, fullname):
         """Load a built-in module."""
-        is_reload = fullname in sys.modules
-        try:
+        with _ManageReload(fullname):
             return _call_with_frames_removed(_imp.init_builtin, fullname)
-        except:
-            if not is_reload and fullname in sys.modules:
-                del sys.modules[fullname]
-            raise
 
     @classmethod
     @_requires_builtin
@@ -792,16 +801,11 @@
     @_requires_frozen
     def load_module(cls, fullname):
         """Load a frozen module."""
-        is_reload = fullname in sys.modules
-        try:
+        with _ManageReload(fullname):
             m = _call_with_frames_removed(_imp.init_frozen, fullname)
             # Let our own module_repr() method produce a suitable repr.
             del m.__file__
             return m
-        except:
-            if not is_reload and fullname in sys.modules:
-                del sys.modules[fullname]
-            raise
 
     @classmethod
     @_requires_frozen
@@ -1147,18 +1151,13 @@
     @set_loader
     def load_module(self, fullname):
         """Load an extension module."""
-        is_reload = fullname in sys.modules
-        try:
+        with _ManageReload(fullname):
             module = _call_with_frames_removed(_imp.load_dynamic,
                                                fullname, self.path)
             _verbose_message('extension module loaded from {!r}', self.path)
             if self.is_package(fullname) and not hasattr(module, '__path__'):
                 module.__path__ = [_path_split(self.path)[0]]
             return module
-        except:
-            if not is_reload and fullname in sys.modules:
-                del sys.modules[fullname]
-            raise
 
     def is_package(self, fullname):
         """Return True if the extension module is a package."""
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 00:06:02 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 00:06:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_importlib=2Eabc=2ESourceLo?=
 =?utf-8?q?ader=2Eget=5Fsource=28=29_was_re-raising_SyntaxError_and?=
Message-ID: <3bYV4V3p4Jz7LjT@mail.python.org>

http://hg.python.org/cpython/rev/e353f64dfd95
changeset:   84172:e353f64dfd95
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 18:05:54 2013 -0400
summary:
  importlib.abc.SourceLoader.get_source() was re-raising SyntaxError and
UnicodeDecodeError as ImportError. That was over-reaching the point of
raising ImportError in get_source() (which is to signal the source
code was not found when it should have). Conflating the two exceptions
with ImportError could lead to masking errors with the source which
should be known outside of whether there was an error simply getting
the source to begin with.

files:
  Doc/whatsnew/3.4.rst        |     9 +
  Lib/importlib/_bootstrap.py |    14 +-
  Misc/NEWS                   |     3 +
  Python/importlib.h          |  3428 +++++++++++-----------
  4 files changed, 1721 insertions(+), 1733 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -282,3 +282,12 @@
   it would write to is a symlink or a non-regular file. This is to act as a
   warning that import will overwrite those files with a regular file regardless
   of what type of file path they were originally.
+
+* :meth:`importlib.abc.SourceLoader.get_source` no longer raises
+  :exc:`ImportError` when the source code being loaded triggers a
+  :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is
+  meant to be raised only when source code cannot be found but it should, it was
+  felt to be over-reaching/overloading of that meaning when the source code is
+  found but improperly structured. If you were catching ImportError before and
+  wish to continue to ignore syntax or decoding issues, catch all three
+  exceptions now.
\ No newline at end of file
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -959,25 +959,17 @@
 
     def get_source(self, fullname):
         """Concrete implementation of InspectLoader.get_source."""
-        import tokenize
         path = self.get_filename(fullname)
         try:
             source_bytes = self.get_data(path)
         except OSError as exc:
             raise ImportError("source not available through get_data()",
                               name=fullname) from exc
+        import tokenize
         readsource = _io.BytesIO(source_bytes).readline
-        try:
-            encoding = tokenize.detect_encoding(readsource)
-        except SyntaxError as exc:
-            raise ImportError("Failed to detect encoding",
-                              name=fullname) from exc
+        encoding = tokenize.detect_encoding(readsource)
         newline_decoder = _io.IncrementalNewlineDecoder(None, True)
-        try:
-            return newline_decoder.decode(source_bytes.decode(encoding[0]))
-        except UnicodeDecodeError as exc:
-            raise ImportError("Failed to decode source file",
-                              name=fullname) from exc
+        return newline_decoder.decode(source_bytes.decode(encoding[0]))
 
     def source_to_code(self, data, path, *, _optimize=-1):
         """Return the code object compiled from source.
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- importlib.abc.SourceLoader.get_source() no longer changes SyntaxError or
+  UnicodeDecodeError into ImportError.
+
 - Issue #18058, 18057: Make the namespace package loader meet the
   importlib.abc.InspectLoader ABC, allowing for namespace packages to work with
   runpy.
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 00:38:02 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 00:38:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318076=3A_Introduc?=
 =?utf-8?q?e_imoportlib=2Eutil=2Edecode=5Fsource=28=29=2E?=
Message-ID: <3bYVnQ3fzTzRdJ@mail.python.org>

http://hg.python.org/cpython/rev/bdd60bedf933
changeset:   84173:bdd60bedf933
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 18:37:53 2013 -0400
summary:
  Issue #18076: Introduce imoportlib.util.decode_source().

The helper function makes it easier to implement
imoprtlib.abc.InspectLoader.get_source() by making that function
require just the raw bytes for source code and handling all other
details.

files:
  Doc/library/importlib.rst            |     8 +
  Lib/importlib/_bootstrap.py          |    18 +-
  Lib/importlib/util.py                |     1 +
  Lib/test/test_importlib/test_util.py |    21 +
  Misc/NEWS                            |     2 +
  Python/importlib.h                   |  7149 +++++++------
  6 files changed, 3628 insertions(+), 3571 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -916,6 +916,14 @@
 
    .. versionadded:: 3.4
 
+.. function:: decode_source(source_bytes)
+
+   Decode the given bytes representing source code and return it as a string
+   with universal newlines (as required by
+   :meth:`importlib.abc.InspectLoader.get_source`).
+
+   .. versionadded:: 3.4
+
 .. function:: resolve_name(name, package)
 
    Resolve a relative module name to an absolute one.
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -723,6 +723,18 @@
     return data
 
 
+def decode_source(source_bytes):
+    """Decode bytes representing source code and return the string.
+
+    Universal newline support is used in the decoding.
+    """
+    import tokenize  # To avoid bootstrap issues.
+    source_bytes_readline = _io.BytesIO(source_bytes).readline
+    encoding = tokenize.detect_encoding(source_bytes_readline)
+    newline_decoder = _io.IncrementalNewlineDecoder(None, True)
+    return newline_decoder.decode(source_bytes.decode(encoding[0]))
+
+
 # Loaders #####################################################################
 
 class BuiltinImporter:
@@ -965,11 +977,7 @@
         except OSError as exc:
             raise ImportError("source not available through get_data()",
                               name=fullname) from exc
-        import tokenize
-        readsource = _io.BytesIO(source_bytes).readline
-        encoding = tokenize.detect_encoding(readsource)
-        newline_decoder = _io.IncrementalNewlineDecoder(None, True)
-        return newline_decoder.decode(source_bytes.decode(encoding[0]))
+        return decode_source(source_bytes)
 
     def source_to_code(self, data, path, *, _optimize=-1):
         """Return the code object compiled from source.
diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py
--- a/Lib/importlib/util.py
+++ b/Lib/importlib/util.py
@@ -2,6 +2,7 @@
 
 from ._bootstrap import MAGIC_NUMBER
 from ._bootstrap import cache_from_source
+from ._bootstrap import decode_source
 from ._bootstrap import module_to_load
 from ._bootstrap import set_loader
 from ._bootstrap import set_package
diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py
--- a/Lib/test/test_importlib/test_util.py
+++ b/Lib/test/test_importlib/test_util.py
@@ -9,6 +9,27 @@
 import warnings
 
 
+class DecodeSourceBytesTests(unittest.TestCase):
+
+    source = "string ='?'"
+
+    def test_ut8_default(self):
+        source_bytes = self.source.encode('utf-8')
+        self.assertEqual(util.decode_source(source_bytes), self.source)
+
+    def test_specified_encoding(self):
+        source = '# coding=latin-1\n' + self.source
+        source_bytes = source.encode('latin-1')
+        assert source_bytes != source.encode('utf-8')
+        self.assertEqual(util.decode_source(source_bytes), source)
+
+    def test_universal_newlines(self):
+        source = '\r\n'.join([self.source, self.source])
+        source_bytes = source.encode('utf-8')
+        self.assertEqual(util.decode_source(source_bytes),
+                         '\n'.join([self.source, self.source]))
+
+
 class ModuleToLoadTests(unittest.TestCase):
 
     module_name = 'ModuleManagerTest_module'
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #18076: Introduce importlib.util.decode_source().
+
 - importlib.abc.SourceLoader.get_source() no longer changes SyntaxError or
   UnicodeDecodeError into ImportError.
 
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 01:09:55 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 01:09:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fix_a_misnamin?=
 =?utf-8?q?g_of_a_method_and_an_argument?=
Message-ID: <3bYWVC3jYcz7LjT@mail.python.org>

http://hg.python.org/cpython/rev/e13371d5b4e4
changeset:   84174:e13371d5b4e4
branch:      3.3
parent:      84144:61a1aa69e83e
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 19:06:55 2013 -0400
summary:
  Fix a misnaming of a method and an argument

files:
  Doc/library/importlib.rst   |     2 +-
  Lib/importlib/_bootstrap.py |     4 +-
  Python/importlib.h          |  2156 +++++++++++-----------
  3 files changed, 1081 insertions(+), 1081 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -714,7 +714,7 @@
 
       The path the finder will search in.
 
-   .. method:: find_module(fullname)
+   .. method:: find_loader(fullname)
 
       Attempt to find the loader to handle *fullname* within :attr:`path`.
 
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -1329,12 +1329,12 @@
 
     """
 
-    def __init__(self, path, *details):
+    def __init__(self, path, *loader_details):
         """Initialize with the path to search on and a variable number of
         2-tuples containing the loader and the file suffixes the loader
         recognizes."""
         loaders = []
-        for loader, suffixes in details:
+        for loader, suffixes in loader_details:
             loaders.extend((suffix, loader) for suffix in suffixes)
         self._loaders = loaders
         # Base (directory) path
diff --git a/Python/importlib.h b/Python/importlib.h
--- a/Python/importlib.h
+++ b/Python/importlib.h
[stripped]

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 01:09:56 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 01:09:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuMyk6?=
	=?utf-8?q?_merge?=
Message-ID: <3bYWVD75lNz7Ljg@mail.python.org>

http://hg.python.org/cpython/rev/e7ab65cf1e2a
changeset:   84175:e7ab65cf1e2a
branch:      3.3
parent:      84174:e13371d5b4e4
parent:      84165:55f611f55952
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 19:07:16 2013 -0400
summary:
  merge

files:
  Doc/library/codecs.rst    |   6 +++-
  Doc/library/functions.rst |  40 ++++++++++++++++++------
  Lib/codecs.py             |   1 +
  Lib/subprocess.py         |  43 +++++++++++++++-----------
  Lib/test/test_curses.py   |  13 ++++++++
  Misc/NEWS                 |   7 ++++
  Modules/_curses_panel.c   |   4 ++
  Modules/_io/_iomodule.c   |   4 +-
  Modules/_io/textio.c      |   5 +-
  9 files changed, 90 insertions(+), 33 deletions(-)


diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -78,7 +78,11 @@
      reference (for encoding only)
    * ``'backslashreplace'``: replace with backslashed escape sequences (for
      encoding only)
-   * ``'surrogateescape'``: replace with surrogate U+DCxx, see :pep:`383`
+   * ``'surrogateescape'``: on decoding, replace with code points in the Unicode
+     Private Use Area ranging from U+DC80 to U+DCFF.  These private code
+     points will then be turned back into the same bytes when the
+     ``surrogateescape`` error handler is used when encoding the data.
+     (See :pep:`383` for more.)
 
    as well as any other error handling name defined via :func:`register_error`.
 
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -895,16 +895,36 @@
    the list of supported encodings.
 
    *errors* is an optional string that specifies how encoding and decoding
-   errors are to be handled--this cannot be used in binary mode.  Pass
-   ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding
-   error (the default of ``None`` has the same effect), or pass ``'ignore'`` to
-   ignore errors.  (Note that ignoring encoding errors can lead to data loss.)
-   ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
-   where there is malformed data.  When writing, ``'xmlcharrefreplace'``
-   (replace with the appropriate XML character reference) or
-   ``'backslashreplace'`` (replace with backslashed escape sequences) can be
-   used.  Any other error handling name that has been registered with
-   :func:`codecs.register_error` is also valid.
+   errors are to be handled--this cannot be used in binary mode.
+   A variety of standard error handlers are available, though any
+   error handling name that has been registered with
+   :func:`codecs.register_error` is also valid.  The standard names
+   are:
+
+   * ``'strict'`` to raise a :exc:`ValueError` exception if there is
+     an encoding error.  The default value of ``None`` has the same
+     effect.
+
+   * ``'ignore'`` ignores errors.  Note that ignoring encoding errors
+     can lead to data loss.
+
+   * ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
+     where there is malformed data.
+
+   * ``'surrogateescape'`` will represent any incorrect bytes as code
+     points in the Unicode Private Use Area ranging from U+DC80 to
+     U+DCFF.  These private code points will then be turned back into
+     the same bytes when the ``surrogateescape`` error handler is used
+     when writing data.  This is useful for processing files in an
+     unknown encoding.
+
+   * ``'xmlcharrefreplace'`` is only supported when writing to a file.
+     Characters not supported by the encoding are replaced with the
+     appropriate XML character reference ``&#nnn;``.
+
+   * ``'backslashreplace'`` (also only supported when writing)
+     replaces unsupported characters with Python's backslashed escape
+     sequences.
 
    .. index::
       single: universal newlines; open() built-in function
diff --git a/Lib/codecs.py b/Lib/codecs.py
--- a/Lib/codecs.py
+++ b/Lib/codecs.py
@@ -105,6 +105,7 @@
                     Python will use the official U+FFFD REPLACEMENT
                     CHARACTER for the builtin Unicode codecs on
                     decoding and '?' on encoding.
+         'surrogateescape' - replace with private codepoints U+DCnn.
          'xmlcharrefreplace' - Replace with the appropriate XML
                                character reference (only for encoding).
          'backslashreplace'  - Replace with backslashed escape sequences
diff --git a/Lib/subprocess.py b/Lib/subprocess.py
--- a/Lib/subprocess.py
+++ b/Lib/subprocess.py
@@ -810,6 +810,7 @@
             if universal_newlines:
                 self.stderr = io.TextIOWrapper(self.stderr)
 
+        self._closed_child_pipe_fds = False
         try:
             self._execute_child(args, executable, preexec_fn, close_fds,
                                 pass_fds, cwd, env,
@@ -826,19 +827,21 @@
                 except EnvironmentError:
                     pass  # Ignore EBADF or other errors.
 
-            # Make sure the child pipes are closed as well.
-            to_close = []
-            if stdin == PIPE:
-                to_close.append(p2cread)
-            if stdout == PIPE:
-                to_close.append(c2pwrite)
-            if stderr == PIPE:
-                to_close.append(errwrite)
-            for fd in to_close:
-                try:
-                    os.close(fd)
-                except EnvironmentError:
-                    pass
+            if not self._closed_child_pipe_fds:
+                to_close = []
+                if stdin == PIPE:
+                    to_close.append(p2cread)
+                if stdout == PIPE:
+                    to_close.append(c2pwrite)
+                if stderr == PIPE:
+                    to_close.append(errwrite)
+                if hasattr(self, '_devnull'):
+                    to_close.append(self._devnull)
+                for fd in to_close:
+                    try:
+                        os.close(fd)
+                    except EnvironmentError:
+                        pass
 
             raise
 
@@ -1383,14 +1386,18 @@
                     # be sure the FD is closed no matter what
                     os.close(errpipe_write)
 
-                if p2cread != -1 and p2cwrite != -1:
+                # self._devnull is not always defined.
+                devnull_fd = getattr(self, '_devnull', None)
+                if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                     os.close(p2cread)
-                if c2pwrite != -1 and c2pread != -1:
+                if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                     os.close(c2pwrite)
-                if errwrite != -1 and errread != -1:
+                if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                     os.close(errwrite)
-                if hasattr(self, '_devnull'):
-                    os.close(self._devnull)
+                if devnull_fd is not None:
+                    os.close(devnull_fd)
+                # Prevent a double close of these fds from __init__ on error.
+                self._closed_child_pipe_fds = True
 
                 # Wait for exec to fail or succeed; possibly raising an
                 # exception (limited in size)
diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -252,6 +252,18 @@
     except curses.panel.error:
         pass
 
+def test_userptr_memory_leak(stdscr):
+    w = curses.newwin(10, 10)
+    p = curses.panel.new_panel(w)
+    obj = object()
+    nrefs = sys.getrefcount(obj)
+    for i in range(100):
+        p.set_userptr(obj)
+
+    p.set_userptr(None)
+    if sys.getrefcount(obj) != nrefs:
+        raise RuntimeError("set_userptr leaked references")
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -317,6 +329,7 @@
         module_funcs(stdscr)
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
+        test_userptr_memory_leak(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
         test_unget_wch(stdscr)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -32,6 +32,10 @@
 Library
 -------
 
+- subprocess: Prevent a possible double close of parent pipe fds when the
+  subprocess exec runs into an error.  Prevent a regular multi-close of the
+  /dev/null fd when any of stdin, stdout and stderr was set to DEVNULL.
+
 - Issue #16102: Make uuid._netbios_getnode() work again on Python 3.
 
 - Issue #18109: os.uname() now decodes fields from the locale encoding, and
@@ -78,6 +82,9 @@
   the default for linking if LDSHARED is not also overriden.  This restores
   Distutils behavior introduced in 3.2.3 and inadvertently dropped in 3.3.0.
 
+- Issue #18113: Fixed a refcount leak in the curses.panel module's
+  set_userptr() method.  Reported by Atsuo Ishimoto.
+
 IDLE
 ----
 
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -322,6 +322,10 @@
 static PyObject *
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
+    PyObject *oldobj;
+    PyCursesInitialised;
+    oldobj = (PyObject *) panel_userptr(self->pan);
+    Py_XDECREF(oldobj);
     Py_INCREF(obj);
     return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
                             "set_panel_userptr");
diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
--- a/Modules/_io/_iomodule.c
+++ b/Modules/_io/_iomodule.c
@@ -168,8 +168,8 @@
 "'strict' to raise a ValueError exception if there is an encoding error\n"
 "(the default of None has the same effect), or pass 'ignore' to ignore\n"
 "errors. (Note that ignoring encoding errors can lead to data loss.)\n"
-"See the documentation for codecs.register for a list of the permitted\n"
-"encoding error strings.\n"
+"See the documentation for codecs.register or run 'help(codecs.Codec)'\n"
+"for a list of the permitted encoding error strings.\n"
 "\n"
 "newline controls how universal newlines works (it only applies to text\n"
 "mode). It can be None, '', '\\n', '\\r', and '\\r\\n'.  It works as\n"
diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -642,8 +642,9 @@
     "encoding gives the name of the encoding that the stream will be\n"
     "decoded or encoded with. It defaults to locale.getpreferredencoding(False).\n"
     "\n"
-    "errors determines the strictness of encoding and decoding (see the\n"
-    "codecs.register) and defaults to \"strict\".\n"
+    "errors determines the strictness of encoding and decoding (see\n"
+    "help(codecs.Codec) or the documentation for codecs.register) and\n"
+    "defaults to \"strict\".\n"
     "\n"
     "newline controls how line endings are handled. It can be None, '',\n"
     "'\\n', '\\r', and '\\r\\n'.  It works as follows:\n"

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 01:09:58 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 01:09:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_doc/argument_name_fix?=
Message-ID: <3bYWVG238Lz7LkT@mail.python.org>

http://hg.python.org/cpython/rev/03ad17618ac4
changeset:   84176:03ad17618ac4
parent:      84173:bdd60bedf933
parent:      84175:e7ab65cf1e2a
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 19:09:46 2013 -0400
summary:
  Merge doc/argument name fix

files:
  Doc/library/importlib.rst   |  2 +-
  Lib/importlib/_bootstrap.py |  4 ++--
  2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -751,7 +751,7 @@
 
       The path the finder will search in.
 
-   .. method:: find_module(fullname)
+   .. method:: find_loader(fullname)
 
       Attempt to find the loader to handle *fullname* within :attr:`path`.
 
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -1361,12 +1361,12 @@
 
     """
 
-    def __init__(self, path, *details):
+    def __init__(self, path, *loader_details):
         """Initialize with the path to search on and a variable number of
         2-tuples containing the loader and the file suffixes the loader
         recognizes."""
         loaders = []
-        for loader, suffixes in details:
+        for loader, suffixes in loader_details:
             loaders.extend((suffix, loader) for suffix in suffixes)
         self._loaders = loaders
         # Base (directory) path

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 01:17:20 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 01:17:20 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_documentation_suggesting_s?=
 =?utf-8?q?omething_which_doesn=27t_exist?=
Message-ID: <3bYWfm0f3Lz7Lkc@mail.python.org>

http://hg.python.org/cpython/rev/69b0b713865e
changeset:   84177:69b0b713865e
user:        Brett Cannon <brett at python.org>
date:        Sun Jun 16 19:17:12 2013 -0400
summary:
  documentation suggesting something which doesn't exist

files:
  Doc/library/importlib.rst |  3 +--
  1 files changed, 1 insertions(+), 2 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -524,8 +524,7 @@
     * :meth:`ResourceLoader.get_data`
     * :meth:`ExecutionLoader.get_filename`
           Should only return the path to the source file; sourceless
-          loading is not supported (see :class:`SourcelessLoader` if that
-          functionality is required)
+          loading is not supported.
 
     The abstract methods defined by this class are to add optional bytecode
     file support. Not implementing these optional methods (or causing them to

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Jun 17 05:56:45 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 17 Jun 2013 05:56:45 +0200
Subject: [Python-checkins] Daily reference leaks (69b0b713865e): sum=0
Message-ID: <E1UoQYf-0000NK-1R@ap.vmr.nerim.net>

results for 69b0b713865e on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogy5y16t', '-x']

From python-checkins at python.org  Mon Jun 17 15:11:39 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 17 Jun 2013 15:11:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjIz?=
 =?utf-8?q?=3A_Refactor_test=5Ftarfile=2E?=
Message-ID: <3bYt9R3mG1zSyF@mail.python.org>

http://hg.python.org/cpython/rev/4b2188b13dd2
changeset:   84178:4b2188b13dd2
branch:      3.3
parent:      84175:e7ab65cf1e2a
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Jun 17 15:38:50 2013 +0300
summary:
  Issue #18223: Refactor test_tarfile.

* Use mixins for generating tests for different compression types.
* Make test_tarfile discoverable.
* Use more special tests (i.e. assertEqual, assertIs) instead of assertTrue.
* Add explicit test skips instead of reporting skipped tests as passed.
* Wrap long lines.
* Correct a comment for test_hardlink_extraction1.
* Add support.requires_gzip.

and some other minor enhancements.

files:
  Lib/test/support.py      |    9 +-
  Lib/test/test_tarfile.py |  717 ++++++++++++--------------
  2 files changed, 345 insertions(+), 381 deletions(-)


diff --git a/Lib/test/support.py b/Lib/test/support.py
--- a/Lib/test/support.py
+++ b/Lib/test/support.py
@@ -43,6 +43,11 @@
     zlib = None
 
 try:
+    import gzip
+except ImportError:
+    gzip = None
+
+try:
     import bz2
 except ImportError:
     bz2 = None
@@ -71,7 +76,7 @@
     "TestHandler", "Matcher", "can_symlink", "skip_unless_symlink",
     "skip_unless_xattr", "import_fresh_module", "requires_zlib",
     "PIPE_MAX_SIZE", "failfast", "anticipate_failure", "run_with_tz",
-    "requires_bz2", "requires_lzma", "suppress_crash_popup",
+    "requires_gzip", "requires_bz2", "requires_lzma", "suppress_crash_popup",
     ]
 
 class Error(Exception):
@@ -588,6 +593,8 @@
 
 requires_zlib = unittest.skipUnless(zlib, 'requires zlib')
 
+requires_gzip = unittest.skipUnless(gzip, 'requires gzip')
+
 requires_bz2 = unittest.skipUnless(bz2, 'requires bz2')
 
 requires_lzma = unittest.skipUnless(lzma, 'requires lzma')
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -2,9 +2,7 @@
 import os
 import io
 import shutil
-import io
 from hashlib import md5
-import errno
 
 import unittest
 import tarfile
@@ -14,8 +12,7 @@
 # Check for our compression modules.
 try:
     import gzip
-    gzip.GzipFile
-except (ImportError, AttributeError):
+except ImportError:
     gzip = None
 try:
     import bz2
@@ -40,25 +37,55 @@
 md5_sparse = "a54fbc4ca4f4399a90e1b27164012fc6"
 
 
-class ReadTest(unittest.TestCase):
+class TarTest:
+    tarname = tarname
+    suffix = ''
+    open = io.FileIO
 
-    tarname = tarname
-    mode = "r:"
+    @property
+    def mode(self):
+        return self.prefix + self.suffix
+
+ at support.requires_gzip
+class GzipTest:
+    tarname = gzipname
+    suffix = 'gz'
+    open = gzip.GzipFile if gzip else None
+
+ at support.requires_bz2
+class Bz2Test:
+    tarname = bz2name
+    suffix = 'bz2'
+    open = bz2.BZ2File if bz2 else None
+
+ at support.requires_lzma
+class LzmaTest:
+    tarname = xzname
+    suffix = 'xz'
+    open = lzma.LZMAFile if lzma else None
+
+
+class ReadTest(TarTest):
+
+    prefix = "r:"
 
     def setUp(self):
-        self.tar = tarfile.open(self.tarname, mode=self.mode, encoding="iso8859-1")
+        self.tar = tarfile.open(self.tarname, mode=self.mode,
+                                encoding="iso8859-1")
 
     def tearDown(self):
         self.tar.close()
 
 
-class UstarReadTest(ReadTest):
+class UstarReadTest(ReadTest, unittest.TestCase):
 
     def test_fileobj_regular_file(self):
         tarinfo = self.tar.getmember("ustar/regtype")
         with self.tar.extractfile(tarinfo) as fobj:
             data = fobj.read()
-            self.assertTrue((len(data), md5sum(data)) == (tarinfo.size, md5_regtype),
+            self.assertEqual(len(data), tarinfo.size,
+                    "regular file extraction failed")
+            self.assertEqual(md5sum(data), md5_regtype,
                     "regular file extraction failed")
 
     def test_fileobj_readlines(self):
@@ -70,12 +97,13 @@
         with self.tar.extractfile(tarinfo) as fobj:
             fobj2 = io.TextIOWrapper(fobj)
             lines2 = fobj2.readlines()
-            self.assertTrue(lines1 == lines2,
+            self.assertEqual(lines1, lines2,
                     "fileobj.readlines() failed")
-            self.assertTrue(len(lines2) == 114,
+            self.assertEqual(len(lines2), 114,
                     "fileobj.readlines() failed")
-            self.assertTrue(lines2[83] ==
-                    "I will gladly admit that Python is not the fastest running scripting language.\n",
+            self.assertEqual(lines2[83],
+                    "I will gladly admit that Python is not the fastest "
+                    "running scripting language.\n",
                     "fileobj.readlines() failed")
 
     def test_fileobj_iter(self):
@@ -85,8 +113,8 @@
             lines1 = fobj1.readlines()
         with self.tar.extractfile(tarinfo) as fobj2:
             lines2 = list(io.TextIOWrapper(fobj2))
-            self.assertTrue(lines1 == lines2,
-                         "fileobj.__iter__() failed")
+            self.assertEqual(lines1, lines2,
+                    "fileobj.__iter__() failed")
 
     def test_fileobj_seek(self):
         self.tar.extract("ustar/regtype", TEMPDIR)
@@ -110,12 +138,12 @@
         self.assertEqual(2048, fobj.tell(),
                      "seek() to positive relative position failed")
         s = fobj.read(10)
-        self.assertTrue(s == data[2048:2058],
+        self.assertEqual(s, data[2048:2058],
                      "read() after seek failed")
         fobj.seek(0, 2)
         self.assertEqual(tarinfo.size, fobj.tell(),
                      "seek() to file's end failed")
-        self.assertTrue(fobj.read() == b"",
+        self.assertEqual(fobj.read(), b"",
                      "read() at file's end did not return empty string")
         fobj.seek(-tarinfo.size, 2)
         self.assertEqual(0, fobj.tell(),
@@ -124,13 +152,13 @@
         s1 = fobj.readlines()
         fobj.seek(512)
         s2 = fobj.readlines()
-        self.assertTrue(s1 == s2,
+        self.assertEqual(s1, s2,
                      "readlines() after seek failed")
         fobj.seek(0)
         self.assertEqual(len(fobj.readline()), fobj.tell(),
                      "tell() after readline() failed")
         fobj.seek(512)
-        self.assertTrue(len(fobj.readline()) + 512 == fobj.tell(),
+        self.assertEqual(len(fobj.readline()) + 512, fobj.tell(),
                      "tell() after seek() and readline() failed")
         fobj.seek(0)
         line = fobj.readline()
@@ -154,24 +182,36 @@
     # test link members each point to a regular member whose data is
     # supposed to be exported.
     def _test_fileobj_link(self, lnktype, regtype):
-        with self.tar.extractfile(lnktype) as a, self.tar.extractfile(regtype) as b:
+        with self.tar.extractfile(lnktype) as a, \
+             self.tar.extractfile(regtype) as b:
             self.assertEqual(a.name, b.name)
 
     def test_fileobj_link1(self):
         self._test_fileobj_link("ustar/lnktype", "ustar/regtype")
 
     def test_fileobj_link2(self):
-        self._test_fileobj_link("./ustar/linktest2/lnktype", "ustar/linktest1/regtype")
+        self._test_fileobj_link("./ustar/linktest2/lnktype",
+                                "ustar/linktest1/regtype")
 
     def test_fileobj_symlink1(self):
         self._test_fileobj_link("ustar/symtype", "ustar/regtype")
 
     def test_fileobj_symlink2(self):
-        self._test_fileobj_link("./ustar/linktest2/symtype", "ustar/linktest1/regtype")
+        self._test_fileobj_link("./ustar/linktest2/symtype",
+                                "ustar/linktest1/regtype")
 
     def test_issue14160(self):
         self._test_fileobj_link("symtype2", "ustar/regtype")
 
+class GzipUstarReadTest(GzipTest, UstarReadTest):
+    pass
+
+class Bz2UstarReadTest(Bz2Test, UstarReadTest):
+    pass
+
+class LzmaUstarReadTest(LzmaTest, UstarReadTest):
+    pass
+
 
 class CommonReadTest(ReadTest):
 
@@ -203,37 +243,24 @@
 
     def test_ignore_zeros(self):
         # Test TarFile's ignore_zeros option.
-        if self.mode.endswith(":gz"):
-            _open = gzip.GzipFile
-        elif self.mode.endswith(":bz2"):
-            _open = bz2.BZ2File
-        elif self.mode.endswith(":xz"):
-            _open = lzma.LZMAFile
-        else:
-            _open = io.FileIO
-
         for char in (b'\0', b'a'):
             # Test if EOFHeaderError ('\0') and InvalidHeaderError ('a')
             # are ignored correctly.
-            with _open(tmpname, "w") as fobj:
+            with self.open(tmpname, "w") as fobj:
                 fobj.write(char * 1024)
                 fobj.write(tarfile.TarInfo("foo").tobuf())
 
             tar = tarfile.open(tmpname, mode="r", ignore_zeros=True)
             try:
                 self.assertListEqual(tar.getnames(), ["foo"],
-                    "ignore_zeros=True should have skipped the %r-blocks" % char)
+                    "ignore_zeros=True should have skipped the %r-blocks" %
+                    char)
             finally:
                 tar.close()
 
 
-class MiscReadTest(CommonReadTest):
-
+class MiscReadTestBase(CommonReadTest):
     def test_no_name_argument(self):
-        if self.mode.endswith(("bz2", "xz")):
-            # BZ2File and LZMAFile have no name attribute.
-            self.skipTest("no name attribute")
-
         with open(self.tarname, "rb") as fobj:
             tar = tarfile.open(fileobj=fobj, mode=self.mode)
             self.assertEqual(tar.name, os.path.abspath(fobj.name))
@@ -269,16 +296,7 @@
             tar.close()
 
         # Open the testtar and seek to the offset of the second member.
-        if self.mode.endswith(":gz"):
-            _open = gzip.GzipFile
-        elif self.mode.endswith(":bz2"):
-            _open = bz2.BZ2File
-        elif self.mode.endswith(":xz"):
-            _open = lzma.LZMAFile
-        else:
-            _open = io.FileIO
-
-        with _open(self.tarname) as fobj:
+        with self.open(self.tarname) as fobj:
             fobj.seek(offset)
 
             # Test if the tarfile starts with the second member.
@@ -294,8 +312,6 @@
 
     def test_fail_comp(self):
         # For Gzip and Bz2 Tests: fail with a ReadError on an uncompressed file.
-        if self.mode == "r:":
-            return
         self.assertRaises(tarfile.ReadError, tarfile.open, tarname, self.mode)
         with open(tarname, "rb") as fobj:
             self.assertRaises(tarfile.ReadError, tarfile.open,
@@ -306,7 +322,7 @@
         # Old V7 tars create directory members using an AREGTYPE
         # header with a "/" appended to the filename field.
         tarinfo = self.tar.getmember("misc/dirtype-old-v7")
-        self.assertTrue(tarinfo.type == tarfile.DIRTYPE,
+        self.assertEqual(tarinfo.type, tarfile.DIRTYPE,
                 "v7 dirtype failed")
 
     def test_xstar_type(self):
@@ -320,15 +336,15 @@
 
     def test_check_members(self):
         for tarinfo in self.tar:
-            self.assertTrue(int(tarinfo.mtime) == 0o7606136617,
+            self.assertEqual(int(tarinfo.mtime), 0o7606136617,
                     "wrong mtime for %s" % tarinfo.name)
             if not tarinfo.name.startswith("ustar/"):
                 continue
-            self.assertTrue(tarinfo.uname == "tarfile",
+            self.assertEqual(tarinfo.uname, "tarfile",
                     "wrong uname for %s" % tarinfo.name)
 
     def test_find_members(self):
-        self.assertTrue(self.tar.getmembers()[-1].name == "misc/eof",
+        self.assertEqual(self.tar.getmembers()[-1].name, "misc/eof",
                 "could not find all members")
 
     @unittest.skipUnless(hasattr(os, "link"),
@@ -365,7 +381,8 @@
                 path = os.path.join(DIR, tarinfo.name)
                 if sys.platform != "win32":
                     # Win32 has no support for fine grained permissions.
-                    self.assertEqual(tarinfo.mode & 0o777, os.stat(path).st_mode & 0o777)
+                    self.assertEqual(tarinfo.mode & 0o777,
+                                     os.stat(path).st_mode & 0o777)
                 def format_mtime(mtime):
                     if isinstance(mtime, float):
                         return "{} ({})".format(mtime, mtime.hex())
@@ -423,10 +440,28 @@
                 self.assertEqual(m1.offset, m2.offset)
                 self.assertEqual(m1.get_info(), m2.get_info())
 
+class MiscReadTest(MiscReadTestBase, unittest.TestCase):
+    test_fail_comp = None
 
-class StreamReadTest(CommonReadTest):
+class GzipMiscReadTest(GzipTest, MiscReadTestBase, unittest.TestCase):
+    def test_non_existent_targz_file(self):
+        # Test for issue11513: prevent non-existent gzipped tarfiles raising
+        # multiple exceptions.
+        with self.assertRaisesRegex(FileNotFoundError, "xxx"):
+            tarfile.open("xxx", self.mode)
 
-    mode="r|"
+class Bz2MiscReadTest(Bz2Test, MiscReadTestBase, unittest.TestCase):
+    def test_no_name_argument(self):
+        self.skipTest("BZ2File have no name attribute")
+
+class LzmaMiscReadTest(LzmaTest, MiscReadTestBase, unittest.TestCase):
+    def test_no_name_argument(self):
+        self.skipTest("LZMAFile have no name attribute")
+
+
+class StreamReadTest(CommonReadTest, unittest.TestCase):
+
+    prefix="r|"
 
     def test_read_through(self):
         # Issue #11224: A poorly designed _FileInFile.read() method
@@ -439,7 +474,8 @@
                     try:
                         buf = fobj.read(512)
                     except tarfile.StreamError:
-                        self.fail("simple read-through using TarFile.extractfile() failed")
+                        self.fail("simple read-through using "
+                                  "TarFile.extractfile() failed")
                     if not buf:
                         break
 
@@ -447,7 +483,9 @@
         tarinfo = self.tar.next() # get "regtype" (can't use getmember)
         with self.tar.extractfile(tarinfo) as fobj:
             data = fobj.read()
-        self.assertTrue((len(data), md5sum(data)) == (tarinfo.size, md5_regtype),
+        self.assertEqual(len(data), tarinfo.size,
+                "regular file extraction failed")
+        self.assertEqual(md5sum(data), md5_regtype,
                 "regular file extraction failed")
 
     def test_provoke_stream_error(self):
@@ -465,24 +503,34 @@
                 t2 = tar2.next()
                 if t1 is None:
                     break
-                self.assertTrue(t2 is not None, "stream.next() failed.")
+                self.assertIsNotNone(t2, "stream.next() failed.")
 
                 if t2.islnk() or t2.issym():
-                    self.assertRaises(tarfile.StreamError, tar2.extractfile, t2)
+                    with self.assertRaises(tarfile.StreamError):
+                        tar2.extractfile(t2)
                     continue
 
                 v1 = tar1.extractfile(t1)
                 v2 = tar2.extractfile(t2)
                 if v1 is None:
                     continue
-                self.assertTrue(v2 is not None, "stream.extractfile() failed")
-                self.assertEqual(v1.read(), v2.read(), "stream extraction failed")
+                self.assertIsNotNone(v2, "stream.extractfile() failed")
+                self.assertEqual(v1.read(), v2.read(),
+                        "stream extraction failed")
         finally:
             tar1.close()
 
+class GzipStreamReadTest(GzipTest, StreamReadTest):
+    pass
 
-class DetectReadTest(unittest.TestCase):
+class Bz2StreamReadTest(Bz2Test, StreamReadTest):
+    pass
 
+class LzmaStreamReadTest(LzmaTest, StreamReadTest):
+    pass
+
+
+class DetectReadTest(TarTest, unittest.TestCase):
     def _testfunc_file(self, name, mode):
         try:
             tar = tarfile.open(name, mode)
@@ -501,47 +549,20 @@
             tar.close()
 
     def _test_modes(self, testfunc):
-        testfunc(tarname, "r")
-        testfunc(tarname, "r:")
-        testfunc(tarname, "r:*")
-        testfunc(tarname, "r|")
-        testfunc(tarname, "r|*")
-
-        if gzip:
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:gz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|gz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, gzipname, mode="r:")
-            self.assertRaises(tarfile.ReadError, tarfile.open, gzipname, mode="r|")
-
-            testfunc(gzipname, "r")
-            testfunc(gzipname, "r:*")
-            testfunc(gzipname, "r:gz")
-            testfunc(gzipname, "r|*")
-            testfunc(gzipname, "r|gz")
-
-        if bz2:
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:bz2")
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|bz2")
-            self.assertRaises(tarfile.ReadError, tarfile.open, bz2name, mode="r:")
-            self.assertRaises(tarfile.ReadError, tarfile.open, bz2name, mode="r|")
-
-            testfunc(bz2name, "r")
-            testfunc(bz2name, "r:*")
-            testfunc(bz2name, "r:bz2")
-            testfunc(bz2name, "r|*")
-            testfunc(bz2name, "r|bz2")
-
-        if lzma:
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:xz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|xz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r:")
-            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r|")
-
-            testfunc(xzname, "r")
-            testfunc(xzname, "r:*")
-            testfunc(xzname, "r:xz")
-            testfunc(xzname, "r|*")
-            testfunc(xzname, "r|xz")
+        if self.suffix:
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(tarname, mode="r:" + self.suffix)
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(tarname, mode="r|" + self.suffix)
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(self.tarname, mode="r:")
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(self.tarname, mode="r|")
+        testfunc(self.tarname, "r")
+        testfunc(self.tarname, "r:" + self.suffix)
+        testfunc(self.tarname, "r:*")
+        testfunc(self.tarname, "r|" + self.suffix)
+        testfunc(self.tarname, "r|*")
 
     def test_detect_file(self):
         self._test_modes(self._testfunc_file)
@@ -549,14 +570,15 @@
     def test_detect_fileobj(self):
         self._test_modes(self._testfunc_fileobj)
 
+class GzipDetectReadTest(GzipTest, DetectReadTest):
+    pass
+
+class Bz2DetectReadTest(Bz2Test, DetectReadTest):
     def test_detect_stream_bz2(self):
         # Originally, tarfile's stream detection looked for the string
         # "BZh91" at the start of the file. This is incorrect because
         # the '9' represents the blocksize (900kB). If the file was
         # compressed using another blocksize autodetection fails.
-        if not bz2:
-            return
-
         with open(tarname, "rb") as fobj:
             data = fobj.read()
 
@@ -566,13 +588,17 @@
 
         self._testfunc_file(tmpname, "r|*")
 
+class LzmaDetectReadTest(LzmaTest, DetectReadTest):
+    pass
 
-class MemberReadTest(ReadTest):
+
+class MemberReadTest(ReadTest, unittest.TestCase):
 
     def _test_member(self, tarinfo, chksum=None, **kwargs):
         if chksum is not None:
-            self.assertTrue(md5sum(self.tar.extractfile(tarinfo).read()) == chksum,
-                    "wrong md5sum for %s" % tarinfo.name)
+            with self.tar.extractfile(tarinfo) as f:
+                self.assertEqual(md5sum(f.read()), chksum,
+                        "wrong md5sum for %s" % tarinfo.name)
 
         kwargs["mtime"] = 0o7606136617
         kwargs["uid"] = 1000
@@ -582,7 +608,7 @@
             kwargs["uname"] = "tarfile"
             kwargs["gname"] = "tarfile"
         for k, v in kwargs.items():
-            self.assertTrue(getattr(tarinfo, k) == v,
+            self.assertEqual(getattr(tarinfo, k), v,
                     "wrong value in %s field of %s" % (k, tarinfo.name))
 
     def test_find_regtype(self):
@@ -642,7 +668,8 @@
         self._test_member(tarinfo, size=86016, chksum=md5_sparse)
 
     def test_find_umlauts(self):
-        tarinfo = self.tar.getmember("ustar/umlauts-\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+        tarinfo = self.tar.getmember("ustar/umlauts-"
+                                     "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
         self._test_member(tarinfo, size=7011, chksum=md5_regtype)
 
     def test_find_ustar_longname(self):
@@ -655,12 +682,14 @@
 
     def test_find_pax_umlauts(self):
         self.tar.close()
-        self.tar = tarfile.open(self.tarname, mode=self.mode, encoding="iso8859-1")
-        tarinfo = self.tar.getmember("pax/umlauts-\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+        self.tar = tarfile.open(self.tarname, mode=self.mode,
+                                encoding="iso8859-1")
+        tarinfo = self.tar.getmember("pax/umlauts-"
+                                     "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
         self._test_member(tarinfo, size=7011, chksum=md5_regtype)
 
 
-class LongnameTest(ReadTest):
+class LongnameTest:
 
     def test_read_longname(self):
         # Test reading of longname (bug #1471427).
@@ -669,7 +698,8 @@
             tarinfo = self.tar.getmember(longname)
         except KeyError:
             self.fail("longname not found")
-        self.assertTrue(tarinfo.type != tarfile.DIRTYPE, "read longname as dirtype")
+        self.assertNotEqual(tarinfo.type, tarfile.DIRTYPE,
+                "read longname as dirtype")
 
     def test_read_longlink(self):
         longname = self.subdir + "/" + "123/" * 125 + "longname"
@@ -678,7 +708,7 @@
             tarinfo = self.tar.getmember(longlink)
         except KeyError:
             self.fail("longlink not found")
-        self.assertTrue(tarinfo.linkname == longname, "linkname wrong")
+        self.assertEqual(tarinfo.linkname, longname, "linkname wrong")
 
     def test_truncated_longname(self):
         longname = self.subdir + "/" + "123/" * 125 + "longname"
@@ -686,7 +716,8 @@
         offset = tarinfo.offset
         self.tar.fileobj.seek(offset)
         fobj = io.BytesIO(self.tar.fileobj.read(3 * 512))
-        self.assertRaises(tarfile.ReadError, tarfile.open, name="foo.tar", fileobj=fobj)
+        with self.assertRaises(tarfile.ReadError):
+            tarfile.open(name="foo.tar", fileobj=fobj)
 
     def test_header_offset(self):
         # Test if the start offset of the TarInfo object includes
@@ -695,11 +726,12 @@
         offset = self.tar.getmember(longname).offset
         with open(tarname, "rb") as fobj:
             fobj.seek(offset)
-            tarinfo = tarfile.TarInfo.frombuf(fobj.read(512), "iso8859-1", "strict")
+            tarinfo = tarfile.TarInfo.frombuf(fobj.read(512),
+                                              "iso8859-1", "strict")
             self.assertEqual(tarinfo.type, self.longnametype)
 
 
-class GNUReadTest(LongnameTest):
+class GNUReadTest(LongnameTest, ReadTest, unittest.TestCase):
 
     subdir = "gnu"
     longnametype = tarfile.GNUTYPE_LONGNAME
@@ -721,7 +753,7 @@
 
         if self._fs_supports_holes():
             s = os.stat(filename)
-            self.assertTrue(s.st_blocks * 512 < s.st_size)
+            self.assertLess(s.st_blocks * 512, s.st_size)
 
     def test_sparse_file_old(self):
         self._test_sparse_file("gnu/sparse")
@@ -753,7 +785,7 @@
             return False
 
 
-class PaxReadTest(LongnameTest):
+class PaxReadTest(LongnameTest, ReadTest, unittest.TestCase):
 
     subdir = "pax"
     longnametype = tarfile.XHDTYPE
@@ -764,17 +796,20 @@
             tarinfo = tar.getmember("pax/regtype1")
             self.assertEqual(tarinfo.uname, "foo")
             self.assertEqual(tarinfo.gname, "bar")
-            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"),
+                             "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
 
             tarinfo = tar.getmember("pax/regtype2")
             self.assertEqual(tarinfo.uname, "")
             self.assertEqual(tarinfo.gname, "bar")
-            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"),
+                             "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
 
             tarinfo = tar.getmember("pax/regtype3")
             self.assertEqual(tarinfo.uname, "tarfile")
             self.assertEqual(tarinfo.gname, "tarfile")
-            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"),
+                             "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
         finally:
             tar.close()
 
@@ -794,7 +829,7 @@
             tar.close()
 
 
-class WriteTestBase(unittest.TestCase):
+class WriteTestBase(TarTest):
     # Put all write tests in here that are supposed to be tested
     # in all possible mode combinations.
 
@@ -803,12 +838,12 @@
         tar = tarfile.open(fileobj=fobj, mode=self.mode)
         tar.addfile(tarfile.TarInfo("foo"))
         tar.close()
-        self.assertTrue(fobj.closed is False, "external fileobjs must never closed")
+        self.assertFalse(fobj.closed, "external fileobjs must never closed")
 
 
-class WriteTest(WriteTestBase):
+class WriteTest(WriteTestBase, unittest.TestCase):
 
-    mode = "w:"
+    prefix = "w:"
 
     def test_100_char_name(self):
         # The name field in a tar header stores strings of at most 100 chars.
@@ -825,7 +860,7 @@
 
         tar = tarfile.open(tmpname)
         try:
-            self.assertTrue(tar.getnames()[0] == name,
+            self.assertEqual(tar.getnames()[0], name,
                     "failed to store 100 char filename")
         finally:
             tar.close()
@@ -840,7 +875,7 @@
             tar.add(path)
         finally:
             tar.close()
-        self.assertTrue(os.path.getsize(tmpname) > 0,
+        self.assertGreater(os.path.getsize(tmpname), 0,
                 "tarfile is empty")
 
     # The test_*_size tests test for bug #1167128.
@@ -873,25 +908,26 @@
         finally:
             os.rmdir(path)
 
+    @unittest.skipUnless(hasattr(os, "link"),
+                         "Missing hardlink implementation")
     def test_link_size(self):
-        if hasattr(os, "link"):
-            link = os.path.join(TEMPDIR, "link")
-            target = os.path.join(TEMPDIR, "link_target")
-            with open(target, "wb") as fobj:
-                fobj.write(b"aaa")
-            os.link(target, link)
+        link = os.path.join(TEMPDIR, "link")
+        target = os.path.join(TEMPDIR, "link_target")
+        with open(target, "wb") as fobj:
+            fobj.write(b"aaa")
+        os.link(target, link)
+        try:
+            tar = tarfile.open(tmpname, self.mode)
             try:
-                tar = tarfile.open(tmpname, self.mode)
-                try:
-                    # Record the link target in the inodes list.
-                    tar.gettarinfo(target)
-                    tarinfo = tar.gettarinfo(link)
-                    self.assertEqual(tarinfo.size, 0)
-                finally:
-                    tar.close()
+                # Record the link target in the inodes list.
+                tar.gettarinfo(target)
+                tarinfo = tar.gettarinfo(link)
+                self.assertEqual(tarinfo.size, 0)
             finally:
-                os.remove(target)
-                os.remove(link)
+                tar.close()
+        finally:
+            os.remove(target)
+            os.remove(link)
 
     @support.skip_unless_symlink
     def test_symlink_size(self):
@@ -912,15 +948,18 @@
         dstname = os.path.abspath(tmpname)
         tar = tarfile.open(tmpname, self.mode)
         try:
-            self.assertTrue(tar.name == dstname, "archive name must be absolute")
+            self.assertEqual(tar.name, dstname,
+                    "archive name must be absolute")
             tar.add(dstname)
-            self.assertTrue(tar.getnames() == [], "added the archive to itself")
+            self.assertEqual(tar.getnames(), [],
+                    "added the archive to itself")
 
             cwd = os.getcwd()
             os.chdir(TEMPDIR)
             tar.add(dstname)
             os.chdir(cwd)
-            self.assertTrue(tar.getnames() == [], "added the archive to itself")
+            self.assertEqual(tar.getnames(), [],
+                    "added the archive to itself")
         finally:
             tar.close()
 
@@ -1087,48 +1126,49 @@
             tar = tarfile.open(tmpname, "r")
             try:
                 for t in tar:
-                    self.assertTrue(t.name == "." or t.name.startswith("./"))
+                    if t.name != ".":
+                        self.assertTrue(t.name.startswith("./"), t.name)
             finally:
                 tar.close()
         finally:
             os.chdir(cwd)
 
+class GzipWriteTest(GzipTest, WriteTest):
+    pass
 
-class StreamWriteTest(WriteTestBase):
+class Bz2WriteTest(Bz2Test, WriteTest):
+    pass
 
-    mode = "w|"
+class LzmaWriteTest(LzmaTest, WriteTest):
+    pass
+
+
+class StreamWriteTest(WriteTestBase, unittest.TestCase):
+
+    prefix = "w|"
+    decompressor = None
 
     def test_stream_padding(self):
         # Test for bug #1543303.
         tar = tarfile.open(tmpname, self.mode)
         tar.close()
-
-        if self.mode.endswith("gz"):
-            with gzip.GzipFile(tmpname) as fobj:
-                data = fobj.read()
-        elif self.mode.endswith("bz2"):
-            dec = bz2.BZ2Decompressor()
+        if self.decompressor:
+            dec = self.decompressor()
             with open(tmpname, "rb") as fobj:
                 data = fobj.read()
             data = dec.decompress(data)
-            self.assertTrue(len(dec.unused_data) == 0,
-                    "found trailing data")
-        elif self.mode.endswith("xz"):
-            with lzma.LZMAFile(tmpname) as fobj:
+            self.assertFalse(dec.unused_data, "found trailing data")
+        else:
+            with self.open(tmpname) as fobj:
                 data = fobj.read()
-        else:
-            with open(tmpname, "rb") as fobj:
-                data = fobj.read()
+        self.assertEqual(data.count(b"\0"), tarfile.RECORDSIZE,
+                        "incorrect zero padding")
 
-        self.assertTrue(data.count(b"\0") == tarfile.RECORDSIZE,
-                         "incorrect zero padding")
-
+    @unittest.skipUnless(sys.platform != "win32" and hasattr(os, "umask"),
+                         "Missing umask implementation")
     def test_file_mode(self):
         # Test for issue #8464: Create files with correct
         # permissions.
-        if sys.platform == "win32" or not hasattr(os, "umask"):
-            return
-
         if os.path.exists(tmpname):
             os.remove(tmpname)
 
@@ -1141,15 +1181,22 @@
         finally:
             os.umask(original_umask)
 
+class GzipStreamWriteTest(GzipTest, StreamWriteTest):
+    pass
+
+class Bz2StreamWriteTest(Bz2Test, StreamWriteTest):
+    decompressor = bz2.BZ2Decompressor if bz2 else None
+
+class LzmaStreamWriteTest(LzmaTest, StreamWriteTest):
+    decompressor = lzma.LZMADecompressor if lzma else None
+
 
 class GNUWriteTest(unittest.TestCase):
     # This testcase checks for correct creation of GNU Longname
     # and Longlink extended headers (cp. bug #812325).
 
     def _length(self, s):
-        blocks, remainder = divmod(len(s) + 1, 512)
-        if remainder:
-            blocks += 1
+        blocks = len(s) // 512 + 1
         return blocks * 512
 
     def _calc_size(self, name, link=None):
@@ -1179,7 +1226,7 @@
 
             v1 = self._calc_size(name, link)
             v2 = tar.offset
-            self.assertTrue(v1 == v2, "GNU longname/longlink creation failed")
+            self.assertEqual(v1, v2, "GNU longname/longlink creation failed")
         finally:
             tar.close()
 
@@ -1226,6 +1273,7 @@
                    ("longlnk/" * 127) + "longlink_")
 
 
+ at unittest.skipUnless(hasattr(os, "link"), "Missing hardlink implementation")
 class HardlinkTest(unittest.TestCase):
     # Test the creation of LNKTYPE (hardlink) members in an archive.
 
@@ -1250,18 +1298,18 @@
         # The same name will be added as a REGTYPE every
         # time regardless of st_nlink.
         tarinfo = self.tar.gettarinfo(self.foo)
-        self.assertTrue(tarinfo.type == tarfile.REGTYPE,
+        self.assertEqual(tarinfo.type, tarfile.REGTYPE,
                 "add file as regular failed")
 
     def test_add_hardlink(self):
         tarinfo = self.tar.gettarinfo(self.bar)
-        self.assertTrue(tarinfo.type == tarfile.LNKTYPE,
+        self.assertEqual(tarinfo.type, tarfile.LNKTYPE,
                 "add file as hardlink failed")
 
     def test_dereference_hardlink(self):
         self.tar.dereference = True
         tarinfo = self.tar.gettarinfo(self.bar)
-        self.assertTrue(tarinfo.type == tarfile.REGTYPE,
+        self.assertEqual(tarinfo.type, tarfile.REGTYPE,
                 "dereferencing hardlink failed")
 
 
@@ -1284,10 +1332,10 @@
         try:
             if link:
                 l = tar.getmembers()[0].linkname
-                self.assertTrue(link == l, "PAX longlink creation failed")
+                self.assertEqual(link, l, "PAX longlink creation failed")
             else:
                 n = tar.getmembers()[0].name
-                self.assertTrue(name == n, "PAX longname creation failed")
+                self.assertEqual(name, n, "PAX longname creation failed")
         finally:
             tar.close()
 
@@ -1313,8 +1361,8 @@
             self.assertEqual(tar.getmembers()[0].pax_headers, pax_headers)
             # Test if all the fields are strings.
             for key, val in tar.pax_headers.items():
-                self.assertTrue(type(key) is not bytes)
-                self.assertTrue(type(val) is not bytes)
+                self.assertIsNot(type(key), bytes)
+                self.assertIsNot(type(val), bytes)
                 if key in tarfile.PAX_NUMBER_FIELDS:
                     try:
                         tarfile.PAX_NUMBER_FIELDS[key](val)
@@ -1328,7 +1376,8 @@
         # TarInfo.
         pax_headers = {"path": "foo", "uid": "123"}
 
-        tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT, encoding="iso8859-1")
+        tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT,
+                           encoding="iso8859-1")
         try:
             t = tarfile.TarInfo()
             t.name = "\xe4\xf6\xfc" # non-ASCII
@@ -1362,7 +1411,8 @@
         self._test_unicode_filename("utf-8")
 
     def _test_unicode_filename(self, encoding):
-        tar = tarfile.open(tmpname, "w", format=self.format, encoding=encoding, errors="strict")
+        tar = tarfile.open(tmpname, "w", format=self.format,
+                           encoding=encoding, errors="strict")
         try:
             name = "\xe4\xf6\xfc"
             tar.addfile(tarfile.TarInfo(name))
@@ -1376,11 +1426,8 @@
             tar.close()
 
     def test_unicode_filename_error(self):
-        if self.format == tarfile.PAX_FORMAT:
-            # PAX_FORMAT ignores encoding in write mode.
-            return
-
-        tar = tarfile.open(tmpname, "w", format=self.format, encoding="ascii", errors="strict")
+        tar = tarfile.open(tmpname, "w", format=self.format,
+                           encoding="ascii", errors="strict")
         try:
             tarinfo = tarfile.TarInfo()
 
@@ -1394,13 +1441,14 @@
             tar.close()
 
     def test_unicode_argument(self):
-        tar = tarfile.open(tarname, "r", encoding="iso8859-1", errors="strict")
+        tar = tarfile.open(tarname, "r",
+                           encoding="iso8859-1", errors="strict")
         try:
             for t in tar:
-                self.assertTrue(type(t.name) is str)
-                self.assertTrue(type(t.linkname) is str)
-                self.assertTrue(type(t.uname) is str)
-                self.assertTrue(type(t.gname) is str)
+                self.assertIs(type(t.name), str)
+                self.assertIs(type(t.linkname), str)
+                self.assertIs(type(t.uname), str)
+                self.assertIs(type(t.gname), str)
         finally:
             tar.close()
 
@@ -1409,7 +1457,8 @@
         t.uname = "\xe4\xf6\xfc"
         t.gname = "\xe4\xf6\xfc"
 
-        tar = tarfile.open(tmpname, mode="w", format=self.format, encoding="iso8859-1")
+        tar = tarfile.open(tmpname, mode="w", format=self.format,
+                           encoding="iso8859-1")
         try:
             tar.addfile(t)
         finally:
@@ -1438,9 +1487,11 @@
     def test_bad_pax_header(self):
         # Test for issue #8633. GNU tar <= 1.23 creates raw binary fields
         # without a hdrcharset=BINARY header.
-        for encoding, name in (("utf-8", "pax/bad-pax-\udce4\udcf6\udcfc"),
+        for encoding, name in (
+                ("utf-8", "pax/bad-pax-\udce4\udcf6\udcfc"),
                 ("iso8859-1", "pax/bad-pax-\xe4\xf6\xfc"),):
-            with tarfile.open(tarname, encoding=encoding, errors="surrogateescape") as tar:
+            with tarfile.open(tarname, encoding=encoding,
+                              errors="surrogateescape") as tar:
                 try:
                     t = tar.getmember(name)
                 except KeyError:
@@ -1451,18 +1502,23 @@
 
     format = tarfile.PAX_FORMAT
 
+    # PAX_FORMAT ignores encoding in write mode.
+    test_unicode_filename_error = None
+
     def test_binary_header(self):
         # Test a POSIX.1-2008 compatible header with a hdrcharset=BINARY field.
-        for encoding, name in (("utf-8", "pax/hdrcharset-\udce4\udcf6\udcfc"),
+        for encoding, name in (
+                ("utf-8", "pax/hdrcharset-\udce4\udcf6\udcfc"),
                 ("iso8859-1", "pax/hdrcharset-\xe4\xf6\xfc"),):
-            with tarfile.open(tarname, encoding=encoding, errors="surrogateescape") as tar:
+            with tarfile.open(tarname, encoding=encoding,
+                              errors="surrogateescape") as tar:
                 try:
                     t = tar.getmember(name)
                 except KeyError:
                     self.fail("unable to read POSIX.1-2008 binary header")
 
 
-class AppendTest(unittest.TestCase):
+class AppendTestBase:
     # Test append mode (cp. patch #1652681).
 
     def setUp(self):
@@ -1470,10 +1526,6 @@
         if os.path.exists(self.tarname):
             os.remove(self.tarname)
 
-    def _add_testfile(self, fileobj=None):
-        with tarfile.open(self.tarname, "a", fileobj=fileobj) as tar:
-            tar.addfile(tarfile.TarInfo("bar"))
-
     def _create_testtar(self, mode="w:"):
         with tarfile.open(tarname, encoding="iso8859-1") as src:
             t = src.getmember("ustar/regtype")
@@ -1482,6 +1534,17 @@
                 with tarfile.open(self.tarname, mode) as tar:
                     tar.addfile(t, f)
 
+    def test_append_compressed(self):
+        self._create_testtar("w:" + self.suffix)
+        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
+
+class AppendTest(AppendTestBase, unittest.TestCase):
+    test_append_compressed = None
+
+    def _add_testfile(self, fileobj=None):
+        with tarfile.open(self.tarname, "a", fileobj=fileobj) as tar:
+            tar.addfile(tarfile.TarInfo("bar"))
+
     def _test(self, names=["bar"], fileobj=None):
         with tarfile.open(self.tarname, fileobj=fileobj) as tar:
             self.assertEqual(tar.getnames(), names)
@@ -1515,24 +1578,6 @@
         self._add_testfile()
         self._test(names=["foo", "bar"])
 
-    def test_append_gz(self):
-        if gzip is None:
-            return
-        self._create_testtar("w:gz")
-        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
-
-    def test_append_bz2(self):
-        if bz2 is None:
-            return
-        self._create_testtar("w:bz2")
-        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
-
-    def test_append_lzma(self):
-        if lzma is None:
-            self.skipTest("lzma module not available")
-        self._create_testtar("w:xz")
-        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
-
     # Append mode is supposed to fail if the tarfile to append to
     # does not end with a zero block.
     def _test_error(self, data):
@@ -1557,6 +1602,15 @@
     def test_invalid(self):
         self._test_error(b"a" * 512)
 
+class GzipAppendTest(GzipTest, AppendTestBase, unittest.TestCase):
+    pass
+
+class Bz2AppendTest(Bz2Test, AppendTestBase, unittest.TestCase):
+    pass
+
+class LzmaAppendTest(LzmaTest, AppendTestBase, unittest.TestCase):
+    pass
+
 
 class LimitsTest(unittest.TestCase):
 
@@ -1620,36 +1674,54 @@
 class MiscTest(unittest.TestCase):
 
     def test_char_fields(self):
-        self.assertEqual(tarfile.stn("foo", 8, "ascii", "strict"), b"foo\0\0\0\0\0")
-        self.assertEqual(tarfile.stn("foobar", 3, "ascii", "strict"), b"foo")
-        self.assertEqual(tarfile.nts(b"foo\0\0\0\0\0", "ascii", "strict"), "foo")
-        self.assertEqual(tarfile.nts(b"foo\0bar\0", "ascii", "strict"), "foo")
+        self.assertEqual(tarfile.stn("foo", 8, "ascii", "strict"),
+                         b"foo\0\0\0\0\0")
+        self.assertEqual(tarfile.stn("foobar", 3, "ascii", "strict"),
+                         b"foo")
+        self.assertEqual(tarfile.nts(b"foo\0\0\0\0\0", "ascii", "strict"),
+                         "foo")
+        self.assertEqual(tarfile.nts(b"foo\0bar\0", "ascii", "strict"),
+                         "foo")
 
     def test_read_number_fields(self):
         # Issue 13158: Test if GNU tar specific base-256 number fields
         # are decoded correctly.
         self.assertEqual(tarfile.nti(b"0000001\x00"), 1)
         self.assertEqual(tarfile.nti(b"7777777\x00"), 0o7777777)
-        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\x00\x20\x00\x00"), 0o10000000)
-        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\xff\xff\xff\xff"), 0xffffffff)
-        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\xff"), -1)
-        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\x9c"), -100)
-        self.assertEqual(tarfile.nti(b"\xff\x00\x00\x00\x00\x00\x00\x00"), -0x100000000000000)
+        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\x00\x20\x00\x00"),
+                         0o10000000)
+        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\xff\xff\xff\xff"),
+                         0xffffffff)
+        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\xff"),
+                         -1)
+        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\x9c"),
+                         -100)
+        self.assertEqual(tarfile.nti(b"\xff\x00\x00\x00\x00\x00\x00\x00"),
+                         -0x100000000000000)
 
     def test_write_number_fields(self):
         self.assertEqual(tarfile.itn(1), b"0000001\x00")
         self.assertEqual(tarfile.itn(0o7777777), b"7777777\x00")
-        self.assertEqual(tarfile.itn(0o10000000), b"\x80\x00\x00\x00\x00\x20\x00\x00")
-        self.assertEqual(tarfile.itn(0xffffffff), b"\x80\x00\x00\x00\xff\xff\xff\xff")
-        self.assertEqual(tarfile.itn(-1), b"\xff\xff\xff\xff\xff\xff\xff\xff")
-        self.assertEqual(tarfile.itn(-100), b"\xff\xff\xff\xff\xff\xff\xff\x9c")
-        self.assertEqual(tarfile.itn(-0x100000000000000), b"\xff\x00\x00\x00\x00\x00\x00\x00")
+        self.assertEqual(tarfile.itn(0o10000000),
+                         b"\x80\x00\x00\x00\x00\x20\x00\x00")
+        self.assertEqual(tarfile.itn(0xffffffff),
+                         b"\x80\x00\x00\x00\xff\xff\xff\xff")
+        self.assertEqual(tarfile.itn(-1),
+                         b"\xff\xff\xff\xff\xff\xff\xff\xff")
+        self.assertEqual(tarfile.itn(-100),
+                         b"\xff\xff\xff\xff\xff\xff\xff\x9c")
+        self.assertEqual(tarfile.itn(-0x100000000000000),
+                         b"\xff\x00\x00\x00\x00\x00\x00\x00")
 
     def test_number_field_limits(self):
-        self.assertRaises(ValueError, tarfile.itn, -1, 8, tarfile.USTAR_FORMAT)
-        self.assertRaises(ValueError, tarfile.itn, 0o10000000, 8, tarfile.USTAR_FORMAT)
-        self.assertRaises(ValueError, tarfile.itn, -0x10000000001, 6, tarfile.GNU_FORMAT)
-        self.assertRaises(ValueError, tarfile.itn, 0x10000000000, 6, tarfile.GNU_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(-1, 8, tarfile.USTAR_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(0o10000000, 8, tarfile.USTAR_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(-0x10000000001, 6, tarfile.GNU_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(0x10000000000, 6, tarfile.GNU_FORMAT)
 
 
 class ContextManagerTest(unittest.TestCase):
@@ -1710,19 +1782,19 @@
             self.assertTrue(tar.closed, "context manager failed")
 
 
-class LinkEmulationTest(ReadTest):
+ at unittest.skipIf(hasattr(os, "link"), "requires os.link to be missing")
+class LinkEmulationTest(ReadTest, unittest.TestCase):
 
     # Test for issue #8741 regression. On platforms that do not support
-    # symbolic or hard links tarfile tries to extract these types of members as
-    # the regular files they point to.
+    # symbolic or hard links tarfile tries to extract these types of members
+    # as the regular files they point to.
     def _test_link_extraction(self, name):
         self.tar.extract(name, TEMPDIR)
-        data = open(os.path.join(TEMPDIR, name), "rb").read()
+        with open(os.path.join(TEMPDIR, name), "rb") as f:
+            data = f.read()
         self.assertEqual(md5sum(data), md5_regtype)
 
-    # When 8879 gets fixed, this will need to change. Currently on Windows
-    # we have os.path.islink but no os.link, so these tests fail without the
-    # following skip until link is completed.
+    # See issues #1578269, #8879, and #17689 for some history on these skips
     @unittest.skipIf(hasattr(os.path, "islink"),
                      "Skip emulation - has os.path.islink but not os.link")
     def test_hardlink_extraction1(self):
@@ -1744,44 +1816,7 @@
         self._test_link_extraction("./ustar/linktest2/symtype")
 
 
-class GzipMiscReadTest(MiscReadTest):
-    tarname = gzipname
-    mode = "r:gz"
-
-    def test_non_existent_targz_file(self):
-        # Test for issue11513: prevent non-existent gzipped tarfiles raising
-        # multiple exceptions.
-        with self.assertRaisesRegex(IOError, "xxx") as ex:
-            tarfile.open("xxx", self.mode)
-        self.assertEqual(ex.exception.errno, errno.ENOENT)
-
-class GzipUstarReadTest(UstarReadTest):
-    tarname = gzipname
-    mode = "r:gz"
-class GzipStreamReadTest(StreamReadTest):
-    tarname = gzipname
-    mode = "r|gz"
-class GzipWriteTest(WriteTest):
-    mode = "w:gz"
-class GzipStreamWriteTest(StreamWriteTest):
-    mode = "w|gz"
-
-
-class Bz2MiscReadTest(MiscReadTest):
-    tarname = bz2name
-    mode = "r:bz2"
-class Bz2UstarReadTest(UstarReadTest):
-    tarname = bz2name
-    mode = "r:bz2"
-class Bz2StreamReadTest(StreamReadTest):
-    tarname = bz2name
-    mode = "r|bz2"
-class Bz2WriteTest(WriteTest):
-    mode = "w:bz2"
-class Bz2StreamWriteTest(StreamWriteTest):
-    mode = "w|bz2"
-
-class Bz2PartialReadTest(unittest.TestCase):
+class Bz2PartialReadTest(Bz2Test, unittest.TestCase):
     # Issue5068: The _BZ2Proxy.read() method loops forever
     # on an empty or partial bzipped file.
 
@@ -1790,7 +1825,8 @@
             hit_eof = False
             def read(self, n):
                 if self.hit_eof:
-                    raise AssertionError("infinite loop detected in tarfile.open()")
+                    raise AssertionError("infinite loop detected in "
+                                         "tarfile.open()")
                 self.hit_eof = self.tell() == len(self.getvalue())
                 return super(MyBytesIO, self).read(n)
             def seek(self, *args):
@@ -1811,102 +1847,23 @@
         self._test_partial_input("r:bz2")
 
 
-class LzmaMiscReadTest(MiscReadTest):
-    tarname = xzname
-    mode = "r:xz"
-class LzmaUstarReadTest(UstarReadTest):
-    tarname = xzname
-    mode = "r:xz"
-class LzmaStreamReadTest(StreamReadTest):
-    tarname = xzname
-    mode = "r|xz"
-class LzmaWriteTest(WriteTest):
-    mode = "w:xz"
-class LzmaStreamWriteTest(StreamWriteTest):
-    mode = "w|xz"
-
-
-def test_main():
+def setUpModule():
     support.unlink(TEMPDIR)
     os.makedirs(TEMPDIR)
 
-    tests = [
-        UstarReadTest,
-        MiscReadTest,
-        StreamReadTest,
-        DetectReadTest,
-        MemberReadTest,
-        GNUReadTest,
-        PaxReadTest,
-        WriteTest,
-        StreamWriteTest,
-        GNUWriteTest,
-        PaxWriteTest,
-        UstarUnicodeTest,
-        GNUUnicodeTest,
-        PAXUnicodeTest,
-        AppendTest,
-        LimitsTest,
-        MiscTest,
-        ContextManagerTest,
-    ]
-
-    if hasattr(os, "link"):
-        tests.append(HardlinkTest)
-    else:
-        tests.append(LinkEmulationTest)
-
     with open(tarname, "rb") as fobj:
         data = fobj.read()
 
-    if gzip:
-        # Create testtar.tar.gz and add gzip-specific tests.
-        support.unlink(gzipname)
-        with gzip.open(gzipname, "wb") as tar:
-            tar.write(data)
+    # Create compressed tarfiles.
+    for c in GzipTest, Bz2Test, LzmaTest:
+        if c.open:
+            support.unlink(c.tarname)
+            with c.open(c.tarname, "wb") as tar:
+                tar.write(data)
 
-        tests += [
-            GzipMiscReadTest,
-            GzipUstarReadTest,
-            GzipStreamReadTest,
-            GzipWriteTest,
-            GzipStreamWriteTest,
-        ]
-
-    if bz2:
-        # Create testtar.tar.bz2 and add bz2-specific tests.
-        support.unlink(bz2name)
-        with bz2.BZ2File(bz2name, "wb") as tar:
-            tar.write(data)
-
-        tests += [
-            Bz2MiscReadTest,
-            Bz2UstarReadTest,
-            Bz2StreamReadTest,
-            Bz2WriteTest,
-            Bz2StreamWriteTest,
-            Bz2PartialReadTest,
-        ]
-
-    if lzma:
-        # Create testtar.tar.xz and add lzma-specific tests.
-        support.unlink(xzname)
-        with lzma.LZMAFile(xzname, "w") as tar:
-            tar.write(data)
-
-        tests += [
-            LzmaMiscReadTest,
-            LzmaUstarReadTest,
-            LzmaStreamReadTest,
-            LzmaWriteTest,
-            LzmaStreamWriteTest,
-        ]
-
-    try:
-        support.run_unittest(*tests)
-    finally:
-        if os.path.exists(TEMPDIR):
-            shutil.rmtree(TEMPDIR)
+def tearDownModule():
+    if os.path.exists(TEMPDIR):
+        shutil.rmtree(TEMPDIR)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:11:41 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 17 Jun 2013 15:11:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318223=3A_Refactor_test=5Ftarfile=2E?=
Message-ID: <3bYt9T3DBFz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/5c0816e64aac
changeset:   84179:5c0816e64aac
parent:      84177:69b0b713865e
parent:      84178:4b2188b13dd2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Jun 17 16:11:06 2013 +0300
summary:
  Issue #18223: Refactor test_tarfile.

* Use mixins for generating tests for different compression types.
* Make test_tarfile discoverable.
* Use more special tests (i.e. assertEqual, assertIs) instead of assertTrue.
* Add explicit test skips instead of reporting skipped tests as passed.
* Wrap long lines.
* Correct a comment for test_hardlink_extraction1.
* Add support.requires_gzip.
* Replace ImportError by  ModuleNotFoundError.

and some other minor enhancements.

files:
  Lib/test/support.py      |    9 +-
  Lib/test/test_tarfile.py |  721 ++++++++++++--------------
  2 files changed, 347 insertions(+), 383 deletions(-)


diff --git a/Lib/test/support.py b/Lib/test/support.py
--- a/Lib/test/support.py
+++ b/Lib/test/support.py
@@ -43,6 +43,11 @@
     zlib = None
 
 try:
+    import gzip
+except ModuleNotFoundError:
+    gzip = None
+
+try:
     import bz2
 except ModuleNotFoundError:
     bz2 = None
@@ -71,7 +76,7 @@
     "TestHandler", "Matcher", "can_symlink", "skip_unless_symlink",
     "skip_unless_xattr", "import_fresh_module", "requires_zlib",
     "PIPE_MAX_SIZE", "failfast", "anticipate_failure", "run_with_tz",
-    "requires_bz2", "requires_lzma", "suppress_crash_popup",
+    "requires_gzip", "requires_bz2", "requires_lzma", "suppress_crash_popup",
     ]
 
 class Error(Exception):
@@ -590,6 +595,8 @@
 
 requires_zlib = unittest.skipUnless(zlib, 'requires zlib')
 
+requires_gzip = unittest.skipUnless(gzip, 'requires gzip')
+
 requires_bz2 = unittest.skipUnless(bz2, 'requires bz2')
 
 requires_lzma = unittest.skipUnless(lzma, 'requires lzma')
diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py
--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -2,9 +2,7 @@
 import os
 import io
 import shutil
-import io
 from hashlib import md5
-import errno
 
 import unittest
 import tarfile
@@ -14,16 +12,15 @@
 # Check for our compression modules.
 try:
     import gzip
-    gzip.GzipFile
-except (ImportError, AttributeError):
+except ModuleNotFoundError:
     gzip = None
 try:
     import bz2
-except ImportError:
+except ModuleNotFoundError:
     bz2 = None
 try:
     import lzma
-except ImportError:
+except ModuleNotFoundError:
     lzma = None
 
 def md5sum(data):
@@ -40,25 +37,55 @@
 md5_sparse = "a54fbc4ca4f4399a90e1b27164012fc6"
 
 
-class ReadTest(unittest.TestCase):
+class TarTest:
+    tarname = tarname
+    suffix = ''
+    open = io.FileIO
 
-    tarname = tarname
-    mode = "r:"
+    @property
+    def mode(self):
+        return self.prefix + self.suffix
+
+ at support.requires_gzip
+class GzipTest:
+    tarname = gzipname
+    suffix = 'gz'
+    open = gzip.GzipFile if gzip else None
+
+ at support.requires_bz2
+class Bz2Test:
+    tarname = bz2name
+    suffix = 'bz2'
+    open = bz2.BZ2File if bz2 else None
+
+ at support.requires_lzma
+class LzmaTest:
+    tarname = xzname
+    suffix = 'xz'
+    open = lzma.LZMAFile if lzma else None
+
+
+class ReadTest(TarTest):
+
+    prefix = "r:"
 
     def setUp(self):
-        self.tar = tarfile.open(self.tarname, mode=self.mode, encoding="iso8859-1")
+        self.tar = tarfile.open(self.tarname, mode=self.mode,
+                                encoding="iso8859-1")
 
     def tearDown(self):
         self.tar.close()
 
 
-class UstarReadTest(ReadTest):
+class UstarReadTest(ReadTest, unittest.TestCase):
 
     def test_fileobj_regular_file(self):
         tarinfo = self.tar.getmember("ustar/regtype")
         with self.tar.extractfile(tarinfo) as fobj:
             data = fobj.read()
-            self.assertTrue((len(data), md5sum(data)) == (tarinfo.size, md5_regtype),
+            self.assertEqual(len(data), tarinfo.size,
+                    "regular file extraction failed")
+            self.assertEqual(md5sum(data), md5_regtype,
                     "regular file extraction failed")
 
     def test_fileobj_readlines(self):
@@ -70,12 +97,13 @@
         with self.tar.extractfile(tarinfo) as fobj:
             fobj2 = io.TextIOWrapper(fobj)
             lines2 = fobj2.readlines()
-            self.assertTrue(lines1 == lines2,
+            self.assertEqual(lines1, lines2,
                     "fileobj.readlines() failed")
-            self.assertTrue(len(lines2) == 114,
+            self.assertEqual(len(lines2), 114,
                     "fileobj.readlines() failed")
-            self.assertTrue(lines2[83] ==
-                    "I will gladly admit that Python is not the fastest running scripting language.\n",
+            self.assertEqual(lines2[83],
+                    "I will gladly admit that Python is not the fastest "
+                    "running scripting language.\n",
                     "fileobj.readlines() failed")
 
     def test_fileobj_iter(self):
@@ -85,8 +113,8 @@
             lines1 = fobj1.readlines()
         with self.tar.extractfile(tarinfo) as fobj2:
             lines2 = list(io.TextIOWrapper(fobj2))
-            self.assertTrue(lines1 == lines2,
-                         "fileobj.__iter__() failed")
+            self.assertEqual(lines1, lines2,
+                    "fileobj.__iter__() failed")
 
     def test_fileobj_seek(self):
         self.tar.extract("ustar/regtype", TEMPDIR)
@@ -110,12 +138,12 @@
         self.assertEqual(2048, fobj.tell(),
                      "seek() to positive relative position failed")
         s = fobj.read(10)
-        self.assertTrue(s == data[2048:2058],
+        self.assertEqual(s, data[2048:2058],
                      "read() after seek failed")
         fobj.seek(0, 2)
         self.assertEqual(tarinfo.size, fobj.tell(),
                      "seek() to file's end failed")
-        self.assertTrue(fobj.read() == b"",
+        self.assertEqual(fobj.read(), b"",
                      "read() at file's end did not return empty string")
         fobj.seek(-tarinfo.size, 2)
         self.assertEqual(0, fobj.tell(),
@@ -124,13 +152,13 @@
         s1 = fobj.readlines()
         fobj.seek(512)
         s2 = fobj.readlines()
-        self.assertTrue(s1 == s2,
+        self.assertEqual(s1, s2,
                      "readlines() after seek failed")
         fobj.seek(0)
         self.assertEqual(len(fobj.readline()), fobj.tell(),
                      "tell() after readline() failed")
         fobj.seek(512)
-        self.assertTrue(len(fobj.readline()) + 512 == fobj.tell(),
+        self.assertEqual(len(fobj.readline()) + 512, fobj.tell(),
                      "tell() after seek() and readline() failed")
         fobj.seek(0)
         line = fobj.readline()
@@ -154,24 +182,36 @@
     # test link members each point to a regular member whose data is
     # supposed to be exported.
     def _test_fileobj_link(self, lnktype, regtype):
-        with self.tar.extractfile(lnktype) as a, self.tar.extractfile(regtype) as b:
+        with self.tar.extractfile(lnktype) as a, \
+             self.tar.extractfile(regtype) as b:
             self.assertEqual(a.name, b.name)
 
     def test_fileobj_link1(self):
         self._test_fileobj_link("ustar/lnktype", "ustar/regtype")
 
     def test_fileobj_link2(self):
-        self._test_fileobj_link("./ustar/linktest2/lnktype", "ustar/linktest1/regtype")
+        self._test_fileobj_link("./ustar/linktest2/lnktype",
+                                "ustar/linktest1/regtype")
 
     def test_fileobj_symlink1(self):
         self._test_fileobj_link("ustar/symtype", "ustar/regtype")
 
     def test_fileobj_symlink2(self):
-        self._test_fileobj_link("./ustar/linktest2/symtype", "ustar/linktest1/regtype")
+        self._test_fileobj_link("./ustar/linktest2/symtype",
+                                "ustar/linktest1/regtype")
 
     def test_issue14160(self):
         self._test_fileobj_link("symtype2", "ustar/regtype")
 
+class GzipUstarReadTest(GzipTest, UstarReadTest):
+    pass
+
+class Bz2UstarReadTest(Bz2Test, UstarReadTest):
+    pass
+
+class LzmaUstarReadTest(LzmaTest, UstarReadTest):
+    pass
+
 
 class CommonReadTest(ReadTest):
 
@@ -203,37 +243,24 @@
 
     def test_ignore_zeros(self):
         # Test TarFile's ignore_zeros option.
-        if self.mode.endswith(":gz"):
-            _open = gzip.GzipFile
-        elif self.mode.endswith(":bz2"):
-            _open = bz2.BZ2File
-        elif self.mode.endswith(":xz"):
-            _open = lzma.LZMAFile
-        else:
-            _open = io.FileIO
-
         for char in (b'\0', b'a'):
             # Test if EOFHeaderError ('\0') and InvalidHeaderError ('a')
             # are ignored correctly.
-            with _open(tmpname, "w") as fobj:
+            with self.open(tmpname, "w") as fobj:
                 fobj.write(char * 1024)
                 fobj.write(tarfile.TarInfo("foo").tobuf())
 
             tar = tarfile.open(tmpname, mode="r", ignore_zeros=True)
             try:
                 self.assertListEqual(tar.getnames(), ["foo"],
-                    "ignore_zeros=True should have skipped the %r-blocks" % char)
+                    "ignore_zeros=True should have skipped the %r-blocks" %
+                    char)
             finally:
                 tar.close()
 
 
-class MiscReadTest(CommonReadTest):
-
+class MiscReadTestBase(CommonReadTest):
     def test_no_name_argument(self):
-        if self.mode.endswith(("bz2", "xz")):
-            # BZ2File and LZMAFile have no name attribute.
-            self.skipTest("no name attribute")
-
         with open(self.tarname, "rb") as fobj:
             tar = tarfile.open(fileobj=fobj, mode=self.mode)
             self.assertEqual(tar.name, os.path.abspath(fobj.name))
@@ -269,16 +296,7 @@
             tar.close()
 
         # Open the testtar and seek to the offset of the second member.
-        if self.mode.endswith(":gz"):
-            _open = gzip.GzipFile
-        elif self.mode.endswith(":bz2"):
-            _open = bz2.BZ2File
-        elif self.mode.endswith(":xz"):
-            _open = lzma.LZMAFile
-        else:
-            _open = io.FileIO
-
-        with _open(self.tarname) as fobj:
+        with self.open(self.tarname) as fobj:
             fobj.seek(offset)
 
             # Test if the tarfile starts with the second member.
@@ -294,8 +312,6 @@
 
     def test_fail_comp(self):
         # For Gzip and Bz2 Tests: fail with a ReadError on an uncompressed file.
-        if self.mode == "r:":
-            return
         self.assertRaises(tarfile.ReadError, tarfile.open, tarname, self.mode)
         with open(tarname, "rb") as fobj:
             self.assertRaises(tarfile.ReadError, tarfile.open,
@@ -306,7 +322,7 @@
         # Old V7 tars create directory members using an AREGTYPE
         # header with a "/" appended to the filename field.
         tarinfo = self.tar.getmember("misc/dirtype-old-v7")
-        self.assertTrue(tarinfo.type == tarfile.DIRTYPE,
+        self.assertEqual(tarinfo.type, tarfile.DIRTYPE,
                 "v7 dirtype failed")
 
     def test_xstar_type(self):
@@ -320,15 +336,15 @@
 
     def test_check_members(self):
         for tarinfo in self.tar:
-            self.assertTrue(int(tarinfo.mtime) == 0o7606136617,
+            self.assertEqual(int(tarinfo.mtime), 0o7606136617,
                     "wrong mtime for %s" % tarinfo.name)
             if not tarinfo.name.startswith("ustar/"):
                 continue
-            self.assertTrue(tarinfo.uname == "tarfile",
+            self.assertEqual(tarinfo.uname, "tarfile",
                     "wrong uname for %s" % tarinfo.name)
 
     def test_find_members(self):
-        self.assertTrue(self.tar.getmembers()[-1].name == "misc/eof",
+        self.assertEqual(self.tar.getmembers()[-1].name, "misc/eof",
                 "could not find all members")
 
     @unittest.skipUnless(hasattr(os, "link"),
@@ -365,7 +381,8 @@
                 path = os.path.join(DIR, tarinfo.name)
                 if sys.platform != "win32":
                     # Win32 has no support for fine grained permissions.
-                    self.assertEqual(tarinfo.mode & 0o777, os.stat(path).st_mode & 0o777)
+                    self.assertEqual(tarinfo.mode & 0o777,
+                                     os.stat(path).st_mode & 0o777)
                 def format_mtime(mtime):
                     if isinstance(mtime, float):
                         return "{} ({})".format(mtime, mtime.hex())
@@ -423,10 +440,28 @@
                 self.assertEqual(m1.offset, m2.offset)
                 self.assertEqual(m1.get_info(), m2.get_info())
 
+class MiscReadTest(MiscReadTestBase, unittest.TestCase):
+    test_fail_comp = None
 
-class StreamReadTest(CommonReadTest):
+class GzipMiscReadTest(GzipTest, MiscReadTestBase, unittest.TestCase):
+    def test_non_existent_targz_file(self):
+        # Test for issue11513: prevent non-existent gzipped tarfiles raising
+        # multiple exceptions.
+        with self.assertRaisesRegex(FileNotFoundError, "xxx"):
+            tarfile.open("xxx", self.mode)
 
-    mode="r|"
+class Bz2MiscReadTest(Bz2Test, MiscReadTestBase, unittest.TestCase):
+    def test_no_name_argument(self):
+        self.skipTest("BZ2File have no name attribute")
+
+class LzmaMiscReadTest(LzmaTest, MiscReadTestBase, unittest.TestCase):
+    def test_no_name_argument(self):
+        self.skipTest("LZMAFile have no name attribute")
+
+
+class StreamReadTest(CommonReadTest, unittest.TestCase):
+
+    prefix="r|"
 
     def test_read_through(self):
         # Issue #11224: A poorly designed _FileInFile.read() method
@@ -439,7 +474,8 @@
                     try:
                         buf = fobj.read(512)
                     except tarfile.StreamError:
-                        self.fail("simple read-through using TarFile.extractfile() failed")
+                        self.fail("simple read-through using "
+                                  "TarFile.extractfile() failed")
                     if not buf:
                         break
 
@@ -447,7 +483,9 @@
         tarinfo = self.tar.next() # get "regtype" (can't use getmember)
         with self.tar.extractfile(tarinfo) as fobj:
             data = fobj.read()
-        self.assertTrue((len(data), md5sum(data)) == (tarinfo.size, md5_regtype),
+        self.assertEqual(len(data), tarinfo.size,
+                "regular file extraction failed")
+        self.assertEqual(md5sum(data), md5_regtype,
                 "regular file extraction failed")
 
     def test_provoke_stream_error(self):
@@ -465,24 +503,34 @@
                 t2 = tar2.next()
                 if t1 is None:
                     break
-                self.assertTrue(t2 is not None, "stream.next() failed.")
+                self.assertIsNotNone(t2, "stream.next() failed.")
 
                 if t2.islnk() or t2.issym():
-                    self.assertRaises(tarfile.StreamError, tar2.extractfile, t2)
+                    with self.assertRaises(tarfile.StreamError):
+                        tar2.extractfile(t2)
                     continue
 
                 v1 = tar1.extractfile(t1)
                 v2 = tar2.extractfile(t2)
                 if v1 is None:
                     continue
-                self.assertTrue(v2 is not None, "stream.extractfile() failed")
-                self.assertEqual(v1.read(), v2.read(), "stream extraction failed")
+                self.assertIsNotNone(v2, "stream.extractfile() failed")
+                self.assertEqual(v1.read(), v2.read(),
+                        "stream extraction failed")
         finally:
             tar1.close()
 
+class GzipStreamReadTest(GzipTest, StreamReadTest):
+    pass
 
-class DetectReadTest(unittest.TestCase):
+class Bz2StreamReadTest(Bz2Test, StreamReadTest):
+    pass
 
+class LzmaStreamReadTest(LzmaTest, StreamReadTest):
+    pass
+
+
+class DetectReadTest(TarTest, unittest.TestCase):
     def _testfunc_file(self, name, mode):
         try:
             tar = tarfile.open(name, mode)
@@ -501,47 +549,20 @@
             tar.close()
 
     def _test_modes(self, testfunc):
-        testfunc(tarname, "r")
-        testfunc(tarname, "r:")
-        testfunc(tarname, "r:*")
-        testfunc(tarname, "r|")
-        testfunc(tarname, "r|*")
-
-        if gzip:
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:gz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|gz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, gzipname, mode="r:")
-            self.assertRaises(tarfile.ReadError, tarfile.open, gzipname, mode="r|")
-
-            testfunc(gzipname, "r")
-            testfunc(gzipname, "r:*")
-            testfunc(gzipname, "r:gz")
-            testfunc(gzipname, "r|*")
-            testfunc(gzipname, "r|gz")
-
-        if bz2:
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:bz2")
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|bz2")
-            self.assertRaises(tarfile.ReadError, tarfile.open, bz2name, mode="r:")
-            self.assertRaises(tarfile.ReadError, tarfile.open, bz2name, mode="r|")
-
-            testfunc(bz2name, "r")
-            testfunc(bz2name, "r:*")
-            testfunc(bz2name, "r:bz2")
-            testfunc(bz2name, "r|*")
-            testfunc(bz2name, "r|bz2")
-
-        if lzma:
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r:xz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, tarname, mode="r|xz")
-            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r:")
-            self.assertRaises(tarfile.ReadError, tarfile.open, xzname, mode="r|")
-
-            testfunc(xzname, "r")
-            testfunc(xzname, "r:*")
-            testfunc(xzname, "r:xz")
-            testfunc(xzname, "r|*")
-            testfunc(xzname, "r|xz")
+        if self.suffix:
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(tarname, mode="r:" + self.suffix)
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(tarname, mode="r|" + self.suffix)
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(self.tarname, mode="r:")
+            with self.assertRaises(tarfile.ReadError):
+                tarfile.open(self.tarname, mode="r|")
+        testfunc(self.tarname, "r")
+        testfunc(self.tarname, "r:" + self.suffix)
+        testfunc(self.tarname, "r:*")
+        testfunc(self.tarname, "r|" + self.suffix)
+        testfunc(self.tarname, "r|*")
 
     def test_detect_file(self):
         self._test_modes(self._testfunc_file)
@@ -549,14 +570,15 @@
     def test_detect_fileobj(self):
         self._test_modes(self._testfunc_fileobj)
 
+class GzipDetectReadTest(GzipTest, DetectReadTest):
+    pass
+
+class Bz2DetectReadTest(Bz2Test, DetectReadTest):
     def test_detect_stream_bz2(self):
         # Originally, tarfile's stream detection looked for the string
         # "BZh91" at the start of the file. This is incorrect because
         # the '9' represents the blocksize (900kB). If the file was
         # compressed using another blocksize autodetection fails.
-        if not bz2:
-            return
-
         with open(tarname, "rb") as fobj:
             data = fobj.read()
 
@@ -566,13 +588,17 @@
 
         self._testfunc_file(tmpname, "r|*")
 
+class LzmaDetectReadTest(LzmaTest, DetectReadTest):
+    pass
 
-class MemberReadTest(ReadTest):
+
+class MemberReadTest(ReadTest, unittest.TestCase):
 
     def _test_member(self, tarinfo, chksum=None, **kwargs):
         if chksum is not None:
-            self.assertTrue(md5sum(self.tar.extractfile(tarinfo).read()) == chksum,
-                    "wrong md5sum for %s" % tarinfo.name)
+            with self.tar.extractfile(tarinfo) as f:
+                self.assertEqual(md5sum(f.read()), chksum,
+                        "wrong md5sum for %s" % tarinfo.name)
 
         kwargs["mtime"] = 0o7606136617
         kwargs["uid"] = 1000
@@ -582,7 +608,7 @@
             kwargs["uname"] = "tarfile"
             kwargs["gname"] = "tarfile"
         for k, v in kwargs.items():
-            self.assertTrue(getattr(tarinfo, k) == v,
+            self.assertEqual(getattr(tarinfo, k), v,
                     "wrong value in %s field of %s" % (k, tarinfo.name))
 
     def test_find_regtype(self):
@@ -642,7 +668,8 @@
         self._test_member(tarinfo, size=86016, chksum=md5_sparse)
 
     def test_find_umlauts(self):
-        tarinfo = self.tar.getmember("ustar/umlauts-\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+        tarinfo = self.tar.getmember("ustar/umlauts-"
+                                     "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
         self._test_member(tarinfo, size=7011, chksum=md5_regtype)
 
     def test_find_ustar_longname(self):
@@ -655,12 +682,14 @@
 
     def test_find_pax_umlauts(self):
         self.tar.close()
-        self.tar = tarfile.open(self.tarname, mode=self.mode, encoding="iso8859-1")
-        tarinfo = self.tar.getmember("pax/umlauts-\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+        self.tar = tarfile.open(self.tarname, mode=self.mode,
+                                encoding="iso8859-1")
+        tarinfo = self.tar.getmember("pax/umlauts-"
+                                     "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
         self._test_member(tarinfo, size=7011, chksum=md5_regtype)
 
 
-class LongnameTest(ReadTest):
+class LongnameTest:
 
     def test_read_longname(self):
         # Test reading of longname (bug #1471427).
@@ -669,7 +698,8 @@
             tarinfo = self.tar.getmember(longname)
         except KeyError:
             self.fail("longname not found")
-        self.assertTrue(tarinfo.type != tarfile.DIRTYPE, "read longname as dirtype")
+        self.assertNotEqual(tarinfo.type, tarfile.DIRTYPE,
+                "read longname as dirtype")
 
     def test_read_longlink(self):
         longname = self.subdir + "/" + "123/" * 125 + "longname"
@@ -678,7 +708,7 @@
             tarinfo = self.tar.getmember(longlink)
         except KeyError:
             self.fail("longlink not found")
-        self.assertTrue(tarinfo.linkname == longname, "linkname wrong")
+        self.assertEqual(tarinfo.linkname, longname, "linkname wrong")
 
     def test_truncated_longname(self):
         longname = self.subdir + "/" + "123/" * 125 + "longname"
@@ -686,7 +716,8 @@
         offset = tarinfo.offset
         self.tar.fileobj.seek(offset)
         fobj = io.BytesIO(self.tar.fileobj.read(3 * 512))
-        self.assertRaises(tarfile.ReadError, tarfile.open, name="foo.tar", fileobj=fobj)
+        with self.assertRaises(tarfile.ReadError):
+            tarfile.open(name="foo.tar", fileobj=fobj)
 
     def test_header_offset(self):
         # Test if the start offset of the TarInfo object includes
@@ -695,11 +726,12 @@
         offset = self.tar.getmember(longname).offset
         with open(tarname, "rb") as fobj:
             fobj.seek(offset)
-            tarinfo = tarfile.TarInfo.frombuf(fobj.read(512), "iso8859-1", "strict")
+            tarinfo = tarfile.TarInfo.frombuf(fobj.read(512),
+                                              "iso8859-1", "strict")
             self.assertEqual(tarinfo.type, self.longnametype)
 
 
-class GNUReadTest(LongnameTest):
+class GNUReadTest(LongnameTest, ReadTest, unittest.TestCase):
 
     subdir = "gnu"
     longnametype = tarfile.GNUTYPE_LONGNAME
@@ -721,7 +753,7 @@
 
         if self._fs_supports_holes():
             s = os.stat(filename)
-            self.assertTrue(s.st_blocks * 512 < s.st_size)
+            self.assertLess(s.st_blocks * 512, s.st_size)
 
     def test_sparse_file_old(self):
         self._test_sparse_file("gnu/sparse")
@@ -753,7 +785,7 @@
             return False
 
 
-class PaxReadTest(LongnameTest):
+class PaxReadTest(LongnameTest, ReadTest, unittest.TestCase):
 
     subdir = "pax"
     longnametype = tarfile.XHDTYPE
@@ -764,17 +796,20 @@
             tarinfo = tar.getmember("pax/regtype1")
             self.assertEqual(tarinfo.uname, "foo")
             self.assertEqual(tarinfo.gname, "bar")
-            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"),
+                             "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
 
             tarinfo = tar.getmember("pax/regtype2")
             self.assertEqual(tarinfo.uname, "")
             self.assertEqual(tarinfo.gname, "bar")
-            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"),
+                             "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
 
             tarinfo = tar.getmember("pax/regtype3")
             self.assertEqual(tarinfo.uname, "tarfile")
             self.assertEqual(tarinfo.gname, "tarfile")
-            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"), "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
+            self.assertEqual(tarinfo.pax_headers.get("VENDOR.umlauts"),
+                             "\xc4\xd6\xdc\xe4\xf6\xfc\xdf")
         finally:
             tar.close()
 
@@ -794,7 +829,7 @@
             tar.close()
 
 
-class WriteTestBase(unittest.TestCase):
+class WriteTestBase(TarTest):
     # Put all write tests in here that are supposed to be tested
     # in all possible mode combinations.
 
@@ -803,12 +838,12 @@
         tar = tarfile.open(fileobj=fobj, mode=self.mode)
         tar.addfile(tarfile.TarInfo("foo"))
         tar.close()
-        self.assertTrue(fobj.closed is False, "external fileobjs must never closed")
+        self.assertFalse(fobj.closed, "external fileobjs must never closed")
 
 
-class WriteTest(WriteTestBase):
+class WriteTest(WriteTestBase, unittest.TestCase):
 
-    mode = "w:"
+    prefix = "w:"
 
     def test_100_char_name(self):
         # The name field in a tar header stores strings of at most 100 chars.
@@ -825,7 +860,7 @@
 
         tar = tarfile.open(tmpname)
         try:
-            self.assertTrue(tar.getnames()[0] == name,
+            self.assertEqual(tar.getnames()[0], name,
                     "failed to store 100 char filename")
         finally:
             tar.close()
@@ -840,7 +875,7 @@
             tar.add(path)
         finally:
             tar.close()
-        self.assertTrue(os.path.getsize(tmpname) > 0,
+        self.assertGreater(os.path.getsize(tmpname), 0,
                 "tarfile is empty")
 
     # The test_*_size tests test for bug #1167128.
@@ -873,25 +908,26 @@
         finally:
             os.rmdir(path)
 
+    @unittest.skipUnless(hasattr(os, "link"),
+                         "Missing hardlink implementation")
     def test_link_size(self):
-        if hasattr(os, "link"):
-            link = os.path.join(TEMPDIR, "link")
-            target = os.path.join(TEMPDIR, "link_target")
-            with open(target, "wb") as fobj:
-                fobj.write(b"aaa")
-            os.link(target, link)
+        link = os.path.join(TEMPDIR, "link")
+        target = os.path.join(TEMPDIR, "link_target")
+        with open(target, "wb") as fobj:
+            fobj.write(b"aaa")
+        os.link(target, link)
+        try:
+            tar = tarfile.open(tmpname, self.mode)
             try:
-                tar = tarfile.open(tmpname, self.mode)
-                try:
-                    # Record the link target in the inodes list.
-                    tar.gettarinfo(target)
-                    tarinfo = tar.gettarinfo(link)
-                    self.assertEqual(tarinfo.size, 0)
-                finally:
-                    tar.close()
+                # Record the link target in the inodes list.
+                tar.gettarinfo(target)
+                tarinfo = tar.gettarinfo(link)
+                self.assertEqual(tarinfo.size, 0)
             finally:
-                os.remove(target)
-                os.remove(link)
+                tar.close()
+        finally:
+            os.remove(target)
+            os.remove(link)
 
     @support.skip_unless_symlink
     def test_symlink_size(self):
@@ -912,15 +948,18 @@
         dstname = os.path.abspath(tmpname)
         tar = tarfile.open(tmpname, self.mode)
         try:
-            self.assertTrue(tar.name == dstname, "archive name must be absolute")
+            self.assertEqual(tar.name, dstname,
+                    "archive name must be absolute")
             tar.add(dstname)
-            self.assertTrue(tar.getnames() == [], "added the archive to itself")
+            self.assertEqual(tar.getnames(), [],
+                    "added the archive to itself")
 
             cwd = os.getcwd()
             os.chdir(TEMPDIR)
             tar.add(dstname)
             os.chdir(cwd)
-            self.assertTrue(tar.getnames() == [], "added the archive to itself")
+            self.assertEqual(tar.getnames(), [],
+                    "added the archive to itself")
         finally:
             tar.close()
 
@@ -1087,48 +1126,49 @@
             tar = tarfile.open(tmpname, "r")
             try:
                 for t in tar:
-                    self.assertTrue(t.name == "." or t.name.startswith("./"))
+                    if t.name != ".":
+                        self.assertTrue(t.name.startswith("./"), t.name)
             finally:
                 tar.close()
         finally:
             os.chdir(cwd)
 
+class GzipWriteTest(GzipTest, WriteTest):
+    pass
 
-class StreamWriteTest(WriteTestBase):
+class Bz2WriteTest(Bz2Test, WriteTest):
+    pass
 
-    mode = "w|"
+class LzmaWriteTest(LzmaTest, WriteTest):
+    pass
+
+
+class StreamWriteTest(WriteTestBase, unittest.TestCase):
+
+    prefix = "w|"
+    decompressor = None
 
     def test_stream_padding(self):
         # Test for bug #1543303.
         tar = tarfile.open(tmpname, self.mode)
         tar.close()
-
-        if self.mode.endswith("gz"):
-            with gzip.GzipFile(tmpname) as fobj:
-                data = fobj.read()
-        elif self.mode.endswith("bz2"):
-            dec = bz2.BZ2Decompressor()
+        if self.decompressor:
+            dec = self.decompressor()
             with open(tmpname, "rb") as fobj:
                 data = fobj.read()
             data = dec.decompress(data)
-            self.assertTrue(len(dec.unused_data) == 0,
-                    "found trailing data")
-        elif self.mode.endswith("xz"):
-            with lzma.LZMAFile(tmpname) as fobj:
+            self.assertFalse(dec.unused_data, "found trailing data")
+        else:
+            with self.open(tmpname) as fobj:
                 data = fobj.read()
-        else:
-            with open(tmpname, "rb") as fobj:
-                data = fobj.read()
+        self.assertEqual(data.count(b"\0"), tarfile.RECORDSIZE,
+                        "incorrect zero padding")
 
-        self.assertTrue(data.count(b"\0") == tarfile.RECORDSIZE,
-                         "incorrect zero padding")
-
+    @unittest.skipUnless(sys.platform != "win32" and hasattr(os, "umask"),
+                         "Missing umask implementation")
     def test_file_mode(self):
         # Test for issue #8464: Create files with correct
         # permissions.
-        if sys.platform == "win32" or not hasattr(os, "umask"):
-            return
-
         if os.path.exists(tmpname):
             os.remove(tmpname)
 
@@ -1141,15 +1181,22 @@
         finally:
             os.umask(original_umask)
 
+class GzipStreamWriteTest(GzipTest, StreamWriteTest):
+    pass
+
+class Bz2StreamWriteTest(Bz2Test, StreamWriteTest):
+    decompressor = bz2.BZ2Decompressor if bz2 else None
+
+class LzmaStreamWriteTest(LzmaTest, StreamWriteTest):
+    decompressor = lzma.LZMADecompressor if lzma else None
+
 
 class GNUWriteTest(unittest.TestCase):
     # This testcase checks for correct creation of GNU Longname
     # and Longlink extended headers (cp. bug #812325).
 
     def _length(self, s):
-        blocks, remainder = divmod(len(s) + 1, 512)
-        if remainder:
-            blocks += 1
+        blocks = len(s) // 512 + 1
         return blocks * 512
 
     def _calc_size(self, name, link=None):
@@ -1179,7 +1226,7 @@
 
             v1 = self._calc_size(name, link)
             v2 = tar.offset
-            self.assertTrue(v1 == v2, "GNU longname/longlink creation failed")
+            self.assertEqual(v1, v2, "GNU longname/longlink creation failed")
         finally:
             tar.close()
 
@@ -1226,6 +1273,7 @@
                    ("longlnk/" * 127) + "longlink_")
 
 
+ at unittest.skipUnless(hasattr(os, "link"), "Missing hardlink implementation")
 class HardlinkTest(unittest.TestCase):
     # Test the creation of LNKTYPE (hardlink) members in an archive.
 
@@ -1250,18 +1298,18 @@
         # The same name will be added as a REGTYPE every
         # time regardless of st_nlink.
         tarinfo = self.tar.gettarinfo(self.foo)
-        self.assertTrue(tarinfo.type == tarfile.REGTYPE,
+        self.assertEqual(tarinfo.type, tarfile.REGTYPE,
                 "add file as regular failed")
 
     def test_add_hardlink(self):
         tarinfo = self.tar.gettarinfo(self.bar)
-        self.assertTrue(tarinfo.type == tarfile.LNKTYPE,
+        self.assertEqual(tarinfo.type, tarfile.LNKTYPE,
                 "add file as hardlink failed")
 
     def test_dereference_hardlink(self):
         self.tar.dereference = True
         tarinfo = self.tar.gettarinfo(self.bar)
-        self.assertTrue(tarinfo.type == tarfile.REGTYPE,
+        self.assertEqual(tarinfo.type, tarfile.REGTYPE,
                 "dereferencing hardlink failed")
 
 
@@ -1284,10 +1332,10 @@
         try:
             if link:
                 l = tar.getmembers()[0].linkname
-                self.assertTrue(link == l, "PAX longlink creation failed")
+                self.assertEqual(link, l, "PAX longlink creation failed")
             else:
                 n = tar.getmembers()[0].name
-                self.assertTrue(name == n, "PAX longname creation failed")
+                self.assertEqual(name, n, "PAX longname creation failed")
         finally:
             tar.close()
 
@@ -1313,8 +1361,8 @@
             self.assertEqual(tar.getmembers()[0].pax_headers, pax_headers)
             # Test if all the fields are strings.
             for key, val in tar.pax_headers.items():
-                self.assertTrue(type(key) is not bytes)
-                self.assertTrue(type(val) is not bytes)
+                self.assertIsNot(type(key), bytes)
+                self.assertIsNot(type(val), bytes)
                 if key in tarfile.PAX_NUMBER_FIELDS:
                     try:
                         tarfile.PAX_NUMBER_FIELDS[key](val)
@@ -1328,7 +1376,8 @@
         # TarInfo.
         pax_headers = {"path": "foo", "uid": "123"}
 
-        tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT, encoding="iso8859-1")
+        tar = tarfile.open(tmpname, "w", format=tarfile.PAX_FORMAT,
+                           encoding="iso8859-1")
         try:
             t = tarfile.TarInfo()
             t.name = "\xe4\xf6\xfc" # non-ASCII
@@ -1362,7 +1411,8 @@
         self._test_unicode_filename("utf-8")
 
     def _test_unicode_filename(self, encoding):
-        tar = tarfile.open(tmpname, "w", format=self.format, encoding=encoding, errors="strict")
+        tar = tarfile.open(tmpname, "w", format=self.format,
+                           encoding=encoding, errors="strict")
         try:
             name = "\xe4\xf6\xfc"
             tar.addfile(tarfile.TarInfo(name))
@@ -1376,11 +1426,8 @@
             tar.close()
 
     def test_unicode_filename_error(self):
-        if self.format == tarfile.PAX_FORMAT:
-            # PAX_FORMAT ignores encoding in write mode.
-            return
-
-        tar = tarfile.open(tmpname, "w", format=self.format, encoding="ascii", errors="strict")
+        tar = tarfile.open(tmpname, "w", format=self.format,
+                           encoding="ascii", errors="strict")
         try:
             tarinfo = tarfile.TarInfo()
 
@@ -1394,13 +1441,14 @@
             tar.close()
 
     def test_unicode_argument(self):
-        tar = tarfile.open(tarname, "r", encoding="iso8859-1", errors="strict")
+        tar = tarfile.open(tarname, "r",
+                           encoding="iso8859-1", errors="strict")
         try:
             for t in tar:
-                self.assertTrue(type(t.name) is str)
-                self.assertTrue(type(t.linkname) is str)
-                self.assertTrue(type(t.uname) is str)
-                self.assertTrue(type(t.gname) is str)
+                self.assertIs(type(t.name), str)
+                self.assertIs(type(t.linkname), str)
+                self.assertIs(type(t.uname), str)
+                self.assertIs(type(t.gname), str)
         finally:
             tar.close()
 
@@ -1409,7 +1457,8 @@
         t.uname = "\xe4\xf6\xfc"
         t.gname = "\xe4\xf6\xfc"
 
-        tar = tarfile.open(tmpname, mode="w", format=self.format, encoding="iso8859-1")
+        tar = tarfile.open(tmpname, mode="w", format=self.format,
+                           encoding="iso8859-1")
         try:
             tar.addfile(t)
         finally:
@@ -1438,9 +1487,11 @@
     def test_bad_pax_header(self):
         # Test for issue #8633. GNU tar <= 1.23 creates raw binary fields
         # without a hdrcharset=BINARY header.
-        for encoding, name in (("utf-8", "pax/bad-pax-\udce4\udcf6\udcfc"),
+        for encoding, name in (
+                ("utf-8", "pax/bad-pax-\udce4\udcf6\udcfc"),
                 ("iso8859-1", "pax/bad-pax-\xe4\xf6\xfc"),):
-            with tarfile.open(tarname, encoding=encoding, errors="surrogateescape") as tar:
+            with tarfile.open(tarname, encoding=encoding,
+                              errors="surrogateescape") as tar:
                 try:
                     t = tar.getmember(name)
                 except KeyError:
@@ -1451,18 +1502,23 @@
 
     format = tarfile.PAX_FORMAT
 
+    # PAX_FORMAT ignores encoding in write mode.
+    test_unicode_filename_error = None
+
     def test_binary_header(self):
         # Test a POSIX.1-2008 compatible header with a hdrcharset=BINARY field.
-        for encoding, name in (("utf-8", "pax/hdrcharset-\udce4\udcf6\udcfc"),
+        for encoding, name in (
+                ("utf-8", "pax/hdrcharset-\udce4\udcf6\udcfc"),
                 ("iso8859-1", "pax/hdrcharset-\xe4\xf6\xfc"),):
-            with tarfile.open(tarname, encoding=encoding, errors="surrogateescape") as tar:
+            with tarfile.open(tarname, encoding=encoding,
+                              errors="surrogateescape") as tar:
                 try:
                     t = tar.getmember(name)
                 except KeyError:
                     self.fail("unable to read POSIX.1-2008 binary header")
 
 
-class AppendTest(unittest.TestCase):
+class AppendTestBase:
     # Test append mode (cp. patch #1652681).
 
     def setUp(self):
@@ -1470,10 +1526,6 @@
         if os.path.exists(self.tarname):
             os.remove(self.tarname)
 
-    def _add_testfile(self, fileobj=None):
-        with tarfile.open(self.tarname, "a", fileobj=fileobj) as tar:
-            tar.addfile(tarfile.TarInfo("bar"))
-
     def _create_testtar(self, mode="w:"):
         with tarfile.open(tarname, encoding="iso8859-1") as src:
             t = src.getmember("ustar/regtype")
@@ -1482,6 +1534,17 @@
                 with tarfile.open(self.tarname, mode) as tar:
                     tar.addfile(t, f)
 
+    def test_append_compressed(self):
+        self._create_testtar("w:" + self.suffix)
+        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
+
+class AppendTest(AppendTestBase, unittest.TestCase):
+    test_append_compressed = None
+
+    def _add_testfile(self, fileobj=None):
+        with tarfile.open(self.tarname, "a", fileobj=fileobj) as tar:
+            tar.addfile(tarfile.TarInfo("bar"))
+
     def _test(self, names=["bar"], fileobj=None):
         with tarfile.open(self.tarname, fileobj=fileobj) as tar:
             self.assertEqual(tar.getnames(), names)
@@ -1515,24 +1578,6 @@
         self._add_testfile()
         self._test(names=["foo", "bar"])
 
-    def test_append_gz(self):
-        if gzip is None:
-            return
-        self._create_testtar("w:gz")
-        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
-
-    def test_append_bz2(self):
-        if bz2 is None:
-            return
-        self._create_testtar("w:bz2")
-        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
-
-    def test_append_lzma(self):
-        if lzma is None:
-            self.skipTest("lzma module not available")
-        self._create_testtar("w:xz")
-        self.assertRaises(tarfile.ReadError, tarfile.open, tmpname, "a")
-
     # Append mode is supposed to fail if the tarfile to append to
     # does not end with a zero block.
     def _test_error(self, data):
@@ -1557,6 +1602,15 @@
     def test_invalid(self):
         self._test_error(b"a" * 512)
 
+class GzipAppendTest(GzipTest, AppendTestBase, unittest.TestCase):
+    pass
+
+class Bz2AppendTest(Bz2Test, AppendTestBase, unittest.TestCase):
+    pass
+
+class LzmaAppendTest(LzmaTest, AppendTestBase, unittest.TestCase):
+    pass
+
 
 class LimitsTest(unittest.TestCase):
 
@@ -1620,36 +1674,54 @@
 class MiscTest(unittest.TestCase):
 
     def test_char_fields(self):
-        self.assertEqual(tarfile.stn("foo", 8, "ascii", "strict"), b"foo\0\0\0\0\0")
-        self.assertEqual(tarfile.stn("foobar", 3, "ascii", "strict"), b"foo")
-        self.assertEqual(tarfile.nts(b"foo\0\0\0\0\0", "ascii", "strict"), "foo")
-        self.assertEqual(tarfile.nts(b"foo\0bar\0", "ascii", "strict"), "foo")
+        self.assertEqual(tarfile.stn("foo", 8, "ascii", "strict"),
+                         b"foo\0\0\0\0\0")
+        self.assertEqual(tarfile.stn("foobar", 3, "ascii", "strict"),
+                         b"foo")
+        self.assertEqual(tarfile.nts(b"foo\0\0\0\0\0", "ascii", "strict"),
+                         "foo")
+        self.assertEqual(tarfile.nts(b"foo\0bar\0", "ascii", "strict"),
+                         "foo")
 
     def test_read_number_fields(self):
         # Issue 13158: Test if GNU tar specific base-256 number fields
         # are decoded correctly.
         self.assertEqual(tarfile.nti(b"0000001\x00"), 1)
         self.assertEqual(tarfile.nti(b"7777777\x00"), 0o7777777)
-        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\x00\x20\x00\x00"), 0o10000000)
-        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\xff\xff\xff\xff"), 0xffffffff)
-        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\xff"), -1)
-        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\x9c"), -100)
-        self.assertEqual(tarfile.nti(b"\xff\x00\x00\x00\x00\x00\x00\x00"), -0x100000000000000)
+        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\x00\x20\x00\x00"),
+                         0o10000000)
+        self.assertEqual(tarfile.nti(b"\x80\x00\x00\x00\xff\xff\xff\xff"),
+                         0xffffffff)
+        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\xff"),
+                         -1)
+        self.assertEqual(tarfile.nti(b"\xff\xff\xff\xff\xff\xff\xff\x9c"),
+                         -100)
+        self.assertEqual(tarfile.nti(b"\xff\x00\x00\x00\x00\x00\x00\x00"),
+                         -0x100000000000000)
 
     def test_write_number_fields(self):
         self.assertEqual(tarfile.itn(1), b"0000001\x00")
         self.assertEqual(tarfile.itn(0o7777777), b"7777777\x00")
-        self.assertEqual(tarfile.itn(0o10000000), b"\x80\x00\x00\x00\x00\x20\x00\x00")
-        self.assertEqual(tarfile.itn(0xffffffff), b"\x80\x00\x00\x00\xff\xff\xff\xff")
-        self.assertEqual(tarfile.itn(-1), b"\xff\xff\xff\xff\xff\xff\xff\xff")
-        self.assertEqual(tarfile.itn(-100), b"\xff\xff\xff\xff\xff\xff\xff\x9c")
-        self.assertEqual(tarfile.itn(-0x100000000000000), b"\xff\x00\x00\x00\x00\x00\x00\x00")
+        self.assertEqual(tarfile.itn(0o10000000),
+                         b"\x80\x00\x00\x00\x00\x20\x00\x00")
+        self.assertEqual(tarfile.itn(0xffffffff),
+                         b"\x80\x00\x00\x00\xff\xff\xff\xff")
+        self.assertEqual(tarfile.itn(-1),
+                         b"\xff\xff\xff\xff\xff\xff\xff\xff")
+        self.assertEqual(tarfile.itn(-100),
+                         b"\xff\xff\xff\xff\xff\xff\xff\x9c")
+        self.assertEqual(tarfile.itn(-0x100000000000000),
+                         b"\xff\x00\x00\x00\x00\x00\x00\x00")
 
     def test_number_field_limits(self):
-        self.assertRaises(ValueError, tarfile.itn, -1, 8, tarfile.USTAR_FORMAT)
-        self.assertRaises(ValueError, tarfile.itn, 0o10000000, 8, tarfile.USTAR_FORMAT)
-        self.assertRaises(ValueError, tarfile.itn, -0x10000000001, 6, tarfile.GNU_FORMAT)
-        self.assertRaises(ValueError, tarfile.itn, 0x10000000000, 6, tarfile.GNU_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(-1, 8, tarfile.USTAR_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(0o10000000, 8, tarfile.USTAR_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(-0x10000000001, 6, tarfile.GNU_FORMAT)
+        with self.assertRaises(ValueError):
+            tarfile.itn(0x10000000000, 6, tarfile.GNU_FORMAT)
 
 
 class ContextManagerTest(unittest.TestCase):
@@ -1710,19 +1782,19 @@
             self.assertTrue(tar.closed, "context manager failed")
 
 
-class LinkEmulationTest(ReadTest):
+ at unittest.skipIf(hasattr(os, "link"), "requires os.link to be missing")
+class LinkEmulationTest(ReadTest, unittest.TestCase):
 
     # Test for issue #8741 regression. On platforms that do not support
-    # symbolic or hard links tarfile tries to extract these types of members as
-    # the regular files they point to.
+    # symbolic or hard links tarfile tries to extract these types of members
+    # as the regular files they point to.
     def _test_link_extraction(self, name):
         self.tar.extract(name, TEMPDIR)
-        data = open(os.path.join(TEMPDIR, name), "rb").read()
+        with open(os.path.join(TEMPDIR, name), "rb") as f:
+            data = f.read()
         self.assertEqual(md5sum(data), md5_regtype)
 
-    # When 8879 gets fixed, this will need to change. Currently on Windows
-    # we have os.path.islink but no os.link, so these tests fail without the
-    # following skip until link is completed.
+    # See issues #1578269, #8879, and #17689 for some history on these skips
     @unittest.skipIf(hasattr(os.path, "islink"),
                      "Skip emulation - has os.path.islink but not os.link")
     def test_hardlink_extraction1(self):
@@ -1744,44 +1816,7 @@
         self._test_link_extraction("./ustar/linktest2/symtype")
 
 
-class GzipMiscReadTest(MiscReadTest):
-    tarname = gzipname
-    mode = "r:gz"
-
-    def test_non_existent_targz_file(self):
-        # Test for issue11513: prevent non-existent gzipped tarfiles raising
-        # multiple exceptions.
-        with self.assertRaisesRegex(OSError, "xxx") as ex:
-            tarfile.open("xxx", self.mode)
-        self.assertEqual(ex.exception.errno, errno.ENOENT)
-
-class GzipUstarReadTest(UstarReadTest):
-    tarname = gzipname
-    mode = "r:gz"
-class GzipStreamReadTest(StreamReadTest):
-    tarname = gzipname
-    mode = "r|gz"
-class GzipWriteTest(WriteTest):
-    mode = "w:gz"
-class GzipStreamWriteTest(StreamWriteTest):
-    mode = "w|gz"
-
-
-class Bz2MiscReadTest(MiscReadTest):
-    tarname = bz2name
-    mode = "r:bz2"
-class Bz2UstarReadTest(UstarReadTest):
-    tarname = bz2name
-    mode = "r:bz2"
-class Bz2StreamReadTest(StreamReadTest):
-    tarname = bz2name
-    mode = "r|bz2"
-class Bz2WriteTest(WriteTest):
-    mode = "w:bz2"
-class Bz2StreamWriteTest(StreamWriteTest):
-    mode = "w|bz2"
-
-class Bz2PartialReadTest(unittest.TestCase):
+class Bz2PartialReadTest(Bz2Test, unittest.TestCase):
     # Issue5068: The _BZ2Proxy.read() method loops forever
     # on an empty or partial bzipped file.
 
@@ -1790,7 +1825,8 @@
             hit_eof = False
             def read(self, n):
                 if self.hit_eof:
-                    raise AssertionError("infinite loop detected in tarfile.open()")
+                    raise AssertionError("infinite loop detected in "
+                                         "tarfile.open()")
                 self.hit_eof = self.tell() == len(self.getvalue())
                 return super(MyBytesIO, self).read(n)
             def seek(self, *args):
@@ -1811,102 +1847,23 @@
         self._test_partial_input("r:bz2")
 
 
-class LzmaMiscReadTest(MiscReadTest):
-    tarname = xzname
-    mode = "r:xz"
-class LzmaUstarReadTest(UstarReadTest):
-    tarname = xzname
-    mode = "r:xz"
-class LzmaStreamReadTest(StreamReadTest):
-    tarname = xzname
-    mode = "r|xz"
-class LzmaWriteTest(WriteTest):
-    mode = "w:xz"
-class LzmaStreamWriteTest(StreamWriteTest):
-    mode = "w|xz"
-
-
-def test_main():
+def setUpModule():
     support.unlink(TEMPDIR)
     os.makedirs(TEMPDIR)
 
-    tests = [
-        UstarReadTest,
-        MiscReadTest,
-        StreamReadTest,
-        DetectReadTest,
-        MemberReadTest,
-        GNUReadTest,
-        PaxReadTest,
-        WriteTest,
-        StreamWriteTest,
-        GNUWriteTest,
-        PaxWriteTest,
-        UstarUnicodeTest,
-        GNUUnicodeTest,
-        PAXUnicodeTest,
-        AppendTest,
-        LimitsTest,
-        MiscTest,
-        ContextManagerTest,
-    ]
-
-    if hasattr(os, "link"):
-        tests.append(HardlinkTest)
-    else:
-        tests.append(LinkEmulationTest)
-
     with open(tarname, "rb") as fobj:
         data = fobj.read()
 
-    if gzip:
-        # Create testtar.tar.gz and add gzip-specific tests.
-        support.unlink(gzipname)
-        with gzip.open(gzipname, "wb") as tar:
-            tar.write(data)
+    # Create compressed tarfiles.
+    for c in GzipTest, Bz2Test, LzmaTest:
+        if c.open:
+            support.unlink(c.tarname)
+            with c.open(c.tarname, "wb") as tar:
+                tar.write(data)
 
-        tests += [
-            GzipMiscReadTest,
-            GzipUstarReadTest,
-            GzipStreamReadTest,
-            GzipWriteTest,
-            GzipStreamWriteTest,
-        ]
-
-    if bz2:
-        # Create testtar.tar.bz2 and add bz2-specific tests.
-        support.unlink(bz2name)
-        with bz2.BZ2File(bz2name, "wb") as tar:
-            tar.write(data)
-
-        tests += [
-            Bz2MiscReadTest,
-            Bz2UstarReadTest,
-            Bz2StreamReadTest,
-            Bz2WriteTest,
-            Bz2StreamWriteTest,
-            Bz2PartialReadTest,
-        ]
-
-    if lzma:
-        # Create testtar.tar.xz and add lzma-specific tests.
-        support.unlink(xzname)
-        with lzma.LZMAFile(xzname, "w") as tar:
-            tar.write(data)
-
-        tests += [
-            LzmaMiscReadTest,
-            LzmaUstarReadTest,
-            LzmaStreamReadTest,
-            LzmaWriteTest,
-            LzmaStreamWriteTest,
-        ]
-
-    try:
-        support.run_unittest(*tests)
-    finally:
-        if os.path.exists(TEMPDIR):
-            shutil.rmtree(TEMPDIR)
+def tearDownModule():
+    if os.path.exists(TEMPDIR):
+        shutil.rmtree(TEMPDIR)
 
 if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:33:11 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 17 Jun 2013 15:33:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318207=3A_Fix_test?=
 =?utf-8?q?=5Fssl_for_some_versions_of_OpenSSL_that_ignore_seconds?=
Message-ID: <3bYtfH1xWpzRFT@mail.python.org>

http://hg.python.org/cpython/rev/c484ca129288
changeset:   84180:c484ca129288
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 17 15:32:57 2013 +0200
summary:
  Issue #18207: Fix test_ssl for some versions of OpenSSL that ignore seconds
in ASN1_TIME fields.

files:
  Lib/test/test_ssl.py |  18 ++++++++++++++++--
  Misc/NEWS            |   3 +++
  2 files changed, 19 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -6,6 +6,7 @@
 import socket
 import select
 import time
+import datetime
 import gc
 import os
 import errno
@@ -73,6 +74,19 @@
     # 0.9.7h or higher
     return ssl.OPENSSL_VERSION_INFO >= (0, 9, 7, 8, 15)
 
+def asn1time(cert_time):
+    # Some versions of OpenSSL ignore seconds, see #18207
+    # 0.9.8.i
+    if ssl._OPENSSL_API_VERSION == (0, 9, 8, 9, 15):
+        fmt = "%b %d %H:%M:%S %Y GMT"
+        dt = datetime.datetime.strptime(cert_time, fmt)
+        dt = dt.replace(second=0)
+        cert_time = dt.strftime(fmt)
+        # %d adds leading zero but ASN1_TIME_print() uses leading space
+        if cert_time[4] == "0":
+            cert_time = cert_time[:4] + " " + cert_time[5:]
+
+    return cert_time
 
 # Issue #9415: Ubuntu hijacks their OpenSSL and forcefully disables SSLv2
 def skip_if_broken_ubuntu_ssl(func):
@@ -142,8 +156,8 @@
                           (('commonName', 'localhost'),))
                         )
         # Note the next three asserts will fail if the keys are regenerated
-        self.assertEqual(p['notAfter'], 'Oct  5 23:01:56 2020 GMT')
-        self.assertEqual(p['notBefore'], 'Oct  8 23:01:56 2010 GMT')
+        self.assertEqual(p['notAfter'], asn1time('Oct  5 23:01:56 2020 GMT'))
+        self.assertEqual(p['notBefore'], asn1time('Oct  8 23:01:56 2010 GMT'))
         self.assertEqual(p['serialNumber'], 'D7C7381919AFC24E')
         self.assertEqual(p['subject'],
                          ((('countryName', 'XY'),),
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -409,6 +409,9 @@
 Tests
 -----
 
+- Issue #18207: Fix test_ssl for some versions of OpenSSL that ignore seconds
+  in ASN1_TIME fields.
+
 - Issue #18094: test_uuid no more reports skipped tests as passed.
 
 - Issue #17992: Add timeouts to asyncore and asynchat tests so that they won't

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:38:27 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 17 Jun 2013 15:38:27 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTY3?=
 =?utf-8?q?=3A_cgi=2EFieldStorage_no_more_fails_to_handle_multipart/form-d?=
 =?utf-8?q?ata?=
Message-ID: <3bYtmM0lS3zRFT@mail.python.org>

http://hg.python.org/cpython/rev/63058453a4cc
changeset:   84181:63058453a4cc
branch:      2.7
parent:      84151:aff0bdab358e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Jun 17 16:33:48 2013 +0300
summary:
  Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
when \r\n appears at end of 65535 bytes without other newlines.

files:
  Lib/cgi.py           |   9 +++++++++
  Lib/test/test_cgi.py |  23 +++++++++++++++++++++++
  Misc/NEWS            |   3 +++
  3 files changed, 35 insertions(+), 0 deletions(-)


diff --git a/Lib/cgi.py b/Lib/cgi.py
--- a/Lib/cgi.py
+++ b/Lib/cgi.py
@@ -697,6 +697,9 @@
             if not line:
                 self.done = -1
                 break
+            if delim == "\r":
+                line = delim + line
+                delim = ""
             if line[:2] == "--" and last_line_lfend:
                 strippedline = line.strip()
                 if strippedline == next:
@@ -713,6 +716,12 @@
                 delim = "\n"
                 line = line[:-1]
                 last_line_lfend = True
+            elif line[-1] == "\r":
+                # We may interrupt \r\n sequences if they span the 2**16
+                # byte boundary
+                delim = "\r"
+                line = line[:-1]
+                last_line_lfend = False
             else:
                 delim = ""
                 last_line_lfend = False
diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py
--- a/Lib/test/test_cgi.py
+++ b/Lib/test/test_cgi.py
@@ -266,6 +266,29 @@
                 got = getattr(fs.list[x], k)
                 self.assertEqual(got, exp)
 
+    def test_fieldstorage_multipart_maxline(self):
+        # Issue #18167
+        maxline = 1 << 16
+        self.maxDiff = None
+        def check(content):
+            data = """
+---123
+Content-Disposition: form-data; name="upload"; filename="fake.txt"
+Content-Type: text/plain
+
+%s
+---123--
+""".replace('\n', '\r\n') % content
+            environ = {
+                'CONTENT_LENGTH':   str(len(data)),
+                'CONTENT_TYPE':     'multipart/form-data; boundary=-123',
+                'REQUEST_METHOD':   'POST',
+            }
+            self.assertEqual(gen_result(data, environ), {'upload': content})
+        check('x' * (maxline - 1))
+        check('x' * (maxline - 1) + '\r')
+        check('x' * (maxline - 1) + '\r' + 'y' * (maxline - 1))
+
     _qs_result = {
         'key1': 'value1',
         'key2': ['value2x', 'value2y'],
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -18,6 +18,9 @@
 Library
 -------
 
+- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+  when \r\n appears at end of 65535 bytes without other newlines.
+
 - Issue #17403: urllib.parse.robotparser normalizes the urls before adding to
   ruleline. This helps in handling certain types invalid urls in a conservative
   manner. Patch contributed by Mher Movsisyan.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:38:28 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 17 Jun 2013 15:38:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTY3?=
 =?utf-8?q?=3A_cgi=2EFieldStorage_no_more_fails_to_handle_multipart/form-d?=
 =?utf-8?q?ata?=
Message-ID: <3bYtmN37h3z7Ljd@mail.python.org>

http://hg.python.org/cpython/rev/a48f65bac986
changeset:   84182:a48f65bac986
branch:      3.3
parent:      84178:4b2188b13dd2
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Jun 17 16:34:41 2013 +0300
summary:
  Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
when \r\n appears at end of 65535 bytes without other newlines.

files:
  Lib/cgi.py           |   9 +++++++++
  Lib/test/test_cgi.py |  23 +++++++++++++++++++++++
  Misc/NEWS            |   3 +++
  3 files changed, 35 insertions(+), 0 deletions(-)


diff --git a/Lib/cgi.py b/Lib/cgi.py
--- a/Lib/cgi.py
+++ b/Lib/cgi.py
@@ -786,6 +786,9 @@
             if not line:
                 self.done = -1
                 break
+            if delim == b"\r":
+                line = delim + line
+                delim = b""
             if line.startswith(b"--") and last_line_lfend:
                 strippedline = line.rstrip()
                 if strippedline == next_boundary:
@@ -802,6 +805,12 @@
                 delim = b"\n"
                 line = line[:-1]
                 last_line_lfend = True
+            elif line.endswith(b"\r"):
+                # We may interrupt \r\n sequences if they span the 2**16
+                # byte boundary
+                delim = b"\r"
+                line = line[:-1]
+                last_line_lfend = False
             else:
                 delim = b""
                 last_line_lfend = False
diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py
--- a/Lib/test/test_cgi.py
+++ b/Lib/test/test_cgi.py
@@ -256,6 +256,29 @@
                     got = getattr(fs.list[x], k)
                     self.assertEqual(got, exp)
 
+    def test_fieldstorage_multipart_maxline(self):
+        # Issue #18167
+        maxline = 1 << 16
+        self.maxDiff = None
+        def check(content):
+            data = """---123
+Content-Disposition: form-data; name="upload"; filename="fake.txt"
+Content-Type: text/plain
+
+%s
+---123--
+""".replace('\n', '\r\n') % content
+            environ = {
+                'CONTENT_LENGTH':   str(len(data)),
+                'CONTENT_TYPE':     'multipart/form-data; boundary=-123',
+                'REQUEST_METHOD':   'POST',
+            }
+            self.assertEqual(gen_result(data, environ),
+                             {'upload': content.encode('latin1')})
+        check('x' * (maxline - 1))
+        check('x' * (maxline - 1) + '\r')
+        check('x' * (maxline - 1) + '\r' + 'y' * (maxline - 1))
+
     _qs_result = {
         'key1': 'value1',
         'key2': ['value2x', 'value2y'],
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -32,6 +32,9 @@
 Library
 -------
 
+- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+  when \r\n appears at end of 65535 bytes without other newlines.
+
 - subprocess: Prevent a possible double close of parent pipe fds when the
   subprocess exec runs into an error.  Prevent a regular multi-close of the
   /dev/null fd when any of stdin, stdout and stderr was set to DEVNULL.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:38:29 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 17 Jun 2013 15:38:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318167=3A_cgi=2EFieldStorage_no_more_fails_to_ha?=
 =?utf-8?q?ndle_multipart/form-data?=
Message-ID: <3bYtmP68PFz7Ljk@mail.python.org>

http://hg.python.org/cpython/rev/17ec73a3a854
changeset:   84183:17ec73a3a854
parent:      84179:5c0816e64aac
parent:      84182:a48f65bac986
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Jun 17 16:36:20 2013 +0300
summary:
  Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
when \r\n appears at end of 65535 bytes without other newlines.

files:
  Lib/cgi.py           |   9 +++++++++
  Lib/test/test_cgi.py |  23 +++++++++++++++++++++++
  Misc/NEWS            |   3 +++
  3 files changed, 35 insertions(+), 0 deletions(-)


diff --git a/Lib/cgi.py b/Lib/cgi.py
--- a/Lib/cgi.py
+++ b/Lib/cgi.py
@@ -786,6 +786,9 @@
             if not line:
                 self.done = -1
                 break
+            if delim == b"\r":
+                line = delim + line
+                delim = b""
             if line.startswith(b"--") and last_line_lfend:
                 strippedline = line.rstrip()
                 if strippedline == next_boundary:
@@ -802,6 +805,12 @@
                 delim = b"\n"
                 line = line[:-1]
                 last_line_lfend = True
+            elif line.endswith(b"\r"):
+                # We may interrupt \r\n sequences if they span the 2**16
+                # byte boundary
+                delim = b"\r"
+                line = line[:-1]
+                last_line_lfend = False
             else:
                 delim = b""
                 last_line_lfend = False
diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py
--- a/Lib/test/test_cgi.py
+++ b/Lib/test/test_cgi.py
@@ -256,6 +256,29 @@
                     got = getattr(fs.list[x], k)
                     self.assertEqual(got, exp)
 
+    def test_fieldstorage_multipart_maxline(self):
+        # Issue #18167
+        maxline = 1 << 16
+        self.maxDiff = None
+        def check(content):
+            data = """---123
+Content-Disposition: form-data; name="upload"; filename="fake.txt"
+Content-Type: text/plain
+
+%s
+---123--
+""".replace('\n', '\r\n') % content
+            environ = {
+                'CONTENT_LENGTH':   str(len(data)),
+                'CONTENT_TYPE':     'multipart/form-data; boundary=-123',
+                'REQUEST_METHOD':   'POST',
+            }
+            self.assertEqual(gen_result(data, environ),
+                             {'upload': content.encode('latin1')})
+        check('x' * (maxline - 1))
+        check('x' * (maxline - 1) + '\r')
+        check('x' * (maxline - 1) + '\r' + 'y' * (maxline - 1))
+
     _qs_result = {
         'key1': 'value1',
         'key2': ['value2x', 'value2y'],
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,9 @@
 Library
 -------
 
+- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+  when \r\n appears at end of 65535 bytes without other newlines.
+
 - Issue #18076: Introduce importlib.util.decode_source().
 
 - importlib.abc.SourceLoader.get_source() no longer changes SyntaxError or

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:38:31 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Mon, 17 Jun 2013 15:38:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_heads?=
Message-ID: <3bYtmR1c0qz7Ljn@mail.python.org>

http://hg.python.org/cpython/rev/fd5ff09f1751
changeset:   84184:fd5ff09f1751
parent:      84183:17ec73a3a854
parent:      84180:c484ca129288
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Mon Jun 17 16:38:00 2013 +0300
summary:
  Merge heads

files:
  Lib/test/test_ssl.py |  18 ++++++++++++++++--
  Misc/NEWS            |   3 +++
  2 files changed, 19 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -6,6 +6,7 @@
 import socket
 import select
 import time
+import datetime
 import gc
 import os
 import errno
@@ -73,6 +74,19 @@
     # 0.9.7h or higher
     return ssl.OPENSSL_VERSION_INFO >= (0, 9, 7, 8, 15)
 
+def asn1time(cert_time):
+    # Some versions of OpenSSL ignore seconds, see #18207
+    # 0.9.8.i
+    if ssl._OPENSSL_API_VERSION == (0, 9, 8, 9, 15):
+        fmt = "%b %d %H:%M:%S %Y GMT"
+        dt = datetime.datetime.strptime(cert_time, fmt)
+        dt = dt.replace(second=0)
+        cert_time = dt.strftime(fmt)
+        # %d adds leading zero but ASN1_TIME_print() uses leading space
+        if cert_time[4] == "0":
+            cert_time = cert_time[:4] + " " + cert_time[5:]
+
+    return cert_time
 
 # Issue #9415: Ubuntu hijacks their OpenSSL and forcefully disables SSLv2
 def skip_if_broken_ubuntu_ssl(func):
@@ -142,8 +156,8 @@
                           (('commonName', 'localhost'),))
                         )
         # Note the next three asserts will fail if the keys are regenerated
-        self.assertEqual(p['notAfter'], 'Oct  5 23:01:56 2020 GMT')
-        self.assertEqual(p['notBefore'], 'Oct  8 23:01:56 2010 GMT')
+        self.assertEqual(p['notAfter'], asn1time('Oct  5 23:01:56 2020 GMT'))
+        self.assertEqual(p['notBefore'], asn1time('Oct  8 23:01:56 2010 GMT'))
         self.assertEqual(p['serialNumber'], 'D7C7381919AFC24E')
         self.assertEqual(p['subject'],
                          ((('countryName', 'XY'),),
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -412,6 +412,9 @@
 Tests
 -----
 
+- Issue #18207: Fix test_ssl for some versions of OpenSSL that ignore seconds
+  in ASN1_TIME fields.
+
 - Issue #18094: test_uuid no more reports skipped tests as passed.
 
 - Issue #17992: Add timeouts to asyncore and asynchat tests so that they won't

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:45:21 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 17 Jun 2013 15:45:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318147=3A_Add_diag?=
 =?utf-8?q?nostic_functions_to_ssl=2ESSLContext=28=29=2E?=
Message-ID: <3bYtwK4G3WzRdJ@mail.python.org>

http://hg.python.org/cpython/rev/38e759e4c9e6
changeset:   84185:38e759e4c9e6
parent:      84180:c484ca129288
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 17 15:44:12 2013 +0200
summary:
  Issue #18147: Add diagnostic functions to ssl.SSLContext().

get_ca_list() lists all loaded CA certificates and cert_store_stats() returns
amount of loaded X.509 certs, X.509 CA certs and CRLs.

files:
  Doc/library/ssl.rst  |   24 +++++
  Lib/test/test_ssl.py |   57 ++++++++++++
  Misc/NEWS            |    4 +
  Modules/_ssl.c       |  144 +++++++++++++++++++++++++++---
  4 files changed, 212 insertions(+), 17 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -791,6 +791,19 @@
 
 :class:`SSLContext` objects have the following methods and attributes:
 
+.. method:: SSLContext.cert_store_stats()
+
+   Get statistics about quantities of loaded X.509 certificates, count of
+   X.509 certificates flagged as CA certificates and certificate revocation
+   lists as dictionary.
+
+   Example for a context with one CA cert and one other cert::
+
+      >>> context.cert_store_stats()
+      {'crl': 0, 'x509_ca': 1, 'x509': 2}
+
+   .. versionadded:: 3.4
+
 .. method:: SSLContext.load_cert_chain(certfile, keyfile=None, password=None)
 
    Load a private key and the corresponding certificate.  The *certfile*
@@ -837,6 +850,17 @@
    following an `OpenSSL specific layout
    <http://www.openssl.org/docs/ssl/SSL_CTX_load_verify_locations.html>`_.
 
+.. method:: SSLContext.get_ca_certs(binary_form=False)
+
+   Get a list of loaded "certification authority" (CA) certificates. If the
+   ``binary_form`` parameter is :const:`False` each list
+   entry is a dict like the output of :meth:`SSLSocket.getpeercert`. Otherwise
+   the method returns a list of DER-encoded certificates. The returned list
+   does not contain certificates from *capath* unless a certificate was
+   requested and loaded by a SSL connection.
+
+   ..versionadded:: 3.4
+
 .. method:: SSLContext.set_default_verify_paths()
 
    Load a set of default "certification authority" (CA) certificates from
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -680,6 +680,47 @@
         gc.collect()
         self.assertIs(wr(), None)
 
+    def test_cert_store_stats(self):
+        ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+        self.assertEqual(ctx.cert_store_stats(),
+            {'x509_ca': 0, 'crl': 0, 'x509': 0})
+        ctx.load_cert_chain(CERTFILE)
+        self.assertEqual(ctx.cert_store_stats(),
+            {'x509_ca': 0, 'crl': 0, 'x509': 0})
+        ctx.load_verify_locations(CERTFILE)
+        self.assertEqual(ctx.cert_store_stats(),
+            {'x509_ca': 0, 'crl': 0, 'x509': 1})
+        ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT)
+        self.assertEqual(ctx.cert_store_stats(),
+            {'x509_ca': 1, 'crl': 0, 'x509': 2})
+
+    def test_get_ca_certs(self):
+        ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+        self.assertEqual(ctx.get_ca_certs(), [])
+        # CERTFILE is not flagged as X509v3 Basic Constraints: CA:TRUE
+        ctx.load_verify_locations(CERTFILE)
+        self.assertEqual(ctx.get_ca_certs(), [])
+        # but SVN_PYTHON_ORG_ROOT_CERT is a CA cert
+        ctx.load_verify_locations(SVN_PYTHON_ORG_ROOT_CERT)
+        self.assertEqual(ctx.get_ca_certs(),
+            [{'issuer': ((('organizationName', 'Root CA'),),
+                         (('organizationalUnitName', 'http://www.cacert.org'),),
+                         (('commonName', 'CA Cert Signing Authority'),),
+                         (('emailAddress', 'support at cacert.org'),)),
+              'notAfter': asn1time('Mar 29 12:29:49 2033 GMT'),
+              'notBefore': asn1time('Mar 30 12:29:49 2003 GMT'),
+              'serialNumber': '00',
+              'subject': ((('organizationName', 'Root CA'),),
+                          (('organizationalUnitName', 'http://www.cacert.org'),),
+                          (('commonName', 'CA Cert Signing Authority'),),
+                          (('emailAddress', 'support at cacert.org'),)),
+              'version': 3}])
+
+        with open(SVN_PYTHON_ORG_ROOT_CERT) as f:
+            pem = f.read()
+        der = ssl.PEM_cert_to_DER_cert(pem)
+        self.assertEqual(ctx.get_ca_certs(True), [der])
+
 
 class SSLErrorTests(unittest.TestCase):
 
@@ -995,6 +1036,22 @@
             finally:
                 s.close()
 
+    def test_get_ca_certs_capath(self):
+        # capath certs are loaded on request
+        with support.transient_internet("svn.python.org"):
+            ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
+            ctx.verify_mode = ssl.CERT_REQUIRED
+            ctx.load_verify_locations(capath=CAPATH)
+            self.assertEqual(ctx.get_ca_certs(), [])
+            s = ctx.wrap_socket(socket.socket(socket.AF_INET))
+            s.connect(("svn.python.org", 443))
+            try:
+                cert = s.getpeercert()
+                self.assertTrue(cert)
+            finally:
+                s.close()
+            self.assertEqual(len(ctx.get_ca_certs()), 1)
+
 
 try:
     import threading
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,10 @@
 Library
 -------
 
+- Issue #18147: Add diagnostic functions to ssl.SSLContext(). get_ca_list()
+  lists all loaded CA certificates and cert_store_stats() returns amount of
+  loaded X.509 certs, X.509 CA certs and CRLs.
+
 - Issue #18076: Introduce importlib.util.decode_source().
 
 - importlib.abc.SourceLoader.get_source() no longer changes SyntaxError or
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1023,6 +1023,24 @@
     return NULL;
 }
 
+static PyObject *
+_certificate_to_der(X509 *certificate)
+{
+    unsigned char *bytes_buf = NULL;
+    int len;
+    PyObject *retval;
+
+    bytes_buf = NULL;
+    len = i2d_X509(certificate, &bytes_buf);
+    if (len < 0) {
+        _setSSLError(NULL, 0, __FILE__, __LINE__);
+        return NULL;
+    }
+    /* this is actually an immutable bytes sequence */
+    retval = PyBytes_FromStringAndSize((const char *) bytes_buf, len);
+    OPENSSL_free(bytes_buf);
+    return retval;
+}
 
 static PyObject *
 PySSL_test_decode_certificate (PyObject *mod, PyObject *args) {
@@ -1068,8 +1086,6 @@
 static PyObject *
 PySSL_peercert(PySSLSocket *self, PyObject *args)
 {
-    PyObject *retval = NULL;
-    int len;
     int verification;
     int binary_mode = 0;
 
@@ -1081,21 +1097,7 @@
 
     if (binary_mode) {
         /* return cert in DER-encoded format */
-
-        unsigned char *bytes_buf = NULL;
-
-        bytes_buf = NULL;
-        len = i2d_X509(self->peer_cert, &bytes_buf);
-        if (len < 0) {
-            PySSL_SetError(self, len, __FILE__, __LINE__);
-            return NULL;
-        }
-        /* this is actually an immutable bytes sequence */
-        retval = PyBytes_FromStringAndSize
-          ((const char *) bytes_buf, len);
-        OPENSSL_free(bytes_buf);
-        return retval;
-
+        return _certificate_to_der(self->peer_cert);
     } else {
         verification = SSL_CTX_get_verify_mode(SSL_get_SSL_CTX(self->ssl));
         if ((verification & SSL_VERIFY_PEER) == 0)
@@ -2555,6 +2557,110 @@
 #endif
 }
 
+PyDoc_STRVAR(PySSL_get_stats_doc,
+"cert_store_stats() -> {'crl': int, 'x509_ca': int, 'x509': int}\n\
+\n\
+Returns quantities of loaded X.509 certificates. X.509 certificates with a\n\
+CA extension and certificate revocation lists inside the context's cert\n\
+store.\n\
+NOTE: Certificates in a capath directory aren't loaded unless they have\n\
+been used at least once.");
+
+static PyObject *
+cert_store_stats(PySSLContext *self)
+{
+    X509_STORE *store;
+    X509_OBJECT *obj;
+    int x509 = 0, crl = 0, pkey = 0, ca = 0, i;
+
+    store = SSL_CTX_get_cert_store(self->ctx);
+    for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) {
+        obj = sk_X509_OBJECT_value(store->objs, i);
+        switch (obj->type) {
+            case X509_LU_X509:
+                x509++;
+                if (X509_check_ca(obj->data.x509)) {
+                    ca++;
+                }
+                break;
+            case X509_LU_CRL:
+                crl++;
+                break;
+            case X509_LU_PKEY:
+                pkey++;
+                break;
+            default:
+                /* Ignore X509_LU_FAIL, X509_LU_RETRY, X509_LU_PKEY.
+                 * As far as I can tell they are internal states and never
+                 * stored in a cert store */
+                break;
+        }
+    }
+    return Py_BuildValue("{sisisi}", "x509", x509, "crl", crl,
+        "x509_ca", ca);
+}
+
+PyDoc_STRVAR(PySSL_get_ca_certs_doc,
+"get_ca_certs([der=False]) -> list of loaded certificate\n\
+\n\
+Returns a list of dicts with information of loaded CA certs. If the\n\
+optional argument is True, returns a DER-encoded copy of the CA certificate.\n\
+NOTE: Certificates in a capath directory aren't loaded unless they have\n\
+been used at least once.");
+
+static PyObject *
+get_ca_certs(PySSLContext *self, PyObject *args)
+{
+    X509_STORE *store;
+    PyObject *ci = NULL, *rlist = NULL;
+    int i;
+    int binary_mode = 0;
+
+    if (!PyArg_ParseTuple(args, "|p:get_ca_certs", &binary_mode)) {
+        return NULL;
+    }
+
+    if ((rlist = PyList_New(0)) == NULL) {
+        return NULL;
+    }
+
+    store = SSL_CTX_get_cert_store(self->ctx);
+    for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) {
+        X509_OBJECT *obj;
+        X509 *cert;
+
+        obj = sk_X509_OBJECT_value(store->objs, i);
+        if (obj->type != X509_LU_X509) {
+            /* not a x509 cert */
+            continue;
+        }
+        /* CA for any purpose */
+        cert = obj->data.x509;
+        if (!X509_check_ca(cert)) {
+            continue;
+        }
+        if (binary_mode) {
+            ci = _certificate_to_der(cert);
+        } else {
+            ci = _decode_certificate(cert);
+        }
+        if (ci == NULL) {
+            goto error;
+        }
+        if (PyList_Append(rlist, ci) == -1) {
+            goto error;
+        }
+        Py_CLEAR(ci);
+    }
+    return rlist;
+
+  error:
+    Py_XDECREF(ci);
+    Py_XDECREF(rlist);
+    return NULL;
+}
+
+
 static PyGetSetDef context_getsetlist[] = {
     {"options", (getter) get_options,
                 (setter) set_options, NULL},
@@ -2586,6 +2692,10 @@
 #endif
     {"set_servername_callback", (PyCFunction) set_servername_callback,
                     METH_VARARGS, PySSL_set_servername_callback_doc},
+    {"cert_store_stats", (PyCFunction) cert_store_stats,
+                    METH_NOARGS, PySSL_get_stats_doc},
+    {"get_ca_certs", (PyCFunction) get_ca_certs,
+                    METH_VARARGS, PySSL_get_ca_certs_doc},
     {NULL, NULL}        /* sentinel */
 };
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 15:45:22 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 17 Jun 2013 15:45:22 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge_heads?=
Message-ID: <3bYtwL6GV7z7LjS@mail.python.org>

http://hg.python.org/cpython/rev/ec4b0145f7e5
changeset:   84186:ec4b0145f7e5
parent:      84185:38e759e4c9e6
parent:      84184:fd5ff09f1751
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 17 15:45:11 2013 +0200
summary:
  merge heads

files:
  Lib/cgi.py           |   9 +++++++++
  Lib/test/test_cgi.py |  23 +++++++++++++++++++++++
  Misc/NEWS            |   3 +++
  3 files changed, 35 insertions(+), 0 deletions(-)


diff --git a/Lib/cgi.py b/Lib/cgi.py
--- a/Lib/cgi.py
+++ b/Lib/cgi.py
@@ -786,6 +786,9 @@
             if not line:
                 self.done = -1
                 break
+            if delim == b"\r":
+                line = delim + line
+                delim = b""
             if line.startswith(b"--") and last_line_lfend:
                 strippedline = line.rstrip()
                 if strippedline == next_boundary:
@@ -802,6 +805,12 @@
                 delim = b"\n"
                 line = line[:-1]
                 last_line_lfend = True
+            elif line.endswith(b"\r"):
+                # We may interrupt \r\n sequences if they span the 2**16
+                # byte boundary
+                delim = b"\r"
+                line = line[:-1]
+                last_line_lfend = False
             else:
                 delim = b""
                 last_line_lfend = False
diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py
--- a/Lib/test/test_cgi.py
+++ b/Lib/test/test_cgi.py
@@ -256,6 +256,29 @@
                     got = getattr(fs.list[x], k)
                     self.assertEqual(got, exp)
 
+    def test_fieldstorage_multipart_maxline(self):
+        # Issue #18167
+        maxline = 1 << 16
+        self.maxDiff = None
+        def check(content):
+            data = """---123
+Content-Disposition: form-data; name="upload"; filename="fake.txt"
+Content-Type: text/plain
+
+%s
+---123--
+""".replace('\n', '\r\n') % content
+            environ = {
+                'CONTENT_LENGTH':   str(len(data)),
+                'CONTENT_TYPE':     'multipart/form-data; boundary=-123',
+                'REQUEST_METHOD':   'POST',
+            }
+            self.assertEqual(gen_result(data, environ),
+                             {'upload': content.encode('latin1')})
+        check('x' * (maxline - 1))
+        check('x' * (maxline - 1) + '\r')
+        check('x' * (maxline - 1) + '\r' + 'y' * (maxline - 1))
+
     _qs_result = {
         'key1': 'value1',
         'key2': ['value2x', 'value2y'],
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -127,6 +127,9 @@
   lists all loaded CA certificates and cert_store_stats() returns amount of
   loaded X.509 certs, X.509 CA certs and CRLs.
 
+- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+  when \r\n appears at end of 65535 bytes without other newlines.
+
 - Issue #18076: Introduce importlib.util.decode_source().
 
 - importlib.abc.SourceLoader.get_source() no longer changes SyntaxError or

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 19:54:50 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 17 Jun 2013 19:54:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?devguide=3A_add_me_and_GPS_to_some_mo?=
	=?utf-8?q?dules?=
Message-ID: <3bZ0SB4FKMzRZ3@mail.python.org>

http://hg.python.org/devguide/rev/c92f70308362
changeset:   622:c92f70308362
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 17 19:54:42 2013 +0200
summary:
  add me and GPS to some modules

files:
  experts.rst |  13 +++++++------
  1 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/experts.rst b/experts.rst
--- a/experts.rst
+++ b/experts.rst
@@ -118,9 +118,9 @@
 glob
 grp
 gzip
-hashlib
+hashlib               christian.heimes, gregory.p.smith
 heapq                 rhettinger, stutzbach
-hmac
+hmac                  christian.heimes, gregory.p.smith
 html                  ezio.melotti
 http
 idlelib               kbk, terry.reedy, roger.serwy
@@ -201,8 +201,8 @@
 socketserver
 spwd
 sqlite3               ghaering
-ssl                   janssen, pitrou, giampaolo.rodola
-stat
+ssl                   janssen, pitrou, giampaolo.rodola, christian.heimes
+stat                  christian.heimes
 string                georg.brandl*
 stringprep
 struct                mark.dickinson, meador.inge
@@ -249,8 +249,8 @@
 xml.dom.minidom
 xml.dom.pulldom
 xml.etree             effbot (inactive), eli.bendersky
-xml.parsers.expat
-xml.sax
+xml.parsers.expat     christian.heimes
+xml.sax               christian.heimes
 xml.sax.handler
 xml.sax.saxutils
 xml.sax.xmlreader
@@ -303,6 +303,7 @@
 bytecode            pitrou, georg.brandl
 context managers    ncoghlan
 coverity scan       christian.heimes, brett.cannon
+cryptography        christian.heimes, gregory.p.smith
 data formats        mark.dickinson, georg.brandl
 database            lemburg
 devguide            ncoghlan, eric.araujo, ezio.melotti

-- 
Repository URL: http://hg.python.org/devguide

From python-checkins at python.org  Mon Jun 17 20:28:25 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 20:28:25 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogcmVncnRlc3QucHk6?=
 =?utf-8?q?_Fix_typo_in_the_usage_of_the_faulthandler_module?=
Message-ID: <3bZ1Bx0yJvzSdK@mail.python.org>

http://hg.python.org/cpython/rev/732fb15e856d
changeset:   84187:732fb15e856d
branch:      3.3
parent:      84182:a48f65bac986
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 20:27:10 2013 +0200
summary:
  regrtest.py: Fix typo in the usage of the faulthandler module

files:
  Lib/test/regrtest.py |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -454,13 +454,13 @@
             # join it with the saved CWD so it ends up where the user expects.
             testdir = os.path.join(support.SAVEDCWD, a)
         elif o == '--timeout':
-            if hasattr(faulthandler, 'dump_tracebacks_later'):
+            if hasattr(faulthandler, 'dump_traceback_later'):
                 timeout = float(a)
                 if timeout <= 0:
                     timeout = None
             else:
                 print("Warning: The timeout option requires "
-                      "faulthandler.dump_tracebacks_later")
+                      "faulthandler.dump_traceback_later")
                 timeout = None
         elif o == '--wait':
             input("Press any key to continue...")
@@ -904,7 +904,7 @@
         support.use_resources = use_resources
     use_timeout = (timeout is not None)
     if use_timeout:
-        faulthandler.dump_tracebacks_later(timeout, exit=True)
+        faulthandler.dump_traceback_later(timeout, exit=True)
     try:
         support.match_tests = match_tests
         if failfast:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 20:28:26 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 20:28:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=28Merge_3=2E3=29_regrtest=2Epy=3A_Fix_typo_in_the_usage?=
 =?utf-8?q?_of_the_faulthandler_module?=
Message-ID: <3bZ1By2yp6z7Ljv@mail.python.org>

http://hg.python.org/cpython/rev/c026dbe95f78
changeset:   84188:c026dbe95f78
parent:      84186:ec4b0145f7e5
parent:      84187:732fb15e856d
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 20:28:02 2013 +0200
summary:
  (Merge 3.3) regrtest.py: Fix typo in the usage of the faulthandler module

files:
  Lib/test/regrtest.py |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -536,13 +536,13 @@
             # join it with the saved CWD so it ends up where the user expects.
             testdir = os.path.join(support.SAVEDCWD, a)
         elif o == '--timeout':
-            if hasattr(faulthandler, 'dump_tracebacks_later'):
+            if hasattr(faulthandler, 'dump_traceback_later'):
                 timeout = float(a)
                 if timeout <= 0:
                     timeout = None
             else:
                 print("Warning: The timeout option requires "
-                      "faulthandler.dump_tracebacks_later")
+                      "faulthandler.dump_traceback_later")
                 timeout = None
         elif o == '--wait':
             input("Press any key to continue...")
@@ -972,7 +972,7 @@
         support.use_resources = use_resources
     use_timeout = (timeout is not None)
     if use_timeout:
-        faulthandler.dump_tracebacks_later(timeout, exit=True)
+        faulthandler.dump_traceback_later(timeout, exit=True)
     try:
         support.match_tests = match_tests
         if failfast:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 20:35:59 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 20:35:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogcmVncnRlc3QucHk6?=
 =?utf-8?q?_Fix_another_typo_in_the_usage_of_the_faulthandler_module?=
Message-ID: <3bZ1Mg3qjHz7Lk6@mail.python.org>

http://hg.python.org/cpython/rev/dac8d28c78a6
changeset:   84189:dac8d28c78a6
branch:      3.3
parent:      84187:732fb15e856d
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 20:35:08 2013 +0200
summary:
  regrtest.py: Fix another typo in the usage of the faulthandler module

files:
  Lib/test/regrtest.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -944,7 +944,7 @@
         return result
     finally:
         if use_timeout:
-            faulthandler.cancel_dump_tracebacks_later()
+            faulthandler.cancel_dump_traceback_later()
         cleanup_test_droppings(test, verbose)
 runtest.stringio = None
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 20:36:00 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 20:36:00 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=28Merge_3=2E3=29_regrtest=2Epy=3A_Fix_another_typo_in_t?=
 =?utf-8?q?he_usage_of_the_faulthandler?=
Message-ID: <3bZ1Mh5mFWz7Ljv@mail.python.org>

http://hg.python.org/cpython/rev/9786b37082dd
changeset:   84190:9786b37082dd
parent:      84188:c026dbe95f78
parent:      84189:dac8d28c78a6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 20:35:42 2013 +0200
summary:
  (Merge 3.3) regrtest.py: Fix another typo in the usage of the faulthandler module

files:
  Lib/test/regrtest.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1012,7 +1012,7 @@
         return result
     finally:
         if use_timeout:
-            faulthandler.cancel_dump_tracebacks_later()
+            faulthandler.cancel_dump_traceback_later()
         cleanup_test_droppings(test, verbose)
 runtest.stringio = None
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 20:40:21 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 20:40:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318228=3A_Fix_loca?=
 =?utf-8?q?le_test_of_test=2Eregrtest=2Esaved=5Ftest=5Fenvironment?=
Message-ID: <3bZ1Sj29psz7Ljn@mail.python.org>

http://hg.python.org/cpython/rev/00824f9e29f3
changeset:   84191:00824f9e29f3
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 20:40:05 2013 +0200
summary:
  Issue #18228: Fix locale test of test.regrtest.saved_test_environment

Skip LC_ALL becore getlocale(locale.LC_ALL) always fail, and catch also
ValueError.

files:
  Lib/test/regrtest.py |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1231,13 +1231,14 @@
             elif os.path.isdir(support.TESTFN):
                 shutil.rmtree(support.TESTFN)
 
-    _lc = [getattr(locale, lc) for lc in dir(locale) if lc.startswith('LC_')]
+    _lc = [getattr(locale, lc) for lc in dir(locale)
+           if lc.startswith('LC_') and lc != 'LC_ALL']
     def get_locale(self):
         pairings = []
         for lc in self._lc:
             try:
                 pairings.append((lc, locale.getlocale(lc)))
-            except TypeError:
+            except (TypeError, ValueError):
                 continue
         return pairings
     def restore_locale(self, saved):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 21:19:46 2013
From: python-checkins at python.org (ethan.furman)
Date: Mon, 17 Jun 2013 21:19:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?devguide=3A_Added_enum_module_with_et?=
 =?utf-8?q?han=2Efurman=2C_eli=2Ebendersky=2C_and_barry_as_experts=2E?=
Message-ID: <3bZ2LB6Hlvz7LjP@mail.python.org>

http://hg.python.org/devguide/rev/c8581129d2c4
changeset:   623:c8581129d2c4
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Mon Jun 17 12:19:40 2013 -0700
summary:
  Added enum module with ethan.furman, eli.bendersky, and barry as experts.

files:
  experts.rst |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/experts.rst b/experts.rst
--- a/experts.rst
+++ b/experts.rst
@@ -100,6 +100,7 @@
 dummy_threading       brett.cannon
 email                 barry, r.david.murray*
 encodings             lemburg, loewis
+enum                  eli.bendersky, barry, ethan.furman* 
 errno
 exceptions
 fcntl

-- 
Repository URL: http://hg.python.org/devguide

From python-checkins at python.org  Mon Jun 17 21:28:40 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 21:28:40 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318228=3A_Use_loca?=
 =?utf-8?q?le=2Esetlocale=28name=2C_None=29_instead_of?=
Message-ID: <3bZ2XS0mnQzMK5@mail.python.org>

http://hg.python.org/cpython/rev/6cbd992d3411
changeset:   84192:6cbd992d3411
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 21:28:14 2013 +0200
summary:
  Issue #18228: Use locale.setlocale(name, None) instead of
locale.getlocale(name) in test.regrtest.saved_test_environment

locale.getlocale() parses the locale, which is useless for
saved_test_environment.

files:
  Lib/test/regrtest.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -1232,12 +1232,12 @@
                 shutil.rmtree(support.TESTFN)
 
     _lc = [getattr(locale, lc) for lc in dir(locale)
-           if lc.startswith('LC_') and lc != 'LC_ALL']
+           if lc.startswith('LC_')]
     def get_locale(self):
         pairings = []
         for lc in self._lc:
             try:
-                pairings.append((lc, locale.getlocale(lc)))
+                pairings.append((lc, locale.setlocale(lc, None)))
             except (TypeError, ValueError):
                 continue
         return pairings

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 21:54:18 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 21:54:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjM4?=
 =?utf-8?q?=3A_Skip_test=5Fsignal=2Etest=5Fsigwaitinfo=5Finterrupted=28=29?=
 =?utf-8?q?_on_AIX?=
Message-ID: <3bZ3620PpCz7LjS@mail.python.org>

http://hg.python.org/cpython/rev/a0e234e10da6
changeset:   84193:a0e234e10da6
branch:      3.3
parent:      84189:dac8d28c78a6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 21:51:56 2013 +0200
summary:
  Issue #18238: Skip test_signal.test_sigwaitinfo_interrupted() on AIX

sigwaitinfo() can be interrupted on Linux (raises InterruptedError), but not on
AIX.

files:
  Lib/test/test_signal.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py
--- a/Lib/test/test_signal.py
+++ b/Lib/test/test_signal.py
@@ -705,6 +705,10 @@
 
     @unittest.skipUnless(hasattr(signal, 'sigwaitinfo'),
                          'need signal.sigwaitinfo()')
+    # Issue #18238: sigwaitinfo() can be interrupted on Linux (raises
+    # InterruptedError), but not on AIX
+    @unittest.skipIf(sys.platform.startswith("aix"),
+                     'signal.sigwaitinfo() cannot be interrupted on AIX')
     def test_sigwaitinfo_interrupted(self):
         self.wait_helper(signal.SIGUSR1, '''
         def test(signum):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 21:54:19 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 21:54:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogKE1lcmdlIDMuMykgSXNzdWUgIzE4MjM4OiBTa2lwIHRlc3Rfc2lnbmFs?=
 =?utf-8?q?=2Etest=5Fsigwaitinfo=5Finterrupted=28=29_on_AIX?=
Message-ID: <3bZ3632Bg5zSv0@mail.python.org>

http://hg.python.org/cpython/rev/8e277e6bd11b
changeset:   84194:8e277e6bd11b
parent:      84192:6cbd992d3411
parent:      84193:a0e234e10da6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 21:52:24 2013 +0200
summary:
  (Merge 3.3) Issue #18238: Skip test_signal.test_sigwaitinfo_interrupted() on AIX

sigwaitinfo() can be interrupted on Linux (raises InterruptedError), but not on
AIX.

files:
  Lib/test/test_signal.py |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py
--- a/Lib/test/test_signal.py
+++ b/Lib/test/test_signal.py
@@ -702,6 +702,10 @@
 
     @unittest.skipUnless(hasattr(signal, 'sigwaitinfo'),
                          'need signal.sigwaitinfo()')
+    # Issue #18238: sigwaitinfo() can be interrupted on Linux (raises
+    # InterruptedError), but not on AIX
+    @unittest.skipIf(sys.platform.startswith("aix"),
+                     'signal.sigwaitinfo() cannot be interrupted on AIX')
     def test_sigwaitinfo_interrupted(self):
         self.wait_helper(signal.SIGUSR1, '''
         def test(signum):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 22:02:28 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 22:02:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogY3R5cGVzOiBBSVgg?=
 =?utf-8?q?needs_an_explicit_=23include_=3Calloca=2Eh=3E_to_get_alloca=28?=
 =?utf-8?q?=29?=
Message-ID: <3bZ3HS2pvhzQxV@mail.python.org>

http://hg.python.org/cpython/rev/cffb497552b8
changeset:   84195:cffb497552b8
branch:      3.3
parent:      84193:a0e234e10da6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 22:01:53 2013 +0200
summary:
  ctypes: AIX needs an explicit #include <alloca.h> to get alloca()

files:
  Modules/_ctypes/callproc.c |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -70,6 +70,7 @@
 
 #include <ffi.h>
 #include "ctypes.h"
+#include <alloca.h>
 
 #if defined(_DEBUG) || defined(__MINGW32__)
 /* Don't use structured exception handling on Windows if this is defined.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 22:02:29 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 22:02:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=28Merge_3=2E3=29_ctypes=3A_AIX_needs_an_explicit_=23inc?=
 =?utf-8?q?lude_=3Calloca=2Eh=3E_to_get_alloca=28=29?=
Message-ID: <3bZ3HT4zkJzSyF@mail.python.org>

http://hg.python.org/cpython/rev/f6f70f1ab124
changeset:   84196:f6f70f1ab124
parent:      84194:8e277e6bd11b
parent:      84195:cffb497552b8
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 22:02:14 2013 +0200
summary:
  (Merge 3.3) ctypes: AIX needs an explicit #include <alloca.h> to get alloca()

files:
  Modules/_ctypes/callproc.c |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -70,6 +70,7 @@
 
 #include <ffi.h>
 #include "ctypes.h"
+#include <alloca.h>
 
 #if defined(_DEBUG) || defined(__MINGW32__)
 /* Don't use structured exception handling on Windows if this is defined.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 22:05:56 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 22:05:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogY3R5cGVzOiBBSVgg?=
 =?utf-8?q?needs_an_explicit_=23include_=3Calloca=2Eh=3E_to_get_alloca=28?=
 =?utf-8?q?=29?=
Message-ID: <3bZ3MS6Nm1zT1f@mail.python.org>

http://hg.python.org/cpython/rev/cf615d161f8f
changeset:   84197:cf615d161f8f
branch:      2.7
parent:      84181:63058453a4cc
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 22:03:35 2013 +0200
summary:
  ctypes: AIX needs an explicit #include <alloca.h> to get alloca()

files:
  Modules/_ctypes/callproc.c |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -75,6 +75,7 @@
 
 #include <ffi.h>
 #include "ctypes.h"
+#include <alloca.h>
 
 #if defined(_DEBUG) || defined(__MINGW32__)
 /* Don't use structured exception handling on Windows if this is defined.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 23:39:19 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 23:39:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogdGVzdF9mYXVsdGhh?=
 =?utf-8?b?bmRsZXI6IHVzZSBfc2lnc2VndigpIGluc3RlYWQgb2YgX3JlYWRfbnVsbCgp?=
Message-ID: <3bZ5RC0cg7z7LjP@mail.python.org>

http://hg.python.org/cpython/rev/f7441b9ef363
changeset:   84198:f7441b9ef363
branch:      3.3
parent:      84195:cffb497552b8
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 23:37:59 2013 +0200
summary:
  test_faulthandler: use _sigsegv() instead of _read_null()

faulthandler._read_null() is not reliable: it does not crash on AIX.

files:
  Lib/test/test_faulthandler.py |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -206,20 +206,20 @@
 import faulthandler
 output = open({filename}, 'wb')
 faulthandler.enable(output)
-faulthandler._read_null()
+faulthandler._sigsegv()
 """.strip().format(filename=repr(filename)),
                 4,
-                '(?:Segmentation fault|Bus error|Illegal instruction)',
+                'Segmentation fault',
                 filename=filename)
 
     def test_enable_single_thread(self):
         self.check_fatal_error("""
 import faulthandler
 faulthandler.enable(all_threads=False)
-faulthandler._read_null()
+faulthandler._sigsegv()
 """.strip(),
             3,
-            '(?:Segmentation fault|Bus error|Illegal instruction)',
+            'Segmentation fault',
             all_threads=False)
 
     def test_disable(self):
@@ -227,7 +227,7 @@
 import faulthandler
 faulthandler.enable()
 faulthandler.disable()
-faulthandler._read_null()
+faulthandler._sigsegv()
 """.strip()
         not_expected = 'Fatal Python error'
         with support.suppress_crash_popup():

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 23:39:20 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 17 Jun 2013 23:39:20 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=28Merge_3=2E3=29_test=5Ffaulthandler=3A_use_=5Fsigsegv?=
 =?utf-8?b?KCkgaW5zdGVhZCBvZiBfcmVhZF9udWxsKCk=?=
Message-ID: <3bZ5RD2VmBz7Ljk@mail.python.org>

http://hg.python.org/cpython/rev/e71b00ff5bae
changeset:   84199:e71b00ff5bae
parent:      84196:f6f70f1ab124
parent:      84198:f7441b9ef363
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 17 23:39:06 2013 +0200
summary:
  (Merge 3.3) test_faulthandler: use _sigsegv() instead of _read_null()

faulthandler._read_null() is not reliable: it does not crash on AIX.

files:
  Lib/test/test_faulthandler.py |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -206,20 +206,20 @@
 import faulthandler
 output = open({filename}, 'wb')
 faulthandler.enable(output)
-faulthandler._read_null()
+faulthandler._sigsegv()
 """.strip().format(filename=repr(filename)),
                 4,
-                '(?:Segmentation fault|Bus error|Illegal instruction)',
+                'Segmentation fault',
                 filename=filename)
 
     def test_enable_single_thread(self):
         self.check_fatal_error("""
 import faulthandler
 faulthandler.enable(all_threads=False)
-faulthandler._read_null()
+faulthandler._sigsegv()
 """.strip(),
             3,
-            '(?:Segmentation fault|Bus error|Illegal instruction)',
+            'Segmentation fault',
             all_threads=False)
 
     def test_disable(self):
@@ -227,7 +227,7 @@
 import faulthandler
 faulthandler.enable()
 faulthandler.disable()
-faulthandler._read_null()
+faulthandler._sigsegv()
 """.strip()
         not_expected = 'Fatal Python error'
         with support.suppress_crash_popup():

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 17 23:48:36 2013
From: python-checkins at python.org (brett.cannon)
Date: Mon, 17 Jun 2013 23:48:36 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317222=3A_fix_a_mi?=
 =?utf-8?q?x-up_in_some_exception_messages=2E?=
Message-ID: <3bZ5dw1l2Pz7Ljn@mail.python.org>

http://hg.python.org/cpython/rev/e2ccfb096e6a
changeset:   84200:e2ccfb096e6a
user:        Brett Cannon <brett at python.org>
date:        Mon Jun 17 17:48:30 2013 -0400
summary:
  Issue #17222: fix a mix-up in some exception messages.

Reported by Arfrever Frehtes Taifersar Arahesis.

files:
  Lib/py_compile.py |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Lib/py_compile.py b/Lib/py_compile.py
--- a/Lib/py_compile.py
+++ b/Lib/py_compile.py
@@ -112,11 +112,11 @@
     if os.path.islink(cfile):
         msg = ('{} is a symlink and will be changed into a regular file if '
                'import writes a byte-compiled file to it')
-        raise FileExistsError(msg.format(file, cfile))
+        raise FileExistsError(msg.format(cfile))
     elif os.path.exists(cfile) and not os.path.isfile(cfile):
         msg = ('{} is a non-regular file and will be changed into a regular '
                'one if import writes a byte-compiled file to it')
-        raise FileExistsError(msg.format(file, cfile))
+        raise FileExistsError(msg.format(cfile))
     loader = importlib.machinery.SourceFileLoader('<py_compile>', file)
     source_bytes = loader.get_data(file)
     try:

-- 
Repository URL: http://hg.python.org/cpython

From root at python.org  Tue Jun 18 00:15:22 2013
From: root at python.org (Cron Daemon)
Date: Tue, 18 Jun 2013 00:15:22 +0200
Subject: [Python-checkins] Cron <docs@dinsdale> /home/docs/build-devguide
Message-ID: <E1Uohhq-00046D-Em@dinsdale.python.org>

abort: error: Connection timed out

From python-checkins at python.org  Tue Jun 18 01:02:34 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 01:02:34 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_0445=3A_examples?=
Message-ID: <3bZ7HG15z3zRZ3@mail.python.org>

http://hg.python.org/peps/rev/9f1d6908bc70
changeset:   4935:9f1d6908bc70
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 01:02:16 2013 +0200
summary:
  PEP 0445: examples

files:
  pep-0445.txt                |  169 ++++++++++++++++++++---
  pep-0445/alloc_hooks.c      |  114 ++++++++++++++++
  pep-0445/replace_allocs.c   |   44 ++++++
  pep-0445/replace_pymalloc.c |   34 ++++
  4 files changed, 339 insertions(+), 22 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -40,25 +40,32 @@
 Proposal
 ========
 
+API changes
+-----------
+
 * Add a new ``PyMemAllocators`` structure
 
 * Add new GIL-free memory allocator functions:
 
-  - ``PyMem_RawMalloc()``
-  - ``PyMem_RawRealloc()``
-  - ``PyMem_RawFree()``
+  - ``void* PyMem_RawMalloc(size_t size)``
+  - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)``
+  - ``void PyMem_RawFree(void *ptr)``
 
 * Add new functions to get and set memory allocators:
 
-  - ``PyMem_GetRawAllocators()``, ``PyMem_SetRawAllocators()``
-  - ``PyMem_GetAllocators()``, ``PyMem_SetAllocators()``
-  - ``PyObject_GetAllocators()``, ``PyObject_SetAllocators()``
-  - ``_PyObject_GetArenaAllocators()``, ``_PyObject_SetArenaAllocators()``
+  - ``void PyMem_GetRawAllocators(PyMemAllocators *allocators)``
+  - ``void PyMem_SetRawAllocators(PyMemAllocators *allocators)``
+  - ``void PyMem_GetAllocators(PyMemAllocators *allocators)``
+  - ``void PyMem_SetAllocators(PyMemAllocators *allocators)``
+  - ``void PyObject_GetAllocators(PyMemAllocators *allocators)``
+  - ``void PyObject_SetAllocators(PyMemAllocators *allocators)``
+  - ``void _PyObject_GetArenaAllocators(void **ctx_p, void* (**malloc_p) (void *ctx, size_t size), void (**free_p) (void *ctx, void *ptr, size_t size))``
+  - ``void _PyObject_SetArenaAllocators(void *ctx, void* (*malloc) (void *ctx, size_t size), void (*free) (void *ctx, void *ptr, size_t size))``
 
 * Add a new function to setup debug hooks after memory allocators were
   replaced:
 
-  - ``PyMem_SetupDebugHooks()``
+  - ``void PyMem_SetupDebugHooks(void)``
 
 * ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call ``malloc()`` and
   ``realloc()``, instead of calling ``PyObject_Malloc()`` and
@@ -70,16 +77,140 @@
   ``realloc()``
 
 
+Examples
+========
+
+Use case 1: replace memory allocators, keeping pymalloc
+-------------------------------------------------------
+
+Setup your custom memory allocators, keeping pymalloc::
+
+    /* global variable, don't use a variable allocated on the stack! */
+    int magic = 42;
+
+    int my_malloc(void *ctx, size_t size);
+    int my_realloc(void *ctx, void *ptr, size_t new_size);
+    void my_free(void *ctx, void *ptr);
+
+    int my_alloc_arena(void *ctx, size_t size);
+    int my_free_arena(void *ctx, void *ptr, size_t size);
+
+    void setup_custom_allocators(void)
+    {
+        PyMemAllocators alloc;
+        alloc.ctx = &magic;
+        alloc.malloc = my_malloc;
+        alloc.realloc = my_realloc;
+        alloc.free = my_free;
+
+        PyMem_SetRawAllocators(&alloc);
+        PyMem_SetAllocators(&alloc);
+        _PyObject_SetArenaAllocators(&magic, my_alloc_arena, my_free_arena);
+
+        PyMem_SetupDebugHooks();
+    }
+
+.. warning::
+   Remove call ``PyMem_SetRawAllocators(&alloc);`` if the new allocators are
+   not thread-safe.
+
+Full example:
+`replace_allocs.c <http://hg.python.org/peps/file/tip/pep-0445/replace_allocs.c>`_.
+
+
+Use case 2: replace memory allocators, overriding pymalloc
+----------------------------------------------------------
+
+If your allocator is optimized for allocation of small objects (less than 512
+bytes) with a short liftime, you can replace override pymalloc (replace
+``PyObject_Malloc()``).  Example::
+
+    /* global variable, don't use a variable allocated on the stack! */
+    int magic = 42;
+
+    int my_malloc(void *ctx, size_t size);
+    int my_realloc(void *ctx, void *ptr, size_t new_size);
+    void my_free(void *ctx, void *ptr);
+
+    void setup_custom_allocators(void)
+    {
+        PyMemAllocators alloc;
+        alloc.ctx = &magic;
+        alloc.malloc = my_malloc;
+        alloc.realloc = my_realloc;
+        alloc.free = my_free;
+
+        PyMem_SetRawAllocators(&alloc);
+        PyMem_SetAllocators(&alloc);
+        PyObject_SetAllocators(&areana);
+        PyMem_SetupDebugHooks();
+    }
+
+Full example:
+`replace_pymalloc.c <http://hg.python.org/peps/file/tip/pep-0445/replace_pymalloc.c>`_.
+
+
+Use case 3: hook allocators
+---------------------------
+
+Setup hooks on memory allocators::
+
+    /* global variable, don't use a variable allocated on the stack! */
+    struct {
+        PyMemAllocators pymem;
+        PyMemAllocators pymem_raw;
+        PyMemAllocators pyobj;
+        int magic;
+    } hook;
+
+    int hook_malloc(void *ctx, size_t size);
+    int hook_realloc(void *ctx, void *ptr, size_t new_size);
+    void hook_free(void *ctx, void *ptr);
+
+    /* Must be called before the first allocation, or hook_realloc() and
+       hook_free() will crash */
+    void setup_custom_allocators(void)
+    {
+        PyMemAllocators alloc;
+
+        alloc.ctx = &magic;
+        alloc.malloc = hook_malloc;
+        alloc.realloc = hook_realloc;
+        alloc.free = hook_free;
+
+        PyMem_GetRawAllocators(&alloc.pymem_raw);
+        alloc.ctx = &alloc.pymem_raw;
+        PyMem_SetRawAllocators(&alloc);
+
+        PyMem_GetAllocators(&alloc.pymem);
+        alloc.ctx = &alloc.pymem;
+        PyMem_SetAllocators(&alloc);
+
+        PyObject_GetAllocators(&alloc.pyobj);
+        alloc.ctx = &alloc.pyobj;
+        PyObject_SetAllocators(&alloc);
+    }
+
+.. note::
+   No need to call ``PyMem_SetupDebugHooks()``: it is already installed by
+   default.
+
+Full example tracking memory usage:
+`alloc_hooks.c <http://hg.python.org/peps/file/tip/pep-0445/alloc_hooks.c>`_.
+
+
+
 Performances
 ============
 
-The Python benchmarks suite (-b 2n3): some tests are 1.04x faster, some tests
-are 1.04 slower, significant is between 115 and -191. I don't understand these
-output, but I guess that the overhead cannot be seen with such test.
+The `Python benchmarks suite <http://hg.python.org/benchmarks>`_ (-b 2n3): some
+tests are 1.04x faster, some tests are 1.04 slower, significant is between 115
+and -191. I don't understand these output, but I guess that the overhead cannot
+be seen with such test.
 
 pybench: "+0.1%" (diff between -4.9% and +5.6%).
 
-Full output attached to the issue #3329.
+The full output is attached to the issue #3329.
 
 
 Alternatives
@@ -163,6 +294,10 @@
 * glib: `g_mem_set_vtable()
   <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
 
+See also the `GNU libc: Memory Allocation Hooks
+<http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_.
+
+
 
 Memory allocators
 =================
@@ -225,13 +360,3 @@
 * `PySizer (developed for Python 2.4)
   <http://pysizer.8325.org/>`_
 
-APIs to set a custom memory allocator and/or hook memory allocators:
-
-* `GNU libc: Memory Allocation Hooks
-  <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_
-
-Other:
-
-* `Python benchmark suite
-  <http://hg.python.org/benchmarks>`_
-
diff --git a/pep-0445/alloc_hooks.c b/pep-0445/alloc_hooks.c
new file mode 100644
--- /dev/null
+++ b/pep-0445/alloc_hooks.c
@@ -0,0 +1,114 @@
+/* global variable, don't use a variable allocated on the stack! */
+struct {
+    PyMemAllocators pymem;
+    PyMemAllocators pymem_raw;
+    PyMemAllocators pyobj;
+    size_t allocated;
+} hook;
+
+/* read_size_t() and write_size_t() are not needed if malloc() and realloc()
+   always return a pointer aligned to sizeof(size_t) bytes */
+static size_t read_size_t(const void *p)
+{
+    const unsigned char *q = (const unsigned char *)p;
+    size_t result = *q++;
+    int i;
+
+    for (i = SST; --i > 0; ++q)
+        result = (result << 8) | *q;
+    return result;
+}
+
+static void write_size_t(void *p, size_t n)
+{
+    unsigned char *q = (unsigned char *)p + SST - 1;
+    int i;
+
+    for (i = SST; --i >= 0; --q) {
+        *q = (unsigned char)(n & 0xff);
+        n >>= 8;
+    }
+}
+
+static int hook_malloc(void *ctx, size_t size)
+{
+    PyMemAllocators *alloc;
+    char *ptr;
+
+    size += sizeof(size_t);
+    ptr = alloc->malloc(size);
+    if (ptr != NULL) {
+        write_size_t(ptr, size);
+        ptr += sizeof(size_t);
+        allocated += size;
+    }
+    return ptr;
+}
+
+static int hook_realloc(void *ctx, void *void_ptr, size_t new_size)
+{
+    PyMemAllocators *alloc;
+    char *ptr, *ptr2;
+    size_t old_size;
+
+    ptr = void_ptr;
+    if (ptr) {
+        ptr -= sizeof(size_t);
+        old_size = read_size_t(ptr);
+    }
+    else {
+        old_size = 0;
+    }
+
+    ptr2 = alloc->realloc(ptr, size);
+    if (ptr2 != NULL) {
+        write_size_t(ptr2, size);
+        ptr2 += sizeof(size_t);
+        allocated -= old_size;
+        allocated += new_size;
+    }
+    return ptr2;
+}
+
+static void hook_free(void *ctx, void *void_ptr)
+{
+    PyMemAllocators *alloc;
+    char *ptr;
+    size_t size;
+
+    ptr = void_ptr;
+    if (!ptr)
+        return;
+
+    ptr -= sizeof(size_t);
+    size = read_size_t(ptr);
+
+    alloc->free(ptr);
+    allocated -= size;
+}
+
+/* Must be called before the first allocation, or hook_realloc() and
+   hook_free() will crash */
+void setup_custom_allocators(void)
+{
+    PyMemAllocators alloc;
+
+    alloc.malloc = my_malloc;
+    alloc.realloc = my_realloc;
+    alloc.free = my_free;
+
+    /* disabled: the hook is not thread-safe */
+#if 0
+    PyMem_GetRawAllocators(&alloc.pymem_raw);
+    alloc.ctx = &alloc.pymem_raw;
+    PyMem_SetRawAllocators(&alloc);
+#endif
+
+    PyMem_GetAllocators(&alloc.pymem);
+    alloc.ctx = &alloc.pymem;
+    PyMem_SetAllocators(&alloc);
+
+    PyObject_GetAllocators(&alloc.pyobj);
+    alloc.ctx = &alloc.pyobj;
+    PyObject_SetAllocators(&alloc);
+}
diff --git a/pep-0445/replace_allocs.c b/pep-0445/replace_allocs.c
new file mode 100644
--- /dev/null
+++ b/pep-0445/replace_allocs.c
@@ -0,0 +1,44 @@
+#include <stdlib.h>
+
+/* global variable, don't use a variable allocated on the stack! */
+int magic = 42;
+
+int my_malloc(void *ctx, size_t size)
+{
+    return malloc(size);
+}
+
+int my_realloc(void *ctx, void *ptr, size_t new_size)
+{
+    return realloc(ptr, new_size);
+}
+
+void my_free(void *ctx, void *ptr)
+{
+    free(ptr);
+}
+
+int my_alloc_arena(void *ctx, size_t size)
+{
+    return malloc(size);
+}
+
+int my_free_arena(void *ctx, void *ptr, size_t size)
+{
+    free(ptr);
+}
+
+void setup_custom_allocators(void)
+{
+    PyMemAllocators alloc;
+    alloc.ctx = &magic;
+    alloc.malloc = my_malloc;
+    alloc.realloc = my_realloc;
+    alloc.free = my_free;
+
+    PyMem_SetRawAllocators(&alloc);
+    PyMem_SetAllocators(&alloc);
+    _PyObject_SetArenaAllocators(&magic, my_alloc_arena, my_free_arena);
+    PyMem_SetupDebugHooks();
+}
+
diff --git a/pep-0445/replace_pymalloc.c b/pep-0445/replace_pymalloc.c
new file mode 100644
--- /dev/null
+++ b/pep-0445/replace_pymalloc.c
@@ -0,0 +1,34 @@
+#include <stdlib.h>
+
+/* global variable, don't use a variable allocated on the stack! */
+int magic = 42;
+
+int my_malloc(void *ctx, size_t size)
+{
+    return malloc(size);
+}
+
+int my_realloc(void *ctx, void *ptr, size_t new_size)
+{
+    return realloc(ptr, new_size);
+}
+
+void my_free(void *ctx, void *ptr)
+{
+    free(ptr);
+}
+
+void setup_custom_allocators(void)
+{
+    PyMemAllocators alloc;
+    alloc.ctx = &magic;
+    alloc.malloc = my_malloc;
+    alloc.realloc = my_realloc;
+    alloc.free = my_free;
+
+    PyMem_SetRawAllocators(&alloc);
+    PyMem_SetAllocators(&alloc);
+    PyObject_SetAllocators(&areana);
+    PyMem_SetupDebugHooks();
+}
+

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 01:30:24 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 01:30:24 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_cleanup_and_inline?=
	=?utf-8?q?_examples?=
Message-ID: <3bZ7vN4G0hz7LjP@mail.python.org>

http://hg.python.org/peps/rev/14bdb8297ae8
changeset:   4936:14bdb8297ae8
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 01:30:05 2013 +0200
summary:
  PEP 445: cleanup and inline examples

files:
  pep-0445.txt                |  154 ++++++++++++++++-------
  pep-0445/alloc_hooks.c      |  114 -----------------
  pep-0445/replace_allocs.c   |   44 ------
  pep-0445/replace_pymalloc.c |   34 -----
  4 files changed, 107 insertions(+), 239 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -80,114 +80,178 @@
 Examples
 ========
 
-Use case 1: replace memory allocators, keeping pymalloc
--------------------------------------------------------
+Use case 1: Replace Memory Allocators, keep pymalloc
+----------------------------------------------------
 
 Setup your custom memory allocators, keeping pymalloc::
 
-    /* global variable, don't use a variable allocated on the stack! */
-    int magic = 42;
+    #include <stdlib.h>
 
-    int my_malloc(void *ctx, size_t size);
-    int my_realloc(void *ctx, void *ptr, size_t new_size);
-    void my_free(void *ctx, void *ptr);
+    int alloc_padding = 2;
+    int arena_padding = 10;
 
-    int my_alloc_arena(void *ctx, size_t size);
-    int my_free_arena(void *ctx, void *ptr, size_t size);
+    void* my_malloc(void *ctx, size_t size)
+    {
+        int padding = *(int *)ctx;
+        return malloc(size + padding);
+    }
+
+    void* my_realloc(void *ctx, void *ptr, size_t new_size)
+    {
+        int padding = *(int *)ctx;
+        return realloc(ptr, new_size + padding);
+    }
+
+    void my_free(void *ctx, void *ptr)
+    {
+        free(ptr);
+    }
+
+    void* my_alloc_arena(void *ctx, size_t size)
+    {
+        int padding = *(int *)ctx;
+        return malloc(size + padding);
+    }
+
+    void my_free_arena(void *ctx, void *ptr, size_t size)
+    {
+        free(ptr);
+    }
 
     void setup_custom_allocators(void)
     {
         PyMemAllocators alloc;
-        alloc.ctx = &magic;
+
+        alloc.ctx = &alloc_padding;
         alloc.malloc = my_malloc;
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
         PyMem_SetRawAllocators(&alloc);
         PyMem_SetAllocators(&alloc);
-        _PyObject_SetArenaAllocators(&magic, my_alloc_arena, my_free_arena);
+
+        _PyObject_SetArenaAllocators(&arena_padding,
+                                     my_alloc_arena, my_free_arena);
 
         PyMem_SetupDebugHooks();
     }
 
 .. warning::
-   Remove call ``PyMem_SetRawAllocators(&alloc);`` if the new allocators are
-   not thread-safe.
+   Remove the call ``PyMem_SetRawAllocators(&alloc);`` if the new allocators
+   are not thread-safe.
 
-Full example:
-`replace_allocs.c <http://hg.python.org/peps/file/tip/pep-0445/replace_allocs.c>`_.
 
-
-Use case 2: replace memory allocators, overriding pymalloc
+Use case 2: Replace Memory Allocators, overriding pymalloc
 ----------------------------------------------------------
 
 If your allocator is optimized for allocation of small objects (less than 512
 bytes) with a short liftime, you can replace override pymalloc (replace
 ``PyObject_Malloc()``).  Example::
 
-    /* global variable, don't use a variable allocated on the stack! */
-    int magic = 42;
+    #include <stdlib.h>
 
-    int my_malloc(void *ctx, size_t size);
-    int my_realloc(void *ctx, void *ptr, size_t new_size);
-    void my_free(void *ctx, void *ptr);
+    int padding = 2;
+
+    void* my_malloc(void *ctx, size_t size)
+    {
+        int padding = *(int *)ctx;
+        return malloc(size + padding);
+    }
+
+    void* my_realloc(void *ctx, void *ptr, size_t new_size)
+    {
+        int padding = *(int *)ctx;
+        return realloc(ptr, new_size + padding);
+    }
+
+    void my_free(void *ctx, void *ptr)
+    {
+        free(ptr);
+    }
 
     void setup_custom_allocators(void)
     {
         PyMemAllocators alloc;
-        alloc.ctx = &magic;
+        alloc.ctx = &padding;
         alloc.malloc = my_malloc;
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
         PyMem_SetRawAllocators(&alloc);
         PyMem_SetAllocators(&alloc);
-        PyObject_SetAllocators(&areana);
+        PyObject_SetAllocators(&alloc);
+
         PyMem_SetupDebugHooks();
     }
 
-Full example:
-`replace_pymalloc.c <http://hg.python.org/peps/file/tip/pep-0445/replace_pymalloc.c>`_.
+.. warning::
+   Remove the call ``PyMem_SetRawAllocators(&alloc);`` if the new allocators
+   are not thread-safe.
 
 
-Use case 3: hook allocators
----------------------------
+
+Use case 3: Setup Allocator Hooks
+---------------------------------
 
 Setup hooks on memory allocators::
 
-    /* global variable, don't use a variable allocated on the stack! */
     struct {
         PyMemAllocators pymem;
         PyMemAllocators pymem_raw;
         PyMemAllocators pyobj;
-        int magic;
+        /* ... */
     } hook;
 
-    int hook_malloc(void *ctx, size_t size);
-    int hook_realloc(void *ctx, void *ptr, size_t new_size);
-    void hook_free(void *ctx, void *ptr);
+    static void* hook_malloc(void *ctx, size_t size)
+    {
+        PyMemAllocators *alloc = (PyMemAllocators *)ctx;
+        /* ... */
+        ptr = alloc->malloc(alloc->ctx, size);
+        /* ... */
+        return ptr;
+    }
 
-    /* Must be called before the first allocation, or hook_realloc() and
-       hook_free() will crash */
-    void setup_custom_allocators(void)
+    static void* hook_realloc(void *ctx, void *ptr, size_t new_size)
+    {
+        PyMemAllocators *alloc = (PyMemAllocators *)ctx;
+        void *ptr2;
+        /* ... */
+        ptr2 = alloc->realloc(alloc->ctx, ptr, new_size);
+        /* ... */
+        return ptr2;
+    }
+
+    static void hook_free(void *ctx, void *ptr)
+    {
+        PyMemAllocators *alloc = (PyMemAllocators *)ctx;
+        /* ... */
+        alloc->free(alloc->ctx, ptr);
+        /* ... */
+    }
+
+    void setup_hooks(void)
     {
         PyMemAllocators alloc;
+        static int registered = 0;
 
-        alloc.ctx = &magic;
+        if (registered)
+            return;
+        registered = 1;
+
         alloc.malloc = hook_malloc;
         alloc.realloc = hook_realloc;
         alloc.free = hook_free;
 
-        PyMem_GetRawAllocators(&alloc.pymem_raw);
-        alloc.ctx = &alloc.pymem_raw;
+        PyMem_GetRawAllocators(&hook.pymem_raw);
+        alloc.ctx = &hook.pymem_raw;
         PyMem_SetRawAllocators(&alloc);
 
-        PyMem_GetAllocators(&alloc.pymem);
-        alloc.ctx = &alloc.pymem;
+        PyMem_GetAllocators(&hook.pymem);
+        alloc.ctx = &hook.pymem;
         PyMem_SetAllocators(&alloc);
 
-        PyObject_GetAllocators(&alloc.pyobj);
-        alloc.ctx = &alloc.pyobj;
+        PyObject_GetAllocators(&hook.pyobj);
+        alloc.ctx = &hook.pyobj;
         PyObject_SetAllocators(&alloc);
     }
 
@@ -195,10 +259,6 @@
    No need to call ``PyMem_SetupDebugHooks()``: it is already installed by
    default.
 
-Full example tracking memory usage:
-`alloc_hooks.c <http://hg.python.org/peps/file/tip/pep-0445/alloc_hooks.c>`_.
-
-
 
 Performances
 ============
diff --git a/pep-0445/alloc_hooks.c b/pep-0445/alloc_hooks.c
deleted file mode 100644
--- a/pep-0445/alloc_hooks.c
+++ /dev/null
@@ -1,114 +0,0 @@
-/* global variable, don't use a variable allocated on the stack! */
-struct {
-    PyMemAllocators pymem;
-    PyMemAllocators pymem_raw;
-    PyMemAllocators pyobj;
-    size_t allocated;
-} hook;
-
-/* read_size_t() and write_size_t() are not needed if malloc() and realloc()
-   always return a pointer aligned to sizeof(size_t) bytes */
-static size_t read_size_t(const void *p)
-{
-    const unsigned char *q = (const unsigned char *)p;
-    size_t result = *q++;
-    int i;
-
-    for (i = SST; --i > 0; ++q)
-        result = (result << 8) | *q;
-    return result;
-}
-
-static void write_size_t(void *p, size_t n)
-{
-    unsigned char *q = (unsigned char *)p + SST - 1;
-    int i;
-
-    for (i = SST; --i >= 0; --q) {
-        *q = (unsigned char)(n & 0xff);
-        n >>= 8;
-    }
-}
-
-static int hook_malloc(void *ctx, size_t size)
-{
-    PyMemAllocators *alloc;
-    char *ptr;
-
-    size += sizeof(size_t);
-    ptr = alloc->malloc(size);
-    if (ptr != NULL) {
-        write_size_t(ptr, size);
-        ptr += sizeof(size_t);
-        allocated += size;
-    }
-    return ptr;
-}
-
-static int hook_realloc(void *ctx, void *void_ptr, size_t new_size)
-{
-    PyMemAllocators *alloc;
-    char *ptr, *ptr2;
-    size_t old_size;
-
-    ptr = void_ptr;
-    if (ptr) {
-        ptr -= sizeof(size_t);
-        old_size = read_size_t(ptr);
-    }
-    else {
-        old_size = 0;
-    }
-
-    ptr2 = alloc->realloc(ptr, size);
-    if (ptr2 != NULL) {
-        write_size_t(ptr2, size);
-        ptr2 += sizeof(size_t);
-        allocated -= old_size;
-        allocated += new_size;
-    }
-    return ptr2;
-}
-
-static void hook_free(void *ctx, void *void_ptr)
-{
-    PyMemAllocators *alloc;
-    char *ptr;
-    size_t size;
-
-    ptr = void_ptr;
-    if (!ptr)
-        return;
-
-    ptr -= sizeof(size_t);
-    size = read_size_t(ptr);
-
-    alloc->free(ptr);
-    allocated -= size;
-}
-
-/* Must be called before the first allocation, or hook_realloc() and
-   hook_free() will crash */
-void setup_custom_allocators(void)
-{
-    PyMemAllocators alloc;
-
-    alloc.malloc = my_malloc;
-    alloc.realloc = my_realloc;
-    alloc.free = my_free;
-
-    /* disabled: the hook is not thread-safe */
-#if 0
-    PyMem_GetRawAllocators(&alloc.pymem_raw);
-    alloc.ctx = &alloc.pymem_raw;
-    PyMem_SetRawAllocators(&alloc);
-#endif
-
-    PyMem_GetAllocators(&alloc.pymem);
-    alloc.ctx = &alloc.pymem;
-    PyMem_SetAllocators(&alloc);
-
-    PyObject_GetAllocators(&alloc.pyobj);
-    alloc.ctx = &alloc.pyobj;
-    PyObject_SetAllocators(&alloc);
-}
diff --git a/pep-0445/replace_allocs.c b/pep-0445/replace_allocs.c
deleted file mode 100644
--- a/pep-0445/replace_allocs.c
+++ /dev/null
@@ -1,44 +0,0 @@
-#include <stdlib.h>
-
-/* global variable, don't use a variable allocated on the stack! */
-int magic = 42;
-
-int my_malloc(void *ctx, size_t size)
-{
-    return malloc(size);
-}
-
-int my_realloc(void *ctx, void *ptr, size_t new_size)
-{
-    return realloc(ptr, new_size);
-}
-
-void my_free(void *ctx, void *ptr)
-{
-    free(ptr);
-}
-
-int my_alloc_arena(void *ctx, size_t size)
-{
-    return malloc(size);
-}
-
-int my_free_arena(void *ctx, void *ptr, size_t size)
-{
-    free(ptr);
-}
-
-void setup_custom_allocators(void)
-{
-    PyMemAllocators alloc;
-    alloc.ctx = &magic;
-    alloc.malloc = my_malloc;
-    alloc.realloc = my_realloc;
-    alloc.free = my_free;
-
-    PyMem_SetRawAllocators(&alloc);
-    PyMem_SetAllocators(&alloc);
-    _PyObject_SetArenaAllocators(&magic, my_alloc_arena, my_free_arena);
-    PyMem_SetupDebugHooks();
-}
-
diff --git a/pep-0445/replace_pymalloc.c b/pep-0445/replace_pymalloc.c
deleted file mode 100644
--- a/pep-0445/replace_pymalloc.c
+++ /dev/null
@@ -1,34 +0,0 @@
-#include <stdlib.h>
-
-/* global variable, don't use a variable allocated on the stack! */
-int magic = 42;
-
-int my_malloc(void *ctx, size_t size)
-{
-    return malloc(size);
-}
-
-int my_realloc(void *ctx, void *ptr, size_t new_size)
-{
-    return realloc(ptr, new_size);
-}
-
-void my_free(void *ctx, void *ptr)
-{
-    free(ptr);
-}
-
-void setup_custom_allocators(void)
-{
-    PyMemAllocators alloc;
-    alloc.ctx = &magic;
-    alloc.malloc = my_malloc;
-    alloc.realloc = my_realloc;
-    alloc.free = my_free;
-
-    PyMem_SetRawAllocators(&alloc);
-    PyMem_SetAllocators(&alloc);
-    PyObject_SetAllocators(&areana);
-    PyMem_SetupDebugHooks();
-}
-

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 01:52:26 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 01:52:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZ8Np6F55zSdK@mail.python.org>

http://hg.python.org/peps/rev/0883803524a0
changeset:   4937:0883803524a0
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 01:52:14 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  45 +++++++++++++++++++++++++++++++++++----
  1 files changed, 40 insertions(+), 5 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -67,6 +67,10 @@
 
   - ``void PyMem_SetupDebugHooks(void)``
 
+
+Use these new APIs
+------------------
+
 * ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call ``malloc()`` and
   ``realloc()``, instead of calling ``PyObject_Malloc()`` and
   ``PyObject_Realloc()`` in debug mode
@@ -76,6 +80,12 @@
   ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of
   ``realloc()``
 
+* Replace direct calls to ``malloc()`` with ``PyMem_Malloc()``, or
+  ``PyMem_RawMalloc()`` if the GIL is not held
+
+* Configure external libraries like zlib or OpenSSL to use
+  ``PyMem_RawMalloc()``
+
 
 Examples
 ========
@@ -137,7 +147,7 @@
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocators(&alloc);`` if the new allocators
+   Remove the call ``PyMem_SetRawAllocators(&alloc)`` if the new allocators
    are not thread-safe.
 
 
@@ -185,7 +195,7 @@
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocators(&alloc);`` if the new allocators
+   Remove the call ``PyMem_SetRawAllocators(&alloc)`` if the new allocators
    are not thread-safe.
 
 
@@ -255,9 +265,13 @@
         PyObject_SetAllocators(&alloc);
     }
 
+.. warning::
+   Remove the call ``PyMem_SetRawAllocators(&alloc)`` if hooks are not
+   thread-safe.
+
 .. note::
-   No need to call ``PyMem_SetupDebugHooks()``: it is already installed by
-   default.
+   ``PyMem_SetupDebugHooks()`` does not need to be called: Python debug hooks
+   are installed automatically at startup.
 
 
 Performances
@@ -339,7 +353,28 @@
 PyMem_Malloc() GIL-free
 -----------------------
 
-There is no real reason to require the GIL when calling PyMem_Malloc().
+There is no real reason to require the GIL when calling ``PyMem_Malloc()``.
+
+Allowing to call ``PyMem_Malloc()`` without holding the GIL might break
+applications which setup their own allocator or their allocator hooks. Holding
+the GIL is very convinient to develop a custom allocator or a hook (no need to
+care of other threads, no need to handle mutexes, etc.).
+
+
+Don't add PyMem_RawMalloc()
+---------------------------
+
+Replace ``malloc()`` with ``PyMem_Malloc()``, but if the GIL is not held: keep
+``malloc()`` unchanged.
+
+The ``PyMem_Malloc()`` is sometimes already misused. For example, the
+``main()`` and ``Py_Main()`` functions of Python call ``PyMem_Malloc()``
+whereas the GIL do not exist yet. In this case, ``PyMem_Malloc()`` should
+be replaced with ``malloc()``.
+
+If an hook is used to the track memory usage, the ``malloc()`` memory will not
+be seen. Remaining ``malloc()`` may allocate a lot of memory and so would be
+missed in reports.
 
 
 CCP API

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 01:59:08 2013
From: python-checkins at python.org (brett.cannon)
Date: Tue, 18 Jun 2013 01:59:08 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Grammar_tweak?=
Message-ID: <3bZ8XX6GKMzRhj@mail.python.org>

http://hg.python.org/cpython/rev/d04444f21b73
changeset:   84201:d04444f21b73
user:        Brett Cannon <brett at python.org>
date:        Mon Jun 17 19:58:57 2013 -0400
summary:
  Grammar tweak

files:
  Doc/library/zipfile.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -144,7 +144,7 @@
    and should be :const:`ZIP_STORED`, :const:`ZIP_DEFLATED`,
    :const:`ZIP_BZIP2` or :const:`ZIP_LZMA`; unrecognized
    values will cause :exc:`RuntimeError` to be raised.  If :const:`ZIP_DEFLATED`,
-   :const:`ZIP_BZIP2` or :const:`ZIP_LZMA` is specified but the corresponded module
+   :const:`ZIP_BZIP2` or :const:`ZIP_LZMA` is specified but the corresponding module
    (:mod:`zlib`, :mod:`bz2` or :mod:`lzma`) is not available, :exc:`RuntimeError`
    is also raised. The default is :const:`ZIP_STORED`.  If *allowZip64* is
    ``True`` zipfile will create ZIP files that use the ZIP64 extensions when

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 18 02:02:44 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 02:02:44 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_background?=
Message-ID: <3bZ8ch18Gkz7LjR@mail.python.org>

http://hg.python.org/peps/rev/708d5a17a989
changeset:   4938:708d5a17a989
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 02:02:27 2013 +0200
summary:
  PEP 445: background

files:
  pep-0445.txt |  21 ++++++++++++++-------
  1 files changed, 14 insertions(+), 7 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -409,13 +409,20 @@
 
 The heap is handled by ``brk()`` and ``sbrk()`` system calls on Linux, and is
 contiguous.  Memory mappings are handled by ``mmap()`` on UNIX and
-``VirtualAlloc()`` on Windows, they are discontiguous. Releasing a memory
-mapping gives back the memory immediatly to the system. For the heap, memory is
-only gave back to the system if it is at the end of the heap. Otherwise, the
-memory will only gave back to the system when all the memory located after the
-released memory are also released. This limitation causes an issue called the
-"memory fragmentation": the memory usage seen by the system may be much higher
-than real usage.
+``VirtualAlloc()`` on Windows, they may be discontiguous.
+
+Releasing a memory mapping gives back immediatly the memory to the system. For
+the heap, memory is only given back to the system if it is at the end of the
+heap. Otherwise, the memory will only be given back to the system when all the
+memory located after the released memory are also released. To allocate memory
+in the heap, the allocator tries to reuse free space. If there is no contiguous
+space big enough, the heap must be increased, even if we have more free space
+than required size.  This issue is called the "memory fragmentation": the
+memory usage seen by the system may be much higher than real usage.
+
+CPython has a pymalloc allocator using arenas of 256 KB for allocations smaller
+than 512 bytes. This allocator is optimized for small objects with a short
+lifetime.
 
 Windows provides a `Low-fragmentation Heap
 <http://msdn.microsoft.com/en-us/library/windows/desktop/aa366750%28v=vs.85%29.aspx>`_.

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 02:46:23 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 02:46:23 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZ9b35PQbz7Ljh@mail.python.org>

http://hg.python.org/peps/rev/0971c633e30b
changeset:   4939:0971c633e30b
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 02:46:10 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  70 +++++++++++++++++++++++++--------------
  1 files changed, 45 insertions(+), 25 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -12,7 +12,7 @@
 Abstract
 ========
 
-Add new APIs to customize memory allocators
+Add new APIs to customize memory allocators.
 
 
 Rationale
@@ -20,15 +20,19 @@
 
 Use cases:
 
-* Application embedding Python wanting to use a custom memory allocator
-  to allocate all Python memory somewhere else or with a different algorithm
+* Application embedding Python may want to isolate Python memory from the
+  memory of the application, or may want to different memory allocator
+  optimized for its Python usage
 * Python running on embedded devices with low memory and slow CPU.
   A custom memory allocator may be required to use efficiently the memory
   and/or to be able to use all memory of the device.
-* Debug tool to track memory leaks
-* Debug tool to detect buffer underflow, buffer overflow and misuse
-  of Python allocator APIs
-* Debug tool to inject bugs, simulate out of memory for example
+* Debug tool to:
+
+  - track memory leaks
+  - get the Python filename and line number where an object was allocated
+  - detect buffer underflow, buffer overflow and detect misuse of Python
+    allocator APIs (builtin Python debug hooks)
+  - force allocation to fail to test handling of ``MemoryError`` exception
 
 API:
 
@@ -62,8 +66,8 @@
   - ``void _PyObject_GetArenaAllocators(void **ctx_p, void* (**malloc_p) (void *ctx, size_t size), void (**free_p) (void *ctx, void *ptr, size_t size))``
   - ``void _PyObject_SetArenaAllocators(void *ctx, void* (*malloc) (void *ctx, size_t size), void (*free) (void *ctx, void *ptr, size_t size))``
 
-* Add a new function to setup debug hooks after memory allocators were
-  replaced:
+* Add a new function to setup Python builtin debug hooks when memory
+  allocators are replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
 
@@ -71,19 +75,19 @@
 Use these new APIs
 ------------------
 
-* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call ``malloc()`` and
+* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()`` and
   ``realloc()``, instead of calling ``PyObject_Malloc()`` and
   ``PyObject_Realloc()`` in debug mode
 
-* ``PyObject_Malloc()`` now falls back on ``PyMem_Malloc()`` instead of
-  ``malloc()`` if size is bigger than ``SMALL_REQUEST_THRESHOLD``, and
-  ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of
-  ``realloc()``
+* ``PyObject_Malloc()`` falls back on ``PyMem_Malloc()`` instead of
+  ``malloc()`` if size is greater or equal than ``SMALL_REQUEST_THRESHOLD``
+  (512 bytes), and ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()``
+  instead of ``realloc()``
 
 * Replace direct calls to ``malloc()`` with ``PyMem_Malloc()``, or
   ``PyMem_RawMalloc()`` if the GIL is not held
 
-* Configure external libraries like zlib or OpenSSL to use
+* Configure external libraries like zlib or OpenSSL to allocate memory using
   ``PyMem_RawMalloc()``
 
 
@@ -93,7 +97,8 @@
 Use case 1: Replace Memory Allocators, keep pymalloc
 ----------------------------------------------------
 
-Setup your custom memory allocators, keeping pymalloc::
+Setup your custom memory allocators, keeping pymalloc. Dummy example wasting 2
+bytes per allocation, and 10 bytes per arena::
 
     #include <stdlib.h>
 
@@ -156,7 +161,9 @@
 
 If your allocator is optimized for allocation of small objects (less than 512
 bytes) with a short liftime, you can replace override pymalloc (replace
-``PyObject_Malloc()``).  Example::
+``PyObject_Malloc()``).
+
+Dummy Example wasting 2 bytes per allocation::
 
     #include <stdlib.h>
 
@@ -203,7 +210,7 @@
 Use case 3: Setup Allocator Hooks
 ---------------------------------
 
-Setup hooks on memory allocators::
+Example to setup hooks on memory allocators::
 
     struct {
         PyMemAllocators pymem;
@@ -314,13 +321,20 @@
 * ``PYALLOC_PYOBJECT``
 
 
-Setup Builtin Debug Hooks
--------------------------
+Add a new PYDEBUGMALLOC environment variable
+--------------------------------------------
 
-To be able to use Python debug functions (like ``_PyMem_DebugMalloc()``) even
-when a custom memory allocator is set, an environment variable
-``PYDEBUGMALLOC`` can be added to set these debug function hooks, instead of
-the new function ``PyMem_SetupDebugHooks()``.
+To be able to use Python builtin debug hooks even when a custom memory
+allocator is set, an environment variable ``PYDEBUGMALLOC`` can be added to
+setup these debug function hooks, instead of adding the new function
+``PyMem_SetupDebugHooks()``. If the environment variable is present,
+``PyMem_SetRawAllocators()``, ``PyMem_SetAllocators()`` and
+``PyObject_SetAllocators()`` will reinstall automatically the hook on top of
+the new allocator.
+
+An new environment variable would make the Python initialization even more
+complex. The `PEP 432 <http://www.python.org/dev/peps/pep-0432/>`_ tries to
+simply the CPython startup sequence.
 
 
 Use macros to get customizable allocators
@@ -336,6 +350,12 @@
 Use C macros using ``__FILE__`` and ``__LINE__`` to get the C filename
 and line number of a memory allocation.
 
+Passing a filename and a line number to each allocator makes the API more
+complex: pass 3 new arguments instead of just a context argument, to each
+allocator function. GC allocator functions should also be patched,
+``_PyObject_GC_Malloc()`` is used in many C functions for example. Such changes
+add too much complexity, for a little gain.
+
 
 No context argument
 -------------------
@@ -446,7 +466,7 @@
   <http://bugs.python.org/issue13483>`_
 * `Issue #16742: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which
   isn't thread safe <http://bugs.python.org/issue16742>`_
-* `Issue #18203: Replace calls to malloc() with PyMem_Malloc()
+* `Issue #18203: Replace calls to malloc() with PyMem_Malloc() or PyMem_RawMalloc()
   <http://bugs.python.org/issue18203>`_
 * `Issue #18227: Use Python memory allocators in external libraries like zlib
   or OpenSSL <http://bugs.python.org/issue18227>`_

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 03:00:27 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 03:00:27 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZ9vH4ZYrz7Ljk@mail.python.org>

http://hg.python.org/peps/rev/37b3159ea506
changeset:   4940:37b3159ea506
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 03:00:17 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  11 +++++++++++
  1 files changed, 11 insertions(+), 0 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -343,6 +343,9 @@
 To have no overhead in the default configuration, customizable allocators would
 be an optional feature enabled by a configuration option or by macros.
 
+Not having to recompile Python makes debug hooks easy to use in practice.
+Extensions modules don't have to be compiled with or without macros.
+
 
 Pass the C filename and line number
 -----------------------------------
@@ -408,6 +411,14 @@
 
 * glib: `g_mem_set_vtable()
   <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
+* OpenSSL: `CRYPTO_set_mem_functions()
+  <http://git.openssl.org/gitweb/?p=openssl.git;a=blob;f=crypto/mem.c;h=f7984fa958eb1edd6c61f6667f3f2b29753be662;hb=HEAD#l124>`_
+  to set memory management functions globally
+* expat: `parserCreate()
+  <http://hg.python.org/cpython/file/cc27d50bd91a/Modules/expat/xmlparse.c#l724>`_
+  has a per-instance memory handler
+* libxml2: `xmlGcMemSetup() <http://xmlsoft.org/html/libxml-xmlmemory.html>`_,
+  global
 
 See also the `GNU libc: Memory Allocation Hooks
 <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_.

-- 
Repository URL: http://hg.python.org/peps

From solipsis at pitrou.net  Tue Jun 18 05:51:54 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 18 Jun 2013 05:51:54 +0200
Subject: [Python-checkins] Daily reference leaks (d04444f21b73): sum=1
Message-ID: <E1UomxW-0004z5-Cu@ap.vmr.nerim.net>

results for d04444f21b73 on branch "default"
--------------------------------------------

test_unittest leaked [-1, 2, 0] memory blocks, sum=1


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog9o1OBM', '-x']

From jeremy.kloth at gmail.com  Tue Jun 18 12:56:37 2013
From: jeremy.kloth at gmail.com (Jeremy Kloth)
Date: Tue, 18 Jun 2013 04:56:37 -0600
Subject: [Python-checkins] cpython (3.3): ctypes: AIX needs an explicit
 #include <alloca.h> to get alloca()
In-Reply-To: <3bZ3HS2pvhzQxV@mail.python.org>
References: <3bZ3HS2pvhzQxV@mail.python.org>
Message-ID: <CAGvrs3JXKUB5eRcNsTn+FC_PvVCau1-O4ajDHY2YPp652-iNTA@mail.gmail.com>

On Mon, Jun 17, 2013 at 2:02 PM, victor.stinner
<python-checkins at python.org> wrote:
> diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
> --- a/Modules/_ctypes/callproc.c
> +++ b/Modules/_ctypes/callproc.c
> @@ -70,6 +70,7 @@
>
>  #include <ffi.h>
>  #include "ctypes.h"
> +#include <alloca.h>

This header is not present on Windows, thus breaking all the Windows
buildbots.  Perhaps it should be wrapped in an AIX-specific #ifdef?

-- 
Jeremy Kloth

From python-checkins at python.org  Tue Jun 18 13:25:35 2013
From: python-checkins at python.org (christian.heimes)
Date: Tue, 18 Jun 2013 13:25:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fix_for_r84195?=
 =?utf-8?q?=3A_add_HAVE=5FALLOCA=5FH_to_configure_and_only_include_alloca?=
 =?utf-8?q?=2Eh_if?=
Message-ID: <3bZRmb5Wqfz7LjW@mail.python.org>

http://hg.python.org/cpython/rev/7b6ae19dd116
changeset:   84202:7b6ae19dd116
branch:      3.3
parent:      84198:f7441b9ef363
user:        Christian Heimes <christian at cheimes.de>
date:        Tue Jun 18 13:22:17 2013 +0200
summary:
  Fix for r84195: add HAVE_ALLOCA_H to configure and only include alloca.h if it's available

files:
  Modules/_ctypes/callproc.c |  3 +++
  configure                  |  2 +-
  configure.ac               |  2 +-
  pyconfig.h.in              |  3 +++
  4 files changed, 8 insertions(+), 2 deletions(-)


diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -70,7 +70,10 @@
 
 #include <ffi.h>
 #include "ctypes.h"
+#ifdef HAVE_ALLOCA_H
+/* AIX needs alloca.h for alloca() */
 #include <alloca.h>
+#endif
 
 #if defined(_DEBUG) || defined(__MINGW32__)
 /* Don't use structured exception handling on Windows if this is defined.
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -6928,7 +6928,7 @@
 sys/stat.h sys/syscall.h sys/sys_domain.h sys/termio.h sys/time.h \
 sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \
 libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \
-bluetooth/bluetooth.h linux/tipc.h spawn.h util.h
+bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h
 do :
   as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
 ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1526,7 +1526,7 @@
 sys/stat.h sys/syscall.h sys/sys_domain.h sys/termio.h sys/time.h \
 sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \
 libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \
-bluetooth/bluetooth.h linux/tipc.h spawn.h util.h)
+bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h)
 CPPFLAGS=$ac_save_cppflags
 AC_HEADER_DIRENT
 AC_HEADER_MAJOR
diff --git a/pyconfig.h.in b/pyconfig.h.in
--- a/pyconfig.h.in
+++ b/pyconfig.h.in
@@ -49,6 +49,9 @@
 /* Define to 1 if you have the `alarm' function. */
 #undef HAVE_ALARM
 
+/* Define to 1 if you have the <alloca.h> header file. */
+#undef HAVE_ALLOCA_H
+
 /* Define this if your time.h defines altzone. */
 #undef HAVE_ALTZONE
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 18 13:25:37 2013
From: python-checkins at python.org (christian.heimes)
Date: Tue, 18 Jun 2013 13:25:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Fix_for_r84195=3A_add_HAVE=5FALLOCA=5FH_to_configure_and?=
 =?utf-8?q?_only_include_alloca=2Eh_if?=
Message-ID: <3bZRmd0gq1z7LjX@mail.python.org>

http://hg.python.org/cpython/rev/390e7fe9a1d3
changeset:   84203:390e7fe9a1d3
parent:      84201:d04444f21b73
parent:      84202:7b6ae19dd116
user:        Christian Heimes <christian at cheimes.de>
date:        Tue Jun 18 13:22:37 2013 +0200
summary:
  Fix for r84195: add HAVE_ALLOCA_H to configure and only include alloca.h if it's available

files:
  Modules/_ctypes/callproc.c |  3 +++
  configure                  |  2 +-
  configure.ac               |  2 +-
  pyconfig.h.in              |  3 +++
  4 files changed, 8 insertions(+), 2 deletions(-)


diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -70,7 +70,10 @@
 
 #include <ffi.h>
 #include "ctypes.h"
+#ifdef HAVE_ALLOCA_H
+/* AIX needs alloca.h for alloca() */
 #include <alloca.h>
+#endif
 
 #if defined(_DEBUG) || defined(__MINGW32__)
 /* Don't use structured exception handling on Windows if this is defined.
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -6937,7 +6937,7 @@
 sys/stat.h sys/syscall.h sys/sys_domain.h sys/termio.h sys/time.h \
 sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \
 libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \
-bluetooth/bluetooth.h linux/tipc.h spawn.h util.h
+bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h
 do :
   as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
 ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1532,7 +1532,7 @@
 sys/stat.h sys/syscall.h sys/sys_domain.h sys/termio.h sys/time.h \
 sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \
 libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \
-bluetooth/bluetooth.h linux/tipc.h spawn.h util.h)
+bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h)
 CPPFLAGS=$ac_save_cppflags
 AC_HEADER_DIRENT
 AC_HEADER_MAJOR
diff --git a/pyconfig.h.in b/pyconfig.h.in
--- a/pyconfig.h.in
+++ b/pyconfig.h.in
@@ -49,6 +49,9 @@
 /* Define to 1 if you have the `alarm' function. */
 #undef HAVE_ALARM
 
+/* Define to 1 if you have the <alloca.h> header file. */
+#undef HAVE_ALLOCA_H
+
 /* Define this if your time.h defines altzone. */
 #undef HAVE_ALTZONE
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 18 13:25:38 2013
From: python-checkins at python.org (christian.heimes)
Date: Tue, 18 Jun 2013 13:25:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_for_r84195?=
 =?utf-8?q?=3A_add_HAVE=5FALLOCA=5FH_to_configure_and_only_include_alloca?=
 =?utf-8?q?=2Eh_if?=
Message-ID: <3bZRmf2l77z7Ljx@mail.python.org>

http://hg.python.org/cpython/rev/4f4433b12757
changeset:   84204:4f4433b12757
branch:      2.7
parent:      84197:cf615d161f8f
user:        Christian Heimes <christian at cheimes.de>
date:        Tue Jun 18 13:25:24 2013 +0200
summary:
  Fix for r84195: add HAVE_ALLOCA_H to configure and only include alloca.h if it's available

files:
  Modules/_ctypes/callproc.c |  3 +++
  configure                  |  2 +-
  configure.ac               |  2 +-
  pyconfig.h.in              |  3 +++
  4 files changed, 8 insertions(+), 2 deletions(-)


diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
--- a/Modules/_ctypes/callproc.c
+++ b/Modules/_ctypes/callproc.c
@@ -75,7 +75,10 @@
 
 #include <ffi.h>
 #include "ctypes.h"
+#ifdef HAVE_ALLOCA_H
+/* AIX needs alloca.h for alloca() */
 #include <alloca.h>
+#endif
 
 #if defined(_DEBUG) || defined(__MINGW32__)
 /* Don't use structured exception handling on Windows if this is defined.
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -6654,7 +6654,7 @@
 sys/termio.h sys/time.h \
 sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \
 sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \
-bluetooth/bluetooth.h linux/tipc.h spawn.h util.h
+bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h
 do :
   as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
 ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -1519,7 +1519,7 @@
 sys/termio.h sys/time.h \
 sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \
 sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \
-bluetooth/bluetooth.h linux/tipc.h spawn.h util.h)
+bluetooth/bluetooth.h linux/tipc.h spawn.h util.h alloca.h)
 AC_HEADER_DIRENT
 AC_HEADER_MAJOR
 
diff --git a/pyconfig.h.in b/pyconfig.h.in
--- a/pyconfig.h.in
+++ b/pyconfig.h.in
@@ -55,6 +55,9 @@
 /* Define to 1 if you have the `alarm' function. */
 #undef HAVE_ALARM
 
+/* Define to 1 if you have the <alloca.h> header file. */
+#undef HAVE_ALLOCA_H
+
 /* Define this if your time.h defines altzone. */
 #undef HAVE_ALTZONE
 

-- 
Repository URL: http://hg.python.org/cpython

From christian at python.org  Tue Jun 18 13:28:13 2013
From: christian at python.org (Christian Heimes)
Date: Tue, 18 Jun 2013 13:28:13 +0200
Subject: [Python-checkins] cpython (3.3): ctypes: AIX needs an explicit
 #include <alloca.h> to get alloca()
In-Reply-To: <CAGvrs3JXKUB5eRcNsTn+FC_PvVCau1-O4ajDHY2YPp652-iNTA@mail.gmail.com>
References: <3bZ3HS2pvhzQxV@mail.python.org>
	<CAGvrs3JXKUB5eRcNsTn+FC_PvVCau1-O4ajDHY2YPp652-iNTA@mail.gmail.com>
Message-ID: <51C0444D.8030402@python.org>

Am 18.06.2013 12:56, schrieb Jeremy Kloth:
> On Mon, Jun 17, 2013 at 2:02 PM, victor.stinner
> <python-checkins at python.org> wrote:
>> diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c
>> --- a/Modules/_ctypes/callproc.c
>> +++ b/Modules/_ctypes/callproc.c
>> @@ -70,6 +70,7 @@
>>
>>  #include <ffi.h>
>>  #include "ctypes.h"
>> +#include <alloca.h>
> 
> This header is not present on Windows, thus breaking all the Windows
> buildbots.  Perhaps it should be wrapped in an AIX-specific #ifdef?

I have added HAVE_ALLOCA_H to configure:

http://hg.python.org/cpython/rev/7b6ae19dd116

Christian

From christian at python.org  Tue Jun 18 13:45:05 2013
From: christian at python.org (Christian Heimes)
Date: Tue, 18 Jun 2013 13:45:05 +0200
Subject: [Python-checkins] cpython (3.3): ctypes: AIX needs an explicit
 #include <alloca.h> to get alloca()
In-Reply-To: <CAMpsgwb=5Gtc2MHUhfZC5trofVXxJASCbaZQC=Oz7K-aG4GdBg@mail.gmail.com>
References: <3bZ3HS2pvhzQxV@mail.python.org>
	<CAGvrs3JXKUB5eRcNsTn+FC_PvVCau1-O4ajDHY2YPp652-iNTA@mail.gmail.com>
	<51C0444D.8030402@python.org>
	<CAMpsgwb=5Gtc2MHUhfZC5trofVXxJASCbaZQC=Oz7K-aG4GdBg@mail.gmail.com>
Message-ID: <51C04841.8080302@python.org>

Am 18.06.2013 13:32, schrieb Victor Stinner:
> 2013/6/18 Christian Heimes <christian at python.org>:
>> Am 18.06.2013 12:56, schrieb Jeremy Kloth:
>>> On Mon, Jun 17, 2013 at 2:02 PM, victor.stinner
>>>> +#include <alloca.h>
>>>
>>> This header is not present on Windows, thus breaking all the Windows
>>> buildbots.  Perhaps it should be wrapped in an AIX-specific #ifdef?
> 
> Oh really? Portability is complex :-) I only tested on Linux, but I
> expected this header to be part of the C standard...

It's neither C99 nor POSIX.1-2001. I guess it's just too fragile and not
portable enough. http://c-faq.com/malloc/alloca.html

> 
>> I have added HAVE_ALLOCA_H to configure:
>>
>> http://hg.python.org/cpython/rev/7b6ae19dd116
> 
> Cool, thanks.

You are welcome. alloca() is documented to require <alloca.h>. It merely
works with GCC because the compiler translates the function call to
inline code.

Christian


From python-checkins at python.org  Tue Jun 18 14:17:58 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 14:17:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZSx20myBz7LjR@mail.python.org>

http://hg.python.org/peps/rev/7cc36550c084
changeset:   4941:7cc36550c084
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 14:14:17 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  168 +++++++++++++++++++++++++-------------
  1 files changed, 110 insertions(+), 58 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -34,12 +34,6 @@
     allocator APIs (builtin Python debug hooks)
   - force allocation to fail to test handling of ``MemoryError`` exception
 
-API:
-
-* Setup a custom memory allocator for all memory allocated by Python
-* Hook memory allocator functions to call extra code before and/or after
-  the underlying allocator function
-
 
 Proposal
 ========
@@ -47,15 +41,29 @@
 API changes
 -----------
 
-* Add a new ``PyMemAllocators`` structure
-
 * Add new GIL-free memory allocator functions:
 
   - ``void* PyMem_RawMalloc(size_t size)``
   - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)``
   - ``void PyMem_RawFree(void *ptr)``
 
-* Add new functions to get and set memory allocators:
+* Add a new ``PyMemAllocators`` structure::
+
+    typedef struct {
+        /* user context passed as the first argument to the 3 functions */
+        void *ctx;
+
+        /* allocate memory */
+        void* (*malloc) (void *ctx, size_t size);
+
+        /* allocate memory or resize a memory buffer */
+        void* (*realloc) (void *ctx, void *ptr, size_t new_size);
+
+        /* release memory */
+        void (*free) (void *ctx, void *ptr);
+    } PyMemAllocators;
+
+* Add new functions to get and set memory block allocators:
 
   - ``void PyMem_GetRawAllocators(PyMemAllocators *allocators)``
   - ``void PyMem_SetRawAllocators(PyMemAllocators *allocators)``
@@ -63,17 +71,32 @@
   - ``void PyMem_SetAllocators(PyMemAllocators *allocators)``
   - ``void PyObject_GetAllocators(PyMemAllocators *allocators)``
   - ``void PyObject_SetAllocators(PyMemAllocators *allocators)``
+
+* Add new functions to get and set memory mapping allocators:
+
   - ``void _PyObject_GetArenaAllocators(void **ctx_p, void* (**malloc_p) (void *ctx, size_t size), void (**free_p) (void *ctx, void *ptr, size_t size))``
   - ``void _PyObject_SetArenaAllocators(void *ctx, void* (*malloc) (void *ctx, size_t size), void (*free) (void *ctx, void *ptr, size_t size))``
 
-* Add a new function to setup Python builtin debug hooks when memory
+* Add a new function to setup the builtin Python debug hooks when memory
   allocators are replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
 
+.. note::
 
-Use these new APIs
-------------------
+   The builtin Python debug hooks were introduced in Python 2.3 and implement the
+   following checks:
+
+   * Newly allocated memory is filled with the byte 0xCB, freed memory is filled
+     with the byte 0xDB.
+   * Detect API violations, ex: ``PyObject_Free()`` called on a memory block
+     allocated by ``PyMem_Malloc()``
+   * Detect write before the start of the buffer (buffer underflow)
+   * Detect write after the end of the buffer (buffer overflow)
+
+
+Make usage of these new APIs
+----------------------------
 
 * ``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()`` and
   ``realloc()``, instead of calling ``PyObject_Malloc()`` and
@@ -156,12 +179,12 @@
    are not thread-safe.
 
 
-Use case 2: Replace Memory Allocators, overriding pymalloc
-----------------------------------------------------------
+Use case 2: Replace Memory Allocators, override pymalloc
+--------------------------------------------------------
 
 If your allocator is optimized for allocation of small objects (less than 512
-bytes) with a short liftime, you can replace override pymalloc (replace
-``PyObject_Malloc()``).
+bytes) with a short lifetime, pymalloc can be overriden: replace
+``PyObject_Malloc()``.
 
 Dummy Example wasting 2 bytes per allocation::
 
@@ -210,7 +233,7 @@
 Use case 3: Setup Allocator Hooks
 ---------------------------------
 
-Example to setup hooks on memory allocators::
+Example to setup hooks on all memory allocators::
 
     struct {
         PyMemAllocators pymem;
@@ -249,11 +272,11 @@
     void setup_hooks(void)
     {
         PyMemAllocators alloc;
-        static int registered = 0;
+        static int installed = 0;
 
-        if (registered)
+        if (installed)
             return;
-        registered = 1;
+        installed = 1;
 
         alloc.malloc = hook_malloc;
         alloc.realloc = hook_realloc;
@@ -284,30 +307,27 @@
 Performances
 ============
 
-The `Python benchmarks suite <http://hg.python.org/benchmarks>`_ (-b 2n3): some
-tests are 1.04x faster, some tests are 1.04 slower, significant is between 115
-and -191. I don't understand these output, but I guess that the overhead cannot
-be seen with such test.
+Results of the `Python benchmarks suite <http://hg.python.org/benchmarks>`_ (-b
+2n3): some tests are 1.04x faster, some tests are 1.04 slower, significant is
+between 115 and -191. I don't understand these output, but I guess that the
+overhead cannot be seen with such test.
 
-pybench: "+0.1%" (diff between -4.9% and +5.6%).
+Results of pybench benchmark: "+0.1%" slower globally (diff between -4.9% and
++5.6%).
 
-The full output is attached to the issue #3329.
+The full reports are attached to the issue #3329.
 
 
 Alternatives
 ============
 
-Only one get and one set function
----------------------------------
+Only have one generic get/set function
+--------------------------------------
 
 Replace the 6 functions:
 
-* ``PyMem_GetRawAllocators()``
-* ``PyMem_GetAllocators()``
-* ``PyObject_GetAllocators()``
-* ``PyMem_SetRawAllocators(allocators)``
-* ``PyMem_SetAllocators(allocators)``
-* ``PyObject_SetAllocators(allocators)``
+* ``PyMem_GetRawAllocators()``, ``PyMem_GetAllocators()``, ``PyObject_GetAllocators()``
+* ``PyMem_SetRawAllocators(allocators)``, ``PyMem_SetAllocators(allocators)``, ``PyObject_SetAllocators(allocators)``
 
 with 2 functions with an additional *domain* argument:
 
@@ -321,16 +341,21 @@
 * ``PYALLOC_PYOBJECT``
 
 
+``_PyObject_GetArenaAllocators()`` and ``_PyObject_SetArenaAllocators()`` are
+not merged and kept private because their prototypes are different and they are
+specific to pymalloc.
+
+
 Add a new PYDEBUGMALLOC environment variable
 --------------------------------------------
 
-To be able to use Python builtin debug hooks even when a custom memory
-allocator is set, an environment variable ``PYDEBUGMALLOC`` can be added to
-setup these debug function hooks, instead of adding the new function
-``PyMem_SetupDebugHooks()``. If the environment variable is present,
-``PyMem_SetRawAllocators()``, ``PyMem_SetAllocators()`` and
-``PyObject_SetAllocators()`` will reinstall automatically the hook on top of
-the new allocator.
+To be able to use the Python builtin debug hooks even when a custom memory
+allocator replaces the default Python allocator, an environment variable
+``PYDEBUGMALLOC`` can be added to setup these debug function hooks, instead of
+adding the new function ``PyMem_SetupDebugHooks()``. If the environment
+variable is present, ``PyMem_SetRawAllocators()``, ``PyMem_SetAllocators()``
+and ``PyObject_SetAllocators()`` will reinstall automatically the hook on top
+of the new allocator.
 
 An new environment variable would make the Python initialization even more
 complex. The `PEP 432 <http://www.python.org/dev/peps/pep-0432/>`_ tries to
@@ -343,8 +368,8 @@
 To have no overhead in the default configuration, customizable allocators would
 be an optional feature enabled by a configuration option or by macros.
 
-Not having to recompile Python makes debug hooks easy to use in practice.
-Extensions modules don't have to be compiled with or without macros.
+Not having to recompile Python makes debug hooks easier to use in practice.
+Extensions modules don't have to be recompiled with macros.
 
 
 Pass the C filename and line number
@@ -354,10 +379,11 @@
 and line number of a memory allocation.
 
 Passing a filename and a line number to each allocator makes the API more
-complex: pass 3 new arguments instead of just a context argument, to each
-allocator function. GC allocator functions should also be patched,
-``_PyObject_GC_Malloc()`` is used in many C functions for example. Such changes
-add too much complexity, for a little gain.
+complex: pass 3 new arguments, instead of just a context argument, to each
+allocator function. The GC allocator functions should also be patched.
+``_PyObject_GC_Malloc()`` is used in many C functions for example and so
+objects of differenet types would have the same allocation location. Such
+changes add too much complexity for a little gain.
 
 
 No context argument
@@ -369,41 +395,67 @@
 * ``void* realloc(void *ptr, size_t new_size)``
 * ``void free(void *ptr)``
 
-The context is a convenient way to reuse the same allocator for different APIs
-(ex: PyMem and PyObject).
+It is likely for an allocator hook to be reused for ``PyMem_SetAllocators()``
+and ``PyObject_SetAllocators()``, but the hook must call a different function
+depending on the allocator. The context is a convenient way to reuse the same
+allocator or hook for different APIs.
 
 
 PyMem_Malloc() GIL-free
 -----------------------
 
-There is no real reason to require the GIL when calling ``PyMem_Malloc()``.
+``PyMem_Malloc()`` must be called with the GIL held because in debug mode, it
+calls indirectly ``PyObject_Malloc()`` which requires the GIL to be held.  This
+PEP proposes to "fix" ``PyMem_Malloc()`` to make it always call ``malloc()``.
+So the "GIL must be held" restriction may be removed no ``PyMem_Malloc()``.
 
 Allowing to call ``PyMem_Malloc()`` without holding the GIL might break
 applications which setup their own allocator or their allocator hooks. Holding
-the GIL is very convinient to develop a custom allocator or a hook (no need to
-care of other threads, no need to handle mutexes, etc.).
+the GIL is very convinient to develop a custom allocator: no need to care of
+other threads nor mutexes. It is also convinient for an allocator hook: Python
+internals can be safetly inspected.
+
+Calling ``PyGILState_Ensure()`` in a memory allocator may have unexpected
+behaviour, especially at Python startup and at creation of a new Python thread
+state.
 
 
 Don't add PyMem_RawMalloc()
 ---------------------------
 
-Replace ``malloc()`` with ``PyMem_Malloc()``, but if the GIL is not held: keep
-``malloc()`` unchanged.
+Replace ``malloc()`` with ``PyMem_Malloc()``, but only if the GIL is held.
+Otherwise, keep ``malloc()`` unchanged.
 
 The ``PyMem_Malloc()`` is sometimes already misused. For example, the
 ``main()`` and ``Py_Main()`` functions of Python call ``PyMem_Malloc()``
 whereas the GIL do not exist yet. In this case, ``PyMem_Malloc()`` should
-be replaced with ``malloc()``.
+be replaced with ``malloc()`` (or ``PyMem_RawMalloc()``).
 
 If an hook is used to the track memory usage, the ``malloc()`` memory will not
 be seen. Remaining ``malloc()`` may allocate a lot of memory and so would be
 missed in reports.
 
 
-CCP API
--------
 
-XXX To be done (Kristj?n Valur J?nsson) XXX
+Use existing debug tools to analyze the memory
+----------------------------------------------
+
+There are many existing debug tools to analyze the memory. Some examples:
+`Valgrind <http://valgrind.org/>`_,
+`Purify <http://ibm.com/software/awdtools/purify/>`_,
+`Clang AddressSanitizer <http://code.google.com/p/address-sanitizer/>`_,
+`failmalloc <http://www.nongnu.org/failmalloc/>`_,
+etc.
+
+The problem is retrieve the Python object related to a memory pointer to read
+its type and/or content. Another issue is to retrieve the location of the
+memory allocation: the C backtrace is usually useless (same reasoning than
+macros using ``__FILE__`` and ``__LINE__``), the Python filename and line
+number (or even the Python traceback) is more useful.
+
+Classic tools are unable to introspect the Python internal to collect such
+information. Being able to setup a hook on allocators called with the GIL held
+allow to read a lot of useful data from Python internals.
 
 
 External libraries

-- 
Repository URL: http://hg.python.org/peps

From victor.stinner at gmail.com  Tue Jun 18 13:32:14 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 18 Jun 2013 13:32:14 +0200
Subject: [Python-checkins] cpython (3.3): ctypes: AIX needs an explicit
 #include <alloca.h> to get alloca()
In-Reply-To: <51C0444D.8030402@python.org>
References: <3bZ3HS2pvhzQxV@mail.python.org>
	<CAGvrs3JXKUB5eRcNsTn+FC_PvVCau1-O4ajDHY2YPp652-iNTA@mail.gmail.com>
	<51C0444D.8030402@python.org>
Message-ID: <CAMpsgwb=5Gtc2MHUhfZC5trofVXxJASCbaZQC=Oz7K-aG4GdBg@mail.gmail.com>

2013/6/18 Christian Heimes <christian at python.org>:
> Am 18.06.2013 12:56, schrieb Jeremy Kloth:
>> On Mon, Jun 17, 2013 at 2:02 PM, victor.stinner
>>> +#include <alloca.h>
>>
>> This header is not present on Windows, thus breaking all the Windows
>> buildbots.  Perhaps it should be wrapped in an AIX-specific #ifdef?

Oh really? Portability is complex :-) I only tested on Linux, but I
expected this header to be part of the C standard...

> I have added HAVE_ALLOCA_H to configure:
>
> http://hg.python.org/cpython/rev/7b6ae19dd116

Cool, thanks.

Victor

From python-checkins at python.org  Tue Jun 18 21:04:47 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 21:04:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZdyR2MSGz7LjR@mail.python.org>

http://hg.python.org/peps/rev/304b8dfd3f9a
changeset:   4942:304b8dfd3f9a
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 21:04:34 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  236 ++++++++++++++++++++++++--------------
  1 files changed, 151 insertions(+), 85 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -46,53 +46,84 @@
   - ``void* PyMem_RawMalloc(size_t size)``
   - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)``
   - ``void PyMem_RawFree(void *ptr)``
+  - the behaviour of requesting zero bytes is not defined: return *NULL* or a
+    distinct non-*NULL* pointer depending on the platform.
 
-* Add a new ``PyMemAllocators`` structure::
+* Add a new ``PyMemBlockAllocator`` structure::
 
     typedef struct {
         /* user context passed as the first argument to the 3 functions */
         void *ctx;
 
-        /* allocate memory */
+        /* allocate a memory block */
         void* (*malloc) (void *ctx, size_t size);
 
-        /* allocate memory or resize a memory buffer */
+        /* allocate or resize a memory block */
         void* (*realloc) (void *ctx, void *ptr, size_t new_size);
 
-        /* release memory */
+        /* release a memory block */
         void (*free) (void *ctx, void *ptr);
-    } PyMemAllocators;
+    } PyMemBlockAllocator;
 
 * Add new functions to get and set memory block allocators:
 
-  - ``void PyMem_GetRawAllocators(PyMemAllocators *allocators)``
-  - ``void PyMem_SetRawAllocators(PyMemAllocators *allocators)``
-  - ``void PyMem_GetAllocators(PyMemAllocators *allocators)``
-  - ``void PyMem_SetAllocators(PyMemAllocators *allocators)``
-  - ``void PyObject_GetAllocators(PyMemAllocators *allocators)``
-  - ``void PyObject_SetAllocators(PyMemAllocators *allocators)``
+  - Get/Set internal functions of ``PyMem_RawMalloc()``,
+    ``PyMem_RawRealloc()`` and ``PyMem_RawFree()``:
 
-* Add new functions to get and set memory mapping allocators:
+    * ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
+    * ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
 
-  - ``void _PyObject_GetArenaAllocators(void **ctx_p, void* (**malloc_p) (void *ctx, size_t size), void (**free_p) (void *ctx, void *ptr, size_t size))``
-  - ``void _PyObject_SetArenaAllocators(void *ctx, void* (*malloc) (void *ctx, size_t size), void (*free) (void *ctx, void *ptr, size_t size))``
+  - Get/Set internal functions of ``PyMem_Malloc()``,
+    ``PyMem_Realloc()`` and ``PyMem_Free()``:
+
+    * ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
+    * ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
+    * ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*:
+      it would be treated as an error.
+
+  - Get/Set internal functions of ``PyObject_Malloc()``,,
+    ``PyObject_Realloc()`` and ``PyObject_Free()``:
+
+    * ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
+    * ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
+
+* Add a new ``PyMemMappingAllocator`` structure::
+
+    typedef struct {
+        /* user context passed as the first argument to the 2 functions */
+        void *ctx;
+
+        /* allocate a memory mapping */
+        void* (*malloc) (void *ctx, size_t size);
+
+        /* release a memory mapping */
+        void (*free) (void *ctx, void *ptr, size_t size);
+    } PyMemMappingAllocator;
+
+* Add a new function to get and set memory mapping allocator:
+
+  - ``void PyMem_GetMappingAllocator(PyMemMappingAllocator *allocator)``
+  - ``void PyMem_SetMappingAllocator(PyMemMappingAllocator *allocator)``
+  - Currently, this allocator is only used internally by *pymalloc* to allocate
+    arenas.
 
 * Add a new function to setup the builtin Python debug hooks when memory
   allocators are replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
 
-.. note::
+The builtin Python debug hooks were introduced in Python 2.3 and implement the
+following checks:
 
-   The builtin Python debug hooks were introduced in Python 2.3 and implement the
-   following checks:
+* Newly allocated memory is filled with the byte ``0xCB``, freed memory is
+  filled with the byte ``0xDB``.
+* Detect API violations, ex: ``PyObject_Free()`` called on a memory block
+  allocated by ``PyMem_Malloc()``
+* Detect write before the start of the buffer (buffer underflow)
+* Detect write after the end of the buffer (buffer overflow)
 
-   * Newly allocated memory is filled with the byte 0xCB, freed memory is filled
-     with the byte 0xDB.
-   * Detect API violations, ex: ``PyObject_Free()`` called on a memory block
-     allocated by ``PyMem_Malloc()``
-   * Detect write before the start of the buffer (buffer underflow)
-   * Detect write after the end of the buffer (buffer overflow)
+The *pymalloc* allocator is used by default for:
+``PyObject_Malloc()``, ``PyObject_Realloc()`` and ``PyObject_Free()``.
 
 
 Make usage of these new APIs
@@ -117,10 +148,10 @@
 Examples
 ========
 
-Use case 1: Replace Memory Allocators, keep pymalloc
+Use case 1: Replace Memory Allocator, keep pymalloc
 ----------------------------------------------------
 
-Setup your custom memory allocators, keeping pymalloc. Dummy example wasting 2
+Setup your custom memory allocator, keeping pymalloc. Dummy example wasting 2
 bytes per allocation, and 10 bytes per arena::
 
     #include <stdlib.h>
@@ -156,30 +187,30 @@
         free(ptr);
     }
 
-    void setup_custom_allocators(void)
+    void setup_custom_allocator(void)
     {
-        PyMemAllocators alloc;
+        PyMemBlockAllocator alloc;
 
         alloc.ctx = &alloc_padding;
         alloc.malloc = my_malloc;
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
-        PyMem_SetRawAllocators(&alloc);
-        PyMem_SetAllocators(&alloc);
+        PyMem_SetRawAllocator(&alloc);
+        PyMem_SetAllocator(&alloc);
 
-        _PyObject_SetArenaAllocators(&arena_padding,
+        _PyObject_SetArenaAllocator(&arena_padding,
                                      my_alloc_arena, my_free_arena);
 
         PyMem_SetupDebugHooks();
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocators(&alloc)`` if the new allocators
+   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new allocator
    are not thread-safe.
 
 
-Use case 2: Replace Memory Allocators, override pymalloc
+Use case 2: Replace Memory Allocator, override pymalloc
 --------------------------------------------------------
 
 If your allocator is optimized for allocation of small objects (less than 512
@@ -209,23 +240,23 @@
         free(ptr);
     }
 
-    void setup_custom_allocators(void)
+    void setup_custom_allocator(void)
     {
-        PyMemAllocators alloc;
+        PyMemBlockAllocator alloc;
         alloc.ctx = &padding;
         alloc.malloc = my_malloc;
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
-        PyMem_SetRawAllocators(&alloc);
-        PyMem_SetAllocators(&alloc);
-        PyObject_SetAllocators(&alloc);
+        PyMem_SetRawAllocator(&alloc);
+        PyMem_SetAllocator(&alloc);
+        PyObject_SetAllocator(&alloc);
 
         PyMem_SetupDebugHooks();
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocators(&alloc)`` if the new allocators
+   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new allocator
    are not thread-safe.
 
 
@@ -236,15 +267,15 @@
 Example to setup hooks on all memory allocators::
 
     struct {
-        PyMemAllocators pymem;
-        PyMemAllocators pymem_raw;
-        PyMemAllocators pyobj;
+        PyMemBlockAllocator pymem;
+        PyMemBlockAllocator pymem_raw;
+        PyMemBlockAllocator pyobj;
         /* ... */
     } hook;
 
     static void* hook_malloc(void *ctx, size_t size)
     {
-        PyMemAllocators *alloc = (PyMemAllocators *)ctx;
+        PyMemBlockAllocator *alloc = (PyMemBlockAllocator *)ctx;
         /* ... */
         ptr = alloc->malloc(alloc->ctx, size);
         /* ... */
@@ -253,7 +284,7 @@
 
     static void* hook_realloc(void *ctx, void *ptr, size_t new_size)
     {
-        PyMemAllocators *alloc = (PyMemAllocators *)ctx;
+        PyMemBlockAllocator *alloc = (PyMemBlockAllocator *)ctx;
         void *ptr2;
         /* ... */
         ptr2 = alloc->realloc(alloc->ctx, ptr, new_size);
@@ -263,7 +294,7 @@
 
     static void hook_free(void *ctx, void *ptr)
     {
-        PyMemAllocators *alloc = (PyMemAllocators *)ctx;
+        PyMemBlockAllocator *alloc = (PyMemBlockAllocator *)ctx;
         /* ... */
         alloc->free(alloc->ctx, ptr);
         /* ... */
@@ -271,7 +302,7 @@
 
     void setup_hooks(void)
     {
-        PyMemAllocators alloc;
+        PyMemBlockAllocator alloc;
         static int installed = 0;
 
         if (installed)
@@ -282,21 +313,21 @@
         alloc.realloc = hook_realloc;
         alloc.free = hook_free;
 
-        PyMem_GetRawAllocators(&hook.pymem_raw);
+        PyMem_GetRawAllocator(&hook.pymem_raw);
         alloc.ctx = &hook.pymem_raw;
-        PyMem_SetRawAllocators(&alloc);
+        PyMem_SetRawAllocator(&alloc);
 
-        PyMem_GetAllocators(&hook.pymem);
+        PyMem_GetAllocator(&hook.pymem);
         alloc.ctx = &hook.pymem;
-        PyMem_SetAllocators(&alloc);
+        PyMem_SetAllocator(&alloc);
 
-        PyObject_GetAllocators(&hook.pyobj);
+        PyObject_GetAllocator(&hook.pyobj);
         alloc.ctx = &hook.pyobj;
-        PyObject_SetAllocators(&alloc);
+        PyObject_SetAllocator(&alloc);
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocators(&alloc)`` if hooks are not
+   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if hooks are not
    thread-safe.
 
 .. note::
@@ -326,13 +357,19 @@
 
 Replace the 6 functions:
 
-* ``PyMem_GetRawAllocators()``, ``PyMem_GetAllocators()``, ``PyObject_GetAllocators()``
-* ``PyMem_SetRawAllocators(allocators)``, ``PyMem_SetAllocators(allocators)``, ``PyObject_SetAllocators(allocators)``
+* ``PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
+* ``PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
+* ``PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
+* ``PyMem_SetRawAllocator(allocator)``
+* ``PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
+* ``PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
 
 with 2 functions with an additional *domain* argument:
 
-* ``Py_GetAllocators(domain)``
-* ``Py_SetAllocators(domain, allocators)``
+* ``int Py_GetAllocator(int domain, PyMemBlockAllocator *allocator)``
+* ``int Py_SetAllocator(int domain, PyMemBlockAllocator *allocator)``
+
+These functions return 0 on success, or -1 if the domain is unknown.
 
 where domain is one of these values:
 
@@ -341,9 +378,19 @@
 * ``PYALLOC_PYOBJECT``
 
 
-``_PyObject_GetArenaAllocators()`` and ``_PyObject_SetArenaAllocators()`` are
-not merged and kept private because their prototypes are different and they are
-specific to pymalloc.
+PyMem_Malloc() reuses PyMem_RawMalloc() by default
+--------------------------------------------------
+
+``PyMem_Malloc()`` should call ``PyMem_RawMalloc()`` by default. So calling
+``PyMem_SetRawAllocator()`` would also also patch ``PyMem_Malloc()``
+indirectly.
+
+Such change is less optimal, it adds another level of indirection.
+
+In the proposed implementation of this PEP (issue #3329), ``PyMem_RawMalloc()``
+calls directly ``malloc()``, whereas ``PyMem_Malloc()`` returns ``NULL`` if
+size is larger than ``PY_SSIZE_T_MAX``, and the default allocator of
+``PyMem_Malloc()`` calls ``malloc(1)`` if the size is zero.
 
 
 Add a new PYDEBUGMALLOC environment variable
@@ -353,8 +400,8 @@
 allocator replaces the default Python allocator, an environment variable
 ``PYDEBUGMALLOC`` can be added to setup these debug function hooks, instead of
 adding the new function ``PyMem_SetupDebugHooks()``. If the environment
-variable is present, ``PyMem_SetRawAllocators()``, ``PyMem_SetAllocators()``
-and ``PyObject_SetAllocators()`` will reinstall automatically the hook on top
+variable is present, ``PyMem_SetRawAllocator()``, ``PyMem_SetAllocator()``
+and ``PyObject_SetAllocator()`` will reinstall automatically the hook on top
 of the new allocator.
 
 An new environment variable would make the Python initialization even more
@@ -386,21 +433,6 @@
 changes add too much complexity for a little gain.
 
 
-No context argument
--------------------
-
-Simplify the signature of allocator functions, remove the context argument:
-
-* ``void* malloc(size_t size)``
-* ``void* realloc(void *ptr, size_t new_size)``
-* ``void free(void *ptr)``
-
-It is likely for an allocator hook to be reused for ``PyMem_SetAllocators()``
-and ``PyObject_SetAllocators()``, but the hook must call a different function
-depending on the allocator. The context is a convenient way to reuse the same
-allocator or hook for different APIs.
-
-
 PyMem_Malloc() GIL-free
 -----------------------
 
@@ -436,7 +468,6 @@
 missed in reports.
 
 
-
 Use existing debug tools to analyze the memory
 ----------------------------------------------
 
@@ -458,6 +489,36 @@
 allow to read a lot of useful data from Python internals.
 
 
+Add msize()
+-----------
+
+Add another field to ``PyMemBlockAllocator`` and ``PyMemMappingAllocator``::
+
+    size_t msize(void *ptr);
+
+This function returns the size of a memory block or a memory mapping. Return
+(size_t)-1 if the function is not implemented or if the pointer is unknown
+(ex: NULL pointer).
+
+On Windows, this function can be implemented using ``_msize()`` and
+``VirtualQuery()``.
+
+
+No context argument
+-------------------
+
+Simplify the signature of allocator functions, remove the context argument:
+
+* ``void* malloc(size_t size)``
+* ``void* realloc(void *ptr, size_t new_size)``
+* ``void free(void *ptr)``
+
+It is likely for an allocator hook to be reused for ``PyMem_SetAllocator()``
+and ``PyObject_SetAllocator()``, but the hook must call a different function
+depending on the allocator. The context is a convenient way to reuse the same
+allocator or hook for different APIs.
+
+
 External libraries
 ==================
 
@@ -476,7 +537,6 @@
 <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_.
 
 
-
 Memory allocators
 =================
 
@@ -490,18 +550,24 @@
 mappings are usually used for large allocations (ex: larger than 256 KB),
 whereas the heap is used for small allocations.
 
-The heap is handled by ``brk()`` and ``sbrk()`` system calls on Linux, and is
-contiguous.  Memory mappings are handled by ``mmap()`` on UNIX and
+On UNIX, the heap is handled by ``brk()`` and ``sbrk()`` system calls on Linux,
+and it is contiguous.  On Windows, the heap is handled by ``HeapAlloc()`` and
+may be discontiguous. Memory mappings are handled by ``mmap()`` on UNIX and
 ``VirtualAlloc()`` on Windows, they may be discontiguous.
 
 Releasing a memory mapping gives back immediatly the memory to the system. For
 the heap, memory is only given back to the system if it is at the end of the
 heap. Otherwise, the memory will only be given back to the system when all the
-memory located after the released memory are also released. To allocate memory
-in the heap, the allocator tries to reuse free space. If there is no contiguous
-space big enough, the heap must be increased, even if we have more free space
-than required size.  This issue is called the "memory fragmentation": the
-memory usage seen by the system may be much higher than real usage.
+memory located after the released memory are also released.
+
+To allocate memory in the heap, the allocator tries to reuse free space. If
+there is no contiguous space big enough, the heap must be increased, even if we
+have more free space than required size.  This issue is called the "memory
+fragmentation": the memory usage seen by the system may be much higher than
+real usage.
+
+On Windows, ``HeapAlloc()`` creates a new memory mapping with
+``VirtualAlloc()`` if there is not enough free contiguous memory.
 
 CPython has a pymalloc allocator using arenas of 256 KB for allocations smaller
 than 512 bytes. This allocator is optimized for small objects with a short

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 22:05:33 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 22:05:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZgJY1M3RzNfc@mail.python.org>

http://hg.python.org/peps/rev/6d91715570b5
changeset:   4943:6d91715570b5
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 21:59:48 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  218 +++++++++++++++++++++++---------------
  1 files changed, 132 insertions(+), 86 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -25,10 +25,10 @@
   optimized for its Python usage
 * Python running on embedded devices with low memory and slow CPU.
   A custom memory allocator may be required to use efficiently the memory
-  and/or to be able to use all memory of the device.
+  and/or to be able to use all the memory of the device.
 * Debug tool to:
 
-  - track memory leaks
+  - track memory usage (memory leaks)
   - get the Python filename and line number where an object was allocated
   - detect buffer underflow, buffer overflow and detect misuse of Python
     allocator APIs (builtin Python debug hooks)
@@ -41,7 +41,7 @@
 API changes
 -----------
 
-* Add new GIL-free memory allocator functions:
+* Add new GIL-free (no need to hold the GIL) memory allocator functions:
 
   - ``void* PyMem_RawMalloc(size_t size)``
   - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)``
@@ -65,27 +65,27 @@
         void (*free) (void *ctx, void *ptr);
     } PyMemBlockAllocator;
 
-* Add new functions to get and set memory block allocators:
+* Add new functions to get and set internal functions of ``PyMem_RawMalloc()``,
+  ``PyMem_RawRealloc()`` and ``PyMem_RawFree()``:
 
-  - Get/Set internal functions of ``PyMem_RawMalloc()``,
-    ``PyMem_RawRealloc()`` and ``PyMem_RawFree()``:
+  - ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
+  - ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
 
-    * ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
-    * ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
+* Add new functions to get and set internal functions of ``PyMem_Malloc()``,
+  ``PyMem_Realloc()`` and ``PyMem_Free()``:
 
-  - Get/Set internal functions of ``PyMem_Malloc()``,
-    ``PyMem_Realloc()`` and ``PyMem_Free()``:
+  - ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
+  - ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
+  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*:
+    it would be treated as an error.
 
-    * ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
-    * ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
-    * ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*:
-      it would be treated as an error.
+* Add new functions to get and set internal functions of
+  ``PyObject_Malloc()``,, ``PyObject_Realloc()`` and ``PyObject_Free()``:
 
-  - Get/Set internal functions of ``PyObject_Malloc()``,,
-    ``PyObject_Realloc()`` and ``PyObject_Free()``:
-
-    * ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
-    * ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
+  - ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
+  - ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
+  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*:
+    it would be treated as an error.
 
 * Add a new ``PyMemMappingAllocator`` structure::
 
@@ -94,7 +94,7 @@
         void *ctx;
 
         /* allocate a memory mapping */
-        void* (*malloc) (void *ctx, size_t size);
+        void* (*alloc) (void *ctx, size_t size);
 
         /* release a memory mapping */
         void (*free) (void *ctx, void *ptr, size_t size);
@@ -112,6 +112,11 @@
 
   - ``void PyMem_SetupDebugHooks(void)``
 
+* The following memory allocators always returns *NULL* if size is greater
+  than ``PY_SSIZE_T_MAX``: ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``,
+  ``PyMem_Malloc()``, ``PyMem_Realloc()``, ``PyObject_Malloc()``,
+  ``PyObject_Realloc()``.
+
 The builtin Python debug hooks were introduced in Python 2.3 and implement the
 following checks:
 
@@ -151,8 +156,7 @@
 Use case 1: Replace Memory Allocator, keep pymalloc
 ----------------------------------------------------
 
-Setup your custom memory allocator, keeping pymalloc. Dummy example wasting 2
-bytes per allocation, and 10 bytes per arena::
+Dummy example wasting 2 bytes per allocation, and 10 bytes per arena::
 
     #include <stdlib.h>
 
@@ -189,18 +193,21 @@
 
     void setup_custom_allocator(void)
     {
-        PyMemBlockAllocator alloc;
+        PyMemBlockAllocator block;
+        PyMemMappingAllocator mapping;
 
-        alloc.ctx = &alloc_padding;
-        alloc.malloc = my_malloc;
-        alloc.realloc = my_realloc;
-        alloc.free = my_free;
+        block.ctx = &alloc_padding;
+        block.malloc = my_malloc;
+        block.realloc = my_realloc;
+        block.free = my_free;
 
-        PyMem_SetRawAllocator(&alloc);
-        PyMem_SetAllocator(&alloc);
+        PyMem_SetRawAllocator(&block);
+        PyMem_SetAllocator(&block);
 
-        _PyObject_SetArenaAllocator(&arena_padding,
-                                     my_alloc_arena, my_free_arena);
+        mapping.ctx = &arena_padding;
+        mapping.alloc = my_alloc_arena;
+        mapping.free = my_free_arena;
+        PyMem_SetMappingAllocator(mapping);
 
         PyMem_SetupDebugHooks();
     }
@@ -267,9 +274,9 @@
 Example to setup hooks on all memory allocators::
 
     struct {
-        PyMemBlockAllocator pymem;
-        PyMemBlockAllocator pymem_raw;
-        PyMemBlockAllocator pyobj;
+        PyMemBlockAllocator raw;
+        PyMemBlockAllocator mem;
+        PyMemBlockAllocator obj;
         /* ... */
     } hook;
 
@@ -313,16 +320,16 @@
         alloc.realloc = hook_realloc;
         alloc.free = hook_free;
 
-        PyMem_GetRawAllocator(&hook.pymem_raw);
-        alloc.ctx = &hook.pymem_raw;
+        PyMem_GetRawAllocator(&hook.raw);
+        alloc.ctx = &hook.raw;
         PyMem_SetRawAllocator(&alloc);
 
-        PyMem_GetAllocator(&hook.pymem);
-        alloc.ctx = &hook.pymem;
+        PyMem_GetAllocator(&hook.mem);
+        alloc.ctx = &hook.mem;
         PyMem_SetAllocator(&alloc);
 
-        PyObject_GetAllocator(&hook.pyobj);
-        alloc.ctx = &hook.pyobj;
+        PyObject_GetAllocator(&hook.obj);
+        alloc.ctx = &hook.obj;
         PyObject_SetAllocator(&alloc);
     }
 
@@ -366,8 +373,8 @@
 
 with 2 functions with an additional *domain* argument:
 
-* ``int Py_GetAllocator(int domain, PyMemBlockAllocator *allocator)``
-* ``int Py_SetAllocator(int domain, PyMemBlockAllocator *allocator)``
+* ``int PyMem_GetBlockAllocator(int domain, PyMemBlockAllocator *allocator)``
+* ``int PyMem_SetBlockAllocator(int domain, PyMemBlockAllocator *allocator)``
 
 These functions return 0 on success, or -1 if the domain is unknown.
 
@@ -377,6 +384,8 @@
 * ``PYALLOC_PYMEM_RAW``
 * ``PYALLOC_PYOBJECT``
 
+Drawback: the caller has to check if the result is 0, or handle the error.
+
 
 PyMem_Malloc() reuses PyMem_RawMalloc() by default
 --------------------------------------------------
@@ -385,12 +394,11 @@
 ``PyMem_SetRawAllocator()`` would also also patch ``PyMem_Malloc()``
 indirectly.
 
-Such change is less optimal, it adds another level of indirection.
+.. note::
 
-In the proposed implementation of this PEP (issue #3329), ``PyMem_RawMalloc()``
-calls directly ``malloc()``, whereas ``PyMem_Malloc()`` returns ``NULL`` if
-size is larger than ``PY_SSIZE_T_MAX``, and the default allocator of
-``PyMem_Malloc()`` calls ``malloc(1)`` if the size is zero.
+   In the implementation of this PEP (issue #3329),
+   ``PyMem_RawMalloc(0)`` calls ``malloc(0)``,
+   whereas ``PyMem_Malloc(0)`` calls ``malloc(1)``.
 
 
 Add a new PYDEBUGMALLOC environment variable
@@ -422,30 +430,58 @@
 Pass the C filename and line number
 -----------------------------------
 
-Use C macros using ``__FILE__`` and ``__LINE__`` to get the C filename
-and line number of a memory allocation.
+Define allocator functions using macros and use ``__FILE__`` and ``__LINE__``
+to get the C filename and line number of a memory allocation.
+
+Example::
+
+    typedef struct {
+        /* user context passed as the first argument to the 3 functions */
+        void *ctx;
+
+        /* allocate a memory block */
+        void* (*malloc) (void *ctx, const char *filename, int lineno,
+                         size_t size);
+
+        /* allocate or resize a memory block */
+        void* (*realloc) (void *ctx, const char *filename, int lineno,
+                          void *ptr, size_t new_size);
+
+        /* release a memory block */
+        void (*free) (void *ctx, const char *filename, int lineno,
+                      void *ptr);
+    } PyMemBlockAllocator;
+
+    void* _PyMem_MallocTrace(const char *filename, int lineno, size_t size);
+
+    /* need also a function for the Python stable ABI */
+    void* PyMem_Malloc(size_t size);
+
+    #define PyMem_Malloc(size) _PyMem_MallocTrace(__FILE__, __LINE__, size)
 
 Passing a filename and a line number to each allocator makes the API more
 complex: pass 3 new arguments, instead of just a context argument, to each
 allocator function. The GC allocator functions should also be patched.
-``_PyObject_GC_Malloc()`` is used in many C functions for example and so
+For example, ``_PyObject_GC_Malloc()`` is used in many C functions and so
 objects of differenet types would have the same allocation location. Such
 changes add too much complexity for a little gain.
 
 
-PyMem_Malloc() GIL-free
+GIL-free PyMem_Malloc()
 -----------------------
 
-``PyMem_Malloc()`` must be called with the GIL held because in debug mode, it
-calls indirectly ``PyObject_Malloc()`` which requires the GIL to be held.  This
-PEP proposes to "fix" ``PyMem_Malloc()`` to make it always call ``malloc()``.
-So the "GIL must be held" restriction may be removed no ``PyMem_Malloc()``.
+When Python is compiled in debug mode, ``PyMem_Malloc()`` calls indirectly ``PyObject_Malloc()`` which requires the GIL to be held.
+That's why ``PyMem_Malloc()`` must be called with the GIL held.
+
+This PEP proposes to "fix" ``PyMem_Malloc()`` to make it always call
+``malloc()``.  So the "GIL must be held" restriction may be removed from
+``PyMem_Malloc()``.
 
 Allowing to call ``PyMem_Malloc()`` without holding the GIL might break
-applications which setup their own allocator or their allocator hooks. Holding
-the GIL is very convinient to develop a custom allocator: no need to care of
-other threads nor mutexes. It is also convinient for an allocator hook: Python
-internals can be safetly inspected.
+applications which setup their own allocators or allocator hooks. Holding the
+GIL is convinient to develop a custom allocator: no need to care of other
+threads. It is also convinient for a debug allocator hook: Python internal
+objects can be safetly inspected.
 
 Calling ``PyGILState_Ensure()`` in a memory allocator may have unexpected
 behaviour, especially at Python startup and at creation of a new Python thread
@@ -458,10 +494,11 @@
 Replace ``malloc()`` with ``PyMem_Malloc()``, but only if the GIL is held.
 Otherwise, keep ``malloc()`` unchanged.
 
-The ``PyMem_Malloc()`` is sometimes already misused. For example, the
-``main()`` and ``Py_Main()`` functions of Python call ``PyMem_Malloc()``
-whereas the GIL do not exist yet. In this case, ``PyMem_Malloc()`` should
-be replaced with ``malloc()`` (or ``PyMem_RawMalloc()``).
+The ``PyMem_Malloc()`` is used without the GIL held in some Python functions.
+For example, the ``main()`` and ``Py_Main()`` functions of Python call
+``PyMem_Malloc()`` whereas the GIL do not exist yet. In this case,
+``PyMem_Malloc()`` should be replaced with ``malloc()`` (or
+``PyMem_RawMalloc()``).
 
 If an hook is used to the track memory usage, the ``malloc()`` memory will not
 be seen. Remaining ``malloc()`` may allocate a lot of memory and so would be
@@ -478,15 +515,15 @@
 `failmalloc <http://www.nongnu.org/failmalloc/>`_,
 etc.
 
-The problem is retrieve the Python object related to a memory pointer to read
+The problem is to retrieve the Python object related to a memory pointer to read
 its type and/or content. Another issue is to retrieve the location of the
 memory allocation: the C backtrace is usually useless (same reasoning than
 macros using ``__FILE__`` and ``__LINE__``), the Python filename and line
 number (or even the Python traceback) is more useful.
 
-Classic tools are unable to introspect the Python internal to collect such
+Classic tools are unable to introspect Python internals to collect such
 information. Being able to setup a hook on allocators called with the GIL held
-allow to read a lot of useful data from Python internals.
+allow to collect a lot of useful data from Python internals.
 
 
 Add msize()
@@ -514,22 +551,31 @@
 * ``void free(void *ptr)``
 
 It is likely for an allocator hook to be reused for ``PyMem_SetAllocator()``
-and ``PyObject_SetAllocator()``, but the hook must call a different function
-depending on the allocator. The context is a convenient way to reuse the same
-allocator or hook for different APIs.
+and ``PyObject_SetAllocator()``, or even ``PyMem_SetRawAllocator()``, but the
+hook must call a different function depending on the allocator. The context is
+a convenient way to reuse the same custom allocator or hook for different
+Python allocators.
 
 
 External libraries
 ==================
 
-* glib: `g_mem_set_vtable()
-  <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
+Python should try to reuse the same prototypes for allocator functions than
+other libraries.
+
+Libraries used by Python:
+
 * OpenSSL: `CRYPTO_set_mem_functions()
   <http://git.openssl.org/gitweb/?p=openssl.git;a=blob;f=crypto/mem.c;h=f7984fa958eb1edd6c61f6667f3f2b29753be662;hb=HEAD#l124>`_
   to set memory management functions globally
 * expat: `parserCreate()
   <http://hg.python.org/cpython/file/cc27d50bd91a/Modules/expat/xmlparse.c#l724>`_
   has a per-instance memory handler
+
+Other libraries:
+
+* glib: `g_mem_set_vtable()
+  <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
 * libxml2: `xmlGcMemSetup() <http://xmlsoft.org/html/libxml-xmlmemory.html>`_,
   global
 
@@ -555,8 +601,8 @@
 may be discontiguous. Memory mappings are handled by ``mmap()`` on UNIX and
 ``VirtualAlloc()`` on Windows, they may be discontiguous.
 
-Releasing a memory mapping gives back immediatly the memory to the system. For
-the heap, memory is only given back to the system if it is at the end of the
+Releasing a memory mapping gives back immediatly the memory to the system. On
+UNIX, heap memory is only given back to the system if it is at the end of the
 heap. Otherwise, the memory will only be given back to the system when all the
 memory located after the released memory are also released.
 
@@ -564,24 +610,24 @@
 there is no contiguous space big enough, the heap must be increased, even if we
 have more free space than required size.  This issue is called the "memory
 fragmentation": the memory usage seen by the system may be much higher than
-real usage.
-
-On Windows, ``HeapAlloc()`` creates a new memory mapping with
+real usage. On Windows, ``HeapAlloc()`` creates a new memory mapping with
 ``VirtualAlloc()`` if there is not enough free contiguous memory.
 
-CPython has a pymalloc allocator using arenas of 256 KB for allocations smaller
-than 512 bytes. This allocator is optimized for small objects with a short
-lifetime.
+CPython has a *pymalloc* allocator for allocations smaller than 512 bytes. This
+allocator is optimized for small objects with a short lifetime. It uses memory
+mappings called "arenas" with a fixed size of 256 KB.
 
-Windows provides a `Low-fragmentation Heap
-<http://msdn.microsoft.com/en-us/library/windows/desktop/aa366750%28v=vs.85%29.aspx>`_.
+Other allocators:
 
-The Linux kernel uses `slab allocation
-<http://en.wikipedia.org/wiki/Slab_allocation>`_.
+* Windows provides a `Low-fragmentation Heap
+  <http://msdn.microsoft.com/en-us/library/windows/desktop/aa366750%28v=vs.85%29.aspx>`_.
 
-The glib library has a `Memory Slice API
-<https://developer.gnome.org/glib/unstable/glib-Memory-Slices.html>`_:
-efficient way to allocate groups of equal-sized chunks of memory
+* The Linux kernel uses `slab allocation
+  <http://en.wikipedia.org/wiki/Slab_allocation>`_.
+
+* The glib library has a `Memory Slice API
+  <https://developer.gnome.org/glib/unstable/glib-Memory-Slices.html>`_:
+  efficient way to allocate groups of equal-sized chunks of memory
 
 
 Links

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 22:05:34 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 22:05:34 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_textwidth=3D72_=28?=
	=?utf-8?q?for_email=29?=
Message-ID: <3bZgJZ5YwtzQTG@mail.python.org>

http://hg.python.org/peps/rev/df41e564ea07
changeset:   4944:df41e564ea07
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 22:05:17 2013 +0200
summary:
  PEP 445: textwidth=72 (for email)

files:
  pep-0445.txt |  308 ++++++++++++++++++++------------------
  1 files changed, 163 insertions(+), 145 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -20,19 +20,21 @@
 
 Use cases:
 
-* Application embedding Python may want to isolate Python memory from the
-  memory of the application, or may want to different memory allocator
-  optimized for its Python usage
+* Application embedding Python may want to isolate Python memory from
+  the memory of the application, or may want to different memory
+  allocator optimized for its Python usage
 * Python running on embedded devices with low memory and slow CPU.
-  A custom memory allocator may be required to use efficiently the memory
-  and/or to be able to use all the memory of the device.
+  A custom memory allocator may be required to use efficiently the
+  memory and/or to be able to use all the memory of the device.
 * Debug tool to:
 
   - track memory usage (memory leaks)
-  - get the Python filename and line number where an object was allocated
+  - get the Python filename and line number where an object was
+    allocated
   - detect buffer underflow, buffer overflow and detect misuse of Python
     allocator APIs (builtin Python debug hooks)
-  - force allocation to fail to test handling of ``MemoryError`` exception
+  - force allocation to fail to test handling of ``MemoryError``
+    exception
 
 
 Proposal
@@ -46,13 +48,14 @@
   - ``void* PyMem_RawMalloc(size_t size)``
   - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)``
   - ``void PyMem_RawFree(void *ptr)``
-  - the behaviour of requesting zero bytes is not defined: return *NULL* or a
-    distinct non-*NULL* pointer depending on the platform.
+  - the behaviour of requesting zero bytes is not defined: return *NULL*
+    or a distinct non-*NULL* pointer depending on the platform.
 
 * Add a new ``PyMemBlockAllocator`` structure::
 
     typedef struct {
-        /* user context passed as the first argument to the 3 functions */
+        /* user context passed as the first argument
+           to the 3 functions */
         void *ctx;
 
         /* allocate a memory block */
@@ -65,32 +68,34 @@
         void (*free) (void *ctx, void *ptr);
     } PyMemBlockAllocator;
 
-* Add new functions to get and set internal functions of ``PyMem_RawMalloc()``,
-  ``PyMem_RawRealloc()`` and ``PyMem_RawFree()``:
+* Add new functions to get and set internal functions of
+  ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()`` and ``PyMem_RawFree()``:
 
   - ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
   - ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
 
-* Add new functions to get and set internal functions of ``PyMem_Malloc()``,
-  ``PyMem_Realloc()`` and ``PyMem_Free()``:
+* Add new functions to get and set internal functions of
+  ``PyMem_Malloc()``, ``PyMem_Realloc()`` and ``PyMem_Free()``:
 
   - ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
   - ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
-  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*:
-    it would be treated as an error.
+  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return
+    *NULL*: it would be treated as an error.
 
 * Add new functions to get and set internal functions of
-  ``PyObject_Malloc()``,, ``PyObject_Realloc()`` and ``PyObject_Free()``:
+  ``PyObject_Malloc()``,, ``PyObject_Realloc()`` and
+  ``PyObject_Free()``:
 
   - ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
   - ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
-  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return *NULL*:
-    it would be treated as an error.
+  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return
+    *NULL*: it would be treated as an error.
 
 * Add a new ``PyMemMappingAllocator`` structure::
 
     typedef struct {
-        /* user context passed as the first argument to the 2 functions */
+        /* user context passed as the first argument
+           to the 2 functions */
         void *ctx;
 
         /* allocate a memory mapping */
@@ -104,26 +109,26 @@
 
   - ``void PyMem_GetMappingAllocator(PyMemMappingAllocator *allocator)``
   - ``void PyMem_SetMappingAllocator(PyMemMappingAllocator *allocator)``
-  - Currently, this allocator is only used internally by *pymalloc* to allocate
-    arenas.
+  - Currently, this allocator is only used internally by *pymalloc* to
+    allocate arenas.
 
 * Add a new function to setup the builtin Python debug hooks when memory
   allocators are replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
 
-* The following memory allocators always returns *NULL* if size is greater
-  than ``PY_SSIZE_T_MAX``: ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``,
-  ``PyMem_Malloc()``, ``PyMem_Realloc()``, ``PyObject_Malloc()``,
-  ``PyObject_Realloc()``.
+* The following memory allocators always returns *NULL* if size is
+  greater than ``PY_SSIZE_T_MAX``: ``PyMem_RawMalloc()``,
+  ``PyMem_RawRealloc()``, ``PyMem_Malloc()``, ``PyMem_Realloc()``,
+  ``PyObject_Malloc()``, ``PyObject_Realloc()``.
 
-The builtin Python debug hooks were introduced in Python 2.3 and implement the
-following checks:
+The builtin Python debug hooks were introduced in Python 2.3 and
+implement the following checks:
 
-* Newly allocated memory is filled with the byte ``0xCB``, freed memory is
-  filled with the byte ``0xDB``.
-* Detect API violations, ex: ``PyObject_Free()`` called on a memory block
-  allocated by ``PyMem_Malloc()``
+* Newly allocated memory is filled with the byte ``0xCB``, freed memory
+  is filled with the byte ``0xDB``.
+* Detect API violations, ex: ``PyObject_Free()`` called on a memory
+  block allocated by ``PyMem_Malloc()``
 * Detect write before the start of the buffer (buffer underflow)
 * Detect write after the end of the buffer (buffer overflow)
 
@@ -134,20 +139,20 @@
 Make usage of these new APIs
 ----------------------------
 
-* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()`` and
-  ``realloc()``, instead of calling ``PyObject_Malloc()`` and
+* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()``
+  and ``realloc()``, instead of calling ``PyObject_Malloc()`` and
   ``PyObject_Realloc()`` in debug mode
 
 * ``PyObject_Malloc()`` falls back on ``PyMem_Malloc()`` instead of
-  ``malloc()`` if size is greater or equal than ``SMALL_REQUEST_THRESHOLD``
-  (512 bytes), and ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()``
-  instead of ``realloc()``
+  ``malloc()`` if size is greater or equal than
+  ``SMALL_REQUEST_THRESHOLD`` (512 bytes), and ``PyObject_Realloc()``
+  falls back on ``PyMem_Realloc()`` instead of ``realloc()``
 
 * Replace direct calls to ``malloc()`` with ``PyMem_Malloc()``, or
   ``PyMem_RawMalloc()`` if the GIL is not held
 
-* Configure external libraries like zlib or OpenSSL to allocate memory using
-  ``PyMem_RawMalloc()``
+* Configure external libraries like zlib or OpenSSL to allocate memory
+  using ``PyMem_RawMalloc()``
 
 
 Examples
@@ -213,16 +218,16 @@
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new allocator
-   are not thread-safe.
+   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new
+   allocator are not thread-safe.
 
 
 Use case 2: Replace Memory Allocator, override pymalloc
 --------------------------------------------------------
 
-If your allocator is optimized for allocation of small objects (less than 512
-bytes) with a short lifetime, pymalloc can be overriden: replace
-``PyObject_Malloc()``.
+If your allocator is optimized for allocation of small objects (less
+than 512 bytes) with a short lifetime, pymalloc can be overriden:
+replace ``PyObject_Malloc()``.
 
 Dummy Example wasting 2 bytes per allocation::
 
@@ -263,8 +268,8 @@
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new allocator
-   are not thread-safe.
+   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new
+   allocator are not thread-safe.
 
 
 
@@ -338,20 +343,21 @@
    thread-safe.
 
 .. note::
-   ``PyMem_SetupDebugHooks()`` does not need to be called: Python debug hooks
-   are installed automatically at startup.
+   ``PyMem_SetupDebugHooks()`` does not need to be called: Python debug
+   hooks are installed automatically at startup.
 
 
 Performances
 ============
 
-Results of the `Python benchmarks suite <http://hg.python.org/benchmarks>`_ (-b
-2n3): some tests are 1.04x faster, some tests are 1.04 slower, significant is
-between 115 and -191. I don't understand these output, but I guess that the
-overhead cannot be seen with such test.
+Results of the `Python benchmarks suite
+<http://hg.python.org/benchmarks>`_ (-b 2n3): some tests are 1.04x
+faster, some tests are 1.04 slower, significant is between 115 and -191.
+I don't understand these output, but I guess that the overhead cannot be
+seen with such test.
 
-Results of pybench benchmark: "+0.1%" slower globally (diff between -4.9% and
-+5.6%).
+Results of pybench benchmark: "+0.1%" slower globally (diff between
+-4.9% and +5.6%).
 
 The full reports are attached to the issue #3329.
 
@@ -390,9 +396,9 @@
 PyMem_Malloc() reuses PyMem_RawMalloc() by default
 --------------------------------------------------
 
-``PyMem_Malloc()`` should call ``PyMem_RawMalloc()`` by default. So calling
-``PyMem_SetRawAllocator()`` would also also patch ``PyMem_Malloc()``
-indirectly.
+``PyMem_Malloc()`` should call ``PyMem_RawMalloc()`` by default. So
+calling ``PyMem_SetRawAllocator()`` would also also patch
+``PyMem_Malloc()`` indirectly.
 
 .. note::
 
@@ -404,39 +410,42 @@
 Add a new PYDEBUGMALLOC environment variable
 --------------------------------------------
 
-To be able to use the Python builtin debug hooks even when a custom memory
-allocator replaces the default Python allocator, an environment variable
-``PYDEBUGMALLOC`` can be added to setup these debug function hooks, instead of
-adding the new function ``PyMem_SetupDebugHooks()``. If the environment
-variable is present, ``PyMem_SetRawAllocator()``, ``PyMem_SetAllocator()``
-and ``PyObject_SetAllocator()`` will reinstall automatically the hook on top
-of the new allocator.
+To be able to use the Python builtin debug hooks even when a custom
+memory allocator replaces the default Python allocator, an environment
+variable ``PYDEBUGMALLOC`` can be added to setup these debug function
+hooks, instead of adding the new function ``PyMem_SetupDebugHooks()``.
+If the environment variable is present, ``PyMem_SetRawAllocator()``,
+``PyMem_SetAllocator()`` and ``PyObject_SetAllocator()`` will reinstall
+automatically the hook on top of the new allocator.
 
-An new environment variable would make the Python initialization even more
-complex. The `PEP 432 <http://www.python.org/dev/peps/pep-0432/>`_ tries to
-simply the CPython startup sequence.
+An new environment variable would make the Python initialization even
+more complex. The `PEP 432 <http://www.python.org/dev/peps/pep-0432/>`_
+tries to simply the CPython startup sequence.
 
 
 Use macros to get customizable allocators
 -----------------------------------------
 
-To have no overhead in the default configuration, customizable allocators would
-be an optional feature enabled by a configuration option or by macros.
+To have no overhead in the default configuration, customizable
+allocators would be an optional feature enabled by a configuration
+option or by macros.
 
-Not having to recompile Python makes debug hooks easier to use in practice.
-Extensions modules don't have to be recompiled with macros.
+Not having to recompile Python makes debug hooks easier to use in
+practice.  Extensions modules don't have to be recompiled with macros.
 
 
 Pass the C filename and line number
 -----------------------------------
 
-Define allocator functions using macros and use ``__FILE__`` and ``__LINE__``
-to get the C filename and line number of a memory allocation.
+Define allocator functions using macros and use ``__FILE__`` and
+``__LINE__`` to get the C filename and line number of a memory
+allocation.
 
 Example::
 
     typedef struct {
-        /* user context passed as the first argument to the 3 functions */
+        /* user context passed as the first argument
+           to the 3 functions */
         void *ctx;
 
         /* allocate a memory block */
@@ -452,7 +461,8 @@
                       void *ptr);
     } PyMemBlockAllocator;
 
-    void* _PyMem_MallocTrace(const char *filename, int lineno, size_t size);
+    void* _PyMem_MallocTrace(const char *filename, int lineno,
+                             size_t size);
 
     /* need also a function for the Python stable ABI */
     void* PyMem_Malloc(size_t size);
@@ -470,7 +480,8 @@
 GIL-free PyMem_Malloc()
 -----------------------
 
-When Python is compiled in debug mode, ``PyMem_Malloc()`` calls indirectly ``PyObject_Malloc()`` which requires the GIL to be held.
+When Python is compiled in debug mode, ``PyMem_Malloc()`` calls
+indirectly ``PyObject_Malloc()`` which requires the GIL to be held.
 That's why ``PyMem_Malloc()`` must be called with the GIL held.
 
 This PEP proposes to "fix" ``PyMem_Malloc()`` to make it always call
@@ -478,64 +489,65 @@
 ``PyMem_Malloc()``.
 
 Allowing to call ``PyMem_Malloc()`` without holding the GIL might break
-applications which setup their own allocators or allocator hooks. Holding the
-GIL is convinient to develop a custom allocator: no need to care of other
-threads. It is also convinient for a debug allocator hook: Python internal
-objects can be safetly inspected.
+applications which setup their own allocators or allocator hooks.
+Holding the GIL is convinient to develop a custom allocator: no need to
+care of other threads. It is also convinient for a debug allocator hook:
+Python internal objects can be safetly inspected.
 
-Calling ``PyGILState_Ensure()`` in a memory allocator may have unexpected
-behaviour, especially at Python startup and at creation of a new Python thread
-state.
+Calling ``PyGILState_Ensure()`` in a memory allocator may have
+unexpected behaviour, especially at Python startup and at creation of a
+new Python thread state.
 
 
 Don't add PyMem_RawMalloc()
 ---------------------------
 
-Replace ``malloc()`` with ``PyMem_Malloc()``, but only if the GIL is held.
-Otherwise, keep ``malloc()`` unchanged.
+Replace ``malloc()`` with ``PyMem_Malloc()``, but only if the GIL is
+held.  Otherwise, keep ``malloc()`` unchanged.
 
-The ``PyMem_Malloc()`` is used without the GIL held in some Python functions.
-For example, the ``main()`` and ``Py_Main()`` functions of Python call
-``PyMem_Malloc()`` whereas the GIL do not exist yet. In this case,
-``PyMem_Malloc()`` should be replaced with ``malloc()`` (or
+The ``PyMem_Malloc()`` is used without the GIL held in some Python
+functions.  For example, the ``main()`` and ``Py_Main()`` functions of
+Python call ``PyMem_Malloc()`` whereas the GIL do not exist yet. In this
+case, ``PyMem_Malloc()`` should be replaced with ``malloc()`` (or
 ``PyMem_RawMalloc()``).
 
-If an hook is used to the track memory usage, the ``malloc()`` memory will not
-be seen. Remaining ``malloc()`` may allocate a lot of memory and so would be
-missed in reports.
+If an hook is used to the track memory usage, the ``malloc()`` memory
+will not be seen. Remaining ``malloc()`` may allocate a lot of memory
+and so would be missed in reports.
 
 
 Use existing debug tools to analyze the memory
 ----------------------------------------------
 
-There are many existing debug tools to analyze the memory. Some examples:
-`Valgrind <http://valgrind.org/>`_,
-`Purify <http://ibm.com/software/awdtools/purify/>`_,
-`Clang AddressSanitizer <http://code.google.com/p/address-sanitizer/>`_,
-`failmalloc <http://www.nongnu.org/failmalloc/>`_,
-etc.
+There are many existing debug tools to analyze the memory. Some
+examples: `Valgrind <http://valgrind.org/>`_, `Purify
+<http://ibm.com/software/awdtools/purify/>`_, `Clang AddressSanitizer
+<http://code.google.com/p/address-sanitizer/>`_, `failmalloc
+<http://www.nongnu.org/failmalloc/>`_, etc.
 
-The problem is to retrieve the Python object related to a memory pointer to read
-its type and/or content. Another issue is to retrieve the location of the
-memory allocation: the C backtrace is usually useless (same reasoning than
-macros using ``__FILE__`` and ``__LINE__``), the Python filename and line
-number (or even the Python traceback) is more useful.
+The problem is to retrieve the Python object related to a memory pointer
+to read its type and/or content. Another issue is to retrieve the
+location of the memory allocation: the C backtrace is usually useless
+(same reasoning than macros using ``__FILE__`` and ``__LINE__``), the
+Python filename and line number (or even the Python traceback) is more
+useful.
 
 Classic tools are unable to introspect Python internals to collect such
-information. Being able to setup a hook on allocators called with the GIL held
-allow to collect a lot of useful data from Python internals.
+information. Being able to setup a hook on allocators called with the
+GIL held allow to collect a lot of useful data from Python internals.
 
 
 Add msize()
 -----------
 
-Add another field to ``PyMemBlockAllocator`` and ``PyMemMappingAllocator``::
+Add another field to ``PyMemBlockAllocator`` and
+``PyMemMappingAllocator``::
 
     size_t msize(void *ptr);
 
-This function returns the size of a memory block or a memory mapping. Return
-(size_t)-1 if the function is not implemented or if the pointer is unknown
-(ex: NULL pointer).
+This function returns the size of a memory block or a memory mapping.
+Return (size_t)-1 if the function is not implemented or if the pointer
+is unknown (ex: NULL pointer).
 
 On Windows, this function can be implemented using ``_msize()`` and
 ``VirtualQuery()``.
@@ -544,24 +556,25 @@
 No context argument
 -------------------
 
-Simplify the signature of allocator functions, remove the context argument:
+Simplify the signature of allocator functions, remove the context
+argument:
 
 * ``void* malloc(size_t size)``
 * ``void* realloc(void *ptr, size_t new_size)``
 * ``void free(void *ptr)``
 
-It is likely for an allocator hook to be reused for ``PyMem_SetAllocator()``
-and ``PyObject_SetAllocator()``, or even ``PyMem_SetRawAllocator()``, but the
-hook must call a different function depending on the allocator. The context is
-a convenient way to reuse the same custom allocator or hook for different
-Python allocators.
+It is likely for an allocator hook to be reused for
+``PyMem_SetAllocator()`` and ``PyObject_SetAllocator()``, or even
+``PyMem_SetRawAllocator()``, but the hook must call a different function
+depending on the allocator. The context is a convenient way to reuse the
+same custom allocator or hook for different Python allocators.
 
 
 External libraries
 ==================
 
-Python should try to reuse the same prototypes for allocator functions than
-other libraries.
+Python should try to reuse the same prototypes for allocator functions
+than other libraries.
 
 Libraries used by Python:
 
@@ -586,36 +599,41 @@
 Memory allocators
 =================
 
-The C standard library provides the well known ``malloc()`` function. Its
-implementation depends on the platform and of the C library. The GNU C library
-uses a modified ptmalloc2, based on "Doug Lea's Malloc" (dlmalloc). FreeBSD
-uses `jemalloc <http://www.canonware.com/jemalloc/>`_. Google provides
-tcmalloc which is part of `gperftools <http://code.google.com/p/gperftools/>`_.
+The C standard library provides the well known ``malloc()`` function.
+Its implementation depends on the platform and of the C library. The GNU
+C library uses a modified ptmalloc2, based on "Doug Lea's Malloc"
+(dlmalloc). FreeBSD uses `jemalloc
+<http://www.canonware.com/jemalloc/>`_. Google provides tcmalloc which
+is part of `gperftools <http://code.google.com/p/gperftools/>`_.
 
 ``malloc()`` uses two kinds of memory: heap and memory mappings. Memory
-mappings are usually used for large allocations (ex: larger than 256 KB),
-whereas the heap is used for small allocations.
+mappings are usually used for large allocations (ex: larger than 256
+KB), whereas the heap is used for small allocations.
 
-On UNIX, the heap is handled by ``brk()`` and ``sbrk()`` system calls on Linux,
-and it is contiguous.  On Windows, the heap is handled by ``HeapAlloc()`` and
-may be discontiguous. Memory mappings are handled by ``mmap()`` on UNIX and
-``VirtualAlloc()`` on Windows, they may be discontiguous.
+On UNIX, the heap is handled by ``brk()`` and ``sbrk()`` system calls on
+Linux, and it is contiguous.  On Windows, the heap is handled by
+``HeapAlloc()`` and may be discontiguous. Memory mappings are handled by
+``mmap()`` on UNIX and ``VirtualAlloc()`` on Windows, they may be
+discontiguous.
 
-Releasing a memory mapping gives back immediatly the memory to the system. On
-UNIX, heap memory is only given back to the system if it is at the end of the
-heap. Otherwise, the memory will only be given back to the system when all the
-memory located after the released memory are also released.
+Releasing a memory mapping gives back immediatly the memory to the
+system. On UNIX, heap memory is only given back to the system if it is
+at the end of the heap. Otherwise, the memory will only be given back to
+the system when all the memory located after the released memory are
+also released.
 
-To allocate memory in the heap, the allocator tries to reuse free space. If
-there is no contiguous space big enough, the heap must be increased, even if we
-have more free space than required size.  This issue is called the "memory
-fragmentation": the memory usage seen by the system may be much higher than
-real usage. On Windows, ``HeapAlloc()`` creates a new memory mapping with
-``VirtualAlloc()`` if there is not enough free contiguous memory.
+To allocate memory in the heap, the allocator tries to reuse free space.
+If there is no contiguous space big enough, the heap must be increased,
+even if we have more free space than required size.  This issue is
+called the "memory fragmentation": the memory usage seen by the system
+may be much higher than real usage. On Windows, ``HeapAlloc()`` creates
+a new memory mapping with ``VirtualAlloc()`` if there is not enough free
+contiguous memory.
 
-CPython has a *pymalloc* allocator for allocations smaller than 512 bytes. This
-allocator is optimized for small objects with a short lifetime. It uses memory
-mappings called "arenas" with a fixed size of 256 KB.
+CPython has a *pymalloc* allocator for allocations smaller than 512
+bytes. This allocator is optimized for small objects with a short
+lifetime. It uses memory mappings called "arenas" with a fixed size of
+256 KB.
 
 Other allocators:
 
@@ -641,10 +659,10 @@
   <http://bugs.python.org/issue13483>`_
 * `Issue #16742: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which
   isn't thread safe <http://bugs.python.org/issue16742>`_
-* `Issue #18203: Replace calls to malloc() with PyMem_Malloc() or PyMem_RawMalloc()
-  <http://bugs.python.org/issue18203>`_
-* `Issue #18227: Use Python memory allocators in external libraries like zlib
-  or OpenSSL <http://bugs.python.org/issue18227>`_
+* `Issue #18203: Replace calls to malloc() with PyMem_Malloc() or
+  PyMem_RawMalloc() <http://bugs.python.org/issue18203>`_
+* `Issue #18227: Use Python memory allocators in external libraries like
+  zlib or OpenSSL <http://bugs.python.org/issue18227>`_
 
 Projects analyzing the memory usage of Python applications:
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 22:08:02 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 22:08:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_typo?=
Message-ID: <3bZgMQ0ZBCzNfc@mail.python.org>

http://hg.python.org/peps/rev/6dc665025073
changeset:   4945:6dc665025073
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 22:07:52 2013 +0200
summary:
  typo

files:
  pep-0445.txt |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -21,14 +21,14 @@
 Use cases:
 
 * Application embedding Python may want to isolate Python memory from
-  the memory of the application, or may want to different memory
+  the memory of the application, or may want to use a different memory
   allocator optimized for its Python usage
 * Python running on embedded devices with low memory and slow CPU.
   A custom memory allocator may be required to use efficiently the
   memory and/or to be able to use all the memory of the device.
 * Debug tool to:
 
-  - track memory usage (memory leaks)
+  - track the memory usage (memory leaks)
   - get the Python filename and line number where an object was
     allocated
   - detect buffer underflow, buffer overflow and detect misuse of Python

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 22:18:35 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 22:18:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZgbb4FJQzSWv@mail.python.org>

http://hg.python.org/peps/rev/4c100be61da0
changeset:   4946:4c100be61da0
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 22:18:21 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  19 ++++++++++++-------
  1 files changed, 12 insertions(+), 7 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -73,6 +73,7 @@
 
   - ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
   - ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
+  - default allocator: ``malloc()``, ``realloc()``, ``free()``
 
 * Add new functions to get and set internal functions of
   ``PyMem_Malloc()``, ``PyMem_Realloc()`` and ``PyMem_Free()``:
@@ -81,15 +82,19 @@
   - ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
   - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return
     *NULL*: it would be treated as an error.
+  - default allocator: ``malloc()``, ``realloc()``, ``free()``;
+    ``PyMem_Malloc(0)`` calls ``malloc(1)``
+    and ``PyMem_Realloc(NULL, 0)`` calls ``realloc(NULL, 1)``
 
 * Add new functions to get and set internal functions of
-  ``PyObject_Malloc()``,, ``PyObject_Realloc()`` and
+  ``PyObject_Malloc()``, ``PyObject_Realloc()`` and
   ``PyObject_Free()``:
 
   - ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
   - ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
   - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return
     *NULL*: it would be treated as an error.
+  - default allocator: the *pymalloc* allocator
 
 * Add a new ``PyMemMappingAllocator`` structure::
 
@@ -116,11 +121,14 @@
   allocators are replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
+  - the function does nothing is Python is compiled not compiled in
+    debug mode
 
 * The following memory allocators always returns *NULL* if size is
-  greater than ``PY_SSIZE_T_MAX``: ``PyMem_RawMalloc()``,
-  ``PyMem_RawRealloc()``, ``PyMem_Malloc()``, ``PyMem_Realloc()``,
-  ``PyObject_Malloc()``, ``PyObject_Realloc()``.
+  greater than ``PY_SSIZE_T_MAX`` (check before calling the internal
+  function): ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``,
+  ``PyMem_Malloc()``, ``PyMem_Realloc()``, ``PyObject_Malloc()``,
+  ``PyObject_Realloc()``.
 
 The builtin Python debug hooks were introduced in Python 2.3 and
 implement the following checks:
@@ -132,9 +140,6 @@
 * Detect write before the start of the buffer (buffer underflow)
 * Detect write after the end of the buffer (buffer overflow)
 
-The *pymalloc* allocator is used by default for:
-``PyObject_Malloc()``, ``PyObject_Realloc()`` and ``PyObject_Free()``.
-
 
 Make usage of these new APIs
 ----------------------------

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 22:21:46 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 18 Jun 2013 22:21:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjU2?=
 =?utf-8?q?=3A_Compilation_fix_for_recent_AIX_releases=2E__Patch_by_David_?=
 =?utf-8?q?Edelsohn=2E?=
Message-ID: <3bZggG1N2MzQHG@mail.python.org>

http://hg.python.org/cpython/rev/f2e373ddfa00
changeset:   84205:f2e373ddfa00
branch:      3.3
parent:      84202:7b6ae19dd116
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Jun 18 22:17:48 2013 +0200
summary:
  Issue #18256: Compilation fix for recent AIX releases.  Patch by David Edelsohn.

files:
  Misc/ACKS               |  1 +
  Misc/NEWS               |  3 +++
  Python/thread_pthread.h |  3 +++
  3 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -327,6 +327,7 @@
 Walter D?rwald
 Hans Eckardt
 Rodolpho Eckhardt
+David Edelsohn
 John Edmonds
 Grant Edwards
 John Ehresman
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -141,6 +141,9 @@
 Build
 -----
 
+- Issue #18256: Compilation fix for recent AIX releases.  Patch by
+  David Edelsohn.
+
 - Issue #15172: Document NASM 2.10+ as requirement for building OpenSSL 1.0.1
   on Windows.
 
diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h
--- a/Python/thread_pthread.h
+++ b/Python/thread_pthread.h
@@ -170,6 +170,7 @@
 PyThread__init_thread(void)
 {
 #if defined(_AIX) && defined(__GNUC__)
+    extern void pthread_init(void);
     pthread_init();
 #endif
 }
@@ -444,6 +445,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_free_lock(%p) called\n", lock));
 
     /* some pthread-like implementations tie the mutex to the cond
@@ -530,6 +532,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_release_lock(%p) called\n", lock));
 
     status = pthread_mutex_lock( &thelock->mut );

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 18 22:21:47 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 18 Jun 2013 22:21:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318256=3A_Compilation_fix_for_recent_AIX_release?=
 =?utf-8?q?s=2E__Patch_by_David_Edelsohn=2E?=
Message-ID: <3bZggH3RKkzSqX@mail.python.org>

http://hg.python.org/cpython/rev/7081859c1e20
changeset:   84206:7081859c1e20
parent:      84203:390e7fe9a1d3
parent:      84205:f2e373ddfa00
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Jun 18 22:19:11 2013 +0200
summary:
  Issue #18256: Compilation fix for recent AIX releases.  Patch by David Edelsohn.

files:
  Misc/ACKS               |  1 +
  Misc/NEWS               |  3 +++
  Python/thread_pthread.h |  3 +++
  3 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -334,6 +334,7 @@
 Walter D?rwald
 Hans Eckardt
 Rodolpho Eckhardt
+David Edelsohn
 John Edmonds
 Grant Edwards
 John Ehresman
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -537,6 +537,9 @@
 Build
 -----
 
+- Issue #18256: Compilation fix for recent AIX releases.  Patch by
+  David Edelsohn.
+
 - Issue #17547: In configure, explicitly pass -Wformat for the benefit for GCC
   4.8.
 
diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h
--- a/Python/thread_pthread.h
+++ b/Python/thread_pthread.h
@@ -170,6 +170,7 @@
 PyThread__init_thread(void)
 {
 #if defined(_AIX) && defined(__GNUC__)
+    extern void pthread_init(void);
     pthread_init();
 #endif
 }
@@ -444,6 +445,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_free_lock(%p) called\n", lock));
 
     /* some pthread-like implementations tie the mutex to the cond
@@ -530,6 +532,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_release_lock(%p) called\n", lock));
 
     status = pthread_mutex_lock( &thelock->mut );

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 18 22:21:48 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 18 Jun 2013 22:21:48 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MjU2?=
 =?utf-8?q?=3A_Compilation_fix_for_recent_AIX_releases=2E__Patch_by_David_?=
 =?utf-8?q?Edelsohn=2E?=
Message-ID: <3bZggJ5RW0z7Ljv@mail.python.org>

http://hg.python.org/cpython/rev/a5ef439f3c9e
changeset:   84207:a5ef439f3c9e
branch:      2.7
parent:      84204:4f4433b12757
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Jun 18 22:17:48 2013 +0200
summary:
  Issue #18256: Compilation fix for recent AIX releases.  Patch by David Edelsohn.

files:
  Misc/ACKS               |  1 +
  Misc/NEWS               |  3 +++
  Python/thread_pthread.h |  3 +++
  3 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -270,6 +270,7 @@
 Maxim Dzumanenko
 Walter D?rwald
 Hans Eckardt
+David Edelsohn
 Grant Edwards
 John Ehresman
 Eric Eisner
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -49,6 +49,9 @@
 Build
 -----
 
+- Issue #18256: Compilation fix for recent AIX releases.  Patch by
+  David Edelsohn.
+
 - Issue #18098: The deprecated OS X Build Applet.app fails to build on
   OS X 10.8 systems because the Apple-deprecated QuickDraw headers have
   been removed from Xcode 4.  Skip building it in this case.
diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h
--- a/Python/thread_pthread.h
+++ b/Python/thread_pthread.h
@@ -145,6 +145,7 @@
 PyThread__init_thread(void)
 {
 #if defined(_AIX) && defined(__GNUC__)
+    extern void pthread_init(void);
     pthread_init();
 #endif
 }
@@ -394,6 +395,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_free_lock(%p) called\n", lock));
 
     status = pthread_mutex_destroy( &thelock->mut );
@@ -445,6 +447,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_release_lock(%p) called\n", lock));
 
     status = pthread_mutex_lock( &thelock->mut );

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 18 22:33:55 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 18 Jun 2013 22:33:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445?=
Message-ID: <3bZgxH0Vgpz7Ljy@mail.python.org>

http://hg.python.org/peps/rev/c71887660ccd
changeset:   4947:c71887660ccd
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 18 22:33:41 2013 +0200
summary:
  PEP 445

files:
  pep-0445.txt |  68 ++++++++++++++++++++--------------------
  1 files changed, 34 insertions(+), 34 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -1,5 +1,5 @@
 PEP: 445
-Title: Add new APIs to customize memory allocators
+Title: Add new APIs to customize Python memory allocators
 Version: $Revision$
 Last-Modified: $Date$
 Author: Victor Stinner <victor.stinner at gmail.com>
@@ -12,7 +12,7 @@
 Abstract
 ========
 
-Add new APIs to customize memory allocators.
+Add new APIs to customize Python memory allocators.
 
 
 Rationale
@@ -110,7 +110,7 @@
         void (*free) (void *ctx, void *ptr, size_t size);
     } PyMemMappingAllocator;
 
-* Add a new function to get and set memory mapping allocator:
+* Add a new function to get and set the memory mapping allocator:
 
   - ``void PyMem_GetMappingAllocator(PyMemMappingAllocator *allocator)``
   - ``void PyMem_SetMappingAllocator(PyMemMappingAllocator *allocator)``
@@ -141,8 +141,8 @@
 * Detect write after the end of the buffer (buffer overflow)
 
 
-Make usage of these new APIs
-----------------------------
+Other changes
+-------------
 
 * ``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()``
   and ``realloc()``, instead of calling ``PyObject_Malloc()`` and
@@ -166,12 +166,13 @@
 Use case 1: Replace Memory Allocator, keep pymalloc
 ----------------------------------------------------
 
-Dummy example wasting 2 bytes per allocation, and 10 bytes per arena::
+Dummy example wasting 2 bytes per memory block,
+and 10 bytes per memory mapping::
 
     #include <stdlib.h>
 
-    int alloc_padding = 2;
-    int arena_padding = 10;
+    int block_padding = 2;
+    int mapping_padding = 10;
 
     void* my_malloc(void *ctx, size_t size)
     {
@@ -190,13 +191,13 @@
         free(ptr);
     }
 
-    void* my_alloc_arena(void *ctx, size_t size)
+    void* my_alloc_mapping(void *ctx, size_t size)
     {
         int padding = *(int *)ctx;
         return malloc(size + padding);
     }
 
-    void my_free_arena(void *ctx, void *ptr, size_t size)
+    void my_free_mapping(void *ctx, void *ptr, size_t size)
     {
         free(ptr);
     }
@@ -206,7 +207,7 @@
         PyMemBlockAllocator block;
         PyMemMappingAllocator mapping;
 
-        block.ctx = &alloc_padding;
+        block.ctx = &block_padding;
         block.malloc = my_malloc;
         block.realloc = my_realloc;
         block.free = my_free;
@@ -214,9 +215,9 @@
         PyMem_SetRawAllocator(&block);
         PyMem_SetAllocator(&block);
 
-        mapping.ctx = &arena_padding;
-        mapping.alloc = my_alloc_arena;
-        mapping.free = my_free_arena;
+        mapping.ctx = &mapping_padding;
+        mapping.alloc = my_alloc_mapping;
+        mapping.free = my_free_mapping;
         PyMem_SetMappingAllocator(mapping);
 
         PyMem_SetupDebugHooks();
@@ -231,10 +232,10 @@
 --------------------------------------------------------
 
 If your allocator is optimized for allocation of small objects (less
-than 512 bytes) with a short lifetime, pymalloc can be overriden:
-replace ``PyObject_Malloc()``.
+than 512 bytes) with a short lifetime, pymalloc can be overriden
+(replace ``PyObject_Malloc()``).
 
-Dummy Example wasting 2 bytes per allocation::
+Dummy example wasting 2 bytes per memory block::
 
     #include <stdlib.h>
 
@@ -358,8 +359,6 @@
 Results of the `Python benchmarks suite
 <http://hg.python.org/benchmarks>`_ (-b 2n3): some tests are 1.04x
 faster, some tests are 1.04 slower, significant is between 115 and -191.
-I don't understand these output, but I guess that the overhead cannot be
-seen with such test.
 
 Results of pybench benchmark: "+0.1%" slower globally (diff between
 -4.9% and +5.6%).
@@ -370,17 +369,17 @@
 Alternatives
 ============
 
-Only have one generic get/set function
---------------------------------------
+Only one get/set function for block allocators
+----------------------------------------------
 
 Replace the 6 functions:
 
-* ``PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
-* ``PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
-* ``PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
-* ``PyMem_SetRawAllocator(allocator)``
-* ``PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
-* ``PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
 
 with 2 functions with an additional *domain* argument:
 
@@ -398,8 +397,8 @@
 Drawback: the caller has to check if the result is 0, or handle the error.
 
 
-PyMem_Malloc() reuses PyMem_RawMalloc() by default
---------------------------------------------------
+Make PyMem_Malloc() reuse PyMem_RawMalloc() by default
+------------------------------------------------------
 
 ``PyMem_Malloc()`` should call ``PyMem_RawMalloc()`` by default. So
 calling ``PyMem_SetRawAllocator()`` would also also patch
@@ -442,11 +441,11 @@
 Pass the C filename and line number
 -----------------------------------
 
-Define allocator functions using macros and use ``__FILE__`` and
-``__LINE__`` to get the C filename and line number of a memory
-allocation.
+Define allocator functions as macros using ``__FILE__`` and ``__LINE__``
+to get the C filename and line number of a memory allocation.
 
-Example::
+Example of ``PyMem_Malloc`` macro with the modified
+``PyMemBlockAllocator`` structure::
 
     typedef struct {
         /* user context passed as the first argument
@@ -472,7 +471,8 @@
     /* need also a function for the Python stable ABI */
     void* PyMem_Malloc(size_t size);
 
-    #define PyMem_Malloc(size) _PyMem_MallocTrace(__FILE__, __LINE__, size)
+    #define PyMem_Malloc(size) \
+            _PyMem_MallocTrace(__FILE__, __LINE__, size)
 
 Passing a filename and a line number to each allocator makes the API more
 complex: pass 3 new arguments, instead of just a context argument, to each

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Tue Jun 18 23:28:25 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Tue, 18 Jun 2013 23:28:25 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_compilation_warning_wi?=
 =?utf-8?q?th_gcc_4=2E8_=28unused_typedef=29?=
Message-ID: <3bZj895ythzSyF@mail.python.org>

http://hg.python.org/cpython/rev/626a8e49f2a9
changeset:   84208:626a8e49f2a9
parent:      84206:7081859c1e20
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Tue Jun 18 23:28:18 2013 +0200
summary:
  Fix compilation warning with gcc 4.8 (unused typedef)

files:
  Objects/setobject.c |  1 -
  1 files changed, 0 insertions(+), 1 deletions(-)


diff --git a/Objects/setobject.c b/Objects/setobject.c
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -214,7 +214,6 @@
 set_insert_key(register PySetObject *so, PyObject *key, Py_hash_t hash)
 {
     register setentry *entry;
-    typedef setentry *(*lookupfunc)(PySetObject *, PyObject *, Py_hash_t);
 
     assert(so->lookup != NULL);
     entry = so->lookup(so, key, hash);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:09:11 2013
From: python-checkins at python.org (christian.heimes)
Date: Wed, 19 Jun 2013 02:09:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjU5?=
 =?utf-8?q?=3A_Declare_sethostname_in_socketmodule=2Ec_for_AIX?=
Message-ID: <3bZmjg306Zz7Ljh@mail.python.org>

http://hg.python.org/cpython/rev/9f8efcd78d0d
changeset:   84209:9f8efcd78d0d
branch:      3.3
parent:      84202:7b6ae19dd116
user:        Christian Heimes <christian at cheimes.de>
date:        Wed Jun 19 02:06:29 2013 +0200
summary:
  Issue #18259: Declare sethostname in socketmodule.c for AIX

files:
  Misc/NEWS              |  2 ++
  Modules/socketmodule.c |  5 +++++
  2 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -32,6 +32,8 @@
 Library
 -------
 
+- Issue #18259: Declare sethostname in socketmodule.c for AIX
+
 - Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
   when \r\n appears at end of 65535 bytes without other newlines.
 
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -4132,6 +4132,11 @@
     Py_buffer buf;
     int res, flag = 0;
 
+#ifdef _AIX
+/* issue #18259, not declared in any useful header file */
+extern int sethostname(const char *, size_t);
+#endif
+
     if (!PyArg_ParseTuple(args, "S:sethostname", &hnobj)) {
         PyErr_Clear();
         if (!PyArg_ParseTuple(args, "O&:sethostname",

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:09:12 2013
From: python-checkins at python.org (christian.heimes)
Date: Wed, 19 Jun 2013 02:09:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318259=3A_Declare_sethostname_in_socketmodule=2E?=
 =?utf-8?q?c_for_AIX?=
Message-ID: <3bZmjh58H6z7Ljn@mail.python.org>

http://hg.python.org/cpython/rev/14748397fc57
changeset:   84210:14748397fc57
parent:      84203:390e7fe9a1d3
parent:      84209:9f8efcd78d0d
user:        Christian Heimes <christian at cheimes.de>
date:        Wed Jun 19 02:07:20 2013 +0200
summary:
  Issue #18259: Declare sethostname in socketmodule.c for AIX

files:
  Misc/NEWS              |  2 ++
  Modules/socketmodule.c |  5 +++++
  2 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #18259: Declare sethostname in socketmodule.c for AIX
+
 - Issue #18147: Add diagnostic functions to ssl.SSLContext(). get_ca_list()
   lists all loaded CA certificates and cert_store_stats() returns amount of
   loaded X.509 certs, X.509 CA certs and CRLs.
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -4066,6 +4066,11 @@
     Py_buffer buf;
     int res, flag = 0;
 
+#ifdef _AIX
+/* issue #18259, not declared in any useful header file */
+extern int sethostname(const char *, size_t);
+#endif
+
     if (!PyArg_ParseTuple(args, "S:sethostname", &hnobj)) {
         PyErr_Clear();
         if (!PyArg_ParseTuple(args, "O&:sethostname",

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:09:14 2013
From: python-checkins at python.org (christian.heimes)
Date: Wed, 19 Jun 2013 02:09:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge_heads?=
Message-ID: <3bZmjk0GlKz7Lk3@mail.python.org>

http://hg.python.org/cpython/rev/ac94b96b1aa7
changeset:   84211:ac94b96b1aa7
parent:      84210:14748397fc57
parent:      84208:626a8e49f2a9
user:        Christian Heimes <christian at cheimes.de>
date:        Wed Jun 19 02:08:41 2013 +0200
summary:
  merge heads

files:
  Misc/ACKS               |  1 +
  Misc/NEWS               |  3 +++
  Objects/setobject.c     |  1 -
  Python/thread_pthread.h |  3 +++
  4 files changed, 7 insertions(+), 1 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -334,6 +334,7 @@
 Walter D?rwald
 Hans Eckardt
 Rodolpho Eckhardt
+David Edelsohn
 John Edmonds
 Grant Edwards
 John Ehresman
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -539,6 +539,9 @@
 Build
 -----
 
+- Issue #18256: Compilation fix for recent AIX releases.  Patch by
+  David Edelsohn.
+
 - Issue #17547: In configure, explicitly pass -Wformat for the benefit for GCC
   4.8.
 
diff --git a/Objects/setobject.c b/Objects/setobject.c
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -214,7 +214,6 @@
 set_insert_key(register PySetObject *so, PyObject *key, Py_hash_t hash)
 {
     register setentry *entry;
-    typedef setentry *(*lookupfunc)(PySetObject *, PyObject *, Py_hash_t);
 
     assert(so->lookup != NULL);
     entry = so->lookup(so, key, hash);
diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h
--- a/Python/thread_pthread.h
+++ b/Python/thread_pthread.h
@@ -170,6 +170,7 @@
 PyThread__init_thread(void)
 {
 #if defined(_AIX) && defined(__GNUC__)
+    extern void pthread_init(void);
     pthread_init();
 #endif
 }
@@ -444,6 +445,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_free_lock(%p) called\n", lock));
 
     /* some pthread-like implementations tie the mutex to the cond
@@ -530,6 +532,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_release_lock(%p) called\n", lock));
 
     status = pthread_mutex_lock( &thelock->mut );

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:09:15 2013
From: python-checkins at python.org (christian.heimes)
Date: Wed, 19 Jun 2013 02:09:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuMyk6?=
	=?utf-8?q?_merge_heads?=
Message-ID: <3bZmjl2HFpz7Lk3@mail.python.org>

http://hg.python.org/cpython/rev/34e648f2d5f5
changeset:   84212:34e648f2d5f5
branch:      3.3
parent:      84209:9f8efcd78d0d
parent:      84205:f2e373ddfa00
user:        Christian Heimes <christian at cheimes.de>
date:        Wed Jun 19 02:09:00 2013 +0200
summary:
  merge heads

files:
  Misc/ACKS               |  1 +
  Misc/NEWS               |  3 +++
  Python/thread_pthread.h |  3 +++
  3 files changed, 7 insertions(+), 0 deletions(-)


diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -327,6 +327,7 @@
 Walter D?rwald
 Hans Eckardt
 Rodolpho Eckhardt
+David Edelsohn
 John Edmonds
 Grant Edwards
 John Ehresman
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -143,6 +143,9 @@
 Build
 -----
 
+- Issue #18256: Compilation fix for recent AIX releases.  Patch by
+  David Edelsohn.
+
 - Issue #15172: Document NASM 2.10+ as requirement for building OpenSSL 1.0.1
   on Windows.
 
diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h
--- a/Python/thread_pthread.h
+++ b/Python/thread_pthread.h
@@ -170,6 +170,7 @@
 PyThread__init_thread(void)
 {
 #if defined(_AIX) && defined(__GNUC__)
+    extern void pthread_init(void);
     pthread_init();
 #endif
 }
@@ -444,6 +445,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_free_lock(%p) called\n", lock));
 
     /* some pthread-like implementations tie the mutex to the cond
@@ -530,6 +532,7 @@
     pthread_lock *thelock = (pthread_lock *)lock;
     int status, error = 0;
 
+    (void) error; /* silence unused-but-set-variable warning */
     dprintf(("PyThread_release_lock(%p) called\n", lock));
 
     status = pthread_mutex_lock( &thelock->mut );

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:44:26 2013
From: python-checkins at python.org (christian.heimes)
Date: Wed, 19 Jun 2013 02:44:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_fixed_libffi_o?=
 =?utf-8?q?n_PPC_without_=5F=5FNO=5FFPRS=5F=5F?=
Message-ID: <3bZnVL578vz7LjW@mail.python.org>

http://hg.python.org/cpython/rev/fd327ac2e7bd
changeset:   84213:fd327ac2e7bd
branch:      3.3
user:        Christian Heimes <christian at cheimes.de>
date:        Wed Jun 19 02:40:38 2013 +0200
summary:
  fixed libffi on PPC without __NO_FPRS__
ISO C90 forbids mixed declarations and code

files:
  Modules/_ctypes/libffi/src/powerpc/ffi.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Modules/_ctypes/libffi/src/powerpc/ffi.c b/Modules/_ctypes/libffi/src/powerpc/ffi.c
--- a/Modules/_ctypes/libffi/src/powerpc/ffi.c
+++ b/Modules/_ctypes/libffi/src/powerpc/ffi.c
@@ -146,12 +146,14 @@
   } p_argv;
   size_t struct_copy_size;
   unsigned gprvalue;
+#ifndef __NO_FPRS__
+  double double_tmp;
+#endif
 
   stacktop.c = (char *) stack + bytes;
   gpr_base.u = stacktop.u - ASM_NEEDS_REGISTERS - NUM_GPR_ARG_REGISTERS;
   intarg_count = 0;
 #ifndef __NO_FPRS__
-  double double_tmp;
   fpr_base.d = gpr_base.d - NUM_FPR_ARG_REGISTERS;
   fparg_count = 0;
   copy_space.c = ((flags & FLAG_FP_ARGUMENTS) ? fpr_base.c : gpr_base.c);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:44:28 2013
From: python-checkins at python.org (christian.heimes)
Date: Wed, 19 Jun 2013 02:44:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_fixed_libffi_on_PPC_without_=5F=5FNO=5FFPRS=5F=5F?=
Message-ID: <3bZnVN02Gtz7LjW@mail.python.org>

http://hg.python.org/cpython/rev/1638f28f3b2f
changeset:   84214:1638f28f3b2f
parent:      84211:ac94b96b1aa7
parent:      84213:fd327ac2e7bd
user:        Christian Heimes <christian at cheimes.de>
date:        Wed Jun 19 02:43:19 2013 +0200
summary:
  fixed libffi on PPC without __NO_FPRS__
ISO C90 forbids mixed declarations and code

files:
  Modules/_ctypes/libffi/src/powerpc/ffi.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Modules/_ctypes/libffi/src/powerpc/ffi.c b/Modules/_ctypes/libffi/src/powerpc/ffi.c
--- a/Modules/_ctypes/libffi/src/powerpc/ffi.c
+++ b/Modules/_ctypes/libffi/src/powerpc/ffi.c
@@ -146,12 +146,14 @@
   } p_argv;
   size_t struct_copy_size;
   unsigned gprvalue;
+#ifndef __NO_FPRS__
+  double double_tmp;
+#endif
 
   stacktop.c = (char *) stack + bytes;
   gpr_base.u = stacktop.u - ASM_NEEDS_REGISTERS - NUM_GPR_ARG_REGISTERS;
   intarg_count = 0;
 #ifndef __NO_FPRS__
-  double double_tmp;
   fpr_base.d = gpr_base.d - NUM_FPR_ARG_REGISTERS;
   fparg_count = 0;
   copy_space.c = ((flags & FLAG_FP_ARGUMENTS) ? fpr_base.c : gpr_base.c);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:51:51 2013
From: python-checkins at python.org (brett.cannon)
Date: Wed, 19 Jun 2013 02:51:51 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317177=3A_Clarify_?=
	=?utf-8?q?some_deprecations?=
Message-ID: <3bZnfv41Rtz7LjX@mail.python.org>

http://hg.python.org/cpython/rev/ded443c603f0
changeset:   84215:ded443c603f0
parent:      84208:626a8e49f2a9
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 18 20:49:55 2013 -0400
summary:
  Issue #17177: Clarify some deprecations

files:
  Doc/library/imp.rst |  14 +++++++++++---
  1 files changed, 11 insertions(+), 3 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -103,8 +103,10 @@
    using a :keyword:`try` ... :keyword:`finally` statement.
 
    .. deprecated:: 3.3
-      Unneeded as loaders should be used to load modules and
-      :func:`find_module` is deprecated.
+      If previously used in conjunction with :func:`imp.find_module` then
+      call ``load_module()`` on the returned loader. If you wish to load a
+      module from a specific file, then use one of the file-based loaders found
+      in :mod:`importlib.machinery`.
 
 
 .. function:: new_module(name)
@@ -233,7 +235,7 @@
    magic number, as returned by :func:`get_magic`.
 
    .. deprecated:: 3.4
-      You may use :attr:`sys.implementation.cache_tag` directly starting
+      Use :attr:`sys.implementation.cache_tag` directly starting
       in Python 3.3.
 
 
@@ -261,6 +263,8 @@
    the most part.  A global import lock is kept for some critical tasks,
    such as initializing the per-module locks.
 
+.. deprecated:: 3.4
+
 
 .. function:: acquire_lock()
 
@@ -279,6 +283,8 @@
    the most part.  A global import lock is kept for some critical tasks,
    such as initializing the per-module locks.
 
+.. deprecated:: 3.4
+
 
 .. function:: release_lock()
 
@@ -290,6 +296,8 @@
    the most part.  A global import lock is kept for some critical tasks,
    such as initializing the per-module locks.
 
+.. deprecated:: 3.4
+
 
 The following constants with integer values, defined in this module, are used
 to indicate the search result of :func:`find_module`.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 02:51:52 2013
From: python-checkins at python.org (brett.cannon)
Date: Wed, 19 Jun 2013 02:51:52 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bZnfw6gKHz7Ljw@mail.python.org>

http://hg.python.org/cpython/rev/338be75f11c1
changeset:   84216:338be75f11c1
parent:      84215:ded443c603f0
parent:      84214:1638f28f3b2f
user:        Brett Cannon <brett at python.org>
date:        Tue Jun 18 20:51:42 2013 -0400
summary:
  merge

files:
  Misc/NEWS                                |  2 ++
  Modules/_ctypes/libffi/src/powerpc/ffi.c |  4 +++-
  Modules/socketmodule.c                   |  5 +++++
  3 files changed, 10 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #18259: Declare sethostname in socketmodule.c for AIX
+
 - Issue #18147: Add diagnostic functions to ssl.SSLContext(). get_ca_list()
   lists all loaded CA certificates and cert_store_stats() returns amount of
   loaded X.509 certs, X.509 CA certs and CRLs.
diff --git a/Modules/_ctypes/libffi/src/powerpc/ffi.c b/Modules/_ctypes/libffi/src/powerpc/ffi.c
--- a/Modules/_ctypes/libffi/src/powerpc/ffi.c
+++ b/Modules/_ctypes/libffi/src/powerpc/ffi.c
@@ -146,12 +146,14 @@
   } p_argv;
   size_t struct_copy_size;
   unsigned gprvalue;
+#ifndef __NO_FPRS__
+  double double_tmp;
+#endif
 
   stacktop.c = (char *) stack + bytes;
   gpr_base.u = stacktop.u - ASM_NEEDS_REGISTERS - NUM_GPR_ARG_REGISTERS;
   intarg_count = 0;
 #ifndef __NO_FPRS__
-  double double_tmp;
   fpr_base.d = gpr_base.d - NUM_FPR_ARG_REGISTERS;
   fparg_count = 0;
   copy_space.c = ((flags & FLAG_FP_ARGUMENTS) ? fpr_base.c : gpr_base.c);
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -4066,6 +4066,11 @@
     Py_buffer buf;
     int res, flag = 0;
 
+#ifdef _AIX
+/* issue #18259, not declared in any useful header file */
+extern int sethostname(const char *, size_t);
+#endif
+
     if (!PyArg_ParseTuple(args, "S:sethostname", &hnobj)) {
         PyErr_Clear();
         if (!PyArg_ParseTuple(args, "O&:sethostname",

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Wed Jun 19 05:50:15 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 19 Jun 2013 05:50:15 +0200
Subject: [Python-checkins] Daily reference leaks (338be75f11c1): sum=1
Message-ID: <E1Up9PT-0003SA-6m@ap.vmr.nerim.net>

results for 338be75f11c1 on branch "default"
--------------------------------------------

test_concurrent_futures leaked [0, -2, 3] memory blocks, sum=1


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogLwfyOZ', '-x']

From python-checkins at python.org  Wed Jun 19 09:31:50 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 19 Jun 2013 09:31:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjAy?=
 =?utf-8?q?=3A_Fix_minor_bugs_and_cleanup_test=5Fcoding=2Epy=2E?=
Message-ID: <3bZyXQ467Fz7Ll5@mail.python.org>

http://hg.python.org/cpython/rev/410ea970866e
changeset:   84217:410ea970866e
branch:      3.3
parent:      84213:fd327ac2e7bd
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Jun 19 10:23:35 2013 +0300
summary:
  Issue #18202: Fix minor bugs and cleanup test_coding.py.

files:
  Lib/test/test_coding.py |  32 +++++++++++++---------------
  1 files changed, 15 insertions(+), 17 deletions(-)


diff --git a/Lib/test/test_coding.py b/Lib/test/test_coding.py
--- a/Lib/test/test_coding.py
+++ b/Lib/test/test_coding.py
@@ -1,4 +1,4 @@
-import test.support, unittest
+import unittest
 from test.support import TESTFN, unlink, unload
 import importlib, os, sys
 
@@ -22,31 +22,31 @@
 
     def test_exec_valid_coding(self):
         d = {}
-        exec('# coding: cp949\na = 5\n', d)
-        self.assertEqual(d['a'], 5)
+        exec(b'# coding: cp949\na = "\xaa\xa7"\n', d)
+        self.assertEqual(d['a'], '\u3047')
 
     def test_file_parse(self):
         # issue1134: all encodings outside latin-1 and utf-8 fail on
         # multiline strings and long lines (>512 columns)
         unload(TESTFN)
+        filename = TESTFN + ".py"
+        f = open(filename, "w", encoding="cp1252")
         sys.path.insert(0, os.curdir)
-        filename = TESTFN + ".py"
-        f = open(filename, "w")
         try:
-            f.write("# -*- coding: cp1252 -*-\n")
-            f.write("'''A short string\n")
-            f.write("'''\n")
-            f.write("'A very long string %s'\n" % ("X" * 1000))
-            f.close()
+            with f:
+                f.write("# -*- coding: cp1252 -*-\n")
+                f.write("'''A short string\n")
+                f.write("'''\n")
+                f.write("'A very long string %s'\n" % ("X" * 1000))
 
             importlib.invalidate_caches()
             __import__(TESTFN)
         finally:
-            f.close()
+            del sys.path[0]
             unlink(filename)
             unlink(filename + "c")
+            unlink(filename + "o")
             unload(TESTFN)
-            del sys.path[0]
 
     def test_error_from_string(self):
         # See http://bugs.python.org/issue6289
@@ -55,11 +55,9 @@
             compile(input, "<string>", "exec")
         expected = "'ascii' codec can't decode byte 0xe2 in position 16: " \
                    "ordinal not in range(128)"
-        self.assertTrue(c.exception.args[0].startswith(expected))
+        self.assertTrue(c.exception.args[0].startswith(expected),
+                        msg=c.exception.args[0])
 
 
-def test_main():
-    test.support.run_unittest(CodingTest)
-
 if __name__ == "__main__":
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 09:31:51 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Wed, 19 Jun 2013 09:31:51 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318202=3A_Fix_minor_bugs_and_cleanup_test=5Fsour?=
 =?utf-8?b?Y2VfZW5jb2RpbmcucHku?=
Message-ID: <3bZyXR6RJ9z7Ll5@mail.python.org>

http://hg.python.org/cpython/rev/959f4ce4d590
changeset:   84218:959f4ce4d590
parent:      84216:338be75f11c1
parent:      84217:410ea970866e
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Wed Jun 19 10:29:58 2013 +0300
summary:
  Issue #18202: Fix minor bugs and cleanup test_source_encoding.py.

files:
  Lib/test/test_source_encoding.py |  25 ++++++++++---------
  1 files changed, 13 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_source_encoding.py b/Lib/test/test_source_encoding.py
--- a/Lib/test/test_source_encoding.py
+++ b/Lib/test/test_source_encoding.py
@@ -94,31 +94,31 @@
 
     def test_exec_valid_coding(self):
         d = {}
-        exec('# coding: cp949\na = 5\n', d)
-        self.assertEqual(d['a'], 5)
+        exec(b'# coding: cp949\na = "\xaa\xa7"\n', d)
+        self.assertEqual(d['a'], '\u3047')
 
     def test_file_parse(self):
         # issue1134: all encodings outside latin-1 and utf-8 fail on
         # multiline strings and long lines (>512 columns)
         unload(TESTFN)
+        filename = TESTFN + ".py"
+        f = open(filename, "w", encoding="cp1252")
         sys.path.insert(0, os.curdir)
-        filename = TESTFN + ".py"
-        f = open(filename, "w")
         try:
-            f.write("# -*- coding: cp1252 -*-\n")
-            f.write("'''A short string\n")
-            f.write("'''\n")
-            f.write("'A very long string %s'\n" % ("X" * 1000))
-            f.close()
+            with f:
+                f.write("# -*- coding: cp1252 -*-\n")
+                f.write("'''A short string\n")
+                f.write("'''\n")
+                f.write("'A very long string %s'\n" % ("X" * 1000))
 
             importlib.invalidate_caches()
             __import__(TESTFN)
         finally:
-            f.close()
+            del sys.path[0]
             unlink(filename)
             unlink(filename + "c")
+            unlink(filename + "o")
             unload(TESTFN)
-            del sys.path[0]
 
     def test_error_from_string(self):
         # See http://bugs.python.org/issue6289
@@ -127,7 +127,8 @@
             compile(input, "<string>", "exec")
         expected = "'ascii' codec can't decode byte 0xe2 in position 16: " \
                    "ordinal not in range(128)"
-        self.assertTrue(c.exception.args[0].startswith(expected))
+        self.assertTrue(c.exception.args[0].startswith(expected),
+                        msg=c.exception.args[0])
 
 
 if __name__ == "__main__":

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 16:05:21 2013
From: python-checkins at python.org (nick.coghlan)
Date: Wed, 19 Jun 2013 16:05:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_426/440_update?=
Message-ID: <3bb7GT1vlGz7LkC@mail.python.org>

http://hg.python.org/peps/rev/de305af601fa
changeset:   4948:de305af601fa
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Wed Jun 19 23:56:38 2013 +1000
summary:
  PEP 426/440 update

files:
  pep-0426.txt                |  1727 ++++++++++++++--------
  pep-0426/pymeta-schema.json |   244 +++
  pep-0440.txt                |   254 +-
  3 files changed, 1439 insertions(+), 786 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -21,8 +21,7 @@
 
 This PEP describes a mechanism for publishing and exchanging metadata
 related to Python distributions. It includes specifics of the field names,
-and their semantics and
-usage.
+and their semantics and usage.
 
 This document specifies version 2.0 of the metadata format.
 Version 1.0 is specified in PEP 241.
@@ -42,7 +41,9 @@
 
    "I" in this doc refers to Nick Coghlan. Daniel and Donald either wrote or
    contributed to earlier versions, and have been providing feedback as this
-   initial draft of the JSON-based rewrite has taken shape.
+   JSON-based rewrite has taken shape. Daniel and Donald have also been
+   vetting the proposal as we go to ensure it is practical to implement for
+   both clients and index servers.
 
    Metadata 2.0 represents a major upgrade to the Python packaging ecosystem,
    and attempts to incorporate experience gained over the 15 years(!) since
@@ -63,8 +64,7 @@
    * a new PEP to define v2.0 of the sdist format
    * an updated wheel PEP (v1.1) to add pymeta.json
    * an updated installation database PEP both for pymeta.json and to add
-     a linking scheme to better support runtime selection of dependencies,
-     as well as recording which extras are currently available
+     a linking scheme to better support runtime selection of dependencies
    * a new static config PEP to standardise metadata generation and
      creation of sdists
    * PEP 439, covering a bootstrapping mechanism for ``pip``
@@ -83,138 +83,239 @@
    "rationale" section at the end of the document, as it would otherwise be
    an irrelevant distraction for future readers.
 
-
-Definitions
-===========
+Purpose
+=======
+
+The purpose of this PEP is to define a common metadata interchange format
+for communication between software publication tools and software integration
+tools in the Python ecosystem. One key aim is to support full dependency
+analysis in that ecosystem without requiring the execution of arbitrary
+Python code by those doing the analysis. Another aim is to encourage good
+software distribution practices by default, while continuing to support the
+current practices of almost all existing users of the Python Package Index
+(both publishers and integrators).
+
+The design draws on the Python community's 15 years of experience with
+distutils based software distribution, and incorporates ideas and concepts
+from other distribution systems, including Python's setuptools, pip and
+other projects, Ruby's gems, Perl's CPAN, Node.js's npm, PHP's composer
+and Linux packaging systems such as RPM and APT.
+
+
+Development, Distribution and Deployment of Python Software
+===========================================================
+
+The metadata design in this PEP is based on a particular conceptual model
+of the software development and distribution process. This model consists of
+the following phases:
+
+* Software development: this phase involves working with a source checkout
+  for a particular application to add features and fix bugs. It is
+  expected that developers in this phase will need to be able to build the
+  software, run the software's automated test suite, run project specific
+  utility scripts and publish the software.
+
+* Software publication: this phase involves taking the developed software
+  and making it available for use by software integrators. This includes
+  creating the descriptive metadata defined in this PEP, as well making the
+  software available (typically by uploading it to an index server).
+
+* Software integration: this phase involves taking published software
+  components and combining them into a coherent, integrated system. This
+  may be done directly using Python specific cross-platform tools, or it may
+  be handled through conversion to development language neutral platform
+  specific packaging systems.
+
+* Software deployment: this phase involves taking integrated software
+  components and deploying them on to the target system where the software
+  will actually execute.
+
+The publication and integration phases are collectively referred to as
+the distribution phase, and the individual software components distributed
+in that phase are referred to as "distributions".
+
+The exact details of these phases will vary greatly for particular use cases.
+Deploying a web application to a public Platform-as-a-Service provider,
+publishing a new release of a web framework or scientific library,
+creating an integrated Linux distribution or upgrading a custom application
+running in a secure enclave are all situations this metadata design should
+be able to handle.
+
+The complexity of the metadata described in this PEP thus arises directly
+from the actual complexities associated with software development,
+distribution and deployment in a wide range of scenarios.
+
+
+Supporting definitions
+----------------------
 
 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
 "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this
 document are to be interpreted as described in RFC 2119.
 
-"Distributions" are deployable software components published through an index
-server or otherwise made available for installation.
-
-"Versions" are uniquely identified snapshots of a distribution.
-
-"Distribution archives" are the packaged files which are used to publish
-and distribute the software.
-
-"Source archives" require build tools to be available on the target
-system.
+"Projects" are software components that are made available for integration.
+Projects include Python libraries, frameworks, scripts, plugins,
+applications, collections of data or other resources, and various
+combinations thereof. Public Python projects are typically registered on
+the `Python Package Index`_.
+
+"Releases" are uniquely identified snapshots of a project.
+
+"Distributions" are the packaged files which are used to publish
+and distribute a release.
+
+"Source archive" and "VCS checkout" both refer to the raw source code for
+a release, prior to creation of an sdist or binary archive.
+
+An "sdist" is a publication format providing the distribution metadata and
+and any source files that are essential to creating a binary archive for
+the distribution. Creating a binary archive from an sdist requires that
+the appropriate build tools be available on the system.
 
 "Binary archives" only require that prebuilt files be moved to the correct
 location on the target system. As Python is a dynamically bound
-cross-platform language, many "binary" archives will contain only pure
-Python source code.
+cross-platform language, many so-called "binary" archives will contain only
+pure Python source code.
+
+"Contributors" are individuals and organizations that work together to
+develop a software component.
+
+"Publishers" are individuals and organizations that make software components
+available for integration (typically by uploading distributions to an
+index server)
+
+"Integrators" are individuals and organizations that incorporate published
+distributions as components of an application or larger system.
 
 "Build tools" are automated tools intended to run on development systems,
 producing source and binary distribution archives. Build tools may also be
-invoked by installation tools in order to install software distributed as
-source archives rather than prebuilt binary archives.
+invoked by integration tools in order to build software distributed as
+sdists rather than prebuilt binary archives.
 
 "Index servers" are active distribution registries which publish version and
 dependency metadata and place constraints on the permitted metadata.
 
+"Public index servers" are index servers which allow distribution uploads
+from untrusted third parties. The `Python Package Index`_ is a public index
+server.
+
 "Publication tools" are automated tools intended to run on development
 systems and upload source and binary distribution archives to index servers.
 
-"Installation tools" are automated tools intended to run on production
-systems, consuming source and binary distribution archives from an index
-server or other designated location and deploying them to the target system.
+"Integration tools" are automated tools that consume the metadata and
+distribution archives published by an index server or other designated
+source, and make use of them in some fashion, such as installing them or
+converting them to a platform specific packaging format.
+
+"Installation tools" are integration tools specifically intended to run on
+deployment targets, consuming source and binary distribution archives from
+an index server or other designated location and deploying them to the target
+system.
 
 "Automated tools" is a collective term covering build tools, index servers,
-publication tools, installation tools and any other software that produces
+publication tools, integration tools and any other software that produces
 or consumes distribution version and dependency metadata.
 
-"Projects" refers to the developers that manage the creation of a particular
-distribution.
-
 "Legacy metadata" refers to earlier versions of this metadata specification,
 along with the supporting metadata file formats defined by the
 ``setuptools`` project.
 
-
-Development and distribution activities
-=======================================
-
-Making effective use of a common metadata format requires a common
-understanding of the most complex development and distribution model
-the format is intended to support. The metadata format described in this
-PEP is based on the following activities:
-
-* Development: during development, a user is operating from a
-  source checkout (or equivalent) for the current project. Dependencies must
-  be available in order to build, test and create a source archive of the
-  distribution.
-
-  .. note::
-     As a generated file, the full distribution metadata often won't be
-     available in a raw source checkout or tarball. In such cases, the
-     relevant distribution metadata is generally obtained from another
-     location, such as the last published release, or by generating it
-     based on a command given in a standard input file. This spec
-     deliberately avoids handling that scenario, instead falling back on
-     the existing ``setup.py`` functionality.
-
-* Build: the build step is the process of turning a source archive into a
-  binary archive. Dependencies must be available in order to build and
-  create a binary archive of the distribution (including any documentation
-  that is installed on target systems).
-
-* Deployment: the deployment phase consists of two subphases:
-
-  * Installation: the installation phase involves getting the distribution
-    and all of its runtime dependencies onto the target system. In this
-    phase, the distribution may already be on the system (when upgrading or
-    reinstalling) or else it may be a completely new installation.
-
-  * Usage: the usage phase, also referred to as "runtime", is normal usage
-    of the distribution after it has been installed on the target system.
-
-The metadata format described in this PEP is designed to enable the
-following:
-
-* It should be practical to have separate development systems, build systems
-  and deployment systems.
-* It should be practical to install dependencies needed specifically to
-  build source archives only on development systems.
-* It should be practical to install dependencies needed specifically to
-  build the software only on development and build systems, as well as
-  optionally on deployment systems if installation from source archives
-  is needed.
-* It should be practical to install dependencies needed to run the
-  distribution only on development and deployment systems.
-* It should be practical to install the dependencies needed to run a
-  distribution's test suite only on development systems, as well as
-  optionally on deployment systems.
-* It should be practical for repackagers to separate out the build
-  dependencies needed to build the application itself from those required
-  to build its documentation (as the documentation often doesn't need to
-  be rebuilt when porting an application to a different platform).
-
-.. note::
-
-    This "most complex supported scenario" is almost *exactly* what has to
-    happen to get an upstream Python package into a Linux distribution, and
-    is why the current crop of automatic Python metadata -> Linux distro
-    metadata converters have some serious issues, at least from the point of
-    view of complying with distro packaging policies: the information
-    they need to comply with those policies isn't available from the
-    upstream projects, and all current formats for publishing it are
-    distro specific. This means either upstreams have to maintain metadata
-    for multiple distributions (which rarely happens) or else repackagers
-    have to do a lot of work manually in order to separate out these
-    dependencies in a  way that complies with those policies.
-
-    One thing this PEP aims to do is define a metadata format that at least
-    has the *potential* to provide the info repackagers need, thus allowing
-    upstream Python projects and Linux distro repackagers to collaborate more
-    effectively (and, ideally, make it possible to reliably automate
-    the process of converting upstream Python distributions into policy
-    compliant distro packages).
-
-    Some items in this section (and the contents of this note) will likely
-    end up moving down to the "Rationale for changes from PEP 345" section.
+"Entry points" are a scheme for identifying Python callables or other
+objects as strings consisting of a Python module name and a module
+attribute name, separated by a colon. For example: ``"test.regrtest:main"``.
+
+
+Integration and deployment of distributions
+-------------------------------------------
+
+The primary purpose of the distribution metadata is to support integration
+and deployment of distributions as part of larger applications and systems.
+
+Integration and deployment can in turn be broken down into further substeps.
+
+* Build: the build step is the process of turning a VCS checkout, source
+  archive or sdist into a binary archive. Dependencies must be available
+  in order to build and create a binary archive of the distribution
+  (including any documentation that is installed on target systems).
+
+* Installation: the installation step involves getting the distribution
+  and all of its runtime dependencies onto the target system. In this
+  step, the distribution may already be on the system (when upgrading or
+  reinstalling) or else it may be a completely new installation.
+
+* Runtime: this is normal usage of a distribution after it has been
+  installed on the target system.
+
+These three steps may all occur directly on the target system. Alternatively
+the build step may be separated out by using binary archives provided by the
+publisher of the distribution, or by creating the binary archives on a
+separate system prior to deployment.
+
+The published metadata for distributions SHOULD allow integrators, with the
+aid of build and integration tools, to:
+
+* obtain the original source code that was used to create a distribution
+* identify and retrieve the dependencies (if any) required to use a
+  distribution
+* identify and retrieve the dependencies (if any) required to build a distribution
+  from source
+* if supported by the distribution, run the distribution's automatic test
+  suite on an installed instance of the distribution
+* identify and retrieve the dependencies (if any) required to run a
+  distribution's test suite
+* find resources on using and contributing to the project
+* access sufficiently rich metadata to support contacting distribution
+  publishers through appropriate channels, as well as finding distributions
+  that are relevant to particular problems
+
+The current iteration of the metadata relies on the ``distutils`` commands
+system to support other necessary integration and deployment activities:
+
+* ``python setup.py bdist_wheel``: create a binary archive from a source
+  archive or checkout
+* ``python setup.py test``: run the distribution's test suite on a built
+  (but not yet installed) distribution
+
+Future iterations of the metadata will aim to replace these ``distutils``/
+``setuptools`` dependent commands with build system independent entry
+points.
+
+
+Development and publication of distributions
+--------------------------------------------
+
+The secondary purpose of the distribution metadata is to support effective
+collaboration amongst software contributors and publishers during the
+development phase.
+
+The published metadata for distributions SHOULD allow contributors
+and publishers, with the aid of build and publication tools, to:
+
+* perform all the same activities needed to effectively integrate and
+  deploy the distribution
+* identify and retrieve the additional dependencies needed to develop and
+  publish the distribution
+* specify the dependencies (if any) required to use the distribution
+* specify the dependencies (if any) required to build the distribution
+  from source
+* specify the dependencies (if any) required to run the distribution's
+  test suite
+* specify the additional dependencies (if any) required to develop and
+  publish the distribution
+
+The current iteration of the metadata relies on the ``distutils`` commands
+system to support other necessary development and publication activities:
+
+* ``python setup.py dist_info``: generate the ``pymeta.json`` file for a
+  distribution
+* ``python setup.py sdist``: create an sdist from a source archive
+  or VCS checkout
+* ``python setup.py test``: run the distribution's test suite on a built
+  (but not yet installed) distribution
+
+Future iterations of the metadata and associated PEPs will aim to replace
+these ``distutils``/``setuptools`` dependent commands with build system
+independent entry points.
 
 
 Metadata format
@@ -247,14 +348,22 @@
 Automated tools MAY automatically derive valid values from other
 information sources (such as a version control system).
 
+Automated tools, especially public index servers, MAY impose additional
+length restrictions on metadata beyond those enumerated in this PEP. Such
+limits SHOULD be imposed where necessary to protect the integrity of a
+service, based on the available resources and the service provider's
+judgment of reasonable metadata capacity requirements.
+
 
 Metadata files
 --------------
 
 The information defined in this PEP is serialised to ``pymeta.json``
-files for some use cases. As indicated by the extension, these
-are JSON-encoded files. Each file consists of a single serialised mapping,
-with fields as described in this PEP.
+files for some use cases. These are files containing UTF-8 encoded JSON
+metadata.
+
+Each metadata file consists of a single serialised mapping, with fields as
+described in this PEP.
 
 There are three standard locations for these metadata files:
 
@@ -270,21 +379,16 @@
    These locations are to be confirmed, since they depend on the definition
    of sdist 2.0 and the revised installation database standard. There will
    also be a wheel 1.1 format update after this PEP is approved that
-   mandates 2.0+ metadata.
+   mandates provision of 2.0+ metadata.
 
 Other tools involved in Python distribution may also use this format.
 
-It is expected that these metadata files will be generated by build tools
-based on other input formats (such as ``setup.py``) rather than being
-edited by hand.
-
-.. note::
-
-    It may be appropriate to add a "./setup.py dist_info" command to
-    setuptools to allow just the sdist metadata files to be generated
-    without having to build the full sdist archive. This would be
-    similar to the existing "./setup.py egg_info" command in setuptools,
-    which would continue to emit the legacy metadata format.
+As JSON files are generally awkward to edit by hand, it is RECOMMENDED
+that these metadata files be generated by build tools based on other
+input formats (such as ``setup.py``) rather than being used directly as
+a data input format. Generating the metadata as part of the publication
+process also helps to deal with version specific fields (including the
+source URL and the version field itself).
 
 For backwards compatibility with older installation tools, metadata 2.0
 files MAY be distributed alongside legacy metadata.
@@ -292,6 +396,10 @@
 Index servers MAY allow distributions to be uploaded and installation tools
 MAY allow distributions to be installed with only legacy metadata.
 
+Automated tools MAY attempt to automatically translate legacy metadata to
+the format described in this PEP. Advice for doing so effectively is given
+in Appendix A.
+
 
 Essential dependency resolution metadata
 ----------------------------------------
@@ -306,11 +414,13 @@
 * ``metadata_version``
 * ``name``
 * ``version``
-* ``build_label``
-* ``version_url``
+* ``source_label``
+* ``source_url``
 * ``extras``
-* ``requires``
-* ``may_require``
+* ``meta_requires``
+* ``meta_may_require``
+* ``run_requires``
+* ``run_may_require``
 * ``build_requires``
 * ``build_may_require``
 * ``dev_requires``
@@ -320,23 +430,30 @@
 * ``supports_environments``
 
 When serialised to a file, the name used for this metadata set SHOULD
-be ``pymeta-minimal.json``.
-
-Abbreviated metadata
---------------------
-
-Some metadata fields have the potential to contain a lot of information
-that will rarely be referenced, greatly increasing storage requirements
-without providing significant benefits.
-
-The abbreviated metadata for a distribution consists of all fields
-*except* the following:
-
-* ``description``
-* ``contributors``
-
-When serialised to a file, the name used for this metadata set SHOULD
-be ``pymeta-short.json``.
+be ``pymeta-dependencies.json``.
+
+
+Included documents
+------------------
+
+Rather than being incorporated directly into the structured metadata, some
+supporting documents are included alongside the metadata file in the
+``dist-info`` metadata directory.
+
+To accommodate the variety of existing naming conventions for these files,
+they are explicitly identified in the ``document_names`` field, rather
+than expecting index servers and other automated tools to identify them
+automatically.
+
+
+Metadata validation
+-------------------
+
+A `jsonschema <https://pypi.python.org/pypi/jsonschema>`__ description of
+the defined metadata schema will be provided prior to PEP acceptance.
+
+Except where otherwise noted, all URL fields in the metadata MUST comply
+with RFC 3986.
 
 
 Core metadata
@@ -391,7 +508,7 @@
 * hyphens (``-``)
 * periods (``.``)
 
-Distributions named MUST start and end with an ASCII letter or digit.
+Distribution names MUST start and end with an ASCII letter or digit.
 
 Automated tools MUST reject non-compliant names.
 
@@ -399,14 +516,14 @@
 consider hyphens and underscores to be equivalent.
 
 Index servers MAY consider "confusable" characters (as defined by the
-Unicode Consortium in `TR39: Unicode Security Mechanisms <TR39>`__) to be
+Unicode Consortium in `TR39: Unicode Security Mechanisms <TR39>`_) to be
 equivalent.
 
 Index servers that permit arbitrary distribution name registrations from
 untrusted sources SHOULD consider confusable characters to be equivalent
 when registering new distributions (and hence reject them as duplicates).
 
-Installation tools MUST NOT silently accept a confusable alternate
+Integration tools MUST NOT silently accept a confusable alternate
 spelling as matching a requested distribution name.
 
 At time of writing, the characters in the ASCII subset designated as
@@ -439,26 +556,9 @@
     the automated tools in the chain.
 
     PyPI has recently been updated to reject non-compliant names for newly
-    registered projects, but existing non-compliant names are still
-    tolerated when using legacy metadata formats. Affected distributions
-    will need to change their names (typically be replacing spaces with
-    hyphens) before they can migrate to the new metadata formats.
-
-    Donald Stufft ran an analysis, and the new restrictions impact less
-    than 230 projects out of the ~31k already on PyPI. This isn't that
-    surprising given the fact that many existing tools could already
-    exhibit odd behaviour when attempting to deal with non-compliant
-    names, implicitly discouraging the use of more exotic names.
-
-    Of those projects, ~200 have the only non-compliant character as an
-    internal space (e.g. "Twisted Web"). These will be automatically
-    migrated by replacing the spaces with hyphens (e.g. "Twisted-Web"),
-    which is what you have to actually type to install these distributions
-    with ``setuptools`` (which powers both ``easy_install`` and ``pip``).
-
-    The remaining ~30 will be investigated manually and decided upon on a
-    case by case basis how to migrate them to the new naming rules (in
-    consultation with the maintainers of those projects where possible).
+    registered projects, and the ~230 existing non-compliant names have
+    been updated to become compliant (mostly by replacing spaces with
+    hyphens).
 
 
 Version
@@ -469,6 +569,10 @@
 variety of flexible version specification mechanisms (see PEP 440 for
 details).
 
+Version identifiers MUST comply with the format defined in PEP 440.
+
+Version identifiers MUST be unique within each project.
+
 Example::
 
     "version": "1.0a2"
@@ -489,9 +593,13 @@
 ------------
 
 A constrained identifying text string, as defined in PEP 440. Source labels
-cannot be used in ordered version comparisons, but may be used to select
-an exact version (see PEP 440 for details).
-
+cannot be used in version specifiers - they are included for information
+purposes only.
+
+Source labels MUST meet the character restrictions defined in PEP 440.
+
+Source labels MUST be unique within each project and MUST NOT match any
+defined version for the project.
 
 Examples::
 
@@ -508,19 +616,23 @@
 ----------
 
 A string containing a full URL where the source for this specific version of
-the distribution can be downloaded.  (This means that the URL can't be
-something like ``"https://github.com/pypa/pip/archive/master.zip"``, but
-instead must be ``"https://github.com/pypa/pip/archive/1.3.1.zip"``.)
-
-Some appropriate targets for a source URL are a source tarball, an sdist
-archive or a direct reference to a tag or specific commit in an online
-version control system.
-
-All source URL references SHOULD either specify a secure transport
-mechanism (such as ``https``) or else include an expected hash value in the
-URL for verification purposes. If an insecure transport is specified without
-any hash information (or with hash information that the tool doesn't
-understand), automated tools SHOULD at least emit a warning and MAY
+the distribution can be downloaded.
+
+Source URLs MUST be unique within each project. This means that the URL
+can't be something like ``"https://github.com/pypa/pip/archive/master.zip"``,
+but instead must be ``"https://github.com/pypa/pip/archive/1.3.1.zip"``.
+
+The source URL MUST reference either a source archive or a tag or specific
+commit in an online version control system that permits creation of a
+suitable VCS checkout. It is intended primarily for integrators that
+wish to recreate the distribution from the original source form.
+
+All source URL references SHOULD specify a secure transport
+mechanism (such as ``https``), include an expected hash value in the
+URL for verification purposes, or both. If an insecure transport is specified
+without any hash information, with hash information that the tool doesn't
+understand, or with a selected hash algortihm that the tool considers too
+weak to trust, automated tools SHOULD at least emit a warning and MAY
 refuse to rely on the URL.
 
 It is RECOMMENDED that only hashes which are unconditionally provided by
@@ -569,6 +681,10 @@
     change for each version, while the project URLs are expected to
     be fairly stable.
 
+    Note that many translations of legacy metadata won't be able to convert
+    Download-URL to source_url, as many distributions haven't respected the
+    requirement to link to a specific version.
+
 
 Additional descriptive metadata
 ===============================
@@ -580,74 +696,47 @@
 a distribution does not provide them, including failing cleanly when an
 operation depending on one of these fields is requested.
 
+
 Summary
 -------
 
 A one-line summary of what the distribution does.
 
 Publication tools SHOULD emit a warning if this field is not provided. Index
-servers MAY require that this field be present before allowing a
+servers MAY require that this field be provided before allowing a
 distribution to be uploaded.
 
+This field SHOULD contain fewer than 512 characters and MUST contain fewer
+than 2048.
+
+A more complete description SHOULD be included as a separate file in the
+sdist for the distribution. See `Document names`_ for details.
+
 Example::
 
     "summary": "A module that is more fiendish than soft cushions."
 
-.. note::
-
-   This used to be mandatory, and it's still highly recommended, but really,
-   nothing should break even when it's missing.
-
-
-Description
------------
-
-The distribution metadata should include a longer description of the
-distribution that may run to several paragraphs. Software that deals
-with metadata should not assume any maximum size for the description.
-
-The distribution description can be written using reStructuredText
-markup [1]_.  For programs that work with the metadata, supporting
-markup is optional; programs may also display the contents of the
-field as plain text without any special formatting.  This means that
-authors should be conservative in the markup they use.
+
+License
+-------
+
+A short string summarising the license used for this distribution.
+
+Note that distributions that provide this field should still specify any
+applicable license Trove classifiers in the `Classifiers`_ field. Even
+when an appropriate Trove classifier is available, the license summary can
+be a good way to specify a particular version of that license, or to
+indicate any variations or exception to the license.
+
+This field SHOULD contain fewer than 512 characters and MUST contain fewer
+than 2048.
+
+The full license text SHOULD be included as a separate file in the source
+archive for the distribution. See `Document names`_ for details.
 
 Example::
 
-    "description": "The ComfyChair module replaces SoftCushions.\\n\\nUse until lunchtime, but pause for a cup of coffee at eleven."
-
-.. note::
-
-   The difficulty of editing this field in a raw JSON file is one of the
-   main reasons this metadata interchange format is NOT recommended for
-   use as an input format for build tools.
-
-
-Description Format
-------------------
-
-A field indicating the intended format of the text in the description field.
-This allows index servers to render the description field correctly and
-provide feedback on rendering errors, rather than having to guess the
-intended format.
-
-If this field is omitted, or contains an unrecognised value, the default
-rendering format MUST be plain text.
-
-The following format names SHOULD be used for the specified markup formats:
-
-* ``txt``: Plain text (default handling if field is omitted)
-* ``rst``: reStructured Text
-* ``md``: Markdown (exact syntax variant will be implementation dependent)
-* ``adoc``: AsciiDoc
-* ``html``: HTML
-
-Automated tools MAY render one or more of the listed formats as plain
-text and MAY accept other markup formats beyond those listed.
-
-Example::
-
-    "description_format": "rst"
+    "license": "GPL version 3, excluding DRM provisions"
 
 
 Keywords
@@ -661,40 +750,6 @@
     "keywords": ["comfy", "chair", "cushions", "too silly", "monty python"]
 
 
-License
--------
-
-A string indicating the license covering the distribution where the license
-is not a simple selection from the "License" Trove classifiers. See
-Classifiers" below.  This field may also be used to specify a
-particular version of a license which is named via the ``Classifier``
-field, or to indicate a variation or exception to such a license.
-
-Example::
-
-    "license": "GPL version 3, excluding DRM provisions"
-
-
-License URL
------------
-
-A specific URL referencing the full licence text for this version of the
-distribution.
-
-Example::
-
-    "license_url": "https://github.com/pypa/pip/blob/1.3.1/LICENSE.txt"
-
-.. note::
-
-   Like Version URL, this is handled separately from the project URLs
-   as it is important that it remain accurate for this *specific*
-   version of the distribution, even if the project later switches to a
-   different license.
-
-   The project URLs field is intended for more stable references.
-
-
 Classifiers
 -----------
 
@@ -704,11 +759,63 @@
 Example::
 
     "classifiers": [
-        "Development Status :: 4 - Beta",
-        "Environment :: Console (Text Based)"
+      "Development Status :: 4 - Beta",
+      "Environment :: Console (Text Based)",
+      "License :: OSI Approved :: GNU General Public License v3 (GPLv3)"
     ]
 
 
+Document names
+--------------
+
+Filenames for supporting documents included in the distribution's
+``dist-info`` metadata directory.
+
+The following supporting documents can be named:
+
+* ``description``: a file containing a long description of the distribution
+* ``license``: a file with the full text of the distribution's license
+* ``changelog``: a file describing changes made to the distribution
+
+If this field is provided at all, it MUST name at least one included
+document.
+
+Supporting documents MUST be included directly in the ``dist-info``
+directory. Directory separators are NOT permitted in document names.
+
+The markup format (if any) for the file is indicated by the file extension.
+This allows index servers and other automated tools to render included
+text documents correctly and provide feedback on rendering errors, rather
+than having to guess the intended format.
+
+If the filename has no extension, or the extension is not recognised, the
+default rendering format MUST be plain text.
+
+The following markup renderers SHOULD be used for the specified file
+extensions:
+
+* Plain text: ``.txt``, no extension, unknown extension
+* reStructured Text: ``.rst``
+* Markdown: ``.md``
+* AsciiDoc: ``.adoc``, ``.asc``, ``.asciidoc``
+* HTML: ``.html``, ``.htm``
+
+Automated tools MAY render one or more of the specified formats as plain
+text and MAY render other markup formats beyond those listed.
+
+Automated tools SHOULD NOT make any assumptions regarding the maximum length
+of supporting document content, except as necessary to protect the
+integrity of a service.
+
+Example::
+
+    "document_names": {
+        "description": "README.rst",
+        "license": "LICENSE.rst",
+        "changelog": "NEWS"
+    }
+
+
 Contributor metadata
 ====================
 
@@ -726,6 +833,12 @@
 
 If no specific role is stated, the default is ``contributor``.
 
+Email addresses must be in the form ``local-part at domain`` where the
+local-part may be up to 64 characters long and the entire email address
+contains no more than 254 characters. The formal specification of the
+format is in RFC 5322 (sections 3.2.3 and 3.4.1) and RFC 5321, with a more
+readable form given in the informational RFC 3696 and the associated errata.
+
 The defined contributor roles are as follows:
 
 * ``author``: the original creator of a distribution
@@ -789,12 +902,12 @@
 Example::
 
     "contributors": [
-        {"name": "John C."},
-        {"name": "Erik I."},
-        {"name": "Terry G."},
-        {"name": "Mike P."},
-        {"name": "Graeme C."},
-        {"name": "Terry J."}
+      {"name": "John C."},
+      {"name": "Erik I."},
+      {"name": "Terry G."},
+      {"name": "Mike P."},
+      {"name": "Graeme C."},
+      {"name": "Terry J."}
     ]
 
 
@@ -815,20 +928,42 @@
 Example::
 
     "project_urls": {
-        "Documentation": "https://distlib.readthedocs.org"
-        "Home": "https://bitbucket.org/pypa/distlib"
-        "Repository": "https://bitbucket.org/pypa/distlib/src"
-        "Tracker": "https://bitbucket.org/pypa/distlib/issues"
+      "Documentation": "https://distlib.readthedocs.org"
+      "Home": "https://bitbucket.org/pypa/distlib"
+      "Repository": "https://bitbucket.org/pypa/distlib/src"
+      "Tracker": "https://bitbucket.org/pypa/distlib/issues"
     }
 
 
-Dependency metadata
-===================
+Semantic dependencies
+=====================
 
 Dependency metadata allows distributions to make use of functionality
 provided by other distributions, without needing to bundle copies of those
 distributions.
 
+Semantic dependencies allow publishers to indicate not only which other
+distributions are needed, but also *why* they're needed. This additional
+information allows integrators to install just the dependencies they need
+for specific activities, making it easier to minimise installation
+footprints in constrained environments (regardless of the reasons for
+those constraints).
+
+Distributions may declare five differents kinds of dependency:
+
+* "Meta" dependencies: subdistributions that are grouped together into a
+  single larger metadistribution for ease of reference and installation.
+* Runtime dependencies: other distributions that are needed to actually use
+  this distribution (but are not considered subdistributions).
+* Test dependencies: other distributions that are needed to run the
+  automated test suite for this distribution (but are not needed just to
+  use it).
+* Build dependencies: other distributions that are needed to build this
+  distribution.
+* Development dependencies: other distributions that are needed when
+  working on this distribution (but do not fit into one of the other
+  dependency categories).
+
 Dependency management is heavily dependent on the version identification
 and specification scheme defined in PEP 440.
 
@@ -838,10 +973,11 @@
     model in metadata 1.2 (which was in turn derived from the setuptools
     dependency parameters). The translation is that ``dev_requires`` and
     ``build_requires`` both map to ``Setup-Requires-Dist``
-    in 1.2, while ``requires`` and ``distributes`` map to ``Requires-Dist``.
+    in 1.2 (aka ``install_requires`` in setuptools), while ``run_requires``
+    and ``meta_requires`` map to ``Requires-Dist``.
     To go the other  way, ``Setup-Requires-Dist`` maps to ``build_requires``
-    and ``Requires-Dist`` maps to ``distributes`` (for exact comparisons)
-    and ``requires`` (for all other version specifiers).
+    and ``Requires-Dist`` maps to ``meta_requires`` (for exact comparisons)
+    and ``run_requires`` (for all other version specifiers).
 
 All of these fields are optional. Automated tools MUST operate correctly if
 a distribution does not provide them, by assuming that a missing field
@@ -918,26 +1054,34 @@
 and development activities identified above, and govern which dependencies
 should be installed for the specified activities:
 
-* Deployment dependencies:
-
-    * ``distributes``
-    * ``requires``
-    * ``may_require``
-    * Request the ``test`` extra to also install
-
+* Implied runtime dependencies:
+
+    * ``meta_requires``
+    * ``meta_may_require``
+    * ``run_requires``
+    * ``run_may_require``
+
+* Implied build dependencies:
+
+    * ``build_requires``
+    * ``build_may_require``
+    * If running the distribution's test suite as part of the build process,
+      request the ``:meta:``, ``:run:`` and ``:test:`` extras to also
+      install:
+
+      * ``meta_requires``
+      * ``meta_may_require``
+      * ``run_requires``
+      * ``run_may_require``
       * ``test_requires``
       * ``test_may_require``
 
-* Build dependencies:
-
-    * ``build_requires``
-    * ``build_may_require``
-
-* Development dependencies:
-
-    * ``distributes``
-    * ``requires``
-    * ``may_require``
+* Implied development and publication dependencies:
+
+    * ``meta_requires``
+    * ``meta_may_require``
+    * ``run_requires``
+    * ``run_may_require``
     * ``build_requires``
     * ``build_may_require``
     * ``test_requires``
@@ -945,139 +1089,146 @@
     * ``dev_requires``
     * ``dev_may_require``
 
-
-To ease compatibility with existing two phase setup/deployment toolchains,
-installation tools MAY treat ``dev_requires`` and ``dev_may_require`` as
-additions to ``build_requires`` and ``build_may_require`` rather than
-as separate fields.
-
-Installation tools SHOULD allow users to request at least the following
-operations for a named distribution:
-
-* Install the distribution and any deployment dependencies.
-* Install just the build dependencies without installing the distribution
-* Install just the development dependencies without installing
-  the distribution
-* Install just the development dependencies without installing
-  the distribution or any dependencies listed in ``distributes``
-
-The notation described in `Extras (optional dependencies)`_ SHOULD be used to
-request additional optional dependencies when installing deployment
-or build dependencies.
+The notation described in `Extras (optional dependencies)`_ SHOULD be used
+to determine exactly what gets installed for various operations.
 
 Installation tools SHOULD report an error if dependencies cannot be found,
 MUST at least emit a warning, and MAY allow the user to force the
 installation to proceed regardless.
 
-.. note::
-
-    As an example of mapping this to Linux distro packages, assume an
-    example project without any extras defined is split into 2 RPMs
-    in a SPEC file: example and example-devel
-
-    The ``distributes``, ``requires`` and applicable ``may_require``
-    dependencies would be mapped to the Requires dependencies for the
-    "example" RPM (a mapping from environment markers to SPEC file
-    conditions would also allow those to be handled correctly)
-
-    The ``build_requires`` and ``build_may_require`` dependencies would be
-    mapped to the BuildRequires dependencies for the "example" RPM.
-
-    All defined dependencies relevant to Linux, including those in
-    ``dev_requires`` and ``test_requires``, would become Requires
-    dependencies for the "example-devel" RPM.
-
-    If a project defines any extras, those would be mapped to additional
-    virtual RPMs with appropriate BuildRequires and Requires entries based
-    on the details of the dependency specifications.
-
-    A documentation toolchain dependency like Sphinx would either go in
-    ``build_requires`` (for example, if man pages were included in the
-    built distribution) or in ``dev_requires`` (for example, if the
-    documentation is published solely through ReadTheDocs or the
-    project website). This would be enough to allow an automated converter
-    to map it to an appropriate dependency in the spec file.
-
-
-Distributes
------------
-
-A list of subdistributions that can easily be installed and used together
-by depending on this metadistribution.
-
-Automated tools MUST allow strict version matching and source reference
-clauses in this field and MUST NOT allow more permissive version specifiers.
-
-Example::
-
-    "distributes": ["ComfyUpholstery (== 1.0a2)",
-                    "ComfySeatCushion (== 1.0a2)"]
-
-
-Requires
---------
-
-A list of other distributions needed when this distribution is deployed.
-
-Automated tools MAY disallow strict version matching clauses and source
-references in this field and SHOULD at least emit a warning for such clauses.
-
-Example::
-
-    "requires": ["SciPy", "PasteDeploy", "zope.interface (>3.5.0)"]
+See Appendix B for an overview of mapping these dependencies to an RPM
+spec file.
 
 
 Extras
 ------
 
 A list of optional sets of dependencies that may be used to define
-conditional dependencies in ``"may_require"`` and similar fields. See
-`Extras (optional dependencies)`_ for details.
-
-The extra name``"test"`` is reserved for requesting the dependencies
-specified in ``test_requires`` and ``test_may_require`` and is NOT
-permitted in this field.
+conditional dependencies in ``"may_distribute"``, ``"run_may_require"`` and
+similar fields. See `Extras (optional dependencies)`_ for details.
+
+The names of extras MUST abide by the same restrictions as those for
+distribution names.
 
 Example::
 
     "extras": ["warmup"]
 
 
-May require
------------
-
-A list of other distributions that may be needed when this distribution
-is deployed, based on the extras requested and the target deployment
+Meta requires
+-------------
+
+An abbreviation of "metadistribution requires". This is a list of
+subdistributions that can easily be installed and used together by
+depending on this metadistribution.
+
+In this field, automated tools:
+
+* MUST allow strict version matching
+* MAY allow direct reference clauses
+* MUST NOT allow more permissive version specifiers.
+
+Public index servers SHOULD NOT allow the use of direct references in
+uploaded distributions. Direct references are intended as a tool for
+software integrators rather than publishers.
+
+Distributions that rely on direct references to platform specific binary
+archives SHOULD place appropriate entries in their ``supports_environments``
+field.
+
+Example::
+
+    "meta_requires": ["ComfyUpholstery (== 1.0a2)",
+                      "ComfySeatCushion (== 1.0a2)"]
+
+
+Meta may require
+----------------
+
+An abbreviation of "metadistribution may require". This is a list of
+subdistributions that can easily be installed and used together by
+depending on this metadistribution, but are not required in all
+circumstances.
+
+Any extras referenced from this field MUST be named in the `Extras`_ field.
+
+In this field, automated tools:
+
+* MUST allow strict version matching
+* MAY allow direct reference clauses
+* MUST NOT allow more permissive version specifiers.
+
+Public index servers SHOULD NOT allow the use of direct references in
+uploaded distributions. Direct references are intended as a tool for
+software integrators rather than publishers.
+
+Distributions that rely on direct references to platform specific binary
+archives SHOULD place appropriate entries in their ``supports_environments``
+field.
+
+Example::
+
+    "meta_may_require": [
+      {
+        "dependencies": ["CupOfTeaAtEleven (== 1.0a2)"],
+        "environment": "'linux' in sys.platform"
+      }
+    ]
+
+
+Run requires
+------------
+
+A list of other distributions needed to actually run this distribution.
+
+Automated tools MUST NOT allow strict version matching clauses or direct
+references in this field - if permitted at all, such clauses should appear
+in ``meta_requires`` instead.
+
+Example::
+
+    "run_requires": ["SciPy", "PasteDeploy", "zope.interface (>3.5.0)"]
+
+
+Run may require
+---------------
+
+A list of other distributions that may be needed to actually run this
+distribution, based on the extras requested and the target deployment
 environment.
 
 Any extras referenced from this field MUST be named in the `Extras`_ field.
 
-Automated tools MAY disallow strict version matching clauses and source
-references in this field and SHOULD at least emit a warning for such clauses.
+Automated tools MUST NOT allow strict version matching clauses or direct
+references in this field - if permitted at all, such clauses should appear
+in ``meta_may_require`` instead.
 
 Example::
 
-        "may_require": [
-          {
-            "dependencies": ["pywin32 (>1.0)"],
-            "environment": "sys.platform == 'win32'"
-          },
-          {
-            "dependencies": ["SoftCushions"],
-            "extra": "warmup"
-          }
-        ]
+    "run_may_require": [
+      {
+        "dependencies": ["pywin32 (>1.0)"],
+        "environment": "sys.platform == 'win32'"
+      },
+      {
+        "dependencies": ["SoftCushions"],
+        "extra": "warmup"
+      }
+    ]
+
 
 Test requires
 -------------
 
 A list of other distributions needed in order to run the automated tests
-for this distribution, either during development or when running the
-``test_installed_dist`` metabuild when deployed.
-
-Automated tools MAY disallow strict version matching clauses and source
+for this distribution..
+
+Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
+Public index servers SHOULD disallow strict version matching clauses and
+direct references in this field.
+
 Example::
 
     "test_requires": ["unittest2"]
@@ -1087,41 +1238,45 @@
 ----------------
 
 A list of other distributions that may be needed in order to run the
-automated tests for this distribution, either during development or when
-running the ``test_installed_dist`` metabuild when deployed, based on the
-extras requested and the target deployment environment.
+automated tests for this distribution.
 
 Any extras referenced from this field MUST be named in the `Extras`_ field.
 
-Automated tools MAY disallow strict version matching clauses and source
+Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
+Public index servers SHOULD disallow strict version matching clauses and
+direct references in this field.
+
 Example::
 
-        "test_may_require": [
-          {
-            "dependencies": ["pywin32 (>1.0)"],
-            "environment": "sys.platform == 'win32'"
-          },
-          {
-            "dependencies": ["CompressPadding"],
-            "extra": "warmup"
-          }
-        ]
+    "test_may_require": [
+      {
+        "dependencies": ["pywin32 (>1.0)"],
+        "environment": "sys.platform == 'win32'"
+      },
+      {
+        "dependencies": ["CompressPadding"],
+        "extra": "warmup"
+      }
+    ]
 
 
 Build requires
 --------------
 
 A list of other distributions needed when this distribution is being built
-(creating a binary archive from a source archive).
+(creating a binary archive from an sdist, source archive or VCS checkout).
 
 Note that while these are build dependencies for the distribution being
 built, the installation is a *deployment* scenario for the dependencies.
 
-Automated tools MAY disallow strict version matching clauses and source
+Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
+Public index servers SHOULD disallow strict version matching clauses and
+direct references in this field.
+
 Example::
 
     "build_requires": ["setuptools (>= 0.7)"]
@@ -1131,8 +1286,8 @@
 -----------------
 
 A list of other distributions that may be needed when this distribution
-is built (creating a binary archive from a source archive), based on the
-features requested and the build environment.
+is built (creating a binary archive from an sdist, source archive or
+VCS checkout), based on the features requested and the build environment.
 
 Note that while these are build dependencies for the distribution being
 built, the installation is a *deployment* scenario for the dependencies.
@@ -1142,21 +1297,24 @@
 Automated tools MAY assume that all extras are implicitly requested when
 installing build dependencies.
 
-Automated tools MAY disallow strict version matching clauses and source
+Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
+Public index servers SHOULD disallow strict version matching clauses and
+direct references in this field.
+
 Example::
 
-        "build_may_require": [
-          {
-            "dependencies": ["pywin32 (>1.0)"],
-            "environment": "sys.platform == 'win32'"
-          },
-          {
-            "dependencies": ["cython"],
-            "extra": "c-accelerators"
-          }
-        ]
+    "build_may_require": [
+      {
+        "dependencies": ["pywin32 (>1.0)"],
+        "environment": "sys.platform == 'win32'"
+      },
+      {
+        "dependencies": ["cython"],
+        "extra": "c-accelerators"
+      }
+    ]
 
 
 Dev requires
@@ -1168,17 +1326,16 @@
 
 Additional dependencies that may be listed in this field include:
 
-* tools needed to create a source archive
+* tools needed to create an sdist from a source archive or VCS checkout
 * tools needed to generate project documentation that is published online
   rather than distributed along with the rest of the software
-* additional test dependencies for tests which are not executed when the
-  test is invoked through the ``test_installed_dist`` metabuild hook (for
-  example, tests that require a local database server and web server and
-  may not work when fully installed on a production system)
-
-Automated tools MAY disallow strict version matching clauses and source
+
+Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
+Public index servers SHOULD disallow strict version matching clauses and
+direct references in this field.
+
 Example::
 
     "dev_requires": ["hgtools", "sphinx (>= 1.0)"]
@@ -1199,17 +1356,20 @@
 Automated tools MAY assume that all extras are implicitly requested when
 installing development dependencies.
 
-Automated tools MAY disallow strict version matching clauses and source
+Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
+Public index servers SHOULD disallow strict version matching clauses and
+direct references in this field.
+
 Example::
 
-        "dev_may_require": [
-          {
-            "dependencies": ["pywin32 (>1.0)"],
-            "environment": "sys.platform == 'win32'"
-          }
-        ]
+    "dev_may_require": [
+      {
+        "dependencies": ["pywin32 (>1.0)"],
+        "environment": "sys.platform == 'win32'"
+      }
+    ]
 
 
 Provides
@@ -1231,6 +1391,19 @@
 project is able to include a ``"provides": ["distribute"]`` entry to
 satisfy any projects that require the now obsolete distribution's name.
 
+To avoid malicious hijacking of names, when interpreting metadata retrieved
+from a public index server, automated tools MUST prefer the distribution
+named in a version specifier over other distributions using that
+distribution's name in a ``"provides"`` entry. Index servers MAY drop such
+entries from the metadata they republish, but SHOULD NOT refuse to publish
+such distributions.
+
+However, to appropriately handle project forks and mergers, automated tools
+MUST accept ``"provides"`` entries that name other distributions when the
+entry is retrieved from a local installation database or when there is a
+corresponding ``"obsoleted_by"`` entry in the metadata for the named
+distribution.
+
 A distribution may also provide a "virtual" project name, which does
 not correspond to any separately distributed project:  such a name
 might be used to indicate an abstract capability which could be supplied
@@ -1313,41 +1486,27 @@
    as Python 2 only and Python 3 only distributions.
 
 
-Metabuild system
-================
-
-The ``metabuild_hooks`` field is used to define various operations that
-may be invoked on a distribution in a platform independent manner.
-
-The metabuild system currently defines three operations as part of the
-deployment of a distribution:
+Install hooks
+=============
+
+The ``install_hooks`` field is used to define operations to be
+invoked on the distribution  in the following situations:
 
 * Installing to a deployment system
 * Uninstalling from a deployment system
-* Running the distribution's test suite on a deployment system (hence the
-  ``test`` runtime extra)
-
-Distributions may define handles for each of these operations as an
-"entry point", a reference to a Python callable, with the module name
-separated from the reference within the module by a colon (``:``).
-
-Example metabuild hooks::
-
-    "metabuild_hooks": {
-        "postinstall": "myproject.build_hooks:postinstall",
-        "preuininstall": "myproject.build_hooks:preuninstall",
-        "test_installed_dist": "some_test_harness:metabuild_hook"
+
+Distributions may define handlers for each of these operations as an
+"entry point", which is a reference to a Python callable, with the module
+name separated from the reference within the module by a colon (``:``).
+
+Example install hooks::
+
+    "install_hooks": {
+      "postinstall": "ComfyChair.install_hooks:postinstall",
+      "preuininstall": "ComfyChair.install_hooks:preuninstall"
     }
 
-Build and installation tools MAY offer additional operations beyond the
-core metabuild operations. These operations SHOULD be composed from the
-defined metabuild operations where appropriate.
-
-Build and installation tools SHOULD support the legacy ``setup.py`` based
-commands for metabuild operations not yet defined as metabuild hooks.
-
-The metabuild hooks are gathered together into a single top level
-``metabuild_hooks`` field. The individual hooks are:
+The currently defined install hooks are:
 
 * ``postinstall``: run after the distribution has been installed to a
   target deployment system (or after it has been upgraded). If the hook is
@@ -1357,18 +1516,15 @@
   deployment system (or before it is upgraded). If the hook is not defined,
   it indicates no distribution specific actions are needed prior to
   uninstallation.
-* ``test_installed_dist``: test an installed distribution is working. If the
-  hook is not defined, it indicates the distribution does not support
-  execution of the test suite after deployment.
-
-The expected signatures of these hooks are as follows::
+
+The required signatures of these hooks are as follows::
 
     def postinstall(current_meta, previous_meta=None):
         """Run following installation or upgrade of the distribution
 
         *current_meta* is the distribution metadata for the version now
         installed on the current system
-        *previous_meta* is either missing or ``None`` (indicating a fresh
+        *previous_meta* is either omitted or ``None`` (indicating a fresh
         install) or else the distribution metadata for the version that
         was previously installed (indicating an upgrade or downgrade).
         """
@@ -1378,30 +1534,57 @@
 
         *current_meta* is the distribution metadata for the version now
         installed on the current system
-        *next_meta* is either missing or ``None`` (indicating complete
+        *next_meta* is either omitted or ``None`` (indicating complete
         uninstallation) or else the distribution metadata for the version
         that is about to be installed (indicating an upgrade or downgrade).
         """
 
-    def test_installed_dist(current_meta):
-        """Check an installed distribution is working correctly
-
-        Note that this check should always be non-destructive as it may be
-        invoked automatically by some tools.
-
-        Requires that the distribution's test dependencies be installed
-        (indicated by the ``test`` runtime extra).
-
-        Returns ``True`` if the check passes, ``False`` otherwise.
-        """
-
-Metabuild hooks MUST be called with at least abbreviated metadata, and MAY
-be called with full metadata.
-
-Where necessary, metabuild hooks check for the presence or absence of
-optional dependencies defined as extras using the same techniques used
-during normal operation of the distribution (for example, checking for
-import failures for optional dependencies).
+When install hooks are defined, it is assumed that they MUST be executed
+to obtain a properly working installation of the distribution, and to
+properly remove the distribution from a system.
+
+Install hooks SHOULD NOT be used to provide functionality that is
+expected to be provided by installation tools (such as rewriting of
+shebang lines and generation of executable wrappers for Windows).
+
+Installation tools MUST ensure the distribution is fully installed, and
+available through the import system and installation database when invoking
+install hooks.
+
+Installation tools MUST call install hooks with full metadata, rather than
+only the essential dependency resolution metadata.
+
+The given parameter names are considered part of the hook signature.
+Installation tools MUST call install hooks solely with keyword arguments.
+Install hook implementations MUST use the given parameter names.
+
+Installation tools SHOULD invoke install hooks automatically after
+installing a distribution from a binary archive. When installing from
+an sdist, source archive or VCS checkout using ``setup.py install``
+installation tools MUST NOT invoke the install hooks - it is assumed
+that the ``setup.py`` script will already invoke any necessary
+post-installation behaviour.
+
+Installation tools MUST NOT silently ignore install hooks, as failing
+to call these hooks may result in a misconfigured installation that fails
+unexpectedly at runtime. Installation tools MAY refuse to install
+distributions that define install hooks, or require that users
+explicitly opt in to permitting the execution of such hooks.
+
+Install hook implementations MUST NOT make any assumptions regarding the
+current working directory when they are invoked, and MUST NOT make
+persistent alterations to the working directory or any other process global
+state (other than potentially importing additional modules, or other
+expected side effects of running the distribution).
+
+Install hooks have access to the full metadata for the release being
+installed, that of the previous/next release (as appropriate), as well as
+to all the normal runtime information (such as available imports). Hook
+implementations can use this information to perform additional platform
+specific installation steps. To check for the presence or absence of
+"extras", hook implementations should use the same runtime checks that
+would be used during normal operation (such as checking for the availability
+of the relevant dependencies).
 
 
 Metadata Extensions
@@ -1413,11 +1596,11 @@
 in JSON::
 
     "extensions" : {
-        "chili" : { "type" : "Poblano", "heat" : "Mild" },
-        "languages" : [ "French", "Italian", "Hebrew" ]
+      "chili" : { "type" : "Poblano", "heat" : "Mild" },
+      "languages" : [ "French", "Italian", "Hebrew" ]
     }
 
-To avoid name conflicts, it is recommended that distribution names be used
+To avoid name conflicts, it is RECOMMENDED that distribution names be used
 to identify metadata extensions. This practice will also make it easier to
 find authoritative documentation for metadata extensions.
 
@@ -1440,7 +1623,7 @@
 
     "name": "ComfyChair",
     "extras": ["warmup", "c-accelerators"]
-    "may_require": [
+    "run_may_require": [
       {
         "dependencies": ["SoftCushions"],
         "extra": "warmup"
@@ -1457,15 +1640,32 @@
 relevant extra names inside square brackets after the distribution name when
 specifying the dependency.
 
-Extra specifications MUST support the following additional syntax:
-
-* Multiple features can be requested by separating them with a comma within
+Extra specifications MUST allow the following additional syntax:
+
+* Multiple extras can be requested by separating them with a comma within
   the brackets.
-* All explicitly defined extras may be requested with the ``*`` wildcard
-  character. Note that this does NOT request the implicitly defined
-  ``test`` extra - that must always be requested explicitly when it is
-  desired.
-* Extras may be explicitly excluded by prefixing their name with a hyphen.
+* The following special extras request processing of the corresponding
+  lists of dependencies:
+
+  * ``:meta:``: ``meta_requires`` and ``meta_may_require``
+  * ``:run:``: ``run_requires`` and ``run_may_require``
+  * ``:test:``: ``test_requires`` and ``test_may_require``
+  * ``:build:``: ``build_requires`` and ``build_may_require``
+  * ``:dev:``: ``dev_requires`` and ``dev_may_require``
+  * ``:*:``: process *all* dependency lists
+
+* The ``*`` character as an extra is a wild card that enables all of the
+  entries defined in the distribution's ``extras`` field.
+* Extras may be explicitly excluded by prefixing their name with a ``-``
+  character (this is useful in conjunction with ``*`` to exclude only
+  particular extras that are definitely not wanted, while enabling all
+  others).
+
+* The ``-`` character as an extra specification indicates that the
+  distribution itself should NOT be installed, and also disables the
+  normally implied processing of ``:meta:`` and ``:run:`` dependencies
+  (those may still be requested explicitly using the appropriate extra
+  specifications).
 
 Command line based installation tools SHOULD support this same syntax to
 allow extras to be requested explicitly.
@@ -1473,15 +1673,31 @@
 The full set of dependency requirements is then based on the top level
 dependencies, along with those of any requested extras.
 
-Example::
-
-    "requires": ["ComfyChair[warmup]"]
+Dependency examples::
+
+    "run_requires": ["ComfyChair[warmup]"]
         -> requires ``ComfyChair`` and ``SoftCushions`` at run time
 
-    "requires": ["ComfyChair[*]"]
+    "run_requires": ["ComfyChair[*]"]
         -> requires ``ComfyChair`` and ``SoftCushions`` at run time, but
-           will also pick up any new optional dependencies other than those
-           needed solely to run the tests
+           will also pick up any new extras defined in later versions
+
+Command line examples::
+
+    pip install ComfyChair
+        -> installs ComfyChair with applicable :meta: and :run: dependencies
+
+    pip install ComfyChair[*]
+        -> as above, but also installs all extra dependencies
+
+    pip install ComfyChair[-,:build:,*]
+        -> installs just the build dependencies with all extras
+
+    pip install ComfyChair[-,:build:,:run:,:meta:,:test:,*]
+        -> as above, but also installs dependencies needed to run the tests
+
+    pip install ComfyChair[-,:*:,*]
+        -> installs the full set of development dependencies
 
 
 Environment markers
@@ -1504,7 +1720,7 @@
 requires PyWin32 both at runtime and buildtime when using Windows::
 
     "name": "ComfyChair",
-    "may_require": [
+    "run_may_require": [
       {
         "dependencies": ["pywin32 (>1.0)"],
         "environment": "sys.platform == 'win32'"
@@ -1559,6 +1775,10 @@
             version += kind[0] + str(info.serial)
         return version
 
+``python_full_version`` will typically correspond to the leading segment
+of ``sys.version()``.
+
+
 Updating the metadata specification
 ===================================
 
@@ -1570,8 +1790,48 @@
 defined in a new PEP.
 
 
-Summary of differences from \PEP 345
-====================================
+Appendix A: Conversion notes for legacy metadata
+================================================
+
+TBD pending feedback from Daniel & Donald
+
+
+Appendix B: Mapping dependency declarations to an RPM SPEC file
+===============================================================
+
+
+As an example of mapping this to Linux distro packages, assume an
+example project without any extras defined is split into 2 RPMs
+in a SPEC file: ``example`` and ``example-devel``.
+
+The ``meta_requires``, ``run_requires`` and applicable
+``meta_may_require`` ``run_may_require`` dependencies would be mapped
+to the Requires dependencies for the "example" RPM (a mapping from
+environment markers relevant to Linux to SPEC file conditions would
+also allow those to be handled correctly)
+
+The ``build_requires`` and ``build_may_require`` dependencies would be
+mapped to the BuildRequires dependencies for the "example" RPM.
+
+All defined dependencies relevant to Linux, including those in
+``dev_requires``, ``test_requires``, ``dev_may_require``, and
+``test_may_require`` would become Requires dependencies for the
+"example-devel" RPM.
+
+If the project did define any extras, those would likely be mapped to
+additional virtual RPMs with appropriate BuildRequires and Requires
+entries based on the details of the dependency specifications.
+
+A documentation toolchain dependency like Sphinx would either go in
+``build_requires`` (for example, if man pages were included in the
+built distribution) or in ``dev_requires`` (for example, if the
+documentation is published solely through ReadTheDocs or the
+project website). This would be enough to allow an automated converter
+to map it to an appropriate dependency in the spec file.
+
+
+Appendix C: Summary of differences from \PEP 345
+=================================================
 
 * Metadata-Version is now 2.0, with semantics specified for handling
   version changes
@@ -1592,21 +1852,21 @@
 
 * Changed the version scheme to be based on PEP 440 rather than PEP 386
 
-* Added the build label mechanism as described in PEP 440
-
-* Support for different development, build, test and deployment dependencies
+* Added the source label mechanism as described in PEP 440
+
+* Support for different kinds of dependencies
 
 * The "Extras" optional dependency mechanism
 
 * A well-defined metadata extension mechanism
 
-* Metabuild hook system
+* Install hook system
 
 * Clarify and simplify various aspects of environment markers:
 
   * allow use of parentheses for grouping in the pseudo-grammar
   * consistently use underscores instead of periods in the variable names
-  * clarify that chained comparisons are not permitted
+  * allow ordered string comparisons and chained comparisons
 
 * More flexible system for defining contact points and contributors
 
@@ -1616,9 +1876,11 @@
 
 * Updated obsolescence mechanism
 
-* Added "License URL" field
-
-* Explicit declaration of description markup format
+* Identification of supporting documents in the ``dist-info`` directory:
+
+  * Allows markup formats to be indicated through file extensions
+  * Standardises the common practice of taking the description from README
+  * Also supports inclusion of license files and changelogs
 
 * With all due respect to Charles Schulz and Peanuts, many of the examples
   have been updated to be more `thematically appropriate`_ for Python ;)
@@ -1667,7 +1929,7 @@
 subfields.
 
 The old serialisation format also wasn't amenable to easy conversion to
-standard Python data structures for use in the new metabuild hook APIs, or
+standard Python data structures for use in the new install hook APIs, or
 in future extensions to the importer APIs to allow them to provide
 information for inclusion in the installation database.
 
@@ -1691,33 +1953,47 @@
 See PEP 440 for the rationale behind the addition of this field.
 
 
-Development, build and deployment dependencies
-----------------------------------------------
-
-The separation of the ``requires``, ``build_requires`` and ``dev_requires``
-fields allows a distribution to indicate whether a dependency is needed
-specifically to develop, build or deploy the distribution.
-
-As distribution metadata improves, this should allow much greater control
-over where particular dependencies end up being installed .
+Support for different kinds of dependencies
+-------------------------------------------
+
+The separation of the five different kinds of dependency allows a
+distribution to indicate whether a dependency is needed specifically to
+develop, build, test or use the distribution.
+
+To allow for metadistributions like PyObjC, while still actively
+discouraging overly strict dependency specifications, the separate
+``meta`` dependency fields are used to separate out those dependencies
+where exact version specifications are appropriate.
+
+The advantage of having these distinctions supported in the upstream Python
+specific metadata is that even if a project doesn't care about these
+distinction themselves, they may be more amenable to patches from
+downstream redistributors that separate the fields appropriately. Over time,
+this should allow much greater control over where and when particular
+dependencies end up being installed.
+
+The names for the dependency fields have been deliberately chosen to avoid
+conflicting with the existing terminology in setuptools and previous
+versions of the metadata standard. Specifically, the names ``requires``,
+``install_requires`` and ``setup_requires`` are not used, which will
+hopefully reduce confustion when converting legacy metadata to the new
+standard.
 
 
 Support for optional dependencies for distributions
 ---------------------------------------------------
 
 The new extras system allows distributions to declare optional
-features, and to use the ``may_require`` and ``build_may_require`` fields
-to indicate when particular dependencies are needed only to support those
-features. It is derived from the equivalent system that is already in
-widespread use as part of ``setuptools`` and allows that aspect of the
-legacy ``setuptools`` metadata to be accurately represented in the new
-metadata format.
-
-The ``test`` extra is implicitly defined for all distributions, as it
-ties in with the new metabuild hook offering a standard way to request
-execution of a distribution's test suite. Identifying test suite
-dependencies is already one of the most popular uses of the extras system
-in ``setuptools``.
+behaviour, and to use the ``*may_require`` fields to indicate when
+particular dependencies are needed only to support that behaviour. It is
+derived from the equivalent system that is already in widespread use as
+part of ``setuptools`` and allows that aspect of the legacy ``setuptools``
+metadata to be accurately represented in the new metadata format.
+
+The additions to the extras syntax relative to setuptools are defined to
+make it easier to express the various possible combinations of dependencies,
+in particular those associated with build systems (with optional support
+for running the test suite) and development systems.
 
 
 Support for metadata extensions
@@ -1734,21 +2010,38 @@
 particular extensions to be provided as optional features.
 
 
-Support for metabuild hooks
+Support for install hooks
 ---------------------------
 
-The new metabuild system is designed to allow the wheel format to fully
-replace direct installation on deployment targets, by allowing projects like
-Twisted to still execute code following installation from a wheel file.
-
-Falling back to invoking ``setup.py`` directly rather than using a
-metabuild hook will remain an option when relying on version 1.x metadata,
-and is also used as the interim solution for installation from source
-archives.
-
-The ``test_installed_dist`` metabuild hook is included in order to integrate
-with build systems that can automatically invoke test suites, and as
-a complement to the ability to explicitly specify test dependencies.
+The new install hook system is designed to allow the wheel format to fully
+replace direct installation on deployment targets, by allowing projects to
+explicitly define code that should be executed following installation from
+a wheel file.
+
+This may be something relatively simple, like the `two line
+refresh <https://twistedmatrix.com/documents/current/core/howto/plugin.html#auto3>`__
+of the Twisted plugin caches that the Twisted developers recommend for
+any project that provides Twisted plugins, to more complex platform
+dependent behaviour, potentially in conjunction with appropriate
+metadata extensions and ``supports_environments`` entries.
+
+For example, upstream declaration of external dependencies for various
+Linux distributions in a distribution neutral format may be supported by
+defining an appropriate metadata extension that is read by a postinstall
+hook and converted into an appropriate invocation of the system package
+manager. Other operations (such as registering COM DLLs on Windows,
+registering services for automatic startup on any platform, or altering
+firewall settings) may need to be undertaken with elevated privileges,
+meaning they cannot be deferred to implicit execution on first use of the
+distribution.
+
+The install hook and metadata extension systems allow support for such
+activities to be pursued independently by the individual platform
+communities, while still interoperating with the cross-platform Python
+tools.
+
+Falling back to invoking ``setup.py install`` directly remains as the
+interim solution for installation from source archives.
 
 
 Changes to environment markers
@@ -1805,8 +2098,9 @@
 has been used to replace several older fields with poorly defined semantics.
 
 For the moment, the old ``Requires-External`` field has been removed
-entirely. Possible replacements may be explored through the metadata
-extension mechanism.
+entirely. The combination of explicit support for post install hooks and the
+metadata extension mechanism will hopefully prove to be a more useful
+replacement.
 
 
 Updated obsolescence mechanism
@@ -1824,22 +2118,55 @@
 is not widely supported, and so removing it does not present any significant
 barrier to tools and projects adopting the new metadata format.
 
-Explicit markup for description
--------------------------------
-
-Currently, PyPI attempts to detect the markup format by rendering it as
-reStructuredText, and if that fails, treating it as plain text. Allowing
-the intended format to be stated explicitly will allow this guessing to be
-removed, and more informative error reports to be provided to users when
-a rendering error occurs.
-
-This is especially necessary since PyPI applies additional restrictions to
+
+Included text documents
+-----------------------
+
+Currently, PyPI attempts to determine the description's markup format by
+rendering it as reStructuredText, and if that fails, treating it as plain
+text.
+
+Furthermore, many projects simply read their long description in from an
+existing README file in ``setup.py``. The popularity of this practice is
+only expected to increase, as many online version control systems
+(including both GitHub and BitBucket) automatically display such files
+on the landing page for the project.
+
+Standardising on the inclusion of the long description as a separate
+file in the ``dist-info`` directory allows this to be simplified:
+
+* An existing file can just be copied into the ``dist-info`` directory as
+  part of creating the sdist
+* The expected markup format can be determined by inspecting the file
+  extension of the specified path
+
+Allowing the intended format to be stated explicitly in the path allows
+the format guessing to be removed and more informative error reports to be
+provided to users when a rendering error occurs.
+
+This is especially helpful since PyPI applies additional restrictions to
 the rendering process for security reasons, thus a description that renders
 correctly on a developer's system may still fail to render on the server.
 
-
-Deferred features
-=================
+The document naming system used to achieve this then makes it relatively
+straightforward to allow declaration of alternative markup formats like
+HTML, Markdown and AsciiDoc through the use of appropriate file
+extensions, as well as to define similar included documents for the
+project's license and changelog.
+
+Grouping the included document names into a single top level field gives
+automated tools the option of treating them as arbitrary documents without
+worrying about their contents.
+
+Requiring that the included documents be added to the ``dist-info`` metadata
+directory means that the complete metadata for the distribution can be
+extracted from an sdist or binary archive simply by extracting that
+directory, without needing to check for references to other files in the
+sdist.
+
+
+Appendix D: Deferred features
+=============================
 
 Several potentially useful features have been deliberately deferred in
 order to better prioritise our efforts in migrating to the new metadata
@@ -1847,15 +2174,26 @@
 new metadata, but which can be readily added in metadata 2.1 without
 breaking any use cases already supported by metadata 2.0.
 
-Once the ``pypi``, ``setuptools``, ``pip`` and ``distlib`` projects
-support creation and consumption of metadata 2.0, then we may revisit
-the creation of metadata 2.1 with these additional features.
-
-.. note::
-
-   Given the nature of this PEP as an interoperability specification,
-   this section will probably be removed before the PEP is accepted.
-   However, it's useful to have it here while discussion is ongoing.
+Once the ``pypi``, ``setuptools``, ``pip``, ``wheel`` and ``distlib``
+projects support creation and consumption of metadata 2.0, then we may
+revisit the creation of metadata 2.1 with some or all of these additional
+features.
+
+
+MIME type registration
+----------------------
+
+At some point after acceptance of the PEP, I will likely submit the
+following MIME type registration requests to IANA:
+
+* Full metadata: ``application/vnd.python.pymeta+json``
+* Abbreviated metadata: ``application/vnd.python.pymeta-short+json``
+* Essential dependency resolution metadata:
+  ``application/vnd.python.pymeta-dependencies+json``
+
+It's even possible we may be able to just register the ``vnd.python``
+namespace under the banner of the PSF rather than having to register
+the individual subformats.
 
 
 String methods in environment markers
@@ -1870,61 +2208,61 @@
 than a little strange.
 
 
-Module listing
---------------
-
-A top level ``"module"`` key, referencing a list of strings, with each
-giving the fully qualified name of a public package or module provided
-by the distribution.
-
-A flat list would be used in order to correctly accommodate namespace
-packages (where a distribution may provide subpackages or submodules without
-explicitly providing the parent namespace package).
-
-Example::
-
-    "modules": [
-        "comfy.chair"
-    ]
+Module and file listings
+------------------------
+
+Derived metadata giving the modules and files included in built
+distributions may be useful at some point in the future. (At least RPM
+provides this, and I believe the APT equivalent does as well)
 
 Explicitly providing a list of public module names will likely help
 with enabling features in RPM like "Requires: python(requests)", as well
 as providing richer static metadata for analysis from PyPI.
 
-However, this is just extra info that doesn't impact installing from wheels,
-so it is a good candidate for postponing to metadata 2.1.
-
-
-Additional metabuild hooks
---------------------------
-
-The following draft metabuild operations have been deferred for now:
+However, this is just extra info that doesn't impact reliably installing
+from wheels, so it is a good candidate for postponing to metadata 2.1
+(at the earliest).
+
+
+Metabuild system
+----------------
+
+This version of the metadata specification continues to use ``setup.py``
+and the distutils command syntax to invoke build and test related
+operations on a source archive or VCS checkout.
+
+It may be desirable to replace these in the future with tool independent
+entry points that support:
 
 * Generating the metadata file on a development system
-* Generating a source archive on a development system
+* Generating an sdist on a development system
 * Generating a binary archive on a build system
+* Running the test suite on a built (but not installed) distribution
 
 Metadata 2.0 deliberately focuses on wheel based installation, leaving
-tarball and sdist based installation to use the existing ``setup.py``
-based ``distutils`` command interface.
-
-In the meantime, the above three operations will continue to be handled
-through the ``distutils``/``setuptools`` command system:
+sdist, source archive, and VCS checkout based installation to use the
+existing ``setup.py`` based ``distutils`` command interface.
+
+In the meantime, the above operations will be handled through the
+``distutils``/``setuptools`` command system:
 
 * ``python setup.py dist_info``
 * ``python setup.py sdist``
 * ``python setup.py bdist_wheel``
-
-The following additional metabuild hooks may be added in metadata 2.1 to
+* ``python setup.py test``
+
+The following metabuild hooks may be defined in metadata 2.1 to
 cover these operations without relying on ``setup.py``:
 
-* ``make_dist_info``: generate the source archive's dist_info directory
-* ``make_sdist``: construct a source archive
-* ``build_wheel``: construct a binary wheel archive from an sdist source
-  archive
-
-Tentative signatures have been designed for those hooks, but they will
-not be pursued further until 2.1::
+* ``make_dist_info``: generate the sdist's dist_info directory
+* ``make_sdist``: create the contents of an sdist
+* ``build_dist``: create the contents of a binary wheel archive from an
+  unpacked sdist
+* ``test_built_dist``: run the test suite for a built distribution
+
+Tentative signatures have been designed for those hooks, but in order to
+better focus initial development efforts on the integration and installation
+use cases, they will not be pursued further until metadata 2.1::
 
     def make_dist_info(source_dir, info_dir):
         """Generate the contents of dist_info for an sdist archive
@@ -1949,11 +2287,11 @@
         Returns the distribution metadata as a dictionary.
         """
 
-    def build_wheel(sdist_dir, contents_dir, info_dir, compatibility=None):
-        """Generate the contents of a wheel archive
-
-        *source_dir* points to an unpacked source archive
-        *contents_dir* is the destination where the wheel contents should be
+    def build_dist(sdist_dir, built_dir, info_dir, compatibility=None):
+        """Generate the contents of a binary wheel archive
+
+        *sdist_dir* points to an unpacked sdist
+        *built_dir* is the destination where the wheel contents should be
         written (note that archiving the contents is the responsibility of
         the metabuild tool rather than the hook function)
         *info_dir* is the destination where the wheel metadata files should
@@ -1965,36 +2303,41 @@
         Returns the actual compatibility tag for the build
         """
 
-As with the existing metabuild hooks, checking for extras would be done
+    def test_built_dist(sdist_dir, built_dir, info_dir):
+        """Check a built (but not installed) distribution works as expected
+
+        *sdist_dir* points to an unpacked sdist
+        *built_dir* points to a platform appropriate unpacked wheel archive
+        (which may be missing the wheel metadata directory)
+        *info_dir* points to the appropriate wheel metadata directory
+
+        Requires that the distribution's test dependencies be installed
+        (indicated by the ``:test:`` extra).
+
+        Returns ``True`` if the check passes, ``False`` otherwise.
+        """
+
+As with the existing install hooks, checking for extras would be done
 using the same import based checks as are used for runtime extras. That
 way it doesn't matter if the additional dependencies were requested
 explicitly or just happen to be available on the system.
 
 
-Rejected Features
-=================
+Appendix E: Rejected features
+=============================
 
 The following features have been explicitly considered and rejected as
 introducing too much additional complexity for too small a gain in
 expressiveness.
 
-.. note::
-
-   Given the nature of this PEP as an interoperability specification,
-   this section will probably be removed before the PEP is accepted.
-   However, it's useful to have it here while discussion is ongoing.
-
-
-Detached metadata
------------------
-
-Rather than allowing some large items (such as the description field) to
-be distributed separately, this PEP instead defines two metadata subsets
-that should support more reasonable caching and API designs (for example,
-only the essential dependency resolution metadata would be distributed
-through TUF, and it is entirely possible the updated sdist, wheel and
-installation database specs will use the abbreviated metadata, leaving
-the full metadata as the province of index servers).
+
+Depending on source labels
+--------------------------
+
+There is no mechanism to express a dependency on a source label - they
+are included in the metadata for internal project reference only. Instead,
+dependencies must be expressed in terms of either public versions or else
+direct URL references.
 
 
 Alternative dependencies
@@ -2019,7 +2362,7 @@
 database driver" metadata extension where a project depends on SQL Alchemy,
 and then declares in the extension which database drivers are checked for
 compatibility by the upstream project (similar to the advisory
-``supports-platform`` field in the main metadata).
+``supports_environments`` field in the main metadata).
 
 We're also getting better support for "virtual provides" in this version of
 the metadata standard, so this may end up being an installer and index
@@ -2047,9 +2390,67 @@
 
 Under the revised metadata design, conditional "provides" based on runtime
 features or the environment would go in a separate "may_provide" field.
-However, I'm not convinced there's a great use case for that, so the idea
+However, it isn't clear there's any use case for doing that, so the idea
 is rejected unless someone can present a compelling use case (and even then
-the idea wouldn't be reconsidered until metadata 2.1 at the earliest).
+the idea won't be reconsidered until metadata 2.1 at the earliest).
+
+
+A hook to run tests against installed distributions
+---------------------------------------------------
+
+Earlier drafts of this PEP defined a hook for running automated
+tests against an *installed* distribution. This isn't actually what you
+generally want - you want the ability to test a *built* distribution,
+potentially relying on files which won't be included in the binary archives.
+
+RPM's "check" step also runs between the build step and the install step,
+rather than after the install step.
+
+Accordingly, the ``test_installed_dist`` hook has been removed, and the
+``test_built_dist`` metabuild hook has been tentatively defined. However,
+along with the rest of the metabuild hooks, further consideration has been
+deferred until metadata 2.1 at the earliest.
+
+
+Extensible signatures for the install hooks
+-------------------------------------------
+
+The install hooks have been deliberately designed to NOT accept arbitary
+keyword arguments that the hook implementation is then expected to ignore.
+
+The argument in favour of that API design technique is to allow the addition
+of new optional arguments in the future, without requiring the definition
+of a new install hook, or migration to version 3.0 of the metadata
+specification. It is a technique very commonly seen in function wrappers
+which merely pass arguments along to the inner function rather than
+processing them directly.
+
+However, the install hooks are already designed to have access to the full
+metadata for the distribution (including all metadata extensions and
+the previous/next version when appropriate), as well as to the full target
+deployment environment.
+
+This means there are two candidates for additional information that
+could be passed as arbitrary keyword arguments:
+
+* installer dependent settings
+* user provided installation options
+
+The first of those runs explicitly counter to one of the core goals of the
+metadata 2.0 specification: decoupling the software developer's choice of
+development and publication tools from the software integrator's choice of
+integration and deployment tools.
+
+The second is a complex problem that has a readily available workaround in
+the form of operating system level environment variables (this is also
+one way to interoperate with platform specific installation tools).
+
+Alternatively, installer developers may either implicitly inject an
+additional metadata extension when invoking the install hook, or else
+define an alternate hook signature as a distinct metadata extension to be
+provided by the distribution. Either of these approaches makes the
+reliance on installer-dependent behaviour suitably explicit in either
+the install hook implementation or the distribution metadata.
 
 
 References
diff --git a/pep-0426/pymeta-schema.json b/pep-0426/pymeta-schema.json
new file mode 100644
--- /dev/null
+++ b/pep-0426/pymeta-schema.json
@@ -0,0 +1,244 @@
+{
+  "id": "http://www.python.org/dev/peps/pep-0426/",
+  "$schema": "http://json-schema.org/draft-04/schema#",
+  "title": "Metadata for Python Software Packages 2.0",
+  "type": "object",
+  "properties": {
+    "metadata_version": {
+      "description": "Version of the file format",
+      "type": "string",
+      "pattern": "^(\d+(\.\d+)*)$"
+    },
+    "name": {
+      "description": "The name of the distribution.",
+      "type": "string",
+      "pattern": "^[0-9A-Za-z]([0-9A-Za-z_.-]*[0-9A-Za-z])?$"
+    },
+    "version": {
+      "description": "The distribution's public version identifier",
+      "type": "string",
+      "pattern": "^(\d+(\.\d+)*)((a|b|c|rc)(\d+))?(\.(post)(\d+))?(\.(dev)(\d+))?$"
+    },
+    "source_label": {
+      "description": "A constrained identifying text string",
+      "type": "string",
+      "pattern": "^[0-9a-z_.-+]+$"
+    },
+    "source_url": {
+      "description": "A string containing a full URL where the source for this specific version of the distribution can be downloaded.",
+      "type": "string",
+      "format": "uri"
+    },
+    "summary": {
+      "description": "A one-line summary of what the distribution does.",
+      "type": "string"
+    },
+    "document_names": {
+      "description": "Names of supporting metadata documents",
+      "type": "object",
+      "properties": {
+        "description": {
+          "type": "string",
+          "$ref": "#/definitions/document_name"
+        },
+        "changelog": {
+          "type": "string",
+          "$ref": "#/definitions/document_name"
+        },
+        "license": {
+          "type": "string",
+          "$ref": "#/definitions/document_name"
+        }
+      },
+      "additionalProperties": false
+    },
+    "keywords": {
+      "description": "A list of additional keywords to be used to assist searching for the distribution in a larger catalog.",
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    },
+    "license": {
+      "description": "A string indicating the license covering the distribution.",
+      "type": "string"
+    },
+    "classifiers": {
+      "description": "A list of strings, with each giving a single classification value for the distribution.",
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    },
+    "contacts": {
+      "description": "A list of contributor entries giving the recommended contact points for getting more information about the project.",
+      "type": "array",
+      "items": {
+        "type": "object",
+        "$ref": "#/definitions/contact"
+      }
+    },
+    "contributors": {
+      "description": "A list of contributor entries for other contributors not already listed as current project points of contact.",
+      "type": "array",
+      "items": {
+        "type": "object",
+        "$ref": "#/definitions/contact"
+      }
+    },
+    "project_urls": {
+      "description": "A mapping of arbitrary text labels to additional URLs relevant to the project.",
+      "type": "object"
+    },
+    "extras": {
+      "description": "A list of optional sets of dependencies that may be used to define conditional dependencies in \"may_require\" and similar fields.",
+      "type": "array",
+      "items": {
+        "type": "string",
+        "$ref": "#/definitions/extra_name"
+      }
+    },
+    "distributes": {
+      "description": "A list of subdistributions made available through this metadistribution.",
+      "type": "array",
+      "$ref": "#/definitions/dependencies"
+    },
+    "may_distribute": {
+      "description": "A list of subdistributions that may be made available through this metadistribution, based on the extras requested and the target deployment environment.",
+      "$ref": "#/definitions/conditional_dependencies"
+    },
+    "run_requires": {
+      "description": "A list of other distributions needed when to run this distribution.",
+      "type": "array",
+      "$ref": "#/definitions/dependencies"
+    },
+    "run_may_require": {
+      "description": "A list of other distributions that may be needed when this distribution is deployed, based on the extras requested and the target deployment environment.",
+      "$ref": "#/definitions/conditional_dependencies"
+    },
+    "test_requires": {
+      "description": "A list of other distributions needed when this distribution is tested.",
+      "type": "array",
+      "$ref": "#/definitions/dependencies"
+    },
+    "test_may_require": {
+      "description": "A list of other distributions that may be needed when this distribution is tested, based on the extras requested and the target deployment environment.",
+      "type": "array",
+      "$ref": "#/definitions/conditional_dependencies"
+    },
+    "build_requires": {
+      "description": "A list of other distributions needed when this distribution is built.",
+      "type": "array",
+      "$ref": "#/definitions/dependencies"
+    },
+    "build_may_require": {
+      "description": "A list of other distributions that may be needed when this distribution is built, based on the extras requested and the target deployment environment.",
+      "type": "array",
+      "$ref": "#/definitions/conditional_dependencies"
+    },
+    "dev_requires": {
+      "description": "A list of other distributions needed when this distribution is developed.",
+      "type": "array",
+      "$ref": "#/definitions/dependencies"
+    },
+    "dev_may_require": {
+      "description": "A list of other distributions that may be needed when this distribution is developed, based on the extras requested and the target deployment environment.",
+      "type": "array",
+      "$ref": "#/definitions/conditional_dependencies"
+    },
+    "provides": {
+      "description": "A list of strings naming additional dependency requirements that are satisfied by installing this distribution. These strings must be of the form Name or Name (Version), as for the requires field.",
+      "type": "array",
+      "items": {
+        "type": "string"
+      }
+    },
+    "obsoleted_by": {
+      "description": "A string that indicates that this project is no longer being developed. The named project provides a substitute or replacement.",
+      "type": "string",
+      "$ref": "#/definitions/version_specifier"
+    },
+    "supports_environments": {
+      "description": "A list of strings specifying the environments that the distribution explicitly supports.",
+      "type": "array",
+      "items": {
+        "type": "string",
+        "$ref": "#/definitions/environment_marker"
+      }
+    },
+    "metabuild_hooks": {
+      "description": "The metabuild_hooks field is used to define various operations that may be invoked on a distribution in a platform independent manner.",
+      "type": "object"
+    },
+    "extensions": {
+      "description": "Extensions to the metadata may be present in a mapping under the 'extensions' key.",
+      "type": "object"
+    }
+  },
+
+  "required": ["metadata_version", "name", "version"],
+  "additionalProperties": false,
+
+  "definitions": {
+    "contact": {
+      "type": "object",
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "email": {
+          "type": "string"
+        },
+        "url": {
+          "type": "string"
+        },
+        "role": {
+          "type": "string"
+        }
+      },
+      "required": ["name"],
+      "additionalProperties": false
+    },
+    "dependencies": {
+      "type": "array",
+      "items": {
+        "type": "string",
+        "$ref": "#/definitions/version_specifier"
+      }
+    },
+    "conditional_dependencies": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "extra": {
+            "type": "string",
+            "$ref": "#/definitions/extra_name"
+          },
+          "environment": {
+            "type": "string",
+            "$ref": "#/definitions/environment_marker"
+          },
+          "dependencies": {
+            "type": "array",
+            "$ref": "#/definitions/dependencies"
+          }
+        },
+        "required": ["dependencies"],
+        "additionalProperties": false
+      }
+    },
+    "version_specifier": {
+        "type": "string"
+    },
+    "extra_name": {
+        "type": "string"
+    },
+    "environment_marker": {
+        "type": "string"
+    },
+    "document_name": {
+        "type": "string"
+    }
+  }
+}
diff --git a/pep-0440.txt b/pep-0440.txt
--- a/pep-0440.txt
+++ b/pep-0440.txt
@@ -27,7 +27,7 @@
    This PEP was broken out of the metadata 2.0 specification in PEP 426.
 
    Unlike PEP 426, the notes that remain in this document are intended as
-   part of the final specification.
+   part of the final specification (except for this one).
 
 
 Definitions
@@ -40,7 +40,7 @@
 The following terms are to be interpreted as described in PEP 426:
 
 * "Distributions"
-* "Versions"
+* "Releases"
 * "Build tools"
 * "Index servers"
 * "Publication tools"
@@ -52,9 +52,13 @@
 Version scheme
 ==============
 
-Distribution versions are identified by both a public version identifier,
-which supports all defined version comparison operations, and a build
-label, which supports only strict equality comparisons.
+Distributions are identified by a public version identifier which
+supports all defined version comparison operations
+
+Distributions may also define a source label, which is not used by
+automated tools. Source labels are useful when a project internal
+versioning scheme requires translation to create a compliant public
+version identifier.
 
 The version scheme is used both to describe the distribution version
 provided by a particular distribution archive, as well as to place
@@ -84,7 +88,7 @@
 * Post-release segment: ``.postN``
 * Development release segment: ``.devN``
 
-Any given version will be a "release", "pre-release", "post-release" or
+Any given release will be a "final release", "pre-release", "post-release" or
 "developmental release" as defined in the following sections.
 
 .. note::
@@ -105,28 +109,37 @@
 Source labels are text strings with minimal defined semantics.
 
 To ensure source labels can be readily incorporated as part of file names
-and URLs, they MUST be comprised of only ASCII alphanumerics, plus signs,
-periods and hyphens.
+and URLs, and to avoid formatting inconsistences in hexadecimal hash
+representations they MUST be limited to the following set of permitted
+characters:
 
-In addition, source labels MUST be unique within a given distribution.
+* Lowercase ASCII letters (``[a-z]``)
+* ASCII digits (``[0-9]``)
+* underscores (``_``)
+* hyphens (``-``)
+* periods (``.``)
+* plus signs (``+``)
 
-As with distribution names, all comparisons of source labels MUST be case
-insensitive.
+Source labels MUST start and end with an ASCII letter or digit.
 
+Source labels MUST be unique within each project and MUST NOT match any
+defined version for the project.
 
-Releases
---------
 
-A version identifier that consists solely of a release segment is termed
-a "release".
+Final releases
+--------------
 
-The release segment consists of one or more non-negative integer values,
-separated by dots::
+A version identifier that consists solely of a release segment is
+termed a "final release".
+
+The release segment consists of one or more non-negative integer
+values, separated by dots::
 
     N[.N]+
 
-Releases within a project will typically be numbered in a consistently
-increasing fashion.
+Final releases within a project MUST be numbered in a consistently
+increasing fashion, otherwise automated tools will not be able to upgrade
+them correctly.
 
 Comparison and ordering of release segments considers the numeric value
 of each component of the release segment in turn. When comparing release
@@ -157,8 +170,8 @@
     2.0
     2.0.1
 
-A release series is any set of release numbers that start with a common
-prefix. For example, ``3.3.1``, ``3.3.5`` and ``3.3.9.45`` are all
+A release series is any set of final release numbers that start with a
+common prefix. For example, ``3.3.1``, ``3.3.5`` and ``3.3.9.45`` are all
 part of the ``3.3`` release series.
 
 .. note::
@@ -206,8 +219,8 @@
 Post-releases
 -------------
 
-Some projects use post-releases to address minor errors in a release that
-do not affect the distributed software (for example, correcting an error
+Some projects use post-releases to address minor errors in a final release
+that do not affect the distributed software (for example, correcting an error
 in the release notes).
 
 If used as part of a project's development cycle, these post-releases are
@@ -371,7 +384,7 @@
    .devN, aN, bN, cN, rcN, <no suffix>, .postN
 
 Note that `rc` will always sort after `c` (regardless of the numeric
-component) although they are semantically equivalent. Tools are free to
+component) although they are semantically equivalent. Tools MAY
 reject this case as ambiguous and remain in compliance with the PEP.
 
 Within an alpha (``1.0a1``), beta (``1.0b1``), or release candidate
@@ -506,6 +519,22 @@
 version comparison semantics.
 
 
+Olson database versioning
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``pytz`` project inherits its versioning scheme from the corresponding
+Olson timezone database versioning scheme: the year followed by a lowercase
+character indicating the version of the database within that year.
+
+This can be translated to a compliant 3-part version identifier as
+``0.<year>.<serial>``, where the serial starts at zero (for the '<year>a'
+release) and is incremented with each subsequent database update within the
+year.
+
+As with other translated version identifiers, the corresponding Olson
+database version would be recorded in the source label field.
+
+
 Version specifiers
 ==================
 
@@ -521,7 +550,7 @@
 * ``~=``: `Compatible release`_ clause
 * ``==``: `Version matching`_ clause
 * ``!=``: `Version exclusion`_ clause
-* ``is``: `Source reference`_ clause
+* ``is``: `Direct reference`_ clause
 * ``<=``, ``>=``: `Inclusive ordered comparison`_ clause
 * ``<``, ``>``: `Exclusive ordered comparison`_ clause
 
@@ -605,6 +634,11 @@
 release segment to ensure the release segments are compared with the same
 length.
 
+Whether or not strict version matching is appropriate depends on the specific
+use case for the version specifier. Automated tools SHOULD at least issue
+warnings and MAY reject them entirely when strict version matches are used
+inappropriately.
+
 Prefix matching may be requested instead of strict comparison, by appending
 a trailing ``.*`` to the version identifier in the version matching clause.
 This means that additional trailing segments will be ignored when
@@ -645,49 +679,42 @@
     != 1.1.*      # Same prefix, so 1.1.post1 does not match clause
 
 
-Source reference
+Direct reference
 ----------------
 
-A source reference includes the source reference operator ``is`` and
-a source label or a source URL.
+A direct reference includes the direct reference operator ``is`` and
+an explicit URL.
 
-Installation tools MAY also permit direct references to a platform
-appropriate binary archive in a source reference clause.
+Whether or not direct references are appropriate depends on the specific
+use case for the version specifier. Automated tools SHOULD at least issue
+warnings and MAY reject them entirely when direct references are used
+inappropriately.
 
-Publication tools and public index servers SHOULD NOT permit direct
-references to a platform appropriate binary archive in a source
-reference clause.
-
-Source label matching works solely on strict equality comparisons: the
-candidate source label must be exactly the same as the source label in the
-version clause for the clause to match the candidate distribution.
-
-For example, a source reference could be used to depend directly on a
-version control hash based identifier rather than the translated public
-version::
-
-    exact-dependency (is 1.3.7+build.11.e0f985a)
-
-A source URL is distinguished from a source label by the presence of
-``:`` and ``/`` characters in the source reference. As these characters
-are not permitted in source labels, they indicate that the reference uses
-a source URL.
-
-Some appropriate targets for a source URL are a source tarball, an sdist
-archive or a direct reference to a tag or specific commit in an online
-version control system. The exact URLs and
-targets supported will be installation tool specific.
+Depending on the use case, some appropriate targets for a direct URL
+reference are a source archive, an sdist, a wheel binary archive or a
+direct reference to a tag or specific commit in an online version control
+system. The exact URLs and targets supported will be automated tool
+specific.
 
 For example, a local source archive may be referenced directly::
 
     pip (is file:///localbuilds/pip-1.3.1.zip)
 
-All source URL references SHOULD either specify a local file URL, a secure
-transport mechanism (such as ``https``) or else include an expected hash
-value in the URL for verification purposes. If an insecure network
-transport is specified without any hash information (or with hash
-information that the tool doesn't understand), automated tools SHOULD
-at least emit a warning and MAY refuse to rely on the URL.
+Alternatively, a prebuilt archive may also be referenced::
+
+    pip (is file:///localbuilds/pip-1.3.1-py33-none-any.whl)
+
+
+All direct URL references that do not refer to a local file URL SHOULD
+specify a secure transport mechanism (such as ``https``), include an
+expected hash value in the URL for verification purposes, or both. If an
+insecure transport is specified without any hash information, with hash
+information that the tool doesn't understand, or with a selected hash
+algortihm that the tool considers too weak to trust, automated tools
+SHOULD at least emit a warning and MAY refuse to rely on the URL.
+
+Index servers MAY place additional restrictions on direct references (such
+as requiring all references be to files hosted on the index server itself).
 
 It is RECOMMENDED that only hashes which are unconditionally provided by
 the latest version of the standard library's ``hashlib`` module be used
@@ -695,24 +722,17 @@
 ``'md5'``, ``'sha1'``, ``'sha224'``, ``'sha256'``, ``'sha384'``, and
 ``'sha512'``.
 
-For source archive references, an expected hash value may be
+For source archive and wheel references, an expected hash value may be
 specified by including a ``<hash-algorithm>=<expected-hash>`` as part of
 the URL fragment.
 
-For version control references, the ``VCS+protocol`` scheme SHOULD be
+Version control references, the ``VCS+protocol`` scheme SHOULD be
 used to identify both the version control system and the secure transport.
 
 To support version control systems that do not support including commit or
 tag references directly in the URL, that information may be appended to the
 end of the URL using the ``@<tag>`` notation.
 
-The use of ``is`` when defining dependencies for published distributions
-is strongly discouraged as it greatly complicates the deployment of
-security fixes. The source label matching operator is intended primarily
-for use when defining dependencies for repeatable *deployments of
-applications* while using a shared distribution index, as well as to
-reference dependencies which are not published through an index server.
-
 
 Inclusive ordered comparison
 ----------------------------
@@ -752,62 +772,47 @@
 ------------------------
 
 Pre-releases of any kind, including developmental releases, are implicitly
-excluded from all version specifiers, *unless* a pre-release or developmental
-release is explicitly mentioned in one of the clauses. For example, these
-specifiers implicitly exclude all pre-releases and development
-releases of later versions::
-
-    2.2
-    >= 1.0
-
-While these specifiers would include at least some of them::
-
-    2.2.dev0
-    2.2, != 2.3b2
-    >= 1.0a1
-    >= 1.0c1
-    >= 1.0, != 1.0b2
-    >= 1.0, < 2.0.dev123
+excluded from all version specifiers, *unless* they are already present
+on the system, explicitly requested by the user, or if the only available
+version that satisfies the version specifier is a pre-release.
 
 By default, dependency resolution tools SHOULD:
 
 * accept already installed pre-releases for all version specifiers
-* accept remotely available pre-releases for version specifiers which
-  include at least one version clauses that references a pre-release
+* accept remotely available pre-releases for version specifiers where
+  there is no final or post release that satisfies the version specifier
 * exclude all other pre-releases from consideration
 
+Dependency resolution tools MAY issue a warning if a pre-release is needed
+to satisfy a version specifier.
+
 Dependency resolution tools SHOULD also allow users to request the
 following alternative behaviours:
 
 * accepting pre-releases for all version specifiers
 * excluding pre-releases for all version specifiers (reporting an error or
-  warning if a pre-release is already installed locally)
+  warning if a pre-release is already installed locally, or if a
+  pre-release is the only way to satisfy a particular specifier)
 
 Dependency resolution tools MAY also allow the above behaviour to be
 controlled on a per-distribution basis.
 
-Post-releases and purely numeric releases receive no special treatment in
-version specifiers - they are always included unless explicitly excluded.
+Post-releases and final releases receive no special treatment in version
+specifiers - they are always included unless explicitly excluded.
 
 
 Examples
 --------
 
-* ``3.1``: version 3.1 or later, but not
-   version 4.0 or later. Excludes pre-releases and developmental releases.
-* ``3.1.2``: version 3.1.2 or later, but not
-   version 3.2.0 or later. Excludes pre-releases and developmental releases.
-* ``3.1a1``: version 3.1a1 or later, but not
-   version 4.0 or later. Allows pre-releases like 3.2a4 and developmental
-   releases like 3.2.dev1.
+* ``3.1``: version 3.1 or later, but not version 4.0 or later.
+* ``3.1.2``: version 3.1.2 or later, but not version 3.2.0 or later.
+* ``3.1a1``: version 3.1a1 or later, but not version 4.0 or later.
 * ``== 3.1``: specifically version 3.1 (or 3.1.0), excludes all pre-releases,
   post releases, developmental releases and any 3.1.x maintenance releases.
-* ``== 3.1.*``: any version that starts with 3.1, excluding pre-releases and
-  developmental releases. Equivalent to the ``3.1.0`` compatible release
-  clause.
+* ``== 3.1.*``: any version that starts with 3.1. Equivalent to the
+  ``3.1.0`` compatible release clause.
 * ``3.1.0, != 3.1.3``: version 3.1.0 or later, but not version 3.1.3 and
-  not version 3.2.0 or later. Excludes pre-releases and developmental
-  releases.
+  not version 3.2.0 or later.
 
 
 Updating the versioning specification
@@ -825,28 +830,29 @@
 
 * Moved the description of version specifiers into the versioning PEP
 
-* added the "source label" concept to better handle projects that wish to
+* Aadded the "source label" concept to better handle projects that wish to
   use a non-compliant versioning scheme internally, especially those based
   on DVCS hashes
   
-* added the "compatible release" clause
+* Added the "compatible release" clause
 
-* added the "source reference" clause
+* Added the "direct reference" clause
 
-* added the trailing wildcard syntax for prefix based version matching
+* Added the trailing wildcard syntax for prefix based version matching
   and exclusion
 
-* changed the top level sort position of the ``.devN`` suffix
+* Changed the top level sort position of the ``.devN`` suffix
 
-* allowed single value version numbers
+* Allowed single value version numbers
 
-* explicit exclusion of leading or trailing whitespace
+* Explicit exclusion of leading or trailing whitespace
 
-* explicit criterion for the exclusion of date based versions
+* Explicit criterion for the exclusion of date based versions
 
-* implicitly exclude pre-releases unless explicitly requested
+* Implicitly exclude pre-releases unless they're already present or
+  needed to satisfy a dependency
 
-* treat post releases the same way as unqualified releases
+* Treat post releases the same way as unqualified releases
 
 * Discuss ordering and dependencies across metadata versions
 
@@ -995,11 +1001,12 @@
 specifiers for no adequately justified reason.
 
 The updated interpretation is intended to make it difficult to accidentally
-accept a pre-release version as satisfying a dependency, while allowing
-pre-release versions to be explicitly requested when needed.
+accept a pre-release version as satisfying a dependency, while still
+allowing pre-release versions to be retrieved automatically when that's the
+only way to satisfy a dependency.
 
 The "some forward compatibility assumed" default version constraint is
-taken directly from the Ruby community's "pessimistic version constraint"
+derived from the Ruby community's "pessimistic version constraint"
 operator [2]_ to allow projects to take a cautious approach to forward
 compatibility promises, while still easily setting a minimum required
 version for their dependencies. It is made the default behaviour rather
@@ -1022,16 +1029,17 @@
 
 The trailing wildcard syntax to request prefix based version matching was
 added to make it possible to sensibly define both compatible release clauses
-and the desired pre-release handling semantics for ``<`` and ``>`` ordered
-comparison clauses.
+and the desired pre- and post-release handling semantics for ``<`` and ``>``
+ordered comparison clauses.
 
-Source references are added for two purposes. In conjunction with source
-labels, they allow hash based references to exact versions that aren't
-compliant with the fully ordered public version scheme, such as those
-generated from version control. In combination with source URLs, they
-also allow the new metadata standard to natively support an existing
-feature of ``pip``, which allows arbitrary URLs like
-``file:///localbuilds/exampledist-1.0-py33-none-any.whl``.
+Direct references are added as an "escape clause" to handle messy real
+world situations that don't map neatly to the standard distribution model.
+This includes dependencies on unpublished software for internal use, as well
+as handling the more complex compatibility issues that may arise when
+wrapping third party libraries as C extensions (this is of especial concern
+to the scientific community). Index servers are given a lot of freedom to
+disallow them, since they're intended primarily as a tool for integrators
+rather than publishers.
 
 
 References

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Wed Jun 19 18:02:53 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 19 Jun 2013 18:02:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_fix_libffi_bui?=
 =?utf-8?q?ld_on_AIX_=28closes_=2318248=29?=
Message-ID: <3bb9t55hlyzRBT@mail.python.org>

http://hg.python.org/cpython/rev/00082406e13f
changeset:   84219:00082406e13f
branch:      3.3
parent:      84217:410ea970866e
user:        Benjamin Peterson <benjamin at python.org>
date:        Wed Jun 19 09:01:42 2013 -0700
summary:
  fix libffi build on AIX (closes #18248)

files:
  Misc/NEWS                              |  2 ++
  Modules/_ctypes/libffi.diff            |  2 +-
  Modules/_ctypes/libffi/fficonfig.py.in |  2 +-
  3 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -32,6 +32,8 @@
 Library
 -------
 
+- Issue #18248: Fix libffi build on AIX.
+
 - Issue #18259: Declare sethostname in socketmodule.c for AIX
 
 - Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
diff --git a/Modules/_ctypes/libffi.diff b/Modules/_ctypes/libffi.diff
--- a/Modules/_ctypes/libffi.diff
+++ b/Modules/_ctypes/libffi.diff
@@ -135,7 +135,7 @@
 +    'M32R': ['src/m32r/sysv.S', 'src/m32r/ffi.c'],
 +    'M68K': ['src/m68k/ffi.c', 'src/m68k/sysv.S'],
 +    'POWERPC': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S', 'src/powerpc/linux64.S', 'src/powerpc/linux64_closure.S'],
-+    'POWERPC_AIX': ['src/powerpc/ffi.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
++    'POWERPC_AIX': ['src/powerpc/ffi_darwin.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
 +    'POWERPC_FREEBSD': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S'],
 +    'ARM': ['src/arm/sysv.S', 'src/arm/ffi.c'],
 +    'LIBFFI_CRIS': ['src/cris/sysv.S', 'src/cris/ffi.c'],
diff --git a/Modules/_ctypes/libffi/fficonfig.py.in b/Modules/_ctypes/libffi/fficonfig.py.in
--- a/Modules/_ctypes/libffi/fficonfig.py.in
+++ b/Modules/_ctypes/libffi/fficonfig.py.in
@@ -16,7 +16,7 @@
     'M32R': ['src/m32r/sysv.S', 'src/m32r/ffi.c'],
     'M68K': ['src/m68k/ffi.c', 'src/m68k/sysv.S'],
     'POWERPC': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S', 'src/powerpc/linux64.S', 'src/powerpc/linux64_closure.S'],
-    'POWERPC_AIX': ['src/powerpc/ffi.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
+    'POWERPC_AIX': ['src/powerpc/ffi_darwin.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
     'POWERPC_FREEBSD': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S'],
     'ARM': ['src/arm/sysv.S', 'src/arm/ffi.c'],
     'LIBFFI_CRIS': ['src/cris/sysv.S', 'src/cris/ffi.c'],

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 18:02:55 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Wed, 19 Jun 2013 18:02:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogbWVyZ2UgMy4zICgjMTgyNDgp?=
Message-ID: <3bb9t70fT5zR95@mail.python.org>

http://hg.python.org/cpython/rev/974d4844d5a7
changeset:   84220:974d4844d5a7
parent:      84218:959f4ce4d590
parent:      84219:00082406e13f
user:        Benjamin Peterson <benjamin at python.org>
date:        Wed Jun 19 09:01:58 2013 -0700
summary:
  merge 3.3 (#18248)

files:
  Misc/NEWS                              |  2 ++
  Modules/_ctypes/libffi.diff            |  2 +-
  Modules/_ctypes/libffi/fficonfig.py.in |  2 +-
  3 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #18248: Fix libffi build on AIX.
+
 - Issue #18259: Declare sethostname in socketmodule.c for AIX
 
 - Issue #18147: Add diagnostic functions to ssl.SSLContext(). get_ca_list()
diff --git a/Modules/_ctypes/libffi.diff b/Modules/_ctypes/libffi.diff
--- a/Modules/_ctypes/libffi.diff
+++ b/Modules/_ctypes/libffi.diff
@@ -135,7 +135,7 @@
 +    'M32R': ['src/m32r/sysv.S', 'src/m32r/ffi.c'],
 +    'M68K': ['src/m68k/ffi.c', 'src/m68k/sysv.S'],
 +    'POWERPC': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S', 'src/powerpc/linux64.S', 'src/powerpc/linux64_closure.S'],
-+    'POWERPC_AIX': ['src/powerpc/ffi.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
++    'POWERPC_AIX': ['src/powerpc/ffi_darwin.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
 +    'POWERPC_FREEBSD': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S'],
 +    'ARM': ['src/arm/sysv.S', 'src/arm/ffi.c'],
 +    'LIBFFI_CRIS': ['src/cris/sysv.S', 'src/cris/ffi.c'],
diff --git a/Modules/_ctypes/libffi/fficonfig.py.in b/Modules/_ctypes/libffi/fficonfig.py.in
--- a/Modules/_ctypes/libffi/fficonfig.py.in
+++ b/Modules/_ctypes/libffi/fficonfig.py.in
@@ -16,7 +16,7 @@
     'M32R': ['src/m32r/sysv.S', 'src/m32r/ffi.c'],
     'M68K': ['src/m68k/ffi.c', 'src/m68k/sysv.S'],
     'POWERPC': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S', 'src/powerpc/linux64.S', 'src/powerpc/linux64_closure.S'],
-    'POWERPC_AIX': ['src/powerpc/ffi.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
+    'POWERPC_AIX': ['src/powerpc/ffi_darwin.c', 'src/powerpc/aix.S', 'src/powerpc/aix_closure.S'],
     'POWERPC_FREEBSD': ['src/powerpc/ffi.c', 'src/powerpc/sysv.S', 'src/powerpc/ppc_closure.S'],
     'ARM': ['src/arm/sysv.S', 'src/arm/ffi.c'],
     'LIBFFI_CRIS': ['src/cris/sysv.S', 'src/cris/ffi.c'],

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 23:01:08 2013
From: python-checkins at python.org (r.david.murray)
Date: Wed, 19 Jun 2013 23:01:08 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Tweak_programm?=
 =?utf-8?q?ing_faq_examples_so_that_it_=28mostly=29_passes_doctest=2E?=
Message-ID: <3bbJVD4T8Dz7Lkl@mail.python.org>

http://hg.python.org/cpython/rev/0113247f894b
changeset:   84221:0113247f894b
branch:      3.3
parent:      84219:00082406e13f
user:        R David Murray <rdmurray at bitdance.com>
date:        Wed Jun 19 16:58:26 2013 -0400
summary:
  Tweak programming faq examples so that it (mostly) passes doctest.

The exception is the import related questions at the end, which
need to be rewritten anyway.

files:
  Doc/faq/programming.rst |  37 ++++++++++++++--------------
  1 files changed, 19 insertions(+), 18 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -214,9 +214,9 @@
 Assume you use a for loop to define a few different lambdas (or even plain
 functions), e.g.::
 
-   squares = []
-   for x in range(5):
-      squares.append(lambda: x**2)
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda: x**2)
 
 This gives you a list that contains 5 lambdas that calculate ``x**2``.  You
 might expect that, when called, they would return, respectively, ``0``, ``1``,
@@ -241,9 +241,9 @@
 In order to avoid this, you need to save the values in variables local to the
 lambdas, so that they don't rely on the value of the global ``x``::
 
-   squares = []
-   for x in range(5):
-      squares.append(lambda n=x: n**2)
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda n=x: n**2)
 
 Here, ``n=x`` creates a new variable ``n`` local to the lambda and computed
 when the lambda is defined so that it has the same value that ``x`` had at
@@ -592,11 +592,11 @@
 Since the comma is not an operator, but a separator between expressions the
 above is evaluated as if you had entered::
 
-    >>> ("a" in "b"), "a"
+    ("a" in "b"), "a"
 
 not::
 
-    >>> "a" in ("b", "a")
+    "a" in ("b", "a")
 
 The same is true of the various assignment operators (``=``, ``+=`` etc).  They
 are not truly operators but syntactic delimiters in assignment statements.
@@ -744,6 +744,7 @@
 unicode data, try using a :class:`io.StringIO` object or the :mod:`array`
 module::
 
+   >>> import io
    >>> s = "Hello, world"
    >>> sio = io.StringIO(s)
    >>> sio.getvalue()
@@ -761,7 +762,7 @@
    array('u', 'Hello, world')
    >>> a[0] = 'y'
    >>> print(a)
-   array('u', 'yello world')
+   array('u', 'yello, world')
    >>> a.tounicode()
    'yello, world'
 
@@ -1060,7 +1061,7 @@
 
 You probably tried to make a multidimensional array like this::
 
-   A = [[None] * 2] * 3
+   >>> A = [[None] * 2] * 3
 
 This looks correct if you print it::
 
@@ -1615,13 +1616,13 @@
 (permissions, free space, etc...) to write the compiled module back to the
 directory.
 
-Running Python on a top level script is not considered an import and no ``.pyc``
-will be created.  For example, if you have a top-level module ``abc.py`` that
-imports another module ``xyz.py``, when you run abc, ``xyz.pyc`` will be created
-since xyz is imported, but no ``abc.pyc`` file will be created since ``abc.py``
-isn't being imported.
+Running Python on a top level script is not considered an import and no
+``.pyc`` will be created.  For example, if you have a top-level module
+``foo.py`` that imports another module ``xyz.py``, when you run ``foo``,
+``xyz.pyc`` will be created since ``xyz`` is imported, but no ``foo.pyc`` file
+will be created since ``foo.py`` isn't being imported.
 
-If you need to create abc.pyc -- that is, to create a .pyc file for a module
+If you need to create ``foo.pyc`` -- that is, to create a ``.pyc`` file for a module
 that is not imported -- you can, using the :mod:`py_compile` and
 :mod:`compileall` modules.
 
@@ -1629,9 +1630,9 @@
 the ``compile()`` function in that module interactively::
 
    >>> import py_compile
-   >>> py_compile.compile('abc.py')
+   >>> py_compile.compile('foo.py')                 # doctest: +SKIP
 
-This will write the ``.pyc`` to the same location as ``abc.py`` (or you can
+This will write the ``.pyc`` to the same location as ``foo.py`` (or you can
 override that with the optional parameter ``cfile``).
 
 You can also automatically compile all files in a directory or directories using

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 23:01:10 2013
From: python-checkins at python.org (r.david.murray)
Date: Wed, 19 Jun 2013 23:01:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge=3A_Tweak_programming_faq_examples_so_that_it_=28mo?=
 =?utf-8?q?stly=29_passes_doctest=2E?=
Message-ID: <3bbJVG0RVwz7Lkd@mail.python.org>

http://hg.python.org/cpython/rev/dcb4985d82bc
changeset:   84222:dcb4985d82bc
parent:      84220:974d4844d5a7
parent:      84221:0113247f894b
user:        R David Murray <rdmurray at bitdance.com>
date:        Wed Jun 19 16:59:22 2013 -0400
summary:
  Merge: Tweak programming faq examples so that it (mostly) passes doctest.

files:
  Doc/faq/programming.rst |  37 ++++++++++++++--------------
  1 files changed, 19 insertions(+), 18 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -214,9 +214,9 @@
 Assume you use a for loop to define a few different lambdas (or even plain
 functions), e.g.::
 
-   squares = []
-   for x in range(5):
-      squares.append(lambda: x**2)
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda: x**2)
 
 This gives you a list that contains 5 lambdas that calculate ``x**2``.  You
 might expect that, when called, they would return, respectively, ``0``, ``1``,
@@ -241,9 +241,9 @@
 In order to avoid this, you need to save the values in variables local to the
 lambdas, so that they don't rely on the value of the global ``x``::
 
-   squares = []
-   for x in range(5):
-      squares.append(lambda n=x: n**2)
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda n=x: n**2)
 
 Here, ``n=x`` creates a new variable ``n`` local to the lambda and computed
 when the lambda is defined so that it has the same value that ``x`` had at
@@ -592,11 +592,11 @@
 Since the comma is not an operator, but a separator between expressions the
 above is evaluated as if you had entered::
 
-    >>> ("a" in "b"), "a"
+    ("a" in "b"), "a"
 
 not::
 
-    >>> "a" in ("b", "a")
+    "a" in ("b", "a")
 
 The same is true of the various assignment operators (``=``, ``+=`` etc).  They
 are not truly operators but syntactic delimiters in assignment statements.
@@ -744,6 +744,7 @@
 unicode data, try using a :class:`io.StringIO` object or the :mod:`array`
 module::
 
+   >>> import io
    >>> s = "Hello, world"
    >>> sio = io.StringIO(s)
    >>> sio.getvalue()
@@ -761,7 +762,7 @@
    array('u', 'Hello, world')
    >>> a[0] = 'y'
    >>> print(a)
-   array('u', 'yello world')
+   array('u', 'yello, world')
    >>> a.tounicode()
    'yello, world'
 
@@ -1060,7 +1061,7 @@
 
 You probably tried to make a multidimensional array like this::
 
-   A = [[None] * 2] * 3
+   >>> A = [[None] * 2] * 3
 
 This looks correct if you print it::
 
@@ -1615,13 +1616,13 @@
 (permissions, free space, etc...) to write the compiled module back to the
 directory.
 
-Running Python on a top level script is not considered an import and no ``.pyc``
-will be created.  For example, if you have a top-level module ``abc.py`` that
-imports another module ``xyz.py``, when you run abc, ``xyz.pyc`` will be created
-since xyz is imported, but no ``abc.pyc`` file will be created since ``abc.py``
-isn't being imported.
+Running Python on a top level script is not considered an import and no
+``.pyc`` will be created.  For example, if you have a top-level module
+``foo.py`` that imports another module ``xyz.py``, when you run ``foo``,
+``xyz.pyc`` will be created since ``xyz`` is imported, but no ``foo.pyc`` file
+will be created since ``foo.py`` isn't being imported.
 
-If you need to create abc.pyc -- that is, to create a .pyc file for a module
+If you need to create ``foo.pyc`` -- that is, to create a ``.pyc`` file for a module
 that is not imported -- you can, using the :mod:`py_compile` and
 :mod:`compileall` modules.
 
@@ -1629,9 +1630,9 @@
 the ``compile()`` function in that module interactively::
 
    >>> import py_compile
-   >>> py_compile.compile('abc.py')
+   >>> py_compile.compile('foo.py')                 # doctest: +SKIP
 
-This will write the ``.pyc`` to the same location as ``abc.py`` (or you can
+This will write the ``.pyc`` to the same location as ``foo.py`` (or you can
 override that with the optional parameter ``cfile``).
 
 You can also automatically compile all files in a directory or directories using

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 19 23:01:11 2013
From: python-checkins at python.org (r.david.murray)
Date: Wed, 19 Jun 2013 23:01:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Tweak_programm?=
 =?utf-8?q?ing_faq_examples_so_that_it_=28mostly=29_passes_doctest=2E?=
Message-ID: <3bbJVH3Mcmz7Lk3@mail.python.org>

http://hg.python.org/cpython/rev/6f535c725b27
changeset:   84223:6f535c725b27
branch:      2.7
parent:      84207:a5ef439f3c9e
user:        R David Murray <rdmurray at bitdance.com>
date:        Wed Jun 19 17:00:43 2013 -0400
summary:
  Tweak programming faq examples so that it (mostly) passes doctest.

Back port of 0113247f894b from 3.3.

The exceptions are the import related questions at the end, which
need to be rewritten anyway, and a math example that doesn't
exist in the 3.3+ docs that I didn't bother trying to fix.

files:
  Doc/faq/programming.rst |  41 ++++++++++++++--------------
  1 files changed, 21 insertions(+), 20 deletions(-)


diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -360,9 +360,9 @@
 Assume you use a for loop to define a few different lambdas (or even plain
 functions), e.g.::
 
-   squares = []
-   for x in range(5):
-      squares.append(lambda: x**2)
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda: x**2)
 
 This gives you a list that contains 5 lambdas that calculate ``x**2``.  You
 might expect that, when called, they would return, respectively, ``0``, ``1``,
@@ -387,9 +387,9 @@
 In order to avoid this, you need to save the values in variables local to the
 lambdas, so that they don't rely on the value of the global ``x``::
 
-   squares = []
-   for x in range(5):
-      squares.append(lambda n=x: n**2)
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda n=x: n**2)
 
 Here, ``n=x`` creates a new variable ``n`` local to the lambda and computed
 when the lambda is defined so that it has the same value that ``x`` had at
@@ -748,11 +748,11 @@
 Since the comma is not an operator, but a separator between expressions the
 above is evaluated as if you had entered::
 
-    >>> ("a" in "b"), "a"
+    ("a" in "b"), "a"
 
 not::
 
-    >>> "a" in ("b", "a")
+    "a" in ("b", "a")
 
 The same is true of the various assignment operators (``=``, ``+=`` etc).  They
 are not truly operators but syntactic delimiters in assignment statements.
@@ -897,6 +897,7 @@
 You can't, because strings are immutable.  If you need an object with this
 ability, try converting the string to a list or use the array module::
 
+   >>> import io
    >>> s = "Hello, world"
    >>> a = list(s)
    >>> print a
@@ -910,7 +911,7 @@
    >>> print a
    array('c', 'Hello, world')
    >>> a[0] = 'y' ; print a
-   array('c', 'yello world')
+   array('c', 'yello, world')
    >>> a.tostring()
    'yello, world'
 
@@ -1172,7 +1173,7 @@
 
 You probably tried to make a multidimensional array like this::
 
-   A = [[None] * 2] * 3
+   >>> A = [[None] * 2] * 3
 
 This looks correct if you print it::
 
@@ -1740,13 +1741,13 @@
 (permissions, free space, etc...) to write the compiled module back to the
 directory.
 
-Running Python on a top level script is not considered an import and no ``.pyc``
-will be created.  For example, if you have a top-level module ``abc.py`` that
-imports another module ``xyz.py``, when you run abc, ``xyz.pyc`` will be created
-since xyz is imported, but no ``abc.pyc`` file will be created since ``abc.py``
-isn't being imported.
-
-If you need to create abc.pyc -- that is, to create a .pyc file for a module
+Running Python on a top level script is not considered an import and no
+``.pyc`` will be created.  For example, if you have a top-level module
+``foo.py`` that imports another module ``xyz.py``, when you run ``foo``,
+``xyz.pyc`` will be created since ``xyz`` is imported, but no ``foo.pyc`` file
+will be created since ``foo.py`` isn't being imported.
+
+If you need to create ``foo.pyc`` -- that is, to create a ``.pyc`` file for a module
 that is not imported -- you can, using the :mod:`py_compile` and
 :mod:`compileall` modules.
 
@@ -1754,9 +1755,9 @@
 the ``compile()`` function in that module interactively::
 
    >>> import py_compile
-   >>> py_compile.compile('abc.py')
-
-This will write the ``.pyc`` to the same location as ``abc.py`` (or you can
+   >>> py_compile.compile('foo.py')                 # doctest: +SKIP
+
+This will write the ``.pyc`` to the same location as ``foo.py`` (or you can
 override that with the optional parameter ``cfile``).
 
 You can also automatically compile all files in a directory or directories using

-- 
Repository URL: http://hg.python.org/cpython

From root at python.org  Thu Jun 20 03:00:22 2013
From: root at python.org (Cron Daemon)
Date: Thu, 20 Jun 2013 03:00:22 +0200
Subject: [Python-checkins] Cron <docs@dinsdale> /home/docs/build-devguide
Message-ID: <E1UpTEc-0002by-QK@dinsdale.python.org>

abort: error: Connection timed out

From root at python.org  Thu Jun 20 03:05:26 2013
From: root at python.org (Cron Daemon)
Date: Thu, 20 Jun 2013 03:05:26 +0200
Subject: [Python-checkins] Cron <docs@dinsdale> /home/docs/build-devguide
Message-ID: <E1UpTJW-0003GK-3I@dinsdale.python.org>

abort: error: Connection timed out

From python-checkins at python.org  Thu Jun 20 05:20:33 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Thu, 20 Jun 2013 05:20:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Add_-b_and_-X_?=
 =?utf-8?q?options_to_python_man_page=2E?=
Message-ID: <3bbSw10tm2z7Ljf@mail.python.org>

http://hg.python.org/cpython/rev/dfead0696a71
changeset:   84224:dfead0696a71
branch:      3.3
parent:      84221:0113247f894b
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Wed Jun 19 22:19:46 2013 -0500
summary:
  Add -b and -X options to python man page.
Patch contributed by Corey Brune.

files:
  Misc/python.man |  22 ++++++++++++++++++----
  1 files changed, 18 insertions(+), 4 deletions(-)


diff --git a/Misc/python.man b/Misc/python.man
--- a/Misc/python.man
+++ b/Misc/python.man
@@ -11,6 +11,9 @@
 .B \-B
 ]
 [
+.B \-b
+]
+[
 .B \-d
 ]
 [
@@ -23,14 +26,14 @@
 .B \-i
 ]
 [
-.B \-m 
+.B \-m
 .I module-name
 ]
-[
-.B \-q
-]
 .br
        [
+.B \-q
+]
+[
 .B \-O
 ]
 [
@@ -60,6 +63,10 @@
 .B \-x
 ]
 [
+[
+.B \-X
+.I option
+]
 .B \-?
 ]
 .br
@@ -105,6 +112,10 @@
 .I .py[co]
 files on import. See also PYTHONDONTWRITEBYTECODE.
 .TP
+.B \-b
+Issue warnings about str(bytes_instance), str(bytearray_instance)
+and comparing bytes/bytearray with str. (-bb: issue errors)
+.TP
 .BI "\-c " command
 Specify the command to execute (see next section).
 This terminates the option list (following options are passed as
@@ -243,6 +254,9 @@
 field matches the line number, where zero matches all line numbers and
 is thus equivalent to an omitted line number.
 .TP
+.BI "\-X " option
+Set implementation specific option.
+.TP
 .B \-x
 Skip the first line of the source.  This is intended for a DOS
 specific hack only.  Warning: the line numbers in error messages will

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 20 05:20:34 2013
From: python-checkins at python.org (senthil.kumaran)
Date: Thu, 20 Jun 2013 05:20:34 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_merge_from_3=2E3?=
Message-ID: <3bbSw22qP3z7LkL@mail.python.org>

http://hg.python.org/cpython/rev/e26b00adb7ba
changeset:   84225:e26b00adb7ba
parent:      84222:dcb4985d82bc
parent:      84224:dfead0696a71
user:        Senthil Kumaran <senthil at uthcode.com>
date:        Wed Jun 19 22:20:26 2013 -0500
summary:
  merge from 3.3

Add -b and -X options to python man page.
Patch contributed by Corey Brune.

files:
  Misc/python.man |  22 ++++++++++++++++++----
  1 files changed, 18 insertions(+), 4 deletions(-)


diff --git a/Misc/python.man b/Misc/python.man
--- a/Misc/python.man
+++ b/Misc/python.man
@@ -11,6 +11,9 @@
 .B \-B
 ]
 [
+.B \-b
+]
+[
 .B \-d
 ]
 [
@@ -23,14 +26,14 @@
 .B \-i
 ]
 [
-.B \-m 
+.B \-m
 .I module-name
 ]
-[
-.B \-q
-]
 .br
        [
+.B \-q
+]
+[
 .B \-O
 ]
 [
@@ -60,6 +63,10 @@
 .B \-x
 ]
 [
+[
+.B \-X
+.I option
+]
 .B \-?
 ]
 .br
@@ -105,6 +112,10 @@
 .I .py[co]
 files on import. See also PYTHONDONTWRITEBYTECODE.
 .TP
+.B \-b
+Issue warnings about str(bytes_instance), str(bytearray_instance)
+and comparing bytes/bytearray with str. (-bb: issue errors)
+.TP
 .BI "\-c " command
 Specify the command to execute (see next section).
 This terminates the option list (following options are passed as
@@ -243,6 +254,9 @@
 field matches the line number, where zero matches all line numbers and
 is thus equivalent to an omitted line number.
 .TP
+.BI "\-X " option
+Set implementation specific option.
+.TP
 .B \-x
 Skip the first line of the source.  This is intended for a DOS
 specific hack only.  Warning: the line numbers in error messages will

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Thu Jun 20 05:44:00 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 20 Jun 2013 05:44:00 +0200
Subject: [Python-checkins] Daily reference leaks (dcb4985d82bc): sum=0
Message-ID: <E1UpVmy-0006yW-Do@ap.vmr.nerim.net>

results for dcb4985d82bc on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog5JkVFX', '-x']

From python-checkins at python.org  Thu Jun 20 11:24:41 2013
From: python-checkins at python.org (nick.coghlan)
Date: Thu, 20 Jun 2013 11:24:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Update_post_dates_for_426/440?=
Message-ID: <3bbd094X1lz7Ljs@mail.python.org>

http://hg.python.org/peps/rev/a06fa086ef45
changeset:   4949:a06fa086ef45
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Thu Jun 20 19:24:19 2013 +1000
summary:
  Update post dates for 426/440

files:
  pep-0426.txt |  3 ++-
  pep-0440.txt |  2 +-
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -12,7 +12,8 @@
 Content-Type: text/x-rst
 Requires: 440
 Created: 30 Aug 2012
-Post-History: 14 Nov 2012, 5 Feb 2013, 7 Feb 2013, 9 Feb 2013, 27-May-2013
+Post-History: 14 Nov 2012, 5 Feb 2013, 7 Feb 2013, 9 Feb 2013,
+              27 May 2013, 20 Jun 2013
 Replaces: 345
 
 
diff --git a/pep-0440.txt b/pep-0440.txt
--- a/pep-0440.txt
+++ b/pep-0440.txt
@@ -9,7 +9,7 @@
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 18 Mar 2013
-Post-History: 30 Mar 2013, 27-May-2013
+Post-History: 30 Mar 2013, 27 May 2013, 20 Jun 2013
 Replaces: 386
 
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Thu Jun 20 13:21:22 2013
From: python-checkins at python.org (victor.stinner)
Date: Thu, 20 Jun 2013 13:21:22 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A?=
Message-ID: <3bbgZp13z5z7LjQ@mail.python.org>

http://hg.python.org/peps/rev/a17ebebe52ca
changeset:   4950:a17ebebe52ca
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Jun 20 13:20:58 2013 +0200
summary:
  PEP 445:

 * add PyMemAllocatorDomain enum: PYALLOC_PYMEM_RAW, PYALLOC_PYMEM or
   PYALLOC_PYOBJECT
 * rename:

   - PyMemBlockAllocator structure => PyMemAllocator
   - PyMem_GetMappingAllocator() => PyObject_GetArenaAllocator()
   - PyMemMappingAllocator structure => PyObjectArenaAllocator
   - PyMem_SetMappingAllocator() => PyObject_SetArenaAllocator()

 * group get/set functions to only keep 2 functions:
   PyMem_GetAllocator() and PyMem_SetAllocator()
 * PyMem_RawMalloc(0) now calls malloc(1) to have a well defined behaviour
 * PYALLOC_PYMEM_RAW and PYALLOC_PYMEM are now using exactly the same allocator
 * Add more references for external libraries

files:
  pep-0445.txt |  301 ++++++++++++++++++++------------------
  1 files changed, 157 insertions(+), 144 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -40,18 +40,20 @@
 Proposal
 ========
 
-API changes
------------
+New functions and new structure
+-------------------------------
 
-* Add new GIL-free (no need to hold the GIL) memory allocator functions:
+* Add a new GIL-free (no need to hold the GIL) memory allocator:
 
   - ``void* PyMem_RawMalloc(size_t size)``
   - ``void* PyMem_RawRealloc(void *ptr, size_t new_size)``
   - ``void PyMem_RawFree(void *ptr)``
-  - the behaviour of requesting zero bytes is not defined: return *NULL*
-    or a distinct non-*NULL* pointer depending on the platform.
+  - The newly allocated memory will not have been initialized in any
+    way.
+  - Requesting zero bytes returns a distinct non-*NULL* pointer if
+    possible, as if ``PyMem_Malloc(1)`` had been called instead.
 
-* Add a new ``PyMemBlockAllocator`` structure::
+* Add a new ``PyMemAllocator`` structure::
 
     typedef struct {
         /* user context passed as the first argument
@@ -66,69 +68,70 @@
 
         /* release a memory block */
         void (*free) (void *ctx, void *ptr);
-    } PyMemBlockAllocator;
+    } PyMemAllocator;
 
-* Add new functions to get and set internal functions of
-  ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()`` and ``PyMem_RawFree()``:
+* Add a new ``PyMemAllocatorDomain`` enum to choose the Python
+  allocator domain. Domains:
 
-  - ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
-  - ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
-  - default allocator: ``malloc()``, ``realloc()``, ``free()``
+  - ``PYALLOC_PYMEM_RAW``: ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``
+    and ``PyMem_RawRealloc()``
 
-* Add new functions to get and set internal functions of
-  ``PyMem_Malloc()``, ``PyMem_Realloc()`` and ``PyMem_Free()``:
+  - ``PYALLOC_PYMEM``: ``PyMem_Malloc()``, ``PyMem_Realloc()`` and
+    ``PyMem_Realloc()``
 
-  - ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
-  - ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
-  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return
-    *NULL*: it would be treated as an error.
-  - default allocator: ``malloc()``, ``realloc()``, ``free()``;
-    ``PyMem_Malloc(0)`` calls ``malloc(1)``
-    and ``PyMem_Realloc(NULL, 0)`` calls ``realloc(NULL, 1)``
+  - ``PYALLOC_PYOBJECT``: ``PyObject_Malloc()``, ``PyObject_Realloc()``
+    and ``PyObject_Realloc()``
 
-* Add new functions to get and set internal functions of
-  ``PyObject_Malloc()``, ``PyObject_Realloc()`` and
-  ``PyObject_Free()``:
+* Add new functions to get and set memory allocators:
 
-  - ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
-  - ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
-  - ``malloc(ctx, 0)`` and ``realloc(ctx, ptr, 0)`` must not return
-    *NULL*: it would be treated as an error.
-  - default allocator: the *pymalloc* allocator
+  - ``void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
+  - ``void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
+  - The new allocator must return a distinct non-*NULL* pointer when
+    requesting zero bytes
 
-* Add a new ``PyMemMappingAllocator`` structure::
+* Add a new ``PyObjectArenaAllocator`` structure::
 
     typedef struct {
         /* user context passed as the first argument
            to the 2 functions */
         void *ctx;
 
-        /* allocate a memory mapping */
+        /* allocate an arena */
         void* (*alloc) (void *ctx, size_t size);
 
-        /* release a memory mapping */
+        /* release an arena */
         void (*free) (void *ctx, void *ptr, size_t size);
-    } PyMemMappingAllocator;
+    } PyObjectArenaAllocator;
 
-* Add a new function to get and set the memory mapping allocator:
+* Add new functions to get and set the arena allocator used by
+  *pymalloc*:
 
-  - ``void PyMem_GetMappingAllocator(PyMemMappingAllocator *allocator)``
-  - ``void PyMem_SetMappingAllocator(PyMemMappingAllocator *allocator)``
-  - Currently, this allocator is only used internally by *pymalloc* to
-    allocate arenas.
+  - ``void PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)``
+  - ``void PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator)``
 
-* Add a new function to setup the builtin Python debug hooks when memory
-  allocators are replaced:
+* Add a new function to setup the builtin Python debug hooks when a
+  memory allocator is replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
-  - the function does nothing is Python is compiled not compiled in
-    debug mode
+  - the function does nothing is Python is not compiled in debug mode
 
-* The following memory allocators always returns *NULL* if size is
-  greater than ``PY_SSIZE_T_MAX`` (check before calling the internal
-  function): ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``,
-  ``PyMem_Malloc()``, ``PyMem_Realloc()``, ``PyObject_Malloc()``,
-  ``PyObject_Realloc()``.
+* Memory allocators always returns *NULL* if size is greater than
+  ``PY_SSIZE_T_MAX``. The check is done before calling the
+  inner function.
+
+The *pymalloc* allocator is optimized for objects smaller than 512 bytes
+with a short lifetime. It uses memory mappings with a fixed size of 256
+KB called "arenas".
+
+Default allocators:
+
+* ``PYALLOC_PYMEM_RAW``, ``PYALLOC_PYMEM``: ``malloc()``,
+  ``realloc()``, ``free()`` (and *ctx* is NULL); call ``malloc(1)`` when
+  requesting zero bytes
+* ``PYALLOC_PYOBJECT``: *pymalloc* allocator which fall backs on
+  ``PyMem_Malloc()`` for allocations larger than 512 bytes
+* *pymalloc* arena allocator: ``mmap()``, ``munmap()`` (and *ctx* is
+  NULL), or ``malloc()`` and ``free()`` if ``mmap()`` is not available
 
 The builtin Python debug hooks were introduced in Python 2.3 and
 implement the following checks:
@@ -141,23 +144,34 @@
 * Detect write after the end of the buffer (buffer overflow)
 
 
-Other changes
--------------
+Don't call malloc() directly anymore
+------------------------------------
 
-* ``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()``
-  and ``realloc()``, instead of calling ``PyObject_Malloc()`` and
-  ``PyObject_Realloc()`` in debug mode
+``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()`` and
+``realloc()``, instead of calling ``PyObject_Malloc()`` and
+``PyObject_Realloc()`` in debug mode.
 
-* ``PyObject_Malloc()`` falls back on ``PyMem_Malloc()`` instead of
-  ``malloc()`` if size is greater or equal than
-  ``SMALL_REQUEST_THRESHOLD`` (512 bytes), and ``PyObject_Realloc()``
-  falls back on ``PyMem_Realloc()`` instead of ``realloc()``
+``PyObject_Malloc()`` falls back on ``PyMem_Malloc()`` instead of
+``malloc()`` if size is greater or equal than 512 bytes, and
+``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of
+``realloc()``
 
-* Replace direct calls to ``malloc()`` with ``PyMem_Malloc()``, or
-  ``PyMem_RawMalloc()`` if the GIL is not held
+Replace direct calls to ``malloc()`` with ``PyMem_Malloc()``, or
+``PyMem_RawMalloc()`` if the GIL is not held.
 
-* Configure external libraries like zlib or OpenSSL to allocate memory
-  using ``PyMem_RawMalloc()``
+Configure external libraries like zlib or OpenSSL to allocate memory
+using ``PyMem_Malloc()`` or ``PyMem_RawMalloc()``. If the allocator of a
+library can only be replaced globally, the allocator is not replaced if
+Python is embedded in an application.
+
+For the "track memory usage" use case, it is important to track memory
+allocated in external libraries to have accurate reports, because these
+allocations may be large.
+
+If an hook is used to the track memory usage, the memory allocated by
+``malloc()`` will not be tracked. Remaining ``malloc()`` in external
+libraries like OpenSSL or bz2 may allocate large memory blocks and so
+would be missed in memory usage reports.
 
 
 Examples
@@ -171,8 +185,8 @@
 
     #include <stdlib.h>
 
-    int block_padding = 2;
-    int mapping_padding = 10;
+    int alloc_padding = 2;
+    int arena_padding = 10;
 
     void* my_malloc(void *ctx, size_t size)
     {
@@ -191,49 +205,49 @@
         free(ptr);
     }
 
-    void* my_alloc_mapping(void *ctx, size_t size)
+    void* my_alloc_arena(void *ctx, size_t size)
     {
         int padding = *(int *)ctx;
         return malloc(size + padding);
     }
 
-    void my_free_mapping(void *ctx, void *ptr, size_t size)
+    void my_free_arena(void *ctx, void *ptr, size_t size)
     {
         free(ptr);
     }
 
     void setup_custom_allocator(void)
     {
-        PyMemBlockAllocator block;
-        PyMemMappingAllocator mapping;
+        PyMemAllocator alloc;
+        PyObjectArenaAllocator arena;
 
-        block.ctx = &block_padding;
-        block.malloc = my_malloc;
-        block.realloc = my_realloc;
-        block.free = my_free;
+        alloc.ctx = &alloc_padding;
+        alloc.malloc = my_malloc;
+        alloc.realloc = my_realloc;
+        alloc.free = my_free;
 
-        PyMem_SetRawAllocator(&block);
-        PyMem_SetAllocator(&block);
+        PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc);
+        PyMem_SetAllocator(PYALLOC_PYMEM, &alloc);
 
-        mapping.ctx = &mapping_padding;
-        mapping.alloc = my_alloc_mapping;
-        mapping.free = my_free_mapping;
-        PyMem_SetMappingAllocator(mapping);
+        arena.ctx = &arena_padding;
+        arena.alloc = my_alloc_arena;
+        arena.free = my_free_arena;
+        PyObject_SetArenaAllocator(&arena);
 
         PyMem_SetupDebugHooks();
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new
-   allocator are not thread-safe.
+   Remove the call ``PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc)`` if
+   the new allocator is not thread-safe.
 
 
 Use case 2: Replace Memory Allocator, override pymalloc
 --------------------------------------------------------
 
-If your allocator is optimized for allocation of small objects (less
-than 512 bytes) with a short lifetime, pymalloc can be overriden
-(replace ``PyObject_Malloc()``).
+If your allocator is optimized for allocations of objects smaller than
+512 bytes with a short lifetime, pymalloc can be overriden (replace
+``PyObject_Malloc()``).
 
 Dummy example wasting 2 bytes per memory block::
 
@@ -260,22 +274,22 @@
 
     void setup_custom_allocator(void)
     {
-        PyMemBlockAllocator alloc;
+        PyMemAllocator alloc;
         alloc.ctx = &padding;
         alloc.malloc = my_malloc;
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
-        PyMem_SetRawAllocator(&alloc);
-        PyMem_SetAllocator(&alloc);
-        PyObject_SetAllocator(&alloc);
+        PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc);
+        PyMem_SetAllocator(PYALLOC_PYMEM, &alloc);
+        PyMem_SetAllocator(PYALLOC_PYOBJECT, &alloc);
 
         PyMem_SetupDebugHooks();
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if the new
-   allocator are not thread-safe.
+   Remove the call ``PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc)`` if
+   the new allocator is not thread-safe.
 
 
 
@@ -285,15 +299,15 @@
 Example to setup hooks on all memory allocators::
 
     struct {
-        PyMemBlockAllocator raw;
-        PyMemBlockAllocator mem;
-        PyMemBlockAllocator obj;
+        PyMemAllocator raw;
+        PyMemAllocator mem;
+        PyMemAllocator obj;
         /* ... */
     } hook;
 
     static void* hook_malloc(void *ctx, size_t size)
     {
-        PyMemBlockAllocator *alloc = (PyMemBlockAllocator *)ctx;
+        PyMemAllocator *alloc = (PyMemAllocator *)ctx;
         /* ... */
         ptr = alloc->malloc(alloc->ctx, size);
         /* ... */
@@ -302,7 +316,7 @@
 
     static void* hook_realloc(void *ctx, void *ptr, size_t new_size)
     {
-        PyMemBlockAllocator *alloc = (PyMemBlockAllocator *)ctx;
+        PyMemAllocator *alloc = (PyMemAllocator *)ctx;
         void *ptr2;
         /* ... */
         ptr2 = alloc->realloc(alloc->ctx, ptr, new_size);
@@ -312,7 +326,7 @@
 
     static void hook_free(void *ctx, void *ptr)
     {
-        PyMemBlockAllocator *alloc = (PyMemBlockAllocator *)ctx;
+        PyMemAllocator *alloc = (PyMemAllocator *)ctx;
         /* ... */
         alloc->free(alloc->ctx, ptr);
         /* ... */
@@ -320,7 +334,7 @@
 
     void setup_hooks(void)
     {
-        PyMemBlockAllocator alloc;
+        PyMemAllocator alloc;
         static int installed = 0;
 
         if (installed)
@@ -330,27 +344,28 @@
         alloc.malloc = hook_malloc;
         alloc.realloc = hook_realloc;
         alloc.free = hook_free;
+        PyMem_GetAllocator(PYALLOC_PYMEM_RAW, &hook.raw);
+        PyMem_GetAllocator(PYALLOC_PYMEM, &hook.mem);
+        PyMem_GetAllocator(PYALLOC_PYOBJECT, &hook.obj);
 
-        PyMem_GetRawAllocator(&hook.raw);
         alloc.ctx = &hook.raw;
-        PyMem_SetRawAllocator(&alloc);
+        PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc);
 
-        PyMem_GetAllocator(&hook.mem);
         alloc.ctx = &hook.mem;
-        PyMem_SetAllocator(&alloc);
+        PyMem_SetAllocator(PYALLOC_PYMEM, &alloc);
 
-        PyObject_GetAllocator(&hook.obj);
         alloc.ctx = &hook.obj;
-        PyObject_SetAllocator(&alloc);
+        PyMem_SetAllocator(PYALLOC_PYOBJECT, &alloc);
     }
 
 .. warning::
-   Remove the call ``PyMem_SetRawAllocator(&alloc)`` if hooks are not
-   thread-safe.
+   Remove the call ``PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc)`` if
+   hooks are not thread-safe.
 
 .. note::
-   ``PyMem_SetupDebugHooks()`` does not need to be called: Python debug
-   hooks are installed automatically at startup.
+   ``PyMem_SetupDebugHooks()`` does not need to be called because the
+   allocator is not replaced: Python debug hooks are installed
+   automatically at startup.
 
 
 Performances
@@ -369,32 +384,22 @@
 Alternatives
 ============
 
-Only one get/set function for block allocators
-----------------------------------------------
+More specific functions to get/set memory allocators
+----------------------------------------------------
 
-Replace the 6 functions:
+Replace the 2 functions:
 
-* ``void PyMem_GetRawAllocator(PyMemBlockAllocator *allocator)``
-* ``void PyMem_GetAllocator(PyMemBlockAllocator *allocator)``
-* ``void PyObject_GetAllocator(PyMemBlockAllocator *allocator)``
-* ``void PyMem_SetRawAllocator(PyMemBlockAllocator *allocator)``
-* ``void PyMem_SetAllocator(PyMemBlockAllocator *allocator)``
-* ``void PyObject_SetAllocator(PyMemBlockAllocator *allocator)``
+* ``void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
+* ``void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
 
-with 2 functions with an additional *domain* argument:
+with:
 
-* ``int PyMem_GetBlockAllocator(int domain, PyMemBlockAllocator *allocator)``
-* ``int PyMem_SetBlockAllocator(int domain, PyMemBlockAllocator *allocator)``
-
-These functions return 0 on success, or -1 if the domain is unknown.
-
-where domain is one of these values:
-
-* ``PYALLOC_PYMEM``
-* ``PYALLOC_PYMEM_RAW``
-* ``PYALLOC_PYOBJECT``
-
-Drawback: the caller has to check if the result is 0, or handle the error.
+* ``void PyMem_GetRawAllocator(PyMemAllocator *allocator)``
+* ``void PyMem_GetAllocator(PyMemAllocator *allocator)``
+* ``void PyObject_GetAllocator(PyMemAllocator *allocator)``
+* ``void PyMem_SetRawAllocator(PyMemAllocator *allocator)``
+* ``void PyMem_SetAllocator(PyMemAllocator *allocator)``
+* ``void PyObject_SetAllocator(PyMemAllocator *allocator)``
 
 
 Make PyMem_Malloc() reuse PyMem_RawMalloc() by default
@@ -404,12 +409,6 @@
 calling ``PyMem_SetRawAllocator()`` would also also patch
 ``PyMem_Malloc()`` indirectly.
 
-.. note::
-
-   In the implementation of this PEP (issue #3329),
-   ``PyMem_RawMalloc(0)`` calls ``malloc(0)``,
-   whereas ``PyMem_Malloc(0)`` calls ``malloc(1)``.
-
 
 Add a new PYDEBUGMALLOC environment variable
 --------------------------------------------
@@ -445,7 +444,7 @@
 to get the C filename and line number of a memory allocation.
 
 Example of ``PyMem_Malloc`` macro with the modified
-``PyMemBlockAllocator`` structure::
+``PyMemAllocator`` structure::
 
     typedef struct {
         /* user context passed as the first argument
@@ -463,7 +462,7 @@
         /* release a memory block */
         void (*free) (void *ctx, const char *filename, int lineno,
                       void *ptr);
-    } PyMemBlockAllocator;
+    } PyMemAllocator;
 
     void* _PyMem_MallocTrace(const char *filename, int lineno,
                              size_t size);
@@ -485,12 +484,12 @@
 GIL-free PyMem_Malloc()
 -----------------------
 
-When Python is compiled in debug mode, ``PyMem_Malloc()`` calls
-indirectly ``PyObject_Malloc()`` which requires the GIL to be held.
-That's why ``PyMem_Malloc()`` must be called with the GIL held.
+In Python 3.3, when Python is compiled in debug mode, ``PyMem_Malloc()``
+calls indirectly ``PyObject_Malloc()`` which requires the GIL to be
+held.  That's why ``PyMem_Malloc()`` must be called with the GIL held.
 
-This PEP proposes to "fix" ``PyMem_Malloc()`` to make it always call
-``malloc()``.  So the "GIL must be held" restriction may be removed from
+This PEP proposes changes ``PyMem_Malloc()``: it now always call
+``malloc()``.  The "GIL must be held" restriction can be removed from
 ``PyMem_Malloc()``.
 
 Allowing to call ``PyMem_Malloc()`` without holding the GIL might break
@@ -516,9 +515,10 @@
 case, ``PyMem_Malloc()`` should be replaced with ``malloc()`` (or
 ``PyMem_RawMalloc()``).
 
-If an hook is used to the track memory usage, the ``malloc()`` memory
-will not be seen. Remaining ``malloc()`` may allocate a lot of memory
-and so would be missed in reports.
+If an hook is used to the track memory usage, the memory allocated by
+direct calls to ``malloc()`` will not be tracked. External libraries
+like OpenSSL or bz2 should not call ``malloc()`` directly, so large
+allocated will be included in memory usage reports.
 
 
 Use existing debug tools to analyze the memory
@@ -545,8 +545,7 @@
 Add msize()
 -----------
 
-Add another field to ``PyMemBlockAllocator`` and
-``PyMemMappingAllocator``::
+Add another field to ``PyMemAllocator`` and ``PyObjectArenaAllocator``::
 
     size_t msize(void *ptr);
 
@@ -574,6 +573,8 @@
 depending on the allocator. The context is a convenient way to reuse the
 same custom allocator or hook for different Python allocators.
 
+In C++, the context can be used to pass *this*.
+
 
 External libraries
 ==================
@@ -589,6 +590,15 @@
 * expat: `parserCreate()
   <http://hg.python.org/cpython/file/cc27d50bd91a/Modules/expat/xmlparse.c#l724>`_
   has a per-instance memory handler
+* zlib: `zlib 1.2.8 Manual <http://www.zlib.net/manual.html#Usage>`_,
+  pass an opaque pointer
+* bz2: `bzip2 and libbzip2, version 1.0.5
+  <http://www.bzip.org/1.0.5/bzip2-manual-1.0.5.html>`_,
+  pass an opaque pointer
+* lzma: `LZMA SDK - How to Use
+  <http://www.asawicki.info/news_1368_lzma_sdk_-_how_to_use.html>`_,
+  pass an opaque pointer
+* lipmpdec doesn't have this extra *ctx* parameter
 
 Other libraries:
 
@@ -596,6 +606,9 @@
   <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
 * libxml2: `xmlGcMemSetup() <http://xmlsoft.org/html/libxml-xmlmemory.html>`_,
   global
+* Oracle's OCI: `Oracle Call Interface Programmer's Guide,
+  Release 2 (9.2)
+  <http://docs.oracle.com/cd/B10501_01/appdev.920/a96584/oci15re4.htm>`_
 
 See also the `GNU libc: Memory Allocation Hooks
 <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_.

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Thu Jun 20 13:45:46 2013
From: python-checkins at python.org (nick.coghlan)
Date: Thu, 20 Jun 2013 13:45:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Tweak_PEP_426_install_hooks?=
Message-ID: <3bbh6y2fHsz7Lky@mail.python.org>

http://hg.python.org/peps/rev/535f09e95d89
changeset:   4951:535f09e95d89
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Thu Jun 20 21:44:51 2013 +1000
summary:
  Tweak PEP 426 install hooks

files:
  pep-0426.txt |  30 +++++++++++++++++++++++++++---
  1 files changed, 27 insertions(+), 3 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -778,9 +778,6 @@
 * ``license``: a file with the full text of the distribution's license
 * ``changelog``: a file describing changes made to the distribution
 
-If this field is provided at all, it MUST name at least one included
-document.
-
 Supporting documents MUST be included directly in the ``dist-info``
 directory. Directory separators are NOT permitted in document names.
 
@@ -1566,6 +1563,13 @@
 that the ``setup.py`` script will already invoke any necessary
 post-installation behaviour.
 
+Installation tools SHOULD treat an exception thrown by a postinstall hook
+as a failure of the installation and revert any other changes made to the
+system.
+
+Installation tools SHOULD treat an exception thrown by a preuninstall hook
+as an indication the removal of the distribution should be aborted.
+
 Installation tools MUST NOT silently ignore install hooks, as failing
 to call these hooks may result in a misconfigured installation that fails
 unexpectedly at runtime. Installation tools MAY refuse to install
@@ -2225,6 +2229,26 @@
 (at the earliest).
 
 
+Additional install hooks
+------------------------
+
+In addition to the postinstall and preuninstall hooks described in the PEP,
+other distribution systems (like RPM) include the notion of preinstall
+and postuninstall hooks. These hooks would run with the runtime dependencies
+installed, but without the distribution itself. These have been deliberately
+omitted, as they're well suited to being explored further as metadata
+extensions.
+
+Similarly, the idea of "optional" postinstall and preuninstall hooks can
+be pursued as a metadata extension.
+
+By contrast, the mandatory postinstall and preuninstall hooks have been
+included directly in the PEP, specifically to ensure installation tools
+don't silently ignore them. This ensures users will either be able to
+install such distributions, or else receive an explicit error at installation
+time.
+
+
 Metabuild system
 ----------------
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Thu Jun 20 13:51:39 2013
From: python-checkins at python.org (nick.coghlan)
Date: Thu, 20 Jun 2013 13:51:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Add_jsonschema_ref_to_PEP_426?=
Message-ID: <3bbhFl0CFcz7Lks@mail.python.org>

http://hg.python.org/peps/rev/aab6742e22bf
changeset:   4952:aab6742e22bf
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Thu Jun 20 21:51:16 2013 +1000
summary:
  Add jsonschema ref to PEP 426

files:
  pep-0426.txt |  6 +++++-
  1 files changed, 5 insertions(+), 1 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -451,7 +451,11 @@
 -------------------
 
 A `jsonschema <https://pypi.python.org/pypi/jsonschema>`__ description of
-the defined metadata schema will be provided prior to PEP acceptance.
+the distribution metadata is `available
+<http://hg.python.org/peps/file/default/pep-0426/pymeta-schema.json>`__.
+
+This schema does NOT currently handle validation of some of the more complex
+string fields (instead treating them as opaque strings).
 
 Except where otherwise noted, all URL fields in the metadata MUST comply
 with RFC 3986.

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Thu Jun 20 14:42:41 2013
From: python-checkins at python.org (nick.coghlan)
Date: Thu, 20 Jun 2013 14:42:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_426_-_add_missing_depende?=
	=?utf-8?q?ncy_fields?=
Message-ID: <3bbjNd0PY8zSm2@mail.python.org>

http://hg.python.org/peps/rev/bdb4e9024f45
changeset:   4953:bdb4e9024f45
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Thu Jun 20 22:42:23 2013 +1000
summary:
  PEP 426 - add missing dependency fields

files:
  pep-0426.txt |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -422,6 +422,8 @@
 * ``meta_may_require``
 * ``run_requires``
 * ``run_may_require``
+* ``test_requires``
+* ``test_may_require``
 * ``build_requires``
 * ``build_may_require``
 * ``dev_requires``

-- 
Repository URL: http://hg.python.org/peps

From brett at python.org  Thu Jun 20 15:36:07 2013
From: brett at python.org (Brett Cannon)
Date: Thu, 20 Jun 2013 09:36:07 -0400
Subject: [Python-checkins] cpython (3.3): Add -b and -X options to
 python man page.
In-Reply-To: <3bbSw10tm2z7Ljf@mail.python.org>
References: <3bbSw10tm2z7Ljf@mail.python.org>
Message-ID: <CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>

On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran <
python-checkins at python.org> wrote:

> http://hg.python.org/cpython/rev/dfead0696a71
> changeset:   84224:dfead0696a71
> branch:      3.3
> parent:      84221:0113247f894b
> user:        Senthil Kumaran <senthil at uthcode.com>
> date:        Wed Jun 19 22:19:46 2013 -0500
> summary:
>   Add -b and -X options to python man page.
> Patch contributed by Corey Brune.
>
> files:
>   Misc/python.man |  22 ++++++++++++++++++----
>   1 files changed, 18 insertions(+), 4 deletions(-)
>
>
> diff --git a/Misc/python.man b/Misc/python.man
> --- a/Misc/python.man
> +++ b/Misc/python.man
> @@ -11,6 +11,9 @@
>  .B \-B
>  ]
>  [
> +.B \-b
> +]
> +[
>  .B \-d
>  ]
>  [
> @@ -23,14 +26,14 @@
>  .B \-i
>  ]
>  [
> -.B \-m
> +.B \-m
>  .I module-name
>  ]
> -[
> -.B \-q
> -]
>  .br
>         [
> +.B \-q
> +]
> +[
>  .B \-O
>  ]
>  [
> @@ -60,6 +63,10 @@
>  .B \-x
>  ]
>  [
> +[
> +.B \-X
> +.I option
> +]
>  .B \-?
>  ]
>  .br
> @@ -105,6 +112,10 @@
>  .I .py[co]
>  files on import. See also PYTHONDONTWRITEBYTECODE.
>  .TP
> +.B \-b
> +Issue warnings about str(bytes_instance), str(bytearray_instance)
> +and comparing bytes/bytearray with str. (-bb: issue errors)
> +.TP
>  .BI "\-c " command
>  Specify the command to execute (see next section).
>  This terminates the option list (following options are passed as
> @@ -243,6 +254,9 @@
>  field matches the line number, where zero matches all line numbers and
>  is thus equivalent to an omitted line number.
>  .TP
> +.BI "\-X " option
> +Set implementation specific option.
>

Should probably be "Set the implementation-specific option."


> +.TP
>  .B \-x
>  Skip the first line of the source.  This is intended for a DOS
>  specific hack only.  Warning: the line numbers in error messages will
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130620/f12fa034/attachment.html>

From python-checkins at python.org  Thu Jun 20 15:46:28 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Thu, 20 Jun 2013 15:46:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzQxNTM6IHVwZGF0?=
 =?utf-8?q?e_Unicode_howto_for_Python_3=2E3?=
Message-ID: <3bbkpD1M0Jz7Lkc@mail.python.org>

http://hg.python.org/cpython/rev/1dbbed06a163
changeset:   84226:1dbbed06a163
branch:      3.3
parent:      84224:dfead0696a71
user:        Andrew Kuchling <amk at amk.ca>
date:        Thu Jun 20 09:29:09 2013 -0400
summary:
  #4153: update Unicode howto for Python 3.3

* state that python3 source encoding is UTF-8, and give examples

* mention surrogateescape in the 'tips and tricks' section, and
  backslashreplace in the "Python's Unicode Support" section.

* Describe Unicode support provided by the re module.

* link to Nick Coghlan's and Ned Batchelder's notes/presentations.

* default filesystem encoding is now UTF-8, not ascii.

* Describe StreamRecoder class.

* update acks section

* remove usage of "I think", "I'm not going to", etc.

* various edits

* remove revision history and original outline

files:
  Doc/howto/unicode.rst |  255 ++++++++++++++++++-----------
  1 files changed, 161 insertions(+), 94 deletions(-)


diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst
--- a/Doc/howto/unicode.rst
+++ b/Doc/howto/unicode.rst
@@ -28,15 +28,15 @@
 as 'na?ve' and 'caf?', and some publications have house styles which require
 spellings such as 'co?perate'.)
 
-For a while people just wrote programs that didn't display accents.  I remember
-looking at Apple ][ BASIC programs, published in French-language publications in
-the mid-1980s, that had lines like these::
+For a while people just wrote programs that didn't display accents.
+In the mid-1980s an Apple II BASIC program written by a French speaker
+might have lines like these::
 
    PRINT "FICHIER EST COMPLETE."
    PRINT "CARACTERE NON ACCEPTE."
 
-Those messages should contain accents, and they just look wrong to someone who
-can read French.
+Those messages should contain accents (complet?, caract?re, accept?),
+and they just look wrong to someone who can read French.
 
 In the 1980s, almost all personal computers were 8-bit, meaning that bytes could
 hold values ranging from 0 to 255.  ASCII codes only went up to 127, so some
@@ -69,9 +69,12 @@
 originally separate efforts, but the specifications were merged with the 1.1
 revision of Unicode.
 
-(This discussion of Unicode's history is highly simplified.  I don't think the
-average Python programmer needs to worry about the historical details; consult
-the Unicode consortium site listed in the References for more information.)
+(This discussion of Unicode's history is highly simplified.  The
+precise historical details aren't necessary for understanding how to
+use Unicode effectively, but if you're curious, consult the Unicode
+consortium site listed in the References or
+the `Wikipedia entry for Unicode <http://en.wikipedia.org/wiki/Unicode#History>`_
+for more information.)
 
 
 Definitions
@@ -216,10 +219,8 @@
 
 Another `good introductory article <http://www.joelonsoftware.com/articles/Unicode.html>`_
 was written by Joel Spolsky.
-If this introduction didn't make things clear to you, you should try reading this
-alternate article before continuing.
-
-.. Jason Orendorff XXX http://www.jorendorff.com/articles/unicode/ is broken
+If this introduction didn't make things clear to you, you should try
+reading this alternate article before continuing.
 
 Wikipedia entries are often helpful; see the entries for "`character encoding
 <http://en.wikipedia.org/wiki/Character_encoding>`_" and `UTF-8
@@ -239,8 +240,31 @@
 characters, meaning any string created using ``"unicode rocks!"``, ``'unicode
 rocks!'``, or the triple-quoted string syntax is stored as Unicode.
 
-To insert a non-ASCII Unicode character, e.g., any letters with
-accents, one can use escape sequences in their string literals as such::
+The default encoding for Python source code is UTF-8, so you can simply
+include a Unicode character in a string literal::
+
+   try:
+       with open('/tmp/input.txt', 'r') as f:
+           ...
+   except IOError:
+       # 'File not found' error message.
+       print("Fichier non trouv?")
+
+You can use a different encoding from UTF-8 by putting a specially-formatted
+comment as the first or second line of the source code::
+
+   # -*- coding: <encoding name> -*-
+
+Side note: Python 3 also supports using Unicode characters in identifiers::
+
+   r?pertoire = "/tmp/records.log"
+   with open(r?pertoire, "w") as f:
+       f.write("test\n")
+
+If you can't enter a particular character in your editor or want to
+keep the source code ASCII-only for some reason, you can also use
+escape sequences in string literals. (Depending on your system,
+you may see the actual capital-delta glyph instead of a \u escape.) ::
 
    >>> "\N{GREEK CAPITAL LETTER DELTA}"  # Using the character name
    '\u0394'
@@ -251,7 +275,7 @@
 
 In addition, one can create a string using the :func:`~bytes.decode` method of
 :class:`bytes`.  This method takes an *encoding* argument, such as ``UTF-8``,
-and optionally, an *errors* argument.
+and optionally an *errors* argument.
 
 The *errors* argument specifies the response when the input string can't be
 converted according to the encoding's rules.  Legal values for this argument are
@@ -295,11 +319,15 @@
 
 The opposite method of :meth:`bytes.decode` is :meth:`str.encode`,
 which returns a :class:`bytes` representation of the Unicode string, encoded in the
-requested *encoding*.  The *errors* parameter is the same as the parameter of
-the :meth:`~bytes.decode` method, with one additional possibility; as well as
-``'strict'``, ``'ignore'``, and ``'replace'`` (which in this case inserts a
-question mark instead of the unencodable character), you can also pass
-``'xmlcharrefreplace'`` which uses XML's character references.
+requested *encoding*.
+
+The *errors* parameter is the same as the parameter of the
+:meth:`~bytes.decode` method but supports a few more possible handlers. As well as
+``'strict'``, ``'ignore'``, and ``'replace'`` (which in this case
+inserts a question mark instead of the unencodable character), there is
+also ``'xmlcharrefreplace'`` (inserts an XML character reference) and
+``backslashreplace`` (inserts a ``\uNNNN`` escape sequence).
+
 The following example shows the different results::
 
     >>> u = chr(40960) + 'abcd' + chr(1972)
@@ -316,16 +344,15 @@
     b'?abcd?'
     >>> u.encode('ascii', 'xmlcharrefreplace')
     b'&#40960;abcd&#1972;'
+    >>> u.encode('ascii', 'backslashreplace')
+    b'\\ua000abcd\\u07b4'
 
-.. XXX mention the surrogate* error handlers
-
-The low-level routines for registering and accessing the available encodings are
-found in the :mod:`codecs` module.  However, the encoding and decoding functions
-returned by this module are usually more low-level than is comfortable, so I'm
-not going to describe the :mod:`codecs` module here.  If you need to implement a
-completely new encoding, you'll need to learn about the :mod:`codecs` module
-interfaces, but implementing encodings is a specialized task that also won't be
-covered here.  Consult the Python documentation to learn more about this module.
+The low-level routines for registering and accessing the available
+encodings are found in the :mod:`codecs` module.  Implementing new
+encodings also requires understanding the :mod:`codecs` module.
+However, the encoding and decoding functions returned by this module
+are usually more low-level than is comfortable, and writing new encodings
+is a specialized task, so the module won't be covered in this HOWTO.
 
 
 Unicode Literals in Python Source Code
@@ -415,12 +442,50 @@
 from the above output, ``'Ll'`` means 'Letter, lowercase', ``'No'`` means
 "Number, other", ``'Mn'`` is "Mark, nonspacing", and ``'So'`` is "Symbol,
 other".  See
-<http://www.unicode.org/reports/tr44/#General_Category_Values> for a
+`the General Category Values section of the Unicode Character Database documentation <http://www.unicode.org/reports/tr44/#General_Category_Values>`_ for a
 list of category codes.
 
+
+Unicode Regular Expressions
+---------------------------
+
+The regular expressions supported by the :mod:`re` module can be provided
+either as bytes or strings.  Some of the special character sequences such as
+``\d`` and ``\w`` have different meanings depending on whether
+the pattern is supplied as bytes or a string.  For example,
+``\d`` will match the characters ``[0-9]`` in bytes but
+in strings will match any character that's in the ``'Nd'`` category.
+
+The string in this example has the number 57 written in both Thai and
+Arabic numerals::
+
+   import re
+   p = re.compile('\d+')
+
+   s = "Over \u0e55\u0e57 57 flavours"
+   m = p.search(s)
+   print(repr(m.group()))
+
+When executed, ``\d+`` will match the Thai numerals and print them
+out.  If you supply the :const:`re.ASCII` flag to
+:func:`~re.compile`, ``\d+`` will match the substring "57" instead.
+
+Similarly, ``\w`` matches a wide variety of Unicode characters but
+only ``[a-zA-Z0-9_]`` in bytes or if :const:`re.ASCII` is supplied,
+and ``\s`` will match either Unicode whitespace characters or
+``[ \t\n\r\f\v]``.
+
+
 References
 ----------
 
+.. comment should these be mentioned earlier, e.g. at the start of the "introduction to Unicode" first section?
+
+Some good alternative discussions of Python's Unicode support are:
+
+* `Processing Text Files in Python 3 <http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html>`_, by Nick Coghlan.
+* `Pragmatic Unicode <http://nedbatchelder.com/text/unipain.html>`_, a PyCon 2012 presentation by Ned Batchelder.
+
 The :class:`str` type is described in the Python library reference at
 :ref:`textseq`.
 
@@ -428,12 +493,10 @@
 
 The documentation for the :mod:`codecs` module.
 
-Marc-Andr? Lemburg gave a presentation at EuroPython 2002 titled "Python and
-Unicode".  A PDF version of his slides is available at
-<http://downloads.egenix.com/python/Unicode-EPC2002-Talk.pdf>, and is an
-excellent overview of the design of Python's Unicode features (based on Python
-2, where the Unicode string type is called ``unicode`` and literals start with
-``u``).
+Marc-Andr? Lemburg gave `a presentation titled "Python and Unicode" (PDF slides) <http://downloads.egenix.com/python/Unicode-EPC2002-Talk.pdf>`_ at
+EuroPython 2002.  The slides are an excellent overview of the design
+of Python 2's Unicode features (where the Unicode string type is
+called ``unicode`` and literals start with ``u``).
 
 
 Reading and Writing Unicode Data
@@ -512,7 +575,7 @@
 Windows, Python uses the name "mbcs" to refer to whatever the currently
 configured encoding is.  On Unix systems, there will only be a filesystem
 encoding if you've set the ``LANG`` or ``LC_CTYPE`` environment variables; if
-you haven't, the default encoding is ASCII.
+you haven't, the default encoding is UTF-8.
 
 The :func:`sys.getfilesystemencoding` function returns the encoding to use on
 your current system, in case you want to do the encoding manually, but there's
@@ -527,13 +590,13 @@
 Functions in the :mod:`os` module such as :func:`os.stat` will also accept Unicode
 filenames.
 
-Function :func:`os.listdir`, which returns filenames, raises an issue: should it return
+The :func:`os.listdir` function returns filenames and raises an issue: should it return
 the Unicode version of filenames, or should it return bytes containing
 the encoded versions?  :func:`os.listdir` will do both, depending on whether you
 provided the directory path as bytes or a Unicode string.  If you pass a
 Unicode string as the path, filenames will be decoded using the filesystem's
 encoding and a list of Unicode strings will be returned, while passing a byte
-path will return the bytes versions of the filenames.  For example,
+path will return the filenames as bytes.  For example,
 assuming the default filesystem encoding is UTF-8, running the following
 program::
 
@@ -548,13 +611,13 @@
 will produce the following output::
 
    amk:~$ python t.py
-   [b'.svn', b'filename\xe4\x94\x80abc', ...]
-   ['.svn', 'filename\u4500abc', ...]
+   [b'filename\xe4\x94\x80abc', ...]
+   ['filename\u4500abc', ...]
 
 The first list contains UTF-8-encoded filenames, and the second list contains
 the Unicode versions.
 
-Note that in most occasions, the Unicode APIs should be used.  The bytes APIs
+Note that on most occasions, the Unicode APIs should be used.  The bytes APIs
 should only be used on systems where undecodable file names can be present,
 i.e. Unix systems.
 
@@ -585,65 +648,69 @@
 clever way to hide malicious text in the encoded bytestream.
 
 
+Converting Between File Encodings
+'''''''''''''''''''''''''''''''''
+
+The :class:`~codecs.StreamRecoder` class can transparently convert between
+encodings, taking a stream that returns data in encoding #1
+and behaving like a stream returning data in encoding #2.
+
+For example, if you have an input file *f* that's in Latin-1, you
+can wrap it with a :class:`StreamRecoder` to return bytes encoded in UTF-8::
+
+    new_f = codecs.StreamRecoder(f,
+        # en/decoder: used by read() to encode its results and
+        # by write() to decode its input.
+        codecs.getencoder('utf-8'), codecs.getdecoder('utf-8'),
+
+        # reader/writer: used to read and write to the stream.
+        codecs.getreader('latin-1'), codecs.getwriter('latin-1') )
+
+
+Files in an Unknown Encoding
+''''''''''''''''''''''''''''
+
+What can you do if you need to make a change to a file, but don't know
+the file's encoding?  If you know the encoding is ASCII-compatible and
+only want to examine or modify the ASCII parts, you can open the file
+with the ``surrogateescape`` error handler::
+
+   with open(fname, 'r', encoding="ascii", errors="surrogateescape") as f:
+       data = f.read()
+
+   # make changes to the string 'data'
+
+   with open(fname + '.new', 'w',
+              encoding="ascii", errors="surrogateescape") as f:
+       f.write(data)
+
+The ``surrogateescape`` error handler will decode any non-ASCII bytes
+as code points in the Unicode Private Use Area ranging from U+DC80 to
+U+DCFF.  These private code points will then be turned back into the
+same bytes when the ``surrogateescape`` error handler is used when
+encoding the data and writing it back out.
+
+
 References
 ----------
 
-The PDF slides for Marc-Andr? Lemburg's presentation "Writing Unicode-aware
-Applications in Python" are available at
-<http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf>
-and discuss questions of character encodings as well as how to internationalize
+One section of `Mastering Python 3 Input/Output <http://pyvideo.org/video/289/pycon-2010--mastering-python-3-i-o>`_,  a PyCon 2010 talk by David Beazley, discusses text processing and binary data handling.
+
+The `PDF slides for Marc-Andr? Lemburg's presentation "Writing Unicode-aware Applications in Python" <http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf>`_
+discuss questions of character encodings as well as how to internationalize
 and localize an application.  These slides cover Python 2.x only.
 
+`The Guts of Unicode in Python <http://pyvideo.org/video/1768/the-guts-of-unicode-in-python>`_ is a PyCon 2013 talk by Benjamin Peterson that discusses the internal Unicode representation in Python 3.3.
+
 
 Acknowledgements
 ================
 
-Thanks to the following people who have noted errors or offered suggestions on
-this article: Nicholas Bastin, Marius Gedminas, Kent Johnson, Ken Krugler,
-Marc-Andr? Lemburg, Martin von L?wis, Chad Whitacre.
+The initial draft of this document was written by Andrew Kuchling.
+It has since been revised further by Alexander Belopolsky, Georg Brandl,
+Andrew Kuchling, and Ezio Melotti.
 
-.. comment
-   Revision History
-
-   Version 1.0: posted August 5 2005.
-
-   Version 1.01: posted August 7 2005.  Corrects factual and markup errors; adds
-   several links.
-
-   Version 1.02: posted August 16 2005.  Corrects factual errors.
-
-   Version 1.1: Feb-Nov 2008.  Updates the document with respect to Python 3 changes.
-
-   Version 1.11: posted June 20 2010.  Notes that Python 3.x is not covered,
-   and that the HOWTO only covers 2.x.
-
-.. comment Describe Python 3.x support (new section? new document?)
-.. comment Describe use of codecs.StreamRecoder and StreamReaderWriter
-
-.. comment
-   Original outline:
-
-   - [ ] Unicode introduction
-       - [ ] ASCII
-       - [ ] Terms
-           - [ ] Character
-           - [ ] Code point
-         - [ ] Encodings
-            - [ ] Common encodings: ASCII, Latin-1, UTF-8
-       - [ ] Unicode Python type
-           - [ ] Writing unicode literals
-               - [ ] Obscurity: -U switch
-           - [ ] Built-ins
-               - [ ] unichr()
-               - [ ] ord()
-               - [ ] unicode() constructor
-           - [ ] Unicode type
-               - [ ] encode(), decode() methods
-       - [ ] Unicodedata module for character properties
-       - [ ] I/O
-           - [ ] Reading/writing Unicode data into files
-               - [ ] Byte-order marks
-           - [ ] Unicode filenames
-       - [ ] Writing Unicode programs
-           - [ ] Do everything in Unicode
-           - [ ] Declaring source code encodings (PEP 263)
+Thanks to the following people who have noted errors or offered
+suggestions on this article: ?ric Araujo, Nicholas Bastin, Nick
+Coghlan, Marius Gedminas, Kent Johnson, Ken Krugler, Marc-Andr?
+Lemburg, Martin von L?wis, Terry J. Reedy, Chad Whitacre.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 20 15:52:24 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Thu, 20 Jun 2013 15:52:24 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_from_3=2E3?=
Message-ID: <3bbkx41B4wzRbb@mail.python.org>

http://hg.python.org/cpython/rev/3acbb23c73bc
changeset:   84227:3acbb23c73bc
parent:      84225:e26b00adb7ba
parent:      84226:1dbbed06a163
user:        Andrew Kuchling <amk at amk.ca>
date:        Thu Jun 20 09:51:51 2013 -0400
summary:
  Merge from 3.3

files:
  Doc/howto/unicode.rst |  255 ++++++++++++++++++-----------
  1 files changed, 161 insertions(+), 94 deletions(-)


diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst
--- a/Doc/howto/unicode.rst
+++ b/Doc/howto/unicode.rst
@@ -28,15 +28,15 @@
 as 'na?ve' and 'caf?', and some publications have house styles which require
 spellings such as 'co?perate'.)
 
-For a while people just wrote programs that didn't display accents.  I remember
-looking at Apple ][ BASIC programs, published in French-language publications in
-the mid-1980s, that had lines like these::
+For a while people just wrote programs that didn't display accents.
+In the mid-1980s an Apple II BASIC program written by a French speaker
+might have lines like these::
 
    PRINT "FICHIER EST COMPLETE."
    PRINT "CARACTERE NON ACCEPTE."
 
-Those messages should contain accents, and they just look wrong to someone who
-can read French.
+Those messages should contain accents (complet?, caract?re, accept?),
+and they just look wrong to someone who can read French.
 
 In the 1980s, almost all personal computers were 8-bit, meaning that bytes could
 hold values ranging from 0 to 255.  ASCII codes only went up to 127, so some
@@ -69,9 +69,12 @@
 originally separate efforts, but the specifications were merged with the 1.1
 revision of Unicode.
 
-(This discussion of Unicode's history is highly simplified.  I don't think the
-average Python programmer needs to worry about the historical details; consult
-the Unicode consortium site listed in the References for more information.)
+(This discussion of Unicode's history is highly simplified.  The
+precise historical details aren't necessary for understanding how to
+use Unicode effectively, but if you're curious, consult the Unicode
+consortium site listed in the References or
+the `Wikipedia entry for Unicode <http://en.wikipedia.org/wiki/Unicode#History>`_
+for more information.)
 
 
 Definitions
@@ -216,10 +219,8 @@
 
 Another `good introductory article <http://www.joelonsoftware.com/articles/Unicode.html>`_
 was written by Joel Spolsky.
-If this introduction didn't make things clear to you, you should try reading this
-alternate article before continuing.
-
-.. Jason Orendorff XXX http://www.jorendorff.com/articles/unicode/ is broken
+If this introduction didn't make things clear to you, you should try
+reading this alternate article before continuing.
 
 Wikipedia entries are often helpful; see the entries for "`character encoding
 <http://en.wikipedia.org/wiki/Character_encoding>`_" and `UTF-8
@@ -239,8 +240,31 @@
 characters, meaning any string created using ``"unicode rocks!"``, ``'unicode
 rocks!'``, or the triple-quoted string syntax is stored as Unicode.
 
-To insert a non-ASCII Unicode character, e.g., any letters with
-accents, one can use escape sequences in their string literals as such::
+The default encoding for Python source code is UTF-8, so you can simply
+include a Unicode character in a string literal::
+
+   try:
+       with open('/tmp/input.txt', 'r') as f:
+           ...
+   except IOError:
+       # 'File not found' error message.
+       print("Fichier non trouv?")
+
+You can use a different encoding from UTF-8 by putting a specially-formatted
+comment as the first or second line of the source code::
+
+   # -*- coding: <encoding name> -*-
+
+Side note: Python 3 also supports using Unicode characters in identifiers::
+
+   r?pertoire = "/tmp/records.log"
+   with open(r?pertoire, "w") as f:
+       f.write("test\n")
+
+If you can't enter a particular character in your editor or want to
+keep the source code ASCII-only for some reason, you can also use
+escape sequences in string literals. (Depending on your system,
+you may see the actual capital-delta glyph instead of a \u escape.) ::
 
    >>> "\N{GREEK CAPITAL LETTER DELTA}"  # Using the character name
    '\u0394'
@@ -251,7 +275,7 @@
 
 In addition, one can create a string using the :func:`~bytes.decode` method of
 :class:`bytes`.  This method takes an *encoding* argument, such as ``UTF-8``,
-and optionally, an *errors* argument.
+and optionally an *errors* argument.
 
 The *errors* argument specifies the response when the input string can't be
 converted according to the encoding's rules.  Legal values for this argument are
@@ -295,11 +319,15 @@
 
 The opposite method of :meth:`bytes.decode` is :meth:`str.encode`,
 which returns a :class:`bytes` representation of the Unicode string, encoded in the
-requested *encoding*.  The *errors* parameter is the same as the parameter of
-the :meth:`~bytes.decode` method, with one additional possibility; as well as
-``'strict'``, ``'ignore'``, and ``'replace'`` (which in this case inserts a
-question mark instead of the unencodable character), you can also pass
-``'xmlcharrefreplace'`` which uses XML's character references.
+requested *encoding*.
+
+The *errors* parameter is the same as the parameter of the
+:meth:`~bytes.decode` method but supports a few more possible handlers. As well as
+``'strict'``, ``'ignore'``, and ``'replace'`` (which in this case
+inserts a question mark instead of the unencodable character), there is
+also ``'xmlcharrefreplace'`` (inserts an XML character reference) and
+``backslashreplace`` (inserts a ``\uNNNN`` escape sequence).
+
 The following example shows the different results::
 
     >>> u = chr(40960) + 'abcd' + chr(1972)
@@ -316,16 +344,15 @@
     b'?abcd?'
     >>> u.encode('ascii', 'xmlcharrefreplace')
     b'&#40960;abcd&#1972;'
+    >>> u.encode('ascii', 'backslashreplace')
+    b'\\ua000abcd\\u07b4'
 
-.. XXX mention the surrogate* error handlers
-
-The low-level routines for registering and accessing the available encodings are
-found in the :mod:`codecs` module.  However, the encoding and decoding functions
-returned by this module are usually more low-level than is comfortable, so I'm
-not going to describe the :mod:`codecs` module here.  If you need to implement a
-completely new encoding, you'll need to learn about the :mod:`codecs` module
-interfaces, but implementing encodings is a specialized task that also won't be
-covered here.  Consult the Python documentation to learn more about this module.
+The low-level routines for registering and accessing the available
+encodings are found in the :mod:`codecs` module.  Implementing new
+encodings also requires understanding the :mod:`codecs` module.
+However, the encoding and decoding functions returned by this module
+are usually more low-level than is comfortable, and writing new encodings
+is a specialized task, so the module won't be covered in this HOWTO.
 
 
 Unicode Literals in Python Source Code
@@ -415,12 +442,50 @@
 from the above output, ``'Ll'`` means 'Letter, lowercase', ``'No'`` means
 "Number, other", ``'Mn'`` is "Mark, nonspacing", and ``'So'`` is "Symbol,
 other".  See
-<http://www.unicode.org/reports/tr44/#General_Category_Values> for a
+`the General Category Values section of the Unicode Character Database documentation <http://www.unicode.org/reports/tr44/#General_Category_Values>`_ for a
 list of category codes.
 
+
+Unicode Regular Expressions
+---------------------------
+
+The regular expressions supported by the :mod:`re` module can be provided
+either as bytes or strings.  Some of the special character sequences such as
+``\d`` and ``\w`` have different meanings depending on whether
+the pattern is supplied as bytes or a string.  For example,
+``\d`` will match the characters ``[0-9]`` in bytes but
+in strings will match any character that's in the ``'Nd'`` category.
+
+The string in this example has the number 57 written in both Thai and
+Arabic numerals::
+
+   import re
+   p = re.compile('\d+')
+
+   s = "Over \u0e55\u0e57 57 flavours"
+   m = p.search(s)
+   print(repr(m.group()))
+
+When executed, ``\d+`` will match the Thai numerals and print them
+out.  If you supply the :const:`re.ASCII` flag to
+:func:`~re.compile`, ``\d+`` will match the substring "57" instead.
+
+Similarly, ``\w`` matches a wide variety of Unicode characters but
+only ``[a-zA-Z0-9_]`` in bytes or if :const:`re.ASCII` is supplied,
+and ``\s`` will match either Unicode whitespace characters or
+``[ \t\n\r\f\v]``.
+
+
 References
 ----------
 
+.. comment should these be mentioned earlier, e.g. at the start of the "introduction to Unicode" first section?
+
+Some good alternative discussions of Python's Unicode support are:
+
+* `Processing Text Files in Python 3 <http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html>`_, by Nick Coghlan.
+* `Pragmatic Unicode <http://nedbatchelder.com/text/unipain.html>`_, a PyCon 2012 presentation by Ned Batchelder.
+
 The :class:`str` type is described in the Python library reference at
 :ref:`textseq`.
 
@@ -428,12 +493,10 @@
 
 The documentation for the :mod:`codecs` module.
 
-Marc-Andr? Lemburg gave a presentation at EuroPython 2002 titled "Python and
-Unicode".  A PDF version of his slides is available at
-<http://downloads.egenix.com/python/Unicode-EPC2002-Talk.pdf>, and is an
-excellent overview of the design of Python's Unicode features (based on Python
-2, where the Unicode string type is called ``unicode`` and literals start with
-``u``).
+Marc-Andr? Lemburg gave `a presentation titled "Python and Unicode" (PDF slides) <http://downloads.egenix.com/python/Unicode-EPC2002-Talk.pdf>`_ at
+EuroPython 2002.  The slides are an excellent overview of the design
+of Python 2's Unicode features (where the Unicode string type is
+called ``unicode`` and literals start with ``u``).
 
 
 Reading and Writing Unicode Data
@@ -512,7 +575,7 @@
 Windows, Python uses the name "mbcs" to refer to whatever the currently
 configured encoding is.  On Unix systems, there will only be a filesystem
 encoding if you've set the ``LANG`` or ``LC_CTYPE`` environment variables; if
-you haven't, the default encoding is ASCII.
+you haven't, the default encoding is UTF-8.
 
 The :func:`sys.getfilesystemencoding` function returns the encoding to use on
 your current system, in case you want to do the encoding manually, but there's
@@ -527,13 +590,13 @@
 Functions in the :mod:`os` module such as :func:`os.stat` will also accept Unicode
 filenames.
 
-Function :func:`os.listdir`, which returns filenames, raises an issue: should it return
+The :func:`os.listdir` function returns filenames and raises an issue: should it return
 the Unicode version of filenames, or should it return bytes containing
 the encoded versions?  :func:`os.listdir` will do both, depending on whether you
 provided the directory path as bytes or a Unicode string.  If you pass a
 Unicode string as the path, filenames will be decoded using the filesystem's
 encoding and a list of Unicode strings will be returned, while passing a byte
-path will return the bytes versions of the filenames.  For example,
+path will return the filenames as bytes.  For example,
 assuming the default filesystem encoding is UTF-8, running the following
 program::
 
@@ -548,13 +611,13 @@
 will produce the following output::
 
    amk:~$ python t.py
-   [b'.svn', b'filename\xe4\x94\x80abc', ...]
-   ['.svn', 'filename\u4500abc', ...]
+   [b'filename\xe4\x94\x80abc', ...]
+   ['filename\u4500abc', ...]
 
 The first list contains UTF-8-encoded filenames, and the second list contains
 the Unicode versions.
 
-Note that in most occasions, the Unicode APIs should be used.  The bytes APIs
+Note that on most occasions, the Unicode APIs should be used.  The bytes APIs
 should only be used on systems where undecodable file names can be present,
 i.e. Unix systems.
 
@@ -585,65 +648,69 @@
 clever way to hide malicious text in the encoded bytestream.
 
 
+Converting Between File Encodings
+'''''''''''''''''''''''''''''''''
+
+The :class:`~codecs.StreamRecoder` class can transparently convert between
+encodings, taking a stream that returns data in encoding #1
+and behaving like a stream returning data in encoding #2.
+
+For example, if you have an input file *f* that's in Latin-1, you
+can wrap it with a :class:`StreamRecoder` to return bytes encoded in UTF-8::
+
+    new_f = codecs.StreamRecoder(f,
+        # en/decoder: used by read() to encode its results and
+        # by write() to decode its input.
+        codecs.getencoder('utf-8'), codecs.getdecoder('utf-8'),
+
+        # reader/writer: used to read and write to the stream.
+        codecs.getreader('latin-1'), codecs.getwriter('latin-1') )
+
+
+Files in an Unknown Encoding
+''''''''''''''''''''''''''''
+
+What can you do if you need to make a change to a file, but don't know
+the file's encoding?  If you know the encoding is ASCII-compatible and
+only want to examine or modify the ASCII parts, you can open the file
+with the ``surrogateescape`` error handler::
+
+   with open(fname, 'r', encoding="ascii", errors="surrogateescape") as f:
+       data = f.read()
+
+   # make changes to the string 'data'
+
+   with open(fname + '.new', 'w',
+              encoding="ascii", errors="surrogateescape") as f:
+       f.write(data)
+
+The ``surrogateescape`` error handler will decode any non-ASCII bytes
+as code points in the Unicode Private Use Area ranging from U+DC80 to
+U+DCFF.  These private code points will then be turned back into the
+same bytes when the ``surrogateescape`` error handler is used when
+encoding the data and writing it back out.
+
+
 References
 ----------
 
-The PDF slides for Marc-Andr? Lemburg's presentation "Writing Unicode-aware
-Applications in Python" are available at
-<http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf>
-and discuss questions of character encodings as well as how to internationalize
+One section of `Mastering Python 3 Input/Output <http://pyvideo.org/video/289/pycon-2010--mastering-python-3-i-o>`_,  a PyCon 2010 talk by David Beazley, discusses text processing and binary data handling.
+
+The `PDF slides for Marc-Andr? Lemburg's presentation "Writing Unicode-aware Applications in Python" <http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf>`_
+discuss questions of character encodings as well as how to internationalize
 and localize an application.  These slides cover Python 2.x only.
 
+`The Guts of Unicode in Python <http://pyvideo.org/video/1768/the-guts-of-unicode-in-python>`_ is a PyCon 2013 talk by Benjamin Peterson that discusses the internal Unicode representation in Python 3.3.
+
 
 Acknowledgements
 ================
 
-Thanks to the following people who have noted errors or offered suggestions on
-this article: Nicholas Bastin, Marius Gedminas, Kent Johnson, Ken Krugler,
-Marc-Andr? Lemburg, Martin von L?wis, Chad Whitacre.
+The initial draft of this document was written by Andrew Kuchling.
+It has since been revised further by Alexander Belopolsky, Georg Brandl,
+Andrew Kuchling, and Ezio Melotti.
 
-.. comment
-   Revision History
-
-   Version 1.0: posted August 5 2005.
-
-   Version 1.01: posted August 7 2005.  Corrects factual and markup errors; adds
-   several links.
-
-   Version 1.02: posted August 16 2005.  Corrects factual errors.
-
-   Version 1.1: Feb-Nov 2008.  Updates the document with respect to Python 3 changes.
-
-   Version 1.11: posted June 20 2010.  Notes that Python 3.x is not covered,
-   and that the HOWTO only covers 2.x.
-
-.. comment Describe Python 3.x support (new section? new document?)
-.. comment Describe use of codecs.StreamRecoder and StreamReaderWriter
-
-.. comment
-   Original outline:
-
-   - [ ] Unicode introduction
-       - [ ] ASCII
-       - [ ] Terms
-           - [ ] Character
-           - [ ] Code point
-         - [ ] Encodings
-            - [ ] Common encodings: ASCII, Latin-1, UTF-8
-       - [ ] Unicode Python type
-           - [ ] Writing unicode literals
-               - [ ] Obscurity: -U switch
-           - [ ] Built-ins
-               - [ ] unichr()
-               - [ ] ord()
-               - [ ] unicode() constructor
-           - [ ] Unicode type
-               - [ ] encode(), decode() methods
-       - [ ] Unicodedata module for character properties
-       - [ ] I/O
-           - [ ] Reading/writing Unicode data into files
-               - [ ] Byte-order marks
-           - [ ] Unicode filenames
-       - [ ] Writing Unicode programs
-           - [ ] Do everything in Unicode
-           - [ ] Declaring source code encodings (PEP 263)
+Thanks to the following people who have noted errors or offered
+suggestions on this article: ?ric Araujo, Nicholas Bastin, Nick
+Coghlan, Marius Gedminas, Kent Johnson, Ken Krugler, Marc-Andr?
+Lemburg, Martin von L?wis, Terry J. Reedy, Chad Whitacre.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Thu Jun 20 23:42:43 2013
From: python-checkins at python.org (victor.stinner)
Date: Thu, 20 Jun 2013 23:42:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_Rename_domains=3A_?=
 =?utf-8?q?PYMEM=5FDOMAIN=5FRAW=2C_PYMEM=5FDOMAIN=5FMEM=2C_PYMEM=5FDOMAIN?=
 =?utf-8?q?=5FOBJ?=
Message-ID: <3bbxMl0LXdz7Ljs@mail.python.org>

http://hg.python.org/peps/rev/e883b171f3e5
changeset:   4954:e883b171f3e5
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Jun 20 23:42:27 2013 +0200
summary:
  PEP 445: Rename domains: PYMEM_DOMAIN_RAW, PYMEM_DOMAIN_MEM, PYMEM_DOMAIN_OBJ

files:
  pep-0445.txt |  12 ++++++------
  1 files changed, 6 insertions(+), 6 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -73,14 +73,14 @@
 * Add a new ``PyMemAllocatorDomain`` enum to choose the Python
   allocator domain. Domains:
 
-  - ``PYALLOC_PYMEM_RAW``: ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``
-    and ``PyMem_RawRealloc()``
+  - ``PYMEM_DOMAIN_RAW``: ``PyMem_RawMalloc()``, ``PyMem_RawRealloc()``
+    and ``PyMem_RawFree()``
 
-  - ``PYALLOC_PYMEM``: ``PyMem_Malloc()``, ``PyMem_Realloc()`` and
-    ``PyMem_Realloc()``
+  - ``PYMEM_DOMAIN_MEM``: ``PyMem_Malloc()``, ``PyMem_Realloc()`` and
+    ``PyMem_Free()``
 
-  - ``PYALLOC_PYOBJECT``: ``PyObject_Malloc()``, ``PyObject_Realloc()``
-    and ``PyObject_Realloc()``
+  - ``PYMEM_DOMAIN_OBJ``: ``PyObject_Malloc()``, ``PyObject_Realloc()``
+    and ``PyObject_Free()``
 
 * Add new functions to get and set memory allocators:
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Thu Jun 20 23:56:13 2013
From: python-checkins at python.org (victor.stinner)
Date: Thu, 20 Jun 2013 23:56:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_fix_examples?=
Message-ID: <3bbxgK6mCKz7Llb@mail.python.org>

http://hg.python.org/peps/rev/b9a3b02a551d
changeset:   4955:b9a3b02a551d
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Thu Jun 20 23:55:57 2013 +0200
summary:
  PEP 445: fix examples

files:
  pep-0445.txt |  32 ++++++++++++++++----------------
  1 files changed, 16 insertions(+), 16 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -125,10 +125,10 @@
 
 Default allocators:
 
-* ``PYALLOC_PYMEM_RAW``, ``PYALLOC_PYMEM``: ``malloc()``,
+* ``PYMEM_DOMAIN_RAW``, ``PYMEM_DOMAIN_MEM``: ``malloc()``,
   ``realloc()``, ``free()`` (and *ctx* is NULL); call ``malloc(1)`` when
   requesting zero bytes
-* ``PYALLOC_PYOBJECT``: *pymalloc* allocator which fall backs on
+* ``PYMEM_DOMAIN_OBJ``: *pymalloc* allocator which fall backs on
   ``PyMem_Malloc()`` for allocations larger than 512 bytes
 * *pymalloc* arena allocator: ``mmap()``, ``munmap()`` (and *ctx* is
   NULL), or ``malloc()`` and ``free()`` if ``mmap()`` is not available
@@ -226,8 +226,8 @@
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
-        PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc);
-        PyMem_SetAllocator(PYALLOC_PYMEM, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
 
         arena.ctx = &arena_padding;
         arena.alloc = my_alloc_arena;
@@ -238,7 +238,7 @@
     }
 
 .. warning::
-   Remove the call ``PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc)`` if
+   Remove the call ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc)`` if
    the new allocator is not thread-safe.
 
 
@@ -280,15 +280,15 @@
         alloc.realloc = my_realloc;
         alloc.free = my_free;
 
-        PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc);
-        PyMem_SetAllocator(PYALLOC_PYMEM, &alloc);
-        PyMem_SetAllocator(PYALLOC_PYOBJECT, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &alloc);
 
         PyMem_SetupDebugHooks();
     }
 
 .. warning::
-   Remove the call ``PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc)`` if
+   Remove the call ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc)`` if
    the new allocator is not thread-safe.
 
 
@@ -344,22 +344,22 @@
         alloc.malloc = hook_malloc;
         alloc.realloc = hook_realloc;
         alloc.free = hook_free;
-        PyMem_GetAllocator(PYALLOC_PYMEM_RAW, &hook.raw);
-        PyMem_GetAllocator(PYALLOC_PYMEM, &hook.mem);
-        PyMem_GetAllocator(PYALLOC_PYOBJECT, &hook.obj);
+        PyMem_GetAllocator(PYMEM_DOMAIN_RAW, &hook.raw);
+        PyMem_GetAllocator(PYMEM_DOMAIN_MEM, &hook.mem);
+        PyMem_GetAllocator(PYMEM_DOMAIN_OBJ, &hook.obj);
 
         alloc.ctx = &hook.raw;
-        PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
 
         alloc.ctx = &hook.mem;
-        PyMem_SetAllocator(PYALLOC_PYMEM, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
 
         alloc.ctx = &hook.obj;
-        PyMem_SetAllocator(PYALLOC_PYOBJECT, &alloc);
+        PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &alloc);
     }
 
 .. warning::
-   Remove the call ``PyMem_SetAllocator(PYALLOC_PYMEM_RAW, &alloc)`` if
+   Remove the call ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc)`` if
    hooks are not thread-safe.
 
 .. note::

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Fri Jun 21 00:32:35 2013
From: python-checkins at python.org (victor.stinner)
Date: Fri, 21 Jun 2013 00:32:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew/3=2E4=3A_mention_?=
 =?utf-8?q?the_new_enum_module=2C_PEP_435?=
Message-ID: <3bbyTH5xpZz7LjN@mail.python.org>

http://hg.python.org/cpython/rev/fdc9429d3d93
changeset:   84228:fdc9429d3d93
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Jun 21 00:31:55 2013 +0200
summary:
  whatsnew/3.4: mention the new enum module, PEP 435

files:
  Doc/whatsnew/3.4.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -90,7 +90,7 @@
 
 New library modules:
 
-* None yet.
+* :mod:`enum`: Implementation of the :pep:`435`.
 
 New built-in features:
 
@@ -290,4 +290,4 @@
   felt to be over-reaching/overloading of that meaning when the source code is
   found but improperly structured. If you were catching ImportError before and
   wish to continue to ignore syntax or decoding issues, catch all three
-  exceptions now.
\ No newline at end of file
+  exceptions now.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 00:37:10 2013
From: python-checkins at python.org (victor.stinner)
Date: Fri, 21 Jun 2013 00:37:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_whatsnew/3=2E4=3A_mention_?=
 =?utf-8?q?functools=2Esingledispatch=2C_PEP_443?=
Message-ID: <3bbyZZ3k7lz7Ljt@mail.python.org>

http://hg.python.org/cpython/rev/ad8a184916fc
changeset:   84229:ad8a184916fc
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Jun 21 00:36:30 2013 +0200
summary:
  whatsnew/3.4: mention functools.singledispatch, PEP 443

files:
  Doc/whatsnew/3.4.rst |  6 ++++++
  1 files changed, 6 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -177,6 +177,12 @@
 detected.  (Contributed by R. David Murray and Daniel Urban in :issue:`16522`.)
 
 
+functools
+---------
+
+New :func:`functools.singledispatch` decorator: see the :pep:`443`.
+
+
 smtplib
 -------
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 03:19:29 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 03:19:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMxODI3?=
 =?utf-8?q?2=3A_use_=27builtins=27_for_3=2E3_instead_of_=5F=5Fbuiltin=5F?=
 =?utf-8?q?=5F?=
Message-ID: <3bc29s2ZcqzSqK@mail.python.org>

http://hg.python.org/cpython/rev/b805506b11e0
changeset:   84230:b805506b11e0
branch:      3.3
parent:      84226:1dbbed06a163
user:        Andrew Kuchling <amk at amk.ca>
date:        Thu Jun 20 21:17:41 2013 -0400
summary:
  Closes #18272: use 'builtins' for 3.3 instead of __builtin__

files:
  Doc/library/imp.rst       |  2 +-
  Doc/library/itertools.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -158,7 +158,7 @@
           cache = {}
 
    It is legal though generally not very useful to reload built-in or dynamically
-   loaded modules, except for :mod:`sys`, :mod:`__main__` and :mod:`__builtin__`.
+   loaded modules, except for :mod:`sys`, :mod:`__main__` and :mod:`builtins`.
    In many cases, however, extension modules are not designed to be initialized
    more than once, and may fail in arbitrary ways when reloaded.
 
diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -762,7 +762,7 @@
        """ Call a function repeatedly until an exception is raised.
 
        Converts a call-until-exception interface to an iterator interface.
-       Like __builtin__.iter(func, sentinel) but uses an exception instead
+       Like builtins.iter(func, sentinel) but uses an exception instead
        of a sentinel to end the loop.
 
        Examples:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 03:20:29 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 03:20:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_from_3=2E3?=
Message-ID: <3bc2C102cBzSqK@mail.python.org>

http://hg.python.org/cpython/rev/c1f537c4a8e5
changeset:   84231:c1f537c4a8e5
parent:      84229:ad8a184916fc
parent:      84230:b805506b11e0
user:        Andrew Kuchling <amk at amk.ca>
date:        Thu Jun 20 21:20:20 2013 -0400
summary:
  Merge from 3.3

files:
  Doc/library/imp.rst       |  2 +-
  Doc/library/itertools.rst |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -165,7 +165,7 @@
           cache = {}
 
    It is legal though generally not very useful to reload built-in or dynamically
-   loaded modules, except for :mod:`sys`, :mod:`__main__` and :mod:`__builtin__`.
+   loaded modules, except for :mod:`sys`, :mod:`__main__` and :mod:`builtins`.
    In many cases, however, extension modules are not designed to be initialized
    more than once, and may fail in arbitrary ways when reloaded.
 
diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -762,7 +762,7 @@
        """ Call a function repeatedly until an exception is raised.
 
        Converts a call-until-exception interface to an iterator interface.
-       Like __builtin__.iter(func, sentinel) but uses an exception instead
+       Like builtins.iter(func, sentinel) but uses an exception instead
        of a sentinel to end the loop.
 
        Examples:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 03:30:36 2013
From: python-checkins at python.org (brett.cannon)
Date: Fri, 21 Jun 2013 03:30:36 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Grammatical_mistake_in_a_c?=
	=?utf-8?q?omment?=
Message-ID: <3bc2Qh1wwtzRF7@mail.python.org>

http://hg.python.org/cpython/rev/abebc6c1962b
changeset:   84232:abebc6c1962b
user:        Brett Cannon <brett at python.org>
date:        Thu Jun 20 21:30:32 2013 -0400
summary:
  Grammatical mistake in a comment

files:
  Modules/zipimport.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/zipimport.c b/Modules/zipimport.c
--- a/Modules/zipimport.c
+++ b/Modules/zipimport.c
@@ -1258,7 +1258,7 @@
 }
 
 /* Given a string buffer containing Python source code, compile it
-   return and return a code object as a new reference. */
+   and return a code object as a new reference. */
 static PyObject *
 compile_source(PyObject *pathname, PyObject *source)
 {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 03:40:26 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 03:40:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMxODI2?=
 =?utf-8?q?7=3A_use_floor_division_in_code_example?=
Message-ID: <3bc2f24kDqzSg9@mail.python.org>

http://hg.python.org/cpython/rev/2a3bc6eb2e13
changeset:   84233:2a3bc6eb2e13
branch:      3.3
parent:      84230:b805506b11e0
user:        Andrew Kuchling <amk at amk.ca>
date:        Thu Jun 20 21:33:05 2013 -0400
summary:
  Closes #18267: use floor division in code example

files:
  Doc/library/xmlrpc.client.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/xmlrpc.client.rst b/Doc/library/xmlrpc.client.rst
--- a/Doc/library/xmlrpc.client.rst
+++ b/Doc/library/xmlrpc.client.rst
@@ -435,7 +435,7 @@
    is a :term:`generator`; iterating over this generator yields the individual
    results.
 
-A usage example of this class follows.  The server code ::
+A usage example of this class follows.  The server code::
 
    from xmlrpc.server import SimpleXMLRPCServer
 
@@ -449,7 +449,7 @@
        return x*y
 
    def divide(x, y):
-       return x/y
+       return x // y
 
    # A simple server with simple arithmetic functions
    server = SimpleXMLRPCServer(("localhost", 8000))

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 03:40:27 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 03:40:27 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_from_3=2E3?=
Message-ID: <3bc2f36TymzSg9@mail.python.org>

http://hg.python.org/cpython/rev/f32dad3a243e
changeset:   84234:f32dad3a243e
parent:      84232:abebc6c1962b
parent:      84233:2a3bc6eb2e13
user:        Andrew Kuchling <amk at amk.ca>
date:        Thu Jun 20 21:40:14 2013 -0400
summary:
  Merge from 3.3

files:
  Doc/library/xmlrpc.client.rst |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/xmlrpc.client.rst b/Doc/library/xmlrpc.client.rst
--- a/Doc/library/xmlrpc.client.rst
+++ b/Doc/library/xmlrpc.client.rst
@@ -435,7 +435,7 @@
    is a :term:`generator`; iterating over this generator yields the individual
    results.
 
-A usage example of this class follows.  The server code ::
+A usage example of this class follows.  The server code::
 
    from xmlrpc.server import SimpleXMLRPCServer
 
@@ -449,7 +449,7 @@
        return x*y
 
    def divide(x, y):
-       return x/y
+       return x // y
 
    # A simple server with simple arithmetic functions
    server = SimpleXMLRPCServer(("localhost", 8000))

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Fri Jun 21 05:44:18 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 21 Jun 2013 05:44:18 +0200
Subject: [Python-checkins] Daily reference leaks (c1f537c4a8e5): sum=3
Message-ID: <E1UpsGo-000227-90@ap.vmr.nerim.net>

results for c1f537c4a8e5 on branch "default"
--------------------------------------------

test_unittest leaked [-1, 1, 2] memory blocks, sum=2
test_concurrent_futures leaked [0, -2, 3] memory blocks, sum=1


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogvKOY6n', '-x']

From python-checkins at python.org  Fri Jun 21 13:45:03 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 13:45:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2318247=3A_add_Lib?=
 =?utf-8?q?/test/data/*_to_=2Egitignore?=
Message-ID: <3bcJ3g6BXBz7LjQ@mail.python.org>

http://hg.python.org/cpython/rev/68e1eec01113
changeset:   84235:68e1eec01113
user:        Andrew Kuchling <amk at amk.ca>
date:        Fri Jun 21 07:44:50 2013 -0400
summary:
  Closes #18247: add Lib/test/data/* to .gitignore

files:
  .gitignore |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/.gitignore b/.gitignore
--- a/.gitignore
+++ b/.gitignore
@@ -15,6 +15,7 @@
 Doc/tools/pygments/
 Doc/tools/sphinx/
 Lib/lib2to3/*.pickle
+Lib/test/data/*
 Lib/_sysconfigdata.py
 Lib/plat-mac/errors.rsrc.df.rsrc
 Makefile

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 14:05:58 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 14:05:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMxODIz?=
 =?utf-8?q?9=3A_correct_description_of_count=28=29_in_module_docstring?=
Message-ID: <3bcJWp6Dy1z7Lk6@mail.python.org>

http://hg.python.org/cpython/rev/ad0b44cdae41
changeset:   84236:ad0b44cdae41
branch:      3.3
parent:      84233:2a3bc6eb2e13
user:        Andrew Kuchling <amk at amk.ca>
date:        Fri Jun 21 07:58:35 2013 -0400
summary:
  Closes #18239: correct description of count() in module docstring

files:
  Modules/itertoolsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c
--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -4452,7 +4452,7 @@
 "Functional tools for creating and using iterators.\n\
 \n\
 Infinite iterators:\n\
-count([n]) --> n, n+1, n+2, ...\n\
+count(start=0, step=1) --> start, start+step, start+2*step, ...\n\
 cycle(p) --> p0, p1, ... plast, p0, p1, ...\n\
 repeat(elem [,n]) --> elem, elem, elem, ... endlessly or up to n times\n\
 \n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 14:06:00 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 14:06:00 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMxODIx?=
 =?utf-8?q?8=3A_use_correct_variable_name_for_starting_point?=
Message-ID: <3bcJWr0tp4z7Lk6@mail.python.org>

http://hg.python.org/cpython/rev/12478f549ed8
changeset:   84237:12478f549ed8
branch:      3.3
user:        Andrew Kuchling <amk at amk.ca>
date:        Fri Jun 21 08:00:58 2013 -0400
summary:
  Closes #18218: use correct variable name for starting point

files:
  Doc/library/itertools.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -278,7 +278,7 @@
 
 .. function:: count(start=0, step=1)
 
-   Make an iterator that returns evenly spaced values starting with *n*. Often
+   Make an iterator that returns evenly spaced values starting with number *start*. Often
    used as an argument to :func:`map` to generate consecutive data points.
    Also, used with :func:`zip` to add sequence numbers.  Equivalent to::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 14:13:45 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 14:13:45 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_from_3=2E3?=
Message-ID: <3bcJhn2yhxz7LjN@mail.python.org>

http://hg.python.org/cpython/rev/27a5da85770c
changeset:   84238:27a5da85770c
parent:      84235:68e1eec01113
parent:      84237:12478f549ed8
user:        Andrew Kuchling <amk at amk.ca>
date:        Fri Jun 21 08:07:35 2013 -0400
summary:
  Merge from 3.3

files:
  Doc/library/itertools.rst |  2 +-
  Modules/itertoolsmodule.c |  2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -278,7 +278,7 @@
 
 .. function:: count(start=0, step=1)
 
-   Make an iterator that returns evenly spaced values starting with *n*. Often
+   Make an iterator that returns evenly spaced values starting with number *start*. Often
    used as an argument to :func:`map` to generate consecutive data points.
    Also, used with :func:`zip` to add sequence numbers.  Equivalent to::
 
diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c
--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -4452,7 +4452,7 @@
 "Functional tools for creating and using iterators.\n\
 \n\
 Infinite iterators:\n\
-count([n]) --> n, n+1, n+2, ...\n\
+count(start=0, step=1) --> start, start+step, start+2*step, ...\n\
 cycle(p) --> p0, p1, ... plast, p0, p1, ...\n\
 repeat(elem [,n]) --> elem, elem, elem, ... endlessly or up to n times\n\
 \n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 17:00:42 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 17:00:42 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2313226=3A_update_referen?=
 =?utf-8?q?ces_from_ctypes/DLFCN_modules_to_os_module?=
Message-ID: <3bcNPQ6VmdzQl7@mail.python.org>

http://hg.python.org/cpython/rev/1da78c7d382b
changeset:   84239:1da78c7d382b
user:        Andrew Kuchling <amk at amk.ca>
date:        Fri Jun 21 10:58:41 2013 -0400
summary:
  #13226: update references from ctypes/DLFCN modules to os module

files:
  Doc/library/sys.rst |  9 +++++----
  Python/sysmodule.c  |  4 ++--
  2 files changed, 7 insertions(+), 6 deletions(-)


diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -411,9 +411,10 @@
 
 .. function:: getdlopenflags()
 
-   Return the current value of the flags that are used for :c:func:`dlopen` calls.
-   The flag constants are defined in the :mod:`ctypes` and :mod:`DLFCN` modules.
-   Availability: Unix.
+   Return the current value of the flags that are used for
+   :c:func:`dlopen` calls.  Symbolic names for the flag values can be
+   found in the :mod:`os` module (``RTLD_xxx`` constants, e.g.
+   :data:`os.RTLD_LAZY`).  Availability: Unix.
 
 
 .. function:: getfilesystemencoding()
@@ -906,7 +907,7 @@
    the interpreter loads extension modules.  Among other things, this will enable a
    lazy resolving of symbols when importing a module, if called as
    ``sys.setdlopenflags(0)``.  To share symbols across extension modules, call as
-   ``sys.setdlopenflags(os.RTLD_GLOBAL)``.  Symbolic names for the flag modules
+   ``sys.setdlopenflags(os.RTLD_GLOBAL)``.  Symbolic names for the flag values
    can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g.
    :data:`os.RTLD_LAZY`).
 
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -774,7 +774,7 @@
 interpreter loads extension modules.  Among other things, this will enable\n\
 a lazy resolving of symbols when importing a module, if called as\n\
 sys.setdlopenflags(0).  To share symbols across extension modules, call as\n\
-sys.setdlopenflags(ctypes.RTLD_GLOBAL).  Symbolic names for the flag modules\n\
+sys.setdlopenflags(os.RTLD_GLOBAL).  Symbolic names for the flag modules\n\
 can be found in the os module (RTLD_xxx constants, e.g. os.RTLD_LAZY).");
 
 static PyObject *
@@ -790,7 +790,7 @@
 "getdlopenflags() -> int\n\
 \n\
 Return the current value of the flags that are used for dlopen calls.\n\
-The flag constants are defined in the ctypes and DLFCN modules.");
+The flag constants are defined in the os module.");
 
 #endif  /* HAVE_DLOPEN */
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 17:50:15 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Fri, 21 Jun 2013 17:50:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2312716=3A_reorganize_doc?=
	=?utf-8?q?s_for_os_module_a_bit?=
Message-ID: <3bcPVb1MXXz7Lk3@mail.python.org>

http://hg.python.org/cpython/rev/88edac3bc2fc
changeset:   84240:88edac3bc2fc
user:        Andrew Kuchling <amk at amk.ca>
date:        Fri Jun 21 11:49:57 2013 -0400
summary:
  #12716: reorganize docs for os module a bit

* Move the constants used by os.open() to after the description of os.open(),
  instead of putting them in their own section.

* Move RTLD_* constants to "Miscellaneous System Information" section, and
  mention the sys functions they're used with.

* Mention dir_fd parameter in os.open() text.

* Typo fix ("are a supported by").

files:
  Doc/library/os.rst |  454 ++++++++++++++++----------------
  1 files changed, 224 insertions(+), 230 deletions(-)


diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -851,11 +851,11 @@
 
    For a description of the flag and mode values, see the C run-time documentation;
    flag constants (like :const:`O_RDONLY` and :const:`O_WRONLY`) are defined in
-   this module too (see :ref:`open-constants`).  In particular, on Windows adding
+   the :mod:`os` module.  In particular, on Windows adding
    :const:`O_BINARY` is needed to open files in binary mode.
 
    This function can support :ref:`paths relative to directory descriptors
-   <dir_fd>`.
+   <dir_fd>` with the *dir_fd* parameter.
 
    Availability: Unix, Windows.
 
@@ -869,224 +869,6 @@
    .. versionadded:: 3.3
       The *dir_fd* argument.
 
-
-.. function:: openpty()
-
-   .. index:: module: pty
-
-   Open a new pseudo-terminal pair. Return a pair of file descriptors ``(master,
-   slave)`` for the pty and the tty, respectively. For a (slightly) more portable
-   approach, use the :mod:`pty` module.
-
-   Availability: some flavors of Unix.
-
-
-.. function:: pipe()
-
-   Create a pipe.  Return a pair of file descriptors ``(r, w)`` usable for reading
-   and writing, respectively.
-
-   Availability: Unix, Windows.
-
-
-.. function:: pipe2(flags)
-
-   Create a pipe with *flags* set atomically.
-   *flags* can be constructed by ORing together one or more of these values:
-   :data:`O_NONBLOCK`, :data:`O_CLOEXEC`.
-   Return a pair of file descriptors ``(r, w)`` usable for reading and writing,
-   respectively.
-
-   Availability: some flavors of Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: posix_fallocate(fd, offset, len)
-
-   Ensures that enough disk space is allocated for the file specified by *fd*
-   starting from *offset* and continuing for *len* bytes.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: posix_fadvise(fd, offset, len, advice)
-
-   Announces an intention to access data in a specific pattern thus allowing
-   the kernel to make optimizations.
-   The advice applies to the region of the file specified by *fd* starting at
-   *offset* and continuing for *len* bytes.
-   *advice* is one of :data:`POSIX_FADV_NORMAL`, :data:`POSIX_FADV_SEQUENTIAL`,
-   :data:`POSIX_FADV_RANDOM`, :data:`POSIX_FADV_NOREUSE`,
-   :data:`POSIX_FADV_WILLNEED` or :data:`POSIX_FADV_DONTNEED`.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. data:: POSIX_FADV_NORMAL
-          POSIX_FADV_SEQUENTIAL
-          POSIX_FADV_RANDOM
-          POSIX_FADV_NOREUSE
-          POSIX_FADV_WILLNEED
-          POSIX_FADV_DONTNEED
-
-   Flags that can be used in *advice* in :func:`posix_fadvise` that specify
-   the access pattern that is likely to be used.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: pread(fd, buffersize, offset)
-
-   Read from a file descriptor, *fd*, at a position of *offset*. It will read up
-   to *buffersize* number of bytes. The file offset remains unchanged.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: pwrite(fd, string, offset)
-
-   Write *string* to a file descriptor, *fd*, from *offset*, leaving the file
-   offset unchanged.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: read(fd, n)
-
-   Read at most *n* bytes from file descriptor *fd*. Return a bytestring containing the
-   bytes read.  If the end of the file referred to by *fd* has been reached, an
-   empty bytes object is returned.
-
-   Availability: Unix, Windows.
-
-   .. note::
-
-      This function is intended for low-level I/O and must be applied to a file
-      descriptor as returned by :func:`os.open` or :func:`pipe`.  To read a
-      "file object" returned by the built-in function :func:`open` or by
-      :func:`popen` or :func:`fdopen`, or :data:`sys.stdin`, use its
-      :meth:`~file.read` or :meth:`~file.readline` methods.
-
-
-.. function:: sendfile(out, in, offset, nbytes)
-              sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0)
-
-   Copy *nbytes* bytes from file descriptor *in* to file descriptor *out*
-   starting at *offset*.
-   Return the number of bytes sent. When EOF is reached return 0.
-
-   The first function notation is supported by all platforms that define
-   :func:`sendfile`.
-
-   On Linux, if *offset* is given as ``None``, the bytes are read from the
-   current position of *in* and the position of *in* is updated.
-
-   The second case may be used on Mac OS X and FreeBSD where *headers* and
-   *trailers* are arbitrary sequences of buffers that are written before and
-   after the data from *in* is written. It returns the same as the first case.
-
-   On Mac OS X and FreeBSD, a value of 0 for *nbytes* specifies to send until
-   the end of *in* is reached.
-
-   All platforms support sockets as *out* file descriptor, and some platforms
-   allow other types (e.g. regular file, pipe) as well.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. data:: SF_NODISKIO
-          SF_MNOWAIT
-          SF_SYNC
-
-   Parameters to the :func:`sendfile` function, if the implementation supports
-   them.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: readv(fd, buffers)
-
-   Read from a file descriptor into a number of writable buffers. *buffers* is
-   an arbitrary sequence of writable buffers. Returns the total number of bytes
-   read.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. function:: tcgetpgrp(fd)
-
-   Return the process group associated with the terminal given by *fd* (an open
-   file descriptor as returned by :func:`os.open`).
-
-   Availability: Unix.
-
-
-.. function:: tcsetpgrp(fd, pg)
-
-   Set the process group associated with the terminal given by *fd* (an open file
-   descriptor as returned by :func:`os.open`) to *pg*.
-
-   Availability: Unix.
-
-
-.. function:: ttyname(fd)
-
-   Return a string which specifies the terminal device associated with
-   file descriptor *fd*.  If *fd* is not associated with a terminal device, an
-   exception is raised.
-
-   Availability: Unix.
-
-
-.. function:: write(fd, str)
-
-   Write the bytestring in *str* to file descriptor *fd*. Return the number of
-   bytes actually written.
-
-   Availability: Unix, Windows.
-
-   .. note::
-
-      This function is intended for low-level I/O and must be applied to a file
-      descriptor as returned by :func:`os.open` or :func:`pipe`.  To write a "file
-      object" returned by the built-in function :func:`open` or by :func:`popen` or
-      :func:`fdopen`, or :data:`sys.stdout` or :data:`sys.stderr`, use its
-      :meth:`~file.write` method.
-
-
-.. function:: writev(fd, buffers)
-
-   Write the contents of *buffers* to file descriptor *fd*, where *buffers*
-   is an arbitrary sequence of buffers.
-   Returns the total number of bytes written.
-
-   Availability: Unix.
-
-   .. versionadded:: 3.3
-
-
-.. _open-constants:
-
-``open()`` flag constants
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
 The following constants are options for the *flags* parameter to the
 :func:`~os.open` function.  They can be combined using the bitwise OR operator
 ``|``.  Some of them are not available on all platforms.  For descriptions of
@@ -1142,15 +924,214 @@
    the C library.
 
 
-.. data:: RTLD_LAZY
-          RTLD_NOW
-          RTLD_GLOBAL
-          RTLD_LOCAL
-          RTLD_NODELETE
-          RTLD_NOLOAD
-          RTLD_DEEPBIND
-
-   See the Unix manual page :manpage:`dlopen(3)`.
+.. function:: openpty()
+
+   .. index:: module: pty
+
+   Open a new pseudo-terminal pair. Return a pair of file descriptors ``(master,
+   slave)`` for the pty and the tty, respectively. For a (slightly) more portable
+   approach, use the :mod:`pty` module.
+
+   Availability: some flavors of Unix.
+
+
+.. function:: pipe()
+
+   Create a pipe.  Return a pair of file descriptors ``(r, w)`` usable for reading
+   and writing, respectively.
+
+   Availability: Unix, Windows.
+
+
+.. function:: pipe2(flags)
+
+   Create a pipe with *flags* set atomically.
+   *flags* can be constructed by ORing together one or more of these values:
+   :data:`O_NONBLOCK`, :data:`O_CLOEXEC`.
+   Return a pair of file descriptors ``(r, w)`` usable for reading and writing,
+   respectively.
+
+   Availability: some flavors of Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: posix_fallocate(fd, offset, len)
+
+   Ensures that enough disk space is allocated for the file specified by *fd*
+   starting from *offset* and continuing for *len* bytes.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: posix_fadvise(fd, offset, len, advice)
+
+   Announces an intention to access data in a specific pattern thus allowing
+   the kernel to make optimizations.
+   The advice applies to the region of the file specified by *fd* starting at
+   *offset* and continuing for *len* bytes.
+   *advice* is one of :data:`POSIX_FADV_NORMAL`, :data:`POSIX_FADV_SEQUENTIAL`,
+   :data:`POSIX_FADV_RANDOM`, :data:`POSIX_FADV_NOREUSE`,
+   :data:`POSIX_FADV_WILLNEED` or :data:`POSIX_FADV_DONTNEED`.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. data:: POSIX_FADV_NORMAL
+          POSIX_FADV_SEQUENTIAL
+          POSIX_FADV_RANDOM
+          POSIX_FADV_NOREUSE
+          POSIX_FADV_WILLNEED
+          POSIX_FADV_DONTNEED
+
+   Flags that can be used in *advice* in :func:`posix_fadvise` that specify
+   the access pattern that is likely to be used.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: pread(fd, buffersize, offset)
+
+   Read from a file descriptor, *fd*, at a position of *offset*. It will read up
+   to *buffersize* number of bytes. The file offset remains unchanged.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: pwrite(fd, string, offset)
+
+   Write *string* to a file descriptor, *fd*, from *offset*, leaving the file
+   offset unchanged.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: read(fd, n)
+
+   Read at most *n* bytes from file descriptor *fd*. Return a bytestring containing the
+   bytes read.  If the end of the file referred to by *fd* has been reached, an
+   empty bytes object is returned.
+
+   Availability: Unix, Windows.
+
+   .. note::
+
+      This function is intended for low-level I/O and must be applied to a file
+      descriptor as returned by :func:`os.open` or :func:`pipe`.  To read a
+      "file object" returned by the built-in function :func:`open` or by
+      :func:`popen` or :func:`fdopen`, or :data:`sys.stdin`, use its
+      :meth:`~file.read` or :meth:`~file.readline` methods.
+
+
+.. function:: sendfile(out, in, offset, nbytes)
+              sendfile(out, in, offset, nbytes, headers=None, trailers=None, flags=0)
+
+   Copy *nbytes* bytes from file descriptor *in* to file descriptor *out*
+   starting at *offset*.
+   Return the number of bytes sent. When EOF is reached return 0.
+
+   The first function notation is supported by all platforms that define
+   :func:`sendfile`.
+
+   On Linux, if *offset* is given as ``None``, the bytes are read from the
+   current position of *in* and the position of *in* is updated.
+
+   The second case may be used on Mac OS X and FreeBSD where *headers* and
+   *trailers* are arbitrary sequences of buffers that are written before and
+   after the data from *in* is written. It returns the same as the first case.
+
+   On Mac OS X and FreeBSD, a value of 0 for *nbytes* specifies to send until
+   the end of *in* is reached.
+
+   All platforms support sockets as *out* file descriptor, and some platforms
+   allow other types (e.g. regular file, pipe) as well.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. data:: SF_NODISKIO
+          SF_MNOWAIT
+          SF_SYNC
+
+   Parameters to the :func:`sendfile` function, if the implementation supports
+   them.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: readv(fd, buffers)
+
+   Read from a file descriptor into a number of writable buffers. *buffers* is
+   an arbitrary sequence of writable buffers. Returns the total number of bytes
+   read.
+
+   Availability: Unix.
+
+   .. versionadded:: 3.3
+
+
+.. function:: tcgetpgrp(fd)
+
+   Return the process group associated with the terminal given by *fd* (an open
+   file descriptor as returned by :func:`os.open`).
+
+   Availability: Unix.
+
+
+.. function:: tcsetpgrp(fd, pg)
+
+   Set the process group associated with the terminal given by *fd* (an open file
+   descriptor as returned by :func:`os.open`) to *pg*.
+
+   Availability: Unix.
+
+
+.. function:: ttyname(fd)
+
+   Return a string which specifies the terminal device associated with
+   file descriptor *fd*.  If *fd* is not associated with a terminal device, an
+   exception is raised.
+
+   Availability: Unix.
+
+
+.. function:: write(fd, str)
+
+   Write the bytestring in *str* to file descriptor *fd*. Return the number of
+   bytes actually written.
+
+   Availability: Unix, Windows.
+
+   .. note::
+
+      This function is intended for low-level I/O and must be applied to a file
+      descriptor as returned by :func:`os.open` or :func:`pipe`.  To write a "file
+      object" returned by the built-in function :func:`open` or by :func:`popen` or
+      :func:`fdopen`, or :data:`sys.stdout` or :data:`sys.stderr`, use its
+      :meth:`~file.write` method.
+
+
+.. function:: writev(fd, buffers)
+
+   Write the contents of *buffers* to file descriptor *fd*, where *buffers*
+   is an arbitrary sequence of buffers.
+   Returns the total number of bytes written.
+
+   Availability: Unix.
 
    .. versionadded:: 3.3
 
@@ -3048,7 +3029,7 @@
 
 .. versionadded:: 3.3
 
-The following scheduling policies are exposed if they are a supported by the
+The following scheduling policies are exposed if they are supported by the
 operating system.
 
 .. data:: SCHED_OTHER
@@ -3298,6 +3279,19 @@
    The file path of the null device. For example: ``'/dev/null'`` for
    POSIX, ``'nul'`` for Windows.  Also available via :mod:`os.path`.
 
+.. data:: RTLD_LAZY
+          RTLD_NOW
+          RTLD_GLOBAL
+          RTLD_LOCAL
+          RTLD_NODELETE
+          RTLD_NOLOAD
+          RTLD_DEEPBIND
+
+   Flags for use with the :func:`~sys.setdlopenflags` and
+   :func:`~sys.getdlopenflags` functions.  See the Unix manual page
+   :manpage:`dlopen(3)` for what the different flags mean.
+
+   .. versionadded:: 3.3
 
 .. _os-miscfunc:
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 18:26:28 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 21 Jun 2013 18:26:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Add_tests_for_?=
 =?utf-8?q?untested_features_of_the_=27stat=27_module_=28part_of_issue_=23?=
 =?utf-8?q?11016=29?=
Message-ID: <3bcQJN5nllz7LkV@mail.python.org>

http://hg.python.org/cpython/rev/f8ff61f44aca
changeset:   84241:f8ff61f44aca
branch:      3.3
parent:      84237:12478f549ed8
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 21 18:25:56 2013 +0200
summary:
  Add tests for untested features of the 'stat' module (part of issue #11016)

files:
  Lib/test/test_stat.py |  149 ++++++++++++++++++++++++++---
  1 files changed, 134 insertions(+), 15 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -1,14 +1,52 @@
 import unittest
 import os
+from test.support import TESTFN, run_unittest, import_fresh_module
 import stat
-from test.support import TESTFN, run_unittest
-
-
-def get_mode(fname=TESTFN):
-    return stat.filemode(os.lstat(fname).st_mode)
-
 
 class TestFilemode(unittest.TestCase):
+    file_flags = {'SF_APPEND', 'SF_ARCHIVED', 'SF_IMMUTABLE', 'SF_NOUNLINK',
+                  'SF_SNAPSHOT', 'UF_APPEND', 'UF_COMPRESSED', 'UF_HIDDEN',
+                  'UF_IMMUTABLE', 'UF_NODUMP', 'UF_NOUNLINK', 'UF_OPAQUE'}
+
+    formats = {'S_IFBLK', 'S_IFCHR', 'S_IFDIR', 'S_IFIFO', 'S_IFLNK',
+               'S_IFREG', 'S_IFSOCK'}
+
+    format_funcs = {'S_ISBLK', 'S_ISCHR', 'S_ISDIR', 'S_ISFIFO', 'S_ISLNK',
+                    'S_ISREG', 'S_ISSOCK'}
+
+    stat_struct = {
+        'ST_MODE': 0,
+        'ST_INO': 1,
+        'ST_DEV': 2,
+        'ST_NLINK': 3,
+        'ST_UID': 4,
+        'ST_GID': 5,
+        'ST_SIZE': 6,
+        'ST_ATIME': 7,
+        'ST_MTIME': 8,
+        'ST_CTIME': 9}
+
+    # permission bit value are defined by POSIX
+    permission_bits = {
+        'S_ISUID': 0o4000,
+        'S_ISGID': 0o2000,
+        'S_ENFMT': 0o2000,
+        'S_ISVTX': 0o1000,
+        'S_IRWXU': 0o700,
+        'S_IRUSR': 0o400,
+        'S_IREAD': 0o400,
+        'S_IWUSR': 0o200,
+        'S_IWRITE': 0o200,
+        'S_IXUSR': 0o100,
+        'S_IEXEC': 0o100,
+        'S_IRWXG': 0o070,
+        'S_IRGRP': 0o040,
+        'S_IWGRP': 0o020,
+        'S_IXGRP': 0o010,
+        'S_IRWXO': 0o007,
+        'S_IROTH': 0o004,
+        'S_IWOTH': 0o002,
+        'S_IXOTH': 0o001}
 
     def setUp(self):
         try:
@@ -20,29 +58,75 @@
                 pass
     tearDown = setUp
 
+    def get_mode(self, fname=TESTFN):
+        st_mode = os.lstat(fname).st_mode
+        modestr = stat.filemode(st_mode)
+        return st_mode, modestr
+
+    def assertS_IS(self, name, mode):
+        # test format, lstrip is for S_IFIFO
+        fmt = getattr(stat, "S_IF" + name.lstrip("F"))
+        self.assertEqual(stat.S_IFMT(mode), fmt)
+        # test that just one function returns true
+        testname = "S_IS" + name
+        for funcname in self.format_funcs:
+            func = getattr(stat, funcname, None)
+            if func is None:
+                if funcname == testname:
+                    raise ValueError(funcname)
+                continue
+            if funcname == testname:
+                self.assertTrue(func(mode))
+            else:
+                self.assertFalse(func(mode))
+
     def test_mode(self):
         with open(TESTFN, 'w'):
             pass
         if os.name == 'posix':
             os.chmod(TESTFN, 0o700)
-            self.assertEqual(get_mode(), '-rwx------')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr, '-rwx------')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXU)
+
             os.chmod(TESTFN, 0o070)
-            self.assertEqual(get_mode(), '----rwx---')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr, '----rwx---')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXG)
+
             os.chmod(TESTFN, 0o007)
-            self.assertEqual(get_mode(), '-------rwx')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr, '-------rwx')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXO)
+
             os.chmod(TESTFN, 0o444)
-            self.assertEqual(get_mode(), '-r--r--r--')
+            st_mode, modestr = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(modestr, '-r--r--r--')
+            self.assertEqual(stat.S_IMODE(st_mode), 0o444)
         else:
             os.chmod(TESTFN, 0o700)
-            self.assertEqual(get_mode()[:3], '-rw')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr[:3], '-rw')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IFMT(st_mode),
+                             stat.S_IFREG)
 
     def test_directory(self):
         os.mkdir(TESTFN)
         os.chmod(TESTFN, 0o700)
+        st_mode, modestr = self.get_mode()
+        self.assertS_IS("DIR", st_mode)
         if os.name == 'posix':
-            self.assertEqual(get_mode(), 'drwx------')
+            self.assertEqual(modestr, 'drwx------')
         else:
-            self.assertEqual(get_mode()[0], 'd')
+            self.assertEqual(modestr[0], 'd')
 
     @unittest.skipUnless(hasattr(os, 'symlink'), 'os.symlink not available')
     def test_link(self):
@@ -51,12 +135,47 @@
         except (OSError, NotImplementedError) as err:
             raise unittest.SkipTest(str(err))
         else:
-            self.assertEqual(get_mode()[0], 'l')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr[0], 'l')
+            self.assertS_IS("LNK", st_mode)
 
     @unittest.skipUnless(hasattr(os, 'mkfifo'), 'os.mkfifo not available')
     def test_fifo(self):
         os.mkfifo(TESTFN, 0o700)
-        self.assertEqual(get_mode(), 'prwx------')
+        st_mode, modestr = self.get_mode()
+        self.assertEqual(modestr, 'prwx------')
+        self.assertS_IS("FIFO", st_mode)
+
+    @unittest.skipUnless(os.name == 'posix', 'requires Posix')
+    def test_devices(self):
+        if os.path.exists(os.devnull):
+            st_mode, modestr = self.get_mode(os.devnull)
+            self.assertEqual(modestr[0], 'c')
+            self.assertS_IS("CHR", st_mode)
+        for blockdev in ("/dev/sda", "/dev/hda", "/dev/da0", "/dev/ad0"):
+            if os.path.exists(blockdev):
+                st_mode, modestr = self.get_mode(blockdev)
+                self.assertEqual(modestr[0], 'b')
+                self.assertS_IS("BLK", st_mode)
+                break
+
+    def test_module_attributes(self):
+        for key, value in self.stat_struct.items():
+            modvalue = getattr(stat, key)
+            self.assertEqual(value, modvalue, key)
+        for key, value in self.permission_bits.items():
+            modvalue = getattr(stat, key)
+            self.assertEqual(value, modvalue, key)
+        for key in self.file_flags:
+            modvalue = getattr(stat, key)
+            self.assertIsInstance(modvalue, int)
+        for key in self.formats:
+            modvalue = getattr(stat, key)
+            self.assertIsInstance(modvalue, int)
+        for key in self.format_funcs:
+            func = getattr(stat, key)
+            self.assertTrue(callable(func))
+            self.assertEqual(func(0), 0)
 
 
 def test_main():

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 18:26:30 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 21 Jun 2013 18:26:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Add_tests_for_untested_features_of_the_=27stat=27_module?=
 =?utf-8?q?_=28part_of_issue_=2311016=29?=
Message-ID: <3bcQJQ1gf7z7Lkd@mail.python.org>

http://hg.python.org/cpython/rev/d15aee50e4a0
changeset:   84242:d15aee50e4a0
parent:      84240:88edac3bc2fc
parent:      84241:f8ff61f44aca
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 21 18:26:05 2013 +0200
summary:
  Add tests for untested features of the 'stat' module (part of issue #11016)

files:
  Lib/test/test_stat.py |  149 ++++++++++++++++++++++++++---
  1 files changed, 134 insertions(+), 15 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -1,14 +1,52 @@
 import unittest
 import os
+from test.support import TESTFN, run_unittest, import_fresh_module
 import stat
-from test.support import TESTFN, run_unittest
-
-
-def get_mode(fname=TESTFN):
-    return stat.filemode(os.lstat(fname).st_mode)
-
 
 class TestFilemode(unittest.TestCase):
+    file_flags = {'SF_APPEND', 'SF_ARCHIVED', 'SF_IMMUTABLE', 'SF_NOUNLINK',
+                  'SF_SNAPSHOT', 'UF_APPEND', 'UF_COMPRESSED', 'UF_HIDDEN',
+                  'UF_IMMUTABLE', 'UF_NODUMP', 'UF_NOUNLINK', 'UF_OPAQUE'}
+
+    formats = {'S_IFBLK', 'S_IFCHR', 'S_IFDIR', 'S_IFIFO', 'S_IFLNK',
+               'S_IFREG', 'S_IFSOCK'}
+
+    format_funcs = {'S_ISBLK', 'S_ISCHR', 'S_ISDIR', 'S_ISFIFO', 'S_ISLNK',
+                    'S_ISREG', 'S_ISSOCK'}
+
+    stat_struct = {
+        'ST_MODE': 0,
+        'ST_INO': 1,
+        'ST_DEV': 2,
+        'ST_NLINK': 3,
+        'ST_UID': 4,
+        'ST_GID': 5,
+        'ST_SIZE': 6,
+        'ST_ATIME': 7,
+        'ST_MTIME': 8,
+        'ST_CTIME': 9}
+
+    # permission bit value are defined by POSIX
+    permission_bits = {
+        'S_ISUID': 0o4000,
+        'S_ISGID': 0o2000,
+        'S_ENFMT': 0o2000,
+        'S_ISVTX': 0o1000,
+        'S_IRWXU': 0o700,
+        'S_IRUSR': 0o400,
+        'S_IREAD': 0o400,
+        'S_IWUSR': 0o200,
+        'S_IWRITE': 0o200,
+        'S_IXUSR': 0o100,
+        'S_IEXEC': 0o100,
+        'S_IRWXG': 0o070,
+        'S_IRGRP': 0o040,
+        'S_IWGRP': 0o020,
+        'S_IXGRP': 0o010,
+        'S_IRWXO': 0o007,
+        'S_IROTH': 0o004,
+        'S_IWOTH': 0o002,
+        'S_IXOTH': 0o001}
 
     def setUp(self):
         try:
@@ -20,29 +58,75 @@
                 pass
     tearDown = setUp
 
+    def get_mode(self, fname=TESTFN):
+        st_mode = os.lstat(fname).st_mode
+        modestr = stat.filemode(st_mode)
+        return st_mode, modestr
+
+    def assertS_IS(self, name, mode):
+        # test format, lstrip is for S_IFIFO
+        fmt = getattr(stat, "S_IF" + name.lstrip("F"))
+        self.assertEqual(stat.S_IFMT(mode), fmt)
+        # test that just one function returns true
+        testname = "S_IS" + name
+        for funcname in self.format_funcs:
+            func = getattr(stat, funcname, None)
+            if func is None:
+                if funcname == testname:
+                    raise ValueError(funcname)
+                continue
+            if funcname == testname:
+                self.assertTrue(func(mode))
+            else:
+                self.assertFalse(func(mode))
+
     def test_mode(self):
         with open(TESTFN, 'w'):
             pass
         if os.name == 'posix':
             os.chmod(TESTFN, 0o700)
-            self.assertEqual(get_mode(), '-rwx------')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr, '-rwx------')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXU)
+
             os.chmod(TESTFN, 0o070)
-            self.assertEqual(get_mode(), '----rwx---')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr, '----rwx---')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXG)
+
             os.chmod(TESTFN, 0o007)
-            self.assertEqual(get_mode(), '-------rwx')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr, '-------rwx')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXO)
+
             os.chmod(TESTFN, 0o444)
-            self.assertEqual(get_mode(), '-r--r--r--')
+            st_mode, modestr = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(modestr, '-r--r--r--')
+            self.assertEqual(stat.S_IMODE(st_mode), 0o444)
         else:
             os.chmod(TESTFN, 0o700)
-            self.assertEqual(get_mode()[:3], '-rw')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr[:3], '-rw')
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IFMT(st_mode),
+                             stat.S_IFREG)
 
     def test_directory(self):
         os.mkdir(TESTFN)
         os.chmod(TESTFN, 0o700)
+        st_mode, modestr = self.get_mode()
+        self.assertS_IS("DIR", st_mode)
         if os.name == 'posix':
-            self.assertEqual(get_mode(), 'drwx------')
+            self.assertEqual(modestr, 'drwx------')
         else:
-            self.assertEqual(get_mode()[0], 'd')
+            self.assertEqual(modestr[0], 'd')
 
     @unittest.skipUnless(hasattr(os, 'symlink'), 'os.symlink not available')
     def test_link(self):
@@ -51,12 +135,47 @@
         except (OSError, NotImplementedError) as err:
             raise unittest.SkipTest(str(err))
         else:
-            self.assertEqual(get_mode()[0], 'l')
+            st_mode, modestr = self.get_mode()
+            self.assertEqual(modestr[0], 'l')
+            self.assertS_IS("LNK", st_mode)
 
     @unittest.skipUnless(hasattr(os, 'mkfifo'), 'os.mkfifo not available')
     def test_fifo(self):
         os.mkfifo(TESTFN, 0o700)
-        self.assertEqual(get_mode(), 'prwx------')
+        st_mode, modestr = self.get_mode()
+        self.assertEqual(modestr, 'prwx------')
+        self.assertS_IS("FIFO", st_mode)
+
+    @unittest.skipUnless(os.name == 'posix', 'requires Posix')
+    def test_devices(self):
+        if os.path.exists(os.devnull):
+            st_mode, modestr = self.get_mode(os.devnull)
+            self.assertEqual(modestr[0], 'c')
+            self.assertS_IS("CHR", st_mode)
+        for blockdev in ("/dev/sda", "/dev/hda", "/dev/da0", "/dev/ad0"):
+            if os.path.exists(blockdev):
+                st_mode, modestr = self.get_mode(blockdev)
+                self.assertEqual(modestr[0], 'b')
+                self.assertS_IS("BLK", st_mode)
+                break
+
+    def test_module_attributes(self):
+        for key, value in self.stat_struct.items():
+            modvalue = getattr(stat, key)
+            self.assertEqual(value, modvalue, key)
+        for key, value in self.permission_bits.items():
+            modvalue = getattr(stat, key)
+            self.assertEqual(value, modvalue, key)
+        for key in self.file_flags:
+            modvalue = getattr(stat, key)
+            self.assertIsInstance(modvalue, int)
+        for key in self.formats:
+            modvalue = getattr(stat, key)
+            self.assertIsInstance(modvalue, int)
+        for key in self.format_funcs:
+            func = getattr(stat, key)
+            self.assertTrue(callable(func))
+            self.assertEqual(func(0), 0)
 
 
 def test_main():

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 18:53:36 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 21 Jun 2013 18:53:36 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogRml4IHRlc3Rfc3Rh?=
 =?utf-8?q?t_on_BSD=2C_/dev/da0_and_/dev/ad0_are_links?=
Message-ID: <3bcQvh1HNvzRSf@mail.python.org>

http://hg.python.org/cpython/rev/ea8359a13ce7
changeset:   84243:ea8359a13ce7
branch:      3.3
parent:      84241:f8ff61f44aca
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 21 18:53:13 2013 +0200
summary:
  Fix test_stat on BSD, /dev/da0 and /dev/ad0 are links

files:
  Lib/test/test_stat.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -152,7 +152,8 @@
             st_mode, modestr = self.get_mode(os.devnull)
             self.assertEqual(modestr[0], 'c')
             self.assertS_IS("CHR", st_mode)
-        for blockdev in ("/dev/sda", "/dev/hda", "/dev/da0", "/dev/ad0"):
+        # needs block devices in BSD, /dev/da0, /dev/ad0 are links
+        for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
                 st_mode, modestr = self.get_mode(blockdev)
                 self.assertEqual(modestr[0], 'b')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 18:53:37 2013
From: python-checkins at python.org (christian.heimes)
Date: Fri, 21 Jun 2013 18:53:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Fix_test=5Fstat_on_BSD=2C_/dev/da0_and_/dev/ad0_are_link?=
 =?utf-8?q?s?=
Message-ID: <3bcQvj3gC2z7LjZ@mail.python.org>

http://hg.python.org/cpython/rev/91af2094074a
changeset:   84244:91af2094074a
parent:      84242:d15aee50e4a0
parent:      84243:ea8359a13ce7
user:        Christian Heimes <christian at cheimes.de>
date:        Fri Jun 21 18:53:27 2013 +0200
summary:
  Fix test_stat on BSD, /dev/da0 and /dev/ad0 are links

files:
  Lib/test/test_stat.py |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -152,7 +152,8 @@
             st_mode, modestr = self.get_mode(os.devnull)
             self.assertEqual(modestr[0], 'c')
             self.assertS_IS("CHR", st_mode)
-        for blockdev in ("/dev/sda", "/dev/hda", "/dev/da0", "/dev/ad0"):
+        # needs block devices in BSD, /dev/da0, /dev/ad0 are links
+        for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
                 st_mode, modestr = self.get_mode(blockdev)
                 self.assertEqual(modestr[0], 'b')

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 21 19:09:33 2013
From: python-checkins at python.org (victor.stinner)
Date: Fri, 21 Jun 2013 19:09:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_marshal=3A_optimize_parsin?=
	=?utf-8?q?g_of_empty_Unicode_strings?=
Message-ID: <3bcRG522BTzNRr@mail.python.org>

http://hg.python.org/cpython/rev/942a7061e8ad
changeset:   84245:942a7061e8ad
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Jun 21 19:08:06 2013 +0200
summary:
  marshal: optimize parsing of empty Unicode strings

Don't create a temporary buffer of zeroy byte nor call r_string() if the length
is zero, create directly the empty string.

files:
  Python/marshal.c |  29 +++++++++++++++++------------
  1 files changed, 17 insertions(+), 12 deletions(-)


diff --git a/Python/marshal.c b/Python/marshal.c
--- a/Python/marshal.c
+++ b/Python/marshal.c
@@ -979,20 +979,25 @@
             retval = NULL;
             break;
         }
-        buffer = PyMem_NEW(char, n);
-        if (buffer == NULL) {
-            retval = PyErr_NoMemory();
-            break;
+        if (n != 0) {
+            buffer = PyMem_NEW(char, n);
+            if (buffer == NULL) {
+                retval = PyErr_NoMemory();
+                break;
+            }
+            if (r_string(buffer, n, p) != n) {
+                PyMem_DEL(buffer);
+                PyErr_SetString(PyExc_EOFError,
+                    "EOF read where object expected");
+                retval = NULL;
+                break;
+            }
+            v = PyUnicode_DecodeUTF8(buffer, n, "surrogatepass");
+            PyMem_DEL(buffer);
         }
-        if (r_string(buffer, n, p) != n) {
-            PyMem_DEL(buffer);
-            PyErr_SetString(PyExc_EOFError,
-                "EOF read where object expected");
-            retval = NULL;
-            break;
+        else {
+            v = PyUnicode_New(0, 0);
         }
-        v = PyUnicode_DecodeUTF8(buffer, n, "surrogatepass");
-        PyMem_DEL(buffer);
         if (type == TYPE_INTERNED)
             PyUnicode_InternInPlace(&v);
         retval = v;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 00:37:11 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 22 Jun 2013 00:37:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4Mjc4?=
 =?utf-8?q?=3A_properly_document_how_the_loaders_are_called_for_FileFinder?=
Message-ID: <3bcZX71N9lz7Ljc@mail.python.org>

http://hg.python.org/cpython/rev/6978d7a6692a
changeset:   84246:6978d7a6692a
branch:      3.3
parent:      84243:ea8359a13ce7
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 21 18:31:55 2013 -0400
summary:
  Issue #18278: properly document how the loaders are called for FileFinder

files:
  Doc/library/importlib.rst |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -697,6 +697,8 @@
 
    The *loader_details* argument is a variable number of 2-item tuples each
    containing a loader and a sequence of file suffixes the loader recognizes.
+   The loaders are expected to be callables which accept two arguments of
+   the module's name and the path to the file found.
 
    The finder will cache the directory contents as necessary, making stat calls
    for each module search to verify the cache is not outdated. Because cache

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 00:37:12 2013
From: python-checkins at python.org (brett.cannon)
Date: Sat, 22 Jun 2013 00:37:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_merge_for_issue_=2318278?=
Message-ID: <3bcZX83FpPz7Ljr@mail.python.org>

http://hg.python.org/cpython/rev/a089a8b1f93d
changeset:   84247:a089a8b1f93d
parent:      84245:942a7061e8ad
parent:      84246:6978d7a6692a
user:        Brett Cannon <brett at python.org>
date:        Fri Jun 21 18:37:02 2013 -0400
summary:
  merge for issue #18278

files:
  Doc/library/importlib.rst |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -733,6 +733,8 @@
 
    The *loader_details* argument is a variable number of 2-item tuples each
    containing a loader and a sequence of file suffixes the loader recognizes.
+   The loaders are expected to be callables which accept two arguments of
+   the module's name and the path to the file found.
 
    The finder will cache the directory contents as necessary, making stat calls
    for each module search to verify the cache is not outdated. Because cache

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Jun 22 05:43:42 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 22 Jun 2013 05:43:42 +0200
Subject: [Python-checkins] Daily reference leaks (a089a8b1f93d): sum=7
Message-ID: <E1UqEjm-0006tH-4o@ap.vmr.nerim.net>

results for a089a8b1f93d on branch "default"
--------------------------------------------

test_robotparser leaked [0, 6, 1] memory blocks, sum=7


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogoCd2G5', '-x']

From python-checkins at python.org  Sat Jun 22 09:55:38 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Sat, 22 Jun 2013 09:55:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Arrange_struct?=
 =?utf-8?q?ure_to_match_the_common_access_patterns=2E?=
Message-ID: <3bcpwV5SfqzNFr@mail.python.org>

http://hg.python.org/cpython/rev/f1dc30a1be72
changeset:   84248:f1dc30a1be72
branch:      2.7
parent:      84223:6f535c725b27
user:        Raymond Hettinger <python at rcn.com>
date:        Sat Jun 22 00:51:01 2013 -0700
summary:
  Arrange structure to match the common access patterns.

files:
  Modules/_collectionsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -48,8 +48,8 @@
 
 typedef struct BLOCK {
     struct BLOCK *leftlink;
+    PyObject *data[BLOCKLEN];
     struct BLOCK *rightlink;
-    PyObject *data[BLOCKLEN];
 } block;
 
 #define MAXFREEBLOCKS 10

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 14:48:51 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 22 Jun 2013 14:48:51 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQlNEOiBibG9jayBk?=
	=?utf-8?q?evices_are_gone?=
Message-ID: <3bcxQq545pzSgZ@mail.python.org>

http://hg.python.org/cpython/rev/bfe5507423b1
changeset:   84249:bfe5507423b1
branch:      3.3
parent:      84246:6978d7a6692a
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 22 14:48:32 2013 +0200
summary:
  BSD: block devices are gone
http://www.freebsd.org/doc/en/books/arch-handbook/driverbasics-block.html

files:
  Lib/test/test_stat.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -152,7 +152,7 @@
             st_mode, modestr = self.get_mode(os.devnull)
             self.assertEqual(modestr[0], 'c')
             self.assertS_IS("CHR", st_mode)
-        # needs block devices in BSD, /dev/da0, /dev/ad0 are links
+        # Linux block devices, BSD has no block devices anymore
         for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
                 st_mode, modestr = self.get_mode(blockdev)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 14:48:52 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 22 Jun 2013 14:48:52 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_BSD=3A_block_devices_are_gone?=
Message-ID: <3bcxQr756YzSgZ@mail.python.org>

http://hg.python.org/cpython/rev/4465f273a8a4
changeset:   84250:4465f273a8a4
parent:      84247:a089a8b1f93d
parent:      84249:bfe5507423b1
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 22 14:48:42 2013 +0200
summary:
  BSD: block devices are gone
http://www.freebsd.org/doc/en/books/arch-handbook/driverbasics-block.html

files:
  Lib/test/test_stat.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -152,7 +152,7 @@
             st_mode, modestr = self.get_mode(os.devnull)
             self.assertEqual(modestr[0], 'c')
             self.assertS_IS("CHR", st_mode)
-        # needs block devices in BSD, /dev/da0, /dev/ad0 are links
+        # Linux block devices, BSD has no block devices anymore
         for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
                 st_mode, modestr = self.get_mode(blockdev)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 18:33:21 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 22 Jun 2013 18:33:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTEzOiBhdm9p?=
 =?utf-8?q?d_segfault_if_Py=5FXDECREF_triggers_code_that_calls_set=5Fpanel?=
 =?utf-8?q?=5Fuserptr?=
Message-ID: <3bd2Ps3TPZzSg9@mail.python.org>

http://hg.python.org/cpython/rev/99733ff98a50
changeset:   84251:99733ff98a50
branch:      2.7
parent:      84248:f1dc30a1be72
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 22 12:33:05 2013 -0400
summary:
  #18113: avoid segfault if Py_XDECREF triggers code that calls set_panel_userptr again

Problem noted & original patch by Serhiy Storchaka; I tweaked the patch a bit.

files:
  Lib/test/test_curses.py |   9 +++++++++
  Modules/_curses_panel.c |  11 ++++++++---
  2 files changed, 17 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -262,6 +262,14 @@
     if sys.getrefcount(obj) != nrefs:
         raise RuntimeError, "set_userptr leaked references"
 
+def test_userptr_segfault(stdscr):
+    panel = curses.panel.new_panel(stdscr)
+    class A:
+        def __del__(self):
+            panel.set_userptr(None)
+    panel.set_userptr(A())
+    panel.set_userptr(None)
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -281,6 +289,7 @@
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
         test_userptr_memory_leak(stdscr)
+        test_userptr_segfault(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
     finally:
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -294,12 +294,17 @@
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
     PyObject *oldobj;
+    int rc;
     PyCursesInitialised;
+    Py_INCREF(obj);
     oldobj = (PyObject *) panel_userptr(self->pan);
+    rc = set_panel_userptr(self->pan, (void*)obj);
+    if (rc == ERR) {
+        /* In case of an ncurses error, decref the new object again */
+        Py_DECREF(obj);
+    }
     Py_XDECREF(oldobj);
-    Py_INCREF(obj);
-    return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
-                            "set_panel_userptr");
+    return PyCursesCheckERR(rc, "set_panel_userptr");
 }
 
 static PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 18:58:59 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 22 Jun 2013 18:58:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_transplant_tes?=
 =?utf-8?q?t=5Fstat_from_default_to_2=2E7_in_order_to_make_sure_it_works_o?=
 =?utf-8?q?n_all?=
Message-ID: <3bd2zR6pg9zQlG@mail.python.org>

http://hg.python.org/cpython/rev/d7e2fe965fdd
changeset:   84252:d7e2fe965fdd
branch:      2.7
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 22 18:58:51 2013 +0200
summary:
  transplant test_stat from default to 2.7 in order to make sure it works on all supported platforms

files:
  Lib/test/test_stat.py |  171 ++++++++++++++++++++++++++++++
  1 files changed, 171 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
new file mode 100644
--- /dev/null
+++ b/Lib/test/test_stat.py
@@ -0,0 +1,171 @@
+import unittest
+import os
+from test.test_support import TESTFN, run_unittest
+import stat
+
+class TestFilemode(unittest.TestCase):
+    file_flags = {'SF_APPEND', 'SF_ARCHIVED', 'SF_IMMUTABLE', 'SF_NOUNLINK',
+                  'SF_SNAPSHOT', 'UF_APPEND', 'UF_COMPRESSED', 'UF_HIDDEN',
+                  'UF_IMMUTABLE', 'UF_NODUMP', 'UF_NOUNLINK', 'UF_OPAQUE'}
+
+    formats = {'S_IFBLK', 'S_IFCHR', 'S_IFDIR', 'S_IFIFO', 'S_IFLNK',
+               'S_IFREG', 'S_IFSOCK'}
+
+    format_funcs = {'S_ISBLK', 'S_ISCHR', 'S_ISDIR', 'S_ISFIFO', 'S_ISLNK',
+                    'S_ISREG', 'S_ISSOCK'}
+
+    stat_struct = {
+        'ST_MODE': 0,
+        'ST_INO': 1,
+        'ST_DEV': 2,
+        'ST_NLINK': 3,
+        'ST_UID': 4,
+        'ST_GID': 5,
+        'ST_SIZE': 6,
+        'ST_ATIME': 7,
+        'ST_MTIME': 8,
+        'ST_CTIME': 9}
+
+    # permission bit value are defined by POSIX
+    permission_bits = {
+        'S_ISUID': 0o4000,
+        'S_ISGID': 0o2000,
+        'S_ENFMT': 0o2000,
+        'S_ISVTX': 0o1000,
+        'S_IRWXU': 0o700,
+        'S_IRUSR': 0o400,
+        'S_IREAD': 0o400,
+        'S_IWUSR': 0o200,
+        'S_IWRITE': 0o200,
+        'S_IXUSR': 0o100,
+        'S_IEXEC': 0o100,
+        'S_IRWXG': 0o070,
+        'S_IRGRP': 0o040,
+        'S_IWGRP': 0o020,
+        'S_IXGRP': 0o010,
+        'S_IRWXO': 0o007,
+        'S_IROTH': 0o004,
+        'S_IWOTH': 0o002,
+        'S_IXOTH': 0o001}
+
+    def setUp(self):
+        try:
+            os.remove(TESTFN)
+        except OSError:
+            try:
+                os.rmdir(TESTFN)
+            except OSError:
+                pass
+    tearDown = setUp
+
+    def get_mode(self, fname=TESTFN):
+        return os.lstat(fname).st_mode
+
+    def assertS_IS(self, name, mode):
+        # test format, lstrip is for S_IFIFO
+        fmt = getattr(stat, "S_IF" + name.lstrip("F"))
+        self.assertEqual(stat.S_IFMT(mode), fmt)
+        # test that just one function returns true
+        testname = "S_IS" + name
+        for funcname in self.format_funcs:
+            func = getattr(stat, funcname, None)
+            if func is None:
+                if funcname == testname:
+                    raise ValueError(funcname)
+                continue
+            if funcname == testname:
+                self.assertTrue(func(mode))
+            else:
+                self.assertFalse(func(mode))
+
+    def test_mode(self):
+        with open(TESTFN, 'w'):
+            pass
+        if os.name == 'posix':
+            os.chmod(TESTFN, 0o700)
+            st_mode = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXU)
+
+            os.chmod(TESTFN, 0o070)
+            st_mode = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXG)
+
+            os.chmod(TESTFN, 0o007)
+            st_mode = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode),
+                             stat.S_IRWXO)
+
+            os.chmod(TESTFN, 0o444)
+            st_mode = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IMODE(st_mode), 0o444)
+        else:
+            os.chmod(TESTFN, 0o700)
+            st_mode = self.get_mode()
+            self.assertS_IS("REG", st_mode)
+            self.assertEqual(stat.S_IFMT(st_mode),
+                             stat.S_IFREG)
+
+    def test_directory(self):
+        os.mkdir(TESTFN)
+        os.chmod(TESTFN, 0o700)
+        st_mode = self.get_mode()
+        self.assertS_IS("DIR", st_mode)
+
+    @unittest.skipUnless(hasattr(os, 'symlink'), 'os.symlink not available')
+    def test_link(self):
+        try:
+            os.symlink(os.getcwd(), TESTFN)
+        except (OSError, NotImplementedError) as err:
+            raise unittest.SkipTest(str(err))
+        else:
+            st_mode = self.get_mode()
+            self.assertS_IS("LNK", st_mode)
+
+    @unittest.skipUnless(hasattr(os, 'mkfifo'), 'os.mkfifo not available')
+    def test_fifo(self):
+        os.mkfifo(TESTFN, 0o700)
+        st_mode = self.get_mode()
+        self.assertS_IS("FIFO", st_mode)
+
+    @unittest.skipUnless(os.name == 'posix', 'requires Posix')
+    def test_devices(self):
+        if os.path.exists(os.devnull):
+            st_mode = self.get_mode(os.devnull)
+            self.assertS_IS("CHR", st_mode)
+        # Linux block devices, BSD has no block devices anymore
+        for blockdev in ("/dev/sda", "/dev/hda"):
+            if os.path.exists(blockdev):
+                st_mode = self.get_mode(blockdev)
+                self.assertS_IS("BLK", st_mode)
+                break
+
+    def test_module_attributes(self):
+        for key, value in self.stat_struct.items():
+            modvalue = getattr(stat, key)
+            self.assertEqual(value, modvalue, key)
+        for key, value in self.permission_bits.items():
+            modvalue = getattr(stat, key)
+            self.assertEqual(value, modvalue, key)
+        for key in self.file_flags:
+            modvalue = getattr(stat, key)
+            self.assertIsInstance(modvalue, int)
+        for key in self.formats:
+            modvalue = getattr(stat, key)
+            self.assertIsInstance(modvalue, int)
+        for key in self.format_funcs:
+            func = getattr(stat, key)
+            self.assertTrue(callable(func))
+            self.assertEqual(func(0), 0)
+
+
+def test_main():
+    run_unittest(TestFilemode)
+
+if __name__ == '__main__':
+    test_main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 19:34:38 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 22 Jun 2013 19:34:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_my_SSL_module_patches_?=
	=?utf-8?q?to_whatsnew_3=2E4?=
Message-ID: <3bd3mZ5SvzzPVY@mail.python.org>

http://hg.python.org/cpython/rev/279ab3ec3e64
changeset:   84253:279ab3ec3e64
parent:      84250:4465f273a8a4
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 22 19:31:58 2013 +0200
summary:
  Add my SSL module patches to whatsnew 3.4

files:
  Doc/whatsnew/3.4.rst |  12 ++++++++++++
  1 files changed, 12 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -191,6 +191,18 @@
 try/except statement by code that only cares whether or not an error occurred.
 (:issue:`2118`).
 
+ssl
+---
+
+New diagnostic functions :func:`~ssl.get_default_verify_paths`,
+:meth:`~ssl.SSLContext.cert_store_stats` and
+:meth:`~ssl.SSLContext.get_ca_certs`
+
+Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows'
+cert store.
+
+(Contributed by Christian Heimes in :issue:`18143`, :issue:`18147` and
+:issue:`17134`)
 
 wave
 ----

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 19:34:40 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 22 Jun 2013 19:34:40 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogLi4uIGFsc28gVExTdjEuMSBh?=
	=?utf-8?q?nd_TLSv1=2E2?=
Message-ID: <3bd3mc0Gl5zRFn@mail.python.org>

http://hg.python.org/cpython/rev/7fb25b77e327
changeset:   84254:7fb25b77e327
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 22 19:34:17 2013 +0200
summary:
  ... also TLSv1.1 and TLSv1.2

files:
  Doc/whatsnew/3.4.rst |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -194,6 +194,9 @@
 ssl
 ---
 
+TLSv1.1 and TLSv1.2 support (Contributed by Michele Orr? and Antoine Pitrou
+in :issue:`16692`)
+
 New diagnostic functions :func:`~ssl.get_default_verify_paths`,
 :meth:`~ssl.SSLContext.cert_store_stats` and
 :meth:`~ssl.SSLContext.get_ca_certs`

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 20:16:47 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Sat, 22 Jun 2013 20:16:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_backout_5accb0?=
 =?utf-8?q?ac8bfb=3B_needs_more_discussion_on_python-dev?=
Message-ID: <3bd4jC3DPNzRmN@mail.python.org>

http://hg.python.org/cpython/rev/07eb090d8938
changeset:   84255:07eb090d8938
branch:      2.7
parent:      84252:d7e2fe965fdd
user:        Benjamin Peterson <benjamin at python.org>
date:        Sat Jun 22 11:16:36 2013 -0700
summary:
  backout 5accb0ac8bfb; needs more discussion on python-dev

files:
  Lib/test/test_deque.py       |   2 +-
  Modules/_collectionsmodule.c |  18 +++++++++++-------
  2 files changed, 12 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py
--- a/Lib/test/test_deque.py
+++ b/Lib/test/test_deque.py
@@ -522,7 +522,7 @@
 
     @test_support.cpython_only
     def test_sizeof(self):
-        BLOCKLEN = 64
+        BLOCKLEN = 62
         basesize = test_support.calcobjsize('2P4PlP')
         blocksize = struct.calcsize('2P%dP' % BLOCKLEN)
         self.assertEqual(object.__sizeof__(deque()), basesize)
diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -8,13 +8,12 @@
 */
 
 /* The block length may be set to any number over 1.  Larger numbers
- * reduce the number of calls to the memory allocator, give faster
- * indexing and rotation, and reduce the link::data overhead ratio.
- * Ideally, the block length should be a power-of-two for faster
- * division/modulo computations during indexing.
+ * reduce the number of calls to the memory allocator but take more
+ * memory.  Ideally, BLOCKLEN should be set with an eye to the
+ * length of a cache line.
  */
 
-#define BLOCKLEN 64
+#define BLOCKLEN 62
 #define CENTER ((BLOCKLEN - 1) / 2)
 
 /* A `dequeobject` is composed of a doubly-linked list of `block` nodes.
@@ -59,8 +58,13 @@
 static block *
 newblock(block *leftlink, block *rightlink, Py_ssize_t len) {
     block *b;
-    /* To prevent len from overflowing PY_SSIZE_T_MAX on 32-bit machines, we
-     * refuse to allocate new blocks if the current len is nearing overflow. */
+    /* To prevent len from overflowing PY_SSIZE_T_MAX on 64-bit machines, we
+     * refuse to allocate new blocks if the current len is dangerously
+     * close.  There is some extra margin to prevent spurious arithmetic
+     * overflows at various places.  The following check ensures that
+     * the blocks allocated to the deque, in the worst case, can only
+     * have PY_SSIZE_T_MAX-2 entries in total.
+     */
     if (len >= PY_SSIZE_T_MAX - 2*BLOCKLEN) {
         PyErr_SetString(PyExc_OverflowError,
                         "cannot add more blocks to the deque");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 20:51:15 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 22 Jun 2013 20:51:15 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTEzOiBhdm9p?=
 =?utf-8?q?d_segfault_if_Py=5FXDECREF_triggers_code_that_calls_set=5Fpanel?=
 =?utf-8?q?=5Fuserptr?=
Message-ID: <3bd5Sz6GLhzSYg@mail.python.org>

http://hg.python.org/cpython/rev/61fafef4c8a2
changeset:   84256:61fafef4c8a2
branch:      3.3
parent:      84249:bfe5507423b1
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 22 14:50:56 2013 -0400
summary:
  #18113: avoid segfault if Py_XDECREF triggers code that calls set_panel_userptr again

Problem noted & original patch by Serhiy Storchaka; I tweaked the patch a bit.

files:
  Lib/test/test_curses.py |   9 +++++++++
  Modules/_curses_panel.c |  11 ++++++++---
  2 files changed, 17 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -264,6 +264,14 @@
     if sys.getrefcount(obj) != nrefs:
         raise RuntimeError("set_userptr leaked references")
 
+def test_userptr_segfault(stdscr):
+    panel = curses.panel.new_panel(stdscr)
+    class A:
+        def __del__(self):
+            panel.set_userptr(None)
+    panel.set_userptr(A())
+    panel.set_userptr(None)
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -330,6 +338,7 @@
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
         test_userptr_memory_leak(stdscr)
+        test_userptr_segfault(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
         test_unget_wch(stdscr)
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -323,12 +323,17 @@
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
     PyObject *oldobj;
+    int rc;
     PyCursesInitialised;
+    Py_INCREF(obj);
     oldobj = (PyObject *) panel_userptr(self->pan);
+    rc = set_panel_userptr(self->pan, (void*)obj);
+    if (rc == ERR) {
+        /* In case of an ncurses error, decref the new object again */
+        Py_DECREF(obj);
+    }
     Py_XDECREF(oldobj);
-    Py_INCREF(obj);
-    return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
-                            "set_panel_userptr");
+    return PyCursesCheckERR(rc, "set_panel_userptr");
 }
 
 static PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 20:57:55 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sat, 22 Jun 2013 20:57:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_from_3=2E3?=
Message-ID: <3bd5cg0LgGzMtd@mail.python.org>

http://hg.python.org/cpython/rev/d7d73f48dfda
changeset:   84257:d7d73f48dfda
parent:      84254:7fb25b77e327
parent:      84256:61fafef4c8a2
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 22 14:57:45 2013 -0400
summary:
  Merge from 3.3

files:
  Lib/test/test_curses.py |   9 +++++++++
  Modules/_curses_panel.c |  11 ++++++++---
  2 files changed, 17 insertions(+), 3 deletions(-)


diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py
--- a/Lib/test/test_curses.py
+++ b/Lib/test/test_curses.py
@@ -264,6 +264,14 @@
     if sys.getrefcount(obj) != nrefs:
         raise RuntimeError("set_userptr leaked references")
 
+def test_userptr_segfault(stdscr):
+    panel = curses.panel.new_panel(stdscr)
+    class A:
+        def __del__(self):
+            panel.set_userptr(None)
+    panel.set_userptr(A())
+    panel.set_userptr(None)
+
 def test_resize_term(stdscr):
     if hasattr(curses, 'resizeterm'):
         lines, cols = curses.LINES, curses.COLS
@@ -330,6 +338,7 @@
         window_funcs(stdscr)
         test_userptr_without_set(stdscr)
         test_userptr_memory_leak(stdscr)
+        test_userptr_segfault(stdscr)
         test_resize_term(stdscr)
         test_issue6243(stdscr)
         test_unget_wch(stdscr)
diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c
--- a/Modules/_curses_panel.c
+++ b/Modules/_curses_panel.c
@@ -323,12 +323,17 @@
 PyCursesPanel_set_panel_userptr(PyCursesPanelObject *self, PyObject *obj)
 {
     PyObject *oldobj;
+    int rc;
     PyCursesInitialised;
+    Py_INCREF(obj);
     oldobj = (PyObject *) panel_userptr(self->pan);
+    rc = set_panel_userptr(self->pan, (void*)obj);
+    if (rc == ERR) {
+        /* In case of an ncurses error, decref the new object again */
+        Py_DECREF(obj);
+    }
     Py_XDECREF(oldobj);
-    Py_INCREF(obj);
-    return PyCursesCheckERR(set_panel_userptr(self->pan, (void*)obj),
-                            "set_panel_userptr");
+    return PyCursesCheckERR(rc, "set_panel_userptr");
 }
 
 static PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 22 21:05:13 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 22 Jun 2013 21:05:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2311016=3A_Add_C_im?=
 =?utf-8?q?plementation_of_the_stat_module_as_=5Fstat?=
Message-ID: <3bd5n54czJzRk0@mail.python.org>

http://hg.python.org/cpython/rev/420f70a22b9d
changeset:   84258:420f70a22b9d
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 22 21:05:02 2013 +0200
summary:
  Issue #11016: Add C implementation of the stat module as _stat

files:
  Doc/library/stat.rst       |   51 ++-
  Doc/whatsnew/3.4.rst       |    7 +-
  Lib/stat.py                |    6 +
  Lib/test/test_stat.py      |   57 +-
  Misc/NEWS                  |    2 +
  Modules/Setup.dist         |    1 +
  Modules/_stat.c            |  571 +++++++++++++++++++++++++
  PC/VS9.0/pythoncore.vcproj |    4 +
  PCbuild/pythoncore.vcxproj |    3 +-
  9 files changed, 679 insertions(+), 23 deletions(-)


diff --git a/Doc/library/stat.rst b/Doc/library/stat.rst
--- a/Doc/library/stat.rst
+++ b/Doc/library/stat.rst
@@ -6,7 +6,8 @@
               os.lstat() and os.fstat().
 .. sectionauthor:: Skip Montanaro <skip at automatrix.com>
 
-**Source code:** :source:`Lib/stat.py`
+**Source code:** :source:`Modules/_stat.c`
+                 :source:`Lib/stat.py`
 
 --------------
 
@@ -15,6 +16,9 @@
 exist).  For complete details about the :c:func:`stat`, :c:func:`fstat` and
 :c:func:`lstat` calls, consult the documentation for your system.
 
+.. versionchanged:: 3.4
+   The stat module is backed by a C implementation.
+
 The :mod:`stat` module defines the following functions to test for specific file
 types:
 
@@ -53,6 +57,24 @@
 
    Return non-zero if the mode is from a socket.
 
+.. function:: S_ISDOOR(mode)
+
+   Return non-zero if the mode is from a door.
+
+   .. versionadded:: 3.4
+
+.. function:: S_ISPORT(mode)
+
+   Return non-zero if the mode is from an event port.
+
+   .. versionadded:: 3.4
+
+.. function:: S_ISWHT(mode)
+
+   Return non-zero if the mode is from a whiteout.
+
+   .. versionadded:: 3.4
+
 Two additional functions are defined for more general manipulation of the file's
 mode:
 
@@ -113,6 +135,10 @@
 
    .. versionadded:: 3.3
 
+   .. versionchanged:: 3.4
+      The function supports :data:`S_IFDOOR`, :data:`S_IFPORT` and
+      :data:`S_IFWHT`.
+
 
 All the variables below are simply symbolic indexes into the 10-tuple returned
 by :func:`os.stat`, :func:`os.fstat` or :func:`os.lstat`.
@@ -210,6 +236,29 @@
 
    FIFO.
 
+.. data:: S_IFDOOR
+
+   Door.
+
+   .. versionadded:: 3.4
+
+.. data:: S_IFPORT
+
+   Event port.
+
+   .. versionadded:: 3.4
+
+.. data:: S_IFWHT
+
+   Whiteout.
+
+   .. versionadded:: 3.4
+
+.. note::
+
+   :data:`S_IFDOOR`, :data:`S_IFPORT` or :data:`S_IFWHT` are defined as
+   0 when the platform does not have support for the file types.
+
 The following flags can also be used in the *mode* argument of :func:`os.chmod`:
 
 .. data:: S_ISUID
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -182,7 +182,6 @@
 
 New :func:`functools.singledispatch` decorator: see the :pep:`443`.
 
-
 smtplib
 -------
 
@@ -213,6 +212,12 @@
 The :meth:`~wave.getparams` method now returns a namedtuple rather than a
 plain tuple.  (Contributed by Claudiu Popa in :issue:`17487`.)
 
+stat
+---
+
+The stat module is now backed by a C implementation in :mod:`_stat`. A C
+implementation is required as most of the values aren't standardized and
+platform-dependent.  (Contributed by Christian Heimes in :issue:`11016`.)
 
 Optimizations
 =============
diff --git a/Lib/stat.py b/Lib/stat.py
--- a/Lib/stat.py
+++ b/Lib/stat.py
@@ -147,3 +147,9 @@
         else:
             perm.append("-")
     return "".join(perm)
+
+# If available, use C implementation
+try:
+    from _stat import *
+except ModuleNotFoundError:
+    pass
diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -1,9 +1,13 @@
 import unittest
 import os
 from test.support import TESTFN, run_unittest, import_fresh_module
-import stat
+
+c_stat = import_fresh_module('stat', fresh=['_stat'])
+py_stat = import_fresh_module('stat', blocked=['_stat'])
 
 class TestFilemode(unittest.TestCase):
+    statmod = None
+
     file_flags = {'SF_APPEND', 'SF_ARCHIVED', 'SF_IMMUTABLE', 'SF_NOUNLINK',
                   'SF_SNAPSHOT', 'UF_APPEND', 'UF_COMPRESSED', 'UF_HIDDEN',
                   'UF_IMMUTABLE', 'UF_NODUMP', 'UF_NOUNLINK', 'UF_OPAQUE'}
@@ -60,17 +64,17 @@
 
     def get_mode(self, fname=TESTFN):
         st_mode = os.lstat(fname).st_mode
-        modestr = stat.filemode(st_mode)
+        modestr = self.statmod.filemode(st_mode)
         return st_mode, modestr
 
     def assertS_IS(self, name, mode):
         # test format, lstrip is for S_IFIFO
-        fmt = getattr(stat, "S_IF" + name.lstrip("F"))
-        self.assertEqual(stat.S_IFMT(mode), fmt)
+        fmt = getattr(self.statmod, "S_IF" + name.lstrip("F"))
+        self.assertEqual(self.statmod.S_IFMT(mode), fmt)
         # test that just one function returns true
         testname = "S_IS" + name
         for funcname in self.format_funcs:
-            func = getattr(stat, funcname, None)
+            func = getattr(self.statmod, funcname, None)
             if func is None:
                 if funcname == testname:
                     raise ValueError(funcname)
@@ -88,35 +92,35 @@
             st_mode, modestr = self.get_mode()
             self.assertEqual(modestr, '-rwx------')
             self.assertS_IS("REG", st_mode)
-            self.assertEqual(stat.S_IMODE(st_mode),
-                             stat.S_IRWXU)
+            self.assertEqual(self.statmod.S_IMODE(st_mode),
+                             self.statmod.S_IRWXU)
 
             os.chmod(TESTFN, 0o070)
             st_mode, modestr = self.get_mode()
             self.assertEqual(modestr, '----rwx---')
             self.assertS_IS("REG", st_mode)
-            self.assertEqual(stat.S_IMODE(st_mode),
-                             stat.S_IRWXG)
+            self.assertEqual(self.statmod.S_IMODE(st_mode),
+                             self.statmod.S_IRWXG)
 
             os.chmod(TESTFN, 0o007)
             st_mode, modestr = self.get_mode()
             self.assertEqual(modestr, '-------rwx')
             self.assertS_IS("REG", st_mode)
-            self.assertEqual(stat.S_IMODE(st_mode),
-                             stat.S_IRWXO)
+            self.assertEqual(self.statmod.S_IMODE(st_mode),
+                             self.statmod.S_IRWXO)
 
             os.chmod(TESTFN, 0o444)
             st_mode, modestr = self.get_mode()
             self.assertS_IS("REG", st_mode)
             self.assertEqual(modestr, '-r--r--r--')
-            self.assertEqual(stat.S_IMODE(st_mode), 0o444)
+            self.assertEqual(self.statmod.S_IMODE(st_mode), 0o444)
         else:
             os.chmod(TESTFN, 0o700)
             st_mode, modestr = self.get_mode()
             self.assertEqual(modestr[:3], '-rw')
             self.assertS_IS("REG", st_mode)
-            self.assertEqual(stat.S_IFMT(st_mode),
-                             stat.S_IFREG)
+            self.assertEqual(self.statmod.S_IFMT(st_mode),
+                             self.statmod.S_IFREG)
 
     def test_directory(self):
         os.mkdir(TESTFN)
@@ -162,25 +166,38 @@
 
     def test_module_attributes(self):
         for key, value in self.stat_struct.items():
-            modvalue = getattr(stat, key)
+            modvalue = getattr(self.statmod, key)
             self.assertEqual(value, modvalue, key)
         for key, value in self.permission_bits.items():
-            modvalue = getattr(stat, key)
+            modvalue = getattr(self.statmod, key)
             self.assertEqual(value, modvalue, key)
         for key in self.file_flags:
-            modvalue = getattr(stat, key)
+            modvalue = getattr(self.statmod, key)
             self.assertIsInstance(modvalue, int)
         for key in self.formats:
-            modvalue = getattr(stat, key)
+            modvalue = getattr(self.statmod, key)
             self.assertIsInstance(modvalue, int)
         for key in self.format_funcs:
-            func = getattr(stat, key)
+            func = getattr(self.statmod, key)
             self.assertTrue(callable(func))
             self.assertEqual(func(0), 0)
 
 
+class TestFilemodeCStat(TestFilemode):
+    statmod = c_stat
+
+    formats = TestFilemode.formats | {'S_IFDOOR', 'S_IFPORT', 'S_IFWHT'}
+    format_funcss = TestFilemode.format_funcs | {'S_ISDOOR', 'S_ISPORT',
+                                                 'S_ISWHT'}
+
+
+class TestFilemodePyStat(TestFilemode):
+    statmod = py_stat
+
+
 def test_main():
-    run_unittest(TestFilemode)
+    run_unittest(TestFilemodeCStat)
+    run_unittest(TestFilemodePyStat)
 
 if __name__ == '__main__':
     test_main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -123,6 +123,8 @@
 Library
 -------
 
+- Issue #11016: Add C implementation of the stat module as _stat.
+
 - Issue #18248: Fix libffi build on AIX.
 
 - Issue #18259: Declare sethostname in socketmodule.c for AIX
diff --git a/Modules/Setup.dist b/Modules/Setup.dist
--- a/Modules/Setup.dist
+++ b/Modules/Setup.dist
@@ -117,6 +117,7 @@
 _collections _collectionsmodule.c # Container types
 itertools itertoolsmodule.c    # Functions creating iterators for efficient looping 
 atexit atexitmodule.c      # Register functions to be run at interpreter-shutdown
+_stat _stat.c			# stat.h interface
 
 # access to ISO C locale support
 _locale _localemodule.c  # -lintl
diff --git a/Modules/_stat.c b/Modules/_stat.c
new file mode 100644
--- /dev/null
+++ b/Modules/_stat.c
@@ -0,0 +1,571 @@
+/* stat.h interface
+ *
+ * The module defines all S_IF*, S_I*, UF_*, SF_* and ST_* constants to
+ * sensible default values as well as defines S_IS*() macros in order to keep
+ * backward compatibility with the old stat.py module.
+ *
+ * New constants and macros such as S_IFDOOR / S_ISDOOR() are always defined
+ * as int 0.
+ *
+ * NOTE: POSIX only defines the values of the S_I* permission bits.
+ *
+ */
+
+#define PY_SSIZE_T_CLEAN
+#include "Python.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif /* HAVE_SYS_TYPES_H */
+
+#ifdef HAVE_SYS_STAT_H
+#include <sys/stat.h>
+#endif /* HAVE_SYS_STAT_H */
+
+/* From Python's stat.py */
+#ifndef S_IMODE
+#  define S_IMODE 07777
+#endif
+
+/* S_IFXXX constants (file types)
+ *
+ * Only the names are defined by POSIX but not their value. All common file
+ * types seems to have the same numeric value on all platforms, though.
+ */
+#ifndef S_IFMT
+#  define S_IFMT 0170000
+#endif
+
+#ifndef S_IFDIR
+#  define S_IFDIR 0040000
+#endif
+
+#ifndef S_IFCHR
+#  define S_IFCHR 0020000
+#endif
+
+#ifndef S_IFBLK
+#  define S_IFBLK 0060000
+#endif
+
+#ifndef S_IFREG
+#  define S_IFREG 0100000
+#endif
+
+#ifndef S_IFIFO
+#  define S_IFIFO 0010000
+#endif
+
+#ifndef S_IFLNK
+#  define S_IFLNK 0120000
+#endif
+
+#ifndef S_IFSOCK
+#  define S_IFSOCK 0140000
+#endif
+
+#ifndef S_IFDOOR
+#  define S_IFDOOR 0
+#endif
+
+#ifndef S_IFPORT
+#  define S_IFPORT 0
+#endif
+
+#ifndef S_IFWHT
+#  define S_IFWHT 0
+#endif
+
+
+/* S_ISXXX() */
+#ifndef S_ISDIR
+#  define S_ISDIR(mode) ((mode) & S_IFMT) == S_IDIR
+#endif
+
+#ifndef S_ISCHR
+#  define S_ISCHR(mode) ((mode) & S_IFMT) == S_ICHR
+#endif
+
+#ifndef S_ISBLK
+#  define S_ISBLK(mode) ((mode) & S_IFMT) == S_IBLK
+#endif
+
+#ifndef S_ISREG
+#  define S_ISREG(mode) ((mode) & S_IFMT) == S_IREG
+#endif
+
+#ifndef S_ISFIFO
+#  define S_ISFIFO(mode) ((mode) & S_IFMT) == S_IFIFO
+#endif
+
+#ifndef S_ISLNK
+#  define S_ISLNK(mode) ((mode) & S_IFMT) == S_IFLNK
+#endif
+
+#ifndef S_ISSOCK
+#  define S_ISSOCK(mode) ((mode) & S_IFMT) == S_IFSOCK
+#endif
+
+#ifndef S_ISDOOR
+#  define S_ISDOOR(mode) 0
+#endif
+
+#ifndef S_ISPORT
+#  define S_ISPORT(mode) 0
+#endif
+
+#ifndef S_ISWHT
+#  define S_ISWHT(mode) 0
+#endif
+
+
+/* S_I* file permission
+ *
+ * The permission bit value are defined by POSIX standards.
+ */
+#ifndef S_ISUID
+#  define S_ISUID 04000
+#endif
+
+#ifndef S_ISGID
+#  define S_ISGID 02000
+#endif
+
+/* what is S_ENFMT? */
+#ifndef S_ENFMT
+#  define S_ENFMT S_ISGID
+#endif
+
+#ifndef S_ISVTX
+#  define S_ISVTX 01000
+#endif
+
+#ifndef S_IREAD
+#  define S_IREAD 00400
+#endif
+
+#ifndef S_IWRITE
+#  define S_IWRITE 00200
+#endif
+
+#ifndef S_IEXEC
+#  define S_IEXEC 00100
+#endif
+
+#ifndef S_IRWXU
+#  define S_IRWXU 00700
+#endif
+
+#ifndef S_IRUSR
+#  define S_IRUSR 00400
+#endif
+
+#ifndef S_IWUSR
+#  define S_IWUSR 00200
+#endif
+
+#ifndef S_IXUSR
+#  define S_IXUSR 00100
+#endif
+
+#ifndef S_IRWXG
+#  define S_IRWXG 00070
+#endif
+
+#ifndef S_IRGRP
+#  define S_IRGRP 00040
+#endif
+
+#ifndef S_IWGRP
+#  define S_IWGRP 00020
+#endif
+
+#ifndef S_IXGRP
+#  define S_IXGRP 00010
+#endif
+
+#ifndef S_IRWXO
+#  define S_IRWXO 00007
+#endif
+
+#ifndef S_IROTH
+#  define S_IROTH 00004
+#endif
+
+#ifndef S_IWOTH
+#  define S_IWOTH 00002
+#endif
+
+#ifndef S_IXOTH
+#  define S_IXOTH 00001
+#endif
+
+
+/* Names for file flags */
+#ifndef UF_NODUMP
+#  define UF_NODUMP 0x00000001
+#endif
+
+#ifndef UF_IMMUTABLE
+#  define UF_IMMUTABLE 0x00000002
+#endif
+
+#ifndef UF_APPEND
+#  define UF_APPEND 0x00000004
+#endif
+
+#ifndef UF_OPAQUE
+#  define UF_OPAQUE 0x00000008
+#endif
+
+#ifndef UF_NOUNLINK
+#  define UF_NOUNLINK 0x00000010
+#endif
+
+#ifndef UF_COMPRESSED
+#  define UF_COMPRESSED 0x00000020
+#endif
+
+#ifndef UF_HIDDEN
+#  define UF_HIDDEN 0x00008000
+#endif
+
+#ifndef SF_ARCHIVED
+#  define SF_ARCHIVED 0x00010000
+#endif
+
+#ifndef SF_IMMUTABLE
+#  define SF_IMMUTABLE 0x00020000
+#endif
+
+#ifndef SF_APPEND
+#  define SF_APPEND 0x00040000
+#endif
+
+#ifndef SF_NOUNLINK
+#  define SF_NOUNLINK 0x00100000
+#endif
+
+#ifndef SF_SNAPSHOT
+#  define SF_SNAPSHOT 0x00200000
+#endif
+
+
+#define stat_S_ISFUNC(isfunc, doc)                             \
+    static PyObject *                                          \
+    stat_ ##isfunc (PyObject *self, PyObject *omode)           \
+    {                                                          \
+       unsigned long mode = PyLong_AsUnsignedLong(omode);      \
+       if ((mode == (unsigned long)-1) && PyErr_Occurred()) {  \
+           return NULL;                                        \
+       }                                                       \
+       return PyBool_FromLong(isfunc(mode));                   \
+    }                                                          \
+    PyDoc_STRVAR(stat_ ## isfunc ## _doc, doc)
+
+stat_S_ISFUNC(S_ISDIR,
+    "S_ISDIR(mode) -> bool\n\n"
+    "Return True if mode is from a directory.");
+
+stat_S_ISFUNC(S_ISCHR,
+    "S_ISCHR(mode) -> bool\n\n"
+    "Return True if mode is from a character special device file.");
+
+stat_S_ISFUNC(S_ISBLK,
+    "S_ISBLK(mode) -> bool\n\n"
+    "Return True if mode is from a block special device file.");
+
+stat_S_ISFUNC(S_ISREG,
+    "S_ISREG(mode) -> bool\n\n"
+    "Return True if mode is from a regular file.");
+
+stat_S_ISFUNC(S_ISFIFO,
+    "S_ISFIFO(mode) -> bool\n\n"
+    "Return True if mode is from a FIFO (named pipe).");
+
+stat_S_ISFUNC(S_ISLNK,
+    "S_ISLNK(mode) -> bool\n\n"
+    "Return True if mode is from a symbolic link.");
+
+stat_S_ISFUNC(S_ISSOCK,
+    "S_ISSOCK(mode) -> bool\n\n"
+    "Return True if mode is from a socket.");
+
+stat_S_ISFUNC(S_ISDOOR,
+    "S_ISDOOR(mode) -> bool\n\n"
+    "Return True if mode is from a door.");
+
+stat_S_ISFUNC(S_ISPORT,
+    "S_ISPORT(mode) -> bool\n\n"
+    "Return True if mode is from an event port.");
+
+stat_S_ISFUNC(S_ISWHT,
+    "S_ISWHT(mode) -> bool\n\n"
+    "Return True if mode is from a whiteout.");
+
+
+PyDoc_STRVAR(stat_S_IMODE_doc,
+"Return the portion of the file's mode that can be set by os.chmod().");
+
+static PyObject *
+stat_S_IMODE(PyObject *self, PyObject *omode)
+{
+    unsigned long mode = PyLong_AsUnsignedLong(omode);
+    if ((mode == (unsigned long)-1) && PyErr_Occurred()) {
+        return NULL;
+    }
+    return PyLong_FromUnsignedLong(mode & S_IMODE);
+}
+
+
+PyDoc_STRVAR(stat_S_IFMT_doc,
+"Return the portion of the file's mode that describes the file type.");
+
+static PyObject *
+stat_S_IFMT(PyObject *self, PyObject *omode)
+{
+    unsigned long mode = PyLong_AsUnsignedLong(omode);
+    if ((mode == (unsigned long)-1) && PyErr_Occurred()) {
+        return NULL;
+    }
+    return PyLong_FromUnsignedLong(mode & S_IFMT);
+}
+
+/* file type chars according to
+   http://en.wikibooks.org/wiki/C_Programming/POSIX_Reference/sys/stat.h */
+
+static char
+filetype(mode_t mode)
+{
+    /* common cases first */
+    if (S_ISREG(mode))  return '-';
+    if (S_ISDIR(mode))  return 'd';
+    if (S_ISLNK(mode))  return 'l';
+    /* special files */
+    if (S_ISBLK(mode))  return 'b';
+    if (S_ISCHR(mode))  return 'c';
+    if (S_ISFIFO(mode)) return 'p';
+    if (S_ISSOCK(mode)) return 's';
+    /* non-standard types */
+    if (S_ISDOOR(mode)) return 'D';
+    if (S_ISPORT(mode)) return 'P';
+    if (S_ISWHT(mode))  return 'w';
+    /* unknown */
+    return '?';
+}
+
+static void
+fileperm(mode_t mode, char *buf)
+{
+    buf[0] = mode & S_IRUSR ? 'r' : '-';
+    buf[1] = mode & S_IWUSR ? 'w' : '-';
+    if (mode & S_ISUID) {
+        buf[2] = mode & S_IXUSR ? 's' : 'S';
+    } else {
+        buf[2] = mode & S_IXUSR ? 'x' : '-';
+    }
+    buf[3] = mode & S_IRGRP ? 'r' : '-';
+    buf[4] = mode & S_IWGRP ? 'w' : '-';
+    if (mode & S_ISGID) {
+        buf[5] = mode & S_IXGRP ? 's' : 'S';
+    } else {
+        buf[5] = mode & S_IXGRP ? 'x' : '-';
+    }
+    buf[6] = mode & S_IROTH ? 'r' : '-';
+    buf[7] = mode & S_IWOTH ? 'w' : '-';
+    if (mode & S_ISVTX) {
+        buf[8] = mode & S_IXOTH ? 't' : 'T';
+    } else {
+        buf[8] = mode & S_IXOTH ? 'x' : '-';
+    }
+}
+
+PyDoc_STRVAR(stat_filemode_doc,
+"Convert a file's mode to a string of the form '-rwxrwxrwx'");
+
+static PyObject *
+stat_filemode(PyObject *self, PyObject *omode)
+{
+    char buf[10];
+    unsigned long mode;
+
+    mode = PyLong_AsUnsignedLong(omode);
+    if ((mode == (unsigned long)-1) && PyErr_Occurred()) {
+        return NULL;
+    }
+
+    buf[0] = filetype(mode);
+    fileperm(mode, &buf[1]);
+    return PyUnicode_FromStringAndSize(buf, 10);
+}
+
+
+static PyMethodDef stat_methods[] = {
+    {"S_ISDIR",         stat_S_ISDIR,  METH_O, stat_S_ISDIR_doc},
+    {"S_ISCHR",         stat_S_ISCHR,  METH_O, stat_S_ISCHR_doc},
+    {"S_ISBLK",         stat_S_ISBLK,  METH_O, stat_S_ISBLK_doc},
+    {"S_ISREG",         stat_S_ISREG,  METH_O, stat_S_ISREG_doc},
+    {"S_ISFIFO",        stat_S_ISFIFO, METH_O, stat_S_ISFIFO_doc},
+    {"S_ISLNK",         stat_S_ISLNK,  METH_O, stat_S_ISLNK_doc},
+    {"S_ISSOCK",        stat_S_ISSOCK, METH_O, stat_S_ISSOCK_doc},
+    {"S_ISDOOR",        stat_S_ISDOOR, METH_O, stat_S_ISDOOR_doc},
+    {"S_ISPORT",        stat_S_ISPORT, METH_O, stat_S_ISPORT_doc},
+    {"S_ISWHT",         stat_S_ISWHT,  METH_O, stat_S_ISWHT_doc},
+    {"S_IMODE",         stat_S_IMODE,  METH_O, stat_S_IMODE_doc},
+    {"S_IFMT",          stat_S_IFMT,   METH_O, stat_S_IFMT_doc},
+    {"filemode",        stat_filemode, METH_O, stat_filemode_doc},
+    {NULL,              NULL}           /* sentinel */
+};
+
+
+PyDoc_STRVAR(module_doc,
+"S_IFMT_: file type bits\n\
+S_IFDIR: directory\n\
+S_IFCHR: character device\n\
+S_IFBLK: block device\n\
+S_IFREG: regular file\n\
+S_IFIFO: fifo (named pipe)\n\
+S_IFLNK: symbolic link\n\
+S_IFSOCK: socket file\n\
+S_IFDOOR: door\n\
+S_IFPORT: event port\n\
+S_IFWHT: whiteout\n\
+\n"
+
+"S_ISUID: set UID bit\n\
+S_ISGID: set GID bit\n\
+S_ENFMT: file locking enforcement\n\
+S_ISVTX: sticky bit\n\
+S_IREAD: Unix V7 synonym for S_IRUSR\n\
+S_IWRITE: Unix V7 synonym for S_IWUSR\n\
+S_IEXEC: Unix V7 synonym for S_IXUSR\n\
+S_IRWXU: mask for owner permissions\n\
+S_IRUSR: read by owner\n\
+S_IWUSR: write by owner\n\
+S_IXUSR: execute by owner\n\
+S_IRWXG: mask for group permissions\n\
+S_IRGRP: read by group\n\
+S_IWGRP: write by group\n\
+S_IXGRP: execute by group\n\
+S_IRWXO: mask for others (not in group) permissions\n\
+S_IROTH: read by others\n\
+S_IWOTH: write by others\n\
+S_IXOTH: execute by others\n\
+\n"
+
+"UF_NODUMP: do not dump file\n\
+UF_IMMUTABLE: file may not be changed\n\
+UF_APPEND: file may only be appended to\n\
+UF_OPAQUE: directory is opaque when viewed through a union stack\n\
+UF_NOUNLINK: file may not be renamed or deleted\n\
+UF_COMPRESSED: OS X: file is hfs-compressed\n\
+UF_HIDDEN: OS X: file should not be displayed\n\
+SF_ARCHIVED: file may be archived\n\
+SF_IMMUTABLE: file may not be changed\n\
+SF_APPEND: file may only be appended to\n\
+SF_NOUNLINK: file may not be renamed or deleted\n\
+SF_SNAPSHOT: file is a snapshot file\n\
+\n"
+
+"ST_MODE\n\
+ST_INO\n\
+ST_DEV\n\
+ST_NLINK\n\
+ST_UID\n\
+ST_GID\n\
+ST_SIZE\n\
+ST_ATIME\n\
+ST_MTIME\n\
+ST_CTIME\n\
+");
+
+
+static struct PyModuleDef statmodule = {
+    PyModuleDef_HEAD_INIT,
+    "_stat",
+    module_doc,
+    -1,
+    stat_methods,
+    NULL,
+    NULL,
+    NULL,
+    NULL
+};
+
+PyMODINIT_FUNC
+PyInit__stat(void)
+{
+    PyObject *m;
+    m = PyModule_Create(&statmodule);
+    if (m == NULL)
+        return NULL;
+
+    if (PyModule_AddIntMacro(m, S_IFDIR)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFCHR)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFBLK)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFREG)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFIFO)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFLNK)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFSOCK)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFDOOR)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFPORT)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IFWHT)) return NULL;
+
+    if (PyModule_AddIntMacro(m, S_ISUID)) return NULL;
+    if (PyModule_AddIntMacro(m, S_ISGID)) return NULL;
+    if (PyModule_AddIntMacro(m, S_ISVTX)) return NULL;
+    if (PyModule_AddIntMacro(m, S_ENFMT)) return NULL;
+
+    if (PyModule_AddIntMacro(m, S_IREAD)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IWRITE)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IEXEC)) return NULL;
+
+    if (PyModule_AddIntMacro(m, S_IRWXU)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IRUSR)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IWUSR)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IXUSR)) return NULL;
+
+    if (PyModule_AddIntMacro(m, S_IRWXG)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IRGRP)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IWGRP)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IXGRP)) return NULL;
+
+    if (PyModule_AddIntMacro(m, S_IRWXO)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IROTH)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IWOTH)) return NULL;
+    if (PyModule_AddIntMacro(m, S_IXOTH)) return NULL;
+
+    if (PyModule_AddIntMacro(m, UF_NODUMP)) return NULL;
+    if (PyModule_AddIntMacro(m, UF_IMMUTABLE)) return NULL;
+    if (PyModule_AddIntMacro(m, UF_APPEND)) return NULL;
+    if (PyModule_AddIntMacro(m, UF_OPAQUE)) return NULL;
+    if (PyModule_AddIntMacro(m, UF_NOUNLINK)) return NULL;
+    if (PyModule_AddIntMacro(m, UF_COMPRESSED)) return NULL;
+    if (PyModule_AddIntMacro(m, UF_HIDDEN)) return NULL;
+    if (PyModule_AddIntMacro(m, SF_ARCHIVED)) return NULL;
+    if (PyModule_AddIntMacro(m, SF_IMMUTABLE)) return NULL;
+    if (PyModule_AddIntMacro(m, SF_APPEND)) return NULL;
+    if (PyModule_AddIntMacro(m, SF_NOUNLINK)) return NULL;
+    if (PyModule_AddIntMacro(m, SF_SNAPSHOT)) return NULL;
+
+    if (PyModule_AddIntConstant(m, "ST_MODE", 0)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_INO", 1)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_DEV", 2)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_NLINK", 3)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_UID", 4)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_GID", 5)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_SIZE", 6)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_ATIME", 7)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_MTIME", 8)) return NULL;
+    if (PyModule_AddIntConstant(m, "ST_CTIME", 9)) return NULL;
+
+    return m;
+}
+
+#ifdef __cplusplus
+}
+#endif
diff --git a/PC/VS9.0/pythoncore.vcproj b/PC/VS9.0/pythoncore.vcproj
--- a/PC/VS9.0/pythoncore.vcproj
+++ b/PC/VS9.0/pythoncore.vcproj
@@ -1155,6 +1155,10 @@
 				>
 			</File>
 			<File
+				RelativePath="..\..\Modules\_stat.c"
+				>
+			</File>
+			<File
 				RelativePath="..\..\Modules\symtablemodule.c"
 				>
 			</File>
diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj
--- a/PCbuild/pythoncore.vcxproj
+++ b/PCbuild/pythoncore.vcxproj
@@ -525,6 +525,7 @@
     <ClCompile Include="..\Modules\sha256module.c" />
     <ClCompile Include="..\Modules\sha512module.c" />
     <ClCompile Include="..\Modules\signalmodule.c" />
+    <ClCompile Include="..\Modules\_stat.c" />
     <ClCompile Include="..\Modules\symtablemodule.c" />
     <ClCompile Include="..\Modules\_threadmodule.c" />
     <ClCompile Include="..\Modules\timemodule.c" />
@@ -678,4 +679,4 @@
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
-</Project>
\ No newline at end of file
+</Project>

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 00:38:05 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 23 Jun 2013 00:38:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTUxLCBwYXJ0?=
 =?utf-8?q?_2=3A_Silence_debug_build_resource_warning_for_each_file_opened?=
 =?utf-8?q?_by?=
Message-ID: <3bdBVj0zB5z7LjR@mail.python.org>

http://hg.python.org/cpython/rev/3c0e8e910125
changeset:   84259:3c0e8e910125
branch:      2.7
parent:      84255:07eb090d8938
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 22 18:26:38 2013 -0400
summary:
  #18151, part 2: Silence debug build resource warning for each file opened by
'Find in files' by replacing 'open with implicit close' by 'with open' in
GrepDialog method grep_it. Streamline code with enumerate(), direct file
iteration, and output tweak. Add test for this method, including output format.

files:
  Lib/idlelib/GrepDialog.py          |  42 +++-----
  Lib/idlelib/idle_test/test_grep.py |  82 ++++++++++++++++++
  2 files changed, 100 insertions(+), 24 deletions(-)


diff --git a/Lib/idlelib/GrepDialog.py b/Lib/idlelib/GrepDialog.py
--- a/Lib/idlelib/GrepDialog.py
+++ b/Lib/idlelib/GrepDialog.py
@@ -81,31 +81,19 @@
         hits = 0
         for fn in list:
             try:
-                f = open(fn)
-            except IOError, msg:
+                with open(fn) as f:
+                    for lineno, line in enumerate(f, 1):
+                        if line[-1:] == '\n':
+                            line = line[:-1]
+                        if prog.search(line):
+                            sys.stdout.write("%s: %s: %s\n" %
+                                             (fn, lineno, line))
+                            hits += 1
+            except IOError as msg:
                 print msg
-                continue
-            lineno = 0
-            while 1:
-                block = f.readlines(100000)
-                if not block:
-                    break
-                for line in block:
-                    lineno = lineno + 1
-                    if line[-1:] == '\n':
-                        line = line[:-1]
-                    if prog.search(line):
-                        sys.stdout.write("%s: %s: %s\n" % (fn, lineno, line))
-                        hits = hits + 1
-        if hits:
-            if hits == 1:
-                s = ""
-            else:
-                s = "s"
-            print "Found", hits, "hit%s." % s
-            print "(Hint: right-click to open locations.)"
-        else:
-            print "No hits."
+        print(("Hits found: %s\n"
+              "(Hint: right-click to open locations.)"
+              % hits) if hits else "No hits.")
 
     def findfiles(self, dir, base, rec):
         try:
@@ -131,3 +119,9 @@
         if self.top:
             self.top.grab_release()
             self.top.withdraw()
+
+if __name__ == "__main__":
+    # A human test is a bit tricky since EditorWindow() imports this module.
+    # Hence Idle must be restarted after editing this file for a live test.
+    import unittest
+    unittest.main('idlelib.idle_test.test_grep', verbosity=2, exit=False)
diff --git a/Lib/idlelib/idle_test/test_grep.py b/Lib/idlelib/idle_test/test_grep.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_grep.py
@@ -0,0 +1,82 @@
+""" !Changing this line will break Test_findfile.test_found!
+Non-gui unit tests for idlelib.GrepDialog methods.
+dummy_command calls grep_it calls findfiles.
+An exception raised in one method will fail callers.
+Otherwise, tests are mostly independent.
+*** Currently only test grep_it.
+"""
+import unittest
+from test.test_support import captured_stdout, findfile
+from idlelib.idle_test.mock_tk import Var
+from idlelib.GrepDialog import GrepDialog
+import re
+
+__file__ = findfile('idlelib/idle_test') + '/test_grep.py'
+
+class Dummy_searchengine:
+    '''GrepDialog.__init__ calls parent SearchDiabolBase which attaches the
+    passed in SearchEngine instance as attribute 'engine'. Only a few of the
+    many possible self.engine.x attributes are needed here.
+    '''
+    def getpat(self):
+        return self._pat
+
+searchengine = Dummy_searchengine()
+
+class Dummy_grep:
+    # Methods tested
+    #default_command = GrepDialog.default_command
+    grep_it = GrepDialog.grep_it.im_func
+    findfiles = GrepDialog.findfiles.im_func
+    # Other stuff needed
+    recvar = Var(False)
+    engine = searchengine
+    def close(self):  # gui method
+        pass
+
+grep = Dummy_grep()
+
+class FindfilesTest(unittest.TestCase):
+    # findfiles is really a function, not a method, could be iterator
+    # test that filename return filename
+    # test that idlelib has many .py files
+    # test that recursive flag adds idle_test .py files
+    pass
+
+class Grep_itTest(unittest.TestCase):
+    # Test captured reports with 0 and some hits.
+    # Should test file names, but Windows reports have mixed / and \ separators
+    # from incomplete replacement, so 'later'.
+
+    def report(self, pat):
+        grep.engine._pat = pat
+        with captured_stdout() as s:
+            grep.grep_it(re.compile(pat), __file__)
+        lines = s.getvalue().split('\n')
+        lines.pop()  # remove bogus '' after last \n
+        return lines
+
+    def test_unfound(self):
+        pat = 'xyz*'*7
+        lines = self.report(pat)
+        self.assertEqual(len(lines), 2)
+        self.assertIn(pat, lines[0])
+        self.assertEqual(lines[1], 'No hits.')
+
+    def test_found(self):
+
+        pat = '""" !Changing this line will break Test_findfile.test_found!'
+        lines = self.report(pat)
+        self.assertEqual(len(lines), 5)
+        self.assertIn(pat, lines[0])
+        self.assertIn('py: 1:', lines[1])  # line number 1
+        self.assertIn('2', lines[3])  # hits found 2
+        self.assertTrue(lines[4].startswith('(Hint:'))
+
+class Default_commandTest(unittest.TestCase):
+    # To write this, mode OutputWindow import to top of GrepDialog
+    # so it can be replaced by captured_stdout in class setup/teardown.
+    pass
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 00:38:06 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 23 Jun 2013 00:38:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTUxLCBwYXJ0?=
 =?utf-8?q?_2=3A_Silence_debug_build_resource_warning_for_each_file_opened?=
 =?utf-8?q?_by?=
Message-ID: <3bdBVk4JfHz7LjX@mail.python.org>

http://hg.python.org/cpython/rev/c541073173bb
changeset:   84260:c541073173bb
branch:      3.3
parent:      84256:61fafef4c8a2
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 22 18:26:51 2013 -0400
summary:
  #18151, part 2: Silence debug build resource warning for each file opened by
'Find in files' by replacing 'open with implicit close' by 'with open' in
GrepDialog method grep_it. Streamline code with enumerate(), direct file
iteration, and output tweak. Add test for this method, including output format.

files:
  Lib/idlelib/GrepDialog.py          |  43 ++++-----
  Lib/idlelib/idle_test/test_grep.py |  80 ++++++++++++++++++
  2 files changed, 99 insertions(+), 24 deletions(-)


diff --git a/Lib/idlelib/GrepDialog.py b/Lib/idlelib/GrepDialog.py
--- a/Lib/idlelib/GrepDialog.py
+++ b/Lib/idlelib/GrepDialog.py
@@ -81,36 +81,24 @@
         hits = 0
         for fn in list:
             try:
-                f = open(fn, errors='replace')
+                with open(fn, errors='replace') as f:
+                    for lineno, line in enumerate(f, 1):
+                        if line[-1:] == '\n':
+                            line = line[:-1]
+                        if prog.search(line):
+                            sys.stdout.write("%s: %s: %s\n" %
+                                             (fn, lineno, line))
+                            hits += 1
             except OSError as msg:
                 print(msg)
-                continue
-            lineno = 0
-            while 1:
-                block = f.readlines(100000)
-                if not block:
-                    break
-                for line in block:
-                    lineno = lineno + 1
-                    if line[-1:] == '\n':
-                        line = line[:-1]
-                    if prog.search(line):
-                        sys.stdout.write("%s: %s: %s\n" % (fn, lineno, line))
-                        hits = hits + 1
-        if hits:
-            if hits == 1:
-                s = ""
-            else:
-                s = "s"
-            print("Found", hits, "hit%s." % s)
-            print("(Hint: right-click to open locations.)")
-        else:
-            print("No hits.")
+        print(("Hits found: %s\n"
+              "(Hint: right-click to open locations.)"
+              % hits) if hits else "No hits.")
 
     def findfiles(self, dir, base, rec):
         try:
             names = os.listdir(dir or os.curdir)
-        except os.error as msg:
+        except OSerror as msg:
             print(msg)
             return []
         list = []
@@ -131,3 +119,10 @@
         if self.top:
             self.top.grab_release()
             self.top.withdraw()
+
+if __name__ == "__main__":
+    # A human test is a bit tricky since EditorWindow() imports this module.
+    # Hence Idle must be restarted after editing this file for a live test.
+    import unittest
+    unittest.main('idlelib.idle_test.test_grep', verbosity=2, exit=False)
+
diff --git a/Lib/idlelib/idle_test/test_grep.py b/Lib/idlelib/idle_test/test_grep.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_grep.py
@@ -0,0 +1,80 @@
+""" !Changing this line will break Test_findfile.test_found!
+Non-gui unit tests for idlelib.GrepDialog methods.
+dummy_command calls grep_it calls findfiles.
+An exception raised in one method will fail callers.
+Otherwise, tests are mostly independent.
+*** Currently only test grep_it.
+"""
+import unittest
+from test.support import captured_stdout
+from idlelib.idle_test.mock_tk import Var
+from idlelib.GrepDialog import GrepDialog
+import re
+
+class Dummy_searchengine:
+    '''GrepDialog.__init__ calls parent SearchDiabolBase which attaches the
+    passed in SearchEngine instance as attribute 'engine'. Only a few of the
+    many possible self.engine.x attributes are needed here.
+    '''
+    def getpat(self):
+        return self._pat
+
+searchengine = Dummy_searchengine()
+
+class Dummy_grep:
+    # Methods tested
+    #default_command = GrepDialog.default_command
+    grep_it = GrepDialog.grep_it
+    findfiles = GrepDialog.findfiles
+    # Other stuff needed
+    recvar = Var(False)
+    engine = searchengine
+    def close(self):  # gui method
+        pass
+
+grep = Dummy_grep()
+
+class FindfilesTest(unittest.TestCase):
+    # findfiles is really a function, not a method, could be iterator
+    # test that filename return filename
+    # test that idlelib has many .py files
+    # test that recursive flag adds idle_test .py files
+    pass
+
+class Grep_itTest(unittest.TestCase):
+    # Test captured reports with 0 and some hits.
+    # Should test file names, but Windows reports have mixed / and \ separators
+    # from incomplete replacement, so 'later'.
+
+    def report(self, pat):
+        grep.engine._pat = pat
+        with captured_stdout() as s:
+            grep.grep_it(re.compile(pat), __file__)
+        lines = s.getvalue().split('\n')
+        lines.pop()  # remove bogus '' after last \n
+        return lines
+
+    def test_unfound(self):
+        pat = 'xyz*'*7
+        lines = self.report(pat)
+        self.assertEqual(len(lines), 2)
+        self.assertIn(pat, lines[0])
+        self.assertEqual(lines[1], 'No hits.')
+
+    def test_found(self):
+
+        pat = '""" !Changing this line will break Test_findfile.test_found!'
+        lines = self.report(pat)
+        self.assertEqual(len(lines), 5)
+        self.assertIn(pat, lines[0])
+        self.assertIn('py: 1:', lines[1])  # line number 1
+        self.assertIn('2', lines[3])  # hits found 2
+        self.assertTrue(lines[4].startswith('(Hint:'))
+
+class Default_commandTest(unittest.TestCase):
+    # To write this, mode OutputWindow import to top of GrepDialog
+    # so it can be replaced by captured_stdout in class setup/teardown.
+    pass
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 00:38:08 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 23 Jun 2013 00:38:08 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=2318151_Merge_from_3=2E3?=
Message-ID: <3bdBVm0F2qz7LjZ@mail.python.org>

http://hg.python.org/cpython/rev/1d67c0893719
changeset:   84261:1d67c0893719
parent:      84258:420f70a22b9d
parent:      84260:c541073173bb
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 22 18:37:34 2013 -0400
summary:
  #18151 Merge from 3.3

files:
  Lib/idlelib/GrepDialog.py          |  42 ++++-----
  Lib/idlelib/idle_test/test_grep.py |  80 ++++++++++++++++++
  2 files changed, 98 insertions(+), 24 deletions(-)


diff --git a/Lib/idlelib/GrepDialog.py b/Lib/idlelib/GrepDialog.py
--- a/Lib/idlelib/GrepDialog.py
+++ b/Lib/idlelib/GrepDialog.py
@@ -81,36 +81,24 @@
         hits = 0
         for fn in list:
             try:
-                f = open(fn, errors='replace')
+                with open(fn, errors='replace') as f:
+                    for lineno, line in enumerate(f, 1):
+                        if line[-1:] == '\n':
+                            line = line[:-1]
+                        if prog.search(line):
+                            sys.stdout.write("%s: %s: %s\n" %
+                                             (fn, lineno, line))
+                            hits += 1
             except OSError as msg:
                 print(msg)
-                continue
-            lineno = 0
-            while 1:
-                block = f.readlines(100000)
-                if not block:
-                    break
-                for line in block:
-                    lineno = lineno + 1
-                    if line[-1:] == '\n':
-                        line = line[:-1]
-                    if prog.search(line):
-                        sys.stdout.write("%s: %s: %s\n" % (fn, lineno, line))
-                        hits = hits + 1
-        if hits:
-            if hits == 1:
-                s = ""
-            else:
-                s = "s"
-            print("Found", hits, "hit%s." % s)
-            print("(Hint: right-click to open locations.)")
-        else:
-            print("No hits.")
+        print(("Hits found: %s\n"
+              "(Hint: right-click to open locations.)"
+              % hits) if hits else "No hits.")
 
     def findfiles(self, dir, base, rec):
         try:
             names = os.listdir(dir or os.curdir)
-        except OSError as msg:
+        except OSerror as msg:
             print(msg)
             return []
         list = []
@@ -131,3 +119,9 @@
         if self.top:
             self.top.grab_release()
             self.top.withdraw()
+
+if __name__ == "__main__":
+    # A human test is a bit tricky since EditorWindow() imports this module.
+    # Hence Idle must be restarted after editing this file for a live test.
+    import unittest
+    unittest.main('idlelib.idle_test.test_grep', verbosity=2, exit=False)
diff --git a/Lib/idlelib/idle_test/test_grep.py b/Lib/idlelib/idle_test/test_grep.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_grep.py
@@ -0,0 +1,80 @@
+""" !Changing this line will break Test_findfile.test_found!
+Non-gui unit tests for idlelib.GrepDialog methods.
+dummy_command calls grep_it calls findfiles.
+An exception raised in one method will fail callers.
+Otherwise, tests are mostly independent.
+*** Currently only test grep_it.
+"""
+import unittest
+from test.support import captured_stdout
+from idlelib.idle_test.mock_tk import Var
+from idlelib.GrepDialog import GrepDialog
+import re
+
+class Dummy_searchengine:
+    '''GrepDialog.__init__ calls parent SearchDiabolBase which attaches the
+    passed in SearchEngine instance as attribute 'engine'. Only a few of the
+    many possible self.engine.x attributes are needed here.
+    '''
+    def getpat(self):
+        return self._pat
+
+searchengine = Dummy_searchengine()
+
+class Dummy_grep:
+    # Methods tested
+    #default_command = GrepDialog.default_command
+    grep_it = GrepDialog.grep_it
+    findfiles = GrepDialog.findfiles
+    # Other stuff needed
+    recvar = Var(False)
+    engine = searchengine
+    def close(self):  # gui method
+        pass
+
+grep = Dummy_grep()
+
+class FindfilesTest(unittest.TestCase):
+    # findfiles is really a function, not a method, could be iterator
+    # test that filename return filename
+    # test that idlelib has many .py files
+    # test that recursive flag adds idle_test .py files
+    pass
+
+class Grep_itTest(unittest.TestCase):
+    # Test captured reports with 0 and some hits.
+    # Should test file names, but Windows reports have mixed / and \ separators
+    # from incomplete replacement, so 'later'.
+
+    def report(self, pat):
+        grep.engine._pat = pat
+        with captured_stdout() as s:
+            grep.grep_it(re.compile(pat), __file__)
+        lines = s.getvalue().split('\n')
+        lines.pop()  # remove bogus '' after last \n
+        return lines
+
+    def test_unfound(self):
+        pat = 'xyz*'*7
+        lines = self.report(pat)
+        self.assertEqual(len(lines), 2)
+        self.assertIn(pat, lines[0])
+        self.assertEqual(lines[1], 'No hits.')
+
+    def test_found(self):
+
+        pat = '""" !Changing this line will break Test_findfile.test_found!'
+        lines = self.report(pat)
+        self.assertEqual(len(lines), 5)
+        self.assertIn(pat, lines[0])
+        self.assertIn('py: 1:', lines[1])  # line number 1
+        self.assertIn('2', lines[3])  # hits found 2
+        self.assertTrue(lines[4].startswith('(Hint:'))
+
+class Default_commandTest(unittest.TestCase):
+    # To write this, mode OutputWindow import to top of GrepDialog
+    # so it can be replaced by captured_stdout in class setup/teardown.
+    pass
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 01:17:33 2013
From: python-checkins at python.org (daniel.holth)
Date: Sun, 23 Jun 2013 01:17:33 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_pep-426=3A_properly_escape_ba?=
	=?utf-8?q?ckslashes?=
Message-ID: <3bdCNF20N6z7LjM@mail.python.org>

http://hg.python.org/peps/rev/ab7f630d4865
changeset:   4956:ab7f630d4865
user:        Daniel Holth <dholth at fastmail.fm>
date:        Sat Jun 22 19:17:17 2013 -0400
summary:
  pep-426: properly escape backslashes

files:
  pep-0426/pymeta-schema.json |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/pep-0426/pymeta-schema.json b/pep-0426/pymeta-schema.json
--- a/pep-0426/pymeta-schema.json
+++ b/pep-0426/pymeta-schema.json
@@ -7,7 +7,7 @@
     "metadata_version": {
       "description": "Version of the file format",
       "type": "string",
-      "pattern": "^(\d+(\.\d+)*)$"
+      "pattern": "^(\\d+(\\.\\d+)*)$"
     },
     "name": {
       "description": "The name of the distribution.",
@@ -17,7 +17,7 @@
     "version": {
       "description": "The distribution's public version identifier",
       "type": "string",
-      "pattern": "^(\d+(\.\d+)*)((a|b|c|rc)(\d+))?(\.(post)(\d+))?(\.(dev)(\d+))?$"
+      "pattern": "^(\\d+(\\.\\d+)*)((a|b|c|rc)(\\d+))?(\\.(post)(\\d+))?(\\.(dev)(\\d+))?$"
     },
     "source_label": {
       "description": "A constrained identifying text string",

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sun Jun 23 01:26:11 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sun, 23 Jun 2013 01:26:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2UgIzE4Mjg1?=
 =?utf-8?q?=3A_add_=27repeat=27_parameter_to_docstring_for_product?=
Message-ID: <3bdCZC0gTFz7Lk4@mail.python.org>

http://hg.python.org/cpython/rev/1fad7a709aae
changeset:   84262:1fad7a709aae
branch:      3.3
parent:      84260:c541073173bb
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 22 19:04:11 2013 -0400
summary:
  Close #18285: add 'repeat' parameter to docstring for product

files:
  Modules/itertoolsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c
--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -2229,7 +2229,7 @@
 };
 
 PyDoc_STRVAR(product_doc,
-"product(*iterables) --> product object\n\
+"product(*iterables, repeat=1) --> product object\n\
 \n\
 Cartesian product of input iterables.  Equivalent to nested for-loops.\n\n\
 For example, product(A, B) returns the same as:  ((x,y) for x in A for y in B).\n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 01:26:12 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sun, 23 Jun 2013 01:26:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogQ2xvc2VzICMxODIy?=
 =?utf-8?q?0=3A_expand_itertools=2Eislice_docstring_to_2_lines?=
Message-ID: <3bdCZD2V9Nz7LjW@mail.python.org>

http://hg.python.org/cpython/rev/7ecca1a98220
changeset:   84263:7ecca1a98220
branch:      3.3
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 22 19:20:54 2013 -0400
summary:
  Closes #18220: expand itertools.islice docstring to 2 lines

files:
  Modules/itertoolsmodule.c |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c
--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -1554,7 +1554,8 @@
 };
 
 PyDoc_STRVAR(islice_doc,
-"islice(iterable, [start,] stop [, step]) --> islice object\n\
+"islice(iterable, stop) --> islice object\n\
+islice(iterable, start, stop[, step]) --> islice object\n\
 \n\
 Return an iterator whose next() method returns selected values from an\n\
 iterable.  If start is specified, will skip all preceding elements;\n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 01:28:58 2013
From: python-checkins at python.org (andrew.kuchling)
Date: Sun, 23 Jun 2013 01:28:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_from_3=2E3?=
Message-ID: <3bdCdQ2zyJzQx6@mail.python.org>

http://hg.python.org/cpython/rev/0898e173ebe0
changeset:   84264:0898e173ebe0
parent:      84261:1d67c0893719
parent:      84263:7ecca1a98220
user:        Andrew Kuchling <amk at amk.ca>
date:        Sat Jun 22 19:27:59 2013 -0400
summary:
  Merge from 3.3

files:
  Modules/itertoolsmodule.c |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c
--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -1554,7 +1554,8 @@
 };
 
 PyDoc_STRVAR(islice_doc,
-"islice(iterable, [start,] stop [, step]) --> islice object\n\
+"islice(iterable, stop) --> islice object\n\
+islice(iterable, start, stop[, step]) --> islice object\n\
 \n\
 Return an iterator whose next() method returns selected values from an\n\
 iterable.  If start is specified, will skip all preceding elements;\n\
@@ -2229,7 +2230,7 @@
 };
 
 PyDoc_STRVAR(product_doc,
-"product(*iterables) --> product object\n\
+"product(*iterables, repeat=1) --> product object\n\
 \n\
 Cartesian product of input iterables.  Equivalent to nested for-loops.\n\n\
 For example, product(A, B) returns the same as:  ((x,y) for x in A for y in B).\n\

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 01:49:54 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 01:49:54 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2311016=3A_Try_to_f?=
 =?utf-8?q?ix_compilaton_of_the_new_=5Fstat=2Ec_module_on_Windows?=
Message-ID: <3bdD5Z1LwDz7LjM@mail.python.org>

http://hg.python.org/cpython/rev/e5427b0b2bf7
changeset:   84265:e5427b0b2bf7
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 01:49:42 2013 +0200
summary:
  Issue #11016: Try to fix compilaton of the new _stat.c module on Windows

files:
  Include/pyport.h |  8 ++++++++
  Modules/_stat.c  |  4 ++++
  2 files changed, 12 insertions(+), 0 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -406,6 +406,14 @@
 #define S_ISDIR(x) (((x) & S_IFMT) == S_IFDIR)
 #endif
 
+#ifndef S_ISCHR
+#define S_ISCHR(x) (((x) & S_IFMT) == S_IFCHR)
+#endif
+
+#ifndef S_ISBLK
+#define S_ISBLK(x) (((x) & S_IFMT) == S_IFBLK)
+#endif
+
 
 #ifdef __cplusplus
 /* Move this down here since some C++ #include's don't like to be included
diff --git a/Modules/_stat.c b/Modules/_stat.c
--- a/Modules/_stat.c
+++ b/Modules/_stat.c
@@ -26,6 +26,10 @@
 #include <sys/stat.h>
 #endif /* HAVE_SYS_STAT_H */
 
+#ifdef MS_WINDOWS
+typedef unsigned short mode_t;
+#endif
+
 /* From Python's stat.py */
 #ifndef S_IMODE
 #  define S_IMODE 07777

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 02:28:58 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 02:28:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_fix_hook=5Fmalloc?=
	=?utf-8?q?=28=29_in_example_3?=
Message-ID: <3bdDyf0SDGz7Lk3@mail.python.org>

http://hg.python.org/peps/rev/9eff6b4ce035
changeset:   4957:9eff6b4ce035
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 02:28:37 2013 +0200
summary:
  PEP 445: fix hook_malloc() in example 3

Add missing declaration "void *ptr;"

files:
  pep-0445.txt |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -308,6 +308,7 @@
     static void* hook_malloc(void *ctx, size_t size)
     {
         PyMemAllocator *alloc = (PyMemAllocator *)ctx;
+        void *ptr;
         /* ... */
         ptr = alloc->malloc(alloc->ctx, size);
         /* ... */

-- 
Repository URL: http://hg.python.org/peps

From solipsis at pitrou.net  Sun Jun 23 05:43:22 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 23 Jun 2013 05:43:22 +0200
Subject: [Python-checkins] Daily reference leaks (e5427b0b2bf7): sum=1
Message-ID: <E1UqbD0-0001mH-EU@ap.vmr.nerim.net>

results for e5427b0b2bf7 on branch "default"
--------------------------------------------

test_unittest leaked [0, -1, 2] memory blocks, sum=1


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogxx8ag8', '-x']

From python-checkins at python.org  Sun Jun 23 09:01:49 2013
From: python-checkins at python.org (nick.coghlan)
Date: Sun, 23 Jun 2013 09:01:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_426_updates?=
Message-ID: <3bdPgx4NKfz7LjY@mail.python.org>

http://hg.python.org/peps/rev/3f733fe7c06c
changeset:   4958:3f733fe7c06c
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Sun Jun 23 17:01:36 2013 +1000
summary:
  PEP 426 updates

* Add implementation_name and implementation_version marker variables
* Expand on the expected use cases for supports_environments
* Reference wheel and warehouse from legacy metadata appendix
* Drop most of the Sphinx notes (moving some to Rejected Features)

files:
  pep-0426.txt |  215 ++++++++++++++++++++------------------
  1 files changed, 115 insertions(+), 100 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -13,7 +13,7 @@
 Requires: 440
 Created: 30 Aug 2012
 Post-History: 14 Nov 2012, 5 Feb 2013, 7 Feb 2013, 9 Feb 2013,
-              27 May 2013, 20 Jun 2013
+              27 May 2013, 20 Jun 2013, 23 Jun 2013
 Replaces: 345
 
 
@@ -225,6 +225,9 @@
 objects as strings consisting of a Python module name and a module
 attribute name, separated by a colon. For example: ``"test.regrtest:main"``.
 
+"Distros" is used as the preferred term for Linux distributions, to help
+avoid confusion with the Python-specific meaning of the term.
+
 
 Integration and deployment of distributions
 -------------------------------------------
@@ -545,28 +548,6 @@
 
     "name": "ComfyChair"
 
-.. note::
-
-    Debian doesn't actually permit underscores in names, but that seems
-    unduly restrictive for this spec given the common practice of using
-    valid Python identifiers as Python distribution names. A Debian side
-    policy of converting underscores to hyphens seems easy enough to
-    implement (and the requirement to consider hyphens and underscores as
-    equivalent ensures that doing so won't introduce any conflicts).
-
-    We're deliberately *not* following Python 3 down the path of arbitrary
-    unicode identifiers at this time. The security implications of doing so
-    are substantially worse in the software distribution use case (it opens
-    up far more interesting attack vectors than mere code obfuscation), the
-    existing tooling really only works properly if you abide by the stated
-    restrictions and changing it would require a *lot* of work for all
-    the automated tools in the chain.
-
-    PyPI has recently been updated to reject non-compliant names for newly
-    registered projects, and the ~230 existing non-compliant names have
-    been updated to become compliant (mostly by replacing spaces with
-    hyphens).
-
 
 Version
 -------
@@ -665,33 +646,6 @@
     "source_url": "http://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686"
     "source_url": "git+https://github.com/pypa/pip.git at 1.3.1"
 
-.. note::
-
-    This was called "Download-URL" in previous versions of the metadata. It
-    has been renamed, since there are plenty of other download locations and
-    this URL is meant to be a way to get the original source for development
-    purposes (or to generate an SRPM or other platform specific equivalent).
-
-    For extra fun and games, it appears that unlike "svn+ssh://",
-    neither "git+ssh://" nor "hg+ssh://" natively support direct linking to a
-    particular tag (hg does support direct links to bookmarks through the URL
-    fragment, but that doesn't help for git and doesn't appear to be what I
-    want anyway).
-
-    However pip does have a `defined convention
-    <http://www.pip-installer.org/en/latest/logic.html#vcs-support>`__ for
-    this kind of link, which effectively splits a "URL" into "<repo>@<tag>".
-
-    The PEP simply adopts pip's existing solution to this problem.
-
-    This field is separate from the project URLs, as it's expected to
-    change for each version, while the project URLs are expected to
-    be fairly stable.
-
-    Note that many translations of legacy metadata won't be able to convert
-    Download-URL to source_url, as many distributions haven't respected the
-    requirement to link to a specific version.
-
 
 Additional descriptive metadata
 ===============================
@@ -851,15 +805,6 @@
 * ``contributor``: any other individuals or organizations involved in the
   creation of the distribution
 
-.. note::
-
-   The contributor role field is included primarily to replace the
-   Author, Author-Email, Maintainer, Maintainer-Email fields from metadata
-   1.2 in a way that allows those distinctions to be fully represented for
-   lossless translation, while allowing future distributions to pretty
-   much ignore everything other than the contact/contributor distinction
-   if they so choose.
-
 Contact and contributor metadata is optional. Automated tools MUST operate
 correctly if a distribution does not provide it, including failing cleanly
 when an operation depending on one of these fields is requested.
@@ -971,18 +916,6 @@
 Dependency management is heavily dependent on the version identification
 and specification scheme defined in PEP 440.
 
-.. note::
-
-    This substantially changes the old two-phase setup vs runtime dependency
-    model in metadata 1.2 (which was in turn derived from the setuptools
-    dependency parameters). The translation is that ``dev_requires`` and
-    ``build_requires`` both map to ``Setup-Requires-Dist``
-    in 1.2 (aka ``install_requires`` in setuptools), while ``run_requires``
-    and ``meta_requires`` map to ``Requires-Dist``.
-    To go the other  way, ``Setup-Requires-Dist`` maps to ``build_requires``
-    and ``Requires-Dist`` maps to ``meta_requires`` (for exact comparisons)
-    and ``run_requires`` (for all other version specifiers).
-
 All of these fields are optional. Automated tools MUST operate correctly if
 a distribution does not provide them, by assuming that a missing field
 indicates "Not applicable for this distribution".
@@ -1043,13 +976,6 @@
 conditional dependencies. This may happen, for example, if an extra itself
 only needs some of its dependencies in specific environments.
 
-.. note::
-
-   Technically, you could store the conditional and unconditional
-   dependencies in a single list and switch based on the entry type
-   (string or mapping), but the ``*requires`` vs ``*may-require`` two
-   list design seems easier to understand and work with.
-
 
 Mapping dependencies to development and distribution activities
 ---------------------------------------------------------------
@@ -1137,8 +1063,8 @@
 software integrators rather than publishers.
 
 Distributions that rely on direct references to platform specific binary
-archives SHOULD place appropriate entries in their ``supports_environments``
-field.
+archives SHOULD define appropriate constraints in their
+``supports_environments`` field.
 
 Example::
 
@@ -1167,8 +1093,8 @@
 software integrators rather than publishers.
 
 Distributions that rely on direct references to platform specific binary
-archives SHOULD place appropriate entries in their ``supports_environments``
-field.
+archives SHOULD defined appropriate constraints in their
+``supports_environments`` field.
 
 Example::
 
@@ -1468,26 +1394,43 @@
 Individual entries are environment markers, as described in
 `Environment markers`_.
 
-Installation tools SHOULD report an error if supported platforms are
+Installation tools SHOULD report an error if supported environments are
 specified by the distribution and the current platform fails to match
 any of them, MUST at least emit a warning, and MAY allow the user to
 force the installation to proceed regardless.
 
-Examples::
-
+The two main uses of this field are to declare which versions of Python
+and which underlying operating systems are supported.
+
+Examples indicating supported Python versions::
+
+   # Supports Python 2.6+
+   "supports_environments": ["python_version >= '2.6'"]
+
+   # Supports Python 2.6+ (for 2.x) or 3.3+ (for 3.x)
+   "supports_environments": ["python_version >= '3.3'",
+                             "'3.0' > python_version >= '2.6'"]
+
+Examples indicating supported operating systems::
+
+   # Windows only
    "supports_environments": ["sys_platform == 'win32'"]
+
+   # Anything except Windows
    "supports_environments": ["sys_platform != 'win32'"]
+
+   # Linux or BSD only
    "supports_environments": ["'linux' in sys_platform",
                              "'bsd' in sys_platform"]
 
-
-.. note::
-
-   This field replaces the old Platform, Requires-Platform and
-   Requires-Python fields and has been redesigned with environment
-   marker based semantics that should make it possible to reliably flag,
-   for example, Unix specific or Windows specific distributions, as well
-   as Python 2 only and Python 3 only distributions.
+Example where the supported Python version varies by platform::
+
+   # The standard library's os module has long supported atomic renaming
+   # on POSIX systems, but only gained atomic renaming on Windows in Python
+   # 3.3. A distribution that needs atomic renaming support for reliable
+   # operation might declare the following supported environments.
+   "supports_environments": ["python_version >= '2.6' and sys_platform != 'win32'",
+                             "python_version >= '3.3' and sys_platform == 'win32'"]
 
 
 Install hooks
@@ -1615,6 +1558,12 @@
 to identify metadata extensions. This practice will also make it easier to
 find authoritative documentation for metadata extensions.
 
+Metadata extensions allow development tools to record information in the
+metadata that may be useful during later phases of distribution. For
+example, a build tool could include default build options in a metadata
+extension when creating an sdist, and use those when creating the wheel
+files later.
+
 
 Extras (optional dependencies)
 ==============================
@@ -1766,6 +1715,12 @@
 * ``platform_version``: ``platform.version()``
 * ``platform_machine``: ``platform.machine()``
 * ``platform_python_implementation``: ``platform.python_implementation()``
+* ``implementation_name````: ``sys.implementation.name``
+* ``implementation_version````: see definition below
+
+If a particular value is not available (such as the ``sys.implementation``
+subattributes in versions of Python prior to 3.3), the corresponding marker
+variable MUST be considered equivalent to the empty string.
 
 Note that all subexpressions are restricted to strings or one of the
 marker variable names (which refer to string values), meaning that it is
@@ -1775,17 +1730,20 @@
 Chaining of comparison operations is permitted using the normal Python
 semantics of an implied ``and``.
 
-The ``python_full_version`` marker variable is derived from
-``sys.version_info()`` in accordance with the following algorithm::
-
-    def format_full_version():
-        info = sys.version_info
+The ``python_full_version`` and ``implementation_version`` marker variables
+are derived from ``sys.version_info()`` and ``sys.implementation.version``
+respectively, in accordance with the following algorithm::
+
+    def format_full_version(info):
         version = '{0.major}.{0.minor}.{0.micro}'.format(info)
         kind = info.releaselevel
         if kind != 'final':
             version += kind[0] + str(info.serial)
         return version
 
+    python_full_version = format_full_version(sys.version_info)
+    implementation_version = format_full_version(sys.implementation.version)
+
 ``python_full_version`` will typically correspond to the leading segment
 of ``sys.version()``.
 
@@ -1804,14 +1762,29 @@
 Appendix A: Conversion notes for legacy metadata
 ================================================
 
-TBD pending feedback from Daniel & Donald
+The reference implementations for converting from legacy metadata to
+metadata 2.0 are Daniel Holth's `wheel project
+<https://bitbucket.org/dholth/wheel/overview>`__ (which
+adds the ``bdist_wheel`` command to ``setuptools``) and Donald Stufft's
+`Warehouse project <https://github.com/dstufft/warehouse>`__ (which will
+eventually be the next generation Python Package Index implementation).
+
+While it is expected that there may be some edge cases where manual
+intervention is needed for clean conversion, the specification has been
+designed to allow fully automated conversion of almost all projects on
+PyPI.
+
+Metadata conversion (especially on the part of the index server) is a
+necessary step to allow installation and analysis tools to start
+benefiting from the new metadata format, without having to wait for
+developers to upgrade to newer build systems.
 
 
 Appendix B: Mapping dependency declarations to an RPM SPEC file
 ===============================================================
 
 
-As an example of mapping this to Linux distro packages, assume an
+As an example of mapping this PEP to Linux distro packages, assume an
 example project without any extras defined is split into 2 RPMs
 in a SPEC file: ``example`` and ``example-devel``.
 
@@ -2362,6 +2335,48 @@
 expressiveness.
 
 
+Disallowing underscores in distribution names
+---------------------------------------------
+
+Debian doesn't actually permit underscores in names, but that seems
+unduly restrictive for this spec given the common practice of using
+valid Python identifiers as Python distribution names. A Debian side
+policy of converting underscores to hyphens seems easy enough to
+implement (and the requirement to consider hyphens and underscores as
+equivalent ensures that doing so won't introduce any conflicts).
+
+
+Allowing the use of Unicode in distribution names
+-------------------------------------------------
+
+This PEP deliberately avoids following Python 3 down the path of arbitrary
+Unicode identifiers, as the security implications of doing so are
+substantially worse in the software distribution use case (it opens
+up far more interesting attack vectors than mere code obfuscation).
+
+In addition, the existing tools really only work properly if you restrict
+names to ASCII and changing that would require a *lot* of work for all
+the automated tools in the chain.
+
+It may be reasonable to revisit this question at some point in the (distant)
+future, but setting up a more reliable software distribution system is
+challenging enough without adding more general Unicode identifier support
+into the mix.
+
+
+Single list for conditional and unconditional dependencies
+----------------------------------------------------------
+
+It's technically possible to store the conditional and unconditional
+dependencies of each kind in a single list and switch the handling based on
+the entry type (string or mapping).
+
+However, the current ``*requires`` vs ``*may-require`` two list design seems
+easier to understand and work with, since it's only the conditional
+dependencies that need to be checked against the requested extras list and
+the target installation environment.
+
+
 Depending on source labels
 --------------------------
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sun Jun 23 14:34:24 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 14:34:24 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_add_=22Redesign_De?=
 =?utf-8?q?bug_Checks_on__Memory_Allocators_as_Hooks=22_section?=
Message-ID: <3bdY3h3dy5z7LjQ@mail.python.org>

http://hg.python.org/peps/rev/c6199202522a
changeset:   4959:c6199202522a
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 14:34:11 2013 +0200
summary:
  PEP 445: add "Redesign Debug Checks on  Memory Allocators as Hooks" section

files:
  pep-0445.txt |  49 +++++++++++++++++++++++++++++++--------
  1 files changed, 38 insertions(+), 11 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -40,8 +40,8 @@
 Proposal
 ========
 
-New functions and new structure
--------------------------------
+New Functions and Structures
+----------------------------
 
 * Add a new GIL-free (no need to hold the GIL) memory allocator:
 
@@ -109,11 +109,12 @@
   - ``void PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)``
   - ``void PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator)``
 
-* Add a new function to setup the builtin Python debug hooks when a
-  memory allocator is replaced:
+* Add a new function to setup the debug checks on memory allocators when
+  a memory allocator is replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
-  - the function does nothing is Python is not compiled in debug mode
+  - Install the debug hook on all memory block allocators.
+  - The function does nothing is Python is not compiled in debug mode
 
 * Memory allocators always returns *NULL* if size is greater than
   ``PY_SSIZE_T_MAX``. The check is done before calling the
@@ -133,8 +134,12 @@
 * *pymalloc* arena allocator: ``mmap()``, ``munmap()`` (and *ctx* is
   NULL), or ``malloc()`` and ``free()`` if ``mmap()`` is not available
 
-The builtin Python debug hooks were introduced in Python 2.3 and
-implement the following checks:
+
+Redesign Debug Checks on  Memory Allocators as Hooks
+----------------------------------------------------
+
+Since Python 2.3, Python implements different checks on memory
+allocators in debug mode:
 
 * Newly allocated memory is filled with the byte ``0xCB``, freed memory
   is filled with the byte ``0xDB``.
@@ -143,14 +148,36 @@
 * Detect write before the start of the buffer (buffer underflow)
 * Detect write after the end of the buffer (buffer overflow)
 
+In Python 3.3, the checks are installed by replacing
+``PYMEM_DOMAIN_MEM`` and ``PYMEM_DOMAIN_OBJ`` allocators, the previous
+allocator is no more called.  The new allocator is the same for both
+domains: ``PyMem_Malloc()`` and ``PyMem_Realloc()`` call indirectly
+``PyObject_Malloc()`` and ``PyObject_Realloc()``.
+
+This PEP redesigns the debug checks as hooks on the existing allocators
+in debug mode. Examples of call traces without the hooks:
+
+* ``PyMem_Malloc()`` => ``_PyMem_RawMalloc()`` => ``malloc()``
+* ``PyObject_Free()`` => ``_PyObject_Free()``
+
+Call traces when the hooks are installed (debug mode):
+
+* ``PyMem_Malloc()`` => ``_PyMem_DebugMalloc()`` =>
+  ``_PyMem_RawMalloc()`` => ``malloc()``
+* ``PyObject_Free()`` => ``_PyMem_DebugFree()`` => ``_PyObject_Free()``
+
+As a result, ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call
+``malloc()`` and ``realloc()``, instead of calling ``PyObject_Malloc()``
+and ``PyObject_Realloc()`` in debug mode.
+
+When at least one memory allocator is replaced with
+``PyMem_SetAllocator()``, the ``PyMem_SetupDebugHooks()`` function must
+be called to install the debug hooks on top on the new allocator.
+
 
 Don't call malloc() directly anymore
 ------------------------------------
 
-``PyMem_Malloc()`` and ``PyMem_Realloc()`` always call ``malloc()`` and
-``realloc()``, instead of calling ``PyObject_Malloc()`` and
-``PyObject_Realloc()`` in debug mode.
-
 ``PyObject_Malloc()`` falls back on ``PyMem_Malloc()`` instead of
 ``malloc()`` if size is greater or equal than 512 bytes, and
 ``PyObject_Realloc()`` falls back on ``PyMem_Realloc()`` instead of

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sun Jun 23 14:57:36 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 14:57:36 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTM3?=
 =?utf-8?q?=3A_Detect_integer_overflow_on_precision_in_float=2E=5F=5Fforma?=
 =?utf-8?b?dF9fKCkgYW5k?=
Message-ID: <3bdYZS0CJRzRTm@mail.python.org>

http://hg.python.org/cpython/rev/ef5175d08e7e
changeset:   84266:ef5175d08e7e
branch:      3.3
parent:      84263:7ecca1a98220
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 14:54:30 2013 +0200
summary:
  Issue #18137: Detect integer overflow on precision in float.__format__() and
complex.__format__().

files:
  Lib/test/test_format.py    |  17 +++++++++++++++++
  Misc/NEWS                  |   3 +++
  Python/formatter_unicode.c |  16 ++++++++++++++--
  3 files changed, 34 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_format.py b/Lib/test/test_format.py
--- a/Lib/test/test_format.py
+++ b/Lib/test/test_format.py
@@ -312,6 +312,23 @@
 def test_main():
     support.run_unittest(FormatTest)
 
+    def test_precision(self):
+        INT_MAX = 2147483647
+
+        f = 1.2
+        self.assertEqual(format(f, ".0f"), "1")
+        self.assertEqual(format(f, ".3f"), "1.200")
+        with self.assertRaises(ValueError) as cm:
+            format(f, ".%sf" % (INT_MAX + 1))
+        self.assertEqual(str(cm.exception), "precision too big")
+
+        c = complex(f)
+        self.assertEqual(format(f, ".0f"), "1")
+        self.assertEqual(format(f, ".3f"), "1.200")
+        with self.assertRaises(ValueError) as cm:
+            format(f, ".%sf" % (INT_MAX + 1))
+        self.assertEqual(str(cm.exception), "precision too big")
+
 
 if __name__ == "__main__":
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18137: Detect integer overflow on precision in float.__format__()
+  and complex.__format__().
+
 - Issue #18183: Fix various unicode operations on strings with large unicode
   codepoints.
 
diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c
--- a/Python/formatter_unicode.c
+++ b/Python/formatter_unicode.c
@@ -977,7 +977,7 @@
     Py_ssize_t n_total;
     int has_decimal;
     double val;
-    Py_ssize_t precision = format->precision;
+    Py_ssize_t precision;
     Py_ssize_t default_precision = 6;
     Py_UCS4 type = format->type;
     int add_pct = 0;
@@ -994,6 +994,12 @@
        from a hard-code pseudo-locale */
     LocaleInfo locale = STATIC_LOCALE_INFO_INIT;
 
+    if (format->precision > INT_MAX) {
+        PyErr_SetString(PyExc_ValueError, "precision too big");
+        goto done;
+    }
+    precision = (int)format->precision;
+
     if (format->alternate)
         flags |= Py_DTSF_ALT;
 
@@ -1127,7 +1133,7 @@
     Py_ssize_t n_im_total;
     int re_has_decimal;
     int im_has_decimal;
-    Py_ssize_t precision = format->precision;
+    int precision;
     Py_ssize_t default_precision = 6;
     Py_UCS4 type = format->type;
     Py_ssize_t i_re;
@@ -1155,6 +1161,12 @@
        from a hard-code pseudo-locale */
     LocaleInfo locale = STATIC_LOCALE_INFO_INIT;
 
+    if (format->precision > INT_MAX) {
+        PyErr_SetString(PyExc_ValueError, "precision too big");
+        goto done;
+    }
+    precision = (int)format->precision;
+
     /* Zero padding is not allowed. */
     if (format->fill_char == '0') {
         PyErr_SetString(PyExc_ValueError,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 14:57:37 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 14:57:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=28Merge_3=2E3=29_Issue_=2318137=3A_Detect_integer_overf?=
 =?utf-8?q?low_on_precision_in?=
Message-ID: <3bdYZT2C33z7Ljw@mail.python.org>

http://hg.python.org/cpython/rev/81fef2666ebb
changeset:   84267:81fef2666ebb
parent:      84265:e5427b0b2bf7
parent:      84266:ef5175d08e7e
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 14:55:43 2013 +0200
summary:
  (Merge 3.3) Issue #18137: Detect integer overflow on precision in
float.__format__() and complex.__format__().

files:
  Lib/test/test_format.py    |  17 +++++++++++++++++
  Misc/NEWS                  |   3 +++
  Python/formatter_unicode.c |  16 ++++++++++++++--
  3 files changed, 34 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_format.py b/Lib/test/test_format.py
--- a/Lib/test/test_format.py
+++ b/Lib/test/test_format.py
@@ -331,6 +331,23 @@
 def test_main():
     support.run_unittest(FormatTest)
 
+    def test_precision(self):
+        INT_MAX = 2147483647
+
+        f = 1.2
+        self.assertEqual(format(f, ".0f"), "1")
+        self.assertEqual(format(f, ".3f"), "1.200")
+        with self.assertRaises(ValueError) as cm:
+            format(f, ".%sf" % (INT_MAX + 1))
+        self.assertEqual(str(cm.exception), "precision too big")
+
+        c = complex(f)
+        self.assertEqual(format(f, ".0f"), "1")
+        self.assertEqual(format(f, ".3f"), "1.200")
+        with self.assertRaises(ValueError) as cm:
+            format(f, ".%sf" % (INT_MAX + 1))
+        self.assertEqual(str(cm.exception), "precision too big")
+
 
 if __name__ == "__main__":
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18137: Detect integer overflow on precision in float.__format__()
+  and complex.__format__().
+
 - Issue #15767: Introduce ModuleNotFoundError which is raised when a module
   could not be found.
 
diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c
--- a/Python/formatter_unicode.c
+++ b/Python/formatter_unicode.c
@@ -982,7 +982,7 @@
     Py_ssize_t n_total;
     int has_decimal;
     double val;
-    Py_ssize_t precision = format->precision;
+    Py_ssize_t precision;
     Py_ssize_t default_precision = 6;
     Py_UCS4 type = format->type;
     int add_pct = 0;
@@ -999,6 +999,12 @@
        from a hard-code pseudo-locale */
     LocaleInfo locale = STATIC_LOCALE_INFO_INIT;
 
+    if (format->precision > INT_MAX) {
+        PyErr_SetString(PyExc_ValueError, "precision too big");
+        goto done;
+    }
+    precision = (int)format->precision;
+
     if (format->alternate)
         flags |= Py_DTSF_ALT;
 
@@ -1132,7 +1138,7 @@
     Py_ssize_t n_im_total;
     int re_has_decimal;
     int im_has_decimal;
-    Py_ssize_t precision = format->precision;
+    int precision;
     Py_ssize_t default_precision = 6;
     Py_UCS4 type = format->type;
     Py_ssize_t i_re;
@@ -1160,6 +1166,12 @@
        from a hard-code pseudo-locale */
     LocaleInfo locale = STATIC_LOCALE_INFO_INIT;
 
+    if (format->precision > INT_MAX) {
+        PyErr_SetString(PyExc_ValueError, "precision too big");
+        goto done;
+    }
+    precision = (int)format->precision;
+
     /* Zero padding is not allowed. */
     if (format->fill_char == '0') {
         PyErr_SetString(PyExc_ValueError,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 14:57:38 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 14:57:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTM3?=
 =?utf-8?q?=3A_Detect_integer_overflow_on_precision_in_float=2E=5F=5Fforma?=
 =?utf-8?b?dF9fKCk=?=
Message-ID: <3bdYZV4Cl7z7Lk9@mail.python.org>

http://hg.python.org/cpython/rev/d2b4f59943fa
changeset:   84268:d2b4f59943fa
branch:      2.7
parent:      84259:3c0e8e910125
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 14:56:57 2013 +0200
summary:
  Issue #18137: Detect integer overflow on precision in float.__format__()
and complex.__format__().

files:
  Lib/test/test_format.py       |  17 +++++++++++++++++
  Misc/NEWS                     |   3 +++
  Objects/stringlib/formatter.h |  16 ++++++++++++++--
  3 files changed, 34 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_format.py b/Lib/test/test_format.py
--- a/Lib/test/test_format.py
+++ b/Lib/test/test_format.py
@@ -302,6 +302,23 @@
 def test_main():
     test_support.run_unittest(FormatTest)
 
+    def test_precision(self):
+        INT_MAX = 2147483647
+
+        f = 1.2
+        self.assertEqual(format(f, ".0f"), "1")
+        self.assertEqual(format(f, ".3f"), "1.200")
+        with self.assertRaises(ValueError) as cm:
+            format(f, ".%sf" % (INT_MAX + 1))
+        self.assertEqual(str(cm.exception), "precision too big")
+
+        c = complex(f)
+        self.assertEqual(format(f, ".0f"), "1")
+        self.assertEqual(format(f, ".3f"), "1.200")
+        with self.assertRaises(ValueError) as cm:
+            format(f, ".%sf" % (INT_MAX + 1))
+        self.assertEqual(str(cm.exception), "precision too big")
+
 
 if __name__ == "__main__":
     unittest.main()
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -9,6 +9,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18137: Detect integer overflow on precision in float.__format__()
+  and complex.__format__().
+
 - Issue #18038: SyntaxError raised during compilation sources with illegal
   encoding now always contains an encoding name.
 
diff --git a/Objects/stringlib/formatter.h b/Objects/stringlib/formatter.h
--- a/Objects/stringlib/formatter.h
+++ b/Objects/stringlib/formatter.h
@@ -928,7 +928,7 @@
     Py_ssize_t n_total;
     int has_decimal;
     double val;
-    Py_ssize_t precision = format->precision;
+    Py_ssize_t precision;
     Py_ssize_t default_precision = 6;
     STRINGLIB_CHAR type = format->type;
     int add_pct = 0;
@@ -947,6 +947,12 @@
        from a hard-code pseudo-locale */
     LocaleInfo locale;
 
+    if (format->precision > INT_MAX) {
+        PyErr_SetString(PyExc_ValueError, "precision too big");
+        goto done;
+    }
+    precision = (int)format->precision;
+
     /* Alternate is not allowed on floats. */
     if (format->alternate) {
         PyErr_SetString(PyExc_ValueError,
@@ -1078,7 +1084,7 @@
     Py_ssize_t n_im_total;
     int re_has_decimal;
     int im_has_decimal;
-    Py_ssize_t precision = format->precision;
+    Py_ssize_t precision;
     Py_ssize_t default_precision = 6;
     STRINGLIB_CHAR type = format->type;
     STRINGLIB_CHAR *p_re;
@@ -1107,6 +1113,12 @@
        from a hard-code pseudo-locale */
     LocaleInfo locale;
 
+    if (format->precision > INT_MAX) {
+        PyErr_SetString(PyExc_ValueError, "precision too big");
+        goto done;
+    }
+    precision = (int)format->precision;
+
     /* Alternate is not allowed on complex. */
     if (format->alternate) {
         PyErr_SetString(PyExc_ValueError,

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:18:48 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 15:18:48 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogX3NzbC5jOiBzdHJp?=
	=?utf-8?q?p_trailing_spaces?=
Message-ID: <3bdZ2w16lWz7LjQ@mail.python.org>

http://hg.python.org/cpython/rev/1bd49f8a30da
changeset:   84269:1bd49f8a30da
branch:      3.3
parent:      84266:ef5175d08e7e
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 14:58:43 2013 +0200
summary:
  _ssl.c: strip trailing spaces

files:
  Modules/_ssl.c |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1136,7 +1136,7 @@
     const unsigned char *out;
     unsigned int outlen;
 
-    SSL_get0_next_proto_negotiated(self->ssl, 
+    SSL_get0_next_proto_negotiated(self->ssl,
                                    &out, &outlen);
 
     if (out == NULL)
@@ -1763,8 +1763,8 @@
 #ifdef OPENSSL_NPN_NEGOTIATED
 /* this callback gets passed to SSL_CTX_set_next_protos_advertise_cb */
 static int
-_advertiseNPN_cb(SSL *s, 
-                 const unsigned char **data, unsigned int *len, 
+_advertiseNPN_cb(SSL *s,
+                 const unsigned char **data, unsigned int *len,
                  void *args)
 {
     PySSLContext *ssl_ctx = (PySSLContext *) args;
@@ -1781,7 +1781,7 @@
 }
 /* this callback gets passed to SSL_CTX_set_next_proto_select_cb */
 static int
-_selectNPN_cb(SSL *s, 
+_selectNPN_cb(SSL *s,
               unsigned char **out, unsigned char *outlen,
               const unsigned char *server, unsigned int server_len,
               void *args)
@@ -2848,7 +2848,7 @@
     }
     if (PyModule_AddObject(m, "lib_codes_to_names", lib_codes_to_names))
         return NULL;
-    
+
     /* OpenSSL version */
     /* SSLeay() gives us the version of the library linked against,
        which could be different from the headers version.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:18:49 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 15:18:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTM1?=
 =?utf-8?q?=3A_Fix_a_possible_integer_overflow_in_ssl=2ESSLSocket=2Ewrite?=
 =?utf-8?b?KCk=?=
Message-ID: <3bdZ2x3B4Qz7LkG@mail.python.org>

http://hg.python.org/cpython/rev/f0d934732ab1
changeset:   84270:f0d934732ab1
branch:      3.3
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 15:08:23 2013 +0200
summary:
  Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
and in ssl.SSLContext.load_cert_chain() for strings and passwords longer
than 2 gigabytes.

files:
  Misc/NEWS      |   4 ++++
  Modules/_ssl.c |  16 +++++++++++-----
  2 files changed, 15 insertions(+), 5 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -35,6 +35,10 @@
 Library
 -------
 
+- Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
+  and in ssl.SSLContext.load_cert_chain() for strings and passwords longer than
+  2 gigabytes.
+
 - Issue #18248: Fix libffi build on AIX.
 
 - Issue #18259: Declare sethostname in socketmodule.c for AIX
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1284,8 +1284,9 @@
         goto error;
     }
     do {
+        len = (int)Py_MIN(buf.len, INT_MAX);
         PySSL_BEGIN_ALLOW_THREADS
-        len = SSL_write(self->ssl, buf.buf, buf.len);
+        len = SSL_write(self->ssl, buf.buf, len);
         err = SSL_get_error(self->ssl, len);
         PySSL_END_ALLOW_THREADS
         if (PyErr_CheckSignals()) {
@@ -1576,7 +1577,7 @@
 {
     PyObject *retval = NULL;
     char buf[PySSL_CB_MAXLEN];
-    int len;
+    size_t len;
 
     if (SSL_session_reused(self->ssl) ^ !self->socket_type) {
         /* if session is resumed XOR we are the client */
@@ -1588,7 +1589,6 @@
     }
 
     /* It cannot be negative in current OpenSSL version as of July 2011 */
-    assert(len >= 0);
     if (len == 0)
         Py_RETURN_NONE;
 
@@ -1915,7 +1915,7 @@
     PyThreadState *thread_state;
     PyObject *callable;
     char *password;
-    Py_ssize_t size;
+    int size;
     int error;
 } _PySSLPasswordInfo;
 
@@ -1949,6 +1949,12 @@
         goto error;
     }
 
+    if (size > (Py_ssize_t)INT_MAX) {
+        PyErr_Format(PyExc_ValueError,
+                     "password cannot be longer than %d bytes", INT_MAX);
+        goto error;
+    }
+
     free(pw_info->password);
     pw_info->password = malloc(size);
     if (!pw_info->password) {
@@ -1957,7 +1963,7 @@
         goto error;
     }
     memcpy(pw_info->password, data, size);
-    pw_info->size = size;
+    pw_info->size = (int)size;
 
     Py_XDECREF(password_bytes);
     return 1;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:18:50 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 15:18:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_=28Merge_3=2E3=29_Issue_=2318135=3A_Fix_a_possible_integ?=
 =?utf-8?q?er_overflow_in?=
Message-ID: <3bdZ2y57Fkz7LjY@mail.python.org>

http://hg.python.org/cpython/rev/f90d82a75a43
changeset:   84271:f90d82a75a43
parent:      84267:81fef2666ebb
parent:      84270:f0d934732ab1
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 15:09:26 2013 +0200
summary:
  (Merge 3.3) Issue #18135: Fix a possible integer overflow in
ssl.SSLSocket.write() and in ssl.SSLContext.load_cert_chain() for strings and
passwords longer than 2 gigabytes.

files:
  Misc/NEWS      |   4 ++++
  Modules/_ssl.c |  26 ++++++++++++++++----------
  2 files changed, 20 insertions(+), 10 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -126,6 +126,10 @@
 Library
 -------
 
+- Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
+  and in ssl.SSLContext.load_cert_chain() for strings and passwords longer than
+  2 gigabytes.
+
 - Issue #11016: Add C implementation of the stat module as _stat.
 
 - Issue #18248: Fix libffi build on AIX.
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1172,7 +1172,7 @@
     const unsigned char *out;
     unsigned int outlen;
 
-    SSL_get0_next_proto_negotiated(self->ssl, 
+    SSL_get0_next_proto_negotiated(self->ssl,
                                    &out, &outlen);
 
     if (out == NULL)
@@ -1358,8 +1358,9 @@
         goto error;
     }
     do {
+        len = (int)Py_MIN(buf.len, INT_MAX);
         PySSL_BEGIN_ALLOW_THREADS
-        len = SSL_write(self->ssl, buf.buf, buf.len);
+        len = SSL_write(self->ssl, buf.buf, len);
         err = SSL_get_error(self->ssl, len);
         PySSL_END_ALLOW_THREADS
         if (PyErr_CheckSignals()) {
@@ -1650,7 +1651,7 @@
 {
     PyObject *retval = NULL;
     char buf[PySSL_CB_MAXLEN];
-    int len;
+    size_t len;
 
     if (SSL_session_reused(self->ssl) ^ !self->socket_type) {
         /* if session is resumed XOR we are the client */
@@ -1662,7 +1663,6 @@
     }
 
     /* It cannot be negative in current OpenSSL version as of July 2011 */
-    assert(len >= 0);
     if (len == 0)
         Py_RETURN_NONE;
 
@@ -1873,8 +1873,8 @@
 #ifdef OPENSSL_NPN_NEGOTIATED
 /* this callback gets passed to SSL_CTX_set_next_protos_advertise_cb */
 static int
-_advertiseNPN_cb(SSL *s, 
-                 const unsigned char **data, unsigned int *len, 
+_advertiseNPN_cb(SSL *s,
+                 const unsigned char **data, unsigned int *len,
                  void *args)
 {
     PySSLContext *ssl_ctx = (PySSLContext *) args;
@@ -1891,7 +1891,7 @@
 }
 /* this callback gets passed to SSL_CTX_set_next_proto_select_cb */
 static int
-_selectNPN_cb(SSL *s, 
+_selectNPN_cb(SSL *s,
               unsigned char **out, unsigned char *outlen,
               const unsigned char *server, unsigned int server_len,
               void *args)
@@ -2025,7 +2025,7 @@
     PyThreadState *thread_state;
     PyObject *callable;
     char *password;
-    Py_ssize_t size;
+    int size;
     int error;
 } _PySSLPasswordInfo;
 
@@ -2059,6 +2059,12 @@
         goto error;
     }
 
+    if (size > (Py_ssize_t)INT_MAX) {
+        PyErr_Format(PyExc_ValueError,
+                     "password cannot be longer than %d bytes", INT_MAX);
+        goto error;
+    }
+
     free(pw_info->password);
     pw_info->password = malloc(size);
     if (!pw_info->password) {
@@ -2067,7 +2073,7 @@
         goto error;
     }
     memcpy(pw_info->password, data, size);
-    pw_info->size = size;
+    pw_info->size = (int)size;
 
     Py_XDECREF(password_bytes);
     return 1;
@@ -3441,7 +3447,7 @@
     }
     if (PyModule_AddObject(m, "lib_codes_to_names", lib_codes_to_names))
         return NULL;
-    
+
     /* OpenSSL version */
     /* SSLeay() gives us the version of the library linked against,
        which could be different from the headers version.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:18:52 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 15:18:52 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTM1?=
 =?utf-8?q?=3A_Fix_a_possible_integer_overflow_in_ssl=2ESSLSocket=2Ewrite?=
 =?utf-8?b?KCk=?=
Message-ID: <3bdZ3006rvz7Lkb@mail.python.org>

http://hg.python.org/cpython/rev/d7e22acb2315
changeset:   84272:d7e22acb2315
branch:      2.7
parent:      84268:d2b4f59943fa
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 15:15:10 2013 +0200
summary:
  Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
for strings longer than 2 gigabytes.

files:
  Misc/NEWS      |  3 +++
  Modules/_ssl.c |  7 ++++++-
  2 files changed, 9 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -21,6 +21,9 @@
 Library
 -------
 
+- Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
+  for strings longer than 2 gigabytes.
+
 - Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
   when \r\n appears at end of 65535 bytes without other newlines.
 
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1212,8 +1212,13 @@
         goto error;
     }
     do {
+        if (buf.len <= INT_MAX)
+            len = (int)buf.len;
+        else
+            len = INT_MAX;
+
         PySSL_BEGIN_ALLOW_THREADS
-        len = SSL_write(self->ssl, buf.buf, buf.len);
+        len = SSL_write(self->ssl, buf.buf, len);
         err = SSL_get_error(self->ssl, len);
         PySSL_END_ALLOW_THREADS
         if (PyErr_CheckSignals()) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:47:03 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 15:47:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_add_=5Fstat_to_list_of_bui?=
	=?utf-8?q?ltin_Windows_modules?=
Message-ID: <3bdZgW5v8NzR43@mail.python.org>

http://hg.python.org/cpython/rev/838f04e5a690
changeset:   84273:838f04e5a690
parent:      84271:f90d82a75a43
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 15:46:56 2013 +0200
summary:
  add _stat to list of builtin Windows modules

files:
  PC/config.c |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/PC/config.c b/PC/config.c
--- a/PC/config.c
+++ b/PC/config.c
@@ -64,6 +64,7 @@
 extern PyObject* PyInit_atexit(void);
 extern PyObject* _PyWarnings_Init(void);
 extern PyObject* PyInit__string(void);
+extern PyObject* PyInit__stat(void);
 
 /* tools/freeze/makeconfig.py marker for additional "extern" */
 /* -- ADDMODULE MARKER 1 -- */
@@ -154,6 +155,7 @@
     {"_io", PyInit__io},
     {"_pickle", PyInit__pickle},
     {"atexit", PyInit_atexit},
+    {"_stat", PyInit__stat},
 
     /* Sentinel */
     {0, 0}

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:50:54 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 15:50:54 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_=5Fstat_module_to_VS_p?=
 =?utf-8?q?roject_file_--_this_time_with_the_GUI_instead_of?=
Message-ID: <3bdZly64y9z7Ll1@mail.python.org>

http://hg.python.org/cpython/rev/b9e0d7b14934
changeset:   84274:b9e0d7b14934
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 15:50:45 2013 +0200
summary:
  Add _stat module to VS project file -- this time with the GUI instead of manually.

files:
  PCbuild/pythoncore.vcxproj         |  4 ++--
  PCbuild/pythoncore.vcxproj.filters |  3 +++
  2 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj
--- a/PCbuild/pythoncore.vcxproj
+++ b/PCbuild/pythoncore.vcxproj
@@ -501,6 +501,7 @@
     <ClCompile Include="..\Modules\_pickle.c" />
     <ClCompile Include="..\Modules\_randommodule.c" />
     <ClCompile Include="..\Modules\_sre.c" />
+    <ClCompile Include="..\Modules\_stat.c" />
     <ClCompile Include="..\Modules\_struct.c" />
     <ClCompile Include="..\Modules\_weakref.c" />
     <ClCompile Include="..\Modules\arraymodule.c" />
@@ -525,7 +526,6 @@
     <ClCompile Include="..\Modules\sha256module.c" />
     <ClCompile Include="..\Modules\sha512module.c" />
     <ClCompile Include="..\Modules\signalmodule.c" />
-    <ClCompile Include="..\Modules\_stat.c" />
     <ClCompile Include="..\Modules\symtablemodule.c" />
     <ClCompile Include="..\Modules\_threadmodule.c" />
     <ClCompile Include="..\Modules\timemodule.c" />
@@ -679,4 +679,4 @@
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
-</Project>
+</Project>
\ No newline at end of file
diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters
--- a/PCbuild/pythoncore.vcxproj.filters
+++ b/PCbuild/pythoncore.vcxproj.filters
@@ -919,6 +919,9 @@
       <Filter>PC</Filter>
     </ClCompile>
     <ClCompile Include="..\Objects\namespaceobject.c" />
+    <ClCompile Include="..\Modules\_stat.c">
+      <Filter>Modules</Filter>
+    </ClCompile>
   </ItemGroup>
   <ItemGroup>
     <ResourceCompile Include="..\PC\python_nt.rc">

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 15:53:18 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 15:53:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_import=2Ec_does_neither_ne?=
 =?utf-8?q?ed_mode=5Ft_nor_=5Fmkdir=28=29_anymore?=
Message-ID: <3bdZpk2QKJz7Lkv@mail.python.org>

http://hg.python.org/cpython/rev/a39d2223be23
changeset:   84275:a39d2223be23
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 15:53:09 2013 +0200
summary:
  import.c does neither need mode_t nor _mkdir() anymore

files:
  Python/import.c |  8 --------
  1 files changed, 0 insertions(+), 8 deletions(-)


diff --git a/Python/import.c b/Python/import.c
--- a/Python/import.c
+++ b/Python/import.c
@@ -19,14 +19,6 @@
 extern "C" {
 #endif
 
-#ifdef MS_WINDOWS
-/* for stat.st_mode */
-typedef unsigned short mode_t;
-/* for _mkdir */
-#include <direct.h>
-#endif
-
-
 #define CACHEDIR "__pycache__"
 
 /* See _PyImport_FixupExtensionObject() below */

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 16:16:18 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 16:16:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogU29sYXJpcycgL2Rl?=
 =?utf-8?q?v/null_is_a_symlink=2E_The_device_test_now_uses_stat_instead_of?=
 =?utf-8?q?_lstat?=
Message-ID: <3bdbKG4p1vz7LjM@mail.python.org>

http://hg.python.org/cpython/rev/6c23ca1982b3
changeset:   84276:6c23ca1982b3
branch:      3.3
parent:      84270:f0d934732ab1
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 16:10:29 2013 +0200
summary:
  Solaris' /dev/null is a symlink. The device test now uses stat instead of lstat to compensate
for symlinks.

files:
  Lib/test/test_stat.py |  11 +++++++----
  1 files changed, 7 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -58,8 +58,11 @@
                 pass
     tearDown = setUp
 
-    def get_mode(self, fname=TESTFN):
-        st_mode = os.lstat(fname).st_mode
+    def get_mode(self, fname=TESTFN, lstat=True):
+        if lstat:
+            st_mode = os.lstat(fname).st_mode
+        else:
+            st_mode = os.stat(fname).st_mode
         modestr = stat.filemode(st_mode)
         return st_mode, modestr
 
@@ -149,13 +152,13 @@
     @unittest.skipUnless(os.name == 'posix', 'requires Posix')
     def test_devices(self):
         if os.path.exists(os.devnull):
-            st_mode, modestr = self.get_mode(os.devnull)
+            st_mode, modestr = self.get_mode(os.devnull, lstat=False)
             self.assertEqual(modestr[0], 'c')
             self.assertS_IS("CHR", st_mode)
         # Linux block devices, BSD has no block devices anymore
         for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
-                st_mode, modestr = self.get_mode(blockdev)
+                st_mode, modestr = self.get_mode(blockdev, lstat=False)
                 self.assertEqual(modestr[0], 'b')
                 self.assertS_IS("BLK", st_mode)
                 break

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 16:16:19 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 16:16:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Solaris=27_/dev/null_is_a_symlink=2E_The_device_test_now?=
 =?utf-8?q?_uses_stat_instead_of_lstat?=
Message-ID: <3bdbKH6ZTbz7LjY@mail.python.org>

http://hg.python.org/cpython/rev/6fd45b004d78
changeset:   84277:6fd45b004d78
parent:      84275:a39d2223be23
parent:      84276:6c23ca1982b3
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 16:11:37 2013 +0200
summary:
  Solaris' /dev/null is a symlink. The device test now uses stat instead of lstat to compensate
for symlinks.

files:
  Lib/test/test_stat.py |  11 +++++++----
  1 files changed, 7 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -62,8 +62,11 @@
                 pass
     tearDown = setUp
 
-    def get_mode(self, fname=TESTFN):
-        st_mode = os.lstat(fname).st_mode
+    def get_mode(self, fname=TESTFN, lstat=True):
+        if lstat:
+            st_mode = os.lstat(fname).st_mode
+        else:
+            st_mode = os.stat(fname).st_mode
         modestr = self.statmod.filemode(st_mode)
         return st_mode, modestr
 
@@ -153,13 +156,13 @@
     @unittest.skipUnless(os.name == 'posix', 'requires Posix')
     def test_devices(self):
         if os.path.exists(os.devnull):
-            st_mode, modestr = self.get_mode(os.devnull)
+            st_mode, modestr = self.get_mode(os.devnull, lstat=False)
             self.assertEqual(modestr[0], 'c')
             self.assertS_IS("CHR", st_mode)
         # Linux block devices, BSD has no block devices anymore
         for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
-                st_mode, modestr = self.get_mode(blockdev)
+                st_mode, modestr = self.get_mode(blockdev, lstat=False)
                 self.assertEqual(modestr[0], 'b')
                 self.assertS_IS("BLK", st_mode)
                 break

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 16:27:09 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 16:27:09 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogU29sYXJpcycgL2Rl?=
 =?utf-8?q?v/null_is_a_symlink=2E_The_device_test_now_uses_stat_instead_of?=
 =?utf-8?q?_lstat?=
Message-ID: <3bdbYn6xcYzNFr@mail.python.org>

http://hg.python.org/cpython/rev/8f0adcb66633
changeset:   84278:8f0adcb66633
branch:      2.7
parent:      84272:d7e22acb2315
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 16:27:01 2013 +0200
summary:
  Solaris' /dev/null is a symlink. The device test now uses stat instead of lstat to compensate
for symlinks.

files:
  Lib/test/test_stat.py |  12 ++++++++----
  1 files changed, 8 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -58,8 +58,12 @@
                 pass
     tearDown = setUp
 
-    def get_mode(self, fname=TESTFN):
-        return os.lstat(fname).st_mode
+    def get_mode(self, fname=TESTFN, lstat=True):
+        if lstat:
+            st_mode = os.lstat(fname).st_mode
+        else:
+            st_mode = os.stat(fname).st_mode
+        return st_mode
 
     def assertS_IS(self, name, mode):
         # test format, lstrip is for S_IFIFO
@@ -136,12 +140,12 @@
     @unittest.skipUnless(os.name == 'posix', 'requires Posix')
     def test_devices(self):
         if os.path.exists(os.devnull):
-            st_mode = self.get_mode(os.devnull)
+            st_mode = self.get_mode(os.devnull, lstat=False)
             self.assertS_IS("CHR", st_mode)
         # Linux block devices, BSD has no block devices anymore
         for blockdev in ("/dev/sda", "/dev/hda"):
             if os.path.exists(blockdev):
-                st_mode = self.get_mode(blockdev)
+                st_mode = self.get_mode(blockdev, lstat=False)
                 self.assertS_IS("BLK", st_mode)
                 break
 

-- 
Repository URL: http://hg.python.org/cpython

From fijall at gmail.com  Sun Jun 23 17:40:13 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 23 Jun 2013 17:40:13 +0200
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
In-Reply-To: <CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
Message-ID: <CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>

On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
> <python-checkins at python.org> wrote:
>>
>> http://hg.python.org/cpython/rev/dfead0696a71
>> changeset:   84224:dfead0696a71
>> branch:      3.3
>> parent:      84221:0113247f894b
>> user:        Senthil Kumaran <senthil at uthcode.com>
>> date:        Wed Jun 19 22:19:46 2013 -0500
>> summary:
>>   Add -b and -X options to python man page.
>> Patch contributed by Corey Brune.
>>
>> files:
>>   Misc/python.man |  22 ++++++++++++++++++----
>>   1 files changed, 18 insertions(+), 4 deletions(-)
>>
>>
>> diff --git a/Misc/python.man b/Misc/python.man
>> --- a/Misc/python.man
>> +++ b/Misc/python.man
>> @@ -11,6 +11,9 @@
>>  .B \-B
>>  ]
>>  [
>> +.B \-b
>> +]
>> +[
>>  .B \-d
>>  ]
>>  [
>> @@ -23,14 +26,14 @@
>>  .B \-i
>>  ]
>>  [
>> -.B \-m
>> +.B \-m
>>  .I module-name
>>  ]
>> -[
>> -.B \-q
>> -]
>>  .br
>>         [
>> +.B \-q
>> +]
>> +[
>>  .B \-O
>>  ]
>>  [
>> @@ -60,6 +63,10 @@
>>  .B \-x
>>  ]
>>  [
>> +[
>> +.B \-X
>> +.I option
>> +]
>>  .B \-?
>>  ]
>>  .br
>> @@ -105,6 +112,10 @@
>>  .I .py[co]
>>  files on import. See also PYTHONDONTWRITEBYTECODE.
>>  .TP
>> +.B \-b
>> +Issue warnings about str(bytes_instance), str(bytearray_instance)
>> +and comparing bytes/bytearray with str. (-bb: issue errors)
>> +.TP
>>  .BI "\-c " command
>>  Specify the command to execute (see next section).
>>  This terminates the option list (following options are passed as
>> @@ -243,6 +254,9 @@
>>  field matches the line number, where zero matches all line numbers and
>>  is thus equivalent to an omitted line number.
>>  .TP
>> +.BI "\-X " option
>> +Set implementation specific option.
>
>
> Should probably be "Set the implementation-specific option."

Is there anyone respecting this notation? (I know pypy does not, it
uses --jit and stuff)

>
>>
>> +.TP
>>  .B \-x
>>  Skip the first line of the source.  This is intended for a DOS
>>  specific hack only.  Warning: the line numbers in error messages will
>>
>> --
>> Repository URL: http://hg.python.org/cpython
>>
>> _______________________________________________
>> Python-checkins mailing list
>> Python-checkins at python.org
>> http://mail.python.org/mailman/listinfo/python-checkins
>>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>

From rdmurray at bitdance.com  Sun Jun 23 18:14:38 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 23 Jun 2013 12:14:38 -0400
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
	options to python man page.
In-Reply-To: <CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
Message-ID: <20130623161440.2587A250338@webabinitio.net>

On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
> > <python-checkins at python.org> wrote:
> >>  .TP
> >> +.BI "\-X " option
> >> +Set implementation specific option.
> >
> >
> > Should probably be "Set the implementation-specific option."
> 
> Is there anyone respecting this notation? (I know pypy does not, it
> uses --jit and stuff)

CPython does.  We introduced it for ourselves, it is up to other
implementations whether or not to use it, or use something else.

--David

From python-checkins at python.org  Sun Jun 23 19:12:55 2013
From: python-checkins at python.org (lukasz.langa)
Date: Sun, 23 Jun 2013 19:12:55 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fixed_issue_?=
 =?utf-8?q?=2318260=3A_configparser_TypeError_on_source_name_specified_as_?=
 =?utf-8?q?bytes?=
Message-ID: <3bdgF320pxz7Ll1@mail.python.org>

http://hg.python.org/cpython/rev/56c1227f21f5
changeset:   84279:56c1227f21f5
branch:      3.3
parent:      84276:6c23ca1982b3
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Sun Jun 23 19:10:25 2013 +0200
summary:
  Fixed issue #18260: configparser TypeError on source name specified as bytes

files:
  Lib/configparser.py           |   4 +-
  Lib/test/test_configparser.py |  74 +++++++++++++++++++---
  2 files changed, 65 insertions(+), 13 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -191,7 +191,7 @@
     def __init__(self, section, source=None, lineno=None):
         msg = [repr(section), " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": section ")
@@ -217,7 +217,7 @@
         msg = [repr(option), " in section ", repr(section),
                " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": option ")
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -626,15 +626,15 @@
                     oops{equals}this won't
                 """.format(equals=self.delimiters[0])), source='<foo-bar>')
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <foo-bar> [line  5]: "
-                                     "section 'Foo' already exists")
+            self.assertEqual(str(e), "While reading from '<foo-bar>' "
+                                     "[line  5]: section 'Foo' already exists")
             self.assertEqual(e.args, ("Foo", '<foo-bar>', 5))
 
             with self.assertRaises(configparser.DuplicateOptionError) as cm:
                 cf.read_dict({'Bar': {'opt': 'val', 'OPT': 'is really `opt`'}})
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <dict>: option 'opt' "
-                                     "in section 'Bar' already exists")
+            self.assertEqual(str(e), "While reading from '<dict>': option "
+                                     "'opt' in section 'Bar' already exists")
             self.assertEqual(e.args, ("Bar", "opt", "<dict>", None))
 
     def test_write(self):
@@ -1419,13 +1419,18 @@
 
 class ReadFileTestCase(unittest.TestCase):
     def test_file(self):
-        file_path = support.findfile("cfgparser.1")
-        parser = configparser.ConfigParser()
-        with open(file_path) as f:
-            parser.read_file(f)
-        self.assertIn("Foo Bar", parser)
-        self.assertIn("foo", parser["Foo Bar"])
-        self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
+        file_paths = [support.findfile("cfgparser.1")]
+        try:
+            file_paths.append(file_paths[0].encode('utf8'))
+        except UnicodeEncodeError:
+            pass   # unfortunately we can't test bytes on this path
+        for file_path in file_paths:
+            parser = configparser.ConfigParser()
+            with open(file_path) as f:
+                parser.read_file(f)
+            self.assertIn("Foo Bar", parser)
+            self.assertIn("foo", parser["Foo Bar"])
+            self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
     def test_iterable(self):
         lines = textwrap.dedent("""
@@ -1447,6 +1452,53 @@
         self.assertIn("foo", parser["Foo Bar"])
         self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
+    def test_source_as_bytes(self):
+        """Issue #18260."""
+        lines = textwrap.dedent("""
+        [badbad]
+        [badbad]""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateSectionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  2]: section 'badbad' "
+            "already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        bad = bad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateOptionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  3]: option 'bad' in section "
+            "'badbad' already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.ParsingError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "Source contains parsing errors: b'badbad'\n\t[line  2]: '= bad'"
+        )
+        lines = textwrap.dedent("""
+        [badbad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.MissingSectionHeaderError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "File contains no section headers.\nfile: b'badbad', line: 1\n"
+            "'[badbad'"
+        )
+
 
 class CoverageOneHundredTestCase(unittest.TestCase):
     """Covers edge cases in the codebase."""

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:12:56 2013
From: python-checkins at python.org (lukasz.langa)
Date: Sun, 23 Jun 2013 19:12:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merged_fix_for_issue_=2318260_from_3=2E3?=
Message-ID: <3bdgF456ytz7Ll4@mail.python.org>

http://hg.python.org/cpython/rev/06e70937364b
changeset:   84280:06e70937364b
parent:      84277:6fd45b004d78
parent:      84279:56c1227f21f5
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Sun Jun 23 19:12:12 2013 +0200
summary:
  Merged fix for issue #18260 from 3.3

files:
  Lib/configparser.py           |   4 +-
  Lib/test/test_configparser.py |  74 +++++++++++++++++++---
  2 files changed, 65 insertions(+), 13 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -191,7 +191,7 @@
     def __init__(self, section, source=None, lineno=None):
         msg = [repr(section), " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": section ")
@@ -217,7 +217,7 @@
         msg = [repr(option), " in section ", repr(section),
                " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": option ")
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -626,15 +626,15 @@
                     oops{equals}this won't
                 """.format(equals=self.delimiters[0])), source='<foo-bar>')
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <foo-bar> [line  5]: "
-                                     "section 'Foo' already exists")
+            self.assertEqual(str(e), "While reading from '<foo-bar>' "
+                                     "[line  5]: section 'Foo' already exists")
             self.assertEqual(e.args, ("Foo", '<foo-bar>', 5))
 
             with self.assertRaises(configparser.DuplicateOptionError) as cm:
                 cf.read_dict({'Bar': {'opt': 'val', 'OPT': 'is really `opt`'}})
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <dict>: option 'opt' "
-                                     "in section 'Bar' already exists")
+            self.assertEqual(str(e), "While reading from '<dict>': option "
+                                     "'opt' in section 'Bar' already exists")
             self.assertEqual(e.args, ("Bar", "opt", "<dict>", None))
 
     def test_write(self):
@@ -1419,13 +1419,18 @@
 
 class ReadFileTestCase(unittest.TestCase):
     def test_file(self):
-        file_path = support.findfile("cfgparser.1")
-        parser = configparser.ConfigParser()
-        with open(file_path) as f:
-            parser.read_file(f)
-        self.assertIn("Foo Bar", parser)
-        self.assertIn("foo", parser["Foo Bar"])
-        self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
+        file_paths = [support.findfile("cfgparser.1")]
+        try:
+            file_paths.append(file_paths[0].encode('utf8'))
+        except UnicodeEncodeError:
+            pass   # unfortunately we can't test bytes on this path
+        for file_path in file_paths:
+            parser = configparser.ConfigParser()
+            with open(file_path) as f:
+                parser.read_file(f)
+            self.assertIn("Foo Bar", parser)
+            self.assertIn("foo", parser["Foo Bar"])
+            self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
     def test_iterable(self):
         lines = textwrap.dedent("""
@@ -1447,6 +1452,53 @@
         self.assertIn("foo", parser["Foo Bar"])
         self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
+    def test_source_as_bytes(self):
+        """Issue #18260."""
+        lines = textwrap.dedent("""
+        [badbad]
+        [badbad]""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateSectionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  2]: section 'badbad' "
+            "already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        bad = bad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateOptionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  3]: option 'bad' in section "
+            "'badbad' already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.ParsingError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "Source contains parsing errors: b'badbad'\n\t[line  2]: '= bad'"
+        )
+        lines = textwrap.dedent("""
+        [badbad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.MissingSectionHeaderError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "File contains no section headers.\nfile: b'badbad', line: 1\n"
+            "'[badbad'"
+        )
+
 
 class CoverageOneHundredTestCase(unittest.TestCase):
     """Covers edge cases in the codebase."""

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:55:57 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 23 Jun 2013 19:55:57 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTg0?=
 =?utf-8?q?=3A_PyUnicode=5FFromFormat=28=29_and_PyUnicode=5FFromFormatV=28?=
 =?utf-8?q?=29_now_raise?=
Message-ID: <3bdhBj16dFz7Lkl@mail.python.org>

http://hg.python.org/cpython/rev/f8ede55cf92b
changeset:   84281:f8ede55cf92b
branch:      3.3
parent:      84276:6c23ca1982b3
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 23 20:12:14 2013 +0300
summary:
  Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
OverflowError when an argument of %c format is out of range.

files:
  Lib/test/test_unicode.py |  2 ++
  Misc/NEWS                |  3 +++
  Objects/unicodeobject.c  |  9 +++++++--
  3 files changed, 12 insertions(+), 2 deletions(-)


diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py
+++ b/Lib/test/test_unicode.py
@@ -2024,6 +2024,8 @@
         # test "%c"
         self.assertEqual(PyUnicode_FromFormat(b'%c', c_int(0xabcd)), '\uabcd')
         self.assertEqual(PyUnicode_FromFormat(b'%c', c_int(0x10ffff)), '\U0010ffff')
+        with self.assertRaises(OverflowError):
+            PyUnicode_FromFormat(b'%c', c_int(0x110000))
         # Issue #18183
         self.assertEqual(
             PyUnicode_FromFormat(b'%c%c', c_int(0x10000), c_int(0x100000)),
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
+  OverflowError when an argument of %c format is out of range.
+
 - Issue #18137: Detect integer overflow on precision in float.__format__()
   and complex.__format__().
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -2489,8 +2489,13 @@
             switch (*f) {
             case 'c':
             {
-                Py_UCS4 ordinal = va_arg(count, int);
-                maxchar = Py_MAX(maxchar, ordinal);
+                int ordinal = va_arg(count, int);
+                if (ordinal < 0 || ordinal > MAX_UNICODE) {
+                    PyErr_SetString(PyExc_OverflowError,
+                                    "%c arg not in range(0x110000)");
+                    goto fail;
+                }
+                maxchar = Py_MAX(maxchar, (Py_UCS4)ordinal);
                 n++;
                 break;
             }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:55:58 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 23 Jun 2013 19:55:58 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318184=3A_PyUnicode=5FFromFormat=28=29_and_PyUni?=
 =?utf-8?q?code=5FFromFormatV=28=29_now_raise?=
Message-ID: <3bdhBk3VJnz7Lks@mail.python.org>

http://hg.python.org/cpython/rev/42def600210e
changeset:   84282:42def600210e
parent:      84277:6fd45b004d78
parent:      84281:f8ede55cf92b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 23 20:21:16 2013 +0300
summary:
  Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
OverflowError when an argument of %c format is out of range.

files:
  Lib/test/test_unicode.py |  2 ++
  Misc/NEWS                |  3 +++
  Objects/unicodeobject.c  |  2 +-
  3 files changed, 6 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py
--- a/Lib/test/test_unicode.py
+++ b/Lib/test/test_unicode.py
@@ -2048,6 +2048,8 @@
                      b'%c', c_int(0xabcd))
         check_format('\U0010ffff',
                      b'%c', c_int(0x10ffff))
+        with self.assertRaises(OverflowError):
+            PyUnicode_FromFormat(b'%c', c_int(0x110000))
         # Issue #18183
         check_format('\U00010000\U00100000',
                      b'%c%c', c_int(0x10000), c_int(0x100000))
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
+  OverflowError when an argument of %c format is out of range.
+
 - Issue #18137: Detect integer overflow on precision in float.__format__()
   and complex.__format__().
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -2496,7 +2496,7 @@
     {
         int ordinal = va_arg(*vargs, int);
         if (ordinal < 0 || ordinal > MAX_UNICODE) {
-            PyErr_SetString(PyExc_ValueError,
+            PyErr_SetString(PyExc_OverflowError,
                             "character argument not in range(0x110000)");
             return NULL;
         }

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:55:59 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 23 Jun 2013 19:55:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTg0?=
 =?utf-8?q?=3A_PyUnicode=5FFromFormat=28=29_and_PyUnicode=5FFromFormatV=28?=
 =?utf-8?q?=29_now_raise?=
Message-ID: <3bdhBl5fkbz7LlB@mail.python.org>

http://hg.python.org/cpython/rev/2f1e8b7fa534
changeset:   84283:2f1e8b7fa534
branch:      2.7
parent:      84278:8f0adcb66633
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 23 20:22:09 2013 +0300
summary:
  Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
OverflowError when an argument of %c format is out of range.

files:
  Misc/NEWS               |   3 +++
  Objects/unicodeobject.c |  19 ++++++++++++++++++-
  2 files changed, 21 insertions(+), 1 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -9,6 +9,9 @@
 Core and Builtins
 -----------------
 
+- Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
+  OverflowError when an argument of %c format is out of range.
+
 - Issue #18137: Detect integer overflow on precision in float.__format__()
   and complex.__format__().
 
diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -740,8 +740,25 @@
 
             switch (*f) {
             case 'c':
-                (void)va_arg(count, int);
+            {
+                int ordinal = va_arg(count, int);
+#ifdef Py_UNICODE_WIDE
+                if (ordinal < 0 || ordinal > 0x10ffff) {
+                    PyErr_SetString(PyExc_OverflowError,
+                                    "%c arg not in range(0x110000) "
+                                    "(wide Python build)");
+                    goto fail;
+                }
+#else
+                if (ordinal < 0 || ordinal > 0xffff) {
+                    PyErr_SetString(PyExc_OverflowError,
+                                    "%c arg not in range(0x10000) "
+                                    "(narrow Python build)");
+                    goto fail;
+                }
+#endif
                 /* fall through... */
+            }
             case '%':
                 n++;
                 break;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:56:01 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 23 Jun 2013 19:56:01 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy4zIC0+IDMuMyk6?=
	=?utf-8?q?_Merge_heads?=
Message-ID: <3bdhBn2pFXz7LlC@mail.python.org>

http://hg.python.org/cpython/rev/059432647de0
changeset:   84284:059432647de0
branch:      3.3
parent:      84281:f8ede55cf92b
parent:      84279:56c1227f21f5
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 23 20:25:40 2013 +0300
summary:
  Merge heads

files:
  Lib/configparser.py           |   4 +-
  Lib/test/test_configparser.py |  74 +++++++++++++++++++---
  2 files changed, 65 insertions(+), 13 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -191,7 +191,7 @@
     def __init__(self, section, source=None, lineno=None):
         msg = [repr(section), " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": section ")
@@ -217,7 +217,7 @@
         msg = [repr(option), " in section ", repr(section),
                " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": option ")
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -626,15 +626,15 @@
                     oops{equals}this won't
                 """.format(equals=self.delimiters[0])), source='<foo-bar>')
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <foo-bar> [line  5]: "
-                                     "section 'Foo' already exists")
+            self.assertEqual(str(e), "While reading from '<foo-bar>' "
+                                     "[line  5]: section 'Foo' already exists")
             self.assertEqual(e.args, ("Foo", '<foo-bar>', 5))
 
             with self.assertRaises(configparser.DuplicateOptionError) as cm:
                 cf.read_dict({'Bar': {'opt': 'val', 'OPT': 'is really `opt`'}})
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <dict>: option 'opt' "
-                                     "in section 'Bar' already exists")
+            self.assertEqual(str(e), "While reading from '<dict>': option "
+                                     "'opt' in section 'Bar' already exists")
             self.assertEqual(e.args, ("Bar", "opt", "<dict>", None))
 
     def test_write(self):
@@ -1419,13 +1419,18 @@
 
 class ReadFileTestCase(unittest.TestCase):
     def test_file(self):
-        file_path = support.findfile("cfgparser.1")
-        parser = configparser.ConfigParser()
-        with open(file_path) as f:
-            parser.read_file(f)
-        self.assertIn("Foo Bar", parser)
-        self.assertIn("foo", parser["Foo Bar"])
-        self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
+        file_paths = [support.findfile("cfgparser.1")]
+        try:
+            file_paths.append(file_paths[0].encode('utf8'))
+        except UnicodeEncodeError:
+            pass   # unfortunately we can't test bytes on this path
+        for file_path in file_paths:
+            parser = configparser.ConfigParser()
+            with open(file_path) as f:
+                parser.read_file(f)
+            self.assertIn("Foo Bar", parser)
+            self.assertIn("foo", parser["Foo Bar"])
+            self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
     def test_iterable(self):
         lines = textwrap.dedent("""
@@ -1447,6 +1452,53 @@
         self.assertIn("foo", parser["Foo Bar"])
         self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
+    def test_source_as_bytes(self):
+        """Issue #18260."""
+        lines = textwrap.dedent("""
+        [badbad]
+        [badbad]""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateSectionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  2]: section 'badbad' "
+            "already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        bad = bad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateOptionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  3]: option 'bad' in section "
+            "'badbad' already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.ParsingError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "Source contains parsing errors: b'badbad'\n\t[line  2]: '= bad'"
+        )
+        lines = textwrap.dedent("""
+        [badbad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.MissingSectionHeaderError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "File contains no section headers.\nfile: b'badbad', line: 1\n"
+            "'[badbad'"
+        )
+
 
 class CoverageOneHundredTestCase(unittest.TestCase):
     """Covers edge cases in the codebase."""

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:56:02 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 23 Jun 2013 19:56:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_heads?=
Message-ID: <3bdhBp6PCkz7Ll4@mail.python.org>

http://hg.python.org/cpython/rev/0d81489bc0a6
changeset:   84285:0d81489bc0a6
parent:      84282:42def600210e
parent:      84280:06e70937364b
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 23 20:25:57 2013 +0300
summary:
  Merge heads

files:
  Lib/configparser.py           |   4 +-
  Lib/test/test_configparser.py |  74 +++++++++++++++++++---
  2 files changed, 65 insertions(+), 13 deletions(-)


diff --git a/Lib/configparser.py b/Lib/configparser.py
--- a/Lib/configparser.py
+++ b/Lib/configparser.py
@@ -191,7 +191,7 @@
     def __init__(self, section, source=None, lineno=None):
         msg = [repr(section), " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": section ")
@@ -217,7 +217,7 @@
         msg = [repr(option), " in section ", repr(section),
                " already exists"]
         if source is not None:
-            message = ["While reading from ", source]
+            message = ["While reading from ", repr(source)]
             if lineno is not None:
                 message.append(" [line {0:2d}]".format(lineno))
             message.append(": option ")
diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py
--- a/Lib/test/test_configparser.py
+++ b/Lib/test/test_configparser.py
@@ -626,15 +626,15 @@
                     oops{equals}this won't
                 """.format(equals=self.delimiters[0])), source='<foo-bar>')
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <foo-bar> [line  5]: "
-                                     "section 'Foo' already exists")
+            self.assertEqual(str(e), "While reading from '<foo-bar>' "
+                                     "[line  5]: section 'Foo' already exists")
             self.assertEqual(e.args, ("Foo", '<foo-bar>', 5))
 
             with self.assertRaises(configparser.DuplicateOptionError) as cm:
                 cf.read_dict({'Bar': {'opt': 'val', 'OPT': 'is really `opt`'}})
             e = cm.exception
-            self.assertEqual(str(e), "While reading from <dict>: option 'opt' "
-                                     "in section 'Bar' already exists")
+            self.assertEqual(str(e), "While reading from '<dict>': option "
+                                     "'opt' in section 'Bar' already exists")
             self.assertEqual(e.args, ("Bar", "opt", "<dict>", None))
 
     def test_write(self):
@@ -1419,13 +1419,18 @@
 
 class ReadFileTestCase(unittest.TestCase):
     def test_file(self):
-        file_path = support.findfile("cfgparser.1")
-        parser = configparser.ConfigParser()
-        with open(file_path) as f:
-            parser.read_file(f)
-        self.assertIn("Foo Bar", parser)
-        self.assertIn("foo", parser["Foo Bar"])
-        self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
+        file_paths = [support.findfile("cfgparser.1")]
+        try:
+            file_paths.append(file_paths[0].encode('utf8'))
+        except UnicodeEncodeError:
+            pass   # unfortunately we can't test bytes on this path
+        for file_path in file_paths:
+            parser = configparser.ConfigParser()
+            with open(file_path) as f:
+                parser.read_file(f)
+            self.assertIn("Foo Bar", parser)
+            self.assertIn("foo", parser["Foo Bar"])
+            self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
     def test_iterable(self):
         lines = textwrap.dedent("""
@@ -1447,6 +1452,53 @@
         self.assertIn("foo", parser["Foo Bar"])
         self.assertEqual(parser["Foo Bar"]["foo"], "newbar")
 
+    def test_source_as_bytes(self):
+        """Issue #18260."""
+        lines = textwrap.dedent("""
+        [badbad]
+        [badbad]""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateSectionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  2]: section 'badbad' "
+            "already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        bad = bad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.DuplicateOptionError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "While reading from b'badbad' [line  3]: option 'bad' in section "
+            "'badbad' already exists"
+        )
+        lines = textwrap.dedent("""
+        [badbad]
+        = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.ParsingError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "Source contains parsing errors: b'badbad'\n\t[line  2]: '= bad'"
+        )
+        lines = textwrap.dedent("""
+        [badbad
+        bad = bad""").strip().split('\n')
+        parser = configparser.ConfigParser()
+        with self.assertRaises(configparser.MissingSectionHeaderError) as dse:
+            parser.read_file(lines, source=b"badbad")
+        self.assertEqual(
+            str(dse.exception),
+            "File contains no section headers.\nfile: b'badbad', line: 1\n"
+            "'[badbad'"
+        )
+
 
 class CoverageOneHundredTestCase(unittest.TestCase):
     """Covers edge cases in the codebase."""

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 19:56:04 2013
From: python-checkins at python.org (serhiy.storchaka)
Date: Sun, 23 Jun 2013 19:56:04 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_heads?=
Message-ID: <3bdhBr1dBvz7LlW@mail.python.org>

http://hg.python.org/cpython/rev/f2c0e09ef806
changeset:   84286:f2c0e09ef806
parent:      84285:0d81489bc0a6
parent:      84284:059432647de0
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Jun 23 20:27:40 2013 +0300
summary:
  Merge heads

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 20:24:27 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 20:24:27 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2311390=3A_convert_doctes?=
 =?utf-8?q?t_CLI_to_argparse_and_add_-o_and_-f_options=2E?=
Message-ID: <3bdhqb0Qc7z7Lkl@mail.python.org>

http://hg.python.org/cpython/rev/ae802dc4dcd4
changeset:   84287:ae802dc4dcd4
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 14:24:13 2013 -0400
summary:
  #11390: convert doctest CLI to argparse and add -o and -f options.

This provides a way to specify arbitrary doctest options when using
the CLI interface to process test files, just as one can when calling
testmod or testfile programmatically.

files:
  Doc/library/doctest.rst  |    8 +-
  Doc/whatsnew/3.4.rst     |   11 +-
  Lib/doctest.py           |   37 +++-
  Lib/test/test_doctest.py |  226 +++++++++++++++++++++++++++
  Misc/NEWS                |    3 +
  5 files changed, 273 insertions(+), 12 deletions(-)


diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst
--- a/Doc/library/doctest.rst
+++ b/Doc/library/doctest.rst
@@ -495,7 +495,10 @@
 A number of option flags control various aspects of doctest's behavior.
 Symbolic names for the flags are supplied as module constants, which can be
 or'ed together and passed to various functions.  The names can also be used in
-:ref:`doctest directives <doctest-directives>`.
+:ref:`doctest directives <doctest-directives>`, and may be passed to the
+doctest command line interface via the ``-o`` option.
+
+.. versionadded:: 3.4 the ``-o`` command line option
 
 The first group of options define test semantics, controlling aspects of how
 doctest decides whether actual output matches an example's expected output:
@@ -640,6 +643,9 @@
    1.  This flag may be useful during debugging, since examples after the first
    failure won't even produce debugging output.
 
+   The doctest command line accepts the option ``-f`` as a shorthand for ``-o
+   FAIL_FAST``.
+
    .. versionadded:: 3.4
 
 
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -173,8 +173,15 @@
 doctest
 -------
 
-Added ``FAIL_FAST`` flag to halt test running as soon as the first failure is
-detected.  (Contributed by R. David Murray and Daniel Urban in :issue:`16522`.)
+Added :data:`~doctest.FAIL_FAST` flag to halt test running as soon as the first
+failure is detected.  (Contributed by R. David Murray and Daniel Urban in
+:issue:`16522`.)
+
+Updated the doctest command line interface to use :mod:`argparse`, and added
+``-o`` and ``-f`` options to the interface.  ``-o`` allows doctest options to
+be specified on the command line, and ``-f`` is a shorthand for ``-o
+FAIL_FAST`` (to parallel the similar option supported by the :mod:`unittest`
+CLI).  (Contributed by R. David Murray in :issue:`11390`.)
 
 
 functools
diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -93,6 +93,7 @@
 ]
 
 import __future__
+import argparse
 import difflib
 import inspect
 import linecache
@@ -2708,13 +2709,30 @@
 
 
 def _test():
-    testfiles = [arg for arg in sys.argv[1:] if arg and arg[0] != '-']
-    if not testfiles:
-        name = os.path.basename(sys.argv[0])
-        if '__loader__' in globals():          # python -m
-            name, _ = os.path.splitext(name)
-        print("usage: {0} [-v] file ...".format(name))
-        return 2
+    parser = argparse.ArgumentParser(description="doctest runner")
+    parser.add_argument('-v', '--verbose', action='store_true', default=False,
+                        help='print very verbose output for all tests')
+    parser.add_argument('-o', '--option', action='append',
+                        choices=OPTIONFLAGS_BY_NAME.keys(), default=[],
+                        help=('specify a doctest option flag to apply'
+                              ' to the test run; may be specified more'
+                              ' than once to apply multiple options'))
+    parser.add_argument('-f', '--fail-fast', action='store_true',
+                        help=('stop running tests after first failure (this'
+                              ' is a shorthand for -o FAIL_FAST, and is'
+                              ' in addition to any other -o options)'))
+    parser.add_argument('file', nargs='+',
+                        help='file containing the tests to run')
+    args = parser.parse_args()
+    testfiles = args.file
+    # Verbose used to be handled by the "inspect argv" magic in DocTestRunner,
+    # but since we are using argparse we are passing it manually now.
+    verbose = args.verbose
+    options = 0
+    for option in args.option:
+        options |= OPTIONFLAGS_BY_NAME[option]
+    if args.fail_fast:
+        options |= FAIL_FAST
     for filename in testfiles:
         if filename.endswith(".py"):
             # It is a module -- insert its dir into sys.path and try to
@@ -2724,9 +2742,10 @@
             sys.path.insert(0, dirname)
             m = __import__(filename[:-3])
             del sys.path[0]
-            failures, _ = testmod(m)
+            failures, _ = testmod(m, verbose=verbose, optionflags=options)
         else:
-            failures, _ = testfile(filename, module_relative=False)
+            failures, _ = testfile(filename, module_relative=False,
+                                     verbose=verbose, optionflags=options)
         if failures:
             return 1
     return 0
diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -2596,6 +2596,232 @@
     TestResults(failed=1, attempted=1)
     """
 
+def test_CLI(): r"""
+The doctest module can be used to run doctests against an arbitrary file.
+These tests test this CLI functionality.
+
+We'll use the support module's script_helpers for this, and write a test files
+to a temp dir to run the command against.
+
+First, a file with two simple tests and no errors.  We'll run both the
+unadorned doctest command, and the verbose version, and then check the output:
+
+    >>> from test import script_helper
+    >>> with script_helper.temp_dir() as tmpdir:
+    ...     fn = os.path.join(tmpdir, 'myfile.doc')
+    ...     with open(fn, 'w') as f:
+    ...         _ = f.write('This is a very simple test file.\n')
+    ...         _ = f.write('   >>> 1 + 1\n')
+    ...         _ = f.write('   2\n')
+    ...         _ = f.write('   >>> "a"\n')
+    ...         _ = f.write("   'a'\n")
+    ...         _ = f.write('\n')
+    ...         _ = f.write('And that is it.\n')
+    ...     rc1, out1, err1 = script_helper.assert_python_ok(
+    ...             '-m', 'doctest', fn)
+    ...     rc2, out2, err2 = script_helper.assert_python_ok(
+    ...             '-m', 'doctest', '-v', fn)
+
+With no arguments and passing tests, we should get no output:
+
+    >>> rc1, out1, err1
+    (0, b'', b'')
+
+With the verbose flag, we should see the test output, but no error output:
+
+    >>> rc2, err2
+    (0, b'')
+    >>> print(out2.decode())
+    Trying:
+        1 + 1
+    Expecting:
+        2
+    ok
+    Trying:
+        "a"
+    Expecting:
+        'a'
+    ok
+    1 items passed all tests:
+       2 tests in myfile.doc
+    2 tests in 1 items.
+    2 passed and 0 failed.
+    Test passed.
+    <BLANKLINE>
+
+Now we'll write a couple files, one with three tests, the other a python module
+with two tests, both of the files having "errors" in the tests that can be made
+non-errors by applying the appropriate doctest options to the run (ELLIPSIS in
+the first file, NORMALIZE_WHITESPACE in the second).  This combination will
+allow to thoroughly test the -f and -o flags, as well as the doctest command's
+ability to process more than one file on the command line and, since the second
+file ends in '.py', its handling of python module files (as opposed to straight
+text files).
+
+    >>> from test import script_helper
+    >>> with script_helper.temp_dir() as tmpdir:
+    ...     fn = os.path.join(tmpdir, 'myfile.doc')
+    ...     with open(fn, 'w') as f:
+    ...         _ = f.write('This is another simple test file.\n')
+    ...         _ = f.write('   >>> 1 + 1\n')
+    ...         _ = f.write('   2\n')
+    ...         _ = f.write('   >>> "abcdef"\n')
+    ...         _ = f.write("   'a...f'\n")
+    ...         _ = f.write('   >>> "ajkml"\n')
+    ...         _ = f.write("   'a...l'\n")
+    ...         _ = f.write('\n')
+    ...         _ = f.write('And that is it.\n')
+    ...     fn2 = os.path.join(tmpdir, 'myfile2.py')
+    ...     with open(fn2, 'w') as f:
+    ...         _ = f.write('def test_func():\n')
+    ...         _ = f.write('   \"\"\"\n')
+    ...         _ = f.write('   This is simple python test function.\n')
+    ...         _ = f.write('       >>> 1 + 1\n')
+    ...         _ = f.write('       2\n')
+    ...         _ = f.write('       >>> "abc   def"\n')
+    ...         _ = f.write("       'abc def'\n")
+    ...         _ = f.write("\n")
+    ...         _ = f.write('   \"\"\"\n')
+    ...     import shutil
+    ...     rc1, out1, err1 = script_helper.assert_python_failure(
+    ...             '-m', 'doctest', fn, fn2)
+    ...     rc2, out2, err2 = script_helper.assert_python_ok(
+    ...             '-m', 'doctest', '-o', 'ELLIPSIS', fn)
+    ...     rc3, out3, err3 = script_helper.assert_python_ok(
+    ...             '-m', 'doctest', '-o', 'ELLIPSIS',
+    ...             '-o', 'NORMALIZE_WHITESPACE', fn, fn2)
+    ...     rc4, out4, err4 = script_helper.assert_python_failure(
+    ...             '-m', 'doctest', '-f', fn, fn2)
+    ...     rc5, out5, err5 = script_helper.assert_python_ok(
+    ...             '-m', 'doctest', '-v', '-o', 'ELLIPSIS',
+    ...             '-o', 'NORMALIZE_WHITESPACE', fn, fn2)
+
+Our first test run will show the errors from the first file (doctest stops if a
+file has errors).  Note that doctest test-run error output appears on stdout,
+not stderr:
+
+    >>> rc1, err1
+    (1, b'')
+    >>> print(out1.decode())                # doctest: +ELLIPSIS
+    **********************************************************************
+    File "...myfile.doc", line 4, in myfile.doc
+    Failed example:
+        "abcdef"
+    Expected:
+        'a...f'
+    Got:
+        'abcdef'
+    **********************************************************************
+    File "...myfile.doc", line 6, in myfile.doc
+    Failed example:
+        "ajkml"
+    Expected:
+        'a...l'
+    Got:
+        'ajkml'
+    **********************************************************************
+    1 items had failures:
+       2 of   3 in myfile.doc
+    ***Test Failed*** 2 failures.
+    <BLANKLINE>
+
+With -o ELLIPSIS specified, the second run, against just the first file, should
+produce no errors, and with -o NORMALIZE_WHITESPACE also specified, neither
+should the third, which ran against both files:
+
+    >>> rc2, out2, err2
+    (0, b'', b'')
+    >>> rc3, out3, err3
+    (0, b'', b'')
+
+The fourth run uses FAIL_FAST, so we should see only one error:
+
+    >>> rc4, err4
+    (1, b'')
+    >>> print(out4.decode())                # doctest: +ELLIPSIS
+    **********************************************************************
+    File "...myfile.doc", line 4, in myfile.doc
+    Failed example:
+        "abcdef"
+    Expected:
+        'a...f'
+    Got:
+        'abcdef'
+    **********************************************************************
+    1 items had failures:
+       1 of   2 in myfile.doc
+    ***Test Failed*** 1 failures.
+    <BLANKLINE>
+
+The fifth test uses verbose with the two options, so we should get verbose
+success output for the tests in both files:
+
+    >>> rc5, err5
+    (0, b'')
+    >>> print(out5.decode())
+    Trying:
+        1 + 1
+    Expecting:
+        2
+    ok
+    Trying:
+        "abcdef"
+    Expecting:
+        'a...f'
+    ok
+    Trying:
+        "ajkml"
+    Expecting:
+        'a...l'
+    ok
+    1 items passed all tests:
+       3 tests in myfile.doc
+    3 tests in 1 items.
+    3 passed and 0 failed.
+    Test passed.
+    Trying:
+        1 + 1
+    Expecting:
+        2
+    ok
+    Trying:
+        "abc   def"
+    Expecting:
+        'abc def'
+    ok
+    1 items had no tests:
+        myfile2
+    1 items passed all tests:
+       2 tests in myfile2.test_func
+    2 tests in 2 items.
+    2 passed and 0 failed.
+    Test passed.
+    <BLANKLINE>
+
+We should also check some typical error cases.
+
+Invalid file name:
+
+    >>> rc, out, err = script_helper.assert_python_failure(
+    ...         '-m', 'doctest', 'nosuchfile')
+    >>> rc, out
+    (1, b'')
+    >>> print(err.decode())                    # doctest: +ELLIPSIS
+    Traceback (most recent call last):
+      ...
+    FileNotFoundError: [Errno ...] No such file or directory: 'nosuchfile'
+
+Invalid doctest option:
+
+    >>> rc, out, err = script_helper.assert_python_failure(
+    ...         '-m', 'doctest', '-o', 'nosuchoption')
+    >>> rc, out
+    (2, b'')
+    >>> print(err.decode())                    # doctest: +ELLIPSIS
+    usage...invalid...nosuchoption...
+
+"""
+
 ######################################################################
 ## Main
 ######################################################################
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -129,6 +129,9 @@
 Library
 -------
 
+- Issue #11390: Add -o and -f command line options to the doctest CLI to
+  specify doctest options (and convert it to using argparse).
+
 - Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
   and in ssl.SSLContext.load_cert_chain() for strings and passwords longer than
   2 gigabytes.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 20:38:18 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Sun, 23 Jun 2013 20:38:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_also_backout_f?=
 =?utf-8?q?1dc30a1be72_for_not_being_a_bugfix?=
Message-ID: <3bdj7Z3cgkz7Lk9@mail.python.org>

http://hg.python.org/cpython/rev/86d512e0ec66
changeset:   84288:86d512e0ec66
branch:      2.7
parent:      84283:2f1e8b7fa534
user:        Benjamin Peterson <benjamin at python.org>
date:        Sun Jun 23 11:38:11 2013 -0700
summary:
  also backout f1dc30a1be72 for not being a bugfix

files:
  Modules/_collectionsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -47,8 +47,8 @@
 
 typedef struct BLOCK {
     struct BLOCK *leftlink;
+    struct BLOCK *rightlink;
     PyObject *data[BLOCKLEN];
-    struct BLOCK *rightlink;
 } block;
 
 #define MAXFREEBLOCKS 10

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:50 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTc5OiBkb2N1?=
 =?utf-8?q?ment_the_local=5Fhostname_parameter=2E?=
Message-ID: <3bdlDf4c0dz7LlK@mail.python.org>

http://hg.python.org/cpython/rev/3685d8074203
changeset:   84289:3685d8074203
branch:      3.3
parent:      84284:059432647de0
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 15:47:50 2013 -0400
summary:
  #18179: document the local_hostname parameter.

Original patch by Berker Peksag.

files:
  Doc/library/smtplib.rst |  15 +++++++++++----
  Lib/smtplib.py          |  11 +++++++----
  2 files changed, 18 insertions(+), 8 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -25,7 +25,10 @@
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
    host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If the :meth:`connect` call
+   with those parameters during initialization.  If specified, *local_hostname* is
+   used as the FQDN of the local host in the HELO/EHLO command.  Otherwise, the
+   local hostname is found using :func:`socket.getfqdn`.  If the
+   :meth:`connect` call
    returns anything other than a success code, an :exc:`SMTPConnectError` is
    raised. The optional *timeout* parameter specifies a timeout in seconds for
    blocking operations like the connection attempt (if not specified, the
@@ -57,13 +60,17 @@
    .. versionchanged:: 3.3
       source_address argument was added.
 
-.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None, source_address=None)
+.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, \
+                    certfile=None [, timeout], context=None, \
+                    source_address=None)
 
    A :class:`SMTP_SSL` instance behaves exactly the same as instances of
    :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
-   *port* is zero, the standard SMTP-over-SSL port (465) is used. *keyfile*
+   *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
+   arguments *local_hostname* and *source_address* have the same meaning as
+   they do in the :class:`SMTP` class.  *keyfile*
    and *certfile* are also optional, and can contain a PEM formatted private key
    and certificate chain file for the SSL connection. *context* also optional, can contain
    a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
@@ -90,7 +97,7 @@
    standard SMTP client. It's common to use Unix sockets for LMTP, so our
    :meth:`connect` method must support that as well as a regular host:port
    server. The optional arguments local_hostname and source_address have the
-   same meaning as that of SMTP client. To specify a Unix socket, you must use
+   same meaning as they do in the :class:`SMTP` class. To specify a Unix socket, you must use
    an absolute path for *host*, starting with a '/'.
 
    Authentication is supported, using the regular SMTP mechanism. When using a Unix
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -224,7 +224,8 @@
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
         connect method is called, and if it returns anything other than
         a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host.  By default,
+        `local_hostname` is used as the FQDN of the local host in the
+        HELO/EHLO command.  Otherwise,
         the local hostname is found using socket.getfqdn(). The
         `source_address` parameter takes a 2-tuple (host, port) for the socket
         to bind to as its source address before connecting. If the host is ''
@@ -855,8 +856,8 @@
         """ This is a subclass derived from SMTP that connects over an SSL encrypted
         socket (to use this class you need a socket module that was compiled with SSL
         support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used. The optional
-        source_address takes a two-tuple (host,port) for socket to bind to. keyfile and certfile
+        omitted, the standard SMTP-over-SSL port (465) is used.  local_hostname and
+        source_address have the same meaning as they do in the SMTP class.  keyfile and certfile
         are also optional - they can contain a PEM formatted private key and
         certificate chain file for the SSL connection. context also optional, can contain
         a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
@@ -905,7 +906,9 @@
     The LMTP protocol, which is very similar to ESMTP, is heavily based
     on the standard SMTP client. It's common to use Unix sockets for LMTP,
     so our connect() method must support that as well as a regular
-    host:port server. To specify a Unix socket, you must use an absolute
+    host:port server.  local_hostname and source_address have the same
+    meaning as they do in the SMTP class.
+    To specify a Unix socket, you must use an absolute
     path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:52 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:52 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_=2318179=3A_document_the_local=5Fhostname_paramete?=
 =?utf-8?q?r=2E?=
Message-ID: <3bdlDh0VPFz7Lll@mail.python.org>

http://hg.python.org/cpython/rev/b10fae8c185c
changeset:   84290:b10fae8c185c
parent:      84287:ae802dc4dcd4
parent:      84289:3685d8074203
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 15:52:08 2013 -0400
summary:
  Merge #18179: document the local_hostname parameter.

Original patch by Berker Peksag.

files:
  Doc/library/smtplib.rst |  15 +++++++++++----
  Lib/smtplib.py          |  11 +++++++----
  2 files changed, 18 insertions(+), 8 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -25,7 +25,10 @@
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
    host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If the :meth:`connect` call
+   with those parameters during initialization.  If specified, *local_hostname* is
+   used as the FQDN of the local host in the HELO/EHLO command.  Otherwise, the
+   local hostname is found using :func:`socket.getfqdn`.  If the
+   :meth:`connect` call
    returns anything other than a success code, an :exc:`SMTPConnectError` is
    raised. The optional *timeout* parameter specifies a timeout in seconds for
    blocking operations like the connection attempt (if not specified, the
@@ -57,13 +60,17 @@
    .. versionchanged:: 3.3
       source_address argument was added.
 
-.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None, source_address=None)
+.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, \
+                    certfile=None [, timeout], context=None, \
+                    source_address=None)
 
    A :class:`SMTP_SSL` instance behaves exactly the same as instances of
    :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
-   *port* is zero, the standard SMTP-over-SSL port (465) is used. *keyfile*
+   *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
+   arguments *local_hostname* and *source_address* have the same meaning as
+   they do in the :class:`SMTP` class.  *keyfile*
    and *certfile* are also optional, and can contain a PEM formatted private key
    and certificate chain file for the SSL connection. *context* also optional, can contain
    a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
@@ -90,7 +97,7 @@
    standard SMTP client. It's common to use Unix sockets for LMTP, so our
    :meth:`connect` method must support that as well as a regular host:port
    server. The optional arguments local_hostname and source_address have the
-   same meaning as that of SMTP client. To specify a Unix socket, you must use
+   same meaning as they do in the :class:`SMTP` class. To specify a Unix socket, you must use
    an absolute path for *host*, starting with a '/'.
 
    Authentication is supported, using the regular SMTP mechanism. When using a Unix
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -224,7 +224,8 @@
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
         connect method is called, and if it returns anything other than
         a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host.  By default,
+        `local_hostname` is used as the FQDN of the local host in the
+        HELO/EHLO command.  Otherwise,
         the local hostname is found using socket.getfqdn(). The
         `source_address` parameter takes a 2-tuple (host, port) for the socket
         to bind to as its source address before connecting. If the host is ''
@@ -855,8 +856,8 @@
         """ This is a subclass derived from SMTP that connects over an SSL encrypted
         socket (to use this class you need a socket module that was compiled with SSL
         support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used. The optional
-        source_address takes a two-tuple (host,port) for socket to bind to. keyfile and certfile
+        omitted, the standard SMTP-over-SSL port (465) is used.  local_hostname and
+        source_address have the same meaning as they do in the SMTP class.  keyfile and certfile
         are also optional - they can contain a PEM formatted private key and
         certificate chain file for the SSL connection. context also optional, can contain
         a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
@@ -905,7 +906,9 @@
     The LMTP protocol, which is very similar to ESMTP, is heavily based
     on the standard SMTP client. It's common to use Unix sockets for LMTP,
     so our connect() method must support that as well as a regular
-    host:port server. To specify a Unix socket, you must use an absolute
+    host:port server.  local_hostname and source_address have the same
+    meaning as they do in the SMTP class.
+    To specify a Unix socket, you must use an absolute
     path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:53 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTc5OiBkb2N1?=
 =?utf-8?q?ment_the_local=5Fhostname_parameter=2E?=
Message-ID: <3bdlDj3WgNz7Lll@mail.python.org>

http://hg.python.org/cpython/rev/c8914dbe6ead
changeset:   84291:c8914dbe6ead
branch:      2.7
parent:      84278:8f0adcb66633
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 16:02:34 2013 -0400
summary:
  #18179: document the local_hostname parameter.

Original patch by Berker Peksag.

files:
  Doc/library/smtplib.rst |  12 +++++++++---
  Lib/smtplib.py          |   9 ++++++---
  2 files changed, 15 insertions(+), 6 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -25,7 +25,10 @@
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
    host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If the :meth:`connect` call
+   with those parameters during initialization.  If specified, *local_hostname* is
+   used as the FQDN of the local host in the HELO/EHLO command.  Otherwise, the
+   local hostname is found using :func:`socket.getfqdn`.  If the
+   :meth:`connect` call
    returns anything other than a success code, an :exc:`SMTPConnectError` is
    raised. The optional *timeout* parameter specifies a timeout in seconds for
    blocking operations like the connection attempt (if not specified, the
@@ -45,7 +48,9 @@
    :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
-   *port* is omitted, the standard SMTP-over-SSL port (465) is used. *keyfile*
+   *port* is omitted, the standard SMTP-over-SSL port (465) is used.
+   *local_hostname* has the same meaning as it does for the :class:`SMTP` class.
+   *keyfile*
    and *certfile* are also optional, and can contain a PEM formatted private key
    and certificate chain file for the SSL connection. The optional *timeout*
    parameter specifies a timeout in seconds for blocking operations like the
@@ -59,7 +64,8 @@
 
    The LMTP protocol, which is very similar to ESMTP, is heavily based on the
    standard SMTP client. It's common to use Unix sockets for LMTP, so our :meth:`connect`
-   method must support that as well as a regular host:port server. To specify a
+   method must support that as well as a regular host:port server.  *local_hostname*
+   has the same meaning as it does for the :class:`SMTP` class.  To specify a
    Unix socket, you must use an absolute path for *host*, starting with a '/'.
 
    Authentication is supported, using the regular SMTP mechanism. When using a Unix
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -240,7 +240,8 @@
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
         connect method is called, and if it returns anything other than
         a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host.  By default,
+        `local_hostname` is used as the FQDN of the local host for the
+        HELO/EHLO command.  Otherwise,
         the local hostname is found using socket.getfqdn().
 
         """
@@ -762,7 +763,8 @@
         """ This is a subclass derived from SMTP that connects over an SSL encrypted
         socket (to use this class you need a socket module that was compiled with SSL
         support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used. keyfile and certfile
+        omitted, the standard SMTP-over-SSL port (465) is used.  local_hostname
+        has the same meaning as it does in the SMTP class.  keyfile and certfile
         are also optional - they can contain a PEM formatted private key and
         certificate chain file for the SSL connection.
         """
@@ -797,7 +799,8 @@
     The LMTP protocol, which is very similar to ESMTP, is heavily based
     on the standard SMTP client. It's common to use Unix sockets for LMTP,
     so our connect() method must support that as well as a regular
-    host:port server. To specify a Unix socket, you must use an absolute
+    host:port server.  local_hostname has the same meaning as it does in the
+    SMTP class.  To specify a Unix socket, you must use an absolute
     path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:54 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:54 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTc5OiByZWZs?=
	=?utf-8?q?ow_paragraphs=2E?=
Message-ID: <3bdlDk6VLLz7Lld@mail.python.org>

http://hg.python.org/cpython/rev/ffcf46316e1f
changeset:   84292:ffcf46316e1f
branch:      3.3
parent:      84289:3685d8074203
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 16:05:44 2013 -0400
summary:
  #18179: reflow paragraphs.

files:
  Doc/library/smtplib.rst |  48 ++++++++++++++--------------
  Lib/smtplib.py          |  45 +++++++++++++-------------
  2 files changed, 47 insertions(+), 46 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -24,20 +24,20 @@
 
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
-   host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If specified, *local_hostname* is
-   used as the FQDN of the local host in the HELO/EHLO command.  Otherwise, the
-   local hostname is found using :func:`socket.getfqdn`.  If the
-   :meth:`connect` call
-   returns anything other than a success code, an :exc:`SMTPConnectError` is
-   raised. The optional *timeout* parameter specifies a timeout in seconds for
-   blocking operations like the connection attempt (if not specified, the
-   global default timeout setting will be used). The optional source_address
-   parameter allows to bind to some specific source address in a machine with
-   multiple network interfaces, and/or to some specific source TCP port. It
-   takes a 2-tuple (host, port), for the socket to bind to as its source
-   address before connecting. If omitted (or if host or port are ``''`` and/or
-   0 respectively) the OS default behavior will be used.
+   host and port parameters are given, the SMTP :meth:`connect` method is
+   called with those parameters during initialization.  If specified,
+   *local_hostname* is used as the FQDN of the local host in the HELO/EHLO
+   command.  Otherwise, the local hostname is found using
+   :func:`socket.getfqdn`.  If the :meth:`connect` call returns anything other
+   than a success code, an :exc:`SMTPConnectError` is raised. The optional
+   *timeout* parameter specifies a timeout in seconds for blocking operations
+   like the connection attempt (if not specified, the global default timeout
+   setting will be used). The optional source_address parameter allows to bind
+   to some specific source address in a machine with multiple network
+   interfaces, and/or to some specific source TCP port. It takes a 2-tuple
+   (host, port), for the socket to bind to as its source address before
+   connecting. If omitted (or if host or port are ``''`` and/or 0 respectively)
+   the OS default behavior will be used.
 
    For normal use, you should only require the initialization/connect,
    :meth:`sendmail`, and :meth:`~smtplib.quit` methods.
@@ -70,11 +70,11 @@
    not appropriate. If *host* is not specified, the local host is used. If
    *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
    arguments *local_hostname* and *source_address* have the same meaning as
-   they do in the :class:`SMTP` class.  *keyfile*
-   and *certfile* are also optional, and can contain a PEM formatted private key
-   and certificate chain file for the SSL connection. *context* also optional, can contain
-   a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
-   keyfile and certfile must be None.  The optional *timeout*
+   they do in the :class:`SMTP` class.  *keyfile* and *certfile* are also
+   optional, and can contain a PEM formatted private key and certificate chain
+   file for the SSL connection. *context* also optional, can contain a
+   SSLContext, and is an alternative to keyfile and certfile; If it is
+   specified both keyfile and certfile must be None.  The optional *timeout*
    parameter specifies a timeout in seconds for blocking operations like the
    connection attempt (if not specified, the global default timeout setting
    will be used). The optional source_address parameter allows to bind to some
@@ -97,12 +97,12 @@
    standard SMTP client. It's common to use Unix sockets for LMTP, so our
    :meth:`connect` method must support that as well as a regular host:port
    server. The optional arguments local_hostname and source_address have the
-   same meaning as they do in the :class:`SMTP` class. To specify a Unix socket, you must use
-   an absolute path for *host*, starting with a '/'.
+   same meaning as they do in the :class:`SMTP` class. To specify a Unix
+   socket, you must use an absolute path for *host*, starting with a '/'.
 
-   Authentication is supported, using the regular SMTP mechanism. When using a Unix
-   socket, LMTP generally don't support or require any authentication, but your
-   mileage might vary.
+   Authentication is supported, using the regular SMTP mechanism. When using a
+   Unix socket, LMTP generally don't support or require any authentication, but
+   your mileage might vary.
 
 
 A nice selection of exceptions is defined as well:
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -222,14 +222,14 @@
         If specified, `host' is the name of the remote host to which to
         connect.  If specified, `port' specifies the port to which to connect.
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
-        connect method is called, and if it returns anything other than
-        a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host in the
-        HELO/EHLO command.  Otherwise,
-        the local hostname is found using socket.getfqdn(). The
-        `source_address` parameter takes a 2-tuple (host, port) for the socket
-        to bind to as its source address before connecting. If the host is ''
-        and port is 0, the OS default behavior will be used.
+        connect method is called, and if it returns anything other than a
+        success code an SMTPConnectError is raised.  If specified,
+        `local_hostname` is used as the FQDN of the local host in the HELO/EHLO
+        command.  Otherwise, the local hostname is found using
+        socket.getfqdn(). The `source_address` parameter takes a 2-tuple (host,
+        port) for the socket to bind to as its source address before
+        connecting. If the host is '' and port is 0, the OS default behavior
+        will be used.
 
         """
         self.timeout = timeout
@@ -853,15 +853,17 @@
 if _have_ssl:
 
     class SMTP_SSL(SMTP):
-        """ This is a subclass derived from SMTP that connects over an SSL encrypted
-        socket (to use this class you need a socket module that was compiled with SSL
-        support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used.  local_hostname and
-        source_address have the same meaning as they do in the SMTP class.  keyfile and certfile
-        are also optional - they can contain a PEM formatted private key and
-        certificate chain file for the SSL connection. context also optional, can contain
-        a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
-        keyfile and certfile must be None.
+        """ This is a subclass derived from SMTP that connects over an SSL
+        encrypted socket (to use this class you need a socket module that was
+        compiled with SSL support). If host is not specified, '' (the local
+        host) is used. If port is omitted, the standard SMTP-over-SSL port
+        (465) is used.  local_hostname and source_address have the same meaning
+        as they do in the SMTP class.  keyfile and certfile are also optional -
+        they can contain a PEM formatted private key and certificate chain file
+        for the SSL connection. context also optional, can contain a
+        SSLContext, and is an alternative to keyfile and certfile; If it is
+        specified both keyfile and certfile must be None.
+
         """
 
         default_port = SMTP_SSL_PORT
@@ -904,12 +906,11 @@
     """LMTP - Local Mail Transfer Protocol
 
     The LMTP protocol, which is very similar to ESMTP, is heavily based
-    on the standard SMTP client. It's common to use Unix sockets for LMTP,
-    so our connect() method must support that as well as a regular
+    on the standard SMTP client. It's common to use Unix sockets for
+    LMTP, so our connect() method must support that as well as a regular
     host:port server.  local_hostname and source_address have the same
-    meaning as they do in the SMTP class.
-    To specify a Unix socket, you must use an absolute
-    path as the host, starting with a '/'.
+    meaning as they do in the SMTP class.  To specify a Unix socket,
+    you must use an absolute path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When
     using a Unix socket, LMTP generally don't support or require any

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:56 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_=2318179=3A_reflow_paragraphs=2E?=
Message-ID: <3bdlDm2KHsz7Lls@mail.python.org>

http://hg.python.org/cpython/rev/627e3096340e
changeset:   84293:627e3096340e
parent:      84290:b10fae8c185c
parent:      84292:ffcf46316e1f
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 16:06:13 2013 -0400
summary:
  Merge #18179: reflow paragraphs.

files:
  Doc/library/smtplib.rst |  48 ++++++++++++++--------------
  Lib/smtplib.py          |  45 +++++++++++++-------------
  2 files changed, 47 insertions(+), 46 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -24,20 +24,20 @@
 
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
-   host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If specified, *local_hostname* is
-   used as the FQDN of the local host in the HELO/EHLO command.  Otherwise, the
-   local hostname is found using :func:`socket.getfqdn`.  If the
-   :meth:`connect` call
-   returns anything other than a success code, an :exc:`SMTPConnectError` is
-   raised. The optional *timeout* parameter specifies a timeout in seconds for
-   blocking operations like the connection attempt (if not specified, the
-   global default timeout setting will be used). The optional source_address
-   parameter allows to bind to some specific source address in a machine with
-   multiple network interfaces, and/or to some specific source TCP port. It
-   takes a 2-tuple (host, port), for the socket to bind to as its source
-   address before connecting. If omitted (or if host or port are ``''`` and/or
-   0 respectively) the OS default behavior will be used.
+   host and port parameters are given, the SMTP :meth:`connect` method is
+   called with those parameters during initialization.  If specified,
+   *local_hostname* is used as the FQDN of the local host in the HELO/EHLO
+   command.  Otherwise, the local hostname is found using
+   :func:`socket.getfqdn`.  If the :meth:`connect` call returns anything other
+   than a success code, an :exc:`SMTPConnectError` is raised. The optional
+   *timeout* parameter specifies a timeout in seconds for blocking operations
+   like the connection attempt (if not specified, the global default timeout
+   setting will be used). The optional source_address parameter allows to bind
+   to some specific source address in a machine with multiple network
+   interfaces, and/or to some specific source TCP port. It takes a 2-tuple
+   (host, port), for the socket to bind to as its source address before
+   connecting. If omitted (or if host or port are ``''`` and/or 0 respectively)
+   the OS default behavior will be used.
 
    For normal use, you should only require the initialization/connect,
    :meth:`sendmail`, and :meth:`~smtplib.quit` methods.
@@ -70,11 +70,11 @@
    not appropriate. If *host* is not specified, the local host is used. If
    *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
    arguments *local_hostname* and *source_address* have the same meaning as
-   they do in the :class:`SMTP` class.  *keyfile*
-   and *certfile* are also optional, and can contain a PEM formatted private key
-   and certificate chain file for the SSL connection. *context* also optional, can contain
-   a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
-   keyfile and certfile must be None.  The optional *timeout*
+   they do in the :class:`SMTP` class.  *keyfile* and *certfile* are also
+   optional, and can contain a PEM formatted private key and certificate chain
+   file for the SSL connection. *context* also optional, can contain a
+   SSLContext, and is an alternative to keyfile and certfile; If it is
+   specified both keyfile and certfile must be None.  The optional *timeout*
    parameter specifies a timeout in seconds for blocking operations like the
    connection attempt (if not specified, the global default timeout setting
    will be used). The optional source_address parameter allows to bind to some
@@ -97,12 +97,12 @@
    standard SMTP client. It's common to use Unix sockets for LMTP, so our
    :meth:`connect` method must support that as well as a regular host:port
    server. The optional arguments local_hostname and source_address have the
-   same meaning as they do in the :class:`SMTP` class. To specify a Unix socket, you must use
-   an absolute path for *host*, starting with a '/'.
+   same meaning as they do in the :class:`SMTP` class. To specify a Unix
+   socket, you must use an absolute path for *host*, starting with a '/'.
 
-   Authentication is supported, using the regular SMTP mechanism. When using a Unix
-   socket, LMTP generally don't support or require any authentication, but your
-   mileage might vary.
+   Authentication is supported, using the regular SMTP mechanism. When using a
+   Unix socket, LMTP generally don't support or require any authentication, but
+   your mileage might vary.
 
 
 A nice selection of exceptions is defined as well:
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -222,14 +222,14 @@
         If specified, `host' is the name of the remote host to which to
         connect.  If specified, `port' specifies the port to which to connect.
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
-        connect method is called, and if it returns anything other than
-        a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host in the
-        HELO/EHLO command.  Otherwise,
-        the local hostname is found using socket.getfqdn(). The
-        `source_address` parameter takes a 2-tuple (host, port) for the socket
-        to bind to as its source address before connecting. If the host is ''
-        and port is 0, the OS default behavior will be used.
+        connect method is called, and if it returns anything other than a
+        success code an SMTPConnectError is raised.  If specified,
+        `local_hostname` is used as the FQDN of the local host in the HELO/EHLO
+        command.  Otherwise, the local hostname is found using
+        socket.getfqdn(). The `source_address` parameter takes a 2-tuple (host,
+        port) for the socket to bind to as its source address before
+        connecting. If the host is '' and port is 0, the OS default behavior
+        will be used.
 
         """
         self.timeout = timeout
@@ -853,15 +853,17 @@
 if _have_ssl:
 
     class SMTP_SSL(SMTP):
-        """ This is a subclass derived from SMTP that connects over an SSL encrypted
-        socket (to use this class you need a socket module that was compiled with SSL
-        support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used.  local_hostname and
-        source_address have the same meaning as they do in the SMTP class.  keyfile and certfile
-        are also optional - they can contain a PEM formatted private key and
-        certificate chain file for the SSL connection. context also optional, can contain
-        a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
-        keyfile and certfile must be None.
+        """ This is a subclass derived from SMTP that connects over an SSL
+        encrypted socket (to use this class you need a socket module that was
+        compiled with SSL support). If host is not specified, '' (the local
+        host) is used. If port is omitted, the standard SMTP-over-SSL port
+        (465) is used.  local_hostname and source_address have the same meaning
+        as they do in the SMTP class.  keyfile and certfile are also optional -
+        they can contain a PEM formatted private key and certificate chain file
+        for the SSL connection. context also optional, can contain a
+        SSLContext, and is an alternative to keyfile and certfile; If it is
+        specified both keyfile and certfile must be None.
+
         """
 
         default_port = SMTP_SSL_PORT
@@ -904,12 +906,11 @@
     """LMTP - Local Mail Transfer Protocol
 
     The LMTP protocol, which is very similar to ESMTP, is heavily based
-    on the standard SMTP client. It's common to use Unix sockets for LMTP,
-    so our connect() method must support that as well as a regular
+    on the standard SMTP client. It's common to use Unix sockets for
+    LMTP, so our connect() method must support that as well as a regular
     host:port server.  local_hostname and source_address have the same
-    meaning as they do in the SMTP class.
-    To specify a Unix socket, you must use an absolute
-    path as the host, starting with a '/'.
+    meaning as they do in the SMTP class.  To specify a Unix socket,
+    you must use an absolute path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When
     using a Unix socket, LMTP generally don't support or require any

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:57 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:57 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTc5OiByZWZs?=
	=?utf-8?q?ow_paragraphs=2E?=
Message-ID: <3bdlDn5PC2z7Lm6@mail.python.org>

http://hg.python.org/cpython/rev/9f1f83d23ec4
changeset:   84294:9f1f83d23ec4
branch:      2.7
parent:      84291:c8914dbe6ead
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 16:10:37 2013 -0400
summary:
  #18179: reflow paragraphs.

files:
  Doc/library/smtplib.rst |  46 ++++++++++++++--------------
  Lib/smtplib.py          |  32 ++++++++++---------
  2 files changed, 40 insertions(+), 38 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -24,15 +24,15 @@
 
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
-   host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If specified, *local_hostname* is
-   used as the FQDN of the local host in the HELO/EHLO command.  Otherwise, the
-   local hostname is found using :func:`socket.getfqdn`.  If the
-   :meth:`connect` call
-   returns anything other than a success code, an :exc:`SMTPConnectError` is
-   raised. The optional *timeout* parameter specifies a timeout in seconds for
-   blocking operations like the connection attempt (if not specified, the
-   global default timeout setting will be used).
+   host and port parameters are given, the SMTP :meth:`connect` method is
+   called with those parameters during initialization.  If specified,
+   *local_hostname* is used as the FQDN of the local host in the HELO/EHLO
+   command.  Otherwise, the local hostname is found using
+   :func:`socket.getfqdn`.  If the :meth:`connect` call returns anything other
+   than a success code, an :exc:`SMTPConnectError` is raised. The optional
+   *timeout* parameter specifies a timeout in seconds for blocking operations
+   like the connection attempt (if not specified, the global default timeout
+   setting will be used).
 
    For normal use, you should only require the initialization/connect,
    :meth:`sendmail`, and :meth:`~smtplib.quit` methods.
@@ -49,13 +49,12 @@
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
    *port* is omitted, the standard SMTP-over-SSL port (465) is used.
-   *local_hostname* has the same meaning as it does for the :class:`SMTP` class.
-   *keyfile*
-   and *certfile* are also optional, and can contain a PEM formatted private key
-   and certificate chain file for the SSL connection. The optional *timeout*
-   parameter specifies a timeout in seconds for blocking operations like the
-   connection attempt (if not specified, the global default timeout setting
-   will be used).
+   *local_hostname* has the same meaning as it does for the :class:`SMTP`
+   class.  *keyfile* and *certfile* are also optional, and can contain a PEM
+   formatted private key and certificate chain file for the SSL connection. The
+   optional *timeout* parameter specifies a timeout in seconds for blocking
+   operations like the connection attempt (if not specified, the global default
+   timeout setting will be used).
 
    .. versionadded:: 2.6
 
@@ -63,14 +62,15 @@
 .. class:: LMTP([host[, port[, local_hostname]]])
 
    The LMTP protocol, which is very similar to ESMTP, is heavily based on the
-   standard SMTP client. It's common to use Unix sockets for LMTP, so our :meth:`connect`
-   method must support that as well as a regular host:port server.  *local_hostname*
-   has the same meaning as it does for the :class:`SMTP` class.  To specify a
-   Unix socket, you must use an absolute path for *host*, starting with a '/'.
+   standard SMTP client. It's common to use Unix sockets for LMTP, so our
+   :meth:`connect` method must support that as well as a regular host:port
+   server.  *local_hostname* has the same meaning as it does for the
+   :class:`SMTP` class.  To specify a Unix socket, you must use an absolute
+   path for *host*, starting with a '/'.
 
-   Authentication is supported, using the regular SMTP mechanism. When using a Unix
-   socket, LMTP generally don't support or require any authentication, but your
-   mileage might vary.
+   Authentication is supported, using the regular SMTP mechanism. When using a
+   Unix socket, LMTP generally don't support or require any authentication, but
+   your mileage might vary.
 
    .. versionadded:: 2.6
 
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -238,11 +238,11 @@
         If specified, `host' is the name of the remote host to which to
         connect.  If specified, `port' specifies the port to which to connect.
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
-        connect method is called, and if it returns anything other than
-        a success code an SMTPConnectError is raised.  If specified,
+        connect method is called, and if it returns anything other than a
+        success code an SMTPConnectError is raised.  If specified,
         `local_hostname` is used as the FQDN of the local host for the
-        HELO/EHLO command.  Otherwise,
-        the local hostname is found using socket.getfqdn().
+        HELO/EHLO command.  Otherwise, the local hostname is found using
+        socket.getfqdn().
 
         """
         self.timeout = timeout
@@ -760,13 +760,15 @@
 if _have_ssl:
 
     class SMTP_SSL(SMTP):
-        """ This is a subclass derived from SMTP that connects over an SSL encrypted
-        socket (to use this class you need a socket module that was compiled with SSL
-        support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used.  local_hostname
-        has the same meaning as it does in the SMTP class.  keyfile and certfile
-        are also optional - they can contain a PEM formatted private key and
-        certificate chain file for the SSL connection.
+        """ This is a subclass derived from SMTP that connects over an SSL
+        encrypted socket (to use this class you need a socket module that was
+        compiled with SSL support). If host is not specified, '' (the local
+        host) is used. If port is omitted, the standard SMTP-over-SSL port
+        (465) is used.  local_hostname has the same meaning as it does in the
+        SMTP class.  keyfile and certfile are also optional - they can contain
+        a PEM formatted private key and certificate chain file for the SSL
+        connection.
+
         """
 
         default_port = SMTP_SSL_PORT
@@ -797,10 +799,10 @@
     """LMTP - Local Mail Transfer Protocol
 
     The LMTP protocol, which is very similar to ESMTP, is heavily based
-    on the standard SMTP client. It's common to use Unix sockets for LMTP,
-    so our connect() method must support that as well as a regular
-    host:port server.  local_hostname has the same meaning as it does in the
-    SMTP class.  To specify a Unix socket, you must use an absolute
+    on the standard SMTP client. It's common to use Unix sockets for
+    LMTP, so our connect() method must support that as well as a regular
+    host:port server.  local_hostname has the same meaning as it does in
+    the SMTP class.  To specify a Unix socket, you must use an absolute
     path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:12:59 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 23 Jun 2013 22:12:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMi43IC0+IDIuNyk6?=
	=?utf-8?q?_Merge_heads=2E?=
Message-ID: <3bdlDq1FWvz7LmD@mail.python.org>

http://hg.python.org/cpython/rev/a7db9f505e88
changeset:   84295:a7db9f505e88
branch:      2.7
parent:      84288:86d512e0ec66
parent:      84294:9f1f83d23ec4
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 23 16:12:32 2013 -0400
summary:
  Merge heads.

files:
  Doc/library/smtplib.rst |  42 ++++++++++++++++------------
  Lib/smtplib.py          |  31 ++++++++++++--------
  2 files changed, 42 insertions(+), 31 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -24,12 +24,15 @@
 
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
-   host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If the :meth:`connect` call
-   returns anything other than a success code, an :exc:`SMTPConnectError` is
-   raised. The optional *timeout* parameter specifies a timeout in seconds for
-   blocking operations like the connection attempt (if not specified, the
-   global default timeout setting will be used).
+   host and port parameters are given, the SMTP :meth:`connect` method is
+   called with those parameters during initialization.  If specified,
+   *local_hostname* is used as the FQDN of the local host in the HELO/EHLO
+   command.  Otherwise, the local hostname is found using
+   :func:`socket.getfqdn`.  If the :meth:`connect` call returns anything other
+   than a success code, an :exc:`SMTPConnectError` is raised. The optional
+   *timeout* parameter specifies a timeout in seconds for blocking operations
+   like the connection attempt (if not specified, the global default timeout
+   setting will be used).
 
    For normal use, you should only require the initialization/connect,
    :meth:`sendmail`, and :meth:`~smtplib.quit` methods.
@@ -45,12 +48,13 @@
    :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
-   *port* is omitted, the standard SMTP-over-SSL port (465) is used. *keyfile*
-   and *certfile* are also optional, and can contain a PEM formatted private key
-   and certificate chain file for the SSL connection. The optional *timeout*
-   parameter specifies a timeout in seconds for blocking operations like the
-   connection attempt (if not specified, the global default timeout setting
-   will be used).
+   *port* is omitted, the standard SMTP-over-SSL port (465) is used.
+   *local_hostname* has the same meaning as it does for the :class:`SMTP`
+   class.  *keyfile* and *certfile* are also optional, and can contain a PEM
+   formatted private key and certificate chain file for the SSL connection. The
+   optional *timeout* parameter specifies a timeout in seconds for blocking
+   operations like the connection attempt (if not specified, the global default
+   timeout setting will be used).
 
    .. versionadded:: 2.6
 
@@ -58,13 +62,15 @@
 .. class:: LMTP([host[, port[, local_hostname]]])
 
    The LMTP protocol, which is very similar to ESMTP, is heavily based on the
-   standard SMTP client. It's common to use Unix sockets for LMTP, so our :meth:`connect`
-   method must support that as well as a regular host:port server. To specify a
-   Unix socket, you must use an absolute path for *host*, starting with a '/'.
+   standard SMTP client. It's common to use Unix sockets for LMTP, so our
+   :meth:`connect` method must support that as well as a regular host:port
+   server.  *local_hostname* has the same meaning as it does for the
+   :class:`SMTP` class.  To specify a Unix socket, you must use an absolute
+   path for *host*, starting with a '/'.
 
-   Authentication is supported, using the regular SMTP mechanism. When using a Unix
-   socket, LMTP generally don't support or require any authentication, but your
-   mileage might vary.
+   Authentication is supported, using the regular SMTP mechanism. When using a
+   Unix socket, LMTP generally don't support or require any authentication, but
+   your mileage might vary.
 
    .. versionadded:: 2.6
 
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -238,10 +238,11 @@
         If specified, `host' is the name of the remote host to which to
         connect.  If specified, `port' specifies the port to which to connect.
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
-        connect method is called, and if it returns anything other than
-        a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host.  By default,
-        the local hostname is found using socket.getfqdn().
+        connect method is called, and if it returns anything other than a
+        success code an SMTPConnectError is raised.  If specified,
+        `local_hostname` is used as the FQDN of the local host for the
+        HELO/EHLO command.  Otherwise, the local hostname is found using
+        socket.getfqdn().
 
         """
         self.timeout = timeout
@@ -759,12 +760,15 @@
 if _have_ssl:
 
     class SMTP_SSL(SMTP):
-        """ This is a subclass derived from SMTP that connects over an SSL encrypted
-        socket (to use this class you need a socket module that was compiled with SSL
-        support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used. keyfile and certfile
-        are also optional - they can contain a PEM formatted private key and
-        certificate chain file for the SSL connection.
+        """ This is a subclass derived from SMTP that connects over an SSL
+        encrypted socket (to use this class you need a socket module that was
+        compiled with SSL support). If host is not specified, '' (the local
+        host) is used. If port is omitted, the standard SMTP-over-SSL port
+        (465) is used.  local_hostname has the same meaning as it does in the
+        SMTP class.  keyfile and certfile are also optional - they can contain
+        a PEM formatted private key and certificate chain file for the SSL
+        connection.
+
         """
 
         default_port = SMTP_SSL_PORT
@@ -795,9 +799,10 @@
     """LMTP - Local Mail Transfer Protocol
 
     The LMTP protocol, which is very similar to ESMTP, is heavily based
-    on the standard SMTP client. It's common to use Unix sockets for LMTP,
-    so our connect() method must support that as well as a regular
-    host:port server. To specify a Unix socket, you must use an absolute
+    on the standard SMTP client. It's common to use Unix sockets for
+    LMTP, so our connect() method must support that as well as a regular
+    host:port server.  local_hostname has the same meaning as it does in
+    the SMTP class.  To specify a Unix socket, you must use an absolute
     path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:57:42 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 22:57:42 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_a_typo_in_S=5FISDIR=2C?=
 =?utf-8?q?_S=5FISCHR=2C_S=5FISBLK_and_S=5FISREG=2E?=
Message-ID: <3bdmDQ3Ty2z7LlW@mail.python.org>

http://hg.python.org/cpython/rev/bc52faaa50e5
changeset:   84296:bc52faaa50e5
parent:      84287:ae802dc4dcd4
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 22:57:02 2013 +0200
summary:
  Fix a typo in S_ISDIR, S_ISCHR, S_ISBLK and S_ISREG.
Add extra braces to S_IS*() macros

files:
  Modules/_stat.c |  14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/Modules/_stat.c b/Modules/_stat.c
--- a/Modules/_stat.c
+++ b/Modules/_stat.c
@@ -87,31 +87,31 @@
 
 /* S_ISXXX() */
 #ifndef S_ISDIR
-#  define S_ISDIR(mode) ((mode) & S_IFMT) == S_IDIR
+#  define S_ISDIR(mode) (((mode) & S_IFMT) == S_IFDIR)
 #endif
 
 #ifndef S_ISCHR
-#  define S_ISCHR(mode) ((mode) & S_IFMT) == S_ICHR
+#  define S_ISCHR(mode) (((mode) & S_IFMT) == S_IFCHR)
 #endif
 
 #ifndef S_ISBLK
-#  define S_ISBLK(mode) ((mode) & S_IFMT) == S_IBLK
+#  define S_ISBLK(mode) (((mode) & S_IFMT) == S_IFBLK)
 #endif
 
 #ifndef S_ISREG
-#  define S_ISREG(mode) ((mode) & S_IFMT) == S_IREG
+#  define S_ISREG(mode) (((mode) & S_IFMT) == S_IFREG)
 #endif
 
 #ifndef S_ISFIFO
-#  define S_ISFIFO(mode) ((mode) & S_IFMT) == S_IFIFO
+#  define S_ISFIFO(mode) (((mode) & S_IFMT) == S_IFIFO)
 #endif
 
 #ifndef S_ISLNK
-#  define S_ISLNK(mode) ((mode) & S_IFMT) == S_IFLNK
+#  define S_ISLNK(mode) (((mode) & S_IFMT) == S_IFLNK)
 #endif
 
 #ifndef S_ISSOCK
-#  define S_ISSOCK(mode) ((mode) & S_IFMT) == S_IFSOCK
+#  define S_ISSOCK(mode) (((mode) & S_IFMT) == S_IFSOCK)
 #endif
 
 #ifndef S_ISDOOR

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:57:43 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 22:57:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bdmDR6VNGz7Lld@mail.python.org>

http://hg.python.org/cpython/rev/153f7c0df033
changeset:   84297:153f7c0df033
parent:      84296:bc52faaa50e5
parent:      84293:627e3096340e
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 22:57:22 2013 +0200
summary:
  merge

files:
  Doc/library/smtplib.rst |  51 ++++++++++++++++------------
  Lib/smtplib.py          |  44 +++++++++++++-----------
  2 files changed, 53 insertions(+), 42 deletions(-)


diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -24,17 +24,20 @@
 
    A :class:`SMTP` instance encapsulates an SMTP connection.  It has methods
    that support a full repertoire of SMTP and ESMTP operations. If the optional
-   host and port parameters are given, the SMTP :meth:`connect` method is called
-   with those parameters during initialization.  If the :meth:`connect` call
-   returns anything other than a success code, an :exc:`SMTPConnectError` is
-   raised. The optional *timeout* parameter specifies a timeout in seconds for
-   blocking operations like the connection attempt (if not specified, the
-   global default timeout setting will be used). The optional source_address
-   parameter allows to bind to some specific source address in a machine with
-   multiple network interfaces, and/or to some specific source TCP port. It
-   takes a 2-tuple (host, port), for the socket to bind to as its source
-   address before connecting. If omitted (or if host or port are ``''`` and/or
-   0 respectively) the OS default behavior will be used.
+   host and port parameters are given, the SMTP :meth:`connect` method is
+   called with those parameters during initialization.  If specified,
+   *local_hostname* is used as the FQDN of the local host in the HELO/EHLO
+   command.  Otherwise, the local hostname is found using
+   :func:`socket.getfqdn`.  If the :meth:`connect` call returns anything other
+   than a success code, an :exc:`SMTPConnectError` is raised. The optional
+   *timeout* parameter specifies a timeout in seconds for blocking operations
+   like the connection attempt (if not specified, the global default timeout
+   setting will be used). The optional source_address parameter allows to bind
+   to some specific source address in a machine with multiple network
+   interfaces, and/or to some specific source TCP port. It takes a 2-tuple
+   (host, port), for the socket to bind to as its source address before
+   connecting. If omitted (or if host or port are ``''`` and/or 0 respectively)
+   the OS default behavior will be used.
 
    For normal use, you should only require the initialization/connect,
    :meth:`sendmail`, and :meth:`~smtplib.quit` methods.
@@ -57,17 +60,21 @@
    .. versionchanged:: 3.3
       source_address argument was added.
 
-.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout], context=None, source_address=None)
+.. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, \
+                    certfile=None [, timeout], context=None, \
+                    source_address=None)
 
    A :class:`SMTP_SSL` instance behaves exactly the same as instances of
    :class:`SMTP`. :class:`SMTP_SSL` should be used for situations where SSL is
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
-   *port* is zero, the standard SMTP-over-SSL port (465) is used. *keyfile*
-   and *certfile* are also optional, and can contain a PEM formatted private key
-   and certificate chain file for the SSL connection. *context* also optional, can contain
-   a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
-   keyfile and certfile must be None.  The optional *timeout*
+   *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
+   arguments *local_hostname* and *source_address* have the same meaning as
+   they do in the :class:`SMTP` class.  *keyfile* and *certfile* are also
+   optional, and can contain a PEM formatted private key and certificate chain
+   file for the SSL connection. *context* also optional, can contain a
+   SSLContext, and is an alternative to keyfile and certfile; If it is
+   specified both keyfile and certfile must be None.  The optional *timeout*
    parameter specifies a timeout in seconds for blocking operations like the
    connection attempt (if not specified, the global default timeout setting
    will be used). The optional source_address parameter allows to bind to some
@@ -90,12 +97,12 @@
    standard SMTP client. It's common to use Unix sockets for LMTP, so our
    :meth:`connect` method must support that as well as a regular host:port
    server. The optional arguments local_hostname and source_address have the
-   same meaning as that of SMTP client. To specify a Unix socket, you must use
-   an absolute path for *host*, starting with a '/'.
+   same meaning as they do in the :class:`SMTP` class. To specify a Unix
+   socket, you must use an absolute path for *host*, starting with a '/'.
 
-   Authentication is supported, using the regular SMTP mechanism. When using a Unix
-   socket, LMTP generally don't support or require any authentication, but your
-   mileage might vary.
+   Authentication is supported, using the regular SMTP mechanism. When using a
+   Unix socket, LMTP generally don't support or require any authentication, but
+   your mileage might vary.
 
 
 A nice selection of exceptions is defined as well:
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -222,13 +222,14 @@
         If specified, `host' is the name of the remote host to which to
         connect.  If specified, `port' specifies the port to which to connect.
         By default, smtplib.SMTP_PORT is used.  If a host is specified the
-        connect method is called, and if it returns anything other than
-        a success code an SMTPConnectError is raised.  If specified,
-        `local_hostname` is used as the FQDN of the local host.  By default,
-        the local hostname is found using socket.getfqdn(). The
-        `source_address` parameter takes a 2-tuple (host, port) for the socket
-        to bind to as its source address before connecting. If the host is ''
-        and port is 0, the OS default behavior will be used.
+        connect method is called, and if it returns anything other than a
+        success code an SMTPConnectError is raised.  If specified,
+        `local_hostname` is used as the FQDN of the local host in the HELO/EHLO
+        command.  Otherwise, the local hostname is found using
+        socket.getfqdn(). The `source_address` parameter takes a 2-tuple (host,
+        port) for the socket to bind to as its source address before
+        connecting. If the host is '' and port is 0, the OS default behavior
+        will be used.
 
         """
         self.timeout = timeout
@@ -852,15 +853,17 @@
 if _have_ssl:
 
     class SMTP_SSL(SMTP):
-        """ This is a subclass derived from SMTP that connects over an SSL encrypted
-        socket (to use this class you need a socket module that was compiled with SSL
-        support). If host is not specified, '' (the local host) is used. If port is
-        omitted, the standard SMTP-over-SSL port (465) is used. The optional
-        source_address takes a two-tuple (host,port) for socket to bind to. keyfile and certfile
-        are also optional - they can contain a PEM formatted private key and
-        certificate chain file for the SSL connection. context also optional, can contain
-        a SSLContext, and is an alternative to keyfile and certfile; If it is specified both
-        keyfile and certfile must be None.
+        """ This is a subclass derived from SMTP that connects over an SSL
+        encrypted socket (to use this class you need a socket module that was
+        compiled with SSL support). If host is not specified, '' (the local
+        host) is used. If port is omitted, the standard SMTP-over-SSL port
+        (465) is used.  local_hostname and source_address have the same meaning
+        as they do in the SMTP class.  keyfile and certfile are also optional -
+        they can contain a PEM formatted private key and certificate chain file
+        for the SSL connection. context also optional, can contain a
+        SSLContext, and is an alternative to keyfile and certfile; If it is
+        specified both keyfile and certfile must be None.
+
         """
 
         default_port = SMTP_SSL_PORT
@@ -903,10 +906,11 @@
     """LMTP - Local Mail Transfer Protocol
 
     The LMTP protocol, which is very similar to ESMTP, is heavily based
-    on the standard SMTP client. It's common to use Unix sockets for LMTP,
-    so our connect() method must support that as well as a regular
-    host:port server. To specify a Unix socket, you must use an absolute
-    path as the host, starting with a '/'.
+    on the standard SMTP client. It's common to use Unix sockets for
+    LMTP, so our connect() method must support that as well as a regular
+    host:port server.  local_hostname and source_address have the same
+    meaning as they do in the SMTP class.  To specify a Unix socket,
+    you must use an absolute path as the host, starting with a '/'.
 
     Authentication is supported, using the regular SMTP mechanism. When
     using a Unix socket, LMTP generally don't support or require any

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 22:58:18 2013
From: python-checkins at python.org (victor.stinner)
Date: Sun, 23 Jun 2013 22:58:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2311016=3A_Detect_i?=
 =?utf-8?q?nteger_conversion_on_conversion_from_Python_int_to_C?=
Message-ID: <3bdmF64b3fz7LlW@mail.python.org>

http://hg.python.org/cpython/rev/44c8a9d80595
changeset:   84298:44c8a9d80595
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 23 22:57:43 2013 +0200
summary:
  Issue #11016: Detect integer conversion on conversion from Python int to C mode_t

files:
  Modules/_stat.c |  40 ++++++++++++++++++++++++------------
  1 files changed, 27 insertions(+), 13 deletions(-)


diff --git a/Modules/_stat.c b/Modules/_stat.c
--- a/Modules/_stat.c
+++ b/Modules/_stat.c
@@ -258,15 +258,32 @@
 #  define SF_SNAPSHOT 0x00200000
 #endif
 
+static mode_t
+_PyLong_AsMode_t(PyObject *op)
+{
+    unsigned long value;
+    mode_t mode;
+
+    value = PyLong_AsUnsignedLong(op);
+    if ((value == (unsigned long)-1) && PyErr_Occurred())
+        return (mode_t)-1;
+
+    mode = (mode_t)value;
+    if ((unsigned long)mode != value) {
+        PyErr_SetString(PyExc_OverflowError, "mode out of range");
+        return (mode_t)-1;
+    }
+    return mode;
+}
+
 
 #define stat_S_ISFUNC(isfunc, doc)                             \
     static PyObject *                                          \
     stat_ ##isfunc (PyObject *self, PyObject *omode)           \
     {                                                          \
-       unsigned long mode = PyLong_AsUnsignedLong(omode);      \
-       if ((mode == (unsigned long)-1) && PyErr_Occurred()) {  \
+       mode_t mode = _PyLong_AsMode_t(omode);                   \
+       if ((mode == (mode_t)-1) && PyErr_Occurred())           \
            return NULL;                                        \
-       }                                                       \
        return PyBool_FromLong(isfunc(mode));                   \
     }                                                          \
     PyDoc_STRVAR(stat_ ## isfunc ## _doc, doc)
@@ -318,10 +335,9 @@
 static PyObject *
 stat_S_IMODE(PyObject *self, PyObject *omode)
 {
-    unsigned long mode = PyLong_AsUnsignedLong(omode);
-    if ((mode == (unsigned long)-1) && PyErr_Occurred()) {
+    mode_t mode = _PyLong_AsMode_t(omode);
+    if ((mode == (mode_t)-1) && PyErr_Occurred())
         return NULL;
-    }
     return PyLong_FromUnsignedLong(mode & S_IMODE);
 }
 
@@ -332,10 +348,9 @@
 static PyObject *
 stat_S_IFMT(PyObject *self, PyObject *omode)
 {
-    unsigned long mode = PyLong_AsUnsignedLong(omode);
-    if ((mode == (unsigned long)-1) && PyErr_Occurred()) {
+    mode_t mode = _PyLong_AsMode_t(omode);
+    if ((mode == (mode_t)-1) && PyErr_Occurred())
         return NULL;
-    }
     return PyLong_FromUnsignedLong(mode & S_IFMT);
 }
 
@@ -395,12 +410,11 @@
 stat_filemode(PyObject *self, PyObject *omode)
 {
     char buf[10];
-    unsigned long mode;
+    mode_t mode;
 
-    mode = PyLong_AsUnsignedLong(omode);
-    if ((mode == (unsigned long)-1) && PyErr_Occurred()) {
+    mode = _PyLong_AsMode_t(omode);
+    if ((mode == (mode_t)-1) && PyErr_Occurred())
         return NULL;
-    }
 
     buf[0] = filetype(mode);
     fileperm(mode, &buf[1]);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 23:52:49 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 23:52:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Check_for_corr?=
 =?utf-8?q?ect_macro=2C_code_uses_S=5FISDIR=28=29=2E?=
Message-ID: <3bdnS143wYz7Lm9@mail.python.org>

http://hg.python.org/cpython/rev/0762f2419494
changeset:   84299:0762f2419494
branch:      3.3
parent:      84292:ffcf46316e1f
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 23:51:44 2013 +0200
summary:
  Check for correct macro, code uses S_ISDIR().

files:
  Modules/_io/fileio.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -171,7 +171,7 @@
 static int
 dircheck(fileio* self, PyObject *nameobj)
 {
-#if defined(HAVE_FSTAT) && defined(S_IFDIR) && defined(EISDIR)
+#if defined(HAVE_FSTAT) && defined(S_ISDIR) && defined(EISDIR)
     struct stat buf;
     if (self->fd < 0)
         return 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 23:52:50 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 23:52:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Check_for_correct_macro=2C_code_uses_S=5FISDIR=28=29=2E?=
Message-ID: <3bdnS25vgrz7Llx@mail.python.org>

http://hg.python.org/cpython/rev/e290451883d0
changeset:   84300:e290451883d0
parent:      84298:44c8a9d80595
parent:      84299:0762f2419494
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 23:52:40 2013 +0200
summary:
  Check for correct macro, code uses S_ISDIR().

files:
  Modules/_io/fileio.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -171,7 +171,7 @@
 static int
 dircheck(fileio* self, PyObject *nameobj)
 {
-#if defined(HAVE_FSTAT) && defined(S_IFDIR) && defined(EISDIR)
+#if defined(HAVE_FSTAT) && defined(S_ISDIR) && defined(EISDIR)
     struct stat buf;
     if (self->fd < 0)
         return 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 23 23:56:56 2013
From: python-checkins at python.org (christian.heimes)
Date: Sun, 23 Jun 2013 23:56:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Define_S=5FIFMT_and_S=5FIF?=
 =?utf-8?q?LNK_in_pyport=2Eh_so_posixmodule=2Ec_can_use_named_constants?=
Message-ID: <3bdnXm75MGz7Ll4@mail.python.org>

http://hg.python.org/cpython/rev/d26e3940fece
changeset:   84301:d26e3940fece
user:        Christian Heimes <christian at cheimes.de>
date:        Sun Jun 23 23:56:05 2013 +0200
summary:
  Define S_IFMT and S_IFLNK in pyport.h so posixmodule.c can use named constants instead
of arbitrary looking numbers.

files:
  Include/pyport.h      |  15 ++++++++-------
  Modules/posixmodule.c |   4 ++--
  2 files changed, 10 insertions(+), 9 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -393,9 +393,15 @@
 #include <stat.h>
 #endif
 
-#if defined(PYCC_VACPP)
+#ifndef S_IFMT
 /* VisualAge C/C++ Failed to Define MountType Field in sys/stat.h */
-#define S_IFMT (S_IFDIR|S_IFCHR|S_IFREG)
+#define S_IFMT 0170000
+#endif
+
+#ifndef S_IFLNK
+/* Windows doesn't define S_IFLNK but posixmodule.c maps
+ * IO_REPARSE_TAG_SYMLINK to S_IFLNK */
+#  define S_IFLNK 0120000
 #endif
 
 #ifndef S_ISREG
@@ -410,11 +416,6 @@
 #define S_ISCHR(x) (((x) & S_IFMT) == S_IFCHR)
 #endif
 
-#ifndef S_ISBLK
-#define S_ISBLK(x) (((x) & S_IFMT) == S_IFBLK)
-#endif
-
-
 #ifdef __cplusplus
 /* Move this down here since some C++ #include's don't like to be included
    inside an extern "C" */
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -1405,9 +1405,9 @@
     result->st_ino = (((__int64)info->nFileIndexHigh)<<32) + info->nFileIndexLow;
     if (reparse_tag == IO_REPARSE_TAG_SYMLINK) {
         /* first clear the S_IFMT bits */
-        result->st_mode ^= (result->st_mode & 0170000);
+        result->st_mode ^= (result->st_mode & S_IFMT);
         /* now set the bits that make this a symlink */
-        result->st_mode |= 0120000;
+        result->st_mode |= S_IFLNK;
     }
 
     return 0;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 00:13:23 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 24 Jun 2013 00:13:23 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2311016=3A_Don=27t_?=
 =?utf-8?q?define_macros_and_constants_that_are_already_set_by_pyport=2Eh?=
Message-ID: <3bdnvl1D56z7LlN@mail.python.org>

http://hg.python.org/cpython/rev/75bc0ae02bcd
changeset:   84302:75bc0ae02bcd
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 24 00:13:14 2013 +0200
summary:
  Issue #11016: Don't define macros and constants that are already set by pyport.h

files:
  Modules/_stat.c |  36 +++++-------------------------------
  1 files changed, 5 insertions(+), 31 deletions(-)


diff --git a/Modules/_stat.c b/Modules/_stat.c
--- a/Modules/_stat.c
+++ b/Modules/_stat.c
@@ -39,35 +39,18 @@
  *
  * Only the names are defined by POSIX but not their value. All common file
  * types seems to have the same numeric value on all platforms, though.
+ *
+ * pyport.h guarantees S_IFMT, S_IFDIR, S_IFCHR, S_IFREG and S_IFLNK
  */
-#ifndef S_IFMT
-#  define S_IFMT 0170000
-#endif
-
-#ifndef S_IFDIR
-#  define S_IFDIR 0040000
-#endif
-
-#ifndef S_IFCHR
-#  define S_IFCHR 0020000
-#endif
 
 #ifndef S_IFBLK
 #  define S_IFBLK 0060000
 #endif
 
-#ifndef S_IFREG
-#  define S_IFREG 0100000
-#endif
-
 #ifndef S_IFIFO
 #  define S_IFIFO 0010000
 #endif
 
-#ifndef S_IFLNK
-#  define S_IFLNK 0120000
-#endif
-
 #ifndef S_IFSOCK
 #  define S_IFSOCK 0140000
 #endif
@@ -85,23 +68,14 @@
 #endif
 
 
-/* S_ISXXX() */
-#ifndef S_ISDIR
-#  define S_ISDIR(mode) (((mode) & S_IFMT) == S_IFDIR)
-#endif
-
-#ifndef S_ISCHR
-#  define S_ISCHR(mode) (((mode) & S_IFMT) == S_IFCHR)
-#endif
+/* S_ISXXX()
+ * pyport.h defines S_ISDIR(), S_ISREG() and S_ISCHR()
+ */
 
 #ifndef S_ISBLK
 #  define S_ISBLK(mode) (((mode) & S_IFMT) == S_IFBLK)
 #endif
 
-#ifndef S_ISREG
-#  define S_ISREG(mode) (((mode) & S_IFMT) == S_IFREG)
-#endif
-
 #ifndef S_ISFIFO
 #  define S_ISFIFO(mode) (((mode) & S_IFMT) == S_IFIFO)
 #endif

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 00:47:41 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 24 Jun 2013 00:47:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Misc_improvements_to_colle?=
	=?utf-8?q?ctions=2Edeque=28=29?=
Message-ID: <3bdpgK6b8Cz7Lkb@mail.python.org>

http://hg.python.org/cpython/rev/cb70776241bb
changeset:   84303:cb70776241bb
parent:      84298:44c8a9d80595
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Jun 23 15:44:33 2013 -0700
summary:
  Misc improvements to collections.deque()

* Clarified comment on the impact of BLOCKLEN on deque_index
  (with a power-of-two, the division and modulo
   computations are done with a right-shift and bitwise-and).

* Clarified comment on the overflow check to note that
  it is general and not just applicable the 64-bit builds.

* In deque._rotate(), the "deque->" indirections are
  factored-out of the loop (loop invariant code motion),
  leaving the code cleaner looking and slightly faster.

* In deque._rotate(), replaced the memcpy() with an
  equivalent loop.  That saved the memcpy setup time
  and allowed the pointers to move in their natural
  leftward and rightward directions.

See comparative timings at:  http://pastebin.com/p0RJnT5N

files:
  Modules/_collectionsmodule.c |  157 +++++++++++++---------
  1 files changed, 89 insertions(+), 68 deletions(-)


diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -8,9 +8,10 @@
 */
 
 /* The block length may be set to any number over 1.  Larger numbers
- * reduce the number of calls to the memory allocator but take more
- * memory.  Ideally, (BLOCKLEN+2) should be set to a multiple of the
- * length of a cache line.
+ * reduce the number of calls to the memory allocator, give faster
+ * indexing and rotation, and reduce the link::data overhead ratio.
+ * If the block length is a power-of-two, we also get faster
+ * division/modulo computations during indexing.
  */
 
 #define BLOCKLEN 62
@@ -47,8 +48,8 @@
 
 typedef struct BLOCK {
     struct BLOCK *leftlink;
+    PyObject *data[BLOCKLEN];
     struct BLOCK *rightlink;
-    PyObject *data[BLOCKLEN];
 } block;
 
 #define MAXFREEBLOCKS 10
@@ -58,13 +59,8 @@
 static block *
 newblock(block *leftlink, block *rightlink, Py_ssize_t len) {
     block *b;
-    /* To prevent len from overflowing PY_SSIZE_T_MAX on 64-bit machines, we
-     * refuse to allocate new blocks if the current len is dangerously
-     * close.  There is some extra margin to prevent spurious arithmetic
-     * overflows at various places.  The following check ensures that
-     * the blocks allocated to the deque, in the worst case, can only
-     * have PY_SSIZE_T_MAX-2 entries in total.
-     */
+    /* To prevent len from overflowing PY_SSIZE_T_MAX, we refuse to
+     * allocate new blocks if the current len is nearing overflow. */
     if (len >= PY_SSIZE_T_MAX - 2*BLOCKLEN) {
         PyErr_SetString(PyExc_OverflowError,
                         "cannot add more blocks to the deque");
@@ -413,7 +409,12 @@
 static int
 _deque_rotate(dequeobject *deque, Py_ssize_t n)
 {
-    Py_ssize_t m, len=deque->len, halflen=len>>1;
+    block *leftblock = deque->leftblock;
+    block *rightblock = deque->rightblock;
+    Py_ssize_t leftindex = deque->leftindex;
+    Py_ssize_t rightindex = deque->rightindex;
+    Py_ssize_t len=deque->len, halflen=len>>1;
+    int rv = 0;
 
     if (len <= 1)
         return 0;
@@ -429,76 +430,96 @@
 
     deque->state++;
     while (n > 0) {
-        if (deque->leftindex == 0) {
-            block *b = newblock(NULL, deque->leftblock, len);
-            if (b == NULL)
-                return -1;
-            assert(deque->leftblock->leftlink == NULL);
-            deque->leftblock->leftlink = b;
-            deque->leftblock = b;
-            deque->leftindex = BLOCKLEN;
+        if (leftindex == 0) {
+            block *b = newblock(NULL, leftblock, len);
+            if (b == NULL) {
+                rv = -1;
+                goto done;
+            }
+            assert(leftblock->leftlink == NULL);
+            leftblock->leftlink = b;
+            leftblock = b;
+            leftindex = BLOCKLEN;
         }
-        assert(deque->leftindex > 0);
+        assert(leftindex > 0);
 
-        m = n;
-        if (m > deque->rightindex + 1)
-            m = deque->rightindex + 1;
-        if (m > deque->leftindex)
-            m = deque->leftindex;
-        assert (m > 0 && m <= len);
-        memcpy(&deque->leftblock->data[deque->leftindex - m],
-               &deque->rightblock->data[deque->rightindex + 1 - m],
-               m * sizeof(PyObject *));
-        deque->rightindex -= m;
-        deque->leftindex -= m;
-        n -= m;
+        {
+            PyObject **src, **dest;
+            Py_ssize_t m = n;
 
-        if (deque->rightindex == -1) {
-            block *prevblock = deque->rightblock->leftlink;
-            assert(deque->rightblock != NULL);
-            assert(deque->leftblock != deque->rightblock);
-            freeblock(deque->rightblock);
+            if (m > rightindex + 1)
+                m = rightindex + 1;
+            if (m > leftindex)
+                m = leftindex;
+            assert (m > 0 && m <= len);
+            src = &rightblock->data[rightindex];
+            dest = &leftblock->data[leftindex - 1];
+            rightindex -= m;
+            leftindex -= m;
+            n -= m;
+            while (m--)
+                *(dest--) = *(src--);
+        }
+
+        if (rightindex == -1) {
+            block *prevblock = rightblock->leftlink;
+            assert(rightblock != NULL);
+            assert(leftblock != rightblock);
+            freeblock(rightblock);
             prevblock->rightlink = NULL;
-            deque->rightblock = prevblock;
-            deque->rightindex = BLOCKLEN - 1;
+            rightblock = prevblock;
+            rightindex = BLOCKLEN - 1;
         }
     }
     while (n < 0) {
-        if (deque->rightindex == BLOCKLEN - 1) {
-            block *b = newblock(deque->rightblock, NULL, len);
-            if (b == NULL)
-                return -1;
-            assert(deque->rightblock->rightlink == NULL);
-            deque->rightblock->rightlink = b;
-            deque->rightblock = b;
-            deque->rightindex = -1;
+        if (rightindex == BLOCKLEN - 1) {
+            block *b = newblock(rightblock, NULL, len);
+            if (b == NULL) {
+                rv = -1;
+                goto done;
+            }
+            assert(rightblock->rightlink == NULL);
+            rightblock->rightlink = b;
+            rightblock = b;
+            rightindex = -1;
         }
-        assert (deque->rightindex < BLOCKLEN - 1);
+        assert (rightindex < BLOCKLEN - 1);
 
-        m = -n;
-        if (m > BLOCKLEN - deque->leftindex)
-            m = BLOCKLEN - deque->leftindex;
-        if (m > BLOCKLEN - 1 - deque->rightindex)
-            m = BLOCKLEN - 1 - deque->rightindex;
-        assert (m > 0 && m <= len);
-        memcpy(&deque->rightblock->data[deque->rightindex + 1],
-               &deque->leftblock->data[deque->leftindex],
-               m * sizeof(PyObject *));
-        deque->leftindex += m;
-        deque->rightindex += m;
-        n += m;
+        {
+            PyObject **src, **dest;
+            Py_ssize_t m = -n;
 
-        if (deque->leftindex == BLOCKLEN) {
-            block *nextblock = deque->leftblock->rightlink;
-            assert(deque->leftblock != deque->rightblock);
-            freeblock(deque->leftblock);
+            if (m > BLOCKLEN - leftindex)
+                m = BLOCKLEN - leftindex;
+            if (m > BLOCKLEN - 1 - rightindex)
+                m = BLOCKLEN - 1 - rightindex;
+            assert (m > 0 && m <= len);
+            src = &leftblock->data[leftindex];
+            dest = &rightblock->data[rightindex + 1];
+            leftindex += m;
+            rightindex += m;
+            n += m;
+            while (m--)
+                *(dest++) = *(src++);
+        }
+
+        if (leftindex == BLOCKLEN) {
+            block *nextblock = leftblock->rightlink;
+            assert(leftblock != rightblock);
+            freeblock(leftblock);
             assert(nextblock != NULL);
             nextblock->leftlink = NULL;
-            deque->leftblock = nextblock;
-            deque->leftindex = 0;
+            leftblock = nextblock;
+            leftindex = 0;
         }
     }
-    return 0;
+done:
+    deque->leftblock = leftblock;
+    deque->rightblock = rightblock;
+    deque->leftindex = leftindex;
+    deque->rightindex = rightindex;
+
+    return rv;
 }
 
 static PyObject *

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 00:47:43 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Mon, 24 Jun 2013 00:47:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_merge?=
Message-ID: <3bdpgM1T4bz7Lkb@mail.python.org>

http://hg.python.org/cpython/rev/7aab60b70f90
changeset:   84304:7aab60b70f90
parent:      84303:cb70776241bb
parent:      84302:75bc0ae02bcd
user:        Raymond Hettinger <python at rcn.com>
date:        Sun Jun 23 15:47:03 2013 -0700
summary:
  merge

files:
  Include/pyport.h      |  15 ++++++------
  Modules/_io/fileio.c  |   2 +-
  Modules/_stat.c       |  36 ++++--------------------------
  Modules/posixmodule.c |   4 +-
  4 files changed, 16 insertions(+), 41 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -393,9 +393,15 @@
 #include <stat.h>
 #endif
 
-#if defined(PYCC_VACPP)
+#ifndef S_IFMT
 /* VisualAge C/C++ Failed to Define MountType Field in sys/stat.h */
-#define S_IFMT (S_IFDIR|S_IFCHR|S_IFREG)
+#define S_IFMT 0170000
+#endif
+
+#ifndef S_IFLNK
+/* Windows doesn't define S_IFLNK but posixmodule.c maps
+ * IO_REPARSE_TAG_SYMLINK to S_IFLNK */
+#  define S_IFLNK 0120000
 #endif
 
 #ifndef S_ISREG
@@ -410,11 +416,6 @@
 #define S_ISCHR(x) (((x) & S_IFMT) == S_IFCHR)
 #endif
 
-#ifndef S_ISBLK
-#define S_ISBLK(x) (((x) & S_IFMT) == S_IFBLK)
-#endif
-
-
 #ifdef __cplusplus
 /* Move this down here since some C++ #include's don't like to be included
    inside an extern "C" */
diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -171,7 +171,7 @@
 static int
 dircheck(fileio* self, PyObject *nameobj)
 {
-#if defined(HAVE_FSTAT) && defined(S_IFDIR) && defined(EISDIR)
+#if defined(HAVE_FSTAT) && defined(S_ISDIR) && defined(EISDIR)
     struct stat buf;
     if (self->fd < 0)
         return 0;
diff --git a/Modules/_stat.c b/Modules/_stat.c
--- a/Modules/_stat.c
+++ b/Modules/_stat.c
@@ -39,35 +39,18 @@
  *
  * Only the names are defined by POSIX but not their value. All common file
  * types seems to have the same numeric value on all platforms, though.
+ *
+ * pyport.h guarantees S_IFMT, S_IFDIR, S_IFCHR, S_IFREG and S_IFLNK
  */
-#ifndef S_IFMT
-#  define S_IFMT 0170000
-#endif
-
-#ifndef S_IFDIR
-#  define S_IFDIR 0040000
-#endif
-
-#ifndef S_IFCHR
-#  define S_IFCHR 0020000
-#endif
 
 #ifndef S_IFBLK
 #  define S_IFBLK 0060000
 #endif
 
-#ifndef S_IFREG
-#  define S_IFREG 0100000
-#endif
-
 #ifndef S_IFIFO
 #  define S_IFIFO 0010000
 #endif
 
-#ifndef S_IFLNK
-#  define S_IFLNK 0120000
-#endif
-
 #ifndef S_IFSOCK
 #  define S_IFSOCK 0140000
 #endif
@@ -85,23 +68,14 @@
 #endif
 
 
-/* S_ISXXX() */
-#ifndef S_ISDIR
-#  define S_ISDIR(mode) (((mode) & S_IFMT) == S_IFDIR)
-#endif
-
-#ifndef S_ISCHR
-#  define S_ISCHR(mode) (((mode) & S_IFMT) == S_IFCHR)
-#endif
+/* S_ISXXX()
+ * pyport.h defines S_ISDIR(), S_ISREG() and S_ISCHR()
+ */
 
 #ifndef S_ISBLK
 #  define S_ISBLK(mode) (((mode) & S_IFMT) == S_IFBLK)
 #endif
 
-#ifndef S_ISREG
-#  define S_ISREG(mode) (((mode) & S_IFMT) == S_IFREG)
-#endif
-
 #ifndef S_ISFIFO
 #  define S_ISFIFO(mode) (((mode) & S_IFMT) == S_IFIFO)
 #endif
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -1405,9 +1405,9 @@
     result->st_ino = (((__int64)info->nFileIndexHigh)<<32) + info->nFileIndexLow;
     if (reparse_tag == IO_REPARSE_TAG_SYMLINK) {
         /* first clear the S_IFMT bits */
-        result->st_mode ^= (result->st_mode & 0170000);
+        result->st_mode ^= (result->st_mode & S_IFMT);
         /* now set the bits that make this a symlink */
-        result->st_mode |= 0120000;
+        result->st_mode |= S_IFLNK;
     }
 
     return 0;

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Mon Jun 24 05:46:18 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Mon, 24 Jun 2013 05:46:18 +0200
Subject: [Python-checkins] Daily reference leaks (7aab60b70f90): sum=-1
Message-ID: <E1UqxjO-000500-PP@ap.vmr.nerim.net>

results for 7aab60b70f90 on branch "default"
--------------------------------------------

test_unittest leaked [-1, 2, 0] memory blocks, sum=1
test_concurrent_futures leaked [0, 0, -2] memory blocks, sum=-2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogfE6EZY', '-x']

From python-checkins at python.org  Mon Jun 24 13:06:03 2013
From: python-checkins at python.org (nick.coghlan)
Date: Mon, 24 Jun 2013 13:06:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_426=3A_make_summary_field?=
	=?utf-8?q?_mandatory?=
Message-ID: <3bf73H6CxYz7Ljt@mail.python.org>

http://hg.python.org/peps/rev/14818873634f
changeset:   4960:14818873634f
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Mon Jun 24 21:05:51 2013 +1000
summary:
  PEP 426: make summary field mandatory

files:
  pep-0426.txt |  40 ++++++++++++++++++++--------------------
  1 files changed, 20 insertions(+), 20 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -566,6 +566,24 @@
     "version": "1.0a2"
 
 
+Summary
+-------
+
+A short summary of what the distribution does.
+
+This field SHOULD contain fewer than 512 characters and MUST contain fewer
+than 2048.
+
+This field SHOULD NOT contain any line breaks.
+
+A more complete description SHOULD be included as a separate file in the
+sdist for the distribution. See `Document names`_ for details.
+
+Example::
+
+    "summary": "A module that is more fiendish than soft cushions."
+
+
 Source code metadata
 ====================
 
@@ -658,26 +676,6 @@
 operation depending on one of these fields is requested.
 
 
-Summary
--------
-
-A one-line summary of what the distribution does.
-
-Publication tools SHOULD emit a warning if this field is not provided. Index
-servers MAY require that this field be provided before allowing a
-distribution to be uploaded.
-
-This field SHOULD contain fewer than 512 characters and MUST contain fewer
-than 2048.
-
-A more complete description SHOULD be included as a separate file in the
-sdist for the distribution. See `Document names`_ for details.
-
-Example::
-
-    "summary": "A module that is more fiendish than soft cushions."
-
-
 License
 -------
 
@@ -692,6 +690,8 @@
 This field SHOULD contain fewer than 512 characters and MUST contain fewer
 than 2048.
 
+This field SHOULD NOT contain any line breaks.
+
 The full license text SHOULD be included as a separate file in the source
 archive for the distribution. See `Document names`_ for details.
 

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Mon Jun 24 13:38:41 2013
From: python-checkins at python.org (nick.coghlan)
Date: Mon, 24 Jun 2013 13:38:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_426=3A_tweak_standard_bui?=
	=?utf-8?q?ld_system_description?=
Message-ID: <3bf7mx2qxfz7Ljt@mail.python.org>

http://hg.python.org/peps/rev/5be19fe2c1fa
changeset:   4961:5be19fe2c1fa
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Mon Jun 24 21:38:31 2013 +1000
summary:
  PEP 426: tweak standard build system description

files:
  pep-0426.txt |  78 +++++++++++++++++++++++----------------
  1 files changed, 45 insertions(+), 33 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -261,10 +261,8 @@
 * obtain the original source code that was used to create a distribution
 * identify and retrieve the dependencies (if any) required to use a
   distribution
-* identify and retrieve the dependencies (if any) required to build a distribution
-  from source
-* if supported by the distribution, run the distribution's automatic test
-  suite on an installed instance of the distribution
+* identify and retrieve the dependencies (if any) required to build a
+  distribution from source
 * identify and retrieve the dependencies (if any) required to run a
   distribution's test suite
 * find resources on using and contributing to the project
@@ -272,18 +270,6 @@
   publishers through appropriate channels, as well as finding distributions
   that are relevant to particular problems
 
-The current iteration of the metadata relies on the ``distutils`` commands
-system to support other necessary integration and deployment activities:
-
-* ``python setup.py bdist_wheel``: create a binary archive from a source
-  archive or checkout
-* ``python setup.py test``: run the distribution's test suite on a built
-  (but not yet installed) distribution
-
-Future iterations of the metadata will aim to replace these ``distutils``/
-``setuptools`` dependent commands with build system independent entry
-points.
-
 
 Development and publication of distributions
 --------------------------------------------
@@ -307,17 +293,30 @@
 * specify the additional dependencies (if any) required to develop and
   publish the distribution
 
-The current iteration of the metadata relies on the ``distutils`` commands
-system to support other necessary development and publication activities:
-
-* ``python setup.py dist_info``: generate the ``pymeta.json`` file for a
-  distribution
+
+Standard build system
+---------------------
+
+Both development and integration of distributions relies on the ability to
+build extension modules and perform other operations in a distribution
+independent manner.
+
+The current iteration of the metadata relies on the
+``distutils``/``setuptools`` commands system to support these necessary
+development and integration activities:
+
+* ``python setup.py dist_info``: generate distribution metadata in place
+  given a source archive or VCS checkout
 * ``python setup.py sdist``: create an sdist from a source archive
   or VCS checkout
-* ``python setup.py test``: run the distribution's test suite on a built
-  (but not yet installed) distribution
-
-Future iterations of the metadata and associated PEPs will aim to replace
+* ``python setup.py build_ext --inplace``: build extension modules in place
+  given an sdist, source archive or VCS checkout
+* ``python setup.py test``: run the distribution's test suite in place
+  given an sdist, source archive or VCS checkout
+* ``python setup.py bdist_wheel``: create a binary archive from an sdist,
+  source archive or VCS checkout
+
+Future iterations of the metadata and associated PEPs may aim to replace
 these ``distutils``/``setuptools`` dependent commands with build system
 independent entry points.
 
@@ -1506,11 +1505,12 @@
 Install hook implementations MUST use the given parameter names.
 
 Installation tools SHOULD invoke install hooks automatically after
-installing a distribution from a binary archive. When installing from
-an sdist, source archive or VCS checkout using ``setup.py install``
-installation tools MUST NOT invoke the install hooks - it is assumed
-that the ``setup.py`` script will already invoke any necessary
-post-installation behaviour.
+installing a distribution from a binary archive.
+
+When installing from an sdist, source archive or VCS checkout, installation
+tools SHOULD create a binary archive using ``setup.py bdist_wheel`` and
+then install binary archive normally (including invocation of any install
+hooks). Installation tools SHOULD NOT invoke ``setup.py install`` directly.
 
 Installation tools SHOULD treat an exception thrown by a postinstall hook
 as a failure of the installation and revert any other changes made to the
@@ -2024,8 +2024,9 @@
 communities, while still interoperating with the cross-platform Python
 tools.
 
-Falling back to invoking ``setup.py install`` directly remains as the
-interim solution for installation from source archives.
+Legacy packages that expect to able to run code on target systems using
+``setup.py install`` will no longer work correctly. Such packages will
+already break when pip 1.4+ is configured to use a wheel cache directory.
 
 
 Changes to environment markers
@@ -2252,8 +2253,9 @@
 
 * ``python setup.py dist_info``
 * ``python setup.py sdist``
+* ``python setup.py build_ext --inplace``
+* ``python setup.py test``
 * ``python setup.py bdist_wheel``
-* ``python setup.py test``
 
 The following metabuild hooks may be defined in metadata 2.1 to
 cover these operations without relying on ``setup.py``:
@@ -2326,6 +2328,16 @@
 way it doesn't matter if the additional dependencies were requested
 explicitly or just happen to be available on the system.
 
+There are still a number of open questions with this design, such as whether
+a single build hook is sufficient to cover both "build for testing" and
+"prep for deployment", as well as various complexities like support for
+cross-compilation of binaries, specification of target platforms and
+Python versions when creating wheel files, etc.
+
+Opting to retain the status quo for now allows us to make progress on
+improved metadata publication and binary installation support, rather than
+having to delay that awaiting the creation of a viable metabuild framework.
+
 
 Appendix E: Rejected features
 =============================

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Mon Jun 24 13:49:00 2013
From: python-checkins at python.org (nick.coghlan)
Date: Mon, 24 Jun 2013 13:49:00 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_426=3A_add_distlib_as_a_r?=
	=?utf-8?q?eference_impl?=
Message-ID: <3bf80r3x4Tz7Ljt@mail.python.org>

http://hg.python.org/peps/rev/1b71d1e87f64
changeset:   4962:1b71d1e87f64
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Mon Jun 24 21:48:49 2013 +1000
summary:
  PEP 426: add distlib as a reference impl

files:
  pep-0426.txt |  15 ++++++++++-----
  1 files changed, 10 insertions(+), 5 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -1763,11 +1763,16 @@
 ================================================
 
 The reference implementations for converting from legacy metadata to
-metadata 2.0 are Daniel Holth's `wheel project
-<https://bitbucket.org/dholth/wheel/overview>`__ (which
-adds the ``bdist_wheel`` command to ``setuptools``) and Donald Stufft's
-`Warehouse project <https://github.com/dstufft/warehouse>`__ (which will
-eventually be the next generation Python Package Index implementation).
+metadata 2.0 are:
+
+* the `wheel project <https://bitbucket.org/dholth/wheel/overview>`__, which
+  adds the ``bdist_wheel`` command to ``setuptools``
+* the `Warehouse project <https://github.com/dstufft/warehouse>`__, which
+  will eventually be migrated to the Python Packaging Authority as the next
+  generation Python Package Index implementation
+* the `distlib project <https://bitbucket.org/pypa/distlib/>`__ which is
+  derived from the core packaging infrastructure created for the
+  ``distutils2`` project and
 
 While it is expected that there may be some edge cases where manual
 intervention is needed for clean conversion, the specification has been

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Mon Jun 24 15:43:25 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 24 Jun 2013 15:43:25 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Fix_test_for_G?=
 =?utf-8?q?CC_3=2E1+_but_not_strict_ANSI_C?=
Message-ID: <3bfBXs4dVNz7Ln2@mail.python.org>

http://hg.python.org/cpython/rev/b7bdb41aa534
changeset:   84305:b7bdb41aa534
branch:      3.3
parent:      84299:0762f2419494
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 24 15:39:41 2013 +0200
summary:
  Fix test for GCC 3.1+ but not strict ANSI C

files:
  Include/pymacro.h |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Include/pymacro.h b/Include/pymacro.h
--- a/Include/pymacro.h
+++ b/Include/pymacro.h
@@ -33,7 +33,7 @@
 
    Requires at GCC 3.1+ */
 #if (defined(__GNUC__) && !defined(__STRICT_ANSI__) && \
-    ((__GNUC__ == 3) && (__GNU_MINOR__ >= 1)) || (__GNUC__ >= 4))
+    (((__GNUC__ == 3) && (__GNU_MINOR__ >= 1)) || (__GNUC__ >= 4)))
 /* Two gcc extensions.
    &a[0] degrades to a pointer: a different type from an array */
 #define Py_ARRAY_LENGTH(array) \

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 15:43:26 2013
From: python-checkins at python.org (christian.heimes)
Date: Mon, 24 Jun 2013 15:43:26 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Fix_test_for_GCC_3=2E1+_but_not_strict_ANSI_C?=
Message-ID: <3bfBXt6YJ8z7LnW@mail.python.org>

http://hg.python.org/cpython/rev/6915dfddb3f6
changeset:   84306:6915dfddb3f6
parent:      84304:7aab60b70f90
parent:      84305:b7bdb41aa534
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Jun 24 15:39:58 2013 +0200
summary:
  Fix test for GCC 3.1+ but not strict ANSI C

files:
  Include/pymacro.h |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Include/pymacro.h b/Include/pymacro.h
--- a/Include/pymacro.h
+++ b/Include/pymacro.h
@@ -33,7 +33,7 @@
 
    Requires at GCC 3.1+ */
 #if (defined(__GNUC__) && !defined(__STRICT_ANSI__) && \
-    ((__GNUC__ == 3) && (__GNU_MINOR__ >= 1)) || (__GNUC__ >= 4))
+    (((__GNUC__ == 3) && (__GNU_MINOR__ >= 1)) || (__GNUC__ >= 4)))
 /* Two gcc extensions.
    &a[0] degrades to a pointer: a different type from an array */
 #define Py_ARRAY_LENGTH(array) \

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 15:53:27 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 15:53:27 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4Mjc3?=
 =?utf-8?q?=3A_Document_quirks_of_multiprocessing_queue=2E?=
Message-ID: <3bfBmR47hXz7Lls@mail.python.org>

http://hg.python.org/cpython/rev/8dcc4e017d42
changeset:   84307:8dcc4e017d42
branch:      2.7
parent:      84295:a7db9f505e88
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 14:45:24 2013 +0100
summary:
  Issue #18277: Document quirks of multiprocessing queue.

files:
  Doc/library/multiprocessing.rst |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -486,6 +486,23 @@
    the :mod:`multiprocessing` namespace so you need to import them from
    :mod:`Queue`.
 
+.. note::
+
+   When an object is put on a queue, the object is pickled and a
+   background thread later flushes the pickled data to an underlying
+   pipe.  This has some consequences which are a little surprising,
+   but should not cause any pratical difficulties -- you can always
+   use a managed queue if they really bother you.
+
+   (1) After putting an object on an empty queue there may be an
+       infinitessimal delay before the queue's :meth:`~Queue.empty`
+       method returns :const:`False` and :meth:`~Queue.get_nowait` can
+       return without raising :exc:`Queue.Empty`.
+
+   (2) If multiple processes are enqueuing objects, it is possible for
+       the objects to be received at the other end out-of-order.
+       However, objects enqueued by the same process will always be in
+       the expected order with respect to each other.
 
 .. warning::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 15:53:28 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 15:53:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4Mjc3?=
 =?utf-8?q?=3A_Document_quirks_of_multiprocessing_queue=2E?=
Message-ID: <3bfBmS6pvFz7Ln6@mail.python.org>

http://hg.python.org/cpython/rev/0f921e73433a
changeset:   84308:0f921e73433a
branch:      3.3
parent:      84305:b7bdb41aa534
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 14:48:07 2013 +0100
summary:
  Issue #18277: Document quirks of multiprocessing queue.

files:
  Doc/library/multiprocessing.rst |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -515,6 +515,23 @@
    the :mod:`multiprocessing` namespace so you need to import them from
    :mod:`queue`.
 
+.. note::
+
+   When an object is put on a queue, the object is pickled and a
+   background thread later flushes the pickled data to an underlying
+   pipe.  This has some consequences which are a little surprising,
+   but should not cause any pratical difficulties -- you can always
+   use a managed queue if they really bother you.
+
+   (1) After putting an object on an empty queue there may be an
+       infinitessimal delay before the queue's :meth:`~Queue.empty`
+       method returns :const:`False` and :meth:`~Queue.get_nowait` can
+       return without raising :exc:`Queue.Empty`.
+
+   (2) If multiple processes are enqueuing objects, it is possible for
+       the objects to be received at the other end out-of-order.
+       However, objects enqueued by the same process will always be in
+       the expected order with respect to each other.
 
 .. warning::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 15:53:30 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 15:53:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogSXNzdWUgIzE4Mjc3OiBNZXJnZS4=?=
Message-ID: <3bfBmV1xG8z7LnW@mail.python.org>

http://hg.python.org/cpython/rev/06b1447becdc
changeset:   84309:06b1447becdc
parent:      84306:6915dfddb3f6
parent:      84308:0f921e73433a
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 14:52:14 2013 +0100
summary:
  Issue #18277: Merge.

files:
  Doc/library/multiprocessing.rst |  17 +++++++++++++++++
  1 files changed, 17 insertions(+), 0 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -515,6 +515,23 @@
    the :mod:`multiprocessing` namespace so you need to import them from
    :mod:`queue`.
 
+.. note::
+
+   When an object is put on a queue, the object is pickled and a
+   background thread later flushes the pickled data to an underlying
+   pipe.  This has some consequences which are a little surprising,
+   but should not cause any pratical difficulties -- you can always
+   use a managed queue if they really bother you.
+
+   (1) After putting an object on an empty queue there may be an
+       infinitessimal delay before the queue's :meth:`~Queue.empty`
+       method returns :const:`False` and :meth:`~Queue.get_nowait` can
+       return without raising :exc:`Queue.Empty`.
+
+   (2) If multiple processes are enqueuing objects, it is possible for
+       the objects to be received at the other end out-of-order.
+       However, objects enqueued by the same process will always be in
+       the expected order with respect to each other.
 
 .. warning::
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 16:44:41 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 16:44:41 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE1ODE4?=
	=?utf-8?q?=3A_Typo_in_docs=2E?=
Message-ID: <3bfCvY4cNsz7LmF@mail.python.org>

http://hg.python.org/cpython/rev/f50bbae95bc8
changeset:   84310:f50bbae95bc8
branch:      2.7
parent:      84307:8dcc4e017d42
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 15:41:36 2013 +0100
summary:
  Issue #15818: Typo in docs.

files:
  Doc/library/multiprocessing.rst |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -423,9 +423,9 @@
          acquired a lock or semaphore etc. then terminating it is liable to
          cause other processes to deadlock.
 
-   Note that the :meth:`start`, :meth:`join`, :meth:`is_alive` and
-   :attr:`exit_code` methods should only be called by the process that created
-   the process object.
+   Note that the :meth:`start`, :meth:`join`, :meth:`is_alive`,
+   :meth:`terminate` and :attr:`exitcode` methods should only be called by
+   the process that created the process object.
 
    Example usage of some of the methods of :class:`Process`:
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 16:44:42 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 16:44:42 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE1ODE4?=
	=?utf-8?q?=3A_Typo_in_docs=2E?=
Message-ID: <3bfCvZ6YF9z7LnM@mail.python.org>

http://hg.python.org/cpython/rev/3a5e2f1dce5c
changeset:   84311:3a5e2f1dce5c
branch:      3.3
parent:      84308:0f921e73433a
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 15:42:00 2013 +0100
summary:
  Issue #15818: Typo in docs.

files:
  Doc/library/multiprocessing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -443,7 +443,7 @@
          cause other processes to deadlock.
 
    Note that the :meth:`start`, :meth:`join`, :meth:`is_alive`,
-   :meth:`terminate` and :attr:`exit_code` methods should only be called by
+   :meth:`terminate` and :attr:`exitcode` methods should only be called by
    the process that created the process object.
 
    Example usage of some of the methods of :class:`Process`:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 16:44:44 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 16:44:44 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogSXNzdWUgIzE1ODE4OiBNZXJnZS4=?=
Message-ID: <3bfCvc1HHLz7Lnc@mail.python.org>

http://hg.python.org/cpython/rev/4f08d4647f75
changeset:   84312:4f08d4647f75
parent:      84309:06b1447becdc
parent:      84311:3a5e2f1dce5c
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 15:43:35 2013 +0100
summary:
  Issue #15818: Merge.

files:
  Doc/library/multiprocessing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -443,7 +443,7 @@
          cause other processes to deadlock.
 
    Note that the :meth:`start`, :meth:`join`, :meth:`is_alive`,
-   :meth:`terminate` and :attr:`exit_code` methods should only be called by
+   :meth:`terminate` and :attr:`exitcode` methods should only be called by
    the process that created the process object.
 
    Example usage of some of the methods of :class:`Process`:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 19:15:00 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 19:15:00 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Clarify_note_a?=
	=?utf-8?q?nd_fix_typo=2E?=
Message-ID: <3bfHF02vncz7LlV@mail.python.org>

http://hg.python.org/cpython/rev/860fc6a2bd21
changeset:   84313:860fc6a2bd21
branch:      2.7
parent:      84310:f50bbae95bc8
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 18:11:21 2013 +0100
summary:
  Clarify note and fix typo.

files:
  Doc/library/multiprocessing.rst |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -491,8 +491,9 @@
    When an object is put on a queue, the object is pickled and a
    background thread later flushes the pickled data to an underlying
    pipe.  This has some consequences which are a little surprising,
-   but should not cause any pratical difficulties -- you can always
-   use a managed queue if they really bother you.
+   but should not cause any practical difficulties -- if they really
+   bother you then you can instead use a queue created with a
+   :ref:`manager <multiprocessing-managers>`.
 
    (1) After putting an object on an empty queue there may be an
        infinitessimal delay before the queue's :meth:`~Queue.empty`

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 19:15:01 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 19:15:01 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Clarify_note_a?=
	=?utf-8?q?nd_fix_typo=2E?=
Message-ID: <3bfHF14prQz7LlV@mail.python.org>

http://hg.python.org/cpython/rev/347647a1f798
changeset:   84314:347647a1f798
branch:      3.3
parent:      84311:3a5e2f1dce5c
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 18:12:57 2013 +0100
summary:
  Clarify note and fix typo.

files:
  Doc/library/multiprocessing.rst |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -520,8 +520,9 @@
    When an object is put on a queue, the object is pickled and a
    background thread later flushes the pickled data to an underlying
    pipe.  This has some consequences which are a little surprising,
-   but should not cause any pratical difficulties -- you can always
-   use a managed queue if they really bother you.
+   but should not cause any practical difficulties -- if they really
+   bother you then you can instead use a queue created with a
+   :ref:`manager <multiprocessing-managers>`.
 
    (1) After putting an object on an empty queue there may be an
        infinitessimal delay before the queue's :meth:`~Queue.empty`

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 19:15:02 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 19:15:02 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Uu?=
Message-ID: <3bfHF26pbyz7LmH@mail.python.org>

http://hg.python.org/cpython/rev/9dbfe5384301
changeset:   84315:9dbfe5384301
parent:      84312:4f08d4647f75
parent:      84314:347647a1f798
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 18:13:54 2013 +0100
summary:
  Merge.

files:
  Doc/library/multiprocessing.rst |  5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -520,8 +520,9 @@
    When an object is put on a queue, the object is pickled and a
    background thread later flushes the pickled data to an underlying
    pipe.  This has some consequences which are a little surprising,
-   but should not cause any pratical difficulties -- you can always
-   use a managed queue if they really bother you.
+   but should not cause any practical difficulties -- if they really
+   bother you then you can instead use a queue created with a
+   :ref:`manager <multiprocessing-managers>`.
 
    (1) After putting an object on an empty queue there may be an
        infinitessimal delay before the queue's :meth:`~Queue.empty`

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 21:40:37 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 21:40:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4IHR5cG8u?=
Message-ID: <3bfLT12L8Nz7LjY@mail.python.org>

http://hg.python.org/cpython/rev/81a96042cff4
changeset:   84316:81a96042cff4
branch:      2.7
parent:      84313:860fc6a2bd21
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 20:38:22 2013 +0100
summary:
  Fix typo.

files:
  Doc/library/multiprocessing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -496,7 +496,7 @@
    :ref:`manager <multiprocessing-managers>`.
 
    (1) After putting an object on an empty queue there may be an
-       infinitessimal delay before the queue's :meth:`~Queue.empty`
+       infinitesimal delay before the queue's :meth:`~Queue.empty`
        method returns :const:`False` and :meth:`~Queue.get_nowait` can
        return without raising :exc:`Queue.Empty`.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 21:40:38 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 21:40:38 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogRml4IHR5cG8u?=
Message-ID: <3bfLT24GjRz7LnT@mail.python.org>

http://hg.python.org/cpython/rev/6e2f02b59fcf
changeset:   84317:6e2f02b59fcf
branch:      3.3
parent:      84314:347647a1f798
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 20:38:46 2013 +0100
summary:
  Fix typo.

files:
  Doc/library/multiprocessing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -525,7 +525,7 @@
    :ref:`manager <multiprocessing-managers>`.
 
    (1) After putting an object on an empty queue there may be an
-       infinitessimal delay before the queue's :meth:`~Queue.empty`
+       infinitesimal delay before the queue's :meth:`~Queue.empty`
        method returns :const:`False` and :meth:`~Queue.get_nowait` can
        return without raising :exc:`Queue.Empty`.
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 21:40:39 2013
From: python-checkins at python.org (richard.oudkerk)
Date: Mon, 24 Jun 2013 21:40:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Uu?=
Message-ID: <3bfLT367GFz7LjY@mail.python.org>

http://hg.python.org/cpython/rev/efe3c1df3954
changeset:   84318:efe3c1df3954
parent:      84315:9dbfe5384301
parent:      84317:6e2f02b59fcf
user:        Richard Oudkerk <shibturn at gmail.com>
date:        Mon Jun 24 20:39:28 2013 +0100
summary:
  Merge.

files:
  Doc/library/multiprocessing.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -525,7 +525,7 @@
    :ref:`manager <multiprocessing-managers>`.
 
    (1) After putting an object on an empty queue there may be an
-       infinitessimal delay before the queue's :meth:`~Queue.empty`
+       infinitesimal delay before the queue's :meth:`~Queue.empty`
        method returns :const:`False` and :meth:`~Queue.get_nowait` can
        return without raising :exc:`Queue.Empty`.
 

-- 
Repository URL: http://hg.python.org/cpython

From fijall at gmail.com  Mon Jun 24 22:14:46 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 24 Jun 2013 22:14:46 +0200
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
In-Reply-To: <20130623161440.2587A250338@webabinitio.net>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
Message-ID: <CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>

On Sun, Jun 23, 2013 at 6:14 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
>> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
>> > <python-checkins at python.org> wrote:
>> >>  .TP
>> >> +.BI "\-X " option
>> >> +Set implementation specific option.
>> >
>> >
>> > Should probably be "Set the implementation-specific option."
>>
>> Is there anyone respecting this notation? (I know pypy does not, it
>> uses --jit and stuff)
>
> CPython does.  We introduced it for ourselves, it is up to other
> implementations whether or not to use it, or use something else.
>
> --David

you mean "CPython does not have any implementation-specific options"?
I would claim -O behavior should be implementation-specific since it's
nonsense in the optimizations sense, but other than that, it does not
seem that there is any -X options?

From alex.gaynor at gmail.com  Mon Jun 24 22:17:30 2013
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Mon, 24 Jun 2013 13:17:30 -0700
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
In-Reply-To: <CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
Message-ID: <CAFRnB2WU_6p=2TzBbo2v7QvuOzSyk1i-0FU2J4YXx+xmxRfj3Q@mail.gmail.com>

3.3 adds some -X options around faulthandler if I recall correctly.

Alex


On Mon, Jun 24, 2013 at 1:14 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Sun, Jun 23, 2013 at 6:14 PM, R. David Murray <rdmurray at bitdance.com>
> wrote:
> > On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
> >> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
> >> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
> >> > <python-checkins at python.org> wrote:
> >> >>  .TP
> >> >> +.BI "\-X " option
> >> >> +Set implementation specific option.
> >> >
> >> >
> >> > Should probably be "Set the implementation-specific option."
> >>
> >> Is there anyone respecting this notation? (I know pypy does not, it
> >> uses --jit and stuff)
> >
> > CPython does.  We introduced it for ourselves, it is up to other
> > implementations whether or not to use it, or use something else.
> >
> > --David
>
> you mean "CPython does not have any implementation-specific options"?
> I would claim -O behavior should be implementation-specific since it's
> nonsense in the optimizations sense, but other than that, it does not
> seem that there is any -X options?
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130624/4965f04d/attachment.html>

From rdmurray at bitdance.com  Mon Jun 24 22:43:15 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 24 Jun 2013 16:43:15 -0400
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
	options to python man page.
In-Reply-To: <CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
Message-ID: <20130624204316.028CB2500E3@webabinitio.net>

On Mon, 24 Jun 2013 22:14:46 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Sun, Jun 23, 2013 at 6:14 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> > On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
> >> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
> >> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
> >> > <python-checkins at python.org> wrote:
> >> >>  .TP
> >> >> +.BI "\-X " option
> >> >> +Set implementation specific option.
> >> >
> >> >
> >> > Should probably be "Set the implementation-specific option."
> >>
> >> Is there anyone respecting this notation? (I know pypy does not, it
> >> uses --jit and stuff)
> >
> > CPython does.  We introduced it for ourselves, it is up to other
> > implementations whether or not to use it, or use something else.
> >
> > --David
> 
> you mean "CPython does not have any implementation-specific options"?
> I would claim -O behavior should be implementation-specific since it's
> nonsense in the optimizations sense, but other than that, it does not
> seem that there is any -X options?

There is one.  -X faulthandler.  I'm sure others would agree about
-O, but that long predates -X.

So, the idea is that -X *can* be used by other implementations for their
own purposes, but there is certainly no requirement that they do so.
Our promise is that anything CPython uses it for is something we don't
expect other implementations to support.

--David

From benjamin at python.org  Mon Jun 24 22:43:45 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 24 Jun 2013 13:43:45 -0700
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
In-Reply-To: <CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
Message-ID: <CAPZV6o_ZR1pyydwLhatWQPp1caW4=scJff3cPKCRwviBvqE=qw@mail.gmail.com>

2013/6/24 Maciej Fijalkowski <fijall at gmail.com>:
> On Sun, Jun 23, 2013 at 6:14 PM, R. David Murray <rdmurray at bitdance.com> wrote:
>> On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
>>> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
>>> > <python-checkins at python.org> wrote:
>>> >>  .TP
>>> >> +.BI "\-X " option
>>> >> +Set implementation specific option.
>>> >
>>> >
>>> > Should probably be "Set the implementation-specific option."
>>>
>>> Is there anyone respecting this notation? (I know pypy does not, it
>>> uses --jit and stuff)
>>
>> CPython does.  We introduced it for ourselves, it is up to other
>> implementations whether or not to use it, or use something else.
>>
>> --David
>
> you mean "CPython does not have any implementation-specific options"?
> I would claim -O behavior should be implementation-specific since it's
> nonsense in the optimizations sense, but other than that, it does not
> seem that there is any -X options?

I wouldn't object to making that -Xno-docstrings or such, but the ship
sailed long ago on -O.


--
Regards,
Benjamin

From solipsis at pitrou.net  Mon Jun 24 22:56:29 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 24 Jun 2013 22:56:29 +0200
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
	<20130624204316.028CB2500E3@webabinitio.net>
Message-ID: <20130624225629.72d9cf05@fsol>

On Mon, 24 Jun 2013 16:43:15 -0400
"R. David Murray" <rdmurray at bitdance.com> wrote:
> On Mon, 24 Jun 2013 22:14:46 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
> > On Sun, Jun 23, 2013 at 6:14 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> > > On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
> > >> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
> > >> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
> > >> > <python-checkins at python.org> wrote:
> > >> >>  .TP
> > >> >> +.BI "\-X " option
> > >> >> +Set implementation specific option.
> > >> >
> > >> >
> > >> > Should probably be "Set the implementation-specific option."
> > >>
> > >> Is there anyone respecting this notation? (I know pypy does not, it
> > >> uses --jit and stuff)
> > >
> > > CPython does.  We introduced it for ourselves, it is up to other
> > > implementations whether or not to use it, or use something else.
> > >
> > > --David
> > 
> > you mean "CPython does not have any implementation-specific options"?
> > I would claim -O behavior should be implementation-specific since it's
> > nonsense in the optimizations sense, but other than that, it does not
> > seem that there is any -X options?
> 
> There is one.  -X faulthandler.  I'm sure others would agree about
> -O, but that long predates -X.
> 
> So, the idea is that -X *can* be used by other implementations for their
> own purposes, but there is certainly no requirement that they do so.
> Our promise is that anything CPython uses it for is something we don't
> expect other implementations to support.

Yes, basically -X is a private namespace for every implementation to
use as it sees fit without fearing of conflicting with a future
cross-implementation option.

Regards

Antoine.



From python-checkins at python.org  Mon Jun 24 23:05:20 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:05:20 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogSXNzdWUgIzk1NjY6IF9pbzog?=
 =?utf-8?q?Use_Py=5FSAFE=5FDOWNCAST_for_fix_a_compiler_warning_on_Windows_?=
 =?utf-8?q?x64?=
Message-ID: <3bfNLm755Rz7Lpr@mail.python.org>

http://hg.python.org/cpython/rev/5c407b64920c
changeset:   84319:5c407b64920c
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:01:33 2013 +0200
summary:
  Issue #9566: _io: Use Py_SAFE_DOWNCAST for fix a compiler warning on Windows x64

files:
  Modules/_io/textio.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c
--- a/Modules/_io/textio.c
+++ b/Modules/_io/textio.c
@@ -2347,7 +2347,7 @@
 
     /* Note our initial start point. */
     cookie.start_pos += skip_bytes;
-    cookie.chars_to_skip = chars_to_skip;
+    cookie.chars_to_skip = Py_SAFE_DOWNCAST(chars_to_skip, Py_ssize_t, int);
     if (chars_to_skip == 0)
         goto finally;
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:05:22 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:05:22 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_zlib=3A_E?=
 =?utf-8?q?xplicit_cast_to_unsigned_int_to_fix_a_compiler_warning_on?=
Message-ID: <3bfNLp1mY6z7LpG@mail.python.org>

http://hg.python.org/cpython/rev/931e1bc090f6
changeset:   84320:931e1bc090f6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:02:51 2013 +0200
summary:
  Issue #9566: zlib: Explicit cast to unsigned int to fix a compiler warning on Windows x64

files:
  Modules/zlibmodule.c |  8 ++++----
  1 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c
--- a/Modules/zlibmodule.c
+++ b/Modules/zlibmodule.c
@@ -1094,10 +1094,10 @@
             buf += (size_t) UINT_MAX;
             len -= (size_t) UINT_MAX;
         }
-        adler32val = adler32(adler32val, buf, len);
+        adler32val = adler32(adler32val, buf, (unsigned int)len);
         Py_END_ALLOW_THREADS
     } else {
-        adler32val = adler32(adler32val, pbuf.buf, pbuf.len);
+        adler32val = adler32(adler32val, pbuf.buf, (unsigned int)pbuf.len);
     }
     PyBuffer_Release(&pbuf);
     return PyLong_FromUnsignedLong(adler32val & 0xffffffffU);
@@ -1132,10 +1132,10 @@
             buf += (size_t) UINT_MAX;
             len -= (size_t) UINT_MAX;
         }
-        signed_val = crc32(crc32val, buf, len);
+        signed_val = crc32(crc32val, buf, (unsigned int)len);
         Py_END_ALLOW_THREADS
     } else {
-        signed_val = crc32(crc32val, pbuf.buf, pbuf.len);
+        signed_val = crc32(crc32val, pbuf.buf, (unsigned int)pbuf.len);
     }
     PyBuffer_Release(&pbuf);
     return PyLong_FromUnsignedLong(signed_val & 0xffffffffU);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:15:09 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:15:09 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_=5Fwinapi?=
 =?utf-8?q?=2EWriteFile=28=29_now_truncates_length_to_DWORD=5FMAX_=2842949?=
 =?utf-8?q?67295=29?=
Message-ID: <3bfNZ55wfmz7Lkq@mail.python.org>

http://hg.python.org/cpython/rev/c75ab7b802df
changeset:   84321:c75ab7b802df
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:13:24 2013 +0200
summary:
  Issue #9566: _winapi.WriteFile() now truncates length to DWORD_MAX (4294967295)

files:
  Modules/_winapi.c |  7 +++++--
  1 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/Modules/_winapi.c b/Modules/_winapi.c
--- a/Modules/_winapi.c
+++ b/Modules/_winapi.c
@@ -62,6 +62,8 @@
 
 #define T_HANDLE T_POINTER
 
+#define DWORD_MAX 4294967295U
+
 /* Grab CancelIoEx dynamically from kernel32 */
 static int has_CancelIoEx = -1;
 static BOOL (CALLBACK *Py_CancelIoEx)(HANDLE, LPOVERLAPPED);
@@ -1142,7 +1144,7 @@
     HANDLE handle;
     Py_buffer _buf, *buf;
     PyObject *bufobj;
-    DWORD written;
+    DWORD len, written;
     BOOL ret;
     int use_overlapped = 0;
     DWORD err;
@@ -1170,7 +1172,8 @@
     }
 
     Py_BEGIN_ALLOW_THREADS
-    ret = WriteFile(handle, buf->buf, buf->len, &written,
+    len = (DWORD)Py_MIN(buf->len, DWORD_MAX);
+    ret = WriteFile(handle, buf->buf, len, &written,
                     overlapped ? &overlapped->overlapped : NULL);
     Py_END_ALLOW_THREADS
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:24:12 2013
From: python-checkins at python.org (ned.deily)
Date: Mon, 24 Jun 2013 23:24:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTY0?=
 =?utf-8?q?=3A_Backport_the_more_detailed_embedding_compile-and-link_secti?=
 =?utf-8?q?on?=
Message-ID: <3bfNmX0TRhz7LpW@mail.python.org>

http://hg.python.org/cpython/rev/e18b92bae4d6
changeset:   84322:e18b92bae4d6
branch:      2.7
parent:      84316:81a96042cff4
user:        Ned Deily <nad at acm.org>
date:        Mon Jun 24 14:21:43 2013 -0700
summary:
  Issue #18164: Backport the more detailed embedding compile-and-link section
from the Python 3 documentation.

files:
  Doc/extending/embedding.rst |  68 +++++++++++++++---------
  1 files changed, 43 insertions(+), 25 deletions(-)


diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst
--- a/Doc/extending/embedding.rst
+++ b/Doc/extending/embedding.rst
@@ -258,37 +258,55 @@
 
 .. _link-reqs:
 
-Linking Requirements
-====================
+Compiling and Linking under Unix-like systems
+=============================================
 
-While the :program:`configure` script shipped with the Python sources will
-correctly build Python to export the symbols needed by dynamically linked
-extensions, this is not automatically inherited by applications which embed the
-Python library statically, at least on Unix.  This is an issue when the
-application is linked to the static runtime library (:file:`libpython.a`) and
-needs to load dynamic extensions (implemented as :file:`.so` files).
+It is not necessarily trivial to find the right flags to pass to your
+compiler (and linker) in order to embed the Python interpreter into your
+application, particularly because Python needs to load library modules
+implemented as C dynamic extensions (:file:`.so` files) linked against
+it.
 
-The problem is that some entry points are defined by the Python runtime solely
-for extension modules to use.  If the embedding application does not use any of
-these entry points, some linkers will not include those entries in the symbol
-table of the finished executable.  Some additional options are needed to inform
-the linker not to remove these symbols.
+To find out the required compiler and linker flags, you can execute the
+:file:`python{X.Y}-config` script which is generated as part of the
+installation process (a :file:`python-config` script may also be
+available).  This script has several options, of which the following will
+be directly useful to you:
 
-Determining the right options to use for any given platform can be quite
-difficult, but fortunately the Python configuration already has those values.
-To retrieve them from an installed Python interpreter, start an interactive
-interpreter and have a short session like this
+* ``pythonX.Y-config --cflags`` will give you the recommended flags when
+  compiling::
+
+   $ /opt/bin/python2.7-config --cflags
+   -I/opt/include/python2.7 -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
+
+* ``pythonX.Y-config --ldflags`` will give you the recommended flags when
+  linking::
+
+   $ /opt/bin/python2.7-config --ldflags
+   -L/opt/lib/python2.7/config -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic
+
+.. note::
+   To avoid confusion between several Python installations (and especially
+   between the system Python and your own compiled Python), it is recommended
+   that you use the absolute path to :file:`python{X.Y}-config`, as in the above
+   example.
+
+If this procedure doesn't work for you (it is not guaranteed to work for
+all Unix-like platforms; however, we welcome :ref:`bug reports <reporting-bugs>`)
+you will have to read your system's documentation about dynamic linking and/or
+examine Python's :file:`Makefile` (use :func:`sysconfig.get_makefile_filename`
+to find its location) and compilation
+options.  In this case, the :mod:`sysconfig` module is a useful tool to
+programmatically extract the configuration values that you will want to
+combine together.  For example:
 
 .. code-block:: python
 
-   >>> import distutils.sysconfig
-   >>> distutils.sysconfig.get_config_var('LINKFORSHARED')
+   >>> import sysconfig
+   >>> sysconfig.get_config_var('LIBS')
+   '-lpthread -ldl  -lutil'
+   >>> sysconfig.get_config_var('LINKFORSHARED')
    '-Xlinker -export-dynamic'
 
-.. index:: module: distutils.sysconfig
 
-The contents of the string presented will be the options that should be used.
-If the string is empty, there's no need to add any additional options.  The
-:const:`LINKFORSHARED` definition corresponds to the variable of the same name
-in Python's top-level :file:`Makefile`.
-
+.. XXX similar documentation for Windows missing

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:24:13 2013
From: python-checkins at python.org (ned.deily)
Date: Mon, 24 Jun 2013 23:24:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTY0?=
 =?utf-8?q?=3A_Clarify_the_embedding_docs_regarding_link_options=2E?=
Message-ID: <3bfNmY2dcfz7LlN@mail.python.org>

http://hg.python.org/cpython/rev/4a114b0db866
changeset:   84323:4a114b0db866
branch:      3.3
parent:      84317:6e2f02b59fcf
user:        Ned Deily <nad at acm.org>
date:        Mon Jun 24 14:22:09 2013 -0700
summary:
  Issue #18164: Clarify the embedding docs regarding link options.

files:
  Doc/extending/embedding.rst |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst
--- a/Doc/extending/embedding.rst
+++ b/Doc/extending/embedding.rst
@@ -307,11 +307,13 @@
 to find its location) and compilation
 options.  In this case, the :mod:`sysconfig` module is a useful tool to
 programmatically extract the configuration values that you will want to
-combine together:
+combine together.  For example:
 
 .. code-block:: python
 
    >>> import sysconfig
+   >>> sysconfig.get_config_var('LIBS')
+   '-lpthread -ldl  -lutil'
    >>> sysconfig.get_config_var('LINKFORSHARED')
    '-Xlinker -export-dynamic'
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:24:14 2013
From: python-checkins at python.org (ned.deily)
Date: Mon, 24 Jun 2013 23:24:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318164=3A_merge_from_3=2E3?=
Message-ID: <3bfNmZ5HvPz7Lq6@mail.python.org>

http://hg.python.org/cpython/rev/262689e0fa2a
changeset:   84324:262689e0fa2a
parent:      84321:c75ab7b802df
parent:      84323:4a114b0db866
user:        Ned Deily <nad at acm.org>
date:        Mon Jun 24 14:23:35 2013 -0700
summary:
  Issue #18164: merge from 3.3

files:
  Doc/extending/embedding.rst |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst
--- a/Doc/extending/embedding.rst
+++ b/Doc/extending/embedding.rst
@@ -307,11 +307,13 @@
 to find its location) and compilation
 options.  In this case, the :mod:`sysconfig` module is a useful tool to
 programmatically extract the configuration values that you will want to
-combine together:
+combine together.  For example:
 
 .. code-block:: python
 
    >>> import sysconfig
+   >>> sysconfig.get_config_var('LIBS')
+   '-lpthread -ldl  -lutil'
    >>> sysconfig.get_config_var('LINKFORSHARED')
    '-Xlinker -export-dynamic'
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:33:40 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:33:40 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_a_com?=
 =?utf-8?q?piler_warning_in_tupleiter=5Fsetstate=28=29_on_Windows_x64?=
Message-ID: <3bfNzS6mryz7Lpx@mail.python.org>

http://hg.python.org/cpython/rev/6b4d279508a3
changeset:   84325:6b4d279508a3
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:31:48 2013 +0200
summary:
  Issue #9566: Fix a compiler warning in tupleiter_setstate() on Windows x64

files:
  Objects/tupleobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c
--- a/Objects/tupleobject.c
+++ b/Objects/tupleobject.c
@@ -997,7 +997,7 @@
 static PyObject *
 tupleiter_setstate(tupleiterobject *it, PyObject *state)
 {
-    long index = PyLong_AsLong(state);
+    Py_ssize_t index = PyLong_AsLong(state);
     if (index == -1 && PyErr_Occurred())
         return NULL;
     if (it->it_seq != NULL) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:34:37 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:34:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_Fix_a_com?=
 =?utf-8?q?piler_warning_on_Windows_x64?=
Message-ID: <3bfP0Y3MKJz7LlN@mail.python.org>

http://hg.python.org/cpython/rev/72087ebf83f0
changeset:   84326:72087ebf83f0
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:34:15 2013 +0200
summary:
  Issue #9566: Fix a compiler warning on Windows x64

files:
  Python/formatter_unicode.c |  6 ++----
  1 files changed, 2 insertions(+), 4 deletions(-)


diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c
--- a/Python/formatter_unicode.c
+++ b/Python/formatter_unicode.c
@@ -982,8 +982,7 @@
     Py_ssize_t n_total;
     int has_decimal;
     double val;
-    Py_ssize_t precision;
-    Py_ssize_t default_precision = 6;
+    int precision, default_precision = 6;
     Py_UCS4 type = format->type;
     int add_pct = 0;
     Py_ssize_t index;
@@ -1138,8 +1137,7 @@
     Py_ssize_t n_im_total;
     int re_has_decimal;
     int im_has_decimal;
-    int precision;
-    Py_ssize_t default_precision = 6;
+    int precision, default_precision = 6;
     Py_UCS4 type = format->type;
     Py_ssize_t i_re;
     Py_ssize_t i_im;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:37:53 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:37:53 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_pystrtod?=
 =?utf-8?q?=2Ec=3A_Fix_a_compiler_warnings_on_Windows_x64?=
Message-ID: <3bfP4K5VTyz7LlN@mail.python.org>

http://hg.python.org/cpython/rev/5a72adc7c8f7
changeset:   84327:5a72adc7c8f7
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:37:40 2013 +0200
summary:
  Issue #9566: pystrtod.c: Fix a compiler warnings on Windows x64

files:
  Python/pystrtod.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Python/pystrtod.c b/Python/pystrtod.c
--- a/Python/pystrtod.c
+++ b/Python/pystrtod.c
@@ -923,7 +923,7 @@
 
 static char *
 format_float_short(double d, char format_code,
-                   int mode, Py_ssize_t precision,
+                   int mode, int precision,
                    int always_add_sign, int add_dot_0_if_integer,
                    int use_alt_formatting, char **float_strings, int *type)
 {
@@ -1059,7 +1059,7 @@
     /* if using an exponent, reset decimal point position to 1 and adjust
        exponent accordingly.*/
     if (use_exp) {
-        exp = decpt - 1;
+        exp = (int)decpt - 1;
         decpt = 1;
     }
     /* ensure vdigits_start < decpt <= vdigits_end, or vdigits_start <

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:48:01 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:48:01 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_recv=28?=
 =?utf-8?b?KSwgcmVjdmZyb20oKSwgc2VuZCgpLCBzZW5kYWxsKCkgYW5kIHNlbmR0bygp?=
 =?utf-8?q?_methods?=
Message-ID: <3bfPJ14MyCz7LqX@mail.python.org>

http://hg.python.org/cpython/rev/c1a400501db6
changeset:   84328:c1a400501db6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:47:41 2013 +0200
summary:
  Issue #9566: recv(), recvfrom(), send(), sendall() and sendto() methods
of socket.socket objects now truncate the input buffer to INT_MAX bytes on
Windows to avoid an integer overflow.

(sendall() still send the whole buffer.)

files:
  Modules/socketmodule.c |  37 +++++++++++++++++++++++++----
  1 files changed, 32 insertions(+), 5 deletions(-)


diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -2552,8 +2552,15 @@
     BEGIN_SELECT_LOOP(s)
     Py_BEGIN_ALLOW_THREADS
     timeout = internal_select_ex(s, 0, interval);
-    if (!timeout)
+    if (!timeout) {
+#if defined(MS_WIN64) || defined(MS_WINDOWS)
+        if (len > INT_MAX)
+            len = INT_MAX;
+        outlen = recv(s->sock_fd, cbuf, (int)len, flags);
+#else
         outlen = recv(s->sock_fd, cbuf, len, flags);
+#endif
+    }
     Py_END_ALLOW_THREADS
 
     if (timeout == 1) {
@@ -2760,7 +2767,9 @@
     timeout = internal_select_ex(s, 0, interval);
     if (!timeout) {
 #ifndef MS_WINDOWS
-        n = recvfrom(s->sock_fd, cbuf, len, flags,
+        if (len > INT_MAX)
+            len = INT_MAX;
+        n = recvfrom(s->sock_fd, cbuf, (int)len, flags,
                      (void *) &addrbuf, &addrlen);
 #else
         n = recvfrom(s->sock_fd, cbuf, len, flags,
@@ -3239,12 +3248,17 @@
     BEGIN_SELECT_LOOP(s)
     Py_BEGIN_ALLOW_THREADS
     timeout = internal_select_ex(s, 1, interval);
-    if (!timeout)
+    if (!timeout) {
 #ifdef __VMS
         n = sendsegmented(s->sock_fd, buf, len, flags);
+#elif defined(MS_WIN64) || defined(MS_WINDOWS)
+        if (len > INT_MAX)
+            len = INT_MAX;
+        n = send(s->sock_fd, buf, (int)len, flags);
 #else
         n = send(s->sock_fd, buf, len, flags);
 #endif
+    }
     Py_END_ALLOW_THREADS
     if (timeout == 1) {
         PyBuffer_Release(&pbuf);
@@ -3294,6 +3308,10 @@
         if (!timeout) {
 #ifdef __VMS
             n = sendsegmented(s->sock_fd, buf, len, flags);
+#elif defined(MS_WIN64) || defined(MS_WINDOWS)
+            if (len > INT_MAX)
+                len = INT_MAX;
+            n = send(s->sock_fd, buf, (int)len, flags);
 #else
             n = send(s->sock_fd, buf, len, flags);
 #endif
@@ -3388,8 +3406,17 @@
     BEGIN_SELECT_LOOP(s)
     Py_BEGIN_ALLOW_THREADS
     timeout = internal_select_ex(s, 1, interval);
-    if (!timeout)
-        n = sendto(s->sock_fd, buf, len, flags, SAS2SA(&addrbuf), addrlen);
+    if (!timeout) {
+#if defined(MS_WIN64) || defined(MS_WINDOWS)
+        if (len > INT_MAX)
+            len = INT_MAX;
+        n = sendto(s->sock_fd, buf, (int)len, flags,
+                   SAS2SA(&addrbuf), addrlen);
+#else
+        n = sendto(s->sock_fd, buf, len, flags,
+                   SAS2SA(&addrbuf), addrlen);
+#endif
+    }
     Py_END_ALLOW_THREADS
 
     if (timeout == 1) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Mon Jun 24 23:59:39 2013
From: python-checkins at python.org (victor.stinner)
Date: Mon, 24 Jun 2013 23:59:39 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=239566=3A_More_long?=
 =?utf-8?q?/Py=5Fssize=5Ft_fixes_in_tuple_and_list_iterators_=28it=5Findex?=
 =?utf-8?q?=29?=
Message-ID: <3bfPYR29zhz7LqG@mail.python.org>

http://hg.python.org/cpython/rev/3a393fc86b29
changeset:   84329:3a393fc86b29
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Mon Jun 24 23:59:24 2013 +0200
summary:
  Issue #9566: More long/Py_ssize_t fixes in tuple and list iterators (it_index)

files:
  Objects/listobject.c  |  6 +++---
  Objects/tupleobject.c |  4 ++--
  2 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Objects/listobject.c b/Objects/listobject.c
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -2660,7 +2660,7 @@
 
 typedef struct {
     PyObject_HEAD
-    long it_index;
+    Py_ssize_t it_index;
     PyListObject *it_seq; /* Set to NULL when iterator is exhausted */
 } listiterobject;
 
@@ -2797,7 +2797,7 @@
 static PyObject *
 listiter_setstate(listiterobject *it, PyObject *state)
 {
-    long index = PyLong_AsLong(state);
+    Py_ssize_t index = PyLong_AsSsize_t(state);
     if (index == -1 && PyErr_Occurred())
         return NULL;
     if (it->it_seq != NULL) {
@@ -2958,7 +2958,7 @@
     if (forward) {
         listiterobject *it = (listiterobject *)_it;
         if (it->it_seq)
-            return Py_BuildValue("N(O)l", _PyObject_GetBuiltin("iter"),
+            return Py_BuildValue("N(O)n", _PyObject_GetBuiltin("iter"),
                                  it->it_seq, it->it_index);
     } else {
         listreviterobject *it = (listreviterobject *)_it;
diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c
--- a/Objects/tupleobject.c
+++ b/Objects/tupleobject.c
@@ -988,7 +988,7 @@
 tupleiter_reduce(tupleiterobject *it)
 {
     if (it->it_seq)
-        return Py_BuildValue("N(O)l", _PyObject_GetBuiltin("iter"),
+        return Py_BuildValue("N(O)n", _PyObject_GetBuiltin("iter"),
                              it->it_seq, it->it_index);
     else
         return Py_BuildValue("N(())", _PyObject_GetBuiltin("iter"));
@@ -997,7 +997,7 @@
 static PyObject *
 tupleiter_setstate(tupleiterobject *it, PyObject *state)
 {
-    Py_ssize_t index = PyLong_AsLong(state);
+    Py_ssize_t index = PyLong_AsSsize_t(state);
     if (index == -1 && PyErr_Occurred())
         return NULL;
     if (it->it_seq != NULL) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 00:17:52 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 00:17:52 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318081=3A_Workarou?=
 =?utf-8?q?nd_=22=2E/python_-m_test=5Fidle_test=5Flogging=22_failure?=
Message-ID: <3bfPyS4crdz7Lll@mail.python.org>

http://hg.python.org/cpython/rev/2a9e1eb3719c
changeset:   84330:2a9e1eb3719c
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 00:17:37 2013 +0200
summary:
  Issue #18081: Workaround "./python -m test_idle test_logging" failure

"import idlelib" should not install hooks on the warning modules, hooks should
only be installed when IDLE is started.

files:
  Lib/idlelib/PyShell.py |  5 +++--
  Lib/idlelib/run.py     |  4 +++-
  2 files changed, 6 insertions(+), 3 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -61,7 +61,6 @@
                                               lineno, line=line))
         except OSError:
             pass  ## file (probably __stderr__) is invalid, warning dropped.
-    warnings.showwarning = idle_showwarning
     def idle_formatwarning(message, category, filename, lineno, line=None):
         """Format warnings the IDLE way"""
         s = "\nWarning (from warnings module):\n"
@@ -73,7 +72,6 @@
             s += "    %s\n" % line
         s += "%s: %s\n>>> " % (category.__name__, message)
         return s
-    warnings.formatwarning = idle_formatwarning
 
 def extended_linecache_checkcache(filename=None,
                                   orig_checkcache=linecache.checkcache):
@@ -1425,6 +1423,9 @@
 def main():
     global flist, root, use_subprocess
 
+    warnings.showwarning = idle_showwarning
+    warnings.formatwarning = idle_formatwarning
+
     use_subprocess = True
     enable_shell = False
     enable_edit = False
diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -40,7 +40,6 @@
             s += "    %s\n" % line
         s += "%s: %s\n" % (category.__name__, message)
         return s
-    warnings.formatwarning = idle_formatwarning_subproc
 
 
 tcl = tkinter.Tcl()
@@ -82,6 +81,9 @@
     global exit_now
     global quitting
     global no_exitfunc
+
+    warnings.formatwarning = idle_formatwarning_subproc
+
     no_exitfunc = del_exitfunc
     #time.sleep(15) # test subprocess not responding
     try:

-- 
Repository URL: http://hg.python.org/cpython

From victor.stinner at gmail.com  Tue Jun 25 00:28:09 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 25 Jun 2013 00:28:09 +0200
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
In-Reply-To: <20130624204316.028CB2500E3@webabinitio.net>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
	<20130624204316.028CB2500E3@webabinitio.net>
Message-ID: <CAMpsgwZdOHkqiaTwwsahFtQzufhwNGHUuiTXOqrb1gemZL4p-Q@mail.gmail.com>

2013/6/24 R. David Murray <rdmurray at bitdance.com>:
> There is one.  -X faulthandler.  I'm sure others would agree about
> -O, but that long predates -X.

FYI I didn't chose "-X" because it is specific to CPython, but just
because it becomes really hard to choose a random letter to add a new
option... I prefer long options, but Python only supports the long
option --help. IMO "python -X faulthandler" is more explicit than
"python -@" (or "python -f", or any other random letter).

If you didn't know, you can also write "python -X faulthandler=1", it
does also enable faulthandler... As "python -X faulthandler=0" and
"python -X faulthandler=False" :-) (PHP style!)

> So, the idea is that -X *can* be used by other implementations for their
> own purposes, but there is certainly no requirement that they do so.
> Our promise is that anything CPython uses it for is something we don't
> expect other implementations to support.

It would be nice to have faulthandler or something similar (dump
traces on a crash) in other Python implementations :-) faulthandler
implementation is very specific to CPython. It uses internal
structures and low-level OS features like signal handler and a
specific stack for its signal handler. It is better to have it
integrated directly into CPython. (But it is also available as a third
party module for older versions.)

Victor

From python-checkins at python.org  Tue Jun 25 00:37:35 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 00:37:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_If_MS=5FWIN64_is_defined?=
 =?utf-8?q?=2C_MS=5FWINDOWS_is_also_defined=3A_=23ifdef_can_be_simplified?=
 =?utf-8?q?=2E?=
Message-ID: <3bfQPC5MF9z7Lqh@mail.python.org>

http://hg.python.org/cpython/rev/dfc020b4b123
changeset:   84331:dfc020b4b123
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 00:37:25 2013 +0200
summary:
  If MS_WIN64 is defined, MS_WINDOWS is also defined: #ifdef can be simplified.

files:
  Include/pyport.h        |   2 +-
  Modules/_io/_iomodule.h |   2 +-
  Modules/_io/fileio.c    |  14 +++++++-------
  Modules/posixmodule.c   |   8 ++++----
  Modules/socketmodule.c  |   8 ++++----
  Objects/fileobject.c    |   2 +-
  Python/fileutils.c      |   4 ++--
  7 files changed, 20 insertions(+), 20 deletions(-)


diff --git a/Include/pyport.h b/Include/pyport.h
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -263,7 +263,7 @@
  */
 #ifdef HAVE_LONG_LONG
 #   ifndef PY_FORMAT_LONG_LONG
-#       if defined(MS_WIN64) || defined(MS_WINDOWS)
+#       ifdef MS_WINDOWS
 #           define PY_FORMAT_LONG_LONG "I64"
 #       else
 #           error "This platform's pyconfig.h needs to define PY_FORMAT_LONG_LONG"
diff --git a/Modules/_io/_iomodule.h b/Modules/_io/_iomodule.h
--- a/Modules/_io/_iomodule.h
+++ b/Modules/_io/_iomodule.h
@@ -77,7 +77,7 @@
    long with "%lld" even when both long and long long have the same
    precision. */
 
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
 
 /* Windows uses long long for offsets */
 typedef PY_LONG_LONG Py_off_t;
diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -533,7 +533,7 @@
         len = pbuf.len;
         Py_BEGIN_ALLOW_THREADS
         errno = 0;
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         if (len > INT_MAX)
             len = INT_MAX;
         n = read(self->fd, pbuf.buf, (int)len);
@@ -602,7 +602,7 @@
     if (!_PyVerify_fd(self->fd))
         return PyErr_SetFromErrno(PyExc_IOError);
 
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
     pos = _lseeki64(self->fd, 0L, SEEK_CUR);
 #else
     pos = lseek(self->fd, 0L, SEEK_CUR);
@@ -645,7 +645,7 @@
         Py_BEGIN_ALLOW_THREADS
         errno = 0;
         n = bufsize - bytes_read;
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         if (n > INT_MAX)
             n = INT_MAX;
         n = read(self->fd, PyBytes_AS_STRING(result) + bytes_read, (int)n);
@@ -706,7 +706,7 @@
         return fileio_readall(self);
     }
 
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
     if (size > INT_MAX)
         size = INT_MAX;
 #endif
@@ -718,7 +718,7 @@
     if (_PyVerify_fd(self->fd)) {
         Py_BEGIN_ALLOW_THREADS
         errno = 0;
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         n = read(self->fd, ptr, (int)size);
 #else
         n = read(self->fd, ptr, size);
@@ -766,7 +766,7 @@
         Py_BEGIN_ALLOW_THREADS
         errno = 0;
         len = pbuf.len;
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         if (len > 32767 && isatty(self->fd)) {
             /* Issue #11395: the Windows console returns an error (12: not
                enough space error) on writing into stdout if stdout mode is
@@ -839,7 +839,7 @@
 
     if (_PyVerify_fd(fd)) {
         Py_BEGIN_ALLOW_THREADS
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         res = _lseeki64(fd, pos, whence);
 #else
         res = lseek(fd, pos, whence);
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -361,7 +361,7 @@
 #undef STAT
 #undef FSTAT
 #undef STRUCT_STAT
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
 #       define STAT win32_stat
 #       define LSTAT win32_lstat
 #       define FSTAT win32_fstat
@@ -7425,7 +7425,7 @@
 posix_lseek(PyObject *self, PyObject *args)
 {
     int fd, how;
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
     PY_LONG_LONG pos, res;
 #else
     off_t pos, res;
@@ -7453,7 +7453,7 @@
     if (!_PyVerify_fd(fd))
         return posix_error();
     Py_BEGIN_ALLOW_THREADS
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
     res = _lseeki64(fd, pos, how);
 #else
     res = lseek(fd, pos, how);
@@ -7659,7 +7659,7 @@
     }
     len = pbuf.len;
     Py_BEGIN_ALLOW_THREADS
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
     if (len > INT_MAX)
         len = INT_MAX;
     size = write(fd, pbuf.buf, (int)len);
diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
--- a/Modules/socketmodule.c
+++ b/Modules/socketmodule.c
@@ -2553,7 +2553,7 @@
     Py_BEGIN_ALLOW_THREADS
     timeout = internal_select_ex(s, 0, interval);
     if (!timeout) {
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         if (len > INT_MAX)
             len = INT_MAX;
         outlen = recv(s->sock_fd, cbuf, (int)len, flags);
@@ -3251,7 +3251,7 @@
     if (!timeout) {
 #ifdef __VMS
         n = sendsegmented(s->sock_fd, buf, len, flags);
-#elif defined(MS_WIN64) || defined(MS_WINDOWS)
+#elif defined(MS_WINDOWS)
         if (len > INT_MAX)
             len = INT_MAX;
         n = send(s->sock_fd, buf, (int)len, flags);
@@ -3308,7 +3308,7 @@
         if (!timeout) {
 #ifdef __VMS
             n = sendsegmented(s->sock_fd, buf, len, flags);
-#elif defined(MS_WIN64) || defined(MS_WINDOWS)
+#elif defined(MS_WINDOWS)
             if (len > INT_MAX)
                 len = INT_MAX;
             n = send(s->sock_fd, buf, (int)len, flags);
@@ -3407,7 +3407,7 @@
     Py_BEGIN_ALLOW_THREADS
     timeout = internal_select_ex(s, 1, interval);
     if (!timeout) {
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
         if (len > INT_MAX)
             len = INT_MAX;
         n = sendto(s->sock_fd, buf, (int)len, flags,
diff --git a/Objects/fileobject.c b/Objects/fileobject.c
--- a/Objects/fileobject.c
+++ b/Objects/fileobject.c
@@ -390,7 +390,7 @@
 
     Py_BEGIN_ALLOW_THREADS
     errno = 0;
-#if defined(MS_WIN64) || defined(MS_WINDOWS)
+#ifdef MS_WINDOWS
     if (n > INT_MAX)
         n = INT_MAX;
     n = write(self->fd, c, (int)n);
diff --git a/Python/fileutils.c b/Python/fileutils.c
--- a/Python/fileutils.c
+++ b/Python/fileutils.c
@@ -16,13 +16,13 @@
 PyObject *
 _Py_device_encoding(int fd)
 {
-#if defined(MS_WINDOWS) || defined(MS_WIN64)
+#if defined(MS_WINDOWS)
     UINT cp;
 #endif
     if (!_PyVerify_fd(fd) || !isatty(fd)) {
         Py_RETURN_NONE;
     }
-#if defined(MS_WINDOWS) || defined(MS_WIN64)
+#if defined(MS_WINDOWS)
     if (fd == 0)
         cp = GetConsoleCP();
     else if (fd == 1 || fd == 2)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 00:44:03 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 00:44:03 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTM1?=
 =?utf-8?q?=3A_ssl=2ESSLSocket=2Ewrite=28=29_now_raises_an_OverflowError_i?=
 =?utf-8?q?f_the_input?=
Message-ID: <3bfQXg58qQz7LqV@mail.python.org>

http://hg.python.org/cpython/rev/bfede07268a1
changeset:   84332:bfede07268a1
branch:      3.3
parent:      84323:4a114b0db866
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 00:42:31 2013 +0200
summary:
  Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
string in longer than 2 gigabytes, and ssl.SSLContext.load_cert_chain() raises
a ValueError if the password is longer than 2 gigabytes. The ssl module does
not support partial write.

files:
  Misc/NEWS      |  7 ++++---
  Modules/_ssl.c |  9 +++++++--
  2 files changed, 11 insertions(+), 5 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -38,9 +38,10 @@
 Library
 -------
 
-- Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
-  and in ssl.SSLContext.load_cert_chain() for strings and passwords longer than
-  2 gigabytes.
+- Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
+  string in longer than 2 gigabytes, and ssl.SSLContext.load_cert_chain()
+  raises a ValueError if the password is longer than 2 gigabytes. The ssl
+  module does not support partial write.
 
 - Issue #18248: Fix libffi build on AIX.
 
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1264,6 +1264,12 @@
         return NULL;
     }
 
+    if (buf.len > INT_MAX) {
+        PyErr_Format(PyExc_OverflowError,
+                     "string longer than %d bytes", INT_MAX);
+        goto error;
+    }
+
     /* just in case the blocking state of the socket has been changed */
     nonblocking = (sock->sock_timeout >= 0.0);
     BIO_set_nbio(SSL_get_rbio(self->ssl), nonblocking);
@@ -1284,9 +1290,8 @@
         goto error;
     }
     do {
-        len = (int)Py_MIN(buf.len, INT_MAX);
         PySSL_BEGIN_ALLOW_THREADS
-        len = SSL_write(self->ssl, buf.buf, len);
+        len = SSL_write(self->ssl, buf.buf, (int)buf.len);
         err = SSL_get_error(self->ssl, len);
         PySSL_END_ALLOW_THREADS
         if (PyErr_CheckSignals()) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 00:44:05 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 00:44:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogKE1lcmdlIDMuMykgSXNzdWUgIzE4MTM1OiBzc2wuU1NMU29ja2V0Lndy?=
 =?utf-8?q?ite=28=29_now_raises_an_OverflowError_if?=
Message-ID: <3bfQXj04qpz7LqZ@mail.python.org>

http://hg.python.org/cpython/rev/12a388024d5b
changeset:   84333:12a388024d5b
parent:      84331:dfc020b4b123
parent:      84332:bfede07268a1
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 00:43:47 2013 +0200
summary:
  (Merge 3.3) Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if
the input string in longer than 2 gigabytes, and
ssl.SSLContext.load_cert_chain() raises a ValueError if the password is longer
than 2 gigabytes. The ssl module does not support partial write.

files:
  Misc/NEWS      |  8 +++++---
  Modules/_ssl.c |  9 +++++++--
  2 files changed, 12 insertions(+), 5 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -129,12 +129,14 @@
 Library
 -------
 
+
 - Issue #11390: Add -o and -f command line options to the doctest CLI to
   specify doctest options (and convert it to using argparse).
 
-- Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
-  and in ssl.SSLContext.load_cert_chain() for strings and passwords longer than
-  2 gigabytes.
+- Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
+  string in longer than 2 gigabytes, and ssl.SSLContext.load_cert_chain()
+  raises a ValueError if the password is longer than 2 gigabytes. The ssl
+  module does not support partial write.
 
 - Issue #11016: Add C implementation of the stat module as _stat.
 
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1338,6 +1338,12 @@
         return NULL;
     }
 
+    if (buf.len > INT_MAX) {
+        PyErr_Format(PyExc_OverflowError,
+                     "string longer than %d bytes", INT_MAX);
+        goto error;
+    }
+
     /* just in case the blocking state of the socket has been changed */
     nonblocking = (sock->sock_timeout >= 0.0);
     BIO_set_nbio(SSL_get_rbio(self->ssl), nonblocking);
@@ -1358,9 +1364,8 @@
         goto error;
     }
     do {
-        len = (int)Py_MIN(buf.len, INT_MAX);
         PySSL_BEGIN_ALLOW_THREADS
-        len = SSL_write(self->ssl, buf.buf, len);
+        len = SSL_write(self->ssl, buf.buf, (int)buf.len);
         err = SSL_get_error(self->ssl, len);
         PySSL_END_ALLOW_THREADS
         if (PyErr_CheckSignals()) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 00:48:28 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 00:48:28 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=5Fssl=2Ec=3A_strip_traili?=
	=?utf-8?q?ng_spaces?=
Message-ID: <3bfQdm1v7cz7LqZ@mail.python.org>

http://hg.python.org/cpython/rev/28f47a5d59b9
changeset:   84334:28f47a5d59b9
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 00:44:31 2013 +0200
summary:
  _ssl.c: strip trailing spaces

files:
  Modules/_ssl.c |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -205,7 +205,7 @@
     int npn_protocols_len;
 #endif
 #ifndef OPENSSL_NO_TLSEXT
-    PyObject *set_hostname; 
+    PyObject *set_hostname;
 #endif
 } PySSLContext;
 
@@ -1810,7 +1810,7 @@
     self->npn_protocols = NULL;
 #endif
 #ifndef OPENSSL_NO_TLSEXT
-    self->set_hostname = NULL; 
+    self->set_hostname = NULL;
 #endif
     /* Defaults */
     SSL_CTX_set_verify(self->ctx, SSL_VERIFY_NONE, NULL);
@@ -2463,7 +2463,7 @@
     if (ssl_socket == Py_None) {
         goto error;
     }
- 
+
     if (servername == NULL) {
         result = PyObject_CallFunctionObjArgs(ssl_ctx->set_hostname, ssl_socket,
                                               Py_None, ssl_ctx, NULL);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 00:48:29 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 00:48:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTM1?=
 =?utf-8?q?=3A_ssl=2ESSLSocket=2Ewrite=28=29_now_raises_an_OverflowError_i?=
 =?utf-8?q?f_the_input?=
Message-ID: <3bfQdn3vfdz7LqC@mail.python.org>

http://hg.python.org/cpython/rev/a29eaffa7d72
changeset:   84335:a29eaffa7d72
branch:      2.7
parent:      84322:e18b92bae4d6
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 00:48:02 2013 +0200
summary:
  Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
string in longer than 2 gigabytes. The ssl module does not support partial
write.

files:
  Misc/NEWS      |   5 +++--
  Modules/_ssl.c |  13 +++++++------
  2 files changed, 10 insertions(+), 8 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -24,8 +24,9 @@
 Library
 -------
 
-- Issue #18135: Fix a possible integer overflow in ssl.SSLSocket.write()
-  for strings longer than 2 gigabytes.
+- Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
+  string in longer than 2 gigabytes. The ssl module does not support partial
+  write.
 
 - Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
   when \r\n appears at end of 65535 bytes without other newlines.
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -1192,6 +1192,12 @@
     if (!PyArg_ParseTuple(args, "s*:write", &buf))
         return NULL;
 
+    if (buf.len > INT_MAX) {
+        PyErr_Format(PyExc_OverflowError,
+                     "string longer than %d bytes", INT_MAX);
+        goto error;
+    }
+
     /* just in case the blocking state of the socket has been changed */
     nonblocking = (self->Socket->sock_timeout >= 0.0);
     BIO_set_nbio(SSL_get_rbio(self->ssl), nonblocking);
@@ -1212,13 +1218,8 @@
         goto error;
     }
     do {
-        if (buf.len <= INT_MAX)
-            len = (int)buf.len;
-        else
-            len = INT_MAX;
-
         PySSL_BEGIN_ALLOW_THREADS
-        len = SSL_write(self->ssl, buf.buf, len);
+        len = SSL_write(self->ssl, buf.buf, (int)buf.len);
         err = SSL_get_error(self->ssl, len);
         PySSL_END_ALLOW_THREADS
         if (PyErr_CheckSignals()) {

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 02:34:42 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 02:34:42 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogRml4IHRpbWUuc3Ry?=
 =?utf-8?q?ftime=28=22=25Y=22=29_on_AIX=3A_raise_a_ValueError_for_year_=3E?=
 =?utf-8?q?_9999?=
Message-ID: <3bfT0L1qTJz7Lmw@mail.python.org>

http://hg.python.org/cpython/rev/5bf5c461ec00
changeset:   84336:5bf5c461ec00
branch:      3.3
parent:      84332:bfede07268a1
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 02:33:53 2013 +0200
summary:
  Fix time.strftime("%Y") on AIX: raise a ValueError for year > 9999

time.strtime("%Y") returned "2345" when formatting year 12345.

files:
  Modules/timemodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -588,7 +588,7 @@
     else if (!gettmarg(tup, &buf) || !checktm(&buf))
         return NULL;
 
-#if defined(_MSC_VER) || defined(sun)
+#if defined(_MSC_VER) || defined(sun) || defined(_AIX)
     if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
         PyErr_SetString(PyExc_ValueError,
                         "strftime() requires year in [1; 9999]");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 02:34:43 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 02:34:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?b?KTogKE1lcmdlIDMuMykgRml4IHRpbWUuc3RyZnRpbWUoIiVZIikgb24gQUlY?=
 =?utf-8?q?=3A_raise_a_ValueError_for_year_=3E_9999?=
Message-ID: <3bfT0M3f92z7LpP@mail.python.org>

http://hg.python.org/cpython/rev/b78648c6603e
changeset:   84337:b78648c6603e
parent:      84334:28f47a5d59b9
parent:      84336:5bf5c461ec00
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 02:34:13 2013 +0200
summary:
  (Merge 3.3) Fix time.strftime("%Y") on AIX: raise a ValueError for year > 9999

time.strtime("%Y") returned "2345" when formatting year 12345.

files:
  Modules/timemodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -598,7 +598,7 @@
     else if (!gettmarg(tup, &buf) || !checktm(&buf))
         return NULL;
 
-#if defined(_MSC_VER) || defined(sun)
+#if defined(_MSC_VER) || defined(sun) || defined(_AIX)
     if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
         PyErr_SetString(PyExc_ValueError,
                         "strftime() requires year in [1; 9999]");

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Jun 25 05:52:39 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Tue, 25 Jun 2013 05:52:39 +0200
Subject: [Python-checkins] Daily reference leaks (b78648c6603e): sum=3089
Message-ID: <E1UrKJ5-0001qX-Cd@ap.vmr.nerim.net>

results for b78648c6603e on branch "default"
--------------------------------------------

test_support leaked [-1, 1, 0] references, sum=0
test_support leaked [-1, 3, 2] memory blocks, sum=4
test_httplib leaked [1628, 0, 1] references, sum=1629
test_httplib leaked [1452, 2, 2] memory blocks, sum=1456


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog_7mvJT', '-x']

From python-checkins at python.org  Tue Jun 25 07:43:18 2013
From: python-checkins at python.org (raymond.hettinger)
Date: Tue, 25 Jun 2013 07:43:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_18111=3A_Add_a_defau?=
 =?utf-8?q?lt_argument_to_min=28=29_and_max=28=29?=
Message-ID: <3bfbrQ4RLbz7LrS@mail.python.org>

http://hg.python.org/cpython/rev/76196691b5d0
changeset:   84338:76196691b5d0
user:        Raymond Hettinger <python at rcn.com>
date:        Mon Jun 24 22:43:02 2013 -0700
summary:
  Issue 18111: Add a default argument to min() and max()

files:
  Doc/library/functions.rst |  34 +++++++++++++---------
  Lib/test/test_builtin.py  |  28 ++++++++++++++++++
  Misc/NEWS                 |   4 ++
  Python/bltinmodule.c      |  40 +++++++++++++++++---------
  4 files changed, 78 insertions(+), 28 deletions(-)


diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -753,19 +753,22 @@
    already arranged into argument tuples, see :func:`itertools.starmap`\.
 
 
-.. function:: max(iterable, *[, key])
+.. function:: max(iterable, *[, default, key])
               max(arg1, arg2, *args[, key])
 
    Return the largest item in an iterable or the largest of two or more
    arguments.
 
-   If one positional argument is provided, *iterable* must be a non-empty
-   iterable (such as a non-empty string, tuple or list).  The largest item
-   in the iterable is returned.  If two or more positional arguments are
-   provided, the largest of the positional arguments is returned.
+   If one positional argument is provided, it should be an :term:`iterable`.
+   The largest item in the iterable is returned.  If two or more positional
+   arguments are provided, the smallest of the positional arguments is
+   returned.
 
-   The optional keyword-only *key* argument specifies a one-argument ordering
-   function like that used for :meth:`list.sort`.
+   There are two optional keyword-only arguments. The *key* argument specifies
+   a one-argument ordering function like that used for :meth:`list.sort`. The
+   *default* argument specifies an object to return if the provided iterable is
+   empty. If the iterable is empty and *default* is not provided, a
+   :exc:`ValueError` is raised.
 
    If multiple items are maximal, the function returns the first one
    encountered.  This is consistent with other sort-stability preserving tools
@@ -781,19 +784,22 @@
    :ref:`typememoryview` for more information.
 
 
-.. function:: min(iterable, *[, key])
+.. function:: min(iterable, *[, default, key])
               min(arg1, arg2, *args[, key])
 
    Return the smallest item in an iterable or the smallest of two or more
    arguments.
 
-   If one positional argument is provided, *iterable* must be a non-empty
-   iterable (such as a non-empty string, tuple or list).  The smallest item
-   in the iterable is returned.  If two or more positional arguments are
-   provided, the smallest of the positional arguments is returned.
+   If one positional argument is provided, it should be an :term:`iterable`.
+   The smallest item in the iterable is returned.  If two or more positional
+   arguments are provided, the smallest of the positional arguments is
+   returned.
 
-   The optional keyword-only *key* argument specifies a one-argument ordering
-   function like that used for :meth:`list.sort`.
+   There are two optional keyword-only arguments. The *key* argument specifies
+   a one-argument ordering function like that used for :meth:`list.sort`. The
+   *default* argument specifies an object to return if the provided iterable is
+   empty. If the iterable is empty and *default* is not provided, a
+   :exc:`ValueError` is raised.
 
    If multiple items are minimal, the function returns the first one
    encountered.  This is consistent with other sort-stability preserving tools
diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py
--- a/Lib/test/test_builtin.py
+++ b/Lib/test/test_builtin.py
@@ -847,8 +847,19 @@
         self.assertEqual(max(1, 2.0, 3), 3)
         self.assertEqual(max(1.0, 2, 3), 3)
 
+        self.assertRaises(TypeError, max)
+        self.assertRaises(TypeError, max, 42)
+        self.assertRaises(ValueError, max, ())
+        class BadSeq:
+            def __getitem__(self, index):
+                raise ValueError
+        self.assertRaises(ValueError, max, BadSeq())
+
         for stmt in (
             "max(key=int)",                 # no args
+            "max(default=None)",
+            "max(1, 2, default=None)",      # require container for default
+            "max(default=None, key=int)",
             "max(1, key=int)",              # single arg not iterable
             "max(1, 2, keystone=int)",      # wrong keyword
             "max(1, 2, key=int, abc=int)",  # two many keywords
@@ -865,6 +876,13 @@
         self.assertEqual(max((1,2), key=neg), 1)    # two elem iterable
         self.assertEqual(max(1, 2, key=neg), 1)     # two elems
 
+        self.assertEqual(max((), default=None), None)    # zero elem iterable
+        self.assertEqual(max((1,), default=None), 1)     # one elem iterable
+        self.assertEqual(max((1,2), default=None), 2)    # two elem iterable
+
+        self.assertEqual(max((), default=1, key=neg), 1)
+        self.assertEqual(max((1, 2), default=3, key=neg), 1)
+
         data = [random.randrange(200) for i in range(100)]
         keys = dict((elem, random.randrange(50)) for elem in data)
         f = keys.__getitem__
@@ -891,6 +909,9 @@
 
         for stmt in (
             "min(key=int)",                 # no args
+            "min(default=None)",
+            "min(1, 2, default=None)",      # require container for default
+            "min(default=None, key=int)",
             "min(1, key=int)",              # single arg not iterable
             "min(1, 2, keystone=int)",      # wrong keyword
             "min(1, 2, key=int, abc=int)",  # two many keywords
@@ -907,6 +928,13 @@
         self.assertEqual(min((1,2), key=neg), 2)    # two elem iterable
         self.assertEqual(min(1, 2, key=neg), 2)     # two elems
 
+        self.assertEqual(min((), default=None), None)    # zero elem iterable
+        self.assertEqual(min((1,), default=None), 1)     # one elem iterable
+        self.assertEqual(min((1,2), default=None), 1)    # two elem iterable
+
+        self.assertEqual(min((), default=1, key=neg), 1)
+        self.assertEqual(min((1, 2), default=1, key=neg), 2)
+
         data = [random.randrange(200) for i in range(100)]
         keys = dict((elem, random.randrange(50)) for elem in data)
         f = keys.__getitem__
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -13,6 +13,10 @@
 - Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
   OverflowError when an argument of %c format is out of range.
 
+- Issue #18111: The min() and max() functions now support a default argument
+  to be returned instead of raising a ValueError on an empty sequence.
+  (Contributed by Julian Berman.)
+
 - Issue #18137: Detect integer overflow on precision in float.__format__()
   and complex.__format__().
 
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -1329,26 +1329,35 @@
 min_max(PyObject *args, PyObject *kwds, int op)
 {
     PyObject *v, *it, *item, *val, *maxitem, *maxval, *keyfunc=NULL;
+    PyObject *emptytuple, *defaultval = NULL;
+    static char *kwlist[] = {"key", "default", NULL};
     const char *name = op == Py_LT ? "min" : "max";
+    const int positional = PyTuple_Size(args) > 1;
+    int ret;
 
-    if (PyTuple_Size(args) > 1)
+    if (positional)
         v = args;
     else if (!PyArg_UnpackTuple(args, (char *)name, 1, 1, &v))
         return NULL;
 
-    if (kwds != NULL && PyDict_Check(kwds) && PyDict_Size(kwds)) {
-        keyfunc = PyDict_GetItemString(kwds, "key");
-        if (PyDict_Size(kwds)!=1  ||  keyfunc == NULL) {
-            PyErr_Format(PyExc_TypeError,
-                "%s() got an unexpected keyword argument", name);
-            return NULL;
-        }
-        Py_INCREF(keyfunc);
+    emptytuple = PyTuple_New(0);
+    if (emptytuple == NULL)
+        return NULL;
+    ret = PyArg_ParseTupleAndKeywords(emptytuple, kwds, "|$OO", kwlist,
+                                      &keyfunc, &defaultval);
+    Py_DECREF(emptytuple);
+    if (!ret)
+        return NULL;
+
+    if (positional && defaultval != NULL) {
+        PyErr_Format(PyExc_TypeError,
+                        "Cannot specify a default for %s() with multiple "
+                        "positional arguments", name);
+        return NULL;
     }
 
     it = PyObject_GetIter(v);
     if (it == NULL) {
-        Py_XDECREF(keyfunc);
         return NULL;
     }
 
@@ -1392,14 +1401,18 @@
     if (PyErr_Occurred())
         goto Fail_it;
     if (maxval == NULL) {
-        PyErr_Format(PyExc_ValueError,
-                     "%s() arg is an empty sequence", name);
         assert(maxitem == NULL);
+        if (defaultval != NULL) {
+            Py_INCREF(defaultval);
+            maxitem = defaultval;
+        } else {
+            PyErr_Format(PyExc_ValueError,
+                         "%s() arg is an empty sequence", name);
+        }
     }
     else
         Py_DECREF(maxval);
     Py_DECREF(it);
-    Py_XDECREF(keyfunc);
     return maxitem;
 
 Fail_it_item_and_val:
@@ -1410,7 +1423,6 @@
     Py_XDECREF(maxval);
     Py_XDECREF(maxitem);
     Py_DECREF(it);
-    Py_XDECREF(keyfunc);
     return NULL;
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From fijall at gmail.com  Tue Jun 25 11:12:23 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 25 Jun 2013 11:12:23 +0200
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
In-Reply-To: <20130624225629.72d9cf05@fsol>
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
	<20130624204316.028CB2500E3@webabinitio.net>
	<20130624225629.72d9cf05@fsol>
Message-ID: <CAK5idxTT5AGt0GasBi_MScrPkMoajUNvbHVJsPGm_=AVAnCnEQ@mail.gmail.com>

On Mon, Jun 24, 2013 at 10:56 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 24 Jun 2013 16:43:15 -0400
> "R. David Murray" <rdmurray at bitdance.com> wrote:
>> On Mon, 24 Jun 2013 22:14:46 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> > On Sun, Jun 23, 2013 at 6:14 PM, R. David Murray <rdmurray at bitdance.com> wrote:
>> > > On Sun, 23 Jun 2013 17:40:13 +0200, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> > >> On Thu, Jun 20, 2013 at 3:36 PM, Brett Cannon <brett at python.org> wrote:
>> > >> > On Wed, Jun 19, 2013 at 11:20 PM, senthil.kumaran
>> > >> > <python-checkins at python.org> wrote:
>> > >> >>  .TP
>> > >> >> +.BI "\-X " option
>> > >> >> +Set implementation specific option.
>> > >> >
>> > >> >
>> > >> > Should probably be "Set the implementation-specific option."
>> > >>
>> > >> Is there anyone respecting this notation? (I know pypy does not, it
>> > >> uses --jit and stuff)
>> > >
>> > > CPython does.  We introduced it for ourselves, it is up to other
>> > > implementations whether or not to use it, or use something else.
>> > >
>> > > --David
>> >
>> > you mean "CPython does not have any implementation-specific options"?
>> > I would claim -O behavior should be implementation-specific since it's
>> > nonsense in the optimizations sense, but other than that, it does not
>> > seem that there is any -X options?
>>
>> There is one.  -X faulthandler.  I'm sure others would agree about
>> -O, but that long predates -X.
>>
>> So, the idea is that -X *can* be used by other implementations for their
>> own purposes, but there is certainly no requirement that they do so.
>> Our promise is that anything CPython uses it for is something we don't
>> expect other implementations to support.
>
> Yes, basically -X is a private namespace for every implementation to
> use as it sees fit without fearing of conflicting with a future
> cross-implementation option.
>
> Regards
>
> Antoine.

I'm for one looking forward to the day where cpython conflicts on --jit ;-)

From python-checkins at python.org  Tue Jun 25 14:11:46 2013
From: python-checkins at python.org (r.david.murray)
Date: Tue, 25 Jun 2013 14:11:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2311390=3A_fix_test_failu?=
 =?utf-8?q?res_due_to_readline_and_windows_lineneds=2E?=
Message-ID: <3bfmSf4ZCdz7Lpv@mail.python.org>

http://hg.python.org/cpython/rev/8f22e03f5f07
changeset:   84339:8f22e03f5f07
user:        R David Murray <rdmurray at bitdance.com>
date:        Tue Jun 25 08:11:22 2013 -0400
summary:
  #11390: fix test failures due to readline and windows lineneds.

files:
  Lib/test/test_doctest.py |  54 ++++++++++++++++-----------
  1 files changed, 31 insertions(+), 23 deletions(-)


diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py
--- a/Lib/test/test_doctest.py
+++ b/Lib/test/test_doctest.py
@@ -2601,10 +2601,22 @@
 These tests test this CLI functionality.
 
 We'll use the support module's script_helpers for this, and write a test files
-to a temp dir to run the command against.
-
-First, a file with two simple tests and no errors.  We'll run both the
-unadorned doctest command, and the verbose version, and then check the output:
+to a temp dir to run the command against.  Due to a current limitation in
+script_helpers, though, we need a little utility function to turn the returned
+output into something we can doctest against:
+
+    >>> def normalize(s):
+    ...     return '\n'.join(s.decode().splitlines())
+
+Note: we also pass TERM='' to all the assert_python calls to avoid a bug
+in the readline library that is triggered in these tests because we are
+running them in a new python process.  See:
+
+  http://lists.gnu.org/archive/html/bug-readline/2013-06/msg00000.html
+
+With those preliminaries out of the way, we'll start with a file with two
+simple tests and no errors.  We'll run both the unadorned doctest command, and
+the verbose version, and then check the output:
 
     >>> from test import script_helper
     >>> with script_helper.temp_dir() as tmpdir:
@@ -2618,9 +2630,9 @@
     ...         _ = f.write('\n')
     ...         _ = f.write('And that is it.\n')
     ...     rc1, out1, err1 = script_helper.assert_python_ok(
-    ...             '-m', 'doctest', fn)
+    ...             '-m', 'doctest', fn, TERM='')
     ...     rc2, out2, err2 = script_helper.assert_python_ok(
-    ...             '-m', 'doctest', '-v', fn)
+    ...             '-m', 'doctest', '-v', fn, TERM='')
 
 With no arguments and passing tests, we should get no output:
 
@@ -2631,7 +2643,7 @@
 
     >>> rc2, err2
     (0, b'')
-    >>> print(out2.decode())
+    >>> print(normalize(out2))
     Trying:
         1 + 1
     Expecting:
@@ -2647,7 +2659,6 @@
     2 tests in 1 items.
     2 passed and 0 failed.
     Test passed.
-    <BLANKLINE>
 
 Now we'll write a couple files, one with three tests, the other a python module
 with two tests, both of the files having "errors" in the tests that can be made
@@ -2684,17 +2695,17 @@
     ...         _ = f.write('   \"\"\"\n')
     ...     import shutil
     ...     rc1, out1, err1 = script_helper.assert_python_failure(
-    ...             '-m', 'doctest', fn, fn2)
+    ...             '-m', 'doctest', fn, fn2, TERM='')
     ...     rc2, out2, err2 = script_helper.assert_python_ok(
-    ...             '-m', 'doctest', '-o', 'ELLIPSIS', fn)
+    ...             '-m', 'doctest', '-o', 'ELLIPSIS', fn, TERM='')
     ...     rc3, out3, err3 = script_helper.assert_python_ok(
     ...             '-m', 'doctest', '-o', 'ELLIPSIS',
-    ...             '-o', 'NORMALIZE_WHITESPACE', fn, fn2)
+    ...             '-o', 'NORMALIZE_WHITESPACE', fn, fn2, TERM='')
     ...     rc4, out4, err4 = script_helper.assert_python_failure(
-    ...             '-m', 'doctest', '-f', fn, fn2)
+    ...             '-m', 'doctest', '-f', fn, fn2, TERM='')
     ...     rc5, out5, err5 = script_helper.assert_python_ok(
     ...             '-m', 'doctest', '-v', '-o', 'ELLIPSIS',
-    ...             '-o', 'NORMALIZE_WHITESPACE', fn, fn2)
+    ...             '-o', 'NORMALIZE_WHITESPACE', fn, fn2, TERM='')
 
 Our first test run will show the errors from the first file (doctest stops if a
 file has errors).  Note that doctest test-run error output appears on stdout,
@@ -2702,7 +2713,7 @@
 
     >>> rc1, err1
     (1, b'')
-    >>> print(out1.decode())                # doctest: +ELLIPSIS
+    >>> print(normalize(out1))                # doctest: +ELLIPSIS
     **********************************************************************
     File "...myfile.doc", line 4, in myfile.doc
     Failed example:
@@ -2723,7 +2734,6 @@
     1 items had failures:
        2 of   3 in myfile.doc
     ***Test Failed*** 2 failures.
-    <BLANKLINE>
 
 With -o ELLIPSIS specified, the second run, against just the first file, should
 produce no errors, and with -o NORMALIZE_WHITESPACE also specified, neither
@@ -2738,7 +2748,7 @@
 
     >>> rc4, err4
     (1, b'')
-    >>> print(out4.decode())                # doctest: +ELLIPSIS
+    >>> print(normalize(out4))                # doctest: +ELLIPSIS
     **********************************************************************
     File "...myfile.doc", line 4, in myfile.doc
     Failed example:
@@ -2751,14 +2761,13 @@
     1 items had failures:
        1 of   2 in myfile.doc
     ***Test Failed*** 1 failures.
-    <BLANKLINE>
 
 The fifth test uses verbose with the two options, so we should get verbose
 success output for the tests in both files:
 
     >>> rc5, err5
     (0, b'')
-    >>> print(out5.decode())
+    >>> print(normalize(out5))
     Trying:
         1 + 1
     Expecting:
@@ -2796,17 +2805,16 @@
     2 tests in 2 items.
     2 passed and 0 failed.
     Test passed.
-    <BLANKLINE>
 
 We should also check some typical error cases.
 
 Invalid file name:
 
     >>> rc, out, err = script_helper.assert_python_failure(
-    ...         '-m', 'doctest', 'nosuchfile')
+    ...         '-m', 'doctest', 'nosuchfile', TERM='')
     >>> rc, out
     (1, b'')
-    >>> print(err.decode())                    # doctest: +ELLIPSIS
+    >>> print(normalize(err))                    # doctest: +ELLIPSIS
     Traceback (most recent call last):
       ...
     FileNotFoundError: [Errno ...] No such file or directory: 'nosuchfile'
@@ -2814,10 +2822,10 @@
 Invalid doctest option:
 
     >>> rc, out, err = script_helper.assert_python_failure(
-    ...         '-m', 'doctest', '-o', 'nosuchoption')
+    ...         '-m', 'doctest', '-o', 'nosuchoption', TERM='')
     >>> rc, out
     (2, b'')
-    >>> print(err.decode())                    # doctest: +ELLIPSIS
+    >>> print(normalize(err))                    # doctest: +ELLIPSIS
     usage...invalid...nosuchoption...
 
 """

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Tue Jun 25 18:44:38 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 25 Jun 2013 18:44:38 +0200
Subject: [Python-checkins] [Python-Dev] cpython (3.3): Add -b and -X
 options to python man page.
References: <3bbSw10tm2z7Ljf@mail.python.org>
	<CAP1=2W6jxWzd7kP6f55LsLWjZoh-_r2_+3=ZD=BGnbQJiOwB=g@mail.gmail.com>
	<CAK5idxSt053PxaOwDHm9VH5M-NVnp0QbaMU9es+jDGwRoeV1Fw@mail.gmail.com>
	<20130623161440.2587A250338@webabinitio.net>
	<CAK5idxRq8k12AYyxqKztTCgNgd8t_3N_pJHGLOwR7d3xkCuZ_g@mail.gmail.com>
	<20130624204316.028CB2500E3@webabinitio.net>
	<20130624225629.72d9cf05@fsol>
	<CAK5idxTT5AGt0GasBi_MScrPkMoajUNvbHVJsPGm_=AVAnCnEQ@mail.gmail.com>
Message-ID: <20130625184438.334ed5b8@fsol>

On Tue, 25 Jun 2013 11:12:23 +0200
Maciej Fijalkowski <fijall at gmail.com> wrote:
> >
> > Yes, basically -X is a private namespace for every implementation to
> > use as it sees fit without fearing of conflicting with a future
> > cross-implementation option.
> >
> > Regards
> >
> > Antoine.
> 
> I'm for one looking forward to the day where cpython conflicts on --jit ;-)

Don't count on it: we'd use -Xjit instead ;)

Regards

Antoine.



From python-checkins at python.org  Tue Jun 25 20:26:36 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 25 Jun 2013 20:26:36 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_reapply_f1dc30?=
	=?utf-8?q?a1be72?=
Message-ID: <3bfwn82tBBz7Ljw@mail.python.org>

http://hg.python.org/cpython/rev/3dec430055da
changeset:   84340:3dec430055da
branch:      2.7
parent:      84335:a29eaffa7d72
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Jun 25 11:25:26 2013 -0700
summary:
  reapply f1dc30a1be72

files:
  Modules/_collectionsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -47,8 +47,8 @@
 
 typedef struct BLOCK {
     struct BLOCK *leftlink;
+    PyObject *data[BLOCKLEN];
     struct BLOCK *rightlink;
-    PyObject *data[BLOCKLEN];
 } block;
 
 #define MAXFREEBLOCKS 10

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 20:26:37 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 25 Jun 2013 20:26:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_reapply_f1dc30?=
	=?utf-8?q?a1be72?=
Message-ID: <3bfwn94gG5z7Llh@mail.python.org>

http://hg.python.org/cpython/rev/ec4f3d2b4b80
changeset:   84341:ec4f3d2b4b80
branch:      2.7
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Jun 25 11:26:20 2013 -0700
summary:
  reapply f1dc30a1be72

files:
  Modules/_collectionsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -47,8 +47,8 @@
 
 typedef struct BLOCK {
     struct BLOCK *leftlink;
+    struct BLOCK *rightlink;
     PyObject *data[BLOCKLEN];
-    struct BLOCK *rightlink;
 } block;
 
 #define MAXFREEBLOCKS 10

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 20:34:56 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 25 Jun 2013 20:34:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_reapply_5accb0?=
	=?utf-8?q?ac8bfb?=
Message-ID: <3bfwym12Kcz7Llh@mail.python.org>

http://hg.python.org/cpython/rev/db4b807e9706
changeset:   84342:db4b807e9706
branch:      2.7
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Jun 25 11:34:48 2013 -0700
summary:
  reapply 5accb0ac8bfb

files:
  Lib/test/test_deque.py       |   2 +-
  Modules/_collectionsmodule.c |  18 +++++++-----------
  2 files changed, 8 insertions(+), 12 deletions(-)


diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py
--- a/Lib/test/test_deque.py
+++ b/Lib/test/test_deque.py
@@ -522,7 +522,7 @@
 
     @test_support.cpython_only
     def test_sizeof(self):
-        BLOCKLEN = 62
+        BLOCKLEN = 64
         basesize = test_support.calcobjsize('2P4PlP')
         blocksize = struct.calcsize('2P%dP' % BLOCKLEN)
         self.assertEqual(object.__sizeof__(deque()), basesize)
diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -8,12 +8,13 @@
 */
 
 /* The block length may be set to any number over 1.  Larger numbers
- * reduce the number of calls to the memory allocator but take more
- * memory.  Ideally, BLOCKLEN should be set with an eye to the
- * length of a cache line.
+ * reduce the number of calls to the memory allocator, give faster
+ * indexing and rotation, and reduce the link::data overhead ratio.
+ * Ideally, the block length should be a power-of-two for faster
+ * division/modulo computations during indexing.
  */
 
-#define BLOCKLEN 62
+#define BLOCKLEN 64
 #define CENTER ((BLOCKLEN - 1) / 2)
 
 /* A `dequeobject` is composed of a doubly-linked list of `block` nodes.
@@ -58,13 +59,8 @@
 static block *
 newblock(block *leftlink, block *rightlink, Py_ssize_t len) {
     block *b;
-    /* To prevent len from overflowing PY_SSIZE_T_MAX on 64-bit machines, we
-     * refuse to allocate new blocks if the current len is dangerously
-     * close.  There is some extra margin to prevent spurious arithmetic
-     * overflows at various places.  The following check ensures that
-     * the blocks allocated to the deque, in the worst case, can only
-     * have PY_SSIZE_T_MAX-2 entries in total.
-     */
+    /* To prevent len from overflowing PY_SSIZE_T_MAX on 32-bit machines, we
+     * refuse to allocate new blocks if the current len is nearing overflow. */
     if (len >= PY_SSIZE_T_MAX - 2*BLOCKLEN) {
         PyErr_SetString(PyExc_OverflowError,
                         "cannot add more blocks to the deque");

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 20:35:51 2013
From: python-checkins at python.org (benjamin.peterson)
Date: Tue, 25 Jun 2013 20:35:51 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_reapply_f1dc30?=
	=?utf-8?q?a1be72?=
Message-ID: <3bfwzq0DgRz7Ljw@mail.python.org>

http://hg.python.org/cpython/rev/e95161820160
changeset:   84343:e95161820160
branch:      2.7
user:        Benjamin Peterson <benjamin at python.org>
date:        Tue Jun 25 11:35:44 2013 -0700
summary:
  reapply f1dc30a1be72

files:
  Modules/_collectionsmodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c
--- a/Modules/_collectionsmodule.c
+++ b/Modules/_collectionsmodule.c
@@ -48,8 +48,8 @@
 
 typedef struct BLOCK {
     struct BLOCK *leftlink;
+    PyObject *data[BLOCKLEN];
     struct BLOCK *rightlink;
-    PyObject *data[BLOCKLEN];
 } block;
 
 #define MAXFREEBLOCKS 10

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 21:25:14 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 21:25:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317206=3A_test=2Er?=
 =?utf-8?q?egrtest_and_test=2Escript=5Fhelper_enable_faulthandler_module?=
Message-ID: <3bfy4p4psYz7Ljs@mail.python.org>

http://hg.python.org/cpython/rev/c31bec42e411
changeset:   84344:c31bec42e411
parent:      84339:8f22e03f5f07
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 21:24:36 2013 +0200
summary:
  Issue #17206: test.regrtest and test.script_helper enable faulthandler module
in subprocesses.

files:
  Lib/test/regrtest.py      |  3 ++-
  Lib/test/script_helper.py |  2 +-
  2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -701,7 +701,8 @@
         output = Queue()
         pending = MultiprocessTests(tests)
         opt_args = support.args_from_interpreter_flags()
-        base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest']
+        base_cmd = [sys.executable] + opt_args
+        base_cmd += ['-X', 'faulthandler', '-m', 'test.regrtest']
         def work():
             # A worker thread.
             try:
diff --git a/Lib/test/script_helper.py b/Lib/test/script_helper.py
--- a/Lib/test/script_helper.py
+++ b/Lib/test/script_helper.py
@@ -17,7 +17,7 @@
 
 # Executing the interpreter in a subprocess
 def _assert_python(expected_success, *args, **env_vars):
-    cmd_line = [sys.executable]
+    cmd_line = [sys.executable, '-X', 'faulthandler']
     if not env_vars:
         cmd_line.append('-E')
     # Need to preserve the original environment, for in-place testing of

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 21:54:50 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 21:54:50 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317206=3A_Fix_test?=
 =?utf-8?q?=5Fcmd=5Fline_and_test=5Ffaulthandler_for_my_previous_change?=
Message-ID: <3bfyky2r20zRqk@mail.python.org>

http://hg.python.org/cpython/rev/1a9367d5aabc
changeset:   84345:1a9367d5aabc
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 21:54:17 2013 +0200
summary:
  Issue #17206: Fix test_cmd_line and test_faulthandler for my previous change
(test.regrtest and test.script_helper enable faulthandler module in
subprocesses).

files:
  Lib/test/test_cmd_line.py     |  17 ++++++++++++-----
  Lib/test/test_faulthandler.py |   8 +++++---
  2 files changed, 17 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py
--- a/Lib/test/test_cmd_line.py
+++ b/Lib/test/test_cmd_line.py
@@ -54,12 +54,19 @@
         self.assertNotIn(b'stack overflow', err)
 
     def test_xoptions(self):
-        rc, out, err = assert_python_ok('-c', 'import sys; print(sys._xoptions)')
-        opts = eval(out.splitlines()[0])
+        def get_xoptions(*args):
+            # use subprocess module directly because test.script_helper adds
+            # "-X faulthandler" to the command line
+            args = (sys.executable, '-E') + args
+            args += ('-c', 'import sys; print(sys._xoptions)')
+            out = subprocess.check_output(args)
+            opts = eval(out.splitlines()[0])
+            return opts
+
+        opts = get_xoptions()
         self.assertEqual(opts, {})
-        rc, out, err = assert_python_ok(
-            '-Xa', '-Xb=c,d=e', '-c', 'import sys; print(sys._xoptions)')
-        opts = eval(out.splitlines()[0])
+
+        opts = get_xoptions('-Xa', '-Xb=c,d=e')
         self.assertEqual(opts, {'a': True, 'b': 'c,d=e'})
 
     def test_showrefcount(self):
diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py
--- a/Lib/test/test_faulthandler.py
+++ b/Lib/test/test_faulthandler.py
@@ -262,9 +262,11 @@
     def test_disabled_by_default(self):
         # By default, the module should be disabled
         code = "import faulthandler; print(faulthandler.is_enabled())"
-        rc, stdout, stderr = assert_python_ok("-c", code)
-        stdout = (stdout + stderr).strip()
-        self.assertEqual(stdout, b"False")
+        args = (sys.executable, '-E', '-c', code)
+        # use subprocess module directly because test.script_helper adds
+        # "-X faulthandler" to the command line
+        stdout = subprocess.check_output(args)
+        self.assertEqual(stdout.rstrip(), b"False")
 
     def test_sys_xoptions(self):
         # Test python -X faulthandler

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 21:54:51 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 21:54:51 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Fgdb=2Epy=3A_ignore_?=
 =?utf-8?q?also_=22warning=3A_Source_file_is_more_recent_than_executable?=
 =?utf-8?b?LiI=?=
Message-ID: <3bfykz4b9vz7Ljs@mail.python.org>

http://hg.python.org/cpython/rev/8aa6439502c4
changeset:   84346:8aa6439502c4
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 21:54:32 2013 +0200
summary:
  test_gdb.py: ignore also "warning: Source file is more recent than executable." pattern

files:
  Lib/test/test_gdb.py |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py
--- a/Lib/test/test_gdb.py
+++ b/Lib/test/test_gdb.py
@@ -164,6 +164,7 @@
             'linux-gate.so',
             'Do you need "set solib-search-path" or '
             '"set sysroot"?',
+            'warning: Source file is more recent than executable.',
             )
         for line in errlines:
             if not line.startswith(ignore_patterns):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 22:56:21 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 22:56:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_time=2Emktime=28=29_an?=
 =?utf-8?q?d_datetime=2Edatetime=2Etimestamp=28=29_on_AIX?=
Message-ID: <3bg05x0Gfnz7LjT@mail.python.org>

http://hg.python.org/cpython/rev/035d8fed07ad
changeset:   84347:035d8fed07ad
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 22:54:35 2013 +0200
summary:
  Fix time.mktime() and datetime.datetime.timestamp() on AIX

On AIX, the C function mktime() alwaysd sets tm_wday, even on error. So tm_wday
cannot be used as a sentinel to detect an error, we can only check if the
result is (time_t)-1.

files:
  Modules/_datetimemodule.c |  13 ++++++++++---
  Modules/timemodule.c      |  11 ++++++++++-
  2 files changed, 20 insertions(+), 4 deletions(-)


diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -4873,9 +4873,16 @@
         time.tm_wday = -1;
         time.tm_isdst = -1;
         timestamp = mktime(&time);
-        /* Return value of -1 does not necessarily mean an error, but tm_wday
-         * cannot remain set to -1 if mktime succeeded. */
-        if (timestamp == (time_t)(-1) && time.tm_wday == -1) {
+        if (timestamp == (time_t)(-1)
+#ifndef _AIX
+            /* Return value of -1 does not necessarily mean an error,
+             * but tm_wday cannot remain set to -1 if mktime succeeded. */
+            && time.tm_wday == -1
+#else
+            /* on AIX, tm_wday is always sets, even on error */
+#endif
+          )
+        {
             PyErr_SetString(PyExc_OverflowError,
                             "timestamp out of range");
             return NULL;
diff --git a/Modules/timemodule.c b/Modules/timemodule.c
--- a/Modules/timemodule.c
+++ b/Modules/timemodule.c
@@ -807,7 +807,16 @@
     tt = mktime(&buf);
     /* Return value of -1 does not necessarily mean an error, but tm_wday
      * cannot remain set to -1 if mktime succeeded. */
-    if (tt == (time_t)(-1) && buf.tm_wday == -1) {
+    if (tt == (time_t)(-1)
+#ifndef _AIX
+        /* Return value of -1 does not necessarily mean an error, but
+         * tm_wday cannot remain set to -1 if mktime succeeded. */
+        && buf.tm_wday == -1
+#else
+        /* on AIX, tm_wday is always sets, even on error */
+#endif
+       )
+    {
         PyErr_SetString(PyExc_OverflowError,
                         "mktime argument out of range");
         return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Tue Jun 25 23:14:08 2013
From: python-checkins at python.org (victor.stinner)
Date: Tue, 25 Jun 2013 23:14:08 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogRml4IG9zLmNvbmZzdHIoKTog?=
 =?utf-8?q?the_result_type_of_the_C_function_is_size=5Ft=2C_not_int?=
Message-ID: <3bg0VS0WR9z7Lsd@mail.python.org>

http://hg.python.org/cpython/rev/b0668185dc15
changeset:   84348:b0668185dc15
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Tue Jun 25 23:13:47 2013 +0200
summary:
  Fix os.confstr(): the result type of the C function is size_t, not int

files:
  Modules/posixmodule.c |  4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -9140,7 +9140,7 @@
     PyObject *result = NULL;
     int name;
     char buffer[255];
-    int len;
+    size_t len;
 
     if (!PyArg_ParseTuple(args, "O&:confstr", conv_confstr_confname, &name))
         return NULL;
@@ -9157,7 +9157,7 @@
         }
     }
 
-    if ((unsigned int)len >= sizeof(buffer)) {
+    if (len >= sizeof(buffer)) {
         char *buf = PyMem_Malloc(len);
         if (buf == NULL)
             return PyErr_NoMemory();

-- 
Repository URL: http://hg.python.org/cpython

From brett at yvrsfo.ca  Tue Jun 25 17:45:00 2013
From: brett at yvrsfo.ca (Brett Cannon)
Date: Tue, 25 Jun 2013 11:45:00 -0400
Subject: [Python-checkins] cpython: Issue #18081: Workaround "./python
 -m test_idle test_logging" failure
In-Reply-To: <3bfPyS4crdz7Lll@mail.python.org>
References: <3bfPyS4crdz7Lll@mail.python.org>
Message-ID: <CAP1=2W5q9gYUthGQJTq7wie_r-jPq3zgWoEjrqjAfo6KmWAe4w@mail.gmail.com>

This fix is still not correct as warnings.formatwarning should not be
overridden, only showwarning can be.
On Jun 24, 2013 6:18 PM, "victor.stinner" <python-checkins at python.org>
wrote:

> http://hg.python.org/cpython/rev/2a9e1eb3719c
> changeset:   84330:2a9e1eb3719c
> user:        Victor Stinner <victor.stinner at gmail.com>
> date:        Tue Jun 25 00:17:37 2013 +0200
> summary:
>   Issue #18081: Workaround "./python -m test_idle test_logging" failure
>
> "import idlelib" should not install hooks on the warning modules, hooks
> should
> only be installed when IDLE is started.
>
> files:
>   Lib/idlelib/PyShell.py |  5 +++--
>   Lib/idlelib/run.py     |  4 +++-
>   2 files changed, 6 insertions(+), 3 deletions(-)
>
>
> diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
> --- a/Lib/idlelib/PyShell.py
> +++ b/Lib/idlelib/PyShell.py
> @@ -61,7 +61,6 @@
>                                                lineno, line=line))
>          except OSError:
>              pass  ## file (probably __stderr__) is invalid, warning
> dropped.
> -    warnings.showwarning = idle_showwarning
>      def idle_formatwarning(message, category, filename, lineno,
> line=None):
>          """Format warnings the IDLE way"""
>          s = "\nWarning (from warnings module):\n"
> @@ -73,7 +72,6 @@
>              s += "    %s\n" % line
>          s += "%s: %s\n>>> " % (category.__name__, message)
>          return s
> -    warnings.formatwarning = idle_formatwarning
>
>  def extended_linecache_checkcache(filename=None,
>                                    orig_checkcache=linecache.checkcache):
> @@ -1425,6 +1423,9 @@
>  def main():
>      global flist, root, use_subprocess
>
> +    warnings.showwarning = idle_showwarning
> +    warnings.formatwarning = idle_formatwarning
> +
>      use_subprocess = True
>      enable_shell = False
>      enable_edit = False
> diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
> --- a/Lib/idlelib/run.py
> +++ b/Lib/idlelib/run.py
> @@ -40,7 +40,6 @@
>              s += "    %s\n" % line
>          s += "%s: %s\n" % (category.__name__, message)
>          return s
> -    warnings.formatwarning = idle_formatwarning_subproc
>
>
>  tcl = tkinter.Tcl()
> @@ -82,6 +81,9 @@
>      global exit_now
>      global quitting
>      global no_exitfunc
> +
> +    warnings.formatwarning = idle_formatwarning_subproc
> +
>      no_exitfunc = del_exitfunc
>      #time.sleep(15) # test subprocess not responding
>      try:
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130625/28f21240/attachment-0001.html>

From solipsis at pitrou.net  Wed Jun 26 05:46:22 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Wed, 26 Jun 2013 05:46:22 +0200
Subject: [Python-checkins] Daily reference leaks (b0668185dc15): sum=-1
Message-ID: <E1UrggY-0005YK-GG@ap.vmr.nerim.net>

results for b0668185dc15 on branch "default"
--------------------------------------------

test_support leaked [0, 0, -1] references, sum=-1


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogVcKtz1', '-x']

From python-checkins at python.org  Wed Jun 26 13:28:23 2013
From: python-checkins at python.org (nick.coghlan)
Date: Wed, 26 Jun 2013 13:28:23 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Incorporate_Vinay=27s_direct_?=
	=?utf-8?q?reference_feedback?=
Message-ID: <3bgMS74V6hz7LkT@mail.python.org>

http://hg.python.org/peps/rev/16d51a112841
changeset:   4963:16d51a112841
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Wed Jun 26 21:27:57 2013 +1000
summary:
  Incorporate Vinay's direct reference feedback

* Syntax becomes "from URL" instead of "is URL"
* Now a distinct alternative to version specifiers rather
  than a subclause of one

files:
  pep-0426.txt |   33 ++++----
  pep-0440.txt |  141 +++++++++++++++++++++-----------------
  2 files changed, 95 insertions(+), 79 deletions(-)


diff --git a/pep-0426.txt b/pep-0426.txt
--- a/pep-0426.txt
+++ b/pep-0426.txt
@@ -636,7 +636,7 @@
 mechanism (such as ``https``), include an expected hash value in the
 URL for verification purposes, or both. If an insecure transport is specified
 without any hash information, with hash information that the tool doesn't
-understand, or with a selected hash algortihm that the tool considers too
+understand, or with a selected hash algorithm that the tool considers too
 weak to trust, automated tools SHOULD at least emit a warning and MAY
 refuse to rely on the URL.
 
@@ -647,7 +647,7 @@
 ``'sha512'``.
 
 For source archive references, an expected hash value may be specified by
-including a ``<hash-algorithm>=<expected-hash>`` as part of the URL
+including a ``<hash-algorithm>=<expected-hash>`` entry as part of the URL
 fragment.
 
 For version control references, the ``VCS+protocol`` scheme SHOULD be
@@ -926,10 +926,11 @@
 Individual dependencies are typically defined as strings containing a
 distribution name (as found in the ``name`` field). The dependency name
 may be followed by an extras specifier (enclosed in square
-brackets) and by a version specification (within parentheses).
+brackets) and by a version specifier or direct reference (within
+parentheses).
 
 See `Extras (optional dependencies)`_ for details on extras and PEP 440
-for details on version specifiers.
+for details on version specifiers and direct references.
 
 The distribution names should correspond to names as found on the `Python
 Package Index`_; while these names are often the same as the module names
@@ -1054,12 +1055,12 @@
 In this field, automated tools:
 
 * MUST allow strict version matching
-* MAY allow direct reference clauses
 * MUST NOT allow more permissive version specifiers.
+* MAY allow direct references
 
 Public index servers SHOULD NOT allow the use of direct references in
-uploaded distributions. Direct references are intended as a tool for
-software integrators rather than publishers.
+uploaded distributions. Direct references are intended primarily as a
+tool for software integrators rather than publishers.
 
 Distributions that rely on direct references to platform specific binary
 archives SHOULD define appropriate constraints in their
@@ -1084,12 +1085,12 @@
 In this field, automated tools:
 
 * MUST allow strict version matching
-* MAY allow direct reference clauses
 * MUST NOT allow more permissive version specifiers.
+* MAY allow direct references
 
 Public index servers SHOULD NOT allow the use of direct references in
-uploaded distributions. Direct references are intended as a tool for
-software integrators rather than publishers.
+uploaded distributions. Direct references are intended primarily as a
+tool for software integrators rather than publishers.
 
 Distributions that rely on direct references to platform specific binary
 archives SHOULD defined appropriate constraints in their
@@ -1155,7 +1156,7 @@
 Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
-Public index servers SHOULD disallow strict version matching clauses and
+Public index servers SHOULD NOT allow strict version matching clauses or
 direct references in this field.
 
 Example::
@@ -1174,7 +1175,7 @@
 Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
-Public index servers SHOULD disallow strict version matching clauses and
+Public index servers SHOULD NOT allow strict version matching clauses or
 direct references in this field.
 
 Example::
@@ -1203,7 +1204,7 @@
 Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
-Public index servers SHOULD disallow strict version matching clauses and
+Public index servers SHOULD NOT allow strict version matching clauses or
 direct references in this field.
 
 Example::
@@ -1229,7 +1230,7 @@
 Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
-Public index servers SHOULD disallow strict version matching clauses and
+Public index servers SHOULD NOT allow strict version matching clauses or
 direct references in this field.
 
 Example::
@@ -1262,7 +1263,7 @@
 Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
-Public index servers SHOULD disallow strict version matching clauses and
+Public index servers SHOULD NOT allow strict version matching clauses or
 direct references in this field.
 
 Example::
@@ -1288,7 +1289,7 @@
 Automated tools MAY disallow strict version matching clauses and direct
 references in this field and SHOULD at least emit a warning for such clauses.
 
-Public index servers SHOULD disallow strict version matching clauses and
+Public index servers SHOULD NOT allow strict version matching clauses or
 direct references in this field.
 
 Example::
diff --git a/pep-0440.txt b/pep-0440.txt
--- a/pep-0440.txt
+++ b/pep-0440.txt
@@ -550,7 +550,6 @@
 * ``~=``: `Compatible release`_ clause
 * ``==``: `Version matching`_ clause
 * ``!=``: `Version exclusion`_ clause
-* ``is``: `Direct reference`_ clause
 * ``<=``, ``>=``: `Inclusive ordered comparison`_ clause
 * ``<``, ``>``: `Exclusive ordered comparison`_ clause
 
@@ -679,61 +678,6 @@
     != 1.1.*      # Same prefix, so 1.1.post1 does not match clause
 
 
-Direct reference
-----------------
-
-A direct reference includes the direct reference operator ``is`` and
-an explicit URL.
-
-Whether or not direct references are appropriate depends on the specific
-use case for the version specifier. Automated tools SHOULD at least issue
-warnings and MAY reject them entirely when direct references are used
-inappropriately.
-
-Depending on the use case, some appropriate targets for a direct URL
-reference are a source archive, an sdist, a wheel binary archive or a
-direct reference to a tag or specific commit in an online version control
-system. The exact URLs and targets supported will be automated tool
-specific.
-
-For example, a local source archive may be referenced directly::
-
-    pip (is file:///localbuilds/pip-1.3.1.zip)
-
-Alternatively, a prebuilt archive may also be referenced::
-
-    pip (is file:///localbuilds/pip-1.3.1-py33-none-any.whl)
-
-
-All direct URL references that do not refer to a local file URL SHOULD
-specify a secure transport mechanism (such as ``https``), include an
-expected hash value in the URL for verification purposes, or both. If an
-insecure transport is specified without any hash information, with hash
-information that the tool doesn't understand, or with a selected hash
-algortihm that the tool considers too weak to trust, automated tools
-SHOULD at least emit a warning and MAY refuse to rely on the URL.
-
-Index servers MAY place additional restrictions on direct references (such
-as requiring all references be to files hosted on the index server itself).
-
-It is RECOMMENDED that only hashes which are unconditionally provided by
-the latest version of the standard library's ``hashlib`` module be used
-for source archive hashes. At time of writing, that list consists of
-``'md5'``, ``'sha1'``, ``'sha224'``, ``'sha256'``, ``'sha384'``, and
-``'sha512'``.
-
-For source archive and wheel references, an expected hash value may be
-specified by including a ``<hash-algorithm>=<expected-hash>`` as part of
-the URL fragment.
-
-Version control references, the ``VCS+protocol`` scheme SHOULD be
-used to identify both the version control system and the secure transport.
-
-To support version control systems that do not support including commit or
-tag references directly in the URL, that information may be appended to the
-end of the URL using the ``@<tag>`` notation.
-
-
 Inclusive ordered comparison
 ----------------------------
 
@@ -815,6 +759,67 @@
   not version 3.2.0 or later.
 
 
+Direct references
+=================
+
+Some automated tools may permit the use of a direct reference as an
+alternative to a normal version specifier. A direct reference consists of
+the word ``from`` and an explicit URL.
+
+Whether or not direct references are appropriate depends on the specific
+use case for the version specifier. Automated tools SHOULD at least issue
+warnings and MAY reject them entirely when direct references are used
+inappropriately.
+
+Public index servers SHOULD NOT allow the use of direct references in
+uploaded distributions. Direct references are intended as a tool for
+software integrators rather than publishers.
+
+Depending on the use case, some appropriate targets for a direct URL
+reference may be a valid ``source_url`` entry (see PEP 426), an sdist, or
+a wheel binary archive. The exact URLs and targets supported will be tool
+dependent.
+
+For example, a local source archive may be referenced directly::
+
+    pip (from file:///localbuilds/pip-1.3.1.zip)
+
+Alternatively, a prebuilt archive may also be referenced::
+
+    pip (from file:///localbuilds/pip-1.3.1-py33-none-any.whl)
+
+All direct references that do not refer to a local file URL SHOULD
+specify a secure transport mechanism (such as ``https``), include an
+expected hash value in the URL for verification purposes, or both. If an
+insecure transport is specified without any hash information, with hash
+information that the tool doesn't understand, or with a selected hash
+algorithm that the tool considers too weak to trust, automated tools
+SHOULD at least emit a warning and MAY refuse to rely on the URL.
+
+It is RECOMMENDED that only hashes which are unconditionally provided by
+the latest version of the standard library's ``hashlib`` module be used
+for source archive hashes. At time of writing, that list consists of
+``'md5'``, ``'sha1'``, ``'sha224'``, ``'sha256'``, ``'sha384'``, and
+``'sha512'``.
+
+For source archive and wheel references, an expected hash value may be
+specified by including a ``<hash-algorithm>=<expected-hash>`` entry as
+part of the URL fragment.
+
+Version control references, the ``VCS+protocol`` scheme SHOULD be
+used to identify both the version control system and the secure transport.
+
+To support version control systems that do not support including commit or
+tag references directly in the URL, that information may be appended to the
+end of the URL using the ``@<tag>`` notation.
+
+Remote URL examples::
+
+    pip (from https://github.com/pypa/pip/archive/1.3.1.zip)
+    pip (from http://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686)
+    pip (from git+https://github.com/pypa/pip.git at 1.3.1)
+
+
 Updating the versioning specification
 =====================================
 
@@ -830,14 +835,15 @@
 
 * Moved the description of version specifiers into the versioning PEP
 
-* Aadded the "source label" concept to better handle projects that wish to
+* Added the "source label" concept to better handle projects that wish to
   use a non-compliant versioning scheme internally, especially those based
   on DVCS hashes
-  
+
+* Added the "direct reference" concept as a standard notation for direct
+  references to resources (rather than each tool needing to invents its own)
+
 * Added the "compatible release" clause
 
-* Added the "direct reference" clause
-
 * Added the trailing wildcard syntax for prefix based version matching
   and exclusion
 
@@ -1032,14 +1038,23 @@
 and the desired pre- and post-release handling semantics for ``<`` and ``>``
 ordered comparison clauses.
 
+
+Adding direct references
+------------------------
+
 Direct references are added as an "escape clause" to handle messy real
 world situations that don't map neatly to the standard distribution model.
 This includes dependencies on unpublished software for internal use, as well
 as handling the more complex compatibility issues that may arise when
 wrapping third party libraries as C extensions (this is of especial concern
-to the scientific community). Index servers are given a lot of freedom to
-disallow them, since they're intended primarily as a tool for integrators
-rather than publishers.
+to the scientific community).
+
+Index servers are deliberately given a lot of freedom to disallow direct
+references, since they're intended primarily as a tool for integrators
+rather than publishers. PyPI in particular is currently going through the
+process of *eliminating* dependencies on external references, as unreliable
+external services have the effect of slowing down installation operations,
+as well as reducing PyPI's own apparent reliability.
 
 
 References

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Wed Jun 26 17:39:07 2013
From: python-checkins at python.org (lukasz.langa)
Date: Wed, 26 Jun 2013 17:39:07 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Reject_PEP_315=2E?=
Message-ID: <3bgT1R1H5tz7LxL@mail.python.org>

http://hg.python.org/peps/rev/21deefe50c51
changeset:   4964:21deefe50c51
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Wed Jun 26 17:38:57 2013 +0200
summary:
  Reject PEP 315.

files:
  pep-0315.txt |  20 +++++++++++++-------
  1 files changed, 13 insertions(+), 7 deletions(-)


diff --git a/pep-0315.txt b/pep-0315.txt
--- a/pep-0315.txt
+++ b/pep-0315.txt
@@ -4,7 +4,7 @@
 Last-Modified: $Date$
 Author: Raymond Hettinger <python at rcn.com>
         W Isaac Carroll <icarroll at pobox.com>
-Status: Deferred
+Status: Rejected
 Type: Standards Track
 Content-Type: text/plain
 Created: 25-Apr-2003
@@ -21,18 +21,24 @@
 
 Notice
 
-    Deferred; see
+    Rejected; see
+    http://mail.python.org/pipermail/python-ideas/2013-June/021610.html
+
+    This PEP has been deferred since 2006; see
     http://mail.python.org/pipermail/python-dev/2006-February/060718.html
 
     Subsequent efforts to revive the PEP in April 2009 did not
     meet with success because no syntax emerged that could
-    compete with a while-True and an inner if-break.
+    compete with the following form:
 
-    A syntax was found for a basic do-while loop but it found
-    had little support because the condition was at the top:
+        while True:
+            <setup code>
+            if not <condition>:
+                break
+            <loop body>
 
-        do ... while <cond>:
-            <loop body>
+    Users of the language are advised to use that form when a do-while
+    loop would have been appropriate.
 
 
 Motivation

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Wed Jun 26 18:06:35 2013
From: python-checkins at python.org (r.david.murray)
Date: Wed, 26 Jun 2013 18:06:35 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2311454=3A_Reduce_email_m?=
 =?utf-8?q?odule_load_time=2C_improve_surrogate_check_efficiency=2E?=
Message-ID: <3bgTd70qnyz7Lk6@mail.python.org>

http://hg.python.org/cpython/rev/520490c4c388
changeset:   84349:520490c4c388
user:        R David Murray <rdmurray at bitdance.com>
date:        Wed Jun 26 12:06:21 2013 -0400
summary:
  #11454: Reduce email module load time, improve surrogate check efficiency.

The new _has_surrogates code was suggested by Serhiy Storchaka.  See
the issue for timings, but it is far faster than any other alternative,
and also removes the load time that we previously incurred from compiling
the complex regex this replaces.

files:
  Lib/email/utils.py |  14 ++++++++++----
  1 files changed, 10 insertions(+), 4 deletions(-)


diff --git a/Lib/email/utils.py b/Lib/email/utils.py
--- a/Lib/email/utils.py
+++ b/Lib/email/utils.py
@@ -54,10 +54,16 @@
 specialsre = re.compile(r'[][\\()<>@,:;".]')
 escapesre = re.compile(r'[\\"]')
 
-# How to figure out if we are processing strings that come from a byte
-# source with undecodable characters.
-_has_surrogates = re.compile(
-    '([^\ud800-\udbff]|\A)[\udc00-\udfff]([^\udc00-\udfff]|\Z)').search
+def _has_surrogates(s):
+    """Return True if s contains surrogate-escaped binary data."""
+    # This check is based on the fact that unless there are surrogates, utf8
+    # (Python's default encoding) can encode any string.  This is the fastest
+    # way to check for surrogates, see issue 11454 for timings.
+    try:
+        s.encode()
+        return False
+    except UnicodeEncodeError:
+        return True
 
 # How to deal with a string containing bytes before handing it to the
 # application through the 'normal' interface.

-- 
Repository URL: http://hg.python.org/cpython

From rdmurray at bitdance.com  Wed Jun 26 18:18:08 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 26 Jun 2013 12:18:08 -0400
Subject: [Python-checkins] =?utf-8?q?peps=3A_Reject_PEP_315=2E?=
In-Reply-To: <3bgT1R1H5tz7LxL@mail.python.org>
References: <3bgT1R1H5tz7LxL@mail.python.org>
Message-ID: <20130626161809.788D7250B43@webabinitio.net>

On Wed, 26 Jun 2013 17:39:07 +0200, lukasz.langa <python-checkins at python.org> wrote:
> -    Deferred; see
> +    Rejected; see
> +    http://mail.python.org/pipermail/python-ideas/2013-June/021610.html
> +
> +    This PEP has been deferred since 2006; see
>      http://mail.python.org/pipermail/python-dev/2006-February/060718.html
>  
>      Subsequent efforts to revive the PEP in April 2009 did not
>      meet with success because no syntax emerged that could
> -    compete with a while-True and an inner if-break.
> +    compete with the following form:
>  
> -    A syntax was found for a basic do-while loop but it found
> -    had little support because the condition was at the top:
> +        while True:
> +            <setup code>
> +            if not <condition>:
> +                break
> +            <loop body>
>  
> -        do ... while <cond>:
> -            <loop body>
> +    Users of the language are advised to use that form when a do-while
> +    loop would have been appropriate.

Why delete the information about what was found wanting?

--David

From lukasz at langa.pl  Wed Jun 26 19:10:03 2013
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Wed, 26 Jun 2013 19:10:03 +0200
Subject: [Python-checkins] [Python-Dev]  peps: Reject PEP 315.
In-Reply-To: <20130626161809.788D7250B43@webabinitio.net>
References: <3bgT1R1H5tz7LxL@mail.python.org>
	<20130626161809.788D7250B43@webabinitio.net>
Message-ID: <D9A73CE6-4121-4407-8DF7-A11E3643CCF4@langa.pl>

On 26 cze 2013, at 18:18, R. David Murray <rdmurray at bitdance.com> wrote:

> On Wed, 26 Jun 2013 17:39:07 +0200, lukasz.langa <python-checkins at python.org> wrote:
>> -    Deferred; see
>> +    Rejected; see
>> +    http://mail.python.org/pipermail/python-ideas/2013-June/021610.html
>> +
>> +    This PEP has been deferred since 2006; see
>>     http://mail.python.org/pipermail/python-dev/2006-February/060718.html
>> 
>>     Subsequent efforts to revive the PEP in April 2009 did not
>>     meet with success because no syntax emerged that could
>> -    compete with a while-True and an inner if-break.
>> +    compete with the following form:
>> 
>> -    A syntax was found for a basic do-while loop but it found
>> -    had little support because the condition was at the top:
>> +        while True:
>> +            <setup code>
>> +            if not <condition>:
>> +                break
>> +            <loop body>
>> 
>> -        do ... while <cond>:
>> -            <loop body>
>> +    Users of the language are advised to use that form when a do-while
>> +    loop would have been appropriate.
> 
> Why delete the information about what was found wanting?

This was a mention of an alternative possible syntax. I don't feel it was worth keeping this particular one while not discussing any other alternatives at all. If you'd rather keep this information, it would be better off to have it to an "Alternative approaches" section.

-- 
Best regards,
?ukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130626/d466c8a6/attachment.html>

From python-checkins at python.org  Wed Jun 26 21:11:54 2013
From: python-checkins at python.org (r.david.murray)
Date: Wed, 26 Jun 2013 21:11:54 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MzExOiBmaXgg?=
	=?utf-8?q?typo=2E?=
Message-ID: <3bgYky5f2Wz7LjZ@mail.python.org>

http://hg.python.org/cpython/rev/d7ae8a84f443
changeset:   84350:d7ae8a84f443
branch:      3.3
parent:      84336:5bf5c461ec00
user:        R David Murray <rdmurray at bitdance.com>
date:        Wed Jun 26 15:11:12 2013 -0400
summary:
  #18311: fix typo.

files:
  Doc/library/ssl.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -773,7 +773,7 @@
 
 .. method:: SSLContext.set_npn_protocols(protocols)
 
-   Specify which protocols the socket should avertise during the SSL/TLS
+   Specify which protocols the socket should advertise during the SSL/TLS
    handshake. It should be a list of strings, like ``['http/1.1', 'spdy/2']``,
    ordered by preference. The selection of a protocol will happen during the
    handshake, and will play out according to the `NPN draft specification

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Wed Jun 26 21:11:56 2013
From: python-checkins at python.org (r.david.murray)
Date: Wed, 26 Jun 2013 21:11:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_=2318311=3A_fix_typo=2E?=
Message-ID: <3bgYl00PVhz7Ln0@mail.python.org>

http://hg.python.org/cpython/rev/16fe29689f3f
changeset:   84351:16fe29689f3f
parent:      84349:520490c4c388
parent:      84350:d7ae8a84f443
user:        R David Murray <rdmurray at bitdance.com>
date:        Wed Jun 26 15:11:32 2013 -0400
summary:
  Merge #18311: fix typo.

files:
  Doc/library/ssl.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -885,7 +885,7 @@
 
 .. method:: SSLContext.set_npn_protocols(protocols)
 
-   Specify which protocols the socket should avertise during the SSL/TLS
+   Specify which protocols the socket should advertise during the SSL/TLS
    handshake. It should be a list of strings, like ``['http/1.1', 'spdy/2']``,
    ordered by preference. The selection of a protocol will happen during the
    handshake, and will play out according to the `NPN draft specification

-- 
Repository URL: http://hg.python.org/cpython

From ncoghlan at gmail.com  Thu Jun 27 00:51:53 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 27 Jun 2013 08:51:53 +1000
Subject: [Python-checkins] [Python-Dev]  peps: Reject PEP 315.
In-Reply-To: <D9A73CE6-4121-4407-8DF7-A11E3643CCF4@langa.pl>
References: <3bgT1R1H5tz7LxL@mail.python.org>
	<20130626161809.788D7250B43@webabinitio.net>
	<D9A73CE6-4121-4407-8DF7-A11E3643CCF4@langa.pl>
Message-ID: <CADiSq7cb+O0WymgAWozBYVPXYQt49wxp7kv4N4-n-bb6603NGg@mail.gmail.com>

On 27 Jun 2013 03:12, "?ukasz Langa" <lukasz at langa.pl> wrote:
>
> On 26 cze 2013, at 18:18, R. David Murray <rdmurray at bitdance.com> wrote:
>
>> On Wed, 26 Jun 2013 17:39:07 +0200, lukasz.langa <
python-checkins at python.org> wrote:
>>>
>>> -    Deferred; see
>>> +    Rejected; see
>>> +    http://mail.python.org/pipermail/python-ideas/2013-June/021610.html
>>> +
>>> +    This PEP has been deferred since 2006; see
>>>
http://mail.python.org/pipermail/python-dev/2006-February/060718.html
>>>
>>>     Subsequent efforts to revive the PEP in April 2009 did not
>>>     meet with success because no syntax emerged that could
>>> -    compete with a while-True and an inner if-break.
>>> +    compete with the following form:
>>>
>>> -    A syntax was found for a basic do-while loop but it found
>>> -    had little support because the condition was at the top:
>>> +        while True:
>>> +            <setup code>
>>> +            if not <condition>:
>>> +                break
>>> +            <loop body>
>>>
>>> -        do ... while <cond>:
>>> -            <loop body>
>>> +    Users of the language are advised to use that form when a do-while
>>> +    loop would have been appropriate.
>>
>>
>> Why delete the information about what was found wanting?
>
>
> This was a mention of an alternative possible syntax. I don't feel it was
worth keeping this particular one while not discussing any other
alternatives at all. If you'd rather keep this information, it would be
better off to have it to an "Alternative approaches" section.

It was relevant because it was the most acceptable alternative found, and
we *still* didn't think it was an improvement over the status quo.

We don't generally significantly edit PEPs when accepting or rejecting them.

Cheers,
Nick.

>
> --
> Best regards,
> ?ukasz Langa
>
> WWW: http://lukasz.langa.pl/
> Twitter: @llanga
> IRC: ambv on #python-dev
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130627/cf67b1fc/attachment.html>

From solipsis at pitrou.net  Thu Jun 27 05:48:07 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Thu, 27 Jun 2013 05:48:07 +0200
Subject: [Python-checkins] Daily reference leaks (16fe29689f3f): sum=0
Message-ID: <E1Us3Bn-0000St-FV@ap.vmr.nerim.net>

results for 16fe29689f3f on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflog4GpcIa', '-x']

From python-checkins at python.org  Thu Jun 27 10:48:37 2013
From: python-checkins at python.org (lukasz.langa)
Date: Thu, 27 Jun 2013 10:48:37 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Restore_comment_about_an_alte?=
 =?utf-8?q?rnative_syntax_found_in_2009?=
Message-ID: <3bgvsK6B1LzSvY@mail.python.org>

http://hg.python.org/peps/rev/883ab7d71550
changeset:   4965:883ab7d71550
user:        ?ukasz Langa <lukasz at langa.pl>
date:        Thu Jun 27 10:48:28 2013 +0200
summary:
  Restore comment about an alternative syntax found in 2009

files:
  pep-0315.txt |  11 +++++++++--
  1 files changed, 9 insertions(+), 2 deletions(-)


diff --git a/pep-0315.txt b/pep-0315.txt
--- a/pep-0315.txt
+++ b/pep-0315.txt
@@ -37,8 +37,15 @@
                 break
             <loop body>
 
-    Users of the language are advised to use that form when a do-while
-    loop would have been appropriate.
+    A syntax alternative to the one proposed in the PEP was found for
+    a basic do-while loop but it gained little support because the
+    condition was at the top:
+
+        do ... while <cond>:
+            <loop body>
+
+    Users of the language are advised to use the while-True form with
+    an inner if-break when a do-while loop would have been appropriate.
 
 
 Motivation

-- 
Repository URL: http://hg.python.org/peps

From lukasz at langa.pl  Thu Jun 27 10:49:41 2013
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Thu, 27 Jun 2013 10:49:41 +0200
Subject: [Python-checkins] [Python-Dev]  peps: Reject PEP 315.
In-Reply-To: <CADiSq7cb+O0WymgAWozBYVPXYQt49wxp7kv4N4-n-bb6603NGg@mail.gmail.com>
References: <3bgT1R1H5tz7LxL@mail.python.org>
	<20130626161809.788D7250B43@webabinitio.net>
	<D9A73CE6-4121-4407-8DF7-A11E3643CCF4@langa.pl>
	<CADiSq7cb+O0WymgAWozBYVPXYQt49wxp7kv4N4-n-bb6603NGg@mail.gmail.com>
Message-ID: <929D0643-3516-4133-AE53-C73E2C4E5C29@langa.pl>

On 27 cze 2013, at 00:51, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 27 Jun 2013 03:12, "?ukasz Langa" <lukasz at langa.pl> wrote:
> > This was a mention of an alternative possible syntax. I don't feel it was worth keeping this particular one while not discussing any other alternatives at all. If you'd rather keep this information, it would be better off to have it to an "Alternative approaches" section.
> 
> It was relevant because it was the most acceptable alternative found, and we *still* didn't think it was an improvement over the status quo.
> 
> We don't generally significantly edit PEPs when accepting or rejecting them.
> 


Comment restored, point taken. Thanks for clearing this up.


-- 
Best regards,
?ukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-checkins/attachments/20130627/5316fada/attachment.html>

From python-checkins at python.org  Thu Jun 27 12:24:21 2013
From: python-checkins at python.org (martin.v.loewis)
Date: Thu, 27 Jun 2013 12:24:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2313483=3A_Use_Virt?=
 =?utf-8?q?ualAlloc_in_obmalloc_on_Windows=2E?=
Message-ID: <3bgxzn0Q4Rz7LkF@mail.python.org>

http://hg.python.org/cpython/rev/44f455e6163d
changeset:   84352:44f455e6163d
user:        Martin v. L?wis <martin at v.loewis.de>
date:        Thu Jun 27 12:23:29 2013 +0200
summary:
  Issue #13483: Use VirtualAlloc in obmalloc on Windows.

files:
  Misc/NEWS          |   2 ++
  Objects/obmalloc.c |  14 ++++++++++++--
  2 files changed, 14 insertions(+), 2 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,8 @@
 Core and Builtins
 -----------------
 
+- Issue #13483: Use VirtualAlloc in obmalloc on Windows.
+
 - Issue #18184: PyUnicode_FromFormat() and PyUnicode_FromFormatV() now raise
   OverflowError when an argument of %c format is out of range.
 
diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c
--- a/Objects/obmalloc.c
+++ b/Objects/obmalloc.c
@@ -9,6 +9,10 @@
  #endif
 #endif
 
+#ifdef MS_WINDOWS
+#include <windows.h>
+#endif
+
 #ifdef WITH_VALGRIND
 #include <valgrind/valgrind.h>
 
@@ -598,7 +602,11 @@
     arenaobj = unused_arena_objects;
     unused_arena_objects = arenaobj->nextarena;
     assert(arenaobj->address == 0);
-#ifdef ARENAS_USE_MMAP
+#ifdef MS_WINDOWS
+    address = (void*)VirtualAlloc(NULL, ARENA_SIZE,
+                                 MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
+    err = (address == NULL);
+#elif defined(ARENAS_USE_MMAP)
     address = mmap(NULL, ARENA_SIZE, PROT_READ|PROT_WRITE,
                                    MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
     err = (address == MAP_FAILED);
@@ -1093,7 +1101,9 @@
                 unused_arena_objects = ao;
 
                 /* Free the entire arena. */
-#ifdef ARENAS_USE_MMAP
+#ifdef MS_WINDOWS
+                VirtualFree((void *)ao->address, 0, MEM_RELEASE);
+#elif defined(ARENAS_USE_MMAP)
                 munmap((void *)ao->address, ARENA_SIZE);
 #else
                 free((void *)ao->address);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 00:39:04 2013
From: python-checkins at python.org (r.david.murray)
Date: Fri, 28 Jun 2013 00:39:04 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE0MzYwOiBtYWtl?=
 =?utf-8?q?_encoders=2Eencode=5Fquopri_work=2E?=
Message-ID: <3bhGHX1rFnz7Ll7@mail.python.org>

http://hg.python.org/cpython/rev/1d5856849e64
changeset:   84353:1d5856849e64
branch:      3.3
parent:      84350:d7ae8a84f443
user:        R David Murray <rdmurray at bitdance.com>
date:        Thu Jun 27 18:37:00 2013 -0400
summary:
  #14360: make encoders.encode_quopri work.

There were no tests for the encoders module.  encode_base64 worked
because it is the default and so got tested implicitly elsewhere, and
we use encode_7or8bit internally, so that worked, too.  I previously
fixed encode_noop, so this fix means that everythign in the encoders
module now works, hopefully correctly.  Also added an explicit test
for encode_base64.

files:
  Lib/email/encoders.py             |   8 +++-
  Lib/test/test_email/test_email.py |  29 +++++++++++++++++++
  2 files changed, 35 insertions(+), 2 deletions(-)


diff --git a/Lib/email/encoders.py b/Lib/email/encoders.py
--- a/Lib/email/encoders.py
+++ b/Lib/email/encoders.py
@@ -20,7 +20,7 @@
 def _qencode(s):
     enc = _encodestring(s, quotetabs=True)
     # Must encode spaces, which quopri.encodestring() doesn't do
-    return enc.replace(' ', '=20')
+    return enc.replace(b' ', b'=20')
 
 
 def encode_base64(msg):
@@ -41,8 +41,12 @@
     Also, add an appropriate Content-Transfer-Encoding header.
     """
     orig = msg.get_payload()
+    if isinstance(orig, str):
+        # If it is a string, the model data may have binary data encoded in via
+        # surrogateescape.  Convert back to bytes so we can CTE encode it.
+        orig = orig.encode('ascii', 'surrogateescape')
     encdata = _qencode(orig)
-    msg.set_payload(encdata)
+    msg.set_payload(encdata.decode('ascii', 'surrogateescape'))
     msg['Content-Transfer-Encoding'] = 'quoted-printable'
 
 
diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py
--- a/Lib/test/test_email/test_email.py
+++ b/Lib/test/test_email/test_email.py
@@ -1474,6 +1474,35 @@
         self.assertEqual(msg.get_payload(), '\uFFFD' * len(bytesdata))
         self.assertEqual(msg2.get_payload(decode=True), bytesdata)
 
+    def test_binary_body_with_encode_quopri(self):
+        # Issue 14360.
+        bytesdata = b'\xfa\xfb\xfc\xfd\xfe\xff '
+        msg = MIMEApplication(bytesdata, _encoder=encoders.encode_quopri)
+        self.assertEqual(msg.get_payload(), '=FA=FB=FC=FD=FE=FF=20')
+        self.assertEqual(msg.get_payload(decode=True), bytesdata)
+        self.assertEqual(msg['Content-Transfer-Encoding'], 'quoted-printable')
+        s = BytesIO()
+        g = BytesGenerator(s)
+        g.flatten(msg)
+        wireform = s.getvalue()
+        msg2 = email.message_from_bytes(wireform)
+        self.assertEqual(msg.get_payload(), '=FA=FB=FC=FD=FE=FF=20')
+        self.assertEqual(msg2.get_payload(decode=True), bytesdata)
+        self.assertEqual(msg2['Content-Transfer-Encoding'], 'quoted-printable')
+
+    def test_binary_body_with_encode_base64(self):
+        bytesdata = b'\xfa\xfb\xfc\xfd\xfe\xff'
+        msg = MIMEApplication(bytesdata, _encoder=encoders.encode_base64)
+        self.assertEqual(msg.get_payload(), '+vv8/f7/\n')
+        self.assertEqual(msg.get_payload(decode=True), bytesdata)
+        s = BytesIO()
+        g = BytesGenerator(s)
+        g.flatten(msg)
+        wireform = s.getvalue()
+        msg2 = email.message_from_bytes(wireform)
+        self.assertEqual(msg.get_payload(), '+vv8/f7/\n')
+        self.assertEqual(msg2.get_payload(decode=True), bytesdata)
+
 
 # Test the basic MIMEText class
 class TestMIMEText(unittest.TestCase):

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 00:39:05 2013
From: python-checkins at python.org (r.david.murray)
Date: Fri, 28 Jun 2013 00:39:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_=2314360=3A_make_encoders=2Eencode=5Fquopri_work?=
 =?utf-8?q?=2E?=
Message-ID: <3bhGHY3ZSVz7Llx@mail.python.org>

http://hg.python.org/cpython/rev/9046ef201591
changeset:   84354:9046ef201591
parent:      84352:44f455e6163d
parent:      84353:1d5856849e64
user:        R David Murray <rdmurray at bitdance.com>
date:        Thu Jun 27 18:38:36 2013 -0400
summary:
  Merge #14360: make encoders.encode_quopri work.

files:
  Lib/email/encoders.py             |   8 +++-
  Lib/test/test_email/test_email.py |  29 +++++++++++++++++++
  2 files changed, 35 insertions(+), 2 deletions(-)


diff --git a/Lib/email/encoders.py b/Lib/email/encoders.py
--- a/Lib/email/encoders.py
+++ b/Lib/email/encoders.py
@@ -20,7 +20,7 @@
 def _qencode(s):
     enc = _encodestring(s, quotetabs=True)
     # Must encode spaces, which quopri.encodestring() doesn't do
-    return enc.replace(' ', '=20')
+    return enc.replace(b' ', b'=20')
 
 
 def encode_base64(msg):
@@ -41,8 +41,12 @@
     Also, add an appropriate Content-Transfer-Encoding header.
     """
     orig = msg.get_payload()
+    if isinstance(orig, str):
+        # If it is a string, the model data may have binary data encoded in via
+        # surrogateescape.  Convert back to bytes so we can CTE encode it.
+        orig = orig.encode('ascii', 'surrogateescape')
     encdata = _qencode(orig)
-    msg.set_payload(encdata)
+    msg.set_payload(encdata.decode('ascii', 'surrogateescape'))
     msg['Content-Transfer-Encoding'] = 'quoted-printable'
 
 
diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py
--- a/Lib/test/test_email/test_email.py
+++ b/Lib/test/test_email/test_email.py
@@ -1474,6 +1474,35 @@
         self.assertEqual(msg.get_payload(), '\uFFFD' * len(bytesdata))
         self.assertEqual(msg2.get_payload(decode=True), bytesdata)
 
+    def test_binary_body_with_encode_quopri(self):
+        # Issue 14360.
+        bytesdata = b'\xfa\xfb\xfc\xfd\xfe\xff '
+        msg = MIMEApplication(bytesdata, _encoder=encoders.encode_quopri)
+        self.assertEqual(msg.get_payload(), '=FA=FB=FC=FD=FE=FF=20')
+        self.assertEqual(msg.get_payload(decode=True), bytesdata)
+        self.assertEqual(msg['Content-Transfer-Encoding'], 'quoted-printable')
+        s = BytesIO()
+        g = BytesGenerator(s)
+        g.flatten(msg)
+        wireform = s.getvalue()
+        msg2 = email.message_from_bytes(wireform)
+        self.assertEqual(msg.get_payload(), '=FA=FB=FC=FD=FE=FF=20')
+        self.assertEqual(msg2.get_payload(decode=True), bytesdata)
+        self.assertEqual(msg2['Content-Transfer-Encoding'], 'quoted-printable')
+
+    def test_binary_body_with_encode_base64(self):
+        bytesdata = b'\xfa\xfb\xfc\xfd\xfe\xff'
+        msg = MIMEApplication(bytesdata, _encoder=encoders.encode_base64)
+        self.assertEqual(msg.get_payload(), '+vv8/f7/\n')
+        self.assertEqual(msg.get_payload(decode=True), bytesdata)
+        s = BytesIO()
+        g = BytesGenerator(s)
+        g.flatten(msg)
+        wireform = s.getvalue()
+        msg2 = email.message_from_bytes(wireform)
+        self.assertEqual(msg.get_payload(), '+vv8/f7/\n')
+        self.assertEqual(msg2.get_payload(decode=True), bytesdata)
+
 
 # Test the basic MIMEText class
 class TestMIMEText(unittest.TestCase):

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Fri Jun 28 05:47:21 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Fri, 28 Jun 2013 05:47:21 +0200
Subject: [Python-checkins] Daily reference leaks (9046ef201591): sum=2
Message-ID: <E1UsPeb-0004fM-Vp@ap.vmr.nerim.net>

results for 9046ef201591 on branch "default"
--------------------------------------------

test_unittest leaked [-1, 2, 1] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogVjJYdU', '-x']

From python-checkins at python.org  Fri Jun 28 19:26:30 2013
From: python-checkins at python.org (charles-francois.natali)
Date: Fri, 28 Jun 2013 19:26:30 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317914=3A_Use_os?=
 =?utf-8?q?=2Ecpu=5Fcount=28=29_instead_of_multiprocessing=2Ecpu=5Fcount?=
 =?utf-8?q?=28=29_where?=
Message-ID: <3bhlJQ6sVYzSR8@mail.python.org>

http://hg.python.org/cpython/rev/6a0437adafbd
changeset:   84355:6a0437adafbd
user:        Charles-Fran?ois Natali <cf.natali at gmail.com>
date:        Fri Jun 28 19:25:45 2013 +0200
summary:
  Issue #17914: Use os.cpu_count() instead of multiprocessing.cpu_count() where
applicable.

files:
  Doc/library/multiprocessing.rst   |  2 +-
  Lib/concurrent/futures/process.py |  2 +-
  Lib/multiprocessing/pool.py       |  8 +++-----
  Lib/test/regrtest.py              |  8 ++------
  4 files changed, 7 insertions(+), 13 deletions(-)


diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -1664,7 +1664,7 @@
    callbacks and has a parallel map implementation.
 
    *processes* is the number of worker processes to use.  If *processes* is
-   ``None`` then the number returned by :func:`cpu_count` is used.  If
+   ``None`` then the number returned by :func:`os.cpu_count` is used.  If
    *initializer* is not ``None`` then each worker process will call
    ``initializer(*initargs)`` when it starts.
 
diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py
--- a/Lib/concurrent/futures/process.py
+++ b/Lib/concurrent/futures/process.py
@@ -331,7 +331,7 @@
         _check_system_limits()
 
         if max_workers is None:
-            self._max_workers = multiprocessing.cpu_count()
+            self._max_workers = os.cpu_count() or 1
         else:
             self._max_workers = max_workers
 
diff --git a/Lib/multiprocessing/pool.py b/Lib/multiprocessing/pool.py
--- a/Lib/multiprocessing/pool.py
+++ b/Lib/multiprocessing/pool.py
@@ -17,10 +17,11 @@
 import queue
 import itertools
 import collections
+import os
 import time
 import traceback
 
-from multiprocessing import Process, cpu_count, TimeoutError
+from multiprocessing import Process, TimeoutError
 from multiprocessing.util import Finalize, debug
 
 #
@@ -147,10 +148,7 @@
         self._initargs = initargs
 
         if processes is None:
-            try:
-                processes = cpu_count()
-            except NotImplementedError:
-                processes = 1
+            processes = os.cpu_count() or 1
         if processes < 1:
             raise ValueError("Number of processes must be at least 1")
 
diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
--- a/Lib/test/regrtest.py
+++ b/Lib/test/regrtest.py
@@ -508,12 +508,8 @@
         elif o in ('-j', '--multiprocess'):
             use_mp = int(a)
             if use_mp <= 0:
-                try:
-                    import multiprocessing
-                    # Use all cores + extras for tests that like to sleep
-                    use_mp = 2 + multiprocessing.cpu_count()
-                except (ImportError, NotImplementedError):
-                    use_mp = 3
+                # Use all cores + extras for tests that like to sleep
+                use_mp = 2 + (os.cpu_count() or 1)
             if use_mp == 1:
                 use_mp = None
         elif o == '--header':

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 19:31:32 2013
From: python-checkins at python.org (r.david.murray)
Date: Fri, 28 Jun 2013 19:31:32 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2318111=3A_Add_What=27s_N?=
 =?utf-8?q?ew_entry_for_max/min_default=2E?=
Message-ID: <3bhlQD2rRNzSkb@mail.python.org>

http://hg.python.org/cpython/rev/34ff27b431d0
changeset:   84356:34ff27b431d0
user:        R David Murray <rdmurray at bitdance.com>
date:        Fri Jun 28 13:31:19 2013 -0400
summary:
  #18111: Add What's New entry for max/min default.

files:
  Doc/whatsnew/3.4.rst |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -140,6 +140,9 @@
 * Import now raises the new exception :exc:`ModuleNotFoundError` (subclass of
   :exc:`ImportError`) when it cannot find something.
 
+* :func:`min` and :func:`max` now accept a *default* argument that can be used
+  to specify the value they return if the iterable they are evaluating has no
+  elements.  Contributed by Julian Berman in :issue:`18111`.
 
 
 New Modules

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 20:53:16 2013
From: python-checkins at python.org (r.david.murray)
Date: Fri, 28 Jun 2013 20:53:16 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Improve_imap_error_message?=
 =?utf-8?q?_in_unusual_failure_mode=2E?=
Message-ID: <3bhnDX2PbqzRc4@mail.python.org>

http://hg.python.org/cpython/rev/d1cf33b01f11
changeset:   84357:d1cf33b01f11
user:        R David Murray <rdmurray at bitdance.com>
date:        Fri Jun 28 14:52:57 2013 -0400
summary:
  Improve imap error message in unusual failure mode.

We ran into this during the sprits at PyCon and this patch has been
sitting on my disk ever since.  This just adds some information to the
error message that we found useful during debugging.  There's no good
way to add a test, since the message only got generated via code
that we had modified for debugging purposes.

files:
  Lib/imaplib.py |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Lib/imaplib.py b/Lib/imaplib.py
--- a/Lib/imaplib.py
+++ b/Lib/imaplib.py
@@ -1073,7 +1073,7 @@
 
         # Protocol mandates all lines terminated by CRLF
         if not line.endswith(b'\r\n'):
-            raise self.abort('socket error: unterminated line')
+            raise self.abort('socket error: unterminated line: %r' % line)
 
         line = line[:-2]
         if __debug__:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 21:10:05 2013
From: python-checkins at python.org (r.david.murray)
Date: Fri, 28 Jun 2013 21:10:05 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE0MzYwOiBBZGQg?=
	=?utf-8?q?news_item=2E?=
Message-ID: <3bhnbx2vcJzNxG@mail.python.org>

http://hg.python.org/cpython/rev/7c807bc15fa8
changeset:   84358:7c807bc15fa8
branch:      3.3
parent:      84353:1d5856849e64
user:        R David Murray <rdmurray at bitdance.com>
date:        Fri Jun 28 15:08:03 2013 -0400
summary:
  #14360: Add news item.

files:
  Misc/NEWS |  3 +++
  1 files changed, 3 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -38,6 +38,9 @@
 Library
 -------
 
+- Issue #14360: encode_quopri can now be successfully used as an encoder
+  when constructing a MIMEApplication object.
+
 - Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
   string in longer than 2 gigabytes, and ssl.SSLContext.load_cert_chain()
   raises a ValueError if the password is longer than 2 gigabytes. The ssl

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 21:10:06 2013
From: python-checkins at python.org (r.david.murray)
Date: Fri, 28 Jun 2013 21:10:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_=2314360=3A_Add_news_item=2E?=
Message-ID: <3bhnby4yMmzQxh@mail.python.org>

http://hg.python.org/cpython/rev/36cc8b0446b3
changeset:   84359:36cc8b0446b3
parent:      84357:d1cf33b01f11
parent:      84358:7c807bc15fa8
user:        R David Murray <rdmurray at bitdance.com>
date:        Fri Jun 28 15:09:10 2013 -0400
summary:
  Merge #14360: Add news item.

files:
  Misc/NEWS |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -135,6 +135,8 @@
 Library
 -------
 
+- Issue #14360: encode_quopri can now be successfully used as an encoder
+  when constructing a MIMEApplication object.
 
 - Issue #11390: Add -o and -f command line options to the doctest CLI to
   specify doctest options (and convert it to using argparse).

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Fri Jun 28 22:40:31 2013
From: python-checkins at python.org (victor.stinner)
Date: Fri, 28 Jun 2013 22:40:31 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_PEP_445=3A_take_into_account_?=
	=?utf-8?q?Antoine_Pitrou=27s_remarks?=
Message-ID: <3bhqcH03RFzRrF@mail.python.org>

http://hg.python.org/peps/rev/d20763341236
changeset:   4966:d20763341236
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Fri Jun 28 22:39:29 2013 +0200
summary:
  PEP 445: take into account Antoine Pitrou's remarks

files:
  pep-0445.txt |  151 +++++++++++++++++++++++---------------
  1 files changed, 92 insertions(+), 59 deletions(-)


diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -88,6 +88,8 @@
   - ``void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)``
   - The new allocator must return a distinct non-*NULL* pointer when
     requesting zero bytes
+  - For the ``PYMEM_DOMAIN_RAW`` domain, the allocator must be
+    thread-safe: the GIL is not held when the allocator is called.
 
 * Add a new ``PyObjectArenaAllocator`` structure::
 
@@ -113,7 +115,9 @@
   a memory allocator is replaced:
 
   - ``void PyMem_SetupDebugHooks(void)``
-  - Install the debug hook on all memory block allocators.
+  - Install the debug hook on all memory block allocators. The function
+    can be called more than once, hooks are not reinstalled if they
+    were already installed.
   - The function does nothing is Python is not compiled in debug mode
 
 * Memory allocators always returns *NULL* if size is greater than
@@ -127,12 +131,13 @@
 Default allocators:
 
 * ``PYMEM_DOMAIN_RAW``, ``PYMEM_DOMAIN_MEM``: ``malloc()``,
-  ``realloc()``, ``free()`` (and *ctx* is NULL); call ``malloc(1)`` when
-  requesting zero bytes
-* ``PYMEM_DOMAIN_OBJ``: *pymalloc* allocator which fall backs on
+  ``realloc()`` and ``free()``; call ``malloc(1)`` when requesting zero
+  bytes
+* ``PYMEM_DOMAIN_OBJ``: *pymalloc* allocator which falls back on
   ``PyMem_Malloc()`` for allocations larger than 512 bytes
-* *pymalloc* arena allocator: ``mmap()``, ``munmap()`` (and *ctx* is
-  NULL), or ``malloc()`` and ``free()`` if ``mmap()`` is not available
+* *pymalloc* arena allocator: ``VirualAlloc()`` and ``VirtualFree()`` on
+  Windows, ``mmap()`` and ``munmap()`` when available, or ``malloc()``
+  and ``free()``
 
 
 Redesign Debug Checks on  Memory Allocators as Hooks
@@ -148,23 +153,30 @@
 * Detect write before the start of the buffer (buffer underflow)
 * Detect write after the end of the buffer (buffer overflow)
 
-In Python 3.3, the checks are installed by replacing
-``PYMEM_DOMAIN_MEM`` and ``PYMEM_DOMAIN_OBJ`` allocators, the previous
-allocator is no more called.  The new allocator is the same for both
-domains: ``PyMem_Malloc()`` and ``PyMem_Realloc()`` call indirectly
-``PyObject_Malloc()`` and ``PyObject_Realloc()``.
+In Python 3.3, the checks are installed by replacing ``PyMem_Malloc()``,
+``PyMem_Realloc()``, ``PyMem_Free()``, ``PyObject_Malloc()``,
+``PyObject_Realloc()`` and ``PyObject_Free()`` using macros. The new
+allocator allocates a larger buffer and write a pattern to detect buffer
+underflow and overflow. It uses the original ``PyObject_Malloc()``
+function to allocate memory. So ``PyMem_Malloc()`` and
+``PyMem_Realloc()`` call indirectly ``PyObject_Malloc()`` and
+``PyObject_Realloc()``.
 
 This PEP redesigns the debug checks as hooks on the existing allocators
 in debug mode. Examples of call traces without the hooks:
 
-* ``PyMem_Malloc()`` => ``_PyMem_RawMalloc()`` => ``malloc()``
+* ``PyMem_RawMalloc()`` => ``_PyMem_RawMalloc()`` => ``malloc()``
+* ``PyMem_Realloc()`` => ``_PyMem_RawRealloc()`` => ``realloc()``
 * ``PyObject_Free()`` => ``_PyObject_Free()``
 
 Call traces when the hooks are installed (debug mode):
 
-* ``PyMem_Malloc()`` => ``_PyMem_DebugMalloc()`` =>
-  ``_PyMem_RawMalloc()`` => ``malloc()``
-* ``PyObject_Free()`` => ``_PyMem_DebugFree()`` => ``_PyObject_Free()``
+* ``PyMem_RawMalloc()`` => ``_PyMem_DebugMalloc()``
+  => ``_PyMem_RawMalloc()`` => ``malloc()``
+* ``PyMem_Realloc()`` => ``_PyMem_DebugRealloc()``
+  => ``_PyMem_RawRealloc()`` => ``realloc()``
+* ``PyObject_Free()`` => ``_PyMem_DebugFree()``
+  => ``_PyObject_Free()``
 
 As a result, ``PyMem_Malloc()`` and ``PyMem_Realloc()`` now always call
 ``malloc()`` and ``realloc()``, instead of calling ``PyObject_Malloc()``
@@ -212,8 +224,8 @@
 
     #include <stdlib.h>
 
-    int alloc_padding = 2;
-    int arena_padding = 10;
+    size_t alloc_padding = 2;
+    size_t arena_padding = 10;
 
     void* my_malloc(void *ctx, size_t size)
     {
@@ -264,10 +276,6 @@
         PyMem_SetupDebugHooks();
     }
 
-.. warning::
-   Remove the call ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc)`` if
-   the new allocator is not thread-safe.
-
 
 Use case 2: Replace Memory Allocator, override pymalloc
 --------------------------------------------------------
@@ -280,7 +288,7 @@
 
     #include <stdlib.h>
 
-    int padding = 2;
+    size_t padding = 2;
 
     void* my_malloc(void *ctx, size_t size)
     {
@@ -314,11 +322,6 @@
         PyMem_SetupDebugHooks();
     }
 
-.. warning::
-   Remove the call ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc)`` if
-   the new allocator is not thread-safe.
-
-
 
 Use case 3: Setup Allocator Hooks
 ---------------------------------
@@ -386,10 +389,6 @@
         PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &alloc);
     }
 
-.. warning::
-   Remove the call ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc)`` if
-   hooks are not thread-safe.
-
 .. note::
    ``PyMem_SetupDebugHooks()`` does not need to be called because the
    allocator is not replaced: Python debug hooks are installed
@@ -409,8 +408,8 @@
 The full reports are attached to the issue #3329.
 
 
-Alternatives
-============
+Rejected Alternatives
+=====================
 
 More specific functions to get/set memory allocators
 ----------------------------------------------------
@@ -429,13 +428,20 @@
 * ``void PyMem_SetAllocator(PyMemAllocator *allocator)``
 * ``void PyObject_SetAllocator(PyMemAllocator *allocator)``
 
+With more specific functions, it becomes more difficult to write generic
+code, like reusing the same code for different allocator domains.
+
 
 Make PyMem_Malloc() reuse PyMem_RawMalloc() by default
 ------------------------------------------------------
 
-``PyMem_Malloc()`` should call ``PyMem_RawMalloc()`` by default. So
-calling ``PyMem_SetRawAllocator()`` would also also patch
-``PyMem_Malloc()`` indirectly.
+If ``PyMem_Malloc()`` would call ``PyMem_RawMalloc()`` by default,
+calling ``PyMem_SetAllocator(PYMEM_DOMAIN_RAW, alloc)`` would also also
+patch ``PyMem_Malloc()`` indirectly.
+
+This option was rejected because ``PyMem_SetAllocator()`` would have a
+different behaviour depending on the domain. Always having the same
+behaviour is less error-prone.
 
 
 Add a new PYDEBUGMALLOC environment variable
@@ -449,7 +455,7 @@
 ``PyMem_SetAllocator()`` and ``PyObject_SetAllocator()`` will reinstall
 automatically the hook on top of the new allocator.
 
-An new environment variable would make the Python initialization even
+A new environment variable would make the Python initialization even
 more complex. The `PEP 432 <http://www.python.org/dev/peps/pep-0432/>`_
 tries to simply the CPython startup sequence.
 
@@ -461,8 +467,10 @@
 allocators would be an optional feature enabled by a configuration
 option or by macros.
 
-Not having to recompile Python makes debug hooks easier to use in
-practice.  Extensions modules don't have to be recompiled with macros.
+This alternative was rejected because the usage of macros implies having
+to recompile extensions modules to use the new allocator and allocator
+hooks. Not having to recompile Python nor extension modules makes debug
+hooks easier to use in practice.
 
 
 Pass the C filename and line number
@@ -501,12 +509,15 @@
     #define PyMem_Malloc(size) \
             _PyMem_MallocTrace(__FILE__, __LINE__, size)
 
-Passing a filename and a line number to each allocator makes the API more
-complex: pass 3 new arguments, instead of just a context argument, to each
-allocator function. The GC allocator functions should also be patched.
-For example, ``_PyObject_GC_Malloc()`` is used in many C functions and so
-objects of differenet types would have the same allocation location. Such
-changes add too much complexity for a little gain.
+The GC allocator functions would also have to be patched. For example,
+``_PyObject_GC_Malloc()`` is used in many C functions and so objects of
+differenet types would have the same allocation location.
+
+This alternative was rejected because passing a filename and a line
+number to each allocator makes the API more complex: pass 3 new
+arguments (ctx, filename, lineno) to each allocator function, instead of
+just a context argument (ctx). Having to modify also GC allocator
+functions adds too much complexity for a little gain.
 
 
 GIL-free PyMem_Malloc()
@@ -520,15 +531,16 @@
 ``malloc()``.  The "GIL must be held" restriction can be removed from
 ``PyMem_Malloc()``.
 
-Allowing to call ``PyMem_Malloc()`` without holding the GIL might break
-applications which setup their own allocators or allocator hooks.
-Holding the GIL is convinient to develop a custom allocator: no need to
-care of other threads. It is also convinient for a debug allocator hook:
-Python internal objects can be safetly inspected.
+This alternative was rejected because allowing to call
+``PyMem_Malloc()`` without holding the GIL might break applications
+which setup their own allocators or allocator hooks.  Holding the GIL is
+convinient to develop a custom allocator: no need to care of other
+threads. It is also convinient for a debug allocator hook: Python
+internal objects can be safetly inspected.
 
-Calling ``PyGILState_Ensure()`` in a memory allocator may have
-unexpected behaviour, especially at Python startup and at creation of a
-new Python thread state.
+Calling ``PyGILState_Ensure()`` in
+a memory allocator may have unexpected behaviour, especially at Python
+startup and at creation of a new Python thread state.
 
 
 Don't add PyMem_RawMalloc()
@@ -565,15 +577,17 @@
 Python filename and line number (or even the Python traceback) is more
 useful.
 
-Classic tools are unable to introspect Python internals to collect such
-information. Being able to setup a hook on allocators called with the
-GIL held allow to collect a lot of useful data from Python internals.
+This alternative was rejected because classic tools are unable to
+introspect Python internals to collect such information. Being able to
+setup a hook on allocators called with the GIL held allow to collect a
+lot of useful data from Python internals.
 
 
-Add msize()
------------
+Add a msize() function
+----------------------
 
-Add another field to ``PyMemAllocator`` and ``PyObjectArenaAllocator``::
+Add another field to ``PyMemAllocator`` and ``PyObjectArenaAllocator``
+structures::
 
     size_t msize(void *ptr);
 
@@ -584,6 +598,19 @@
 On Windows, this function can be implemented using ``_msize()`` and
 ``VirtualQuery()``.
 
+The function can be used to implement an hook tracking the memory usage.
+The ``free()`` method of an allocator only gets the address of a memory
+block, whereas the size of the memory block is required to update the
+memory usage.
+
+The additional ``msize()`` function was rejected because only few
+platforms implement it. For example, Linux with the GNU libc does not
+provide a function to get the size of a memory block. ``msize()`` is not
+currently used in the Python source code. The function is only used to
+track the memory usage, but makes the API more complex. A debug hook can
+implemente the function internally, there is no need to add it to
+``PyMemAllocator`` and ``PyObjectArenaAllocator`` structures.
+
 
 No context argument
 -------------------
@@ -721,3 +748,9 @@
 * `PySizer (developed for Python 2.4)
   <http://pysizer.8325.org/>`_
 
+
+Copyright
+=========
+
+This document has been placed into the public domain.
+

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Fri Jun 28 23:02:43 2013
From: python-checkins at python.org (ethan.furman)
Date: Fri, 28 Jun 2013 23:02:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_example_to_enum_docs?=
 =?utf-8?q?_show_access_to_name_and_value_attributes_of_enum?=
Message-ID: <3bhr5v19FRzRkl@mail.python.org>

http://hg.python.org/cpython/rev/446ae05de566
changeset:   84360:446ae05de566
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Fri Jun 28 14:02:34 2013 -0700
summary:
  Added example to enum docs show access to name and value attributes of enum members.

files:
  Doc/library/enum.rst |  12 ++++++++++--
  1 files changed, 10 insertions(+), 2 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
--- a/Doc/library/enum.rst
+++ b/Doc/library/enum.rst
@@ -87,8 +87,8 @@
     True
 
 
-Programmatic access to enumeration members
-------------------------------------------
+Programmatic access to enumeration members and their attributes
+---------------------------------------------------------------
 
 Sometimes it's useful to access members in enumerations programmatically (i.e.
 situations where ``Color.red`` won't do because the exact color is not known
@@ -106,6 +106,14 @@
     >>> Color['green']
     <Color.green: 2>
 
+If have an enum member and need its :attr:`name` or :attr:`value`::
+
+    >>> member = Color.red
+    >>> member.name
+    'red'
+    >>> member.value
+    1
+
 
 Duplicating enum members and values
 -----------------------------------

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 01:00:19 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 01:00:19 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MzE1?=
 =?utf-8?q?=3A_Improve_fileinput_docs_by_adding_=27bufsize=27_where_missin?=
 =?utf-8?q?g_and?=
Message-ID: <3bhtjb5ZtczRrF@mail.python.org>

http://hg.python.org/cpython/rev/80b1c5b25ff0
changeset:   84361:80b1c5b25ff0
branch:      2.7
parent:      84343:e95161820160
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 18:59:19 2013 -0400
summary:
  Issue #18315: Improve fileinput docs by adding 'bufsize' where missing and
replacing redundant signature in input() docstring with one-line summary.
Original patch by Terrel Shumway.

files:
  Doc/library/fileinput.rst |   4 ++--
  Lib/fileinput.py          |  11 +++++------
  Misc/ACKS                 |   1 +
  3 files changed, 8 insertions(+), 8 deletions(-)


diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst
--- a/Doc/library/fileinput.rst
+++ b/Doc/library/fileinput.rst
@@ -50,7 +50,7 @@
 The following function is the primary interface of this module:
 
 
-.. function:: input([files[, inplace[, backup[, mode[, openhook]]]]])
+.. function:: input([files[, inplace[, backup[, bufsize[, mode[, openhook]]]]]])
 
    Create an instance of the :class:`FileInput` class.  The instance will be used
    as global state for the functions of this module, and is also returned to use
@@ -122,7 +122,7 @@
 available for subclassing as well:
 
 
-.. class:: FileInput([files[, inplace[, backup[, mode[, openhook]]]]])
+.. class:: FileInput([files[, inplace[, backup[,bufsize[, mode[, openhook]]]]]])
 
    Class :class:`FileInput` is the implementation; its methods :meth:`filename`,
    :meth:`fileno`, :meth:`lineno`, :meth:`filelineno`, :meth:`isfirstline`,
diff --git a/Lib/fileinput.py b/Lib/fileinput.py
--- a/Lib/fileinput.py
+++ b/Lib/fileinput.py
@@ -90,12 +90,11 @@
 
 def input(files=None, inplace=0, backup="", bufsize=0,
           mode="r", openhook=None):
-    """input([files[, inplace[, backup[, mode[, openhook]]]]])
+    """Return an instance of the FileInput class, which can be iterated.
 
-    Create an instance of the FileInput class. The instance will be used
-    as global state for the functions of this module, and is also returned
-    to use during iteration. The parameters to this function will be passed
-    along to the constructor of the FileInput class.
+    The parameters are passed to the constructor of the FileInput class.
+    The returned instance, in addition to being an iterator,
+    keeps global state for the functions of this module,.
     """
     global _state
     if _state and _state._file:
@@ -182,7 +181,7 @@
     return _state.isstdin()
 
 class FileInput:
-    """class FileInput([files[, inplace[, backup[, mode[, openhook]]]]])
+    """FileInput([files[, inplace[, backup[, bufsize[, mode[, openhook]]]]]])
 
     Class FileInput is the implementation of the module; its methods
     filename(), lineno(), fileline(), isfirstline(), isstdin(), fileno(),
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -937,6 +937,7 @@
 Joel Shprentz
 Itamar Shtull-Trauring
 Yue Shuaijie
+Terrel Shumway
 Eric Siegerman
 Paul Sijben
 Tim Silk

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 01:00:21 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 01:00:21 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MzE1?=
 =?utf-8?q?=3A_Improve_fileinput_docs_by_adding_=27bufsize=27_where_missin?=
 =?utf-8?q?g_and?=
Message-ID: <3bhtjd0L1fz7LjW@mail.python.org>

http://hg.python.org/cpython/rev/0760b58526ba
changeset:   84362:0760b58526ba
branch:      3.3
parent:      84358:7c807bc15fa8
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 18:59:28 2013 -0400
summary:
  Issue #18315: Improve fileinput docs by adding 'bufsize' where missing and
replacing redundant signature in input() docstring with one-line summary.
Original patch by Terrel Shumway.

files:
  Lib/fileinput.py |  12 +++++-------
  Misc/ACKS        |   1 +
  2 files changed, 6 insertions(+), 7 deletions(-)


diff --git a/Lib/fileinput.py b/Lib/fileinput.py
--- a/Lib/fileinput.py
+++ b/Lib/fileinput.py
@@ -90,13 +90,11 @@
 
 def input(files=None, inplace=False, backup="", bufsize=0,
           mode="r", openhook=None):
-    """input(files=None, inplace=False, backup="", bufsize=0, \
-mode="r", openhook=None)
+    """Return an instance of the FileInput class, which can be iterated.
 
-    Create an instance of the FileInput class. The instance will be used
-    as global state for the functions of this module, and is also returned
-    to use during iteration. The parameters to this function will be passed
-    along to the constructor of the FileInput class.
+    The parameters are passed to the constructor of the FileInput class.
+    The returned instance, in addition to being an iterator,
+    keeps global state for the functions of this module,.
     """
     global _state
     if _state and _state._file:
@@ -183,7 +181,7 @@
     return _state.isstdin()
 
 class FileInput:
-    """class FileInput([files[, inplace[, backup[, mode[, openhook]]]]])
+    """FileInput([files[, inplace[, backup[, bufsize, [, mode[, openhook]]]]]])
 
     Class FileInput is the implementation of the module; its methods
     filename(), lineno(), fileline(), isfirstline(), isstdin(), fileno(),
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1125,6 +1125,7 @@
 Joel Shprentz
 Itamar Shtull-Trauring
 Yue Shuaijie
+Terrel Shumway
 Eric Siegerman
 Paul Sijben
 SilentGhost

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 01:00:22 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 01:00:22 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bhtjf2Bnrz7Ljb@mail.python.org>

http://hg.python.org/cpython/rev/03747d64c042
changeset:   84363:03747d64c042
parent:      84360:446ae05de566
parent:      84362:0760b58526ba
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 18:59:52 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/fileinput.py |  12 +++++-------
  Misc/ACKS        |   1 +
  2 files changed, 6 insertions(+), 7 deletions(-)


diff --git a/Lib/fileinput.py b/Lib/fileinput.py
--- a/Lib/fileinput.py
+++ b/Lib/fileinput.py
@@ -90,13 +90,11 @@
 
 def input(files=None, inplace=False, backup="", bufsize=0,
           mode="r", openhook=None):
-    """input(files=None, inplace=False, backup="", bufsize=0, \
-mode="r", openhook=None)
+    """Return an instance of the FileInput class, which can be iterated.
 
-    Create an instance of the FileInput class. The instance will be used
-    as global state for the functions of this module, and is also returned
-    to use during iteration. The parameters to this function will be passed
-    along to the constructor of the FileInput class.
+    The parameters are passed to the constructor of the FileInput class.
+    The returned instance, in addition to being an iterator,
+    keeps global state for the functions of this module,.
     """
     global _state
     if _state and _state._file:
@@ -183,7 +181,7 @@
     return _state.isstdin()
 
 class FileInput:
-    """class FileInput([files[, inplace[, backup[, mode[, openhook]]]]])
+    """FileInput([files[, inplace[, backup[, bufsize, [, mode[, openhook]]]]]])
 
     Class FileInput is the implementation of the module; its methods
     filename(), lineno(), fileline(), isfirstline(), isstdin(), fileno(),
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -1154,6 +1154,7 @@
 Joel Shprentz
 Itamar Shtull-Trauring
 Yue Shuaijie
+Terrel Shumway
 Eric Siegerman
 Paul Sijben
 SilentGhost

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 04:32:49 2013
From: python-checkins at python.org (ethan.furman)
Date: Sat, 29 Jun 2013 04:32:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?peps=3A_Added_rationale_for_functiona?=
 =?utf-8?q?l_API_defaulting_to_1_as_the_start=2E?=
Message-ID: <3bhzQn1NCYz7LjP@mail.python.org>

http://hg.python.org/peps/rev/7c185dda44ac
changeset:   4967:7c185dda44ac
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Fri Jun 28 19:32:40 2013 -0700
summary:
  Added rationale for functional API defaulting to 1 as the start.

files:
  pep-0435.txt |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/pep-0435.txt b/pep-0435.txt
--- a/pep-0435.txt
+++ b/pep-0435.txt
@@ -467,6 +467,10 @@
     ...   cat = 3
     ...   dog = 4
 
+The reason for defaulting to ``1`` as the starting number and not ``0`` is
+that ``0`` is ``False`` in a boolean sense, but enum members all evaluate
+to ``True``.
+
 
 Proposed variations
 ===================

-- 
Repository URL: http://hg.python.org/peps

From python-checkins at python.org  Sat Jun 29 04:37:29 2013
From: python-checkins at python.org (ethan.furman)
Date: Sat, 29 Jun 2013 04:37:29 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_rationale_for_defaul?=
 =?utf-8?q?ting_to_1_in_the_functional_API=2E?=
Message-ID: <3bhzX9547gz7LjS@mail.python.org>

http://hg.python.org/cpython/rev/852c90008881
changeset:   84364:852c90008881
user:        Ethan Furman <ethan at stoneleaf.us>
date:        Fri Jun 28 19:37:17 2013 -0700
summary:
  Added rationale for defaulting to 1 in the functional API.

files:
  Doc/library/enum.rst |  4 ++++
  1 files changed, 4 insertions(+), 0 deletions(-)


diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
--- a/Doc/library/enum.rst
+++ b/Doc/library/enum.rst
@@ -335,6 +335,10 @@
     ...   cat = 3
     ...   dog = 4
 
+The reason for defaulting to ``1`` as the starting number and not ``0`` is
+that ``0`` is ``False`` in a boolean sense, but enum members all evaluate
+to ``True``.
+
 Pickling enums created with the functional API can be tricky as frame stack
 implementation details are used to try and figure out which module the
 enumeration is being created in (e.g. it will fail if you use a utility

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sat Jun 29 05:46:01 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sat, 29 Jun 2013 05:46:01 +0200
Subject: [Python-checkins] Daily reference leaks (03747d64c042): sum=0
Message-ID: <E1Usm6r-00012U-6z@ap.vmr.nerim.net>

results for 03747d64c042 on branch "default"
--------------------------------------------



Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogMIfidP', '-x']

From python-checkins at python.org  Sat Jun 29 05:48:04 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 05:48:04 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318081=3A_Back_out?=
 =?utf-8?q?_temporary_changeset=2C_2a9e1eb3719c=2C_to_merge_new_patch=2E?=
Message-ID: <3bj15c3nwKzSq9@mail.python.org>

http://hg.python.org/cpython/rev/2176c1867b34
changeset:   84365:2176c1867b34
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 23:47:40 2013 -0400
summary:
  Issue #18081: Back out temporary changeset, 2a9e1eb3719c, to merge new patch.
If buildbots run before next push, test_logging will (temporarily) fail.

files:
  Lib/idlelib/PyShell.py |  5 ++---
  Lib/idlelib/run.py     |  4 +---
  2 files changed, 3 insertions(+), 6 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -61,6 +61,7 @@
                                               lineno, line=line))
         except OSError:
             pass  ## file (probably __stderr__) is invalid, warning dropped.
+    warnings.showwarning = idle_showwarning
     def idle_formatwarning(message, category, filename, lineno, line=None):
         """Format warnings the IDLE way"""
         s = "\nWarning (from warnings module):\n"
@@ -72,6 +73,7 @@
             s += "    %s\n" % line
         s += "%s: %s\n>>> " % (category.__name__, message)
         return s
+    warnings.formatwarning = idle_formatwarning
 
 def extended_linecache_checkcache(filename=None,
                                   orig_checkcache=linecache.checkcache):
@@ -1423,9 +1425,6 @@
 def main():
     global flist, root, use_subprocess
 
-    warnings.showwarning = idle_showwarning
-    warnings.formatwarning = idle_formatwarning
-
     use_subprocess = True
     enable_shell = False
     enable_edit = False
diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -40,6 +40,7 @@
             s += "    %s\n" % line
         s += "%s: %s\n" % (category.__name__, message)
         return s
+    warnings.formatwarning = idle_formatwarning_subproc
 
 
 tcl = tkinter.Tcl()
@@ -81,9 +82,6 @@
     global exit_now
     global quitting
     global no_exitfunc
-
-    warnings.formatwarning = idle_formatwarning_subproc
-
     no_exitfunc = del_exitfunc
     #time.sleep(15) # test subprocess not responding
     try:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 05:53:43 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 05:53:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E3=29=3A_Issue_*18081?=
 =?utf-8?q?=2C_=2318242=3A_Change_Idle_warnings_capture_in_PyShell_and_run?=
 =?utf-8?q?_to_stop?=
Message-ID: <3bj1D72hhdz7LjN@mail.python.org>

http://hg.python.org/cpython/rev/c15d0baac3d6
changeset:   84366:c15d0baac3d6
branch:      3.3
parent:      84362:0760b58526ba
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 23:50:12 2013 -0400
summary:
  Issue *18081, #18242: Change Idle warnings capture in PyShell and run to stop
replacing warnings.formatwarnings and to reverse replacement of
warnings.showwarnings when import is complete and when main function exits.
Add test_warning.py. Vinay Sajip provided capture_warnings function.

files:
  Lib/idlelib/PyShell.py                |  82 +++++++++-----
  Lib/idlelib/idle_test/test_warning.py |  73 +++++++++++++
  Lib/idlelib/run.py                    |  56 ++++++---
  3 files changed, 162 insertions(+), 49 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -45,35 +45,55 @@
 # internal warnings to the console.  ScriptBinding.check_syntax() will
 # temporarily redirect the stream to the shell window to display warnings when
 # checking user's code.
-global warning_stream
-warning_stream = sys.__stderr__
-try:
-    import warnings
-except ImportError:
-    pass
-else:
-    def idle_showwarning(message, category, filename, lineno,
-                         file=None, line=None):
-        if file is None:
-            file = warning_stream
-        try:
-            file.write(warnings.formatwarning(message, category, filename,
-                                              lineno, line=line))
-        except OSError:
-            pass  ## file (probably __stderr__) is invalid, warning dropped.
-    warnings.showwarning = idle_showwarning
-    def idle_formatwarning(message, category, filename, lineno, line=None):
-        """Format warnings the IDLE way"""
-        s = "\nWarning (from warnings module):\n"
-        s += '  File \"%s\", line %s\n' % (filename, lineno)
-        if line is None:
-            line = linecache.getline(filename, lineno)
-        line = line.strip()
-        if line:
-            s += "    %s\n" % line
-        s += "%s: %s\n>>> " % (category.__name__, message)
-        return s
-    warnings.formatwarning = idle_formatwarning
+warning_stream = sys.__stderr__  # None, at least on Windows, if no console.
+import warnings
+
+def idle_formatwarning(message, category, filename, lineno, line=None):
+    """Format warnings the IDLE way."""
+
+    s = "\nWarning (from warnings module):\n"
+    s += '  File \"%s\", line %s\n' % (filename, lineno)
+    if line is None:
+        line = linecache.getline(filename, lineno)
+    line = line.strip()
+    if line:
+        s += "    %s\n" % line
+    s += "%s: %s\n" % (category.__name__, message)
+    return s
+
+def idle_showwarning(
+        message, category, filename, lineno, file=None, line=None):
+    """Show Idle-format warning (after replacing warnings.showwarning).
+
+    The differences are the formatter called, the file=None replacement,
+    which can be None, the capture of the consequence AttributeError,
+    and the output of a hard-coded prompt.
+    """
+    if file is None:
+        file = warning_stream
+    try:
+        file.write(idle_formatwarning(
+                message, category, filename, lineno, line=line))
+        file.write(">>> ")
+    except (AttributeError, OSError):
+        pass  # if file (probably __stderr__) is invalid, skip warning.
+
+_warnings_showwarning = None
+
+def capture_warnings(capture):
+    "Replace warning.showwarning with idle_showwarning, or reverse."
+
+    global _warnings_showwarning
+    if capture:
+        if _warnings_showwarning is None:
+            _warnings_showwarning = warnings.showwarning
+            warnings.showwarning = idle_showwarning
+    else:
+        if _warnings_showwarning is not None:
+            warnings.showwarning = _warnings_showwarning
+            _warnings_showwarning = None
+
+capture_warnings(True)
 
 def extended_linecache_checkcache(filename=None,
                                   orig_checkcache=linecache.checkcache):
@@ -1421,6 +1441,7 @@
 def main():
     global flist, root, use_subprocess
 
+    capture_warnings(True)
     use_subprocess = True
     enable_shell = False
     enable_edit = False
@@ -1553,7 +1574,10 @@
     while flist.inversedict:  # keep IDLE running while files are open.
         root.mainloop()
     root.destroy()
+    capture_warnings(False)
 
 if __name__ == "__main__":
     sys.modules['PyShell'] = sys.modules['__main__']
     main()
+
+capture_warnings(False)  # Make sure turned off; see issue 18081
diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_warning.py
@@ -0,0 +1,73 @@
+'''Test warnings replacement in PyShell.py and run.py.
+
+This file could be expanded to include traceback overrides
+(in same two modules). If so, change name.
+Revise if output destination changes (http://bugs.python.org/issue18318).
+Make sure warnings module is left unaltered (http://bugs.python.org/issue18081).
+'''
+
+import unittest
+from test.support import captured_stderr
+
+import warnings
+# Try to capture default showwarning before Idle modules are imported.
+showwarning = warnings.showwarning
+# But if we run this file within idle, we are in the middle of the run.main loop
+# and default showwarnings has already been replaced.
+running_in_idle = 'idle' in showwarning.__name__
+
+from idlelib import run
+from idlelib import PyShell as shell
+
+# The following was generated from PyShell.idle_formatwarning
+# and checked as matching expectation.
+idlemsg = '''
+Warning (from warnings module):
+  File "test_warning.py", line 99
+    Line of code
+UserWarning: Test
+'''
+shellmsg = idlemsg + ">>> "
+
+class RunWarnTest(unittest.TestCase):
+
+    @unittest.skipIf(running_in_idle, "Does not work when run within Idle.")
+    def test_showwarnings(self):
+        self.assertIs(warnings.showwarning, showwarning)
+        run.capture_warnings(True)
+        self.assertIs(warnings.showwarning, run.idle_showwarning_subproc)
+        run.capture_warnings(False)
+        self.assertIs(warnings.showwarning, showwarning)
+
+    def test_run_show(self):
+        with captured_stderr() as f:
+            run.idle_showwarning_subproc(
+                    'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code')
+            # The following uses .splitlines to erase line-ending differences
+            self.assertEqual(idlemsg.splitlines(), f.getvalue().splitlines())
+
+class ShellWarnTest(unittest.TestCase):
+
+    @unittest.skipIf(running_in_idle, "Does not work when run within Idle.")
+    def test_showwarnings(self):
+        self.assertIs(warnings.showwarning, showwarning)
+        shell.capture_warnings(True)
+        self.assertIs(warnings.showwarning, shell.idle_showwarning)
+        shell.capture_warnings(False)
+        self.assertIs(warnings.showwarning, showwarning)
+
+    def test_idle_formatter(self):
+        # Will fail if format changed without regenerating idlemsg
+        s = shell.idle_formatwarning(
+                'Test', UserWarning, 'test_warning.py', 99, 'Line of code')
+        self.assertEqual(idlemsg, s)
+
+    def test_shell_show(self):
+        with captured_stderr() as f:
+            shell.idle_showwarning(
+                    'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code')
+            self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines())
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)
diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -23,36 +23,46 @@
 
 LOCALHOST = '127.0.0.1'
 
-try:
-    import warnings
-except ImportError:
-    pass
-else:
-    def idle_formatwarning_subproc(message, category, filename, lineno,
-                                   line=None):
-        """Format warnings the IDLE way"""
-        s = "\nWarning (from warnings module):\n"
-        s += '  File \"%s\", line %s\n' % (filename, lineno)
-        if line is None:
-            line = linecache.getline(filename, lineno)
-        line = line.strip()
-        if line:
-            s += "    %s\n" % line
-        s += "%s: %s\n" % (category.__name__, message)
-        return s
-    warnings.formatwarning = idle_formatwarning_subproc
+import warnings
 
+def idle_showwarning_subproc(
+        message, category, filename, lineno, file=None, line=None):
+    """Show Idle-format warning after replacing warnings.showwarning.
 
+    The only difference is the formatter called.
+    """
+    if file is None:
+        file = sys.stderr
+    try:
+        file.write(PyShell.idle_formatwarning(
+                message, category, filename, lineno, line))
+    except IOError:
+        pass # the file (probably stderr) is invalid - this warning gets lost.
+
+_warnings_showwarning = None
+
+def capture_warnings(capture):
+    "Replace warning.showwarning with idle_showwarning_subproc, or reverse."
+
+    global _warnings_showwarning
+    if capture:
+        if _warnings_showwarning is None:
+            _warnings_showwarning = warnings.showwarning
+            warnings.showwarning = idle_showwarning_subproc
+    else:
+        if _warnings_showwarning is not None:
+            warnings.showwarning = _warnings_showwarning
+            _warnings_showwarning = None
+
+capture_warnings(True)
 tcl = tkinter.Tcl()
 
-
 def handle_tk_events(tcl=tcl):
     """Process any tk events that are ready to be dispatched if tkinter
     has been imported, a tcl interpreter has been created and tk has been
     loaded."""
     tcl.eval("update")
 
-
 # Thread shared globals: Establish a queue between a subthread (which handles
 # the socket) and the main thread (which runs user code), plus global
 # completion, exit and interruptable (the main thread) flags:
@@ -91,6 +101,8 @@
         print("IDLE Subprocess: no IP port passed in sys.argv.",
               file=sys.__stderr__)
         return
+
+    capture_warnings(True)
     sys.argv[:] = [""]
     sockthread = threading.Thread(target=manage_socket,
                                   name='SockThread',
@@ -118,6 +130,7 @@
                 exit_now = True
             continue
         except SystemExit:
+            capture_warnings(False)
             raise
         except:
             type, value, tb = sys.exc_info()
@@ -247,6 +260,7 @@
     if no_exitfunc:
         import atexit
         atexit._clear()
+    capture_warnings(False)
     sys.exit(0)
 
 class MyRPCServer(rpc.RPCServer):
@@ -386,3 +400,5 @@
         sys.last_value = val
         item = StackViewer.StackTreeItem(flist, tb)
         return RemoteObjectBrowser.remote_object_tree_item(item)
+
+capture_warnings(False)  # Make sure turned off; see issue 18081

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 05:53:44 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 05:53:44 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Issue_*18081?=
 =?utf-8?q?=2C_=2318242=3A_Change_Idle_warnings_capture_in_PyShell_and_run?=
 =?utf-8?q?_to_stop?=
Message-ID: <3bj1D85kB7z7Ljg@mail.python.org>

http://hg.python.org/cpython/rev/89e0d33cb978
changeset:   84367:89e0d33cb978
branch:      2.7
parent:      84361:80b1c5b25ff0
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 23:51:34 2013 -0400
summary:
  Issue *18081, #18242: Change Idle warnings capture in PyShell and run to stop
replacing warnings.formatwarnings and to reverse replacement of
warnings.showwarnings when import is complete and when main function exits.
Add test_warning.py. Vinay Sajip provided capture_warnings function.

files:
  Lib/idlelib/PyShell.py                |  82 +++++++++-----
  Lib/idlelib/idle_test/test_warning.py |  73 +++++++++++++
  Lib/idlelib/run.py                    |  56 ++++++---
  3 files changed, 164 insertions(+), 47 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -50,35 +50,55 @@
 # internal warnings to the console.  ScriptBinding.check_syntax() will
 # temporarily redirect the stream to the shell window to display warnings when
 # checking user's code.
-global warning_stream
-warning_stream = sys.__stderr__
-try:
-    import warnings
-except ImportError:
-    pass
-else:
-    def idle_showwarning(message, category, filename, lineno,
-                         file=None, line=None):
-        if file is None:
-            file = warning_stream
-        try:
-            file.write(warnings.formatwarning(message, category, filename,
-                                              lineno, line=line))
-        except IOError:
-            pass  ## file (probably __stderr__) is invalid, warning dropped.
-    warnings.showwarning = idle_showwarning
-    def idle_formatwarning(message, category, filename, lineno, line=None):
-        """Format warnings the IDLE way"""
-        s = "\nWarning (from warnings module):\n"
-        s += '  File \"%s\", line %s\n' % (filename, lineno)
-        if line is None:
-            line = linecache.getline(filename, lineno)
-        line = line.strip()
-        if line:
-            s += "    %s\n" % line
-        s += "%s: %s\n>>> " % (category.__name__, message)
-        return s
-    warnings.formatwarning = idle_formatwarning
+warning_stream = sys.__stderr__  # None, at least on Windows, if no console.
+import warnings
+
+def idle_formatwarning(message, category, filename, lineno, line=None):
+    """Format warnings the IDLE way."""
+
+    s = "\nWarning (from warnings module):\n"
+    s += '  File \"%s\", line %s\n' % (filename, lineno)
+    if line is None:
+        line = linecache.getline(filename, lineno)
+    line = line.strip()
+    if line:
+        s += "    %s\n" % line
+    s += "%s: %s\n" % (category.__name__, message)
+    return s
+
+def idle_showwarning(
+        message, category, filename, lineno, file=None, line=None):
+    """Show Idle-format warning (after replacing warnings.showwarning).
+
+    The differences are the formatter called, the file=None replacement,
+    which can be None, the capture of the consequence AttributeError,
+    and the output of a hard-coded prompt.
+    """
+    if file is None:
+        file = warning_stream
+    try:
+        file.write(idle_formatwarning(
+                message, category, filename, lineno, line=line))
+        file.write(">>> ")
+    except (AttributeError, IOError):
+        pass  # if file (probably __stderr__) is invalid, skip warning.
+
+_warnings_showwarning = None
+
+def capture_warnings(capture):
+    "Replace warning.showwarning with idle_showwarning, or reverse."
+
+    global _warnings_showwarning
+    if capture:
+        if _warnings_showwarning is None:
+            _warnings_showwarning = warnings.showwarning
+            warnings.showwarning = idle_showwarning
+    else:
+        if _warnings_showwarning is not None:
+            warnings.showwarning = _warnings_showwarning
+            _warnings_showwarning = None
+
+capture_warnings(True)
 
 def extended_linecache_checkcache(filename=None,
                                   orig_checkcache=linecache.checkcache):
@@ -1428,6 +1448,7 @@
 def main():
     global flist, root, use_subprocess
 
+    capture_warnings(True)
     use_subprocess = True
     enable_shell = False
     enable_edit = False
@@ -1560,7 +1581,10 @@
     while flist.inversedict:  # keep IDLE running while files are open.
         root.mainloop()
     root.destroy()
+    capture_warnings(False)
 
 if __name__ == "__main__":
     sys.modules['PyShell'] = sys.modules['__main__']
     main()
+
+capture_warnings(False)  # Make sure turned off; see issue 18081
diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_warning.py
@@ -0,0 +1,73 @@
+'''Test warnings replacement in PyShell.py and run.py.
+
+This file could be expanded to include traceback overrides
+(in same two modules). If so, change name.
+Revise if output destination changes (http://bugs.python.org/issue18318).
+Make sure warnings module is left unaltered (http://bugs.python.org/issue18081).
+'''
+
+import unittest
+from test.test_support import captured_stderr
+
+import warnings
+# Try to capture default showwarning before Idle modules are imported.
+showwarning = warnings.showwarning
+# But if we run this file within idle, we are in the middle of the run.main loop
+# and default showwarnings has already been replaced.
+running_in_idle = 'idle' in showwarning.__name__
+
+from idlelib import run
+from idlelib import PyShell as shell
+
+# The following was generated from PyShell.idle_formatwarning
+# and checked as matching expectation.
+idlemsg = '''
+Warning (from warnings module):
+  File "test_warning.py", line 99
+    Line of code
+UserWarning: Test
+'''
+shellmsg = idlemsg + ">>> "
+
+class RunWarnTest(unittest.TestCase):
+
+    @unittest.skipIf(running_in_idle, "Does not work when run within Idle.")
+    def test_showwarnings(self):
+        self.assertIs(warnings.showwarning, showwarning)
+        run.capture_warnings(True)
+        self.assertIs(warnings.showwarning, run.idle_showwarning_subproc)
+        run.capture_warnings(False)
+        self.assertIs(warnings.showwarning, showwarning)
+
+    def test_run_show(self):
+        with captured_stderr() as f:
+            run.idle_showwarning_subproc(
+                    'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code')
+            # The following uses .splitlines to erase line-ending differences
+            self.assertEqual(idlemsg.splitlines(), f.getvalue().splitlines())
+
+class ShellWarnTest(unittest.TestCase):
+
+    @unittest.skipIf(running_in_idle, "Does not work when run within Idle.")
+    def test_showwarnings(self):
+        self.assertIs(warnings.showwarning, showwarning)
+        shell.capture_warnings(True)
+        self.assertIs(warnings.showwarning, shell.idle_showwarning)
+        shell.capture_warnings(False)
+        self.assertIs(warnings.showwarning, showwarning)
+
+    def test_idle_formatter(self):
+        # Will fail if format changed without regenerating idlemsg
+        s = shell.idle_formatwarning(
+                'Test', UserWarning, 'test_warning.py', 99, 'Line of code')
+        self.assertEqual(idlemsg, s)
+
+    def test_shell_show(self):
+        with captured_stderr() as f:
+            shell.idle_showwarning(
+                    'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code')
+            self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines())
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)
diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -22,24 +22,38 @@
 
 LOCALHOST = '127.0.0.1'
 
-try:
-    import warnings
-except ImportError:
-    pass
-else:
-    def idle_formatwarning_subproc(message, category, filename, lineno,
-                                   line=None):
-        """Format warnings the IDLE way"""
-        s = "\nWarning (from warnings module):\n"
-        s += '  File \"%s\", line %s\n' % (filename, lineno)
-        if line is None:
-            line = linecache.getline(filename, lineno)
-        line = line.strip()
-        if line:
-            s += "    %s\n" % line
-        s += "%s: %s\n" % (category.__name__, message)
-        return s
-    warnings.formatwarning = idle_formatwarning_subproc
+import warnings
+
+def idle_showwarning_subproc(
+        message, category, filename, lineno, file=None, line=None):
+    """Show Idle-format warning after replacing warnings.showwarning.
+
+    The only difference is the formatter called.
+    """
+    if file is None:
+        file = sys.stderr
+    try:
+        file.write(PyShell.idle_formatwarning(
+                message, category, filename, lineno, line))
+    except IOError:
+        pass # the file (probably stderr) is invalid - this warning gets lost.
+
+_warnings_showwarning = None
+
+def capture_warnings(capture):
+    "Replace warning.showwarning with idle_showwarning_subproc, or reverse."
+
+    global _warnings_showwarning
+    if capture:
+        if _warnings_showwarning is None:
+            _warnings_showwarning = warnings.showwarning
+            warnings.showwarning = idle_showwarning_subproc
+    else:
+        if _warnings_showwarning is not None:
+            warnings.showwarning = _warnings_showwarning
+            _warnings_showwarning = None
+
+capture_warnings(True)
 
 # Thread shared globals: Establish a queue between a subthread (which handles
 # the socket) and the main thread (which runs user code), plus global
@@ -78,6 +92,8 @@
     except:
         print>>sys.stderr, "IDLE Subprocess: no IP port passed in sys.argv."
         return
+
+    capture_warnings(True)
     sys.argv[:] = [""]
     sockthread = threading.Thread(target=manage_socket,
                                   name='SockThread',
@@ -104,6 +120,7 @@
                 exit_now = True
             continue
         except SystemExit:
+            capture_warnings(False)
             raise
         except:
             type, value, tb = sys.exc_info()
@@ -219,6 +236,7 @@
             del sys.exitfunc
         except AttributeError:
             pass
+    capture_warnings(False)
     sys.exit(0)
 
 class MyRPCServer(rpc.RPCServer):
@@ -352,3 +370,5 @@
         sys.last_value = val
         item = StackViewer.StackTreeItem(flist, tb)
         return RemoteObjectBrowser.remote_object_tree_item(item)
+
+capture_warnings(False)  # Make sure turned off; see issue 18081

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 05:53:46 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 05:53:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bj1DB1kYGz7LkT@mail.python.org>

http://hg.python.org/cpython/rev/373d629bd5fc
changeset:   84368:373d629bd5fc
parent:      84365:2176c1867b34
parent:      84366:c15d0baac3d6
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Fri Jun 28 23:52:05 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/idlelib/PyShell.py                |  82 +++++++++-----
  Lib/idlelib/idle_test/test_warning.py |  73 +++++++++++++
  Lib/idlelib/run.py                    |  56 ++++++---
  3 files changed, 162 insertions(+), 49 deletions(-)


diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -45,35 +45,55 @@
 # internal warnings to the console.  ScriptBinding.check_syntax() will
 # temporarily redirect the stream to the shell window to display warnings when
 # checking user's code.
-global warning_stream
-warning_stream = sys.__stderr__
-try:
-    import warnings
-except ImportError:
-    pass
-else:
-    def idle_showwarning(message, category, filename, lineno,
-                         file=None, line=None):
-        if file is None:
-            file = warning_stream
-        try:
-            file.write(warnings.formatwarning(message, category, filename,
-                                              lineno, line=line))
-        except OSError:
-            pass  ## file (probably __stderr__) is invalid, warning dropped.
-    warnings.showwarning = idle_showwarning
-    def idle_formatwarning(message, category, filename, lineno, line=None):
-        """Format warnings the IDLE way"""
-        s = "\nWarning (from warnings module):\n"
-        s += '  File \"%s\", line %s\n' % (filename, lineno)
-        if line is None:
-            line = linecache.getline(filename, lineno)
-        line = line.strip()
-        if line:
-            s += "    %s\n" % line
-        s += "%s: %s\n>>> " % (category.__name__, message)
-        return s
-    warnings.formatwarning = idle_formatwarning
+warning_stream = sys.__stderr__  # None, at least on Windows, if no console.
+import warnings
+
+def idle_formatwarning(message, category, filename, lineno, line=None):
+    """Format warnings the IDLE way."""
+
+    s = "\nWarning (from warnings module):\n"
+    s += '  File \"%s\", line %s\n' % (filename, lineno)
+    if line is None:
+        line = linecache.getline(filename, lineno)
+    line = line.strip()
+    if line:
+        s += "    %s\n" % line
+    s += "%s: %s\n" % (category.__name__, message)
+    return s
+
+def idle_showwarning(
+        message, category, filename, lineno, file=None, line=None):
+    """Show Idle-format warning (after replacing warnings.showwarning).
+
+    The differences are the formatter called, the file=None replacement,
+    which can be None, the capture of the consequence AttributeError,
+    and the output of a hard-coded prompt.
+    """
+    if file is None:
+        file = warning_stream
+    try:
+        file.write(idle_formatwarning(
+                message, category, filename, lineno, line=line))
+        file.write(">>> ")
+    except (AttributeError, OSError):
+        pass  # if file (probably __stderr__) is invalid, skip warning.
+
+_warnings_showwarning = None
+
+def capture_warnings(capture):
+    "Replace warning.showwarning with idle_showwarning, or reverse."
+
+    global _warnings_showwarning
+    if capture:
+        if _warnings_showwarning is None:
+            _warnings_showwarning = warnings.showwarning
+            warnings.showwarning = idle_showwarning
+    else:
+        if _warnings_showwarning is not None:
+            warnings.showwarning = _warnings_showwarning
+            _warnings_showwarning = None
+
+capture_warnings(True)
 
 def extended_linecache_checkcache(filename=None,
                                   orig_checkcache=linecache.checkcache):
@@ -1425,6 +1445,7 @@
 def main():
     global flist, root, use_subprocess
 
+    capture_warnings(True)
     use_subprocess = True
     enable_shell = False
     enable_edit = False
@@ -1559,7 +1580,10 @@
     while flist.inversedict:  # keep IDLE running while files are open.
         root.mainloop()
     root.destroy()
+    capture_warnings(False)
 
 if __name__ == "__main__":
     sys.modules['PyShell'] = sys.modules['__main__']
     main()
+
+capture_warnings(False)  # Make sure turned off; see issue 18081
diff --git a/Lib/idlelib/idle_test/test_warning.py b/Lib/idlelib/idle_test/test_warning.py
new file mode 100644
--- /dev/null
+++ b/Lib/idlelib/idle_test/test_warning.py
@@ -0,0 +1,73 @@
+'''Test warnings replacement in PyShell.py and run.py.
+
+This file could be expanded to include traceback overrides
+(in same two modules). If so, change name.
+Revise if output destination changes (http://bugs.python.org/issue18318).
+Make sure warnings module is left unaltered (http://bugs.python.org/issue18081).
+'''
+
+import unittest
+from test.support import captured_stderr
+
+import warnings
+# Try to capture default showwarning before Idle modules are imported.
+showwarning = warnings.showwarning
+# But if we run this file within idle, we are in the middle of the run.main loop
+# and default showwarnings has already been replaced.
+running_in_idle = 'idle' in showwarning.__name__
+
+from idlelib import run
+from idlelib import PyShell as shell
+
+# The following was generated from PyShell.idle_formatwarning
+# and checked as matching expectation.
+idlemsg = '''
+Warning (from warnings module):
+  File "test_warning.py", line 99
+    Line of code
+UserWarning: Test
+'''
+shellmsg = idlemsg + ">>> "
+
+class RunWarnTest(unittest.TestCase):
+
+    @unittest.skipIf(running_in_idle, "Does not work when run within Idle.")
+    def test_showwarnings(self):
+        self.assertIs(warnings.showwarning, showwarning)
+        run.capture_warnings(True)
+        self.assertIs(warnings.showwarning, run.idle_showwarning_subproc)
+        run.capture_warnings(False)
+        self.assertIs(warnings.showwarning, showwarning)
+
+    def test_run_show(self):
+        with captured_stderr() as f:
+            run.idle_showwarning_subproc(
+                    'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code')
+            # The following uses .splitlines to erase line-ending differences
+            self.assertEqual(idlemsg.splitlines(), f.getvalue().splitlines())
+
+class ShellWarnTest(unittest.TestCase):
+
+    @unittest.skipIf(running_in_idle, "Does not work when run within Idle.")
+    def test_showwarnings(self):
+        self.assertIs(warnings.showwarning, showwarning)
+        shell.capture_warnings(True)
+        self.assertIs(warnings.showwarning, shell.idle_showwarning)
+        shell.capture_warnings(False)
+        self.assertIs(warnings.showwarning, showwarning)
+
+    def test_idle_formatter(self):
+        # Will fail if format changed without regenerating idlemsg
+        s = shell.idle_formatwarning(
+                'Test', UserWarning, 'test_warning.py', 99, 'Line of code')
+        self.assertEqual(idlemsg, s)
+
+    def test_shell_show(self):
+        with captured_stderr() as f:
+            shell.idle_showwarning(
+                    'Test', UserWarning, 'test_warning.py', 99, f, 'Line of code')
+            self.assertEqual(shellmsg.splitlines(), f.getvalue().splitlines())
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2, exit=False)
diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -23,36 +23,46 @@
 
 LOCALHOST = '127.0.0.1'
 
-try:
-    import warnings
-except ImportError:
-    pass
-else:
-    def idle_formatwarning_subproc(message, category, filename, lineno,
-                                   line=None):
-        """Format warnings the IDLE way"""
-        s = "\nWarning (from warnings module):\n"
-        s += '  File \"%s\", line %s\n' % (filename, lineno)
-        if line is None:
-            line = linecache.getline(filename, lineno)
-        line = line.strip()
-        if line:
-            s += "    %s\n" % line
-        s += "%s: %s\n" % (category.__name__, message)
-        return s
-    warnings.formatwarning = idle_formatwarning_subproc
+import warnings
 
+def idle_showwarning_subproc(
+        message, category, filename, lineno, file=None, line=None):
+    """Show Idle-format warning after replacing warnings.showwarning.
 
+    The only difference is the formatter called.
+    """
+    if file is None:
+        file = sys.stderr
+    try:
+        file.write(PyShell.idle_formatwarning(
+                message, category, filename, lineno, line))
+    except IOError:
+        pass # the file (probably stderr) is invalid - this warning gets lost.
+
+_warnings_showwarning = None
+
+def capture_warnings(capture):
+    "Replace warning.showwarning with idle_showwarning_subproc, or reverse."
+
+    global _warnings_showwarning
+    if capture:
+        if _warnings_showwarning is None:
+            _warnings_showwarning = warnings.showwarning
+            warnings.showwarning = idle_showwarning_subproc
+    else:
+        if _warnings_showwarning is not None:
+            warnings.showwarning = _warnings_showwarning
+            _warnings_showwarning = None
+
+capture_warnings(True)
 tcl = tkinter.Tcl()
 
-
 def handle_tk_events(tcl=tcl):
     """Process any tk events that are ready to be dispatched if tkinter
     has been imported, a tcl interpreter has been created and tk has been
     loaded."""
     tcl.eval("update")
 
-
 # Thread shared globals: Establish a queue between a subthread (which handles
 # the socket) and the main thread (which runs user code), plus global
 # completion, exit and interruptable (the main thread) flags:
@@ -91,6 +101,8 @@
         print("IDLE Subprocess: no IP port passed in sys.argv.",
               file=sys.__stderr__)
         return
+
+    capture_warnings(True)
     sys.argv[:] = [""]
     sockthread = threading.Thread(target=manage_socket,
                                   name='SockThread',
@@ -118,6 +130,7 @@
                 exit_now = True
             continue
         except SystemExit:
+            capture_warnings(False)
             raise
         except:
             type, value, tb = sys.exc_info()
@@ -247,6 +260,7 @@
     if no_exitfunc:
         import atexit
         atexit._clear()
+    capture_warnings(False)
     sys.exit(0)
 
 class MyRPCServer(rpc.RPCServer):
@@ -386,3 +400,5 @@
         sys.last_value = val
         item = StackViewer.StackTreeItem(flist, tb)
         return RemoteObjectBrowser.remote_object_tree_item(item)
+
+capture_warnings(False)  # Make sure turned off; see issue 18081

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 06:59:56 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 06:59:56 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MzE2?=
 =?utf-8?q?=3A_Update_idlelib_2=2E7_except_clauses_to_ease_backports=2E?=
Message-ID: <3bj2hX5DB5z7Lk1@mail.python.org>

http://hg.python.org/cpython/rev/aa1350c7d196
changeset:   84369:aa1350c7d196
branch:      2.7
parent:      84367:89e0d33cb978
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 00:59:34 2013 -0400
summary:
  Issue #18316: Update idlelib 2.7 except clauses to ease backports.

files:
  Lib/idlelib/EditorWindow.py  |  2 +-
  Lib/idlelib/GrepDialog.py    |  2 +-
  Lib/idlelib/IOBinding.py     |  8 ++++----
  Lib/idlelib/PyShell.py       |  8 ++++----
  Lib/idlelib/ScriptBinding.py |  6 +++---
  Lib/idlelib/SearchEngine.py  |  2 +-
  Lib/idlelib/run.py           |  2 +-
  7 files changed, 15 insertions(+), 15 deletions(-)


diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -689,7 +689,7 @@
         # XXX Ought to insert current file's directory in front of path
         try:
             (f, file, (suffix, mode, type)) = _find_module(name)
-        except (NameError, ImportError), msg:
+        except (NameError, ImportError) as msg:
             tkMessageBox.showerror("Import error", str(msg), parent=self.text)
             return
         if type != imp.PY_SOURCE:
diff --git a/Lib/idlelib/GrepDialog.py b/Lib/idlelib/GrepDialog.py
--- a/Lib/idlelib/GrepDialog.py
+++ b/Lib/idlelib/GrepDialog.py
@@ -98,7 +98,7 @@
     def findfiles(self, dir, base, rec):
         try:
             names = os.listdir(dir or os.curdir)
-        except os.error, msg:
+        except os.error as msg:
             print msg
             return []
         list = []
diff --git a/Lib/idlelib/IOBinding.py b/Lib/idlelib/IOBinding.py
--- a/Lib/idlelib/IOBinding.py
+++ b/Lib/idlelib/IOBinding.py
@@ -251,7 +251,7 @@
             f = open(filename,'rb')
             chars = f.read()
             f.close()
-        except IOError, msg:
+        except IOError as msg:
             tkMessageBox.showerror("I/O Error", str(msg), master=self.text)
             return False
 
@@ -294,7 +294,7 @@
         # Next look for coding specification
         try:
             enc = coding_spec(chars)
-        except LookupError, name:
+        except LookupError as name:
             tkMessageBox.showerror(
                 title="Error loading the file",
                 message="The encoding '%s' is not known to this Python "\
@@ -388,7 +388,7 @@
             f.flush()
             f.close()
             return True
-        except IOError, msg:
+        except IOError as msg:
             tkMessageBox.showerror("I/O Error", str(msg),
                                    master=self.text)
             return False
@@ -408,7 +408,7 @@
         try:
             enc = coding_spec(chars)
             failed = None
-        except LookupError, msg:
+        except LookupError as msg:
             failed = msg
             enc = None
         if enc:
diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py
--- a/Lib/idlelib/PyShell.py
+++ b/Lib/idlelib/PyShell.py
@@ -430,7 +430,7 @@
             try:
                 self.rpcclt = MyRPCClient(addr)
                 break
-            except socket.error, err:
+            except socket.error as err:
                 pass
         else:
             self.display_port_binding_error()
@@ -451,7 +451,7 @@
         self.rpcclt.listening_sock.settimeout(10)
         try:
             self.rpcclt.accept()
-        except socket.timeout, err:
+        except socket.timeout as err:
             self.display_no_subprocess_error()
             return None
         self.rpcclt.register("console", self.tkconsole)
@@ -486,7 +486,7 @@
         self.spawn_subprocess()
         try:
             self.rpcclt.accept()
-        except socket.timeout, err:
+        except socket.timeout as err:
             self.display_no_subprocess_error()
             return None
         self.transfer_path(with_cwd=with_cwd)
@@ -1458,7 +1458,7 @@
     startup = False
     try:
         opts, args = getopt.getopt(sys.argv[1:], "c:deihnr:st:")
-    except getopt.error, msg:
+    except getopt.error as msg:
         sys.stderr.write("Error: %s\n" % str(msg))
         sys.stderr.write(usage_msg)
         sys.exit(2)
diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py
--- a/Lib/idlelib/ScriptBinding.py
+++ b/Lib/idlelib/ScriptBinding.py
@@ -70,13 +70,13 @@
         f = open(filename, 'r')
         try:
             tabnanny.process_tokens(tokenize.generate_tokens(f.readline))
-        except tokenize.TokenError, msg:
+        except tokenize.TokenError as msg:
             msgtxt, (lineno, start) = msg
             self.editwin.gotoline(lineno)
             self.errorbox("Tabnanny Tokenizing Error",
                           "Token Error: %s" % msgtxt)
             return False
-        except tabnanny.NannyNag, nag:
+        except tabnanny.NannyNag as nag:
             # The error messages from tabnanny are too confusing...
             self.editwin.gotoline(nag.get_lineno())
             self.errorbox("Tab/space error", indent_message)
@@ -101,7 +101,7 @@
             try:
                 # If successful, return the compiled code
                 return compile(source, filename, "exec")
-            except (SyntaxError, OverflowError, ValueError), err:
+            except (SyntaxError, OverflowError, ValueError) as err:
                 try:
                     msg, (errorfilename, lineno, offset, line) = err
                     if not errorfilename:
diff --git a/Lib/idlelib/SearchEngine.py b/Lib/idlelib/SearchEngine.py
--- a/Lib/idlelib/SearchEngine.py
+++ b/Lib/idlelib/SearchEngine.py
@@ -66,7 +66,7 @@
             flags = flags | re.IGNORECASE
         try:
             prog = re.compile(pat, flags)
-        except re.error, what:
+        except re.error as what:
             try:
                 msg, col = what
             except:
diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py
--- a/Lib/idlelib/run.py
+++ b/Lib/idlelib/run.py
@@ -140,7 +140,7 @@
         try:
             server = MyRPCServer(address, MyHandler)
             break
-        except socket.error, err:
+        except socket.error as err:
             print>>sys.__stderr__,"IDLE Subprocess: socket error: "\
                                         + err.args[1] + ", retrying...."
     else:

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 12:59:06 2013
From: python-checkins at python.org (antoine.pitrou)
Date: Sat, 29 Jun 2013 12:59:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318322=3A_fix_some?=
	=?utf-8?q?_test=5Fstat_nits=2E?=
Message-ID: <3bjBfy0ZQtzPPY@mail.python.org>

http://hg.python.org/cpython/rev/f3f38c84aebc
changeset:   84370:f3f38c84aebc
parent:      84368:373d629bd5fc
user:        Antoine Pitrou <solipsis at pitrou.net>
date:        Sat Jun 29 12:58:57 2013 +0200
summary:
  Issue #18322: fix some test_stat nits.

files:
  Lib/test/test_stat.py |  18 +++++++-----------
  1 files changed, 7 insertions(+), 11 deletions(-)


diff --git a/Lib/test/test_stat.py b/Lib/test/test_stat.py
--- a/Lib/test/test_stat.py
+++ b/Lib/test/test_stat.py
@@ -1,11 +1,11 @@
 import unittest
 import os
-from test.support import TESTFN, run_unittest, import_fresh_module
+from test.support import TESTFN, import_fresh_module
 
 c_stat = import_fresh_module('stat', fresh=['_stat'])
 py_stat = import_fresh_module('stat', blocked=['_stat'])
 
-class TestFilemode(unittest.TestCase):
+class TestFilemode:
     statmod = None
 
     file_flags = {'SF_APPEND', 'SF_ARCHIVED', 'SF_IMMUTABLE', 'SF_NOUNLINK',
@@ -186,21 +186,17 @@
             self.assertEqual(func(0), 0)
 
 
-class TestFilemodeCStat(TestFilemode):
+class TestFilemodeCStat(TestFilemode, unittest.TestCase):
     statmod = c_stat
 
     formats = TestFilemode.formats | {'S_IFDOOR', 'S_IFPORT', 'S_IFWHT'}
-    format_funcss = TestFilemode.format_funcs | {'S_ISDOOR', 'S_ISPORT',
-                                                 'S_ISWHT'}
+    format_funcs = TestFilemode.format_funcs | {'S_ISDOOR', 'S_ISPORT',
+                                                'S_ISWHT'}
 
 
-class TestFilemodePyStat(TestFilemode):
+class TestFilemodePyStat(TestFilemode, unittest.TestCase):
     statmod = py_stat
 
 
-def test_main():
-    run_unittest(TestFilemodeCStat)
-    run_unittest(TestFilemodePyStat)
-
 if __name__ == '__main__':
-    test_main()
+    unittest.main()

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 19:16:44 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 19:16:44 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MjM3?=
 =?utf-8?q?=3A_Fix_assertRaisesRegexp_error_caought_by_Jeff_Tratner=2E?=
Message-ID: <3bjM2h054cz7Ljj@mail.python.org>

http://hg.python.org/cpython/rev/4a714fea95ef
changeset:   84371:4a714fea95ef
branch:      2.7
parent:      84369:aa1350c7d196
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 13:15:36 2013 -0400
summary:
  Issue #18237: Fix assertRaisesRegexp error caought by Jeff Tratner.

files:
  Doc/library/unittest.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -958,7 +958,7 @@
       a regular expression object or a string containing a regular expression
       suitable for use by :func:`re.search`.  Examples::
 
-         self.assertRaisesRegexp(ValueError, 'invalid literal for.*XYZ$',
+         self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$",
                                  int, 'XYZ')
 
       or::

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 19:16:45 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 19:16:45 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjM3?=
 =?utf-8?q?=3A_Fix_assertRaisesRegexp_error_caought_by_Jeff_Tratner=2E?=
Message-ID: <3bjM2j2C9Mz7Lks@mail.python.org>

http://hg.python.org/cpython/rev/b3d19f0494e7
changeset:   84372:b3d19f0494e7
branch:      3.3
parent:      84366:c15d0baac3d6
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 13:15:43 2013 -0400
summary:
  Issue #18237: Fix assertRaisesRegexp error caought by Jeff Tratner.

files:
  Doc/library/unittest.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -873,7 +873,7 @@
       a regular expression object or a string containing a regular expression
       suitable for use by :func:`re.search`.  Examples::
 
-         self.assertRaisesRegex(ValueError, 'invalid literal for.*XYZ$',
+         self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$",
                                 int, 'XYZ')
 
       or::

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 19:16:47 2013
From: python-checkins at python.org (terry.reedy)
Date: Sat, 29 Jun 2013 19:16:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bjM2l1K26z7Lks@mail.python.org>

http://hg.python.org/cpython/rev/b8d95dfcafa3
changeset:   84373:b8d95dfcafa3
parent:      84370:f3f38c84aebc
parent:      84372:b3d19f0494e7
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 13:16:17 2013 -0400
summary:
  Merge with 3.3

files:
  Doc/library/unittest.rst |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst
--- a/Doc/library/unittest.rst
+++ b/Doc/library/unittest.rst
@@ -950,7 +950,7 @@
       a regular expression object or a string containing a regular expression
       suitable for use by :func:`re.search`.  Examples::
 
-         self.assertRaisesRegex(ValueError, 'invalid literal for.*XYZ$',
+         self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$",
                                 int, 'XYZ')
 
       or::

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 20:41:22 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 20:41:22 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_memory_leak_in_endswit?=
	=?utf-8?q?h?=
Message-ID: <3bjNwL58hPz7LlV@mail.python.org>

http://hg.python.org/cpython/rev/787b0f99d77b
changeset:   84374:787b0f99d77b
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 20:41:06 2013 +0200
summary:
  Fix memory leak in endswith
CID 1040368 (#1 of 1): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable substring going out of scope leaks the storage it points to.

files:
  Objects/unicodeobject.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -12941,9 +12941,9 @@
         return NULL;
     }
     result = tailmatch(self, substring, start, end, +1);
+    Py_DECREF(substring);
     if (result == -1)
         return NULL;
-    Py_DECREF(substring);
     return PyBool_FromLong(result);
 }
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 20:43:22 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 20:43:22 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_memory_leak_in_pyexpat?=
	=?utf-8?q?_PyUnknownEncodingHandler?=
Message-ID: <3bjNyf4jvKz7Llt@mail.python.org>

http://hg.python.org/cpython/rev/bd0834b59828
changeset:   84375:bd0834b59828
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 20:43:13 2013 +0200
summary:
  Fix memory leak in pyexpat PyUnknownEncodingHandler
CID 1040367 (#1 of 1): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable u going out of scope leaks the storage it points to.

files:
  Modules/pyexpat.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c
--- a/Modules/pyexpat.c
+++ b/Modules/pyexpat.c
@@ -1128,8 +1128,10 @@
     }
 
     u = PyUnicode_Decode((char*) template_buffer, 256, name, "replace");
-    if (u == NULL || PyUnicode_READY(u))
+    if (u == NULL || PyUnicode_READY(u)) {
+        Py_DECREF(u);
         return XML_STATUS_ERROR;
+    }
 
     if (PyUnicode_GET_LENGTH(u) != 256) {
         Py_DECREF(u);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 20:52:43 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 20:52:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_NULL_ptr_dereferencing?=
 =?utf-8?q?_in_local=5Ftimezone=28=29=2E_nameo_can_be_NULL?=
Message-ID: <3bjP9R1MMbz7Llh@mail.python.org>

http://hg.python.org/cpython/rev/ecc7f6f069f4
changeset:   84376:ecc7f6f069f4
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 20:52:33 2013 +0200
summary:
  Fix NULL ptr dereferencing in local_timezone(). nameo can be NULL
CID 1040362 (#1 of 1): Explicit null dereferenced (FORWARD_NULL)
var_deref_op: Dereferencing null pointer _py_decref_tmp.

files:
  Modules/_datetimemodule.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -4749,7 +4749,7 @@
             goto error;
     }
     result = new_timezone(delta, nameo);
-    Py_DECREF(nameo);
+    Py_XDECREF(nameo);
   error:
     Py_DECREF(delta);
     return result;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 21:03:59 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 21:03:59 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_resource_leak_in_parse?=
	=?utf-8?q?r=2C_free_node_ptr?=
Message-ID: <3bjPQR3qmpz7Lls@mail.python.org>

http://hg.python.org/cpython/rev/117752c21072
changeset:   84377:117752c21072
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 21:03:51 2013 +0200
summary:
  Fix resource leak in parser, free node ptr
CID 1028068 (#1 of 1): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable n going out of scope leaks the storage it points to.

files:
  Parser/pgenmain.c |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/Parser/pgenmain.c b/Parser/pgenmain.c
--- a/Parser/pgenmain.c
+++ b/Parser/pgenmain.c
@@ -113,6 +113,7 @@
         Py_Exit(1);
     }
     g = pgen(n);
+    PyNode_Free(n);
     if (g == NULL) {
         printf("Bad grammar.\n");
         Py_Exit(1);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 21:17:43 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 21:17:43 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_ref_leak_in_error_case?=
 =?utf-8?q?_of_unicode_rindex_and_rfind?=
Message-ID: <3bjPkH0tGrz7LkH@mail.python.org>

http://hg.python.org/cpython/rev/d0fb17896c95
changeset:   84378:d0fb17896c95
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 21:17:34 2013 +0200
summary:
  Fix ref leak in error case of unicode rindex and rfind
CID 983320: Resource leak (RESOURCE_LEAK)
CID 983321: Resource leak (RESOURCE_LEAK)
leaked_storage: Variable substring going out of scope leaks the storage it points to.

files:
  Objects/unicodeobject.c |  24 ++++++++++++++++--------
  1 files changed, 16 insertions(+), 8 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -12248,10 +12248,14 @@
                                             &start, &end))
         return NULL;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-    if (PyUnicode_READY(substring) == -1)
-        return NULL;
+    if (PyUnicode_READY(self) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
+    if (PyUnicode_READY(substring) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
 
     result = any_find_slice(-1, self, substring, start, end);
 
@@ -12280,10 +12284,14 @@
                                             &start, &end))
         return NULL;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-    if (PyUnicode_READY(substring) == -1)
-        return NULL;
+    if (PyUnicode_READY(self) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
+    if (PyUnicode_READY(substring) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
 
     result = any_find_slice(-1, self, substring, start, end);
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 21:21:46 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 21:21:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_ref_leak_in_error_case?=
	=?utf-8?q?_of_unicode_index?=
Message-ID: <3bjPpy1FYxz7Lkm@mail.python.org>

http://hg.python.org/cpython/rev/ec036132cc28
changeset:   84379:ec036132cc28
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 21:21:37 2013 +0200
summary:
  Fix ref leak in error case of unicode index
CID 983319 (#1 of 2): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable substring going out of scope leaks the storage it points to.

files:
  Objects/unicodeobject.c |  12 ++++++++----
  1 files changed, 8 insertions(+), 4 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -11180,10 +11180,14 @@
                                             &start, &end))
         return NULL;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-    if (PyUnicode_READY(substring) == -1)
-        return NULL;
+    if (PyUnicode_READY(self) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
+    if (PyUnicode_READY(substring) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
 
     result = any_find_slice(1, self, substring, start, end);
 

-- 
Repository URL: http://hg.python.org/cpython

From rdmurray at bitdance.com  Sat Jun 29 21:29:09 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 29 Jun 2013 15:29:09 -0400
Subject: [Python-checkins]
	=?utf-8?q?cpython=3A_Fix_memory_leak_in_pyexpat?=
	=?utf-8?q?_PyUnknownEncodingHandler?=
In-Reply-To: <3bjNyf4jvKz7Llt@mail.python.org>
References: <3bjNyf4jvKz7Llt@mail.python.org>
Message-ID: <20130629192910.333AD2500C5@webabinitio.net>

I don't know if this is the commit at fault or not, but we are seeing
segfaults in test_xml_etree on the buildbots now.

--David

On Sat, 29 Jun 2013 20:43:22 +0200, christian.heimes <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/bd0834b59828
> changeset:   84375:bd0834b59828
> user:        Christian Heimes <christian at cheimes.de>
> date:        Sat Jun 29 20:43:13 2013 +0200
> summary:
>   Fix memory leak in pyexpat PyUnknownEncodingHandler
> CID 1040367 (#1 of 1): Resource leak (RESOURCE_LEAK)
> leaked_storage: Variable u going out of scope leaks the storage it points to.
> 
> files:
>   Modules/pyexpat.c |  4 +++-
>   1 files changed, 3 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c
> --- a/Modules/pyexpat.c
> +++ b/Modules/pyexpat.c
> @@ -1128,8 +1128,10 @@
>      }
>  
>      u = PyUnicode_Decode((char*) template_buffer, 256, name, "replace");
> -    if (u == NULL || PyUnicode_READY(u))
> +    if (u == NULL || PyUnicode_READY(u)) {
> +        Py_DECREF(u);
>          return XML_STATUS_ERROR;
> +    }
>  
>      if (PyUnicode_GET_LENGTH(u) != 256) {
>          Py_DECREF(u);
> 
> -- 
> Repository URL: http://hg.python.org/cpython
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins

From python-checkins at python.org  Sat Jun 29 21:33:49 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 21:33:49 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_ref_leak_in_error_case?=
 =?utf-8?q?_of_unicode_find=2C_count=2C_formatlong?=
Message-ID: <3bjQ4s4cMXz7LkD@mail.python.org>

http://hg.python.org/cpython/rev/cad335e0b205
changeset:   84380:cad335e0b205
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 21:33:36 2013 +0200
summary:
  Fix ref leak in error case of unicode find, count, formatlong
CID 983315: Resource leak (RESOURCE_LEAK)
CID 983316: Resource leak (RESOURCE_LEAK)
CID 983317: Resource leak (RESOURCE_LEAK)

files:
  Objects/unicodeobject.c |  18 +++++++++++++-----
  1 files changed, 13 insertions(+), 5 deletions(-)


diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -10875,8 +10875,10 @@
 
     kind1 = PyUnicode_KIND(self);
     kind2 = PyUnicode_KIND(substring);
-    if (kind2 > kind1)
+    if (kind2 > kind1) {
+        Py_DECREF(substring);
         return PyLong_FromLong(0);
+    }
     kind = kind1;
     buf1 = PyUnicode_DATA(self);
     buf2 = PyUnicode_DATA(substring);
@@ -11054,10 +11056,14 @@
                                             &start, &end))
         return NULL;
 
-    if (PyUnicode_READY(self) == -1)
-        return NULL;
-    if (PyUnicode_READY(substring) == -1)
-        return NULL;
+    if (PyUnicode_READY(self) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
+    if (PyUnicode_READY(substring) == -1) {
+        Py_DECREF(substring);
+        return NULL;
+    }
 
     result = any_find_slice(1, self, substring, start, end);
 
@@ -13581,12 +13587,14 @@
 
     /* To modify the string in-place, there can only be one reference. */
     if (Py_REFCNT(result) != 1) {
+        Py_DECREF(result);
         PyErr_BadInternalCall();
         return NULL;
     }
     buf = PyUnicode_DATA(result);
     llen = PyUnicode_GET_LENGTH(result);
     if (llen > INT_MAX) {
+        Py_DECREF(result);
         PyErr_SetString(PyExc_ValueError,
                         "string too large in _PyBytes_FormatLong");
         return NULL;

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 21:37:44 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 21:37:44 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_resource_leak_in_pickl?=
	=?utf-8?q?e_module?=
Message-ID: <3bjQ9N22qjz7LlS@mail.python.org>

http://hg.python.org/cpython/rev/ac7bc6700ac3
changeset:   84381:ac7bc6700ac3
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 21:37:34 2013 +0200
summary:
  Fix resource leak in pickle module
CID 983309 (#1 of 1): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable unicode_str going out of scope leaks the storage it points to.

files:
  Modules/_pickle.c |  4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)


diff --git a/Modules/_pickle.c b/Modules/_pickle.c
--- a/Modules/_pickle.c
+++ b/Modules/_pickle.c
@@ -1748,8 +1748,10 @@
                 return -1;
             if (latin1 == NULL) {
                 latin1 = PyUnicode_InternFromString("latin1");
-                if (latin1 == NULL)
+                if (latin1 == NULL) {
+                    Py_DECREF(unicode_str);
                     return -1;
+                }
             }
             reduce_value = Py_BuildValue("(O(OO))",
                                          codecs_encode, unicode_str, latin1);

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sat Jun 29 21:49:45 2013
From: python-checkins at python.org (christian.heimes)
Date: Sat, 29 Jun 2013 21:49:45 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_segfault_in_pyexpat=2E?=
	=?utf-8?q?c_caused_by_84375?=
Message-ID: <3bjQRF5G7tzT2Q@mail.python.org>

http://hg.python.org/cpython/rev/4c7426c3ceab
changeset:   84382:4c7426c3ceab
user:        Christian Heimes <christian at cheimes.de>
date:        Sat Jun 29 21:49:27 2013 +0200
summary:
  Fix segfault in pyexpat.c caused by 84375
u can be NULL, use XDECREF

files:
  Modules/pyexpat.c |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c
--- a/Modules/pyexpat.c
+++ b/Modules/pyexpat.c
@@ -1129,7 +1129,7 @@
 
     u = PyUnicode_Decode((char*) template_buffer, 256, name, "replace");
     if (u == NULL || PyUnicode_READY(u)) {
-        Py_DECREF(u);
+        Py_XDECREF(u);
         return XML_STATUS_ERROR;
     }
 

-- 
Repository URL: http://hg.python.org/cpython

From christian at python.org  Sat Jun 29 21:51:19 2013
From: christian at python.org (Christian Heimes)
Date: Sat, 29 Jun 2013 21:51:19 +0200
Subject: [Python-checkins] cpython: Fix memory leak in pyexpat
	PyUnknownEncodingHandler
In-Reply-To: <20130629192910.333AD2500C5@webabinitio.net>
References: <3bjNyf4jvKz7Llt@mail.python.org>
	<20130629192910.333AD2500C5@webabinitio.net>
Message-ID: <51CF3AB7.8020308@python.org>

Am 29.06.2013 21:29, schrieb R. David Murray:
> I don't know if this is the commit at fault or not, but we are seeing
> segfaults in test_xml_etree on the buildbots now.

Yeah, it's my fault. Thanks!

>>      u = PyUnicode_Decode((char*) template_buffer, 256, name, "replace");
>> -    if (u == NULL || PyUnicode_READY(u))wr
>> +        Py_DECREF(u);
>>          return XML_STATUS_ERROR;
>> +    }

Py_DECREF() is wrong as u can be NULL. I have changed the code to use
Py_XDECREF().


From storchaka at gmail.com  Sat Jun 29 23:43:06 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 30 Jun 2013 00:43:06 +0300
Subject: [Python-checkins] cpython: Fix memory leak in pyexpat
	PyUnknownEncodingHandler
In-Reply-To: <51CF3AB7.8020308@python.org>
References: <3bjNyf4jvKz7Llt@mail.python.org>
	<20130629192910.333AD2500C5@webabinitio.net>
	<51CF3AB7.8020308@python.org>
Message-ID: <kqnkd7$r0s$1@ger.gmane.org>

29.06.13 22:51, Christian Heimes ???????(??):
> Am 29.06.2013 21:29, schrieb R. David Murray:
>> I don't know if this is the commit at fault or not, but we are seeing
>> segfaults in test_xml_etree on the buildbots now.
>
> Yeah, it's my fault. Thanks!
>
>>>       u = PyUnicode_Decode((char*) template_buffer, 256, name, "replace");
>>> -    if (u == NULL || PyUnicode_READY(u))wr
>>> +        Py_DECREF(u);
>>>           return XML_STATUS_ERROR;
>>> +    }
>
> Py_DECREF() is wrong as u can be NULL. I have changed the code to use
> Py_XDECREF().
>

Oh, there is yet one blindly inherited bug. Should be:

     if (u == NULL || !PyUnicode_READY(u)) {



From storchaka at gmail.com  Sat Jun 29 23:50:00 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 30 Jun 2013 00:50:00 +0300
Subject: [Python-checkins] cpython: Fix memory leak in pyexpat
	PyUnknownEncodingHandler
In-Reply-To: <kqnkd7$r0s$1@ger.gmane.org>
References: <3bjNyf4jvKz7Llt@mail.python.org>
	<20130629192910.333AD2500C5@webabinitio.net>
	<51CF3AB7.8020308@python.org> <kqnkd7$r0s$1@ger.gmane.org>
Message-ID: <kqnkq0$t7t$1@ger.gmane.org>

30.06.13 00:43, Serhiy Storchaka ???????(??):
> 29.06.13 22:51, Christian Heimes ???????(??):
>> Am 29.06.2013 21:29, schrieb R. David Murray:
>>> I don't know if this is the commit at fault or not, but we are seeing
>>> segfaults in test_xml_etree on the buildbots now.
>>
>> Yeah, it's my fault. Thanks!
>>
>>>>       u = PyUnicode_Decode((char*) template_buffer, 256, name,
>>>> "replace");
>>>> -    if (u == NULL || PyUnicode_READY(u))wr
>>>> +        Py_DECREF(u);
>>>>           return XML_STATUS_ERROR;
>>>> +    }
>>
>> Py_DECREF() is wrong as u can be NULL. I have changed the code to use
>> Py_XDECREF().
>>
>
> Oh, there is yet one blindly inherited bug. Should be:
>
>      if (u == NULL || !PyUnicode_READY(u)) {

Oh, no, I were wrong. PyUnicode_READY() returns false (0) on success and 
true (-1) on fail.



From python-checkins at python.org  Sun Jun 30 00:30:46 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 00:30:46 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MTAz?=
 =?utf-8?q?=3A_Update_README=2Etxt_and_test=5Fidle_to_describe_and_run_gui?=
 =?utf-8?q?_tests=2E?=
Message-ID: <3bjV120tG6z7Ljm@mail.python.org>

http://hg.python.org/cpython/rev/c818c215f1a4
changeset:   84383:c818c215f1a4
branch:      3.3
parent:      84372:b3d19f0494e7
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 18:22:02 2013 -0400
summary:
  Issue #18103: Update README.txt and test_idle to describe and run gui tests.

files:
  Lib/idlelib/idle_test/README.txt |  93 +++++++++++++------
  Lib/test/test_idle.py            |   5 +
  2 files changed, 70 insertions(+), 28 deletions(-)


diff --git a/Lib/idlelib/idle_test/README.txt b/Lib/idlelib/idle_test/README.txt
--- a/Lib/idlelib/idle_test/README.txt
+++ b/Lib/idlelib/idle_test/README.txt
@@ -1,14 +1,19 @@
 README FOR IDLE TESTS IN IDLELIB.IDLE_TEST
 
+
+1. Test Files
+
 The idle directory, idlelib, has over 60 xyz.py files. The idle_test
-subdirectory should contain a test_xyy.py for each one. (For test modules,
-make 'xyz' lower case.) Each should start with the following cut-paste
-template, with the blanks after after '.'. 'as', and '_' filled in.
+subdirectory should contain a test_xyy.py for each. (For test modules, make
+'xyz' lower case, and possibly shorten it.) Each file should start with the
+something like the following template, with the blanks after after '.' and 'as',
+and before and after '_' filled in.
 ---
 import unittest
-import idlelib. as 
+from test.support import requires
+import idlelib. as
 
-class Test_(unittest.TestCase):
+class _Test(unittest.TestCase):
 
     def test_(self):
 
@@ -21,43 +26,75 @@
 with xyz (lowercased) added after 'test_'.
 ---
 if __name__ == "__main__":
+    from test import support; support.use_resources = ['gui']
     import unittest
     unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False)
 ---
 
-In Idle, pressing F5 in an editor window with either xyz.py or test_xyz.py
-loaded will then run the test with the version of Python running Idle and
-tracebacks will appear in the Shell window. The options are appropriate for
-developers running (as opposed to importing) either type of file during
-development: verbosity=2 lists all test_y methods; exit=False avoids a
-spurious sys.exit traceback when running in Idle. The following command
-lines also run test_xyz.py
+
+2. Gui Tests
+
+Gui tests need 'requires' and 'use_resources' from test.support
+(test.test_support in 2.7). A test is a gui test if it creates a Tk root or
+master object either directly or indirectly by instantiating a tkinter or
+idle class. For the benefit of buildbot machines that do not have a graphics
+screen, gui tests must be 'guarded' by "requires('gui')" in a setUp
+function or method. This will typically be setUpClass.
+---
+    @classmethod
+    def setUpClass(cls):
+        requires('gui')
+---
+All gui objects must be destroyed by the end of the test, perhaps in a tearDown
+function.
+
+Support.requires('gui') returns true if it is either called in a main module
+(which never happens on buildbots) or if use_resources contains 'gui'.
+Use_resources is set by test.regrtest but not by unittest. So when running
+tests in another module with unittest, we set it ourselves, as in the xyz.py
+template above.
+
+Since non-gui tests always run, but gui tests only sometimes, tests of non-gui
+operations should best avoid needing a gui. Methods that make incidental use of
+tkinter variables and messageboxes can do this by using the mock classes in
+idle_test/mock_tk.py.
+
+
+3. Running Tests
+
+Assume that xyz.py and test_xyz.py end with the "if __name__" statements given
+above. In Idle, pressing F5 in an editor window with either loaded will run all
+tests in the test_xyz file with the version of Python running Idle.  The test
+report and any tracebacks will appear in the Shell window. The options in these
+"if __name__" statements are appropriate for developers running (as opposed to
+importing) either of the files during development: verbosity=2 lists all test
+methods in the file; exit=False avoids a spurious sys.exit traceback that would
+otherwise occur when running in Idle. The following command lines also run
+all test methods, including gui tests, in test_xyz.py. (The exceptions are that
+idlelib and idlelib.idle start Idle and idlelib.PyShell should (issue 18330).)
 
 python -m idlelib.xyz  # With the capitalization of the xyz module
-python -m unittest -v idlelib.idle_test.test_xyz
+python -m idlelib.idle_test.test_xyz
 
-To run all idle tests either interactively ('>>>', with unittest imported)
-or from a command line, use one of the following.
+To run all idle_test/test_*.py tests, either interactively
+('>>>', with unittest imported) or from a command line, use one of the
+following. (Notes: unittest does not run gui tests; in 2.7, 'test ' (with the
+space) is 'test.regrtest '; where present, -v and -ugui can be omitted.)
 
 >>> unittest.main('idlelib.idle_test', verbosity=2, exit=False)
 python -m unittest -v idlelib.idle_test
+python -m test -v -ugui test_idle
 python -m test.test_idle
-python -m test test_idle
 
-The idle tests are 'discovered' in idlelib.idle_test.__init__.load_tests,
+The idle tests are 'discovered' by idlelib.idle_test.__init__.load_tests,
 which is also imported into test.test_idle. Normally, neither file should be
-changed when working on individual test modules. The last command runs runs
+changed when working on individual test modules. The third command runs runs
 unittest indirectly through regrtest. The same happens when the entire test
-suite is run with 'python -m test'. So it must work for buildbots to stay green.
+suite is run with 'python -m test'. So that command must work for buildbots
+to stay green. Idle tests must not disturb the environment in a way that
+makes other tests fail (issue 18081).
 
-To run an individual Testcase or test method, extend the
-dotted name given to unittest on the command line.
+To run an individual Testcase or test method, extend the dotted name given to
+unittest on the command line. (But gui tests will not this way.)
 
 python -m unittest -v idlelib.idle_test.text_xyz.Test_case.test_meth
-
-To disable test/test_idle.py, there are at least two choices.
-a. Comment out 'load_tests' line, no no tests are discovered (simple and safe);
-Running no tests passes, so there is no indication that nothing was run.
-b.Before that line, make module an unexpected skip for regrtest with
-import unittest; raise unittest.SkipTest('skip for buildbots')
-When run directly with unittest, this causes a normal exit and traceback.
\ No newline at end of file
diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py
--- a/Lib/test/test_idle.py
+++ b/Lib/test/test_idle.py
@@ -10,5 +10,10 @@
 load_tests = idletest.load_tests
 
 if __name__ == '__main__':
+    # Until unittest supports resources, we emulate regrtest's -ugui
+    # so loaded tests run the same as if textually present here.
+    # If any Idle test ever needs another resource, add it to the list.
+    from test import support
+    support.use_resources = ['gui']  # use_resources is initially None
     import unittest
     unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 00:30:47 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 00:30:47 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bjV133yxXz7Llr@mail.python.org>

http://hg.python.org/cpython/rev/171561855659
changeset:   84384:171561855659
parent:      84382:4c7426c3ceab
parent:      84383:c818c215f1a4
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 18:22:25 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/idlelib/idle_test/README.txt |  93 +++++++++++++------
  Lib/test/test_idle.py            |   5 +
  2 files changed, 70 insertions(+), 28 deletions(-)


diff --git a/Lib/idlelib/idle_test/README.txt b/Lib/idlelib/idle_test/README.txt
--- a/Lib/idlelib/idle_test/README.txt
+++ b/Lib/idlelib/idle_test/README.txt
@@ -1,14 +1,19 @@
 README FOR IDLE TESTS IN IDLELIB.IDLE_TEST
 
+
+1. Test Files
+
 The idle directory, idlelib, has over 60 xyz.py files. The idle_test
-subdirectory should contain a test_xyy.py for each one. (For test modules,
-make 'xyz' lower case.) Each should start with the following cut-paste
-template, with the blanks after after '.'. 'as', and '_' filled in.
+subdirectory should contain a test_xyy.py for each. (For test modules, make
+'xyz' lower case, and possibly shorten it.) Each file should start with the
+something like the following template, with the blanks after after '.' and 'as',
+and before and after '_' filled in.
 ---
 import unittest
-import idlelib. as 
+from test.support import requires
+import idlelib. as
 
-class Test_(unittest.TestCase):
+class _Test(unittest.TestCase):
 
     def test_(self):
 
@@ -21,43 +26,75 @@
 with xyz (lowercased) added after 'test_'.
 ---
 if __name__ == "__main__":
+    from test import support; support.use_resources = ['gui']
     import unittest
     unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False)
 ---
 
-In Idle, pressing F5 in an editor window with either xyz.py or test_xyz.py
-loaded will then run the test with the version of Python running Idle and
-tracebacks will appear in the Shell window. The options are appropriate for
-developers running (as opposed to importing) either type of file during
-development: verbosity=2 lists all test_y methods; exit=False avoids a
-spurious sys.exit traceback when running in Idle. The following command
-lines also run test_xyz.py
+
+2. Gui Tests
+
+Gui tests need 'requires' and 'use_resources' from test.support
+(test.test_support in 2.7). A test is a gui test if it creates a Tk root or
+master object either directly or indirectly by instantiating a tkinter or
+idle class. For the benefit of buildbot machines that do not have a graphics
+screen, gui tests must be 'guarded' by "requires('gui')" in a setUp
+function or method. This will typically be setUpClass.
+---
+    @classmethod
+    def setUpClass(cls):
+        requires('gui')
+---
+All gui objects must be destroyed by the end of the test, perhaps in a tearDown
+function.
+
+Support.requires('gui') returns true if it is either called in a main module
+(which never happens on buildbots) or if use_resources contains 'gui'.
+Use_resources is set by test.regrtest but not by unittest. So when running
+tests in another module with unittest, we set it ourselves, as in the xyz.py
+template above.
+
+Since non-gui tests always run, but gui tests only sometimes, tests of non-gui
+operations should best avoid needing a gui. Methods that make incidental use of
+tkinter variables and messageboxes can do this by using the mock classes in
+idle_test/mock_tk.py.
+
+
+3. Running Tests
+
+Assume that xyz.py and test_xyz.py end with the "if __name__" statements given
+above. In Idle, pressing F5 in an editor window with either loaded will run all
+tests in the test_xyz file with the version of Python running Idle.  The test
+report and any tracebacks will appear in the Shell window. The options in these
+"if __name__" statements are appropriate for developers running (as opposed to
+importing) either of the files during development: verbosity=2 lists all test
+methods in the file; exit=False avoids a spurious sys.exit traceback that would
+otherwise occur when running in Idle. The following command lines also run
+all test methods, including gui tests, in test_xyz.py. (The exceptions are that
+idlelib and idlelib.idle start Idle and idlelib.PyShell should (issue 18330).)
 
 python -m idlelib.xyz  # With the capitalization of the xyz module
-python -m unittest -v idlelib.idle_test.test_xyz
+python -m idlelib.idle_test.test_xyz
 
-To run all idle tests either interactively ('>>>', with unittest imported)
-or from a command line, use one of the following.
+To run all idle_test/test_*.py tests, either interactively
+('>>>', with unittest imported) or from a command line, use one of the
+following. (Notes: unittest does not run gui tests; in 2.7, 'test ' (with the
+space) is 'test.regrtest '; where present, -v and -ugui can be omitted.)
 
 >>> unittest.main('idlelib.idle_test', verbosity=2, exit=False)
 python -m unittest -v idlelib.idle_test
+python -m test -v -ugui test_idle
 python -m test.test_idle
-python -m test test_idle
 
-The idle tests are 'discovered' in idlelib.idle_test.__init__.load_tests,
+The idle tests are 'discovered' by idlelib.idle_test.__init__.load_tests,
 which is also imported into test.test_idle. Normally, neither file should be
-changed when working on individual test modules. The last command runs runs
+changed when working on individual test modules. The third command runs runs
 unittest indirectly through regrtest. The same happens when the entire test
-suite is run with 'python -m test'. So it must work for buildbots to stay green.
+suite is run with 'python -m test'. So that command must work for buildbots
+to stay green. Idle tests must not disturb the environment in a way that
+makes other tests fail (issue 18081).
 
-To run an individual Testcase or test method, extend the
-dotted name given to unittest on the command line.
+To run an individual Testcase or test method, extend the dotted name given to
+unittest on the command line. (But gui tests will not this way.)
 
 python -m unittest -v idlelib.idle_test.text_xyz.Test_case.test_meth
-
-To disable test/test_idle.py, there are at least two choices.
-a. Comment out 'load_tests' line, no no tests are discovered (simple and safe);
-Running no tests passes, so there is no indication that nothing was run.
-b.Before that line, make module an unexpected skip for regrtest with
-import unittest; raise unittest.SkipTest('skip for buildbots')
-When run directly with unittest, this causes a normal exit and traceback.
\ No newline at end of file
diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py
--- a/Lib/test/test_idle.py
+++ b/Lib/test/test_idle.py
@@ -10,5 +10,10 @@
 load_tests = idletest.load_tests
 
 if __name__ == '__main__':
+    # Until unittest supports resources, we emulate regrtest's -ugui
+    # so loaded tests run the same as if textually present here.
+    # If any Idle test ever needs another resource, add it to the list.
+    from test import support
+    support.use_resources = ['gui']  # use_resources is initially None
     import unittest
     unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 00:41:18 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 00:41:18 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE4MTAz?=
 =?utf-8?q?=3A_Update_README=2Etxt_and_test=5Fidle_to_describe_and_run_gui?=
 =?utf-8?q?_tests=2E?=
Message-ID: <3bjVFB6GhBz7Ljm@mail.python.org>

http://hg.python.org/cpython/rev/0767363a0393
changeset:   84385:0767363a0393
branch:      2.7
parent:      84371:4a714fea95ef
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sat Jun 29 18:22:02 2013 -0400
summary:
  Issue #18103: Update README.txt and test_idle to describe and run gui tests.

files:
  Lib/idlelib/idle_test/README.txt |  91 ++++++++++++++-----
  Lib/test/test_idle.py            |   5 +
  2 files changed, 69 insertions(+), 27 deletions(-)


diff --git a/Lib/idlelib/idle_test/README.txt b/Lib/idlelib/idle_test/README.txt
--- a/Lib/idlelib/idle_test/README.txt
+++ b/Lib/idlelib/idle_test/README.txt
@@ -1,14 +1,19 @@
 README FOR IDLE TESTS IN IDLELIB.IDLE_TEST
 
+
+1. Test Files
+
 The idle directory, idlelib, has over 60 xyz.py files. The idle_test
-subdirectory should contain a test_xyy.py for each one. (For test modules,
-make 'xyz' lower case.) Each should start with the following cut-paste
-template, with the blanks after after '.'. 'as', and '_' filled in.
+subdirectory should contain a test_xyy.py for each. (For test modules, make
+'xyz' lower case, and possibly shorten it.) Each file should start with the
+something like the following template, with the blanks after after '.' and 'as',
+and before and after '_' filled in.
 ---
 import unittest
+from test.support import requires
 import idlelib. as
 
-class Test_(unittest.TestCase):
+class _Test(unittest.TestCase):
 
     def test_(self):
 
@@ -21,43 +26,75 @@
 with xyz (lowercased) added after 'test_'.
 ---
 if __name__ == "__main__":
+    from test import support; support.use_resources = ['gui']
     import unittest
     unittest.main('idlelib.idle_test.test_', verbosity=2, exit=False)
 ---
 
-In Idle, pressing F5 in an editor window with either xyz.py or test_xyz.py
-loaded will then run the test with the version of Python running Idle and
-tracebacks will appear in the Shell window. The options are appropriate for
-developers running (as opposed to importing) either type of file during
-development: verbosity=2 lists all test_y methods; exit=False avoids a
-spurious sys.exit traceback when running in Idle. The following command
-lines also run test_xyz.py
+
+2. Gui Tests
+
+Gui tests need 'requires' and 'use_resources' from test.support
+(test.test_support in 2.7). A test is a gui test if it creates a Tk root or
+master object either directly or indirectly by instantiating a tkinter or
+idle class. For the benefit of buildbot machines that do not have a graphics
+screen, gui tests must be 'guarded' by "requires('gui')" in a setUp
+function or method. This will typically be setUpClass.
+---
+    @classmethod
+    def setUpClass(cls):
+        requires('gui')
+---
+All gui objects must be destroyed by the end of the test, perhaps in a tearDown
+function.
+
+Support.requires('gui') returns true if it is either called in a main module
+(which never happens on buildbots) or if use_resources contains 'gui'.
+Use_resources is set by test.regrtest but not by unittest. So when running
+tests in another module with unittest, we set it ourselves, as in the xyz.py
+template above.
+
+Since non-gui tests always run, but gui tests only sometimes, tests of non-gui
+operations should best avoid needing a gui. Methods that make incidental use of
+tkinter variables and messageboxes can do this by using the mock classes in
+idle_test/mock_tk.py.
+
+
+3. Running Tests
+
+Assume that xyz.py and test_xyz.py end with the "if __name__" statements given
+above. In Idle, pressing F5 in an editor window with either loaded will run all
+tests in the test_xyz file with the version of Python running Idle.  The test
+report and any tracebacks will appear in the Shell window. The options in these
+"if __name__" statements are appropriate for developers running (as opposed to
+importing) either of the files during development: verbosity=2 lists all test
+methods in the file; exit=False avoids a spurious sys.exit traceback that would
+otherwise occur when running in Idle. The following command lines also run
+all test methods, including gui tests, in test_xyz.py. (The exceptions are that
+idlelib and idlelib.idle start Idle and idlelib.PyShell should (issue 18330).)
 
 python -m idlelib.xyz  # With the capitalization of the xyz module
-python -m unittest -v idlelib.idle_test.test_xyz
+python -m idlelib.idle_test.test_xyz
 
-To run all idle tests either interactively ('>>>', with unittest imported)
-or from a command line, use one of the following.
+To run all idle_test/test_*.py tests, either interactively
+('>>>', with unittest imported) or from a command line, use one of the
+following. (Notes: unittest does not run gui tests; in 2.7, 'test ' (with the
+space) is 'test.regrtest '; where present, -v and -ugui can be omitted.)
 
 >>> unittest.main('idlelib.idle_test', verbosity=2, exit=False)
 python -m unittest -v idlelib.idle_test
+python -m test -v -ugui test_idle
 python -m test.test_idle
-python -m test test_idle
 
-The idle tests are 'discovered' in idlelib.idle_test.__init__.load_tests,
+The idle tests are 'discovered' by idlelib.idle_test.__init__.load_tests,
 which is also imported into test.test_idle. Normally, neither file should be
-changed when working on individual test modules. The last command runs runs
+changed when working on individual test modules. The third command runs runs
 unittest indirectly through regrtest. The same happens when the entire test
-suite is run with 'python -m test'. So it must work for buildbots to stay green.
+suite is run with 'python -m test'. So that command must work for buildbots
+to stay green. Idle tests must not disturb the environment in a way that
+makes other tests fail (issue 18081).
 
-To run an individual Testcase or test method, extend the
-dotted name given to unittest on the command line.
+To run an individual Testcase or test method, extend the dotted name given to
+unittest on the command line. (But gui tests will not this way.)
 
 python -m unittest -v idlelib.idle_test.text_xyz.Test_case.test_meth
-
-To disable test/test_idle.py, there are at least two choices.
-a. Comment out 'load_tests' line, no no tests are discovered (simple and safe);
-Running no tests passes, so there is no indication that nothing was run.
-b.Before that line, make module an unexpected skip for regrtest with
-import unittest; raise unittest.SkipTest('skip for buildbots')
-When run directly with unittest, this causes a normal exit and traceback.
\ No newline at end of file
diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py
--- a/Lib/test/test_idle.py
+++ b/Lib/test/test_idle.py
@@ -10,5 +10,10 @@
 load_tests = idletest.load_tests
 
 if __name__ == '__main__':
+    # Until unittest supports resources, we emulate regrtest's -ugui
+    # so loaded tests run the same as if textually present here.
+    # If any Idle test ever needs another resource, add it to the list.
+    from test import support
+    support.use_resources = ['gui']  # use_resources is initially None
     import unittest
     unittest.main(verbosity=2, exit=False)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 00:44:11 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 30 Jun 2013 00:44:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MTU1OiBSZWdl?=
 =?utf-8?q?x-escape_delimiter=2C_in_case_it_is_a_regex_special_char=2E?=
Message-ID: <3bjVJW3Bnrz7Ljm@mail.python.org>

http://hg.python.org/cpython/rev/68ff68f9a0d5
changeset:   84386:68ff68f9a0d5
branch:      3.3
parent:      84383:c818c215f1a4
user:        R David Murray <rdmurray at bitdance.com>
date:        Sat Jun 29 18:40:53 2013 -0400
summary:
  #18155: Regex-escape delimiter, in case it is a regex special char.

Patch by Vajrasky Kok, with slight modification to the tests by me.

files:
  Lib/csv.py           |   5 ++-
  Lib/test/test_csv.py |  42 +++++++++++++++++++++++++++++--
  Misc/ACKS            |   1 +
  Misc/NEWS            |   6 +++-
  4 files changed, 48 insertions(+), 6 deletions(-)


diff --git a/Lib/csv.py b/Lib/csv.py
--- a/Lib/csv.py
+++ b/Lib/csv.py
@@ -264,8 +264,9 @@
 
         # if we see an extra quote between delimiters, we've got a
         # double quoted format
-        dq_regexp = re.compile(r"((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)" % \
-                               {'delim':delim, 'quote':quotechar}, re.MULTILINE)
+        dq_regexp = re.compile(
+                               r"((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)" % \
+                               {'delim':re.escape(delim), 'quote':quotechar}, re.MULTILINE)
 
 
 
diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -796,7 +796,7 @@
 'Tommy''s Place':'Blue Island':'IL':'12/28/02':'Blue Sunday/White Crow'
 'Stonecutters ''Seafood'' and Chop House':'Lemont':'IL':'12/19/02':'Week Back'
 """
-    header = '''\
+    header1 = '''\
 "venue","city","state","date","performers"
 '''
     sample3 = '''\
@@ -815,10 +815,35 @@
     sample6 = "a|b|c\r\nd|e|f\r\n"
     sample7 = "'a'|'b'|'c'\r\n'd'|e|f\r\n"
 
+# Issue 18155: Use a delimiter that is a special char to regex:
+
+    header2 = '''\
+"venue"+"city"+"state"+"date"+"performers"
+'''
+    sample8 = """\
+Harry's+ Arlington Heights+ IL+ 2/1/03+ Kimi Hayes
+Shark City+ Glendale Heights+ IL+ 12/28/02+ Prezence
+Tommy's Place+ Blue Island+ IL+ 12/28/02+ Blue Sunday/White Crow
+Stonecutters Seafood and Chop House+ Lemont+ IL+ 12/19/02+ Week Back
+"""
+    sample9 = """\
+'Harry''s'+ Arlington Heights'+ 'IL'+ '2/1/03'+ 'Kimi Hayes'
+'Shark City'+ Glendale Heights'+' IL'+ '12/28/02'+ 'Prezence'
+'Tommy''s Place'+ Blue Island'+ 'IL'+ '12/28/02'+ 'Blue Sunday/White Crow'
+'Stonecutters ''Seafood'' and Chop House'+ 'Lemont'+ 'IL'+ '12/19/02'+ 'Week Back'
+"""
+
     def test_has_header(self):
         sniffer = csv.Sniffer()
         self.assertEqual(sniffer.has_header(self.sample1), False)
-        self.assertEqual(sniffer.has_header(self.header+self.sample1), True)
+        self.assertEqual(sniffer.has_header(self.header1 + self.sample1),
+                         True)
+
+    def test_has_header_regex_special_delimiter(self):
+        sniffer = csv.Sniffer()
+        self.assertEqual(sniffer.has_header(self.sample8), False)
+        self.assertEqual(sniffer.has_header(self.header2 + self.sample8),
+                         True)
 
     def test_sniff(self):
         sniffer = csv.Sniffer()
@@ -852,13 +877,24 @@
         dialect = sniffer.sniff(self.sample7)
         self.assertEqual(dialect.delimiter, "|")
         self.assertEqual(dialect.quotechar, "'")
+        dialect = sniffer.sniff(self.sample8)
+        self.assertEqual(dialect.delimiter, '+')
+        dialect = sniffer.sniff(self.sample9)
+        self.assertEqual(dialect.delimiter, '+')
+        self.assertEqual(dialect.quotechar, "'")
 
     def test_doublequote(self):
         sniffer = csv.Sniffer()
-        dialect = sniffer.sniff(self.header)
+        dialect = sniffer.sniff(self.header1)
+        self.assertFalse(dialect.doublequote)
+        dialect = sniffer.sniff(self.header2)
         self.assertFalse(dialect.doublequote)
         dialect = sniffer.sniff(self.sample2)
         self.assertTrue(dialect.doublequote)
+        dialect = sniffer.sniff(self.sample8)
+        self.assertFalse(dialect.doublequote)
+        dialect = sniffer.sniff(self.sample9)
+        self.assertTrue(dialect.doublequote)
 
 if not hasattr(sys, "gettotalrefcount"):
     if support.verbose: print("*** skipping leakage tests ***")
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -656,6 +656,7 @@
 Greg Kochanski
 Damon Kohler
 Marko Kohtala
+Vajrasky Kok
 Guido Kollerie
 Jacek Konieczny
 ???? ?????????
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -38,6 +38,10 @@
 Library
 -------
 
+- Issue #18155: The csv module now correctly handles csv files that use
+  a delimter character that has a special meaning in regexes, instead of
+  throwing an exception.
+
 - Issue #14360: encode_quopri can now be successfully used as an encoder
   when constructing a MIMEApplication object.
 
@@ -50,7 +54,7 @@
 
 - Issue #18259: Declare sethostname in socketmodule.c for AIX
 
-- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+- Issue #18167: cgi.FieldStorage no longer fails to handle multipart/form-data
   when \r\n appears at end of 65535 bytes without other newlines.
 
 - subprocess: Prevent a possible double close of parent pipe fds when the

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 00:44:12 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 30 Jun 2013 00:44:12 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_=2318155=3A_Regex-escape_delimiter=2C_in_case_it_i?=
 =?utf-8?q?s_a_regex_special_char=2E?=
Message-ID: <3bjVJX6GZ6z7Lll@mail.python.org>

http://hg.python.org/cpython/rev/acaf73e3d882
changeset:   84387:acaf73e3d882
parent:      84384:171561855659
parent:      84386:68ff68f9a0d5
user:        R David Murray <rdmurray at bitdance.com>
date:        Sat Jun 29 18:42:24 2013 -0400
summary:
  Merge #18155: Regex-escape delimiter, in case it is a regex special char.

files:
  Lib/csv.py           |   5 ++-
  Lib/test/test_csv.py |  42 +++++++++++++++++++++++++++++--
  Misc/NEWS            |   6 +++-
  3 files changed, 47 insertions(+), 6 deletions(-)


diff --git a/Lib/csv.py b/Lib/csv.py
--- a/Lib/csv.py
+++ b/Lib/csv.py
@@ -264,8 +264,9 @@
 
         # if we see an extra quote between delimiters, we've got a
         # double quoted format
-        dq_regexp = re.compile(r"((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)" % \
-                               {'delim':delim, 'quote':quotechar}, re.MULTILINE)
+        dq_regexp = re.compile(
+                               r"((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)" % \
+                               {'delim':re.escape(delim), 'quote':quotechar}, re.MULTILINE)
 
 
 
diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -805,7 +805,7 @@
 'Tommy''s Place':'Blue Island':'IL':'12/28/02':'Blue Sunday/White Crow'
 'Stonecutters ''Seafood'' and Chop House':'Lemont':'IL':'12/19/02':'Week Back'
 """
-    header = '''\
+    header1 = '''\
 "venue","city","state","date","performers"
 '''
     sample3 = '''\
@@ -824,10 +824,35 @@
     sample6 = "a|b|c\r\nd|e|f\r\n"
     sample7 = "'a'|'b'|'c'\r\n'd'|e|f\r\n"
 
+# Issue 18155: Use a delimiter that is a special char to regex:
+
+    header2 = '''\
+"venue"+"city"+"state"+"date"+"performers"
+'''
+    sample8 = """\
+Harry's+ Arlington Heights+ IL+ 2/1/03+ Kimi Hayes
+Shark City+ Glendale Heights+ IL+ 12/28/02+ Prezence
+Tommy's Place+ Blue Island+ IL+ 12/28/02+ Blue Sunday/White Crow
+Stonecutters Seafood and Chop House+ Lemont+ IL+ 12/19/02+ Week Back
+"""
+    sample9 = """\
+'Harry''s'+ Arlington Heights'+ 'IL'+ '2/1/03'+ 'Kimi Hayes'
+'Shark City'+ Glendale Heights'+' IL'+ '12/28/02'+ 'Prezence'
+'Tommy''s Place'+ Blue Island'+ 'IL'+ '12/28/02'+ 'Blue Sunday/White Crow'
+'Stonecutters ''Seafood'' and Chop House'+ 'Lemont'+ 'IL'+ '12/19/02'+ 'Week Back'
+"""
+
     def test_has_header(self):
         sniffer = csv.Sniffer()
         self.assertEqual(sniffer.has_header(self.sample1), False)
-        self.assertEqual(sniffer.has_header(self.header+self.sample1), True)
+        self.assertEqual(sniffer.has_header(self.header1 + self.sample1),
+                         True)
+
+    def test_has_header_regex_special_delimiter(self):
+        sniffer = csv.Sniffer()
+        self.assertEqual(sniffer.has_header(self.sample8), False)
+        self.assertEqual(sniffer.has_header(self.header2 + self.sample8),
+                         True)
 
     def test_sniff(self):
         sniffer = csv.Sniffer()
@@ -861,13 +886,24 @@
         dialect = sniffer.sniff(self.sample7)
         self.assertEqual(dialect.delimiter, "|")
         self.assertEqual(dialect.quotechar, "'")
+        dialect = sniffer.sniff(self.sample8)
+        self.assertEqual(dialect.delimiter, '+')
+        dialect = sniffer.sniff(self.sample9)
+        self.assertEqual(dialect.delimiter, '+')
+        self.assertEqual(dialect.quotechar, "'")
 
     def test_doublequote(self):
         sniffer = csv.Sniffer()
-        dialect = sniffer.sniff(self.header)
+        dialect = sniffer.sniff(self.header1)
+        self.assertFalse(dialect.doublequote)
+        dialect = sniffer.sniff(self.header2)
         self.assertFalse(dialect.doublequote)
         dialect = sniffer.sniff(self.sample2)
         self.assertTrue(dialect.doublequote)
+        dialect = sniffer.sniff(self.sample8)
+        self.assertFalse(dialect.doublequote)
+        dialect = sniffer.sniff(self.sample9)
+        self.assertTrue(dialect.doublequote)
 
 if not hasattr(sys, "gettotalrefcount"):
     if support.verbose: print("*** skipping leakage tests ***")
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -135,6 +135,10 @@
 Library
 -------
 
+- Issue #18155: The csv module now correctly handles csv files that use
+  a delimter character that has a special meaning in regexes, instead of
+  throwing an exception.
+
 - Issue #14360: encode_quopri can now be successfully used as an encoder
   when constructing a MIMEApplication object.
 
@@ -156,7 +160,7 @@
   lists all loaded CA certificates and cert_store_stats() returns amount of
   loaded X.509 certs, X.509 CA certs and CRLs.
 
-- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+- Issue #18167: cgi.FieldStorage no longer fails to handle multipart/form-data
   when \r\n appears at end of 65535 bytes without other newlines.
 
 - Issue #18076: Introduce importlib.util.decode_source().

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 00:44:14 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 30 Jun 2013 00:44:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzE4MTU1OiBSZWdl?=
 =?utf-8?q?x-escape_delimiter=2C_in_case_it_is_a_regex_special_char=2E?=
Message-ID: <3bjVJZ2Dghz7Ljm@mail.python.org>

http://hg.python.org/cpython/rev/0e1d538d36dc
changeset:   84388:0e1d538d36dc
branch:      2.7
parent:      84385:0767363a0393
user:        R David Murray <rdmurray at bitdance.com>
date:        Sat Jun 29 18:43:59 2013 -0400
summary:
  #18155: Regex-escape delimiter, in case it is a regex special char.

Patch by Vajrasky Kok, with slight modification to the tests by me.

files:
  Lib/csv.py           |   5 ++-
  Lib/test/test_csv.py |  42 +++++++++++++++++++++++++++++--
  Misc/ACKS            |   1 +
  Misc/NEWS            |   6 +++-
  4 files changed, 48 insertions(+), 6 deletions(-)


diff --git a/Lib/csv.py b/Lib/csv.py
--- a/Lib/csv.py
+++ b/Lib/csv.py
@@ -261,8 +261,9 @@
 
         # if we see an extra quote between delimiters, we've got a
         # double quoted format
-        dq_regexp = re.compile(r"((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)" % \
-                               {'delim':delim, 'quote':quotechar}, re.MULTILINE)
+        dq_regexp = re.compile(
+                               r"((%(delim)s)|^)\W*%(quote)s[^%(delim)s\n]*%(quote)s[^%(delim)s\n]*%(quote)s\W*((%(delim)s)|$)" % \
+                               {'delim':re.escape(delim), 'quote':quotechar}, re.MULTILINE)
 
 
 
diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -914,7 +914,7 @@
 'Tommy''s Place':'Blue Island':'IL':'12/28/02':'Blue Sunday/White Crow'
 'Stonecutters ''Seafood'' and Chop House':'Lemont':'IL':'12/19/02':'Week Back'
 """
-    header = '''\
+    header1 = '''\
 "venue","city","state","date","performers"
 '''
     sample3 = '''\
@@ -933,10 +933,35 @@
     sample6 = "a|b|c\r\nd|e|f\r\n"
     sample7 = "'a'|'b'|'c'\r\n'd'|e|f\r\n"
 
+# Issue 18155: Use a delimiter that is a special char to regex:
+
+    header2 = '''\
+"venue"+"city"+"state"+"date"+"performers"
+'''
+    sample8 = """\
+Harry's+ Arlington Heights+ IL+ 2/1/03+ Kimi Hayes
+Shark City+ Glendale Heights+ IL+ 12/28/02+ Prezence
+Tommy's Place+ Blue Island+ IL+ 12/28/02+ Blue Sunday/White Crow
+Stonecutters Seafood and Chop House+ Lemont+ IL+ 12/19/02+ Week Back
+"""
+    sample9 = """\
+'Harry''s'+ Arlington Heights'+ 'IL'+ '2/1/03'+ 'Kimi Hayes'
+'Shark City'+ Glendale Heights'+' IL'+ '12/28/02'+ 'Prezence'
+'Tommy''s Place'+ Blue Island'+ 'IL'+ '12/28/02'+ 'Blue Sunday/White Crow'
+'Stonecutters ''Seafood'' and Chop House'+ 'Lemont'+ 'IL'+ '12/19/02'+ 'Week Back'
+"""
+
     def test_has_header(self):
         sniffer = csv.Sniffer()
         self.assertEqual(sniffer.has_header(self.sample1), False)
-        self.assertEqual(sniffer.has_header(self.header+self.sample1), True)
+        self.assertEqual(sniffer.has_header(self.header1 + self.sample1),
+                         True)
+
+    def test_has_header_regex_special_delimiter(self):
+        sniffer = csv.Sniffer()
+        self.assertEqual(sniffer.has_header(self.sample8), False)
+        self.assertEqual(sniffer.has_header(self.header2 + self.sample8),
+                         True)
 
     def test_sniff(self):
         sniffer = csv.Sniffer()
@@ -970,13 +995,24 @@
         dialect = sniffer.sniff(self.sample7)
         self.assertEqual(dialect.delimiter, "|")
         self.assertEqual(dialect.quotechar, "'")
+        dialect = sniffer.sniff(self.sample8)
+        self.assertEqual(dialect.delimiter, '+')
+        dialect = sniffer.sniff(self.sample9)
+        self.assertEqual(dialect.delimiter, '+')
+        self.assertEqual(dialect.quotechar, "'")
 
     def test_doublequote(self):
         sniffer = csv.Sniffer()
-        dialect = sniffer.sniff(self.header)
+        dialect = sniffer.sniff(self.header1)
+        self.assertFalse(dialect.doublequote)
+        dialect = sniffer.sniff(self.header2)
         self.assertFalse(dialect.doublequote)
         dialect = sniffer.sniff(self.sample2)
         self.assertTrue(dialect.doublequote)
+        dialect = sniffer.sniff(self.sample8)
+        self.assertFalse(dialect.doublequote)
+        dialect = sniffer.sniff(self.sample9)
+        self.assertTrue(dialect.doublequote)
 
 if not hasattr(sys, "gettotalrefcount"):
     if test_support.verbose: print "*** skipping leakage tests ***"
diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -545,6 +545,7 @@
 Greg Kochanski
 Damon Kohler
 Marko Kohtala
+Vajrasky Kok
 Guido Kollerie
 Peter A. Koren
 Joseph Koshy
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -24,11 +24,15 @@
 Library
 -------
 
+- Issue #18155: The csv module now correctly handles csv files that use
+  a delimiter character that has a special meaning in regexes, instead of
+  throwing an exception.
+
 - Issue #18135: ssl.SSLSocket.write() now raises an OverflowError if the input
   string in longer than 2 gigabytes. The ssl module does not support partial
   write.
 
-- Issue #18167: cgi.FieldStorage no more fails to handle multipart/form-data
+- Issue #18167: cgi.FieldStorage no longer fails to handle multipart/form-data
   when \r\n appears at end of 65535 bytes without other newlines.
 
 - Issue #17403: urllib.parse.robotparser normalizes the urls before adding to

-- 
Repository URL: http://hg.python.org/cpython

From solipsis at pitrou.net  Sun Jun 30 05:45:26 2013
From: solipsis at pitrou.net (solipsis at pitrou.net)
Date: Sun, 30 Jun 2013 05:45:26 +0200
Subject: [Python-checkins] Daily reference leaks (acaf73e3d882): sum=2
Message-ID: <E1Ut8Zq-0004Ne-Ha@ap.vmr.nerim.net>

results for acaf73e3d882 on branch "default"
--------------------------------------------

test_concurrent_futures leaked [-2, 3, 1] memory blocks, sum=2


Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogldSUu5', '-x']

From python-checkins at python.org  Sun Jun 30 17:51:10 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 30 Jun 2013 17:51:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogIzE4MDM4OiBVc2Ug?=
 =?utf-8?q?non-deprecated_assert_names_in_tests=2E?=
Message-ID: <3bjx5V40D8z7Lk5@mail.python.org>

http://hg.python.org/cpython/rev/19bc00996e74
changeset:   84389:19bc00996e74
branch:      3.3
parent:      84386:68ff68f9a0d5
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 30 11:46:32 2013 -0400
summary:
  #18038: Use non-deprecated assert names in tests.

files:
  Lib/test/test_pep263.py |  10 +++++-----
  1 files changed, 5 insertions(+), 5 deletions(-)


diff --git a/Lib/test/test_pep263.py b/Lib/test/test_pep263.py
--- a/Lib/test/test_pep263.py
+++ b/Lib/test/test_pep263.py
@@ -59,17 +59,17 @@
         compile(b'# -*- coding: iso-8859-15 -*-\n', 'dummy', 'exec')
         compile(b'\xef\xbb\xbf\n', 'dummy', 'exec')
         compile(b'\xef\xbb\xbf# -*- coding: utf-8 -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+        with self.assertRaisesRegex(SyntaxError, 'fake'):
             compile(b'# -*- coding: fake -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'iso-8859-15'):
+        with self.assertRaisesRegex(SyntaxError, 'iso-8859-15'):
             compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
                     'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+        with self.assertRaisesRegex(SyntaxError, 'BOM'):
             compile(b'\xef\xbb\xbf# -*- coding: iso-8859-15 -*-\n',
                     'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'fake'):
+        with self.assertRaisesRegex(SyntaxError, 'fake'):
             compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
-        with self.assertRaisesRegexp(SyntaxError, 'BOM'):
+        with self.assertRaisesRegex(SyntaxError, 'BOM'):
             compile(b'\xef\xbb\xbf# -*- coding: fake -*-\n', 'dummy', 'exec')
 
 

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 17:51:11 2013
From: python-checkins at python.org (r.david.murray)
Date: Sun, 30 Jun 2013 17:51:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Null_merge_=2318038=3A_Use_non-deprecated_assert_names_i?=
 =?utf-8?q?n_tests=2E?=
Message-ID: <3bjx5W5xq0z7Lk5@mail.python.org>

http://hg.python.org/cpython/rev/29e7f6a2dc0d
changeset:   84390:29e7f6a2dc0d
parent:      84387:acaf73e3d882
parent:      84389:19bc00996e74
user:        R David Murray <rdmurray at bitdance.com>
date:        Sun Jun 30 11:50:53 2013 -0400
summary:
  Null merge #18038: Use non-deprecated assert names in tests.

Brett already fixed this in 3.4 in 4f7c25ab2ed2.

files:



-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 19:58:17 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 19:58:17 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogVGVtcGZpbGUucHk6?=
 =?utf-8?q?_stop_buildbot_warning_about_using_deprecated_xreadlines=2E?=
Message-ID: <3bjzw94W6Mz7LmM@mail.python.org>

http://hg.python.org/cpython/rev/1cdd9e680567
changeset:   84391:1cdd9e680567
branch:      2.7
parent:      84388:0e1d538d36dc
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sun Jun 30 13:57:57 2013 -0400
summary:
  Tempfile.py: stop buildbot warning about using deprecated xreadlines.

The slightly odd behavior (the validity of passing a sizehint depends on the
type of self._file) was kept to avoid breaking code that depends on it.
Test_tempfile.test_xreadlines passes (along with everything else).

files:
  Lib/tempfile.py |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/Lib/tempfile.py b/Lib/tempfile.py
--- a/Lib/tempfile.py
+++ b/Lib/tempfile.py
@@ -612,7 +612,7 @@
         return rv
 
     def xreadlines(self, *args):
-        try:
-            return self._file.xreadlines(*args)
-        except AttributeError:
+        if hasattr(self._file, 'xreadlines'):  # real file
+            return iter(self._file)
+        else:  # StringIO()
             return iter(self._file.readlines(*args))

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 22:53:04 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 22:53:04 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgMTgxODk6?=
 =?utf-8?q?_remove_unused_methods_in_idlelib=2EDelegator=2EDelegator=2E?=
Message-ID: <3bk3nr5vBZz7LmX@mail.python.org>

http://hg.python.org/cpython/rev/a568a5426a16
changeset:   84392:a568a5426a16
branch:      2.7
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sun Jun 30 16:51:53 2013 -0400
summary:
  Issue 18189: remove unused methods in idlelib.Delegator.Delegator.
Idle code already uses x.delegate instead of x.getdelegate().
The printed report must have been for testing.

files:
  Lib/idlelib/Delegator.py |  8 --------
  1 files changed, 0 insertions(+), 8 deletions(-)


diff --git a/Lib/idlelib/Delegator.py b/Lib/idlelib/Delegator.py
--- a/Lib/idlelib/Delegator.py
+++ b/Lib/idlelib/Delegator.py
@@ -20,14 +20,6 @@
                 pass
         self.__cache.clear()
 
-    def cachereport(self):
-        keys = self.__cache.keys()
-        keys.sort()
-        print keys
-
     def setdelegate(self, delegate):
         self.resetcache()
         self.delegate = delegate
-
-    def getdelegate(self):
-        return self.delegate

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 22:53:06 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 22:53:06 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgMTgxODk6?=
 =?utf-8?q?_remove_unused_methods_in_idlelib=2EDelegator=2EDelegator=2E?=
Message-ID: <3bk3nt0fHZz7LnF@mail.python.org>

http://hg.python.org/cpython/rev/9d65716367c1
changeset:   84393:9d65716367c1
branch:      3.3
parent:      84389:19bc00996e74
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sun Jun 30 16:52:19 2013 -0400
summary:
  Issue 18189: remove unused methods in idlelib.Delegator.Delegator.
Idle code already uses x.delegate instead of x.getdelegate().
The printed report must have been for testing.

files:
  Lib/idlelib/Delegator.py |  8 --------
  1 files changed, 0 insertions(+), 8 deletions(-)


diff --git a/Lib/idlelib/Delegator.py b/Lib/idlelib/Delegator.py
--- a/Lib/idlelib/Delegator.py
+++ b/Lib/idlelib/Delegator.py
@@ -20,14 +20,6 @@
                 pass
         self.__cache.clear()
 
-    def cachereport(self):
-        keys = list(self.__cache.keys())
-        keys.sort()
-        print(keys)
-
     def setdelegate(self, delegate):
         self.resetcache()
         self.delegate = delegate
-
-    def getdelegate(self):
-        return self.delegate

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 22:53:07 2013
From: python-checkins at python.org (terry.reedy)
Date: Sun, 30 Jun 2013 22:53:07 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
	=?utf-8?q?=29=3A_Merge_with_3=2E3?=
Message-ID: <3bk3nv2SCzz7LnF@mail.python.org>

http://hg.python.org/cpython/rev/6584e1ddc347
changeset:   84394:6584e1ddc347
parent:      84390:29e7f6a2dc0d
parent:      84393:9d65716367c1
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Sun Jun 30 16:52:40 2013 -0400
summary:
  Merge with 3.3

files:
  Lib/idlelib/Delegator.py |  8 --------
  1 files changed, 0 insertions(+), 8 deletions(-)


diff --git a/Lib/idlelib/Delegator.py b/Lib/idlelib/Delegator.py
--- a/Lib/idlelib/Delegator.py
+++ b/Lib/idlelib/Delegator.py
@@ -20,14 +20,6 @@
                 pass
         self.__cache.clear()
 
-    def cachereport(self):
-        keys = list(self.__cache.keys())
-        keys.sort()
-        print(keys)
-
     def setdelegate(self, delegate):
         self.resetcache()
         self.delegate = delegate
-
-    def getdelegate(self):
-        return self.delegate

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 23:12:10 2013
From: python-checkins at python.org (vinay.sajip)
Date: Sun, 30 Jun 2013 23:12:10 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjI0?=
 =?utf-8?q?=3A_Removed_pydoc_script_from_created_venv=2C_as_it_causes_prob?=
 =?utf-8?q?lems_on?=
Message-ID: <3bk4Ct4hrkz7LmM@mail.python.org>

http://hg.python.org/cpython/rev/af837bf390d0
changeset:   84395:af837bf390d0
branch:      3.3
parent:      84393:9d65716367c1
user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
date:        Sun Jun 30 22:06:52 2013 +0100
summary:
  Issue #18224: Removed pydoc script from created venv, as it causes problems on Windows and adds no value over and above python -m pydoc ...

files:
  Lib/venv/scripts/nt/pydoc.py |  4 ----
  Lib/venv/scripts/posix/pydoc |  5 -----
  Misc/NEWS                    |  3 +++
  3 files changed, 3 insertions(+), 9 deletions(-)


diff --git a/Lib/venv/scripts/nt/pydoc.py b/Lib/venv/scripts/nt/pydoc.py
deleted file mode 100644
--- a/Lib/venv/scripts/nt/pydoc.py
+++ /dev/null
@@ -1,4 +0,0 @@
-#!__VENV_PYTHON__
-if __name__ == '__main__':
-    import sys, pydoc
-    sys.exit(pydoc.cli())
diff --git a/Lib/venv/scripts/posix/pydoc b/Lib/venv/scripts/posix/pydoc
deleted file mode 100755
--- a/Lib/venv/scripts/posix/pydoc
+++ /dev/null
@@ -1,5 +0,0 @@
-#!__VENV_PYTHON__
-if __name__ == '__main__':
-    import sys, pydoc
-    sys.exit(pydoc.cli())
-
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -38,6 +38,9 @@
 Library
 -------
 
+- Issue #18224: Removed pydoc script from created venv, as it causes problems
+  on Windows and adds no value over and above python -m pydoc ...
+
 - Issue #18155: The csv module now correctly handles csv files that use
   a delimter character that has a special meaning in regexes, instead of
   throwing an exception.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 23:12:11 2013
From: python-checkins at python.org (vinay.sajip)
Date: Sun, 30 Jun 2013 23:12:11 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Closes_=2318224=3A_Removed_pydoc_script_from_created_ven?=
 =?utf-8?q?v=2C_as_it_causes_problems_on?=
Message-ID: <3bk4Cv6mMjz7Ln0@mail.python.org>

http://hg.python.org/cpython/rev/de73e7ffabb3
changeset:   84396:de73e7ffabb3
parent:      84394:6584e1ddc347
parent:      84395:af837bf390d0
user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
date:        Sun Jun 30 22:08:27 2013 +0100
summary:
  Closes #18224: Removed pydoc script from created venv, as it causes problems on Windows and adds no value over and above python -m pydoc ...

files:
  Lib/venv/scripts/nt/pydoc.py |  4 ----
  Lib/venv/scripts/posix/pydoc |  5 -----
  Misc/NEWS                    |  3 +++
  3 files changed, 3 insertions(+), 9 deletions(-)


diff --git a/Lib/venv/scripts/nt/pydoc.py b/Lib/venv/scripts/nt/pydoc.py
deleted file mode 100644
--- a/Lib/venv/scripts/nt/pydoc.py
+++ /dev/null
@@ -1,4 +0,0 @@
-#!__VENV_PYTHON__
-if __name__ == '__main__':
-    import sys, pydoc
-    sys.exit(pydoc.cli())
diff --git a/Lib/venv/scripts/posix/pydoc b/Lib/venv/scripts/posix/pydoc
deleted file mode 100755
--- a/Lib/venv/scripts/posix/pydoc
+++ /dev/null
@@ -1,5 +0,0 @@
-#!__VENV_PYTHON__
-if __name__ == '__main__':
-    import sys, pydoc
-    sys.exit(pydoc.cli())
-
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -135,6 +135,9 @@
 Library
 -------
 
+- Issue #18224: Removed pydoc script from created venv, as it causes problems
+  on Windows and adds no value over and above python -m pydoc ...
+
 - Issue #18155: The csv module now correctly handles csv files that use
   a delimter character that has a special meaning in regexes, instead of
   throwing an exception.

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 23:12:13 2013
From: python-checkins at python.org (vinay.sajip)
Date: Sun, 30 Jun 2013 23:12:13 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy4zKTogSXNzdWUgIzE4MjI0?=
	=?utf-8?q?=3A_Updated_test=2E?=
Message-ID: <3bk4Cx1WsKz7LnM@mail.python.org>

http://hg.python.org/cpython/rev/c17fa2cbad43
changeset:   84397:c17fa2cbad43
branch:      3.3
parent:      84395:af837bf390d0
user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
date:        Sun Jun 30 22:11:10 2013 +0100
summary:
  Issue #18224: Updated test.

files:
  Lib/test/test_venv.py |  4 ----
  1 files changed, 0 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py
--- a/Lib/test/test_venv.py
+++ b/Lib/test/test_venv.py
@@ -23,12 +23,10 @@
         self.env_dir = os.path.realpath(tempfile.mkdtemp())
         if os.name == 'nt':
             self.bindir = 'Scripts'
-            self.pydocname = 'pydoc.py'
             self.lib = ('Lib',)
             self.include = 'Include'
         else:
             self.bindir = 'bin'
-            self.pydocname = 'pydoc'
             self.lib = ('lib', 'python%s' % sys.version[:3])
             self.include = 'include'
         if sys.platform == 'darwin' and '__PYVENV_LAUNCHER__' in os.environ:
@@ -78,8 +76,6 @@
             executable = sys.executable
         path = os.path.dirname(executable)
         self.assertIn('home = %s' % path, data)
-        data = self.get_text_file_contents(self.bindir, self.pydocname)
-        self.assertTrue(data.startswith('#!%s%s' % (self.env_dir, os.sep)))
         fn = self.get_env_file(self.bindir, self.exe)
         if not os.path.exists(fn):  # diagnostics for Windows buildbot failures
             bd = self.get_env_file(self.bindir)

-- 
Repository URL: http://hg.python.org/cpython

From python-checkins at python.org  Sun Jun 30 23:12:14 2013
From: python-checkins at python.org (vinay.sajip)
Date: Sun, 30 Jun 2013 23:12:14 +0200 (CEST)
Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E3_-=3E_default?=
 =?utf-8?q?=29=3A_Issue_=2318224=3A_Updated_test=2E?=
Message-ID: <3bk4Cy3S8Xz7LnV@mail.python.org>

http://hg.python.org/cpython/rev/ae69436eb7c2
changeset:   84398:ae69436eb7c2
parent:      84396:de73e7ffabb3
parent:      84397:c17fa2cbad43
user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
date:        Sun Jun 30 22:11:46 2013 +0100
summary:
  Issue #18224: Updated test.

files:
  Lib/test/test_venv.py |  4 ----
  1 files changed, 0 insertions(+), 4 deletions(-)


diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py
--- a/Lib/test/test_venv.py
+++ b/Lib/test/test_venv.py
@@ -23,12 +23,10 @@
         self.env_dir = os.path.realpath(tempfile.mkdtemp())
         if os.name == 'nt':
             self.bindir = 'Scripts'
-            self.pydocname = 'pydoc.py'
             self.lib = ('Lib',)
             self.include = 'Include'
         else:
             self.bindir = 'bin'
-            self.pydocname = 'pydoc'
             self.lib = ('lib', 'python%s' % sys.version[:3])
             self.include = 'include'
         if sys.platform == 'darwin' and '__PYVENV_LAUNCHER__' in os.environ:
@@ -78,8 +76,6 @@
             executable = sys.executable
         path = os.path.dirname(executable)
         self.assertIn('home = %s' % path, data)
-        data = self.get_text_file_contents(self.bindir, self.pydocname)
-        self.assertTrue(data.startswith('#!%s%s' % (self.env_dir, os.sep)))
         fn = self.get_env_file(self.bindir, self.exe)
         if not os.path.exists(fn):  # diagnostics for Windows buildbot failures
             bd = self.get_env_file(self.bindir)

-- 
Repository URL: http://hg.python.org/cpython