[Python-checkins] cpython: whatsnew: multiprocessing start methods and context (#8713 and #18999)
r.david.murray
python-checkins at python.org
Mon Mar 10 23:11:38 CET 2014
http://hg.python.org/cpython/rev/b941a320601a
changeset: 89569:b941a320601a
user: R David Murray <rdmurray at bitdance.com>
date: Mon Mar 10 14:27:21 2014 -0400
summary:
whatsnew: multiprocessing start methods and context (#8713 and #18999)
Also tweaked the docs a bit to use our standard style for
versionadded/changed. (I'm guessing there are other places
in the multiprocessing docs where similar tweaks should be made.)
files:
Doc/library/multiprocessing.rst | 35 ++++++++++++--------
Doc/whatsnew/3.4.rst | 24 +++++++++++---
Misc/NEWS | 7 ++++
3 files changed, 47 insertions(+), 19 deletions(-)
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -136,9 +136,11 @@
Available on Unix platforms which support passing file descriptors
over Unix pipes.
-Before Python 3.4 *fork* was the only option available on Unix. Also,
-prior to Python 3.4, child processes would inherit all the parents
-inheritable handles on Windows.
+.. versionchanged:: 3.4
+ *span* added on all unix platforms, and *forkserver* added for
+ some unix platforms.
+ Child processes no longer inherit all of the parents inheritable
+ handles on Windows.
On Unix using the *spawn* or *forkserver* start methods will also
start a *semaphore tracker* process which tracks the unlinked named
@@ -1853,25 +1855,30 @@
callbacks and has a parallel map implementation.
*processes* is the number of worker processes to use. If *processes* is
- ``None`` then the number returned by :func:`os.cpu_count` is used. If
- *initializer* is not ``None`` then each worker process will call
+ ``None`` then the number returned by :func:`os.cpu_count` is used.
+
+ If *initializer* is not ``None`` then each worker process will call
``initializer(*initargs)`` when it starts.
+ *maxtasksperchild* is the number of tasks a worker process can complete
+ before it will exit and be replaced with a fresh worker process, to enable
+ unused resources to be freed. The default *maxtasksperchild* is None, which
+ means worker processes will live as long as the pool.
+
+ *context* can be used to specify the context used for starting
+ the worker processes. Usually a pool is created using the
+ function :func:`multiprocessing.Pool` or the :meth:`Pool` method
+ of a context object. In both cases *context* is set
+ appropriately.
+
Note that the methods of the pool object should only be called by
the process which created the pool.
.. versionadded:: 3.2
- *maxtasksperchild* is the number of tasks a worker process can complete
- before it will exit and be replaced with a fresh worker process, to enable
- unused resources to be freed. The default *maxtasksperchild* is None, which
- means worker processes will live as long as the pool.
+ *maxtasksperchild*
.. versionadded:: 3.4
- *context* can be used to specify the context used for starting
- the worker processes. Usually a pool is created using the
- function :func:`multiprocessing.Pool` or the :meth:`Pool` method
- of a context object. In both cases *context* is set
- appropriately.
+ *context*
.. note::
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -1063,11 +1063,25 @@
(``spawn`` and ``forkserver``) have been added for starting processes using
:mod:`multiprocessing`. These make the mixing of processes with threads more
robust, and the ``spawn`` method matches the semantics that multiprocessing has
-always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`).
-
-Also, except when using the old *fork* start method, child processes
-will no longer inherit unneeded handles/file descriptors from their parents
-(part of :issue:`8713`).
+always used on Windows. New function
+:func:`~multiprocessing.get_all_start_methods` reports all start methods
+available on the platform, :func:`~multiprocessing.get_start_method` reports
+the current start method, and :func:`~multiprocessing.set_start_method` sets
+the start method. (Contributed by Richard Oudkerk in :issue:`8713`).
+
+:mod:`multiprocessing` also now has the concept of a ``context``, which
+determines how child processes are created. New function
+:func:`~multiprocessing.get_context` returns a context that uses a specified
+start method. It has the same API as the :mod:`multiprocessing` module itself,
+so you can use it to create :class:`~multiprocessing.pool.Pool`\ s and other
+objects that will operate within that context. This allows a framework and an
+application or different parts of the same application to use multiprocessing
+without interfering with each other. (Contributed by Richard Oudkerk in
+:issue:`18999`.)
+
+Except when using the old *fork* start method, child processes no longer
+inherit unneeded handles/file descriptors from their parents (part of
+:issue:`8713`).
:mod:`multiprocessing` now relies on :mod:`runpy` (which implements the
``-m`` switch) to initialise ``__main__`` appropriately in child processes
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -1636,6 +1636,9 @@
- Issue #18281: Unused stat constants removed from `tarfile`.
+- Issue #18999: Multiprocessing now supports 'contexts' with the same API
+ as the module, but bound to specified start methods.
+
- Issue #18468: The re.split, re.findall, and re.sub functions and the group()
and groups() methods of match object now always return a string or a bytes
object.
@@ -2051,6 +2054,10 @@
- Issue #18532: Change the builtin hash algorithms' names to lower case names
as promised by hashlib's documentation.
+- Issue #8713: add new spwan and forkserver start methods, and new functions
+ get_all_start_methods, get_start_method, and set_start_method, to
+ multiprocessing.
+
- Issue #18405: Improve the entropy of crypt.mksalt().
- Issue #12015: The tempfile module now uses a suffix of 8 random characters
--
Repository URL: http://hg.python.org/cpython
More information about the Python-checkins
mailing list