[Python-checkins] cpython (3.4): Issue #22952: improve multiprocessing doc introduction and defer notes until
antoine.pitrou
python-checkins at python.org
Sun Jan 11 15:07:08 CET 2015
https://hg.python.org/cpython/rev/a9a9c71f8e15
changeset: 94111:a9a9c71f8e15
branch: 3.4
parent: 94109:5edfc6c929f9
user: Antoine Pitrou <solipsis at pitrou.net>
date: Sun Jan 11 15:05:29 2015 +0100
summary:
Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.
Patch by Davin Potts.
files:
Doc/library/multiprocessing.rst | 92 +++++++++++++-------
1 files changed, 57 insertions(+), 35 deletions(-)
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -16,41 +16,27 @@
leverage multiple processors on a given machine. It runs on both Unix and
Windows.
-.. note::
-
- Some of this package's functionality requires a functioning shared semaphore
- implementation on the host operating system. Without one, the
- :mod:`multiprocessing.synchronize` module will be disabled, and attempts to
- import it will result in an :exc:`ImportError`. See
- :issue:`3770` for additional information.
-
-.. note::
-
- Functionality within this package requires that the ``__main__`` module be
- importable by the children. This is covered in :ref:`multiprocessing-programming`
- however it is worth pointing out here. This means that some examples, such
- as the :class:`multiprocessing.pool.Pool` examples will not work in the
- interactive interpreter. For example::
-
- >>> from multiprocessing import Pool
- >>> p = Pool(5)
- >>> def f(x):
- ... return x*x
- ...
- >>> p.map(f, [1,2,3])
- Process PoolWorker-1:
- Process PoolWorker-2:
- Process PoolWorker-3:
- Traceback (most recent call last):
- Traceback (most recent call last):
- Traceback (most recent call last):
- AttributeError: 'module' object has no attribute 'f'
- AttributeError: 'module' object has no attribute 'f'
- AttributeError: 'module' object has no attribute 'f'
-
- (If you try this it will actually output three full tracebacks
- interleaved in a semi-random fashion, and then you may have to
- stop the master process somehow.)
+The :mod:`multiprocessing` module also introduces APIs which do not have
+analogs in the :mod:`threading` module. A prime example of this is the
+:class:`~multiprocessing.pool.Pool` object which offers a convenient means of
+parallelizing the execution of a function across multiple input values,
+distributing the input data across processes (data parallelism). The following
+example demonstrates the common practice of defining such functions in a module
+so that child processes can successfully import that module. This basic example
+of data parallelism using :class:`~multiprocessing.pool.Pool`, ::
+
+ from multiprocessing import Pool
+
+ def f(x):
+ return x*x
+
+ if __name__ == '__main__':
+ with Pool(5) as p:
+ print(p.map(f, [1, 2, 3]))
+
+will print to standard output ::
+
+ [1, 4, 9]
The :class:`Process` class
@@ -276,6 +262,14 @@
Without using the lock output from the different processes is liable to get all
mixed up.
+.. note::
+
+ Some of this package's functionality requires a functioning shared semaphore
+ implementation on the host operating system. Without one, the
+ :mod:`multiprocessing.synchronize` module will be disabled, and attempts to
+ import it will result in an :exc:`ImportError`. See
+ :issue:`3770` for additional information.
+
Sharing state between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -406,6 +400,34 @@
Note that the methods of a pool should only ever be used by the
process which created it.
+.. note::
+
+ Functionality within this package requires that the ``__main__`` module be
+ importable by the children. This is covered in :ref:`multiprocessing-programming`
+ however it is worth pointing out here. This means that some examples, such
+ as the :class:`multiprocessing.pool.Pool` examples will not work in the
+ interactive interpreter. For example::
+
+ >>> from multiprocessing import Pool
+ >>> p = Pool(5)
+ >>> def f(x):
+ ... return x*x
+ ...
+ >>> p.map(f, [1,2,3])
+ Process PoolWorker-1:
+ Process PoolWorker-2:
+ Process PoolWorker-3:
+ Traceback (most recent call last):
+ Traceback (most recent call last):
+ Traceback (most recent call last):
+ AttributeError: 'module' object has no attribute 'f'
+ AttributeError: 'module' object has no attribute 'f'
+ AttributeError: 'module' object has no attribute 'f'
+
+ (If you try this it will actually output three full tracebacks
+ interleaved in a semi-random fashion, and then you may have to
+ stop the master process somehow.)
+
Reference
---------
--
Repository URL: https://hg.python.org/cpython
More information about the Python-checkins
mailing list