Question about Multi-processing

Anthony Flury anthony.flury at btinternet.com
Tue Oct 2 06:10:08 EDT 2018


I decided to spend this morning to get my head around multi-processing, 
and decided to try to experiment initially in the command line 
interpreter, using the Python 3 documentation 
<https://docs.python.org/3/library/multiprocessing.html?highlight=process#multiprocessing.pool.Pool>

I fired up Python 3.6 and ran this :

    Python 3.6.6 (default, Sep 12 2018, 18:26:19)
    [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
    Type "help", "copyright", "credits" or "license" for more information.
     >>> from multiprocessing import Pool
     >>> def f(x):
    ...     return x*2 + 1
    ...
     >>> with Pool(10) as p:
    ...      print(p.map(f, range(1,100)))
    ...
    [3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37,
    39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69, 71,
    73, 75, 77, 79, 81, 83, 85, 87, 89, 91, 93, 95, 97, 99, 101, 103,
    105, 107, 109, 111, 113, 115, 117, 119, 121, 123, 125, 127, 129,
    131, 133, 135, 137, 139, 141, 143, 145, 147, 149, 151, 153, 155,
    157, 159, 161, 163, 165, 167, 169, 171, 173, 175, 177, 179, 181,
    183, 185, 187, 189, 191, 193, 195, 197, 199]

As you can see - it clearly worked as expected - queue a happy dance.

I continued to read the documentation and came to this quote :

    /Functionality within this package requires that the
    //|__main__|//module be importable by the children. This is covered
    in //Programming guidelines
    <https://docs.python.org/3/library/multiprocessing.html?highlight=process#multiprocessing-programming>//however
    it is worth pointing out here. This means that some examples, such
    as the //|multiprocessing.pool.Pool|
    <https://docs.python.org/3/library/multiprocessing.html?highlight=process#multiprocessing.pool.Pool>//examples
    will not work in the interactive interpreter./

The Documentation then gives a clear example very similar to mine :

    >>> from multiprocessing import Pool >>> p = Pool(5) >>> def f(x): ...
    return x*x ... >>> p.map(f, [1,2,3]) Process PoolWorker-1: Process
    PoolWorker-2: Process PoolWorker-3: Traceback (most recent call
    last): AttributeError: 'module' object has no attribute 'f'
    AttributeError: 'module' object has no attribute 'f' AttributeError:
    'module' object has no attribute 'f'

But when I run this exact example in the command line interpreter it 
works fine :

     >>> p = Pool(5)
     >>> def f(x):
    ...     return x*x
    ...
     >>> p.map(f,[1,2,3])
    [3, 5, 7]

Is the documentation wrong ? is something weird going on ?

-- 

Anthony Flury
*Email* : anthony.flury at btinternet.com <mailto:Anthony.flury at btinternet.com>
*Twitter* : @TonyFlury <https://twitter.com/TonyFlury/>



More information about the Python-list mailing list