Import statements and multiprocessing

[Steven D'Aprano]

On Tue, 30 Jan 2018 15:54:30 +0000, Nicholas Cole wrote:

[...]
> The function I am passing to map calls a function in another file within
> the same model.  And that file has a
> 
> from .some_file_in_the_package import *
> 
> line at the top.
> 
> However, in each function called in that file, I seem to need to put an
> explicit important statement buried within the function in order for the
> code to work:
> 
> def some_function():
>     from .some_file_in_the_package import ThisObject
> 
> It is as if in the worker processes created by Pool.map() the from .
> import * directive is being completely ignored.

[...]
> What could be going wrong?

I would say you're probably misinterpreting the nature of the problem. 
Import * isn't a directive that can be ignored.

Can you show us a *simplified* demonstration? A minimal sample program 
which we can run that demonstrates the issue?

Here's an expensive function for you to call:


import time

def process(value):
    time.sleep(3)
    return value+1



And a data set to process:

data = [1, 2, 3, 4]


Can you put them in a minimal package, show us how you would import them, 
and call worker processes with them? We ought to be able to copy the code 
and run it ourselves to see the issue first hand. See 

http://sscce.org/

for further details.


-- 
Steve