Proper way to pass Queue to process when using multiprocessing.imap()?

[Israel Brewster]

> 
> On Sep 3, 2019, at 11:09 AM, Israel Brewster <ijbrewster at alaska.edu> wrote:
> 
>> 
>> On Sep 3, 2019, at 10:49 AM, Peter Otten <__peter__ at web.de> wrote:
>> 
>> Israel Brewster wrote:
>> 
>>> When using pool.imap to apply a function over a list of values, what is
>>> the proper way to pass additional arguments to the function, specifically
>>> in my case a Queue that the process can use to communicate back to the
>>> main thread (for the purpose of reporting progress)? I have seen
>>> suggestions of using starmap, but this doesn’t appear to have a “lazy”
>>> variant, which I have found to be very beneficial in my use case. The
>>> Queue is the same one for all processes, if that makes a difference.
>>> 
>>> I could just make the Queue global, but I have always been told not too.
>>> Perhaps this is an exception?
>> 
>> How about wrapping the function into another function that takes only one 
>> argument? A concise way is to do that with functools.partial():
>> 
>> def f(value, queue): ...
>> 
>> pool.imap(partial(f, queue=...), values)
> 
> That looks like exactly what I was looking for. I’ll give it a shot. Thanks!

So as it turns out, this doesn’t work after all. I get an error stating that “Queue objects should only be shared between processes through inheritance”. Still a good technique to know though!

---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145

> 
> ---
> Israel Brewster
> Software Engineer
> Alaska Volcano Observatory 
> Geophysical Institute - UAF 
> 2156 Koyukuk Drive 
> Fairbanks AK 99775-7320
> Work: 907-474-5172
> cell:  907-328-9145
> 
>> 
>> 
>> 
>>> 
>>> ---
>>> Israel Brewster
>>> Software Engineer
>>> Alaska Volcano Observatory
>>> Geophysical Institute - UAF
>>> 2156 Koyukuk Drive
>>> Fairbanks AK 99775-7320
>>> Work: 907-474-5172
>>> cell:  907-328-9145
>>> 
>> 
>> 
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>