Proper way to pass Queue to process when using multiprocessing.imap()?

[Israel Brewster]

> 
> On Sep 3, 2019, at 9:27 AM, Rob Gaddi <rgaddi at highlandtechnology.invalid> wrote:
> 
> On 9/3/19 10:17 AM, Israel Brewster wrote:
>> When using pool.imap to apply a function over a list of values, what is the proper way to pass additional arguments to the function, specifically in my case a Queue that the process can use to communicate back to the main thread (for the purpose of reporting progress)? I have seen suggestions of using starmap, but this doesn’t appear to have a “lazy” variant, which I have found to be very beneficial in my use case. The Queue is the same one for all processes, if that makes a difference.
>> I could just make the Queue global, but I have always been told not too. Perhaps this is an exception?
>>  ---
>> Israel Brewster
>> Software Engineer
>> Alaska Volcano Observatory
>> Geophysical Institute - UAF
>> 2156 Koyukuk Drive
>> Fairbanks AK 99775-7320
>> Work: 907-474-5172
>> cell:  907-328-9145
> 
> The first rule is to never use global variables.  The second is to never put too much stock in sweeping generalizations.  So long as you can keep that Queue's usage pattern fairly well constrained, go ahead and make it global.
> 
> One thing to think about that might make this all easier though; have you looked at the concurrent.futures module?  I find it does a fantastic job of handling this sort of parallelization in a straightforward way.

I’ve only briefly looked at it in other situations. I’ll go ahead and take another look for this one. Thanks for the suggestion!

---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145

> 
> -- 
> Rob Gaddi, Highland Technology -- www.highlandtechnology.com
> Email address domain is currently out of order.  See above to fix.
> -- 
> https://mail.python.org/mailman/listinfo/python-list