multiprocess passing arguments double asterisks

[MRAB]

On 2016-10-26 21:44, pic8690 at gmail.com wrote:
> On Monday, October 24, 2016 at 12:39:47 PM UTC-5, Thomas Nyberg wrote:
>> On 10/24/2016 12:45 PM, pic8690 at gmail.com wrote:
>> > Thanks for the reply.
>> >
>> > The code snippet given by Peter is not very clear
>> >
>> > I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value.  This example does not return anything
>> >
>> > Would you please use my example for the map function?
>> >
>> > I appreciate your help,
>> >
>> I'm honestly not totally sure what you want to do. However, say you want
>> to do the following (btw this is basically what Dennis said i nhis last
>> email, but maybe I can help clarify):
>>
>> 	kwargs = {'param1': val1, 'param2': val2})
>>
>> Then you'd like to have the following two operations performed in
>> separate processes:
>>
>> 	bar(param1=val1)
>> 	bar(param2=val2)
>>
>> In that case, I guess I would do something like the following. First
>> define bar_wrapper as follows (*I haven't tested any code here!):
>>
>> def bar_wrapper(pair):
>> 	key, val = pair
>> 	return bar(**{key: val})
>>
>> Then I would probably do something like
>>
>> map(bar_wrapper, kwargs.items())
>>
>> I.e. basically what I'm doing is taking the key-val pairs and producing
>> a list of them (as tuples). This is something that you can apply map
>> too, but not with the original function. So then the wrapper function
>> converts the tuple back to what you want originally.
>>
>> Hopefully I'm understanding correctly and hopefully this helps.
>>
>> Cheers,
>> Thomas
>
> Thomas,
>
> I have strings & numpy.ndarray as arguments.  The wrapper function works great for strings.
>
> Here's what I'm trying to do...
> **************************
>
> I have a serial python fn of the form
>
> def bar(**kwargs):
>   a=kwargs.get("name")
>   print a
>   self.a1d=ma.asanyarray(kwargs.get('a1d'), dtype=float)
>   (****more calculations--takes a long time to compute
>   returns an object)
>
> I am trying to run this function in parallel.
>
> Here's my Main program
>
> import numpy as np
> import numpy.ma as ma
> from delegate import parallelize
> from hashlib import sha1
> a1d=np.zeros((32))
> b1d=np.zeros((32))
> p=open("/tmp/pdata","rb")
> pdata=np.load(p)
> for i in range(0,10):
>   a1d=pdata['t1d']
>   b1d=pdata['gz1d']
>   print a1d,b1d
>   kwargs={'name':'special','a':a1d,'b':b1d}
>   val=parallelize(bar,kwargs)
> ***************************************
> Error:
> line 30, in <module>
>     val=parallelize(bar,kwargs)
>    delegate.py", line 311, in process
>     item, items = items[0], items[1:]
> KeyError: 0
> **************************************
> Error: (with the wrapper function)
> val=parallelize(bar_wrapper,kwargs.items())
> TypeError: unhashable type: 'numpy.ndarray'
> ***************************************
>
> How do I pass the string & numpy.ndarray to the function bar(**kwargs)?
>
> Thank you for suggestions & help,
>
>
> a1d=np.zeros((32))
> b1d=np.zeros((32))
> p=open("/tmp/pdata","rb")
> pdata=np.load(p)
> for i in range(0,100):
>   a1d=pdata['t1d']
>   b1d=pdata['gz1d']
>   print a1d,b1d
>   kwargs={'name':'special','a':a1d,'b':b1d}
>   val=parallelize(bar,kwargs)
>
'parallelize' expects a function and a list of items. It calls the 
function with each item in the list in parallel, passing the item as the 
argument of the function.

That _single_ argument item can be a tuple or a dict.