multiprocess passing arguments double asterisks

[pic8690 at gmail.com]

On Monday, October 24, 2016 at 12:39:47 PM UTC-5, Thomas Nyberg wrote:
> On 10/24/2016 12:45 PM, pic8690 at gmail.com wrote:
> > Thanks for the reply.
> >
> > The code snippet given by Peter is not very clear
> >
> > I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value.  This example does not return anything
> >
> > Would you please use my example for the map function?
> >
> > I appreciate your help,
> >
> I'm honestly not totally sure what you want to do. However, say you want 
> to do the following (btw this is basically what Dennis said i nhis last 
> email, but maybe I can help clarify):
> 
> 	kwargs = {'param1': val1, 'param2': val2})
> 
> Then you'd like to have the following two operations performed in 
> separate processes:
> 
> 	bar(param1=val1)
> 	bar(param2=val2)
> 
> In that case, I guess I would do something like the following. First 
> define bar_wrapper as follows (*I haven't tested any code here!):
> 
> def bar_wrapper(pair):
> 	key, val = pair
> 	return bar(**{key: val})
> 
> Then I would probably do something like
> 
> map(bar_wrapper, kwargs.items())
> 
> I.e. basically what I'm doing is taking the key-val pairs and producing 
> a list of them (as tuples). This is something that you can apply map 
> too, but not with the original function. So then the wrapper function 
> converts the tuple back to what you want originally.
> 
> Hopefully I'm understanding correctly and hopefully this helps.
> 
> Cheers,
> Thomas

Thomas,

I have strings & numpy.ndarray as arguments.  The wrapper function works great for strings.

Here's what I'm trying to do...
**************************

I have a serial python fn of the form

def bar(**kwargs):
  a=kwargs.get("name")
  print a
  self.a1d=ma.asanyarray(kwargs.get('a1d'), dtype=float)
  (****more calculations--takes a long time to compute
  returns an object)

I am trying to run this function in parallel.

Here's my Main program

import numpy as np
import numpy.ma as ma
from delegate import parallelize
from hashlib import sha1
a1d=np.zeros((32))
b1d=np.zeros((32))
p=open("/tmp/pdata","rb")
pdata=np.load(p)
for i in range(0,10):
  a1d=pdata['t1d']
  b1d=pdata['gz1d']
  print a1d,b1d
  kwargs={'name':'special','a':a1d,'b':b1d}
  val=parallelize(bar,kwargs)
***************************************
Error:
line 30, in <module>
    val=parallelize(bar,kwargs)
   delegate.py", line 311, in process
    item, items = items[0], items[1:]
KeyError: 0
**************************************
Error: (with the wrapper function)
val=parallelize(bar_wrapper,kwargs.items())
TypeError: unhashable type: 'numpy.ndarray'
***************************************

How do I pass the string & numpy.ndarray to the function bar(**kwargs)?

Thank you for suggestions & help, 






a1d=np.zeros((32))
b1d=np.zeros((32))
p=open("/tmp/pdata","rb")
pdata=np.load(p)
for i in range(0,100):
  a1d=pdata['t1d']
  b1d=pdata['gz1d']
  print a1d,b1d
  kwargs={'name':'special','a':a1d,'b':b1d}
  val=parallelize(bar,kwargs)