multiprocessing

[Dan Stromberg]

I'm afraid I've not heard of a way of putting more complex objects into
shared memory with multiprocessing.  That is, not without using a queue
along the lines of
http://docs.python.org/library/multiprocessing.html#exchanging-objects-between-processes,
which isn't quite the same thing.

On Thu, Apr 7, 2011 at 5:57 PM, Kerensa McElroy <kerensaelise at hotmail.com>wrote:

>  Hi,
>
> thanks for your response.
>
> I checked out multiprocessing.value, however from what I can make out, it
> works with object of only a very limited type. Is there a way to do this for
> more complex objects? (In reality, my object is a large multi-dimensional
> numpy array).
>
> Thanks,
>
> Elsa.
>
> ------------------------------
> Date: Wed, 6 Apr 2011 22:20:06 -0700
> Subject: Re: multiprocessing
> From: drsalists at gmail.com
> To: kerensaelise at hotmail.com
> CC: python-list at python.org
>
>
>
> On Wed, Apr 6, 2011 at 9:06 PM, elsa <kerensaelise at hotmail.com> wrote:
>
> Hi guys,
>
> I want to try out some pooling of processors, but I'm not sure if it
> is possible to do what I want to do. Basically, I want to have a
> global object, that is updated during the execution of a function, and
> I want to be able to run this function several times on parallel
> processors. The order in which the function runs doesn't matter, and
> the value of the object doesn't matter to the function, but I do want
> the processors to take turns 'nicely' when updating the object, so
> there are no collisions. Here is an extremely simplified and trivial
> example of what I have in mind:
>
> from multiprocessing import Pool
> import random
>
> p=Pool(4)
> myDict={}
>
> def update(value):
>    global myDict
>    index=random.random()
>    myDict[index]+=value
>
> total=1000
>
> p.map(update,range(total))
>
>
> After, I would also like to be able to use several processors to
> access the global object (but not modify it). Again, order doesn't
> matter:
>
> p1=Pool(4)
>
> def getValues(index):
>    global myDict
>    print myDict[index]
>
> p1.map(getValues,keys.myDict)
>
> Is there a way to do this
>
>
> This should give you a synchronized wrapper around an object in shared
> memory:
>
> http://docs.python.org/library/multiprocessing.html#multiprocessing.Value
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110407/5ce7ab0a/attachment-0001.html>