shared dictionary of dictionaries with Manager

[Dan Stromberg]

2011/4/21 Darío Suárez Gracia <dario.suarez at telefonica.net>

> Hi all,
> I was trying to share a dictionary of dictionaries of arrays with Manager
> from multiprocessing. Without multiprocessing the code works perfectly, but
> with the current example the last print does not show the correct result.
>

It appears that if you put a manager.dict A, inside another manager.dict B,
then B gets passed to a subprocess as a manager.dict, but A gets passed
across as a vanilla dict (in 2.7) - so naturally, changes in A go undetected
in other processes.  This might be a bug; the doc says you can nest manager
types.  It's probably getting lost when the manager.dict is pickled and then
unpickled on the other end.  It might be possible to fix this with copy_reg
or __reduce__ or __getstate__, but no guarantees.

You'll probably be better served by rearchitecting a bit anyway - try to use
simpler types in shared memory (these are fast), or pass a little more
complex types via queues (these aren't terribly fast, but at least they're a
cogent abstraction).

Using lots of shared, mutable state is a Bad Thing for concurrency. - it
complicates things fast, and will kill the performance benefits of
concurrency you're trying to get.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110422/97034429/attachment-0001.html>