Sharing objects between processes

[Aaron Brady]

On Mar 9, 12:47 pm, ET <p... at 2drpg.org> wrote:
> > Message: 2
> > Date: Sun, 8 Mar 2009 12:00:40 -0700 (PDT)
> > From: Aaron Brady <castiro... at gmail.com>
> > Subject: Re: Sharing objects between processes
> > To: python-l... at python.org
> > Message-ID:
> >    <5514c3df-d74e-47d8-93fc-34dd5119e... at c11g2000yqj.googlegroups.com>
> > Content-Type: text/plain; charset=ISO-8859-1
>
> > On Mar 8, 1:36?pm, ET <p... at 2drpg.org> wrote:
> > > I have been using the 'threading' library and decided to try swapping it
> > > out for 'processing'... while it's awesome that processing so closely
> > > mirrors the threading interface, I've been having trouble getting my
> > > processes to share an object in a similar way.
>
> > > Using the 'with' keyword didn't work, and using normal locks doesn't
> > > result in the expected behavior (I can get an object to be accessible in
> > > more than one process, and Python indicates that the instances are
> > > living at the same address in memory, but changes in one process are not
> > > reflected in the other[s]). ?I'm sure this is because my expectations
> > > are incorrect. :)
>
> > > The problem, as briefly as possible:
> > > I have three processes which need to safely read and update two objects.
>
> > > I've been working with processing, multiprocessing, and parallel python,
> > > trying to get this working... I suspect it can be accomplished with
> > > managers and/or queues, but if there's an elegant way to handle it, I
> > > have thus far failed to understand it.
>
> > > I don't particularly care which library I use; if someone has done this
> > > or can recommend a good method they're aware of, it'd be incredibly
> > > helpful.
>
> > > Thank you!
>
> > There is POSH: Python Object Sharing, which I learned about a while
> > ago, but never used much.
>
> >http://poshmodule.sourceforge.net/
>
> > It's UNIX only.
>
> Thanks, I'll definitely keep that link handy... unfortunately, this
> particular project needs to run on Windows as well as Linux-based
> systems.

I don't recall whether there was anything in the source that's a deal-
breaker on Windows.  The source is open, as you could see.

Other possibilities are 'shelve' and any database... fixed-length
pickles, a directory of pickles, etc.  Maybe 'multiprocessing' would
work for your synchronization, while you use a more custom technique
for data exchange.

The only other thing I can do is bring to your attention an idea of
mine for sharing primitives.  It's in the drawing board stage if you
want to help.

Of course, it's only after you turned down 'multiprocessing' and
'POSH'.  It does things they don't and vice versa.