Multiprocessing, shared memory vs. pickled copies

[Philip Semanchuk]

On Apr 7, 2011, at 3:41 AM, John Ladasky wrote:

> Following up to my own post...
> 
> On Apr 6, 11:40 pm, John Ladasky <lada... at my-deja.com> wrote:
> 
>> What's up with that?
> 
> Apparently, "what's up" is that I will need to implement a third
> method in my ndarray subclass -- namely, __reduce__.
> 
> http://www.mail-archive.com/numpy-discussion@scipy.org/msg02446.html
> 
> I'm burned out for tonight, I'll attempt to grasp what __reduce__ does
> tomorrow.
> 
> Again, I'm going to point out that, given the extent that
> multiprocessing depends upon pickling, pickling should be made
> easier.  This is Python, for goodness' sake!  I'm still surprised at
> the hoops I've got to jump through.

Hi John,
My own experience has been that when I reach a surprising level of hoop jumping, it usually means there's an easier path somewhere else that I'm neglecting. 

But if pickling subclasses of numpy.ndarray objects is what you really feel you need to do, then yes, I think asking on the numpy list is the best idea. 


Good luck
Philip