Problem with copy.deepcopy and multiprocessing.Queue

[James DeVincentis]

I think I tracked this down and resolved it.

It appears taking an object from a multiprocess.Queue and placing it into a queue.Queue is a no-no even if the queue.Queue isn’t shared across processes. 

I have a series of workers (multiprocessing) that share a multiprocess.Queue across all processes. Each worker has it’s own local queue (Queue.queue) that is shared within only it’s own threads. It appears that when taking an object from a multiprocessing queue and moving it into a Queue.queue it breaks things. I don’t know what or how, but it does. As soon as I switched the local queue to a multiprocess.Queue it fixed the issue. 

Kind of odd to me. Not sure if anyone wants to look into it.

> On Oct 15, 2015, at 5:42 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> 
> On Thu, Oct 15, 2015 at 4:02 PM, James DeVincentis <admin at hexhost.net> wrote:
> >
> > Anyone have any ideas? I feel like this could be a bug with the garbage collector across multiprocessing.
> 
> I'll second MRAB's response from yesterday: could it just be reusing space that it has recently freed?
> 
> As a debugging measure, what happens if you try to keep the objects in memory, e.g. by adding them to some long-lived list? Do you start seeing different addresses?