array.array()'s memory shared with multiprocessing.Process()

[iurly]

Il giorno domenica 10 settembre 2017 18:53:33 UTC+2, MRAB ha scritto:
> On 2017-09-10 12:40, gerlando.falauto at gmail.com wrote:
> >> 
> >> I suspect it's down to timing.
> >> 
> >> What you're putting into the queue is a reference to the array, and it's 
> >> only some time later that the array itself is pickled and then sent (the 
> >> work being done in the 'background').
> >> 
> >> Modifying the array before (or while) it's actually being sent would 
> >> explain the problem you're seeing.
> > 
> > That would also have been my guess. However, according to documentation:
> > 
> >> When an object is put on a queue, the object is pickled and a background 
> >> thread later flushes the pickled data to an underlying pipe.
> > 
> > In my understanding this means the object is pickled *before* the background thread takes care of flushing the data to the pipe. Is that a mistake in the documentation then?
> > 
> > Any suggestion for a way to work around this limitation?
> > Or perhaps a different approach altogether I could use to reduce CPU load?
> > What the main thread actually does is dequeue data from a high-speed USB-to-serial (2.000.000 bps), that's why I came up with the array.array() solution to store collected data, hoping for the smallest possible overhead.
> > Thanks!
> > 
> I've had a quick look at the source code.
> 
> When an object is put into the queue, it's actually put into an internal 
> buffer (a deque), and then the method returns.
> 
> An internal thread works through the buffer and pickles the objects.
> 
> Therefore, just because the method has returned, it doesn't mean that 
> it's now safe to modify the object.

I see. So that explains everything. However, I wonder if that's the intended behavior and/or that should be documented somehow.
As far as I'm concerned, I'm probably better off using double buffers to avoid this kind of issues.
Thanks a lot for your help!