pickle passing client/server design

[castironpi]

On Aug 22, 12:09 pm, DwBear75 <dwbea... at gmail.com> wrote:
> I am contemplating the need for a way to handle high speed data
> passing between two processes. One process would act as a queue that
> would 'buffer' data coming from another processes. Seems that the
> easiest way to handle the data would be to just pass pickles. Further,
> I'm thinking that using a unix domain socket would make this a simple
> way to pass high volumes of pickles. Are there any examples of an
> architecture like these, where a process is a client, sending pickles
> to a server listening on a domain socket?
>
> I'm am thinking there would be a need to have a semaphore, and some
> ACK or NACK that the server process got the whole pickle.

Use mmap for your data, and a pipe to wake the other side on changes.

With moderate extra work, you can even use ctypes structures to store
raw data directly, without pickle.