[issue37652] Multiprocessing shared_memory ValueError on race with ShareableList

[Pierre Glaser]

Pierre Glaser <pierreglaser at msn.com> added the comment:

The root of the error is that struct.pack_into starts by memsetting the underlying memory area with NULL bytes before filling the data with memcpy. If ShareableList._get_packing_format is called between the two operations (through a concurrent __getitem__ call from another process), struct.unpack_from will return an empty tuple which is the direct cause of the error you're seeing.

In the general case though, memcpy is not atomic so even without the memset call before, results of struct.unpack_from may be invalid in a concurrent setting.

shared_memory is a low level python module. Precautions should be made when handling concurrently the shared_memory objects using synchronization primitives for example. I'm not sure this should be done internally in the SharedMemory class -- especially, we don't want to slow down concurrent READ access. +1 For a documentation addition.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37652>
_______________________________________