[New-bugs-announce] [issue31654] ctypes should support atomic operations

[Daniel Colascione]

New submission from Daniel Colascione <dancol at google.com>:

Say we're using multiprocessing to share a counter between two processes and we want to atomically increment that counter. Right now, we need to protect that counter with a multiprocessing semaphore of some sort, then 1) acquire the semaphore, 2) read-modify-write the counter value, and 3) release the semaphore. What if we're preempted by a GIL-acquire request after step #1 but before step #3? We'll hold the semaphore until the OS scheduler gets around to running us again, which might be a while in the case of compute-bound tasks (especially if these tasks call C code that doesn't release the GIL).

Now, if some other process wants to increment the counter, it needs to wait on the first process's GIL! That partially defeats the purpose of multiprocessing: one of the nice things about multiprocessing is avoiding GIL-introduced latency!

If ctypes supported atomic operations, we could skip steps #1 and #3 entirely and operate directly on the shared memory region. Every operating system that supports threads at all also supports some kind of compare-and-exchange primitive. Compare-and-exchange is sufficient for avoiding the GIL contention I describe above.

----------
components: ctypes
messages: 303442
nosy: Daniel Colascione
priority: normal
severity: normal
status: open
title: ctypes should support atomic operations
type: enhancement
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue31654>
_______________________________________