What kind of "thread safe" are deque's actually?

[Chris Angelico]

On Tue, 28 Mar 2023 at 12:26, Travis Griggs <travisgriggs at gmail.com> wrote:
>
> A while ago I chose to use a deque that is shared between two threads. I did so because the docs say:
>
> "Deques support thread-safe, memory efficient appends and pops from either side of the deque with approximately the same O(1) performance in either direction.”
>
> (https://docs.python.org/3.11/library/collections.html?highlight=deque#collections.deque)
>
> Earlier today, looking through some server logs I noticed that from time to I’m getting a
>
> RuntimeError: deque mutated during iteration
>
> I guess this surprised me. When I see “thread safe”, I don’t expect to get errors.
>

I'd like to see your code, but here's an example with no threads
whatsoever that has the same error:

>>> from collections import deque
>>> q = deque([1, 2, 3, 4])
>>> for item in q:
...     if item % 2: q.append(item * 2)
...     print(item)
...
1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: deque mutated during iteration

This error comes from having an active iterator, then mutating the
deque, then continuing to iterate. That MAY be a result of threading,
but it isn't necessarily.

For threaded usage, I would recommend restricting yourself to
append/appendleft/pop/popleft (the standard mutators), with any
operations on the entire queue being done on a copy instead (either
q.copy() or list(q) depending on what you need). The act of taking a
copy should itself be thread-safe, and obviously anything done on a
separate copy will be independent of changes to the original.

ChrisA