[issue34410] itertools.tee not thread-safe; can segfault interpreter when wrapped iterator releases GIL

[Josh Rosenberg]

Josh Rosenberg <shadowranger+python at gmail.com> added the comment:

Carlo: The point of Xiang's post is that this is only tangentially related to multiprocessing; the real problem is that tee-ing an iterator implemented in Python (of which pool.imap_unordered is just one example) and using the resulting tee-ed iterators in multiple threads (which pool.imap_unordered does implicitly, as there is a thread involved in dispatching work).

The problem is *exposed* by multiprocessing.pool.imap_unordered, but it entirely a problem with itertools.tee, and as Xiang's repro indicates, it can be triggered easily without the complexity of multiprocessing being involved.

I've updated the bug title to reflect this.

----------
components: +Library (Lib)
nosy: +josh.r
title: Segfault/TimeoutError: itertools.tee of multiprocessing.pool.imap_unordered -> itertools.tee not thread-safe; can segfault interpreter when wrapped iterator releases GIL
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34410>
_______________________________________