[issue34410] Segfault/TimeoutError: itertools.tee of multiprocessing.pool.imap_unordered

[Xiang Zhang]

Xiang Zhang <angwerzx at 126.com> added the comment:

It seems to me the problem is tee objects might encounter race conditions while  `PyIter_Next` in `teedataobject_getitem` releases GIL. Other threads then might get into the same branch since `tdo->numread` haven't been updated yet. NULL slots are generated then, 2 objects are read from the underlying iterator and `tdo->numread` is updated twice while only one slot is set.

As for multiprocessing.pool, there is a background task handling thread consuming one tee object and main thread consuming another one. The underlying iterator is `IMapIterator` which `next` method would block on a condition.

While trying, I find the following snippet would also crash:

import threading
import itertools

class C:
    def __iter__(self):
        return self
    def __next__(self):
        return 1

def test(i):
    print(list(i))

i1, i2 = itertools.tee(C())
threading.Thread(target=test, args=(i1,)).start()
print(list(i2))

GDB shows it crashs in `teedataobject_dealloc` -> `teedataobject_clear`. I haven't understood what happened.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34410>
_______________________________________