[Python-checkins] Raise a RuntimeError when tee iterator is consumed from different threads (GH-15567)
Raymond Hettinger
webhook-mailer at python.org
Wed Aug 28 23:39:37 EDT 2019
https://github.com/python/cpython/commit/fa220ec7633e9674baccc28dde987f29d7f65141
commit: fa220ec7633e9674baccc28dde987f29d7f65141
branch: master
author: HongWeipeng <961365124 at qq.com>
committer: Raymond Hettinger <rhettinger at users.noreply.github.com>
date: 2019-08-28T20:39:25-07:00
summary:
Raise a RuntimeError when tee iterator is consumed from different threads (GH-15567)
files:
A Misc/NEWS.d/next/Library/2019-08-29-10-26-57.bpo-34410.ttCIpm.rst
M Doc/library/itertools.rst
M Modules/itertoolsmodule.c
diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
index a3f403a5b40b..8e7899e6b613 100644
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -643,7 +643,8 @@ loops that truncate the stream.
Once :func:`tee` has made a split, the original *iterable* should not be
used anywhere else; otherwise, the *iterable* could get advanced without
- the tee objects being informed.
+ the tee objects being informed. the :func:`tee` iterator can not be consumed
+ from different threads, even if an underlying iterator is thread-safe.
This itertool may require significant auxiliary storage (depending on how
much temporary data needs to be stored). In general, if one iterator uses
diff --git a/Misc/NEWS.d/next/Library/2019-08-29-10-26-57.bpo-34410.ttCIpm.rst b/Misc/NEWS.d/next/Library/2019-08-29-10-26-57.bpo-34410.ttCIpm.rst
new file mode 100644
index 000000000000..caddada835fc
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2019-08-29-10-26-57.bpo-34410.ttCIpm.rst
@@ -0,0 +1,2 @@
+Raise a RuntimeError when itertools.tee() iterator is consumed from different
+threads. Patch by hongweipeng.
diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c
index 781d0cca5419..101addcfd3a9 100644
--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -452,6 +452,7 @@ typedef struct {
teedataobject *dataobj;
int index; /* 0 <= index <= LINKCELLS */
PyObject *weakreflist;
+ unsigned long thread_id;
} teeobject;
static PyTypeObject teedataobject_type;
@@ -680,6 +681,11 @@ tee_next(teeobject *to)
{
PyObject *value, *link;
+ if (to->thread_id != PyThread_get_thread_ident()) {
+ PyErr_SetString(PyExc_RuntimeError,
+ "tee() iterator can not be consumed from different threads.");
+ return NULL;
+ }
if (to->index >= LINKCELLS) {
link = teedataobject_jumplink(to->dataobj);
if (link == NULL)
@@ -713,6 +719,7 @@ tee_copy(teeobject *to, PyObject *Py_UNUSED(ignored))
newto->dataobj = to->dataobj;
newto->index = to->index;
newto->weakreflist = NULL;
+ newto->thread_id = to->thread_id;
PyObject_GC_Track(newto);
return (PyObject *)newto;
}
@@ -745,6 +752,7 @@ tee_fromiterable(PyObject *iterable)
to->index = 0;
to->weakreflist = NULL;
+ to->thread_id = PyThread_get_thread_ident();
PyObject_GC_Track(to);
done:
Py_XDECREF(it);
More information about the Python-checkins
mailing list