[issue15108] Incomplete tuple created by PyTuple_New() and accessed via the GC can trigged a crash

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

The general issue here is a the PyTuple_New() is unsafe: it immediately tracks the newly created tuple in the GC, whereas the tuple is not initialized yet. If the GIL is released before the tuple is fully populated and something access to this tuple via the GC (ex: gc.get_objects()), accessing the tuple can crash, especially in the Python land (for example, repr(the_tuple) is likely to crash).

IMO the unsafe PyTuple_New() API should be avoided. For example, allocate an array of PyObject* on the stack memory, and then call _PyTuple_FromArray(). This API is safe because it only tracks the tuple once it's fully initialized, and it calls INCREF on items. Problem: this safe and efficient API is currently private.

There are other safe alternatives like Py_BuildValue("(OOO)", item1, item2, item3).

_pysqlite_fetch_one_row() calls PyTuple_New() and releases the GIL at each sqlite3_column_type() call, so yeah, it has this exact bug. By the way, it doesn't check for PyTuple_SetItem() failure, whereas it's currently possible that there is more than one strong reference to the tuple which is being populated (because of the GC issue).

PyTuple_New() is ok-ish if there is no way to trigger a GC collection and if the GIL cannot be released until the tuple is fully initialized.

Maybe we need a private _PyTuple_NewUntracked() API to create a tuple which is not tracked by the GC, and also a _PyTuple_ResizeUntracked() API. By the way, _PyTuple_Resize() sounds like a nonsense since a tuple is supposed to be immutable ;-)

----------
nosy: +vstinner
title: ERROR: SystemError: ./../Objects/tupleobject.c:118: bad argument to internal function -> Incomplete tuple created by PyTuple_New() and accessed via the GC can trigged a crash

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue15108>
_______________________________________