[Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

[Victor Stinner]

Le mer. 24 avr. 2019 à 01:44, Victor Stinner <vstinner at redhat.com> a écrit :
> The first requirement for the use case is that a Python debug build
> supports the ABI of a release build. (...) I
> propose to no longer imply Py_TRACE_REFS (...)
>
> Apart of Py_TRACE_REFS, I'm not aware of other ABI differences in
> structures. (...)

I tested manually: just by disabling Py_TRACE_REFS, the release ABI
looks *fully* compatible with a debug build!


I modified Python 3.7 to disable Py_TRACE_REFS and to omit "d" from
SOABI when build in debug mode. I built Python in debug mode.

I ran tests on numpy and lxml.etree: I can use .so libraries from
/usr/lib64/python3.7/site-packages (compiled in release mode), it just
works! I was very surprised of not getting any crash on such non
trivial C extension, so I checked manually that I was running a debug
build: yes, sys.gettotalrefcount is present :-)

I also wanted to test an even more complex application: I installed
gajim, a Jabber client written in Python 3 with PyGTK. It uses many C
extensions. Running Gajim with my debug build is slower, well, that's
not a surprise, but it works well! (no crash)


About the SOABI, maybe we should only keep "d" when Py_TRACE_REFS is
used, since technically, the ABI is same between release and debug
mode without Py_TRACE_REFS. In that case, pip doesn't need to be
modified ;-)


If you also want to try, use:

PYTHONPATH=/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages
./python /usr/bin/gajim

On a Python compiled with "./configure --with-pydebug && make" and the
following patch:

diff --git a/Include/object.h b/Include/object.h
index bcf78afe6b..4c807981c4 100644
--- a/Include/object.h
+++ b/Include/object.h
@@ -51,13 +51,8 @@ A standard interface exists for objects that
contain an array of items
 whose size is determined when the object is allocated.
 */

-/* Py_DEBUG implies Py_TRACE_REFS. */
-#if defined(Py_DEBUG) && !defined(Py_TRACE_REFS)
-#define Py_TRACE_REFS
-#endif
-
-/* Py_TRACE_REFS implies Py_REF_DEBUG. */
-#if defined(Py_TRACE_REFS) && !defined(Py_REF_DEBUG)
+/* Py_DEBUG implies Py_REF_DEBUG. */
+#if defined(Py_DEBUG) && !defined(Py_REF_DEBUG)
 #define Py_REF_DEBUG
 #endif

diff --git a/configure b/configure
index 2db11e6e86..7271e9de40 100755
--- a/configure
+++ b/configure
@@ -6365,7 +6365,6 @@ $as_echo "#define Py_DEBUG 1" >>confdefs.h
   { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
 $as_echo "yes" >&6; };
   Py_DEBUG='true'
-  ABIFLAGS="${ABIFLAGS}d"
 else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }; Py_DEBUG='false'
 fi
diff --git a/configure.ac b/configure.ac
index e5fb7e7b0b..fa4bb1944f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1246,7 +1246,6 @@ then
   [Define if you want to build an interpreter with many run-time checks.])
   AC_MSG_RESULT(yes);
   Py_DEBUG='true'
-  ABIFLAGS="${ABIFLAGS}d"
 else AC_MSG_RESULT(no); Py_DEBUG='false'
 fi],
 [AC_MSG_RESULT(no)])

Victor