[issue38962] Reference leaks in subinterpreters

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

All the refleak build bots for master are reporting reference leaks in subinterpreter related tests:

https://buildbot.python.org/all/#/builders/126/builds/6/steps/5/logs/stdio
https://buildbot.python.org/all/#/builders/144/builds/6/steps/5/logs/stdio
https://buildbot.python.org/all/#/builders/206/builds/6
https://buildbot.python.org/all/#/builders/157/builds/6/steps/4/logs/stdio

......
test_atexit leaked [882, 882, 882] references, sum=2646
test_atexit leaked [12, 12, 12] memory blocks, sum=36
5 tests failed again:
    test__xxsubinterpreters test_atexit test_capi test_httpservers
    test_threading
== Tests result: FAILURE then FAILURE ==
401 tests OK.
10 slowest tests:
- test_asyncio: 33 min 34 sec
- test_concurrent_futures: 17 min 22 sec
- test_multiprocessing_spawn: 17 min 6 sec
- test_zipfile: 9 min 25 sec
- test_multiprocessing_forkserver: 9 min 2 sec
- test_multiprocessing_fork: 8 min 43 sec
- test_largefile: 7 min 32 sec
- test_lib2to3: 7 min 3 sec
- test_mailbox: 6 min 27 sec
- test_argparse: 5 min 5 sec
5 tests failed:
    test__xxsubinterpreters test_atexit test_capi test_httpservers
    test_threading
14 tests skipped:
    test_devpoll test_gdb test_ioctl test_kqueue test_msilib
    test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
    test_winconsoleio test_winreg test_winsound test_zipfile64
7 re-run tests:
    test__xxsubinterpreters test_atexit test_capi test_httpservers
    test_nntplib test_pty test_threading


Bisecting shows the following commit as the culprit:

ef5aa9af7c7e493402ac62009e4400aed7c3d54e is the first bad commit
commit ef5aa9af7c7e493402ac62009e4400aed7c3d54e
Author: Victor Stinner <vstinner at python.org>
Date:   Wed Nov 20 00:38:03 2019 +0100

    bpo-38858: Reorganize pycore_init_types() (GH-17265)
    
    * Call _PyLong_Init() and _PyExc_Init() earlier
    * new_interpreter() reuses pycore_init_types()

 Python/pylifecycle.c | 31 +++++++++++--------------------
 1 file changed, 11 insertions(+), 20 deletions(-)
bisect run success


Running * test.test_atexit.SubinterpreterTest.test_callbacks_leak is enough for reproducing the problem.

----------
assignee: vstinner
components: Tests
messages: 357759
nosy: pablogsal, vstinner
priority: high
severity: normal
stage: needs patch
status: open
title: Reference leaks in subinterpreters
type: resource usage
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38962>
_______________________________________