[issue29048] Coverage influence tests, make some of them fail

[Patrik Iselind]

New submission from Patrik Iselind:

I checkout the latest tip from scratch.

When i run the tests i get the following results:
% ./python -m test -j
[...]
376 tests OK.

1 test altered the execution environment:
    test_site

27 tests skipped:
    test_bz2 test_ctypes test_curses test_dbm_gnu test_dbm_ndbm
    test_devpoll test_gdb test_kqueue test_lzma test_msilib
    test_ossaudiodev test_readline test_smtpnet test_socketserver
    test_sqlite test_startfile test_timeout test_tix test_tk
    test_ttk_guionly test_urllib2net test_urllibnet test_winconsoleio
    test_winreg test_winsound test_xmlrpc_net test_zipfile64

Total duration: 2 min 57 sec
Tests result: SUCCESS

Without any changes i run the tests again but this time under coverage. I get a completely different result:
% ./python ../coveragepy run --pylib Lib/test/regrtest.py
[...]
328 tests OK.

8 tests failed:
    test_exceptions test_frame test_ssl test_subprocess test_super
    test_traceback test_unittest test_xml_etree_c

41 tests altered the execution environment:
    test_asynchat test_asyncio test_asyncore test_capi
    test_concurrent_futures test_decimal test_distutils test_docxmlrpc
    test_email test_fork1 test_ftplib test_functools test_gc
    test_hashlib test_httplib test_httpservers test_imaplib
    test_import test_io test_json test_logging test_mailbox
    test_multiprocessing_fork test_multiprocessing_forkserver
    test_multiprocessing_spawn test_nntplib test_os test_poll
    test_poplib test_pydoc test_queue test_robotparser test_sched
    test_smtplib test_socket test_sys test_telnetlib
    test_threaded_import test_threadedtempfile test_threading
    test_xml_etree

27 tests skipped:
    test_bz2 test_ctypes test_curses test_dbm_gnu test_dbm_ndbm
    test_devpoll test_gdb test_kqueue test_lzma test_msilib
    test_ossaudiodev test_readline test_smtpnet test_socketserver
    test_sqlite test_startfile test_timeout test_tix test_tk
    test_ttk_guionly test_urllib2net test_urllibnet test_winconsoleio
    test_winreg test_winsound test_xmlrpc_net test_zipfile64

Total duration: 92 min 34 sec
Tests result: FAILURE

Is the reason for this difference known? Coverage shouldn't affect the outcome of the tests right?

----------
components: Tests
messages: 283828
nosy: patriki
priority: normal
severity: normal
status: open
title: Coverage influence tests, make some of them fail
type: behavior
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29048>
_______________________________________