[New-bugs-announce] [issue36576] Some test_ssl and test_asyncio tests fail with OpenSSL 1.1.1 on Python 3.4 and 3.5

[STINNER Victor]

New submission from STINNER Victor <vstinner at redhat.com>:

On Fedora 29, test_ssl and test_asyncio when Python 3.5 is linked with OpenSSL 1.1.1b (Fedora package openssl-1.1.1b-3.fc29.x86_64):

test_ssl:

* test_options (test.test_ssl.ContextTests)
* test_alpn_protocols (test.test_ssl.ThreadedTests)
* test_default_ecdh_curve (test.test_ssl.ThreadedTests)
* test_shared_ciphers (test.test_ssl.ThreadedTests)

test_asyncio:

* test_create_server_ssl_match_failed (test.test_asyncio.test_events.EPollEventLoopTests)
* test_create_server_ssl_match_failed (test.test_asyncio.test_events.PollEventLoopTests)
* test_create_server_ssl_match_failed (test.test_asyncio.test_events.SelectEventLoopTests)

Fixing these tests would require to backport some ssl features, and I don't think that it's worth it.

Attached PR 12694 skip these tests on OpenSSL 1.1.1.

Note: these tests pass with OpenSSL 1.1.0.

FYI for Fedora, we also care of having the Python 3.4 test suite passing with OpenSSL 1.1.1 and so we will maintain a similar change downstream.


======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 866, in test_options
    self.assertEqual(default, ctx.options)
AssertionError: 2181169236 != 2182217812

======================================================================
FAIL: test_alpn_protocols (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 3205, in test_alpn_protocols
    self.assertIsInstance(stats, ssl.SSLError)
AssertionError: {'client_alpn_protocol': None, 'server_alpn_protocols': [None], 'version': 'TLSv1.2', 'client_npn_protocol': None, 'server_npn_protocols': [None], 'server_shared_ciphers': [[('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256), ('TLS_CHACHA20_POLY1305_SHA256', 'TLSv1.3', 256), ('TLS_AES_128_GCM_SHA256', 'TLSv1.3', 128), ('TLS_AES_128_CCM_SHA256', 'TLSv1.3', 128), ('ECDHE-ECDSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('ECDHE-RSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('DHE-DSS-AES256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-DSS-AES128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-CCM8', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-CCM', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-AES256-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-SHA', 'TLSv1.0', 256), ('ECDHE-RSA-AES256-SHA', 'TLSv1.0', 256), ('DHE-RSA-AES256-CCM8', 'TLSv1.2', 256), ('DHE-RSA-AES256-CCM', 'TLSv1.2', 256), ('DHE-RSA-AES256-SHA256', 'TLSv1.2', 256), ('DHE-DSS-AES256-SHA256', 'TLSv1.2', 256), ('DHE-RSA-AES256-SHA', 'SSLv3', 256), ('DHE-DSS-AES256-SHA', 'SSLv3', 256), ('ECDHE-ECDSA-AES128-CCM8', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-CCM', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-AES128-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-SHA', 'TLSv1.0', 128), ('ECDHE-RSA-AES128-SHA', 'TLSv1.0', 128), ('DHE-RSA-AES128-CCM8', 'TLSv1.2', 128), ('DHE-RSA-AES128-CCM', 'TLSv1.2', 128), ('DHE-RSA-AES128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-AES128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-AES128-SHA', 'SSLv3', 128), ('DHE-DSS-AES128-SHA', 'SSLv3', 128), ('ECDHE-ECDSA-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-CAMELLIA256-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-CAMELLIA256-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-RSA-ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-DSS-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CAMELLIA256-SHA256', 'TLSv1.2', 256), ('DHE-DSS-CAMELLIA256-SHA256', 'TLSv1.2', 256), ('DHE-RSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CAMELLIA256-SHA', 'SSLv3', 256), ('DHE-DSS-CAMELLIA256-SHA', 'SSLv3', 256), ('DHE-RSA-CAMELLIA128-SHA', 'SSLv3', 128), ('DHE-DSS-CAMELLIA128-SHA', 'SSLv3', 128), ('AES256-GCM-SHA384', 'TLSv1.2', 256), ('AES128-GCM-SHA256', 'TLSv1.2', 128), ('AES256-CCM8', 'TLSv1.2', 256), ('AES256-CCM', 'TLSv1.2', 256), ('AES128-CCM8', 'TLSv1.2', 128), ('AES128-CCM', 'TLSv1.2', 128), ('AES256-SHA256', 'TLSv1.2', 256), ('AES128-SHA256', 'TLSv1.2', 128), ('AES256-SHA', 'SSLv3', 256), ('AES128-SHA', 'SSLv3', 128), ('ARIA256-GCM-SHA384', 'TLSv1.2', 256), ('ARIA128-GCM-SHA256', 'TLSv1.2', 128), ('CAMELLIA256-SHA256', 'TLSv1.2', 256), ('CAMELLIA128-SHA256', 'TLSv1.2', 128), ('CAMELLIA256-SHA', 'SSLv3', 256), ('CAMELLIA128-SHA', 'SSLv3', 128)]], 'peercert': {}, 'cipher': ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), 'compression': None} is not an instance of <class 'ssl.SSLError'>

======================================================================
FAIL: test_default_ecdh_curve (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 3064, in test_default_ecdh_curve
    self.assertIn("ECDH", s.cipher()[0])
AssertionError: 'ECDH' not found in 'TLS_AES_256_GCM_SHA384'

======================================================================
FAIL: test_shared_ciphers (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_ssl.py", line 3381, in test_shared_ciphers
    self.fail(name)
AssertionError: TLS_AES_256_GCM_SHA384


======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.EPollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.PollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.SelectEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/3.5/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

----------
components: Tests
messages: 339756
nosy: vstinner
priority: normal
severity: normal
status: open
title: Some test_ssl and test_asyncio tests fail with OpenSSL 1.1.1 on Python 3.4 and 3.5
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36576>
_______________________________________