[issue9146] Segfault in hashlib in OpenSSL FIPS mode using non-FIPS-compliant hashes, if "ssl" imported before "hashlib"

[Dave Malcolm]

Dave Malcolm <dmalcolm at redhat.com> added the comment:

I'm attaching an updated patch which:
  - adds error checking to the various places where EVP_DigestInit is called
  - adds a test to test_hashlib to ensure that hashlib still works gracefully when OPENSSL_FORCE_FIPS_MODE=1 is set in the environment

Note that in this mode:
>>> _hashlib.new('md5')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: error:060800A0:digital envelope routines:EVP_DigestInit_ex:unknown cipher
[57670 refs]

but hashlib falls back to using the "md5" module instead.

I started writing a test for _hashlib (as opposed to hashlib), but it's too hard to express a runtime conditional on whether OPENSSL_FORCE_FIPS_MODE will actually affect the behavior of EVP_DigestInit across the versions of openssl that might be installed on the system.

I'm still waiting to hear back from the Fedora OpenSSL packager for info on how to reproduce this on a vanilla OpenSSL.

----------
Added file: http://bugs.python.org/file17884/hashopenssl-fips-mode-errors-v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9146>
_______________________________________