[New-bugs-announce] [issue4751] Patch for better thread support in hashlib

[ebfe]

New submission from ebfe <knabberknusperhaus at yahoo.de>:

The hashlib functions provided by _hashopenssl.c hold the GIL all the
time although the underlying openssl-library is basically thread-safe.
I've attached a patch (svn diff) which basically does four things:

* If python is compiled with thread-support, the EVPobject is extended
by an additional PyThread_type_lock which protects the objects individually.
* The 'update' function releases the GIL if the to-be-hashed object is a
Bytes-object and therefor provides trustworthy locking (all other types,
including subclasses, are not trustworthy!). This allows multiple
threads to do hashing in parallel.
* The EVP_hash function removes duplicated code.
* The situation regarding unicode objects is now more meaningful. Upon
passing a unicode-string to the .update() function, the original hashlib
throws a "TypeError: object supporting the buffer API required" which is
confusing. I think it's perfectly valid not to accept unicode-strings as
input and people should required to call str.encode() upon their strings
before hashing, so a well-defined byte-representation of their strings
get hashed. Therefor I patched the MY_GET_BUFFER_VIEW_OR_ERROUT-macro to
throw "TypeError: Unicode-objects must be encoded before hashing". This
also fixes issue #1118


I've tested this patch and did not run into problems. CPU occupancy
relies on the buffer-size passed to .update() as releasing the GIL is
basically not worth the effort for very small buffers. More testing may
be needed...

----------
components: Library (Lib)
files: hashopenssl_threads.diff
keywords: patch
messages: 78297
nosy: ebfe
severity: normal
status: open
title: Patch for better thread support in hashlib
type: performance
versions: Python 3.0
Added file: http://bugs.python.org/file12453/hashopenssl_threads.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4751>
_______________________________________