[IronPython] Problems with hashlib and threading

[Brian Merrell]

Thanks!  Given that useful multi-threaded hashing is only available because
of IronPython's lack of GIL I can't complain too much :-)    For reference,
here is the issue link:
http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=20324

I look forward to the fix in a future release.
On Sun, Jan 4, 2009 at 7:09 PM, Curt Hagenlocher <curt at hagenlocher.org>wrote:

> Not without looking at the source code...
>
> We want to be at least as thread-safe as CPython, with the exception that
> we have no interest in implementing the equivalent of a "global interpreter
> lock" :).  This particular case is just bad coding on our part.  I can't
> promise you that there aren't other examples, but at least this one is
> pretty easy to fix.
>
>
> On Sun, Jan 4, 2009 at 8:58 AM, Brian Merrell <brian at merrells.org> wrote:
>
>> Thanks Curt.  I've filed a bug report.  It makes me a bit nervous
>> realizing that it isn't entirely transparent when shared memory/objects are
>> being used.  Normally I would think a local instance of hashlib.md5() would
>> be thread-safe from other local instances.  Is there any documentation of
>> what other modules use global/static instances or other non-thread-safe
>> implementations in IronPython?
>>
>>
>> On Tue, Dec 30, 2008 at 12:13 PM, Curt Hagenlocher <curt at hagenlocher.org>wrote:
>>
>>> I'm going to guess that this is because we use a static instance
>>> of MD5CryptoServiceProvider to do the hashing, something which may not be
>>> thread-safe.  It looks like all of the IronPython crypto implementations do
>>> the same thing, so the SHA functions may have a similar issue.
>>> Please file this as a bug on Codeplex.
>>>
>>> On Sat, Dec 20, 2008 at 1:56 PM, Brian Merrell <brian at merrells.org>wrote:
>>>
>>>> I get the following error periodically:
>>>>
>>>> Exception in thread Thread-2:
>>>> Traceback (most recent call last):
>>>>   File "C:\Program Files\IronPython 2.0\Lib\threading.py", line 486, in
>>>> _Thread__bootstrap_inner
>>>>     self.run()
>>>>   File "initial_ingest.py", line 33, in run
>>>>     md5 = hashlib.md5(contents).hexdigest()
>>>> SystemError: Hash not valid for use in specified state.
>>>>
>>>> I am not sharing a md5() object between threads and have tried to avoid
>>>> any shared memory issues:
>>>>
>>>>
>>>> class Convert(threading.Thread):
>>>>     def __init__(self, queue):
>>>>         threading.Thread.__init__(self)
>>>>         self.queue = queue
>>>>     def run(self):
>>>>         import hashlib
>>>>         while True:
>>>>             abs_filename = self.queue.get()
>>>>             f = open(abs_filename, "rb")
>>>>             contents = f.read()
>>>>             f.close()
>>>>             md5 = hashlib.md5(contents).hexdigest()
>>>>             print md5, abs_filename
>>>>             self.queue.task_done()
>>>>
>>>> Any ideas why this is failing?  Thanks,
>>>>
>>>> -brian
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.ironpython.com
>>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.ironpython.com
>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>
>>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20090104/4699c5e1/attachment.html>