[Cryptography-dev] Thread safety in the OpenSSL backend

[Alex Gaynor]

Maybe a method on `Binding` is the right place? Not sure, let's get the
code up, then we can find a place.

What do you mean "Enable the GIL"?

Alex


On Mon, Jan 20, 2014 at 12:25 PM, Alex Stapleton <alexs at prol.etari.at>wrote:

>  I've already implemented 2 as well so its not too awful :-)  Needs tests
> and things though.
>
> The current PR is all exposed on the binding so PyOpenSSL could easily use
> it. Where do you suggest a Python implementation should live? A utils
> module in the binding.openssl package?
>
> I failed to mention option 4 in my previous post too.
>
> 4. Enable the GIL on all of the OpenSSL bindings.
>
> Not entirely sure how you convince cffi to do that though. Might not be
> possible?
>
> Sent with AquaMail for Android
> http://www.aqua-mail.com
>
> On 20 January 2014 16:10:44 Alex Gaynor  wrote:
>
> How much would you hate me if I asked about doing a prototype of #2? It's
> the kind of performance issue that we can fix later very easily, but makes
> it much easier to review now.
>
> In general +1 on us fixing this, one suggestion though: this shouldn't be
> done in backend, it should be done somewhere ~~generic~~ so that pyOpenSSL
> can just call this function without needing to instantiating a backend.
>
> Alex
>
>
> On Mon, Jan 20, 2014 at 10:07 AM, alexs <alexs at prol.etari.at> wrote:
>
>> TLDR; We would like to merge a thread safety solution for the OpenSSL
>> bindings [1].
>>
>> Probably this https://github.com/pyca/cryptography/pull/377
>>
>> What do we need to do to get it reviewed and in master?
>>
>> As has been recently discovered by Hynek [2], initialising OpenSSL is
>> more difficult than it should be. This is complicated by the Python VM
>> doing its own initialisation of OpenSSL things inside the _ssl module.
>> Normally this works in our favour as Python happens to call the same
>> functions as us. If we have linked cryptography to a different version of
>> OpenSSL to the one Python was compiled against (such as with bottled
>> OpenSSL) however this can result in _ssl not doing any initialisation for
>> us. In particular, it wont setup the threading callback required to make
>> (among other things) the OpenSSL RNG thread safe. This is particularly
>> annoying on OS X which ships with OpenSSL 0.9.8 still.
>>
>> There are a few approaches to this.
>>
>>  1. Ignore it, only support Python OpenSSL, document that somewhere. Will
>> probably make OS X users sad.
>>  2. Expose the callback setup functions and implement locking entirely in
>> Python. This has 4-8x more overhead than the C version but is a small
>> amount of code, relatively easy to test and works cross-platform.
>>  3. Implement our own cross-platform locking wrappers and point OpenSSL
>> at those. This is fast but is harder to test well while we have no Windows
>> build infrastructure. Also puts even more code in C which we have no
>> coverage data on.
>>
>> I have written a PR that implements option 3: https://github.com/pyca/
>> cryptography/pull/377
>>
>> The code was mostly borrowed from PyPy, which is mostly borrowed from an
>> old version of CPython. It seems to solve my small crashing test cases on
>> POSIX.
>>
>> We could really use some people who feel confident (or perhaps even
>> competent) with threading/C/POSIX/WINAPI/OpenSSL etc to review this.
>> It's not had a lot of eyes on and I'm particularly uncomfortable with the
>> Windows parts since I lack the hardware (or software licenses) to test them.
>>
>> [1] https://github.com/pyca/cryptography/issues/330
>> [2] https://github.com/pyca/pyopenssl/issues/7
>> _______________________________________________
>> Cryptography-dev mailing list
>> Cryptography-dev at python.org
>> https://mail.python.org/mailman/listinfo/cryptography-dev
>>
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
>
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev
>
>


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20140120/596aa2a1/attachment-0001.html>