[Cryptography-dev] Adding set_cert_store to pyOpenSSL - patches

[Barry Scott]

On Wednesday, 14 October 2020 10:52:24 BST Barry Scott wrote:
> In an app that uses twisted that uses pyOpenSSL I found that
> to it takes 1s CPU bound to setup a TLS session because the
> twisted code copies the trust store into the context one cert
> at a time.
> 
> I'm using openssl-1.1.1g and python 2.7.18 (yes I know...).
> 
> Here is the function in twisted that causes the 1s CPU bound loop:
> 
> class OpenSSLCertificateAuthorities(object):
>     def __init__(self, caCerts):
>         self._caCerts = caCerts
> 
>     def _addCACertsToContext(self, context):
>         store = context.get_cert_store()
>         for cert in self._caCerts:
>             store.add_cert(cert)
> 
> The obvious way to fix this is to setup the X509Store at app
> startup with the trusted certs. Then set that store on the context.
> 
> The new code would be:
> 
> class OpenSSLCertificateAuthorities(object):
>     def __init__(self, caCerts):
>         self._caCerts = caCerts
>         self._store = X509Store()
>         for cert in self._caCerts:
>             self._store.add_cert(cert)
> 
>     def _addCACertsToContext(self, context):
>         context.set_cert_store(self._store)
> 
> And the patch to pyOpenSSL is:
> 
> --- tmp1/pyopenssl-19.1.0/src/OpenSSL/SSL.py    2019-11-18 04:47:22.000000000 +0000
> +++ tmp2/pyopenssl-19.1.0/src/OpenSSL/SSL.py    2020-10-13 15:11:02.255560148 +0100
> @@ -1357,6 +1357,14 @@
>          pystore._store = store
>          return pystore
>  
> +    def set_cert_store(self, store):
> +        """
> +        Set the certificate store for the context.
> +
> +        :store: A X509Store object or None if it does not have one.
> +        """
> +        _lib.SSL_CTX_set_cert_store(self._context, store)
> +
>      def set_options(self, options):
>          """
>          Add options. Options set before are not cleared!
> 
> But I see this exception:
> 
>   File "ngtls_context_set.py", line 107, in _addCACertsToContext
>     context.set_cert_store(self._store)
>   File "/usr/local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1366, in set_cert_store
>     _lib.SSL_CTX_set_cert_store(self._context, store)
> TypeError: initializer for ctype 'X509_STORE *' must be a cdata pointer, not X509Store
> 
> My searching has not lead me to a way to get a cdata pointer fpr X509Store.
> 
> What do I need to do to get set_cert_store working?

Here are the set of patches that I used to add set_cert_store and get the time to setup a context
from ~1000ms to 0.3ms:

Twisted needs access to set_cert_store.

set_cert_store is missing from cryptography
set_cert_store requires the use of X509_STORE_up_ref

X509_STORE_up_ref is missing from cryptography.

diff -r -u tmp1/cryptography-2.1.4/src/_cffi_src/openssl/x509_vfy.py tmp2/cryptography-2.1.4/src/_cffi_src/openssl/x509_vfy.py
--- tmp1/cryptography-2.1.4/src/_cffi_src/openssl/x509_vfy.py   2017-11-30 01:53:32.000000000 +0000
+++ tmp2/cryptography-2.1.4/src/_cffi_src/openssl/x509_vfy.py   2020-10-15 16:56:44.893834630 +0100
@@ -142,6 +142,7 @@
 int X509_STORE_set_default_paths(X509_STORE *);
 int X509_STORE_set_flags(X509_STORE *, unsigned long);
 void X509_STORE_free(X509_STORE *);
+int X509_STORE_up_ref(X509_STORE *);
 
 /* X509_STORE_CTX */
 X509_STORE_CTX *X509_STORE_CTX_new(void);


--- tmp1/pyopenssl-19.1.0/src/OpenSSL/SSL.py    2019-11-18 04:47:22.000000000 +0000
+++ tmp2/pyopenssl-19.1.0/src/OpenSSL/SSL.py    2020-10-16 11:14:06.933208747 +0100
@@ -1357,6 +1357,15 @@
         pystore._store = store
         return pystore
 
+    def set_cert_store(self, store):
+        """
+        Set the certificate store for the context.
+
+        :store: A X509Store object or None if it does not have one.
+        """
+        _lib.X509_STORE_up_ref( store._store )
+        _lib.SSL_CTX_set_cert_store(self._context, store._store)
+
     def set_options(self, options):
         """
         Add options. Options set before are not cleared!

And in twisted:

--- tmp1/twisted-twisted-19.7.0/src/twisted/internet/_sslverify.py      2019-07-28 10:17:29.000000000 +0100
+++ tmp2/twisted-twisted-19.7.0/src/twisted/internet/_sslverify.py      2020-10-16 12:03:54.653035837 +0100
@@ -963,12 +963,13 @@
         @type caCerts: L{list} of L{OpenSSL.crypto.X509}
         """
         self._caCerts = caCerts
+        self._store = X509Store()
 
+        for cert in self._caCerts:
+            self._store.add_cert(cert)
 
     def _addCACertsToContext(self, context):
-        store = context.get_cert_store()
-        for cert in self._caCerts:
-            store.add_cert(cert)
+        context.set_cert_store(self._store)

Do you want these patches as a PR?

Barry