[Cryptography-dev] Bundling OpenSSL

[Jean-Paul Calderone]

On 10/19/2013 08:26 PM, Donald Stufft wrote:
> I've started working on bundling OpenSSL within cryptography and fixing the packaging up.
>
> There are a number of things I'd like to do that will make things smoother in the long run but require some changes right now. These things are:
>
> Disable implicit compile
>     - This masks problems with the packaging. I tried leaving it on when doing the same thing for PyNaCl and it caused immense frustration with strange edge cases breaking. Further more the cffi devs say they are planning on removing it anyways.
I agree that implicit compile is a little bit crazy.  It ends up
invoking a ton of code when the user probably doesn't expect it.

The only concern I have with disabling this is that it sounds like the
outcome of doing it naively means that source and compiled modules can
easily get out of sync resulting in confusing errors.

It would be nice if there could at *least* be a warning emitted if the
source and compiled modules are mismatched.

> Move the cryptography directory into a src directory
>     - Since cryptography will require an implicit compile step with the above, it's easy to accidentally import the cryptography that is sitting on the file system that hasn't been compiled yet, moving it means you have to install cryptography somehow to import it (or cd into the src directory). This should lower confusing errors when ``python -c 'import cryptography'`` does work. Further since we'll need a folder for the OpenSSL code it makes sense I think to have a single folder for all the things we have in repo, see: https://github.com/pyca/pynacl/tree/master/src

I don't like this idea.  It sounds like the reasoning behind it is
"explicitly compilation makes things more inconvenient so we should
re-arrange the repository to make things even *more* inconvenient than
that" and then somehow conflating "inconvenient" with "not confusing". 
Moreover, I don't find the explanation that people will be confused if
they try to use cryptography without compiling it compelling.  If you
want to remove confusing errors from that usage, detect missing compiled
artifacts and raise an explicit exception about the problem.

"src" directories are a holdover from ancient history when tools worked
differently.  It's almost always pointless now (even if the entire
project is written in C).

>
> Lazily load the compiled library
>     - Right now the compiled library is loaded when the cryptography.bindings.openssl.api library is imported, this is the way CFFI currently works. As i've done in PyNaCl i'd like to make the Api object lazy so that it loads the compiled library on first access and proxies the calls to the exposed functions/variables to the lazily loaded underlying library object. You can see this at: https://github.com/pyca/pynacl/blob/master/src/nacl/_lib/__init__.py#L59-L84

Why?  If someone imports cryptography.bindings.openssl.api then isn't it
pretty likely they're going to use something from the compiled library? 
What is the scenario in which this results in less work being done?

Jean-Paul

> The final solution to this would be that we have a library that is always built at build time with a consistent version of OpenSSL. 
>
> Thoughts on these steps? 
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
>
> _______________________________________________
> Cryptography-dev mailing list
> Cryptography-dev at python.org
> https://mail.python.org/mailman/listinfo/cryptography-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20131020/3ae07223/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/cryptography-dev/attachments/20131020/3ae07223/attachment.sig>