From radix at twistedmatrix.com Thu Apr 17 16:37:30 2014 From: radix at twistedmatrix.com (Christopher Armstrong) Date: Thu, 17 Apr 2014 10:37:30 -0400 Subject: [Cryptography-dev] Towards a new TLS API Message-ID: If you?re eager, the link is at the bottom of this email. I?m not sure I have it in me to describe the context of this project from the beginning, so I will start in the middle: a bunch of people have agreed that designing a new high-level TLS API for Python is a good idea. At PyCon, I sprinted on designing this API with several other people, and got input from many people in the PyCA community: - Ying Li - Paul Kehrer - Jean-Paul Calderone - Laurens Van Houtven - Alex Gaynor - David Reid - Hynek Schlawack - Corbin Simpson - Aaron Gallagher And probably some other people that I missed. It?s still in a very rough state, and certainly needs much more serious thought (as well as some bikeshedding). I?ll describe some of the basic tenets predicating the design: - It will be be easy to use! - It will be opinionated about which math and TLS versions to use, and not allow downgrading to weaker security - It will have no IO (deal only with in-memory buffers) - It will be implemented with multiple backends, such as OpenSSL, SecureTransport, PyTLS, etc. - It will have no global state - It will not allow disabling of security features such as basic security checks, chain validation and hostname validation. - It will support both client and server operation. - We may expose less safe and more flexible lower-level APIs, but they will be clearly delineated from the API that people *should* be using. In addition, we may implement the protocol from scratch (of course using existing cryptography backends) in the future. Some of us feel strongly that protocol code should not be implemented in C (or any memory-unsafe language). I believe that the PyCA community has enough expertise to eventually produce a secure TLS implementation. Right now the design is a sphinx-style .rst file in a branch of cryptography in my github account (mostly so I can run ?make html? on it and get pretty output). Unfortunately this means I?m the only one who can commit to it or accept PRs, so if anyone has a suggestion to improve this situation I?d be happy to oblige. The direct link to the file is:?https://raw.githubusercontent.com/radix/cryptography/tls-api/docs/tls-api.rst The branch is:?https://github.com/radix/cryptography/tree/tls-api Any input would be greatly appreciated. Make sure to check the ?TODO? and ?Future work? sections of the document at the end in case you?re reading through it and immediately think of something to shout about :) PRs are appreciated. Thanks! --? Christopher Armstrong http://twitter.com/radix http://wordeology.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From radix at twistedmatrix.com Sat Apr 19 02:03:21 2014 From: radix at twistedmatrix.com (Christopher Armstrong) Date: Fri, 18 Apr 2014 19:03:21 -0500 Subject: [Cryptography-dev] Towards a new TLS API In-Reply-To: References: Message-ID: Quick note: I?ve created a PR for this proposed document which should make it a lot easier to roundtrip on reviews: https://github.com/radix/cryptography/pull/1 It?s against my own fork of cryptography just so there?s no risk of merging it to cryptography master. Please comment there, and I?ll update the document as needed. --? Christopher Armstrong http://twitter.com/radix http://wordeology.com/ On April 17, 2014 at 9:37:33 AM, Christopher Armstrong (radix at twistedmatrix.com) wrote: If you?re eager, the link is at the bottom of this email. I?m not sure I have it in me to describe the context of this project from the beginning, so I will start in the middle: a bunch of people have agreed that designing a new high-level TLS API for Python is a good idea. At PyCon, I sprinted on designing this API with several other people, and got input from many people in the PyCA community: - Ying Li - Paul Kehrer - Jean-Paul Calderone - Laurens Van Houtven - Alex Gaynor - David Reid - Hynek Schlawack - Corbin Simpson - Aaron Gallagher And probably some other people that I missed. It?s still in a very rough state, and certainly needs much more serious thought (as well as some bikeshedding). I?ll describe some of the basic tenets predicating the design: - It will be be easy to use! - It will be opinionated about which math and TLS versions to use, and not allow downgrading to weaker security - It will have no IO (deal only with in-memory buffers) - It will be implemented with multiple backends, such as OpenSSL, SecureTransport, PyTLS, etc. - It will have no global state - It will not allow disabling of security features such as basic security checks, chain validation and hostname validation. - It will support both client and server operation. - We may expose less safe and more flexible lower-level APIs, but they will be clearly delineated from the API that people *should* be using. In addition, we may implement the protocol from scratch (of course using existing cryptography backends) in the future. Some of us feel strongly that protocol code should not be implemented in C (or any memory-unsafe language). I believe that the PyCA community has enough expertise to eventually produce a secure TLS implementation. Right now the design is a sphinx-style .rst file in a branch of cryptography in my github account (mostly so I can run ?make html? on it and get pretty output). Unfortunately this means I?m the only one who can commit to it or accept PRs, so if anyone has a suggestion to improve this situation I?d be happy to oblige. The direct link to the file is:?https://raw.githubusercontent.com/radix/cryptography/tls-api/docs/tls-api.rst The branch is:?https://github.com/radix/cryptography/tree/tls-api Any input would be greatly appreciated. Make sure to check the ?TODO? and ?Future work? sections of the document at the end in case you?re reading through it and immediately think of something to shout about :) PRs are appreciated. Thanks! --? Christopher Armstrong http://twitter.com/radix http://wordeology.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Sat Apr 19 23:00:24 2014 From: glyph at twistedmatrix.com (Glyph) Date: Sat, 19 Apr 2014 14:00:24 -0700 Subject: [Cryptography-dev] Towards a new TLS API In-Reply-To: References: Message-ID: <7023EA55-52A7-4755-8A23-0FB66DE90AEA@twistedmatrix.com> On Apr 17, 2014, at 7:37 AM, Christopher Armstrong wrote: > as well as some bikeshedding Some meta-bikeshedding, then, I suppose: "bikeshedding" is, by definition, futile. Please don't encourage it. I think you mean something more like "we need to seriously consider all possible preconceptions that our users might be approaching these libraries from, and allow for a longer-than-usual discussion of each name to ensure that it implies the correct type of object so people don't make security mistakes". In the nuclear-power-plant metaphor, this is not "bikeshedding"; the bike shed is still equally irrelevant. This is intentionally enduring a very long and tedious discussion about the fire suppression system which would be unnecessary in a more mundane building because when a nuclear power plant catches fire it is suuuuuper important that that stuff works, and there are unusual challenges in keeping it working (like for example some plutonium at a billion degrees burning its way to the center of the earth). In this case, of course, the fissile material is OpenSSL. -glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4124 bytes Desc: not available URL: From gvwilson at third-bit.com Mon Apr 21 20:31:29 2014 From: gvwilson at third-bit.com (Greg Wilson) Date: Mon, 21 Apr 2014 14:31:29 -0400 Subject: [Cryptography-dev] problem installing on Mac OS X 10.8.5 Message-ID: <53556401.7060102@third-bit.com> Hi, Trying to install cryptography so that I can install Scrapy; as per https://cryptography.io/en/latest/installation/, I'm using: $ brewinstall openssl Warning: openssl-1.0.1e already installed and then: $ env ARCHFLAGS="-arch x86_64" LDFLAGS="-L/usr/local/opt/openssl/lib" CFLAGS="-I/usr/local/opt/openssl/include" pip install cryptography Error message is below; note that I have installed libffi using "brew install libffi", and /usr/local/Cellar/libffi/3.0.13/lib/pkgconfig/libffi.pc exists. Pointers would be welcome... Thanks, Greg -------- Downloading/unpacking cryptography Running setup.py egg_info for package cryptography OS/X: confusion between 'cc' versus 'gcc' (see issue 123) will not use '__thread' in the C code Installed /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg building '_Cryptography_cffi_684bb40axf342507b' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so building '_Cryptography_cffi_8f86901cxc1767c5a' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so building '_Cryptography_cffi_48bbf0ebx93c91939' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Traceback (most recent call last): File "", line 16, in File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 156, in "test": PyTest, File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "", line 14, in replacement_run File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources mm.run() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run self.add_defaults() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults sdist.add_defaults(self) File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults build_py = self.get_finalized_command('build_py') File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options _build_py.finalize_options(self) File "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options ('force', 'force')) File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options src_cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 75, in finalize_options OpenSSLBinding().ffi.verifier.get_extension(), File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in __init__ self._ensure_ffi_initialized() File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in _ensure_ffi_initialized libraries) File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi ext_package="cryptography", File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 75, in load_library return self._load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 151, in _load_library return self._vengine.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/vengine_cpy.py", line 138, in load_library raise ffiplatform.VerificationError(error) cffi.ffiplatform.VerificationError: importing '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, 2): Library not loaded: libcrypto.1.0.0.dylib Referenced from: /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Reason: image not found Complete output from command python setup.py egg_info: OS/X: confusion between 'cc' versus 'gcc' (see issue 123) will not use '__thread' in the C code Installed /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg running egg_info creating pip-egg-info/cryptography.egg-info writing requirements to pip-egg-info/cryptography.egg-info/requires.txt writing pip-egg-info/cryptography.egg-info/PKG-INFO writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt' warning: manifest_maker: standard file '-c' not found running build_ext building '_Cryptography_cffi_684bb40axf342507b' extension creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__ /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so running build_ext building '_Cryptography_cffi_8f86901cxc1767c5a' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so running build_ext building '_Cryptography_cffi_48bbf0ebx93c91939' extension creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__ /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Traceback (most recent call last): File "", line 16, in File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 156, in "test": PyTest, File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "", line 14, in replacement_run File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources mm.run() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run self.add_defaults() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults sdist.add_defaults(self) File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults build_py = self.get_finalized_command('build_py') File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options _build_py.finalize_options(self) File "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options ('force', 'force')) File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options src_cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 75, in finalize_options OpenSSLBinding().ffi.verifier.get_extension(), File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in __init__ self._ensure_ffi_initialized() File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in _ensure_ffi_initialized libraries) File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi ext_package="cryptography", File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 75, in load_library return self._load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 151, in _load_library return self._vengine.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/vengine_cpy.py", line 138, in load_library raise ffiplatform.VerificationError(error) cffi.ffiplatform.VerificationError: importing '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, 2): Library not loaded: libcrypto.1.0.0.dylib Referenced from: /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Reason: image not found ---------------------------------------- Cleaning up... Command python setup.py egg_info failed with error code 1 in /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography Storing complete log in /Users/gwilson/.pip/pip.log From paul.l.kehrer at gmail.com Mon Apr 21 20:31:23 2014 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Mon, 21 Apr 2014 13:31:23 -0500 Subject: [Cryptography-dev] Release Cadences Message-ID: Now that we?re an upstream dependency for at least one major project (PyOpenSSL) it might make sense to try to fall into a monthly release cadence. This way we wouldn't block important features in our bindings from being consumed by downstream and we?ll also be able to get features out more regularly in hazmat/recipes. This would be helpful right now in allowing JP to schedule releases of PyOpenSSL that contain dependencies on new features we add. It would also help prevent situations like issue 941 (https://github.com/pyca/cryptography/issues/941) from arising in the future. If we did want to do monthly release cadence then next Monday would be essentially a month from the release of 0.3. We would need to significantly triage our fourth release issue list (and get some reviews!), but that will always be true for any release. Do people generally support this idea? -Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Mon Apr 21 20:37:53 2014 From: donald at stufft.io (Donald Stufft) Date: Mon, 21 Apr 2014 14:37:53 -0400 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: Message-ID: <575E0189-63C5-48BA-BD5D-AE4EB71CA447@stufft.io> On Apr 21, 2014, at 2:31 PM, Paul Kehrer wrote: > Now that we?re an upstream dependency for at least one major project (PyOpenSSL) it might make sense to try to fall into a monthly release cadence. This way we wouldn't block important features in our bindings from being consumed by downstream and we?ll also be able to get features out more regularly in hazmat/recipes. > > This would be helpful right now in allowing JP to schedule releases of PyOpenSSL that contain dependencies on new features we add. It would also help prevent situations like issue 941 (https://github.com/pyca/cryptography/issues/941) from arising in the future. > > If we did want to do monthly release cadence then next Monday would be essentially a month from the release of 0.3. We would need to significantly triage our fourth release issue list (and get some reviews!), but that will always be true for any release. > > Do people generally support this idea? > > -Paul > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev I think while we're still young, with new features being added at a high pace that a monthly release cycle makes sense. I think we'll want to make sure that this process is as automated as we can make it in order to reasonably achieve that. Probably in the future we'll want to lengthen our release cadence but that's a discussion for another day. I do think that if we essentially say every X we'll issue a new release then having release milestones don't really make a lot of sense though. If we're doing time based releases then what gets released is whatever has landed by the time the next release rolls around. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jarret.raim at RACKSPACE.COM Mon Apr 21 20:43:04 2014 From: jarret.raim at RACKSPACE.COM (Jarret Raim) Date: Mon, 21 Apr 2014 18:43:04 +0000 Subject: [Cryptography-dev] Release Cadences In-Reply-To: <575E0189-63C5-48BA-BD5D-AE4EB71CA447@stufft.io> References: <575E0189-63C5-48BA-BD5D-AE4EB71CA447@stufft.io> Message-ID: +1 with what Donald said for me. Jarret From: Donald Stufft Reply-To: "cryptography-dev at python.org" Date: Monday, April 21, 2014 at 1:37 PM To: "cryptography-dev at python.org" Subject: Re: [Cryptography-dev] Release Cadences > > On Apr 21, 2014, at 2:31 PM, Paul Kehrer wrote: > >> Now that we?re an upstream dependency for at least one major project >> (PyOpenSSL) it might make sense to try to fall into a monthly release >> cadence. This way we wouldn't block important features in our bindings from >> being consumed by downstream and we?ll also be able to get features out more >> regularly in hazmat/recipes. >> >> This would be helpful right now in allowing JP to schedule releases of >> PyOpenSSL that contain dependencies on new features we add. It would also >> help prevent situations like issue 941 >> (https://github.com/pyca/cryptography/issues/941) from arising in the future. >> >> If we did want to do monthly release cadence then next Monday would be >> essentially a month from the release of 0.3. We would need to significantly >> triage our fourth release issue list (and get some reviews!), but that will >> always be true for any release. >> >> Do people generally support this idea? >> >> -Paul >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > > > I think while we're still young, with new features being added at a high pace > that a monthly release cycle makes sense. I think we'll want to make sure that > this process is as automated as we can make it in order to reasonably achieve > that. Probably in the future we'll want to lengthen our release cadence but > that's a discussion for another day. > > I do think that if we essentially say every X we'll issue a new release then > having release milestones don't really make a lot of sense though. If we're > doing time based releases then what gets released is whatever has landed by > the time the next release rolls around. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5551 bytes Desc: not available URL: From donald at stufft.io Mon Apr 21 20:52:34 2014 From: donald at stufft.io (Donald Stufft) Date: Mon, 21 Apr 2014 14:52:34 -0400 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: <575E0189-63C5-48BA-BD5D-AE4EB71CA447@stufft.io> Message-ID: <770B4F40-1866-4819-9490-155BA6DE1D0F@stufft.io> On Apr 21, 2014, at 2:43 PM, Jarret Raim wrote: > +1 with what Donald said for me. > > Jarret > > From: Donald Stufft > Reply-To: "cryptography-dev at python.org" > Date: Monday, April 21, 2014 at 1:37 PM > To: "cryptography-dev at python.org" > Subject: Re: [Cryptography-dev] Release Cadences > >> >> On Apr 21, 2014, at 2:31 PM, Paul Kehrer wrote: >> >>> Now that we?re an upstream dependency for at least one major project (PyOpenSSL) it might make sense to try to fall into a monthly release cadence. This way we wouldn't block important features in our bindings from being consumed by downstream and we?ll also be able to get features out more regularly in hazmat/recipes. >>> >>> This would be helpful right now in allowing JP to schedule releases of PyOpenSSL that contain dependencies on new features we add. It would also help prevent situations like issue 941 (https://github.com/pyca/cryptography/issues/941) from arising in the future. >>> >>> If we did want to do monthly release cadence then next Monday would be essentially a month from the release of 0.3. We would need to significantly triage our fourth release issue list (and get some reviews!), but that will always be true for any release. >>> >>> Do people generally support this idea? >>> >>> -Paul >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> >> >> I think while we're still young, with new features being added at a high pace >> that a monthly release cycle makes sense. I think we'll want to make sure that >> this process is as automated as we can make it in order to reasonably achieve >> that. Probably in the future we'll want to lengthen our release cadence but >> that's a discussion for another day. >> >> I do think that if we essentially say every X we'll issue a new release then >> having release milestones don't really make a lot of sense though. If we're >> doing time based releases then what gets released is whatever has landed by >> the time the next release rolls around. >> >> ----------------- >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA >> > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev More thoughts! If we're going to move to a real time based release schedule then perhaps we want to look at changing our version scheme? Right now we're doing a fairly standard X.Y.Z (although at the moment it's mostly 0.X.Y). I think that probably we're not going to follow semver and thus there's not a lot of point to having a Major.Minor.Patch version scheme like that. Probably we should just move to YY.N where YY is the last two digits of the year (so this should allow us to version our project sanely until 2100 at which point it's someone else's problem!). The N would just be a monotonically increasing integer that represents the Nth release in that year. so we'd have 14.1, 14.2, 14.3, .., 14.N. The only reason I see for wanting to keep using the Major.Minor.Patch release semantics is if we want to leave it open to release a breaking API change and be able to communicate that to our users. However I think that we probably don't want to do that? Even if we are we probably don't want to still be using 0.x since people are using this in production now and 0.x is no longer relevant. One additional thought is that maybe we want to make it YY.N.Y where Y is almost always 0 except in the case that we want to release a security patch for an older version. This isn't strictly requires as the tooling will make YY.N == YY.N.0 however I think if we're likely to ever release a patch like that it's cleaner to always have the same number of digits in our version. One more additional thought stemming from the above, what is our support policy, especially with a high cadence release like monthly? Do we require people to update to the latest to get bug fixes? Security Fixes? Do we backport bug fixes or security fixes? If we do backport how far back to do we backport? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alexs at prol.etari.at Mon Apr 21 22:49:13 2014 From: alexs at prol.etari.at (Alex Stapleton) Date: Mon, 21 Apr 2014 21:49:13 +0100 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: Message-ID: <145860c9618.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> What's actually blocking us pushing a 0.4 master or 0.3.1 hotfix today? I think a more structured release schedule could be a good thing but we need to figure out who we're doing it for and what our aims are clearly. On 21 April 2014 19:31:37 Paul Kehrer wrote: > Now that we?re an upstream dependency for at least one major project > (PyOpenSSL) it might make sense to try to fall into a monthly release > cadence. This way we wouldn't block important features in our bindings from > being consumed by downstream and we?ll also be able to get features out > more regularly in hazmat/recipes. > > This would be helpful right now in allowing JP to schedule releases of > PyOpenSSL that contain dependencies on new features we add. It would also > help prevent situations like issue 941 > (https://github.com/pyca/cryptography/issues/941) from arising in the future. > > If we did want to do monthly release cadence then next Monday would be > essentially a month from the release of 0.3. We would need to significantly > triage our fourth release issue list (and get some reviews!), but that will > always be true for any release. > > Do people generally support this idea? > > -Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Mon Apr 21 23:24:57 2014 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Mon, 21 Apr 2014 16:24:57 -0500 Subject: [Cryptography-dev] Release Cadences In-Reply-To: <145860c9618.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> References: <145860c9618.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> Message-ID: The fourth release needs an official release manager (although I?ll go ahead and volunteer for now), so blockers aren?t really known yet. I think my first email somewhat covers who is it for and why, but another way to say it: Monthly releases would provide some certainty about when merged code will be available to downstream projects and allow projects like PyOpenSSL to move more quickly without requiring careful coordination with us. While we discuss this we could still plan to push the 0.4 release out next week though. On April 21, 2014 at 4:11:15 PM, Alex Stapleton (alexs at prol.etari.at) wrote: What's actually blocking us pushing a 0.4 master or 0.3.1 hotfix today? I think a more structured release schedule could be a good thing but we need to figure out who we're doing it for and what our aims are clearly. On 21 April 2014 19:31:37 Paul Kehrer wrote: Now that we?re an upstream dependency for at least one major project (PyOpenSSL) it might make sense to try to fall into a monthly release cadence. This way we wouldn't block important features in our bindings from being consumed by downstream and we?ll also be able to get features out more regularly in hazmat/recipes. This would be helpful right now in allowing JP to schedule releases of PyOpenSSL that contain dependencies on new features we add. It would also help prevent situations like issue 941 (https://github.com/pyca/cryptography/issues/941) from arising in the future. If we did want to do monthly release cadence then next Monday would be essentially a month from the release of 0.3. We would need to significantly triage our fourth release issue list (and get some reviews!), but that will always be true for any release. Do people generally support this idea? -Paul _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexs at prol.etari.at Tue Apr 22 00:27:10 2014 From: alexs at prol.etari.at (Alex Stapleton) Date: Mon, 21 Apr 2014 23:27:10 +0100 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: <145860c9618.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> Message-ID: <145866652c0.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> I think releasing more often would be good, leaving code on GitHub for ages sucks. If that's called a release cadence then OK :-) I am a bit concerned that without a period of deliberate tidying prior to each release this may result issues though. Currently we've been working on major feature sets which tends to naturally result in periods of tidying as those features near completion. If we are releasing with only some of a features PRs merged then that tidying won't coincide with the release as well. Some of that we can work around and adapt to though. On 21 April 2014 22:25:12 Paul Kehrer wrote: > The fourth release needs an official release manager (although I?ll go > ahead and volunteer for now), so blockers aren?t really known yet. I think > my first email somewhat covers who is it for and why, but another way to > say it: Monthly releases would provide some certainty about when merged > code will be available to downstream projects and allow projects like > PyOpenSSL to move more quickly without requiring careful coordination with us. > > While we discuss this we could still plan to push the 0.4 release out next > week though. > > On April 21, 2014 at 4:11:15 PM, Alex Stapleton (alexs at prol.etari.at) wrote: > > What's actually blocking us pushing a 0.4 master or 0.3.1 hotfix today? > > I think a more structured release schedule could be a good thing but we > need to figure out who we're doing it for and what our aims are clearly. > > On 21 April 2014 19:31:37 Paul Kehrer wrote: > > Now that we?re an upstream dependency for at least one major project > (PyOpenSSL) it might make sense to try to fall into a monthly release > cadence. This way we wouldn't block important features in our bindings from > being consumed by downstream and we?ll also be able to get features out > more regularly in hazmat/recipes. > > This would be helpful right now in allowing JP to schedule releases of > PyOpenSSL that contain dependencies on new features we add. It would also > help prevent situations like issue 941 > (https://github.com/pyca/cryptography/issues/941) from arising in the future. > > If we did want to do monthly release cadence then next Monday would be > essentially a month from the release of 0.3. We would need to significantly > triage our fourth release issue list (and get some reviews!), but that will > always be true for any release. > > Do people generally support this idea? > > -Paul > _______________________________________________ Cryptography-dev mailing > list Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Tue Apr 22 00:35:53 2014 From: donald at stufft.io (Donald Stufft) Date: Mon, 21 Apr 2014 18:35:53 -0400 Subject: [Cryptography-dev] Release Cadences In-Reply-To: <145866652c0.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> References: <145860c9618.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> <145866652c0.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> Message-ID: Time based releases more or less require a policy of ?master is always already to release?. On Apr 21, 2014, at 6:27 PM, Alex Stapleton wrote: > I think releasing more often would be good, leaving code on GitHub for ages sucks. If that's called a release cadence then OK :-) I am a bit concerned that without a period of deliberate tidying prior to each release this may result issues though. Currently we've been working on major feature sets which tends to naturally result in periods of tidying as those features near completion. If we are releasing with only some of a features PRs merged then that tidying won't coincide with the release as well. Some of that we can work around and adapt to though. > > On 21 April 2014 22:25:12 Paul Kehrer wrote: > >> The fourth release needs an official release manager (although I?ll go ahead and volunteer for now), so blockers aren?t really known yet. I think my first email somewhat covers who is it for and why, but another way to say it: Monthly releases would provide some certainty about when merged code will be available to downstream projects and allow projects like PyOpenSSL to move more quickly without requiring careful coordination with us. >> >> While we discuss this we could still plan to push the 0.4 release out next week though. >> >> On April 21, 2014 at 4:11:15 PM, Alex Stapleton (alexs at prol.etari.at) wrote: >> >>> What's actually blocking us pushing a 0.4 master or 0.3.1 hotfix today? >>> >>> I think a more structured release schedule could be a good thing but we need to figure out who we're doing it for and what our aims are clearly. >>> >>> On 21 April 2014 19:31:37 Paul Kehrer wrote: >>> >>>> Now that we?re an upstream dependency for at least one major project (PyOpenSSL) it might make sense to try to fall into a monthly release cadence. This way we wouldn't block important features in our bindings from being consumed by downstream and we?ll also be able to get features out more regularly in hazmat/recipes. >>>> >>>> This would be helpful right now in allowing JP to schedule releases of PyOpenSSL that contain dependencies on new features we add. It would also help prevent situations like issue 941 (https://github.com/pyca/cryptography/issues/941) from arising in the future. >>>> >>>> If we did want to do monthly release cadence then next Monday would be essentially a month from the release of 0.3. We would need to significantly triage our fourth release issue list (and get some reviews!), but that will always be true for any release. >>>> >>>> Do people generally support this idea? >>>> >>>> -Paul >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From steve at steveklabnik.com Tue Apr 22 00:39:44 2014 From: steve at steveklabnik.com (Steve Klabnik) Date: Mon, 21 Apr 2014 15:39:44 -0700 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: Message-ID: I haven't contributed anything, so weigh my words appropriately, but it seems like many big projects are doing a six week release cycle. Ember.js being the latest. They've seen a lot of really good benefits from doing so. From goon12 at gmail.com Tue Apr 22 00:52:18 2014 From: goon12 at gmail.com (Joe Riopel) Date: Mon, 21 Apr 2014 18:52:18 -0400 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: Message-ID: I'm not sure "setting" a cadence is the right idea, let the project grow its own. On Apr 21, 2014 6:46 PM, "Steve Klabnik" wrote: > I haven't contributed anything, so weigh my words appropriately, but > it seems like many big projects are doing a six week release cycle. > Ember.js being the latest. They've seen a lot of really good benefits > from doing so. > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dreid at dreid.org Tue Apr 22 01:15:20 2014 From: dreid at dreid.org (David Reid) Date: Mon, 21 Apr 2014 16:15:20 -0700 Subject: [Cryptography-dev] Thoughts on opaque key material. Message-ID: I apologize for how disorganized these thoughts are, the problems are more fully formed than the solutions and I ended up spending much more time than desired writing this email, so I just sort of cut myself off at 4pm. Hopefully we can figure out some of these answers on the mailing list. -- Currently RSA key material is represented as an object having a series of properties that correspond to the various mathematical components of an RSA key. The initial reasoning for this is that it would make the internal representation of a key backend independent and allow for moving key material from one backend to another (for example, loading the key from one backend and performing encryption with another backend). However, as an initial proponent of this argument I feel comfortable saying that I was completely wrong. It has variety of negative effects, 1) The mathematical components have terrible names that we MUST expose to users. In RSA n, p, q, e, and d. In ECDSA x and y. Even when these components have well known multisyllable names their value as descriptors is somewhat limited, "public exponent" is a much better name than "e" but still requires knowledge of the mathematical foundations of RSA to be of any value to anyone. (You can argue that the hazmat/primitives layers are the place for exposing this kind of detail to users and that if you don't understand the RSA mathematically you shouldn't be using them, however we've never felt any need to point out that AES S-boxes can be described by a system of 23 quadratic equations in 80 terms.) 2) Not all backend representations of keys expose the individual components. Even given a single backend such as openssl there is no guarantee that two internal structures will contain all the necessary data. This is most notable with keys stored in a HSMs or SmartCards where it is a feature that you can not extract the RSA private key from the system. However this is also evident in backends like CommonCrypto which don't have a public API for getting or setting the values of the various key components. To actually use an RSA key from OpenSSL on CommonCrypto you MUST convert it to some intermediate representation that both understand (likely PKCS#8). 3) Optional optimizations get exposed to users as required interface. Similarly, our exposure of the Chinese remainder theorem intermediate calculations on the RSAPrivateKey interface complicates the usage of RSA key material on multiple backends instead of simplifies it. Not all serialization formats or backends support storing/loading the CRT values potentially requiring us to perform duplicate calculations of these values for exposing them on the nonopaque interface. 4) Using the generic key components makes frequent operations (like obtaining a new signing/verify context) more expensive. These backend specific operations now require potentially costly (or even dangerous) conversions between the generic and backend specific representations. Given these issues I think we must seriously consider switching away from these backend independent representations as our primary interface to key materials. Preferring instead opaque backend specific implementations. For example, instead of a concrete RSAPrivateKey instance which can be instantiated with the underlying components, I imagine all key generation and loading being mediated by backends and resulting in objects that provide much more narrow interfaces organized around what you can do with a key (instead of what a key intrinsically is). For example, when generating an RSA key on the openssl backend you would get back an object which provides a basic interface indicating that it is an RSA key, that has some basic attributes for key size, and accessing the public key object, then methods for obtaining backend specific contexts for performing signing/verification and encryption/decryption operations. In addition to those basic operations the object may provide any number of backend specific serialization methods, potentially including "serializing" the key to something similar to the current interface (which we can think of as a bag of bignums). In this way we can facilitate converting between backend specific key representations through a process of serialization/deserialization (in many cases through a formal specification such as PKCS#8). -David -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Tue Apr 22 02:04:12 2014 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Mon, 21 Apr 2014 19:04:12 -0500 Subject: [Cryptography-dev] problem installing on Mac OS X 10.8.5 In-Reply-To: <53556401.7060102@third-bit.com> References: <53556401.7060102@third-bit.com> Message-ID: Does it work if you bind against system OpenSSL (don?t pass any flags)? This looks like the same problem described here:?https://github.com/pyca/cryptography/issues/693 On April 21, 2014 at 1:29:13 PM, Greg Wilson (gvwilson at third-bit.com) wrote: Hi, Trying to install cryptography so that I can install Scrapy; as per https://cryptography.io/en/latest/installation/, I'm using: $ brewinstall openssl Warning: openssl-1.0.1e already installed and then: $ env ARCHFLAGS="-arch x86_64" LDFLAGS="-L/usr/local/opt/openssl/lib" CFLAGS="-I/usr/local/opt/openssl/include" pip install cryptography Error message is below; note that I have installed libffi using "brew install libffi", and /usr/local/Cellar/libffi/3.0.13/lib/pkgconfig/libffi.pc exists. Pointers would be welcome... Thanks, Greg -------- Downloading/unpacking cryptography Running setup.py egg_info for package cryptography OS/X: confusion between 'cc' versus 'gcc' (see issue 123) will not use '__thread' in the C code Installed /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg building '_Cryptography_cffi_684bb40axf342507b' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so building '_Cryptography_cffi_8f86901cxc1767c5a' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so building '_Cryptography_cffi_48bbf0ebx93c91939' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Traceback (most recent call last): File "", line 16, in File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 156, in "test": PyTest, File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "", line 14, in replacement_run File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources mm.run() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run self.add_defaults() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults sdist.add_defaults(self) File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults build_py = self.get_finalized_command('build_py') File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options _build_py.finalize_options(self) File "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options ('force', 'force')) File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options src_cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 75, in finalize_options OpenSSLBinding().ffi.verifier.get_extension(), File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in __init__ self._ensure_ffi_initialized() File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in _ensure_ffi_initialized libraries) File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi ext_package="cryptography", File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 75, in load_library return self._load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 151, in _load_library return self._vengine.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/vengine_cpy.py", line 138, in load_library raise ffiplatform.VerificationError(error) cffi.ffiplatform.VerificationError: importing '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, 2): Library not loaded: libcrypto.1.0.0.dylib Referenced from: /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Reason: image not found Complete output from command python setup.py egg_info: OS/X: confusion between 'cc' versus 'gcc' (see issue 123) will not use '__thread' in the C code Installed /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg running egg_info creating pip-egg-info/cryptography.egg-info writing requirements to pip-egg-info/cryptography.egg-info/requires.txt writing pip-egg-info/cryptography.egg-info/PKG-INFO writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt' warning: manifest_maker: standard file '-c' not found running build_ext building '_Cryptography_cffi_684bb40axf342507b' extension creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__ /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so running build_ext building '_Cryptography_cffi_8f86901cxc1767c5a' extension /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so running build_ext building '_Cryptography_cffi_48bbf0ebx93c91939' extension creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__ /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -arch x86_64 -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Traceback (most recent call last): File "", line 16, in File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 156, in "test": PyTest, File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "", line 14, in replacement_run File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources mm.run() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run self.add_defaults() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults sdist.add_defaults(self) File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults build_py = self.get_finalized_command('build_py') File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options _build_py.finalize_options(self) File "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options ('force', 'force')) File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options src_cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 75, in finalize_options OpenSSLBinding().ffi.verifier.get_extension(), File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in __init__ self._ensure_ffi_initialized() File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in _ensure_ffi_initialized libraries) File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi ext_package="cryptography", File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 75, in load_library return self._load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", line 151, in _load_library return self._vengine.load_library() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/vengine_cpy.py", line 138, in load_library raise ffiplatform.VerificationError(error) cffi.ffiplatform.VerificationError: importing '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, 2): Library not loaded: libcrypto.1.0.0.dylib Referenced from: /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Reason: image not found ---------------------------------------- Cleaning up... Command python setup.py egg_info failed with error code 1 in /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography Storing complete log in /Users/gwilson/.pip/pip.log _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From terrycwk1994 at gmail.com Tue Apr 22 04:31:07 2014 From: terrycwk1994 at gmail.com (Terry Chia) Date: Tue, 22 Apr 2014 10:31:07 +0800 Subject: [Cryptography-dev] Release Cadences In-Reply-To: References: Message-ID: I have no particular preferences either way but I agree with Donald that we have to figure out our backports policy. Another minor concern is that we have to be more vigilant to ensure that we don't release any partially done features which *might* happen given our preferences to split up new features into smaller, more reviewable PRs. On Tue, Apr 22, 2014 at 6:52 AM, Joe Riopel wrote: > I'm not sure "setting" a cadence is the right idea, let the project grow its > own. > > On Apr 21, 2014 6:46 PM, "Steve Klabnik" wrote: >> >> I haven't contributed anything, so weigh my words appropriately, but >> it seems like many big projects are doing a six week release cycle. >> Ember.js being the latest. They've seen a lot of really good benefits >> from doing so. >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > From alexs at prol.etari.at Tue Apr 22 08:49:53 2014 From: alexs at prol.etari.at (Alex Stapleton) Date: Tue, 22 Apr 2014 07:49:53 +0100 Subject: [Cryptography-dev] Thoughts on opaque key material. In-Reply-To: References: Message-ID: <145883296e0.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> Individually I don't think these issues are much of a problem but their combination does sound like it needs addressing. 1) This mostly seems like a documentation issue. If we expose these values anywhere in our API we must document them, even if we hide them inside some backend specific API. Perhaps we can give these values lower priority in the docs and guide people towards the operations somehow? 2) There are 2 problems here. First are HSMs which provide only opaque handles to work on keys. Yet they also often provide APIs for loading existing keys into them. We still need to expose the individual key parameters to write keys to them. If we only support generating keys inside the HSM then we can avoid this to some extent, I suspect we will always need access to *public* key material though. The 2nd problem is APIs like CC which only provide DER encoded keys to the user. This makes implementation annoying for sure. Fortunately we currently have one mandatory backend that is fully capable of translating those DER keys, OpenSSL. There is not yet any good reason not to use it for this purpose. If we do make OpenSSL optional then working with raw DER for key structures isn't impossible and we seem to have some more in depth X.509 plans in progress anyway. 3) The problem is not that CRT is optional. The problem is that not everyone uses PKCS#1. OpenPGP uses a different formulation of CRT, OpenSSH regenerates CRT values on demand in places etc. Now certainly some backends might not care about the CRT values we store. They are rare but maybe we should still accommodate them? It seems like we should actually identify them first. Currently all backends on our to do list have a strong preference for the use of CRT. 4) Having investigated this issue in some depth we certainly have a small timing side channel on conversion in the OpenSSL backend. I'm not confident this is actually a serious danger though. IIRC the side channel leaks a small amount of information about the sum of the number of 1 bits in each part of the key, on Python 2.7. The main slowness is in the constant reallocation of the internal key structures. Abandoning our current API seems like a rather severe optimisation to me. Until the opaque key idea came up our leading solution to this problem was caching the backend internal representation on the key objects. I don't think there's anything blocking us from implementing that? So it seems like there's still a lot of different problems mixed up here. 1) Our documentation has too much emphasis on key material. We can fix this by writing better docs. 2) Implicit key conversion may be a security issue and is "slow". We can fix this within the current API. 3) Insistence on PKCS#1 for RSA. We have taken some steps to make this easier for users. I expect we will find application specific corner cases regardless of representation here. IMO this is best solved at the protocol library and documentation level. 4) Opaque backends. We can use OpenSSL to translate the key for them or just get down to adding more in depth ASN1 support in general. 5) We don't have a good plan for HSMs. We should definitely address this directly. I think they will inherently require their own interfaces though. I would support splitting our current interfaces into *Material and *Operations in anticipation of an HSM backend. We could do this easily today without huge API impacts but multi backend should continue to require a *Material key. I do have concerns that multi backend will turn into a complexity tar pit but so far it hasn't been too horrible. On 22 April 2014 00:15:34 David Reid wrote: > I apologize for how disorganized these thoughts are, the problems are more > fully formed than > the solutions and I ended up spending much more time than desired writing > this email, so I just sort of cut myself off at 4pm. Hopefully we can > figure out some of these answers on the mailing list. > > -- > > Currently RSA key material is represented as an object having a series of > properties that correspond to the various mathematical components of an > RSA key. > > The initial reasoning for this is that it would make the internal > representation of a key backend independent and allow for moving key > material from one backend to another (for example, loading the key from one > backend and performing encryption with another backend). > > However, as an initial proponent of this argument I feel comfortable saying > that I was completely wrong. It has variety of negative effects, > > 1) The mathematical components have terrible names that we MUST expose to > users. In RSA n, p, q, e, and d. In ECDSA x and y. > > Even when these components have well known multisyllable names their > value > as descriptors is somewhat limited, "public exponent" is a much better > name > than "e" but still requires knowledge of the mathematical foundations of > RSA to be of any value to anyone. (You can argue that the > hazmat/primitives layers are the place for exposing this kind of detail > to > users and that if you don't understand the RSA mathematically you > shouldn't > be using them, however we've never felt any need to point out that AES > S-boxes can be described by a system of 23 quadratic equations in 80 > terms.) > > 2) Not all backend representations of keys expose the individual components. > > Even given a single backend such as openssl there is no guarantee that > two internal structures will contain all the necessary data. This is > most notable with keys stored in a HSMs or SmartCards where it is a > feature > that you can not extract the RSA private key from the system. > > However this is also evident in backends like CommonCrypto which don't > have > a public API for getting or setting the values of the various key > components. To actually use an RSA key from OpenSSL on CommonCrypto you > MUST convert it to some intermediate representation that both understand > (likely PKCS#8). > > 3) Optional optimizations get exposed to users as required interface. > > Similarly, our exposure of the Chinese remainder theorem intermediate > calculations on the RSAPrivateKey interface complicates the usage of RSA > key material on multiple backends instead of simplifies it. Not all > serialization formats or backends support storing/loading the CRT values > potentially requiring us to perform duplicate calculations of these > values > for exposing them on the nonopaque interface. > > 4) Using the generic key components makes frequent operations (like > obtaining > a new signing/verify context) more expensive. > > These backend specific operations now require potentially costly (or even > dangerous) conversions between the generic and backend > specific representations. > > > Given these issues I think we must seriously consider switching away from > these backend independent representations as our primary interface to key > materials. Preferring instead opaque backend specific implementations. > > For example, instead of a concrete RSAPrivateKey instance which can be > instantiated with the underlying components, I imagine all key generation > and > loading being mediated by backends and resulting in objects that provide > much > more narrow interfaces organized around what you can do with a key (instead > of > what a key intrinsically is). For example, when generating an RSA key on > the > openssl backend you would get back an object which provides a basic > interface > indicating that it is an RSA key, that has some basic attributes for key > size, and accessing the public key object, then methods for obtaining > backend > specific contexts for performing signing/verification and > encryption/decryption operations. > > In addition to those basic operations the object may provide any number of > backend specific serialization methods, potentially including "serializing" > the > key to something similar to the current interface (which we can think of as > a > bag of bignums). > > In this way we can facilitate converting between backend specific key > representations through a process of serialization/deserialization (in many > cases through a formal specification such as PKCS#8). > > -David -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvwilson at third-bit.com Tue Apr 22 15:42:04 2014 From: gvwilson at third-bit.com (Greg Wilson) Date: Tue, 22 Apr 2014 09:42:04 -0400 Subject: [Cryptography-dev] problem installing on Mac OS X 10.8.5 In-Reply-To: References: <53556401.7060102@third-bit.com> Message-ID: <535671AC.3000102@third-bit.com> Hi Paul, Nope - followed that recipe, still no joy. I'll put up a blog post soon with details of what I've tried. Thx, G On 2014-04-21 8:04 PM, Paul Kehrer wrote: > Does it work if you bind against system OpenSSL (don't pass any > flags)? This looks like the same problem described here: > https://github.com/pyca/cryptography/issues/693 > > On April 21, 2014 at 1:29:13 PM, Greg Wilson (gvwilson at third-bit.com > ) wrote: > >> Hi, >> >> Trying to install cryptography so that I can install Scrapy; as per >> https://cryptography.io/en/latest/installation/, I'm using: >> >> $ brewinstall openssl >> Warning: openssl-1.0.1e already installed >> >> and then: >> >> $ env ARCHFLAGS="-arch x86_64" LDFLAGS="-L/usr/local/opt/openssl/lib" >> CFLAGS="-I/usr/local/opt/openssl/include" pip install cryptography >> >> Error message is below; note that I have installed libffi using "brew >> install libffi", and >> /usr/local/Cellar/libffi/3.0.13/lib/pkgconfig/libffi.pc exists. Pointers >> would be welcome... >> Thanks, >> Greg >> >> -------- >> >> Downloading/unpacking cryptography >> Running setup.py egg_info for package cryptography >> OS/X: confusion between 'cc' versus 'gcc' (see issue 123) >> will not use '__thread' in the C code >> >> Installed >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg >> >> >> building '_Cryptography_cffi_684bb40axf342507b' extension >> /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes >> -I/usr/local/opt/openssl/include -arch x86_64 >> -I/Users/gwilson/anaconda/include/python2.7 -c >> cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c >> >> -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o >> >> /usr/bin/clang -bundle -undefined dynamic_lookup >> -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib >> -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o >> >> -L/Users/gwilson/anaconda/lib -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so >> >> building '_Cryptography_cffi_8f86901cxc1767c5a' extension >> /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes >> -I/usr/local/opt/openssl/include -arch x86_64 >> -I/Users/gwilson/anaconda/include/python2.7 -c >> cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c >> >> -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o >> >> /usr/bin/clang -bundle -undefined dynamic_lookup >> -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib >> -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o >> >> -L/Users/gwilson/anaconda/lib -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so >> >> building '_Cryptography_cffi_48bbf0ebx93c91939' extension >> /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes >> -I/usr/local/opt/openssl/include -arch x86_64 >> -I/Users/gwilson/anaconda/include/python2.7 -c >> cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c >> >> -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o >> >> /usr/bin/clang -bundle -undefined dynamic_lookup >> -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib >> -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o >> >> -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so >> >> Traceback (most recent call last): >> File "", line 16, in >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", >> >> line 156, in >> "test": PyTest, >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", >> line 152, in setup >> dist.run_commands() >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", >> line 953, in run_commands >> self.run_command(cmd) >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", >> line 972, in run_command >> cmd_obj.run() >> File "", line 14, in replacement_run >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", >> >> line 259, in find_sources >> mm.run() >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", >> >> line 325, in run >> self.add_defaults() >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", >> >> line 361, in add_defaults >> sdist.add_defaults(self) >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", >> >> line 199, in add_defaults >> build_py = self.get_finalized_command('build_py') >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", >> line 312, in get_finalized_command >> cmd_obj.ensure_finalized() >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", >> line 109, in ensure_finalized >> self.finalize_options() >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", >> >> line 73, in finalize_options >> _build_py.finalize_options(self) >> File >> "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", >> line 46, in finalize_options >> ('force', 'force')) >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", >> line 298, in set_undefined_options >> src_cmd_obj.ensure_finalized() >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", >> line 109, in ensure_finalized >> self.finalize_options() >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", >> >> line 75, in finalize_options >> OpenSSLBinding().ffi.verifier.get_extension(), >> File "cryptography/hazmat/bindings/openssl/binding.py", line 83, >> in __init__ >> self._ensure_ffi_initialized() >> File "cryptography/hazmat/bindings/openssl/binding.py", line 99, >> in _ensure_ffi_initialized >> libraries) >> File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi >> ext_package="cryptography", >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/api.py", >> >> line 341, in verify >> lib = self.verifier.load_library() >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", >> >> line 75, in load_library >> return self._load_library() >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", >> >> line 151, in _load_library >> return self._vengine.load_library() >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/vengine_cpy.py", >> >> line 138, in load_library >> raise ffiplatform.VerificationError(error) >> cffi.ffiplatform.VerificationError: importing >> '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': >> >> dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, >> >> 2): Library not loaded: libcrypto.1.0.0.dylib >> Referenced from: >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so >> >> Reason: image not found >> Complete output from command python setup.py egg_info: >> OS/X: confusion between 'cc' versus 'gcc' (see issue 123) >> >> will not use '__thread' in the C code >> >> >> >> Installed >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg >> >> >> running egg_info >> >> creating pip-egg-info/cryptography.egg-info >> >> writing requirements to pip-egg-info/cryptography.egg-info/requires.txt >> >> writing pip-egg-info/cryptography.egg-info/PKG-INFO >> >> writing top-level names to >> pip-egg-info/cryptography.egg-info/top_level.txt >> >> writing dependency_links to >> pip-egg-info/cryptography.egg-info/dependency_links.txt >> >> writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt' >> >> warning: manifest_maker: standard file '-c' not found >> >> >> >> running build_ext >> >> building '_Cryptography_cffi_684bb40axf342507b' extension >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography >> >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat >> >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives >> >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__ >> >> >> /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes >> -I/usr/local/opt/openssl/include -arch x86_64 >> -I/Users/gwilson/anaconda/include/python2.7 -c >> cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c >> >> -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o >> >> >> /usr/bin/clang -bundle -undefined dynamic_lookup >> -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib >> -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o >> >> -L/Users/gwilson/anaconda/lib -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so >> >> >> running build_ext >> >> building '_Cryptography_cffi_8f86901cxc1767c5a' extension >> >> /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes >> -I/usr/local/opt/openssl/include -arch x86_64 >> -I/Users/gwilson/anaconda/include/python2.7 -c >> cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c >> >> -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o >> >> >> /usr/bin/clang -bundle -undefined dynamic_lookup >> -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib >> -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o >> >> -L/Users/gwilson/anaconda/lib -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so >> >> >> running build_ext >> >> building '_Cryptography_cffi_48bbf0ebx93c91939' extension >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography >> >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat >> >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings >> >> >> creating >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__ >> >> >> /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes >> -I/usr/local/opt/openssl/include -arch x86_64 >> -I/Users/gwilson/anaconda/include/python2.7 -c >> cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c >> >> -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o >> >> >> /usr/bin/clang -bundle -undefined dynamic_lookup >> -L/Users/gwilson/anaconda/lib -L/usr/local/opt/openssl/lib >> -I/usr/local/opt/openssl/include -arch x86_64 -arch x86_64 >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o >> >> -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so >> >> >> Traceback (most recent call last): >> >> File "", line 16, in >> >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", >> >> line 156, in >> >> "test": PyTest, >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line >> 152, in setup >> >> dist.run_commands() >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line >> 953, in run_commands >> >> self.run_command(cmd) >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line >> 972, in run_command >> >> cmd_obj.run() >> >> File "", line 14, in replacement_run >> >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", >> >> line 259, in find_sources >> >> mm.run() >> >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", >> >> line 325, in run >> >> self.add_defaults() >> >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", >> >> line 361, in add_defaults >> >> sdist.add_defaults(self) >> >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", >> >> line 199, in add_defaults >> >> build_py = self.get_finalized_command('build_py') >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line >> 312, in get_finalized_command >> >> cmd_obj.ensure_finalized() >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line >> 109, in ensure_finalized >> >> self.finalize_options() >> >> File >> "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", >> >> line 73, in finalize_options >> >> _build_py.finalize_options(self) >> >> File >> "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", >> line 46, in finalize_options >> >> ('force', 'force')) >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line >> 298, in set_undefined_options >> >> src_cmd_obj.ensure_finalized() >> >> File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line >> 109, in ensure_finalized >> >> self.finalize_options() >> >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", >> >> line 75, in finalize_options >> >> OpenSSLBinding().ffi.verifier.get_extension(), >> >> File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in >> __init__ >> >> self._ensure_ffi_initialized() >> >> File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in >> _ensure_ffi_initialized >> >> libraries) >> >> File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi >> >> ext_package="cryptography", >> >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/api.py", >> >> line 341, in verify >> >> lib = self.verifier.load_library() >> >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", >> >> line 75, in load_library >> >> return self._load_library() >> >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/verifier.py", >> >> line 151, in _load_library >> >> return self._vengine.load_library() >> >> File >> "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cffi-0.8.2-py2.7-macosx-10.5-x86_64.egg/cffi/vengine_cpy.py", >> >> line 138, in load_library >> >> raise ffiplatform.VerificationError(error) >> >> cffi.ffiplatform.VerificationError: importing >> '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': >> >> dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, >> >> 2): Library not loaded: libcrypto.1.0.0.dylib >> >> Referenced from: >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so >> >> >> Reason: image not found >> >> ---------------------------------------- >> Cleaning up... >> Command python setup.py egg_info failed with error code 1 in >> /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography >> >> Storing complete log in /Users/gwilson/.pip/pip.log >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvwilson at third-bit.com Tue Apr 22 16:09:47 2014 From: gvwilson at third-bit.com (Greg Wilson) Date: Tue, 22 Apr 2014 10:09:47 -0400 Subject: [Cryptography-dev] problem installing on Mac OS X 10.8.5 In-Reply-To: <535671AC.3000102@third-bit.com> References: <53556401.7060102@third-bit.com> <535671AC.3000102@third-bit.com> Message-ID: <5356782B.2040003@third-bit.com> Here's a typescript of what I've tried and what's going wrong. Thanks Greg Script started on Tue Apr 22 09:56:21 2014 ~: which python /Users/gwilson/anaconda/bin/python ~: python --version Python 2.7.6 :: Anaconda 1.8.0 (x86_64) ~: which ipython /Users/gwilson/anaconda/bin/ipython ~: ipython --version 2.0.0 ~: which conda /Users/gwilson/anaconda/bin/conda ~: conda --version conda 3.4.2 ~: conda update openssl Fetching package metadata: .. # All requested packages already installed. # packages in environment at /Users/gwilson/anaconda: # openssl 1.0.1g 0 ~: brew install libffi Warning: Your Xcode (4.5.1) is outdated Please install Xcode 4.6.2. Warning: libffi-3.0.13 already installed ~: ls /usr/local/Cellar/libffi/3.0.13/lib/pkgconfig libffi.pc ~: conda update cryptography Error: package 'cryptography' is not installed in /Users/gwilson/anaconda ~: pip install cryptography Downloading/unpacking cryptography Downloading cryptography-0.3.tar.gz (208kB): 208kB downloaded Running setup.py egg_info for package cryptography building '_Cryptography_cffi_684bb40axf342507b' extension /usr/bin/clang -fno-strict-aliasing -I/Users/gwilson/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so building '_Cryptography_cffi_8f86901cxc1767c5a' extension /usr/bin/clang -fno-strict-aliasing -I/Users/gwilson/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so building '_Cryptography_cffi_48bbf0ebx93c91939' extension /usr/bin/clang -fno-strict-aliasing -I/Users/gwilson/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Traceback (most recent call last): File "", line 16, in File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 156, in "test": PyTest, File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "", line 14, in replacement_run File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources mm.run() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run self.add_defaults() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults sdist.add_defaults(self) File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults build_py = self.get_finalized_command('build_py') File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options _build_py.finalize_options(self) File "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options ('force', 'force')) File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options src_cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 75, in finalize_options OpenSSLBinding().ffi.verifier.get_extension(), File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in __init__ self._ensure_ffi_initialized() File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in _ensure_ffi_initialized libraries) File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi ext_package="cryptography", File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/verifier.py", line 75, in load_library return self._load_library() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/verifier.py", line 151, in _load_library return self._vengine.load_library() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/vengine_cpy.py", line 138, in load_library raise ffiplatform.VerificationError(error) cffi.ffiplatform.VerificationError: importing '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, 2): Library not loaded: libcrypto.1.0.0.dylib Referenced from: /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Reason: image not found Complete output from command python setup.py egg_info: running egg_info creating pip-egg-info/cryptography.egg-info writing requirements to pip-egg-info/cryptography.egg-info/requires.txt writing pip-egg-info/cryptography.egg-info/PKG-INFO writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt' warning: manifest_maker: standard file '-c' not found running build_ext building '_Cryptography_cffi_684bb40axf342507b' extension creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__ /usr/bin/clang -fno-strict-aliasing -I/Users/gwilson/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_684bb40axf342507b.so running build_ext building '_Cryptography_cffi_8f86901cxc1767c5a' extension /usr/bin/clang -fno-strict-aliasing -I/Users/gwilson/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.o -L/Users/gwilson/anaconda/lib -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/primitives/__pycache__/_Cryptography_cffi_8f86901cxc1767c5a.so running build_ext building '_Cryptography_cffi_48bbf0ebx93c91939' extension creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings creating /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__ /usr/bin/clang -fno-strict-aliasing -I/Users/gwilson/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/gwilson/anaconda/include/python2.7 -c cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.c -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o /usr/bin/clang -bundle -undefined dynamic_lookup -L/Users/gwilson/anaconda/lib -arch x86_64 -arch x86_64 /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.o -L/Users/gwilson/anaconda/lib -lcrypto -lssl -o /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Traceback (most recent call last): File "", line 16, in File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 156, in "test": PyTest, File "/Users/gwilson/anaconda/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/Users/gwilson/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "", line 14, in replacement_run File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources mm.run() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run self.add_defaults() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults sdist.add_defaults(self) File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults build_py = self.get_finalized_command('build_py') File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options _build_py.finalize_options(self) File "/Users/gwilson/anaconda/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options ('force', 'force')) File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options src_cmd_obj.ensure_finalized() File "/Users/gwilson/anaconda/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized self.finalize_options() File "/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/setup.py", line 75, in finalize_options OpenSSLBinding().ffi.verifier.get_extension(), File "cryptography/hazmat/bindings/openssl/binding.py", line 83, in __init__ self._ensure_ffi_initialized() File "cryptography/hazmat/bindings/openssl/binding.py", line 99, in _ensure_ffi_initialized libraries) File "cryptography/hazmat/bindings/utils.py", line 77, in build_ffi ext_package="cryptography", File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/verifier.py", line 75, in load_library return self._load_library() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/verifier.py", line 151, in _load_library return self._vengine.load_library() File "/Users/gwilson/anaconda/lib/python2.7/site-packages/cffi/vengine_cpy.py", line 138, in load_library raise ffiplatform.VerificationError(error) cffi.ffiplatform.VerificationError: importing '/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so': dlopen(/private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so, 2): Library not loaded: libcrypto.1.0.0.dylib Referenced from: /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography/cryptography/hazmat/bindings/__pycache__/_Cryptography_cffi_48bbf0ebx93c91939.so Reason: image not found ---------------------------------------- Cleaning up... Command python setup.py egg_info failed with error code 1 in /private/var/folders/vz/m_8fd1kj2kn9t2gq0d8fqgg00000gn/T/pip_build_gwilson/cryptography Storing complete log in /Users/gwilson/.pip/pip.log ~: exit Script done on Tue Apr 22 09:58:15 2014 From gvwilson at third-bit.com Tue Apr 22 16:15:03 2014 From: gvwilson at third-bit.com (Greg Wilson) Date: Tue, 22 Apr 2014 10:15:03 -0400 Subject: [Cryptography-dev] problem installing on Mac OS X 10.8.5 In-Reply-To: <5356782B.2040003@third-bit.com> References: <53556401.7060102@third-bit.com> <535671AC.3000102@third-bit.com> <5356782B.2040003@third-bit.com> Message-ID: <53567967.4010700@third-bit.com> Moving discussion to https://github.com/pyca/cryptography/issues/693. From alex.gaynor at gmail.com Wed Apr 23 18:58:15 2014 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 23 Apr 2014 09:58:15 -0700 Subject: [Cryptography-dev] Thoughts on opaque key material. In-Reply-To: <145883296e0.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> References: <145883296e0.2771.91731227630488f1d17e3e780e9eb50a@prol.etari.at> Message-ID: I'm not as good at writing very long emails as you two :-), but my thoughts are: Getting some integers out of a key should be treated as a serialization question, not as a fundamental property of the key, the fundamental nature of the key is to be able to sign/verify/encrypt/decrypt. Alex On Mon, Apr 21, 2014 at 11:49 PM, Alex Stapleton wrote: > Individually I don't think these issues are much of a problem but their > combination does sound like it needs addressing. > > 1) This mostly seems like a documentation issue. If we expose these values > anywhere in our API we must document them, even if we hide them inside some > backend specific API. Perhaps we can give these values lower priority in > the docs and guide people towards the operations somehow? > > 2) There are 2 problems here. First are HSMs which provide only opaque > handles to work on keys. Yet they also often provide APIs for loading > existing keys into them. We still need to expose the individual key > parameters to write keys to them. If we only support generating keys inside > the HSM then we can avoid this to some extent, I suspect we will always > need access to *public* key material though. > > The 2nd problem is APIs like CC which only provide DER encoded keys to the > user. This makes implementation annoying for sure. Fortunately we currently > have one mandatory backend that is fully capable of translating those DER > keys, OpenSSL. There is not yet any good reason not to use it for this > purpose. If we do make OpenSSL optional then working with raw DER for key > structures isn't impossible and we seem to have some more in depth X.509 > plans in progress anyway. > > 3) The problem is not that CRT is optional. The problem is that not > everyone uses PKCS#1. OpenPGP uses a different formulation of CRT, OpenSSH > regenerates CRT values on demand in places etc. Now certainly some backends > might not care about the CRT values we store. They are rare but maybe we > should still accommodate them? It seems like we should actually identify > them first. Currently all backends on our to do list have a strong > preference for the use of CRT. > > 4) Having investigated this issue in some depth we certainly have a small > timing side channel on conversion in the OpenSSL backend. I'm not confident > this is actually a serious danger though. IIRC the side channel leaks a > small amount of information about the sum of the number of 1 bits in each > part of the key, on Python 2.7. > > The main slowness is in the constant reallocation of the internal key > structures. Abandoning our current API seems like a rather severe > optimisation to me. Until the opaque key idea came up our leading solution > to this problem was caching the backend internal representation on the key > objects. I don't think there's anything blocking us from implementing that? > > So it seems like there's still a lot of different problems mixed up here. > > 1) Our documentation has too much emphasis on key material. We can fix > this by writing better docs. > > 2) Implicit key conversion may be a security issue and is "slow". We can > fix this within the current API. > > 3) Insistence on PKCS#1 for RSA. We have taken some steps to make this > easier for users. I expect we will find application specific corner cases > regardless of representation here. IMO this is best solved at the protocol > library and documentation level. > > 4) Opaque backends. We can use OpenSSL to translate the key for them or > just get down to adding more in depth ASN1 support in general. > > 5) We don't have a good plan for HSMs. We should definitely address this > directly. I think they will inherently require their own interfaces though. > I would support splitting our current interfaces into *Material and > *Operations in anticipation of an HSM backend. We could do this easily > today without huge API impacts but multi backend should continue to require > a *Material key. > > I do have concerns that multi backend will turn into a complexity tar pit > but so far it hasn't been too horrible. > > On 22 April 2014 00:15:34 David Reid wrote: > >> I apologize for how disorganized these thoughts are, the problems are >> more fully formed than >> the solutions and I ended up spending much more time than desired writing >> this email, so I just sort of cut myself off at 4pm. Hopefully we can >> figure out some of these answers on the mailing list. >> >> -- >> >> Currently RSA key material is represented as an object having a series of >> properties that correspond to the various mathematical components of an >> RSA key. >> >> The initial reasoning for this is that it would make the internal >> representation of a key backend independent and allow for moving key >> material from one backend to another (for example, loading the key from >> one >> backend and performing encryption with another backend). >> >> However, as an initial proponent of this argument I feel comfortable >> saying >> that I was completely wrong. It has variety of negative effects, >> >> 1) The mathematical components have terrible names that we MUST expose to >> users. In RSA n, p, q, e, and d. In ECDSA x and y. >> >> Even when these components have well known multisyllable names their >> value >> as descriptors is somewhat limited, "public exponent" is a much better >> name >> than "e" but still requires knowledge of the mathematical foundations >> of >> RSA to be of any value to anyone. (You can argue that the >> hazmat/primitives layers are the place for exposing this kind of >> detail to >> users and that if you don't understand the RSA mathematically you >> shouldn't >> be using them, however we've never felt any need to point out that AES >> S-boxes can be described by a system of 23 quadratic equations in 80 >> terms.) >> >> 2) Not all backend representations of keys expose the individual >> components. >> >> Even given a single backend such as openssl there is no guarantee that >> two internal structures will contain all the necessary data. This is >> most notable with keys stored in a HSMs or SmartCards where it is a >> feature >> that you can not extract the RSA private key from the system. >> >> However this is also evident in backends like CommonCrypto which don't >> have >> a public API for getting or setting the values of the various key >> components. To actually use an RSA key from OpenSSL on CommonCrypto >> you >> MUST convert it to some intermediate representation that both >> understand >> (likely PKCS#8). >> >> 3) Optional optimizations get exposed to users as required interface. >> >> Similarly, our exposure of the Chinese remainder theorem intermediate >> calculations on the RSAPrivateKey interface complicates the usage of >> RSA >> key material on multiple backends instead of simplifies it. Not all >> serialization formats or backends support storing/loading the CRT >> values >> potentially requiring us to perform duplicate calculations of these >> values >> for exposing them on the nonopaque interface. >> >> 4) Using the generic key components makes frequent operations (like >> obtaining >> a new signing/verify context) more expensive. >> >> These backend specific operations now require potentially costly (or >> even >> dangerous) conversions between the generic and backend >> specific representations. >> >> >> Given these issues I think we must seriously consider switching away from >> these backend independent representations as our primary interface to key >> materials. Preferring instead opaque backend specific implementations. >> >> For example, instead of a concrete RSAPrivateKey instance which can be >> instantiated with the underlying components, I imagine all key generation >> and >> loading being mediated by backends and resulting in objects that provide >> much >> more narrow interfaces organized around what you can do with a key >> (instead of >> what a key intrinsically is). For example, when generating an RSA key on >> the >> openssl backend you would get back an object which provides a basic >> interface >> indicating that it is an RSA key, that has some basic attributes for key >> size, and accessing the public key object, then methods for obtaining >> backend >> specific contexts for performing signing/verification and >> encryption/decryption operations. >> >> In addition to those basic operations the object may provide any number of >> backend specific serialization methods, potentially including >> "serializing" the >> key to something similar to the current interface (which we can think of >> as a >> bag of bignums). >> >> In this way we can facilitate converting between backend specific key >> representations through a process of serialization/deserialization (in >> many >> cases through a formal specification such as PKCS#8). >> >> -David >> > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Thu Apr 24 01:48:32 2014 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Wed, 23 Apr 2014 16:48:32 -0700 Subject: [Cryptography-dev] Thoughts on opaque key material. In-Reply-To: References: Message-ID: <1CC8196F-805A-490C-BB5D-3A3FD67D4549@twistedmatrix.com> On Apr 21, 2014, at 4:15 PM, David Reid wrote: > I apologize for how disorganized these thoughts are, the problems are more fully formed than > the solutions and I ended up spending much more time than desired writing this email, so I just sort of cut myself off at 4pm. Hopefully we can figure out some of these answers on the mailing list. > > -- > > Currently RSA key material is represented as an object having a series of > properties that correspond to the various mathematical components of an > RSA key. > > The initial reasoning for this is that it would make the internal > representation of a key backend independent and allow for moving key > material from one backend to another (for example, loading the key from one > backend and performing encryption with another backend). > > However, as an initial proponent of this argument I feel comfortable saying > that I was completely wrong. It has variety of negative effects, (agreed on all points) > Given these issues I think we must seriously consider switching away from > these backend independent representations as our primary interface to key > materials. Preferring instead opaque backend specific implementations. > > For example, instead of a concrete RSAPrivateKey instance which can be > instantiated with the underlying components, I imagine all key generation and > loading being mediated by backends and resulting in objects that provide much > more narrow interfaces organized around what you can do with a key (instead of > what a key intrinsically is). For example, when generating an RSA key on the > openssl backend you would get back an object which provides a basic interface > indicating that it is an RSA key, that has some basic attributes for key > size, and accessing the public key object, then methods for obtaining backend > specific contexts for performing signing/verification and > encryption/decryption operations. There's a conceptual shift here in what an RSAPrivateKey really is which you alluded to when you said (in the elided point 2 above): > This is most notable with keys stored in a HSMs or SmartCards where it is a feature that you can not extract the RSA private key from the system The really important point I think you're getting at here is that RSAPrivateKey is really a pointer to a key stored somewhere inside a backend. Maybe you can't access the contents of that pointer in a particular way because the backend doesn't expose public APIs to do so. Maybe you can't access it because it's physically isolated from the general purpose computer running Cryptography. From an API perspective, it doesn't really matter; what the RSAPrivateKey interface should expose is the ability to do some crypto. (And of course the "RSA" isn't here, this applies to any kind of private key.) Maybe it would also be valuable, at some point, to have a backend-neutral representation of the values associated with an RSA private key, but that's more of an advanced feature and, at least at first, should be left to the backends themselves; they already have representations of this stuff they need to use, obviously, and the first and most important task is to expose it in a way which clearly reflects its capabilities. > In addition to those basic operations the object may provide any number of > backend specific serialization methods, potentially including "serializing" the > key to something similar to the current interface (which we can think of as a > bag of bignums). > > In this way we can facilitate converting between backend specific key > representations through a process of serialization/deserialization (in many > cases through a formal specification such as PKCS#8). I would suggest that the backend conversion process go like this: Have a function that translates keys between different backends, that works for "any" key and "any" backend (scare quotes because they need to have some kind of common representation). Expose an interrogation interface on the private key to allow the translation function to ask what formats its backend supports serializing to. Expose an interrogation interface on the backend to allow the translation function to ask what formats new backend supports loading from. The translation function can then be nice and general, and support any new formats that come along, and also order things by preference of efficiently (like, for example, the TotallyUnsafeMaybeTheMathIsWrongBagOfBignums format that would just move the bignums between backends, potentially including things like chinese remainder theorem optimizations, without re-verifying that they're accurate, since presumably the first backend that loaded it already did that in some way). Anyway this all seems like a much better way to expose asymmetric crypto to me! -glyph -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4124 bytes Desc: not available URL: From paul.l.kehrer at gmail.com Thu Apr 24 23:38:34 2014 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Thu, 24 Apr 2014 16:38:34 -0500 Subject: [Cryptography-dev] Cryptography 0.4 Release Schedule Message-ID: We?re discussing longer term release strategies in the release cadences thread, but we?ve also got a large backlog of work that is sitting in master unreleased right now so it?s time to talk about a 0.4 release (which I?ve previously volunteered to manage). The release milestone (https://github.com/pyca/cryptography/issues?milestone=4&page=1&state=open) has an up-to-date view of the remaining issues blocking release, but here?s a bullet list for posterity. * RSA encryption support (PR currently under review) * DSA verification (WIP PR) * DSA signing * Key serialization (this will probably only be TraditionalOpenSSL aka PKCS1-ish loading/serializing) I think we can probably close these items out and get a release out next week. If you think something else belongs in the release let me know or add the issue/PR to it. General list of new stuff for 0.4 that has already landed: * RSA decryption (OAEP, PKCS1v15) * SEED support * CMAC support * DSA key generation, interfaces * Many new OpenSSL bindings (ECDSA, NPN, CMS, etc). Some of these are very useful for PyOpenSSL and other downstream projects that need things we don?t offer in hazmat at this time. -Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Mon Apr 28 18:32:09 2014 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Mon, 28 Apr 2014 09:32:09 -0700 Subject: [Cryptography-dev] Packaging ideas Message-ID: So, before I can start talking about solutions, I have to describe the use cases. 1) Some folks need a very specific backend (i.e. pyOpenSSL) and right now it's just assumed that that will come with `pip install cryptography`. 2) Some folks want to use cryptography as the API for non-traditional primitives (e.g. scrypt), and `pip install cryptography` brings a lot more than what they want. 3) Some folks want to type the letters AES, and don't really care where that comes from. 4) Some folks want to type the letters "fab", and super don't care that fabric needs a paramiko which needs an AES. 5) Some folks want to be able `pip install something` without getting an OpenSSL, because OpenSSL is literally the worst thing in the world. Invariants that must hold true: * `pip install cryptography` gets you a thing that can AES (and whatever other core algorithms we think are important) * There's a thing you can `pip install` that gets you scrypt, but not OpenSSL * There's a thing you can `pip install` to definitely get some OpenSSL. Solution: We break things down into a few different packages: * `pip install cryptography-core`: This contains all the primitives, all the recipes, all the everything besides the backends/bindings * `pip install cryptography-{openssl,scrypt,etc.}`: This contains the backend/bindings for the specific thing * `pip install cryptography`: This gets you a cryptography, and whatever set of core backends we think are reasonable for your platform for doing stuff like typing AES (and, god willing, then typing HMAC). Thoughts? Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Mon Apr 28 18:56:36 2014 From: donald at stufft.io (Donald Stufft) Date: Mon, 28 Apr 2014 12:56:36 -0400 Subject: [Cryptography-dev] Packaging ideas In-Reply-To: References: Message-ID: On Apr 28, 2014, at 12:32 PM, Alex Gaynor wrote: > So, before I can start talking about solutions, I have to describe the use cases. > > 1) Some folks need a very specific backend (i.e. pyOpenSSL) and right now it's just assumed that that will come with `pip install cryptography`. > 2) Some folks want to use cryptography as the API for non-traditional primitives (e.g. scrypt), and `pip install cryptography` brings a lot more than what they want. > 3) Some folks want to type the letters AES, and don't really care where that comes from. > 4) Some folks want to type the letters "fab", and super don't care that fabric needs a paramiko which needs an AES. > 5) Some folks want to be able `pip install something` without getting an OpenSSL, because OpenSSL is literally the worst thing in the world. > > Invariants that must hold true: > > * `pip install cryptography` gets you a thing that can AES (and whatever other core algorithms we think are important) > * There's a thing you can `pip install` that gets you scrypt, but not OpenSSL > * There's a thing you can `pip install` to definitely get some OpenSSL. > > Solution: > > We break things down into a few different packages: > > * `pip install cryptography-core`: This contains all the primitives, all the recipes, all the everything besides the backends/bindings > * `pip install cryptography-{openssl,scrypt,etc.}`: This contains the backend/bindings for the specific thing > * `pip install cryptography`: This gets you a cryptography, and whatever set of core backends we think are reasonable for your platform for doing stuff like typing AES (and, god willing, then typing HMAC). > > Thoughts? > Alex > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev So I (obviously given i?ve been saying it for awhile now ;) ) agree with splitting out the backends! First off, my question is how important is #2 on platforms which do not ship OpenSSL or have it easily installable which extends into how important is #5 at all? I ask because I think that cryptography-core (versus just keeping the ?core? stuff in cryptography) is a hack around a failing in python packaging and I would much rather solve that by fixing python packaging. What that would mean is that supporting #5 (and #2 on platforms where OpenSSL isn?t readably available) isn?t super nice at first until packaging ?catches? up. It is my belief that the people with the use case of #2 are not going to care if the bits to enable OpenSSL are residing on their machine unused unless those bits forced them to go and manually install OpenSSL, with their level of care increasing based on how painful it was for them to manually install OpenSSL. I also believe that the #5 case will be the people who actually care ?No there must be ZERO OpenSSL bits on my system? and I also believe that they will be a minority use case. Given those beliefs I feel like it is reasonable to leave the ?core? stuff inside of the cryptography package, and simple say that if you want to ``pip install cryptography`` without getting the OpenSSL bits then you?ll need to do ``pip install cryptography ?no-use-wheel ?install-option=??no-openssl?`` or ``CRYPTOGRAPHY_WITHOUT_OPENSSL=1 pip install ?no-use-wheel cryptography``. Once packaging catches up to this use case then we can clean that up to something like ``pip install cryptography[nss]`` (replace nss with any other backend here!). Beyond that there is one small sticky point to this: Imports. Currently we only ship one package (well two if you count cryptography_vectors) however this is going to cause us to ship more than that. However we already have a public API on ``cryptography.hazmat.bindings.*`` and ``cryptography.hazmat.backends.*``. I do not think we can break those which leaves us with two choices: A) Namespace packages - These are the ?obvious? first top to reach too, however I can often be found saying that nobody has ever used namespace packages without regretting it. In particular there are issues when combining easy_install and pip. B) Use some sort of shim to enable ``cryptography.hazmat.bindings.*`` and ``cryptography.hazmat.backends.*`` to act as ?aliases? for something else, either something based on setuptools entry points, or something based on naming (ex: https://github.com/mitsuhiko/flask/blob/master/flask/ext/__init__.py) ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From donald at stufft.io Mon Apr 28 21:34:08 2014 From: donald at stufft.io (Donald Stufft) Date: Mon, 28 Apr 2014 15:34:08 -0400 Subject: [Cryptography-dev] Packaging ideas In-Reply-To: References: Message-ID: <3C70FD1B-474B-41DA-AB5E-CD86ABBB1574@stufft.io> On Apr 28, 2014, at 12:56 PM, Donald Stufft wrote: > > On Apr 28, 2014, at 12:32 PM, Alex Gaynor wrote: > >> So, before I can start talking about solutions, I have to describe the use cases. >> >> 1) Some folks need a very specific backend (i.e. pyOpenSSL) and right now it's just assumed that that will come with `pip install cryptography`. >> 2) Some folks want to use cryptography as the API for non-traditional primitives (e.g. scrypt), and `pip install cryptography` brings a lot more than what they want. >> 3) Some folks want to type the letters AES, and don't really care where that comes from. >> 4) Some folks want to type the letters "fab", and super don't care that fabric needs a paramiko which needs an AES. >> 5) Some folks want to be able `pip install something` without getting an OpenSSL, because OpenSSL is literally the worst thing in the world. >> >> Invariants that must hold true: >> >> * `pip install cryptography` gets you a thing that can AES (and whatever other core algorithms we think are important) >> * There's a thing you can `pip install` that gets you scrypt, but not OpenSSL >> * There's a thing you can `pip install` to definitely get some OpenSSL. >> >> Solution: >> >> We break things down into a few different packages: >> >> * `pip install cryptography-core`: This contains all the primitives, all the recipes, all the everything besides the backends/bindings >> * `pip install cryptography-{openssl,scrypt,etc.}`: This contains the backend/bindings for the specific thing >> * `pip install cryptography`: This gets you a cryptography, and whatever set of core backends we think are reasonable for your platform for doing stuff like typing AES (and, god willing, then typing HMAC). >> >> Thoughts? >> Alex >> >> -- >> "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >> "The people's good is the highest law." -- Cicero >> GPG Key fingerprint: 125F 5C67 DFE9 4084 >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > > So I (obviously given i?ve been saying it for awhile now ;) ) agree with > splitting out the backends! > > First off, my question is how important is #2 on platforms which do not ship > OpenSSL or have it easily installable which extends into how important is #5 at > all? > > I ask because I think that cryptography-core (versus just keeping the ?core? > stuff in cryptography) is a hack around a failing in python packaging and I > would much rather solve that by fixing python packaging. What that would mean > is that supporting #5 (and #2 on platforms where OpenSSL isn?t readably > available) isn?t super nice at first until packaging ?catches? up. > > It is my belief that the people with the use case of #2 are not going to care > if the bits to enable OpenSSL are residing on their machine unused unless those > bits forced them to go and manually install OpenSSL, with their level of care > increasing based on how painful it was for them to manually install OpenSSL. I > also believe that the #5 case will be the people who actually care ?No there > must be ZERO OpenSSL bits on my system? and I also believe that they will be a > minority use case. > > Given those beliefs I feel like it is reasonable to leave the ?core? stuff > inside of the cryptography package, and simple say that if you want to > ``pip install cryptography`` without getting the OpenSSL bits then you?ll need > to do ``pip install cryptography ?no-use-wheel ?install-option=??no-openssl?`` > or ``CRYPTOGRAPHY_WITHOUT_OPENSSL=1 pip install ?no-use-wheel cryptography``. > Once packaging catches up to this use case then we can clean that up to > something like ``pip install cryptography[nss]`` (replace nss with any other > backend here!). > > Beyond that there is one small sticky point to this: Imports. > > Currently we only ship one package (well two if you count cryptography_vectors) > however this is going to cause us to ship more than that. However we already > have a public API on ``cryptography.hazmat.bindings.*`` and > ``cryptography.hazmat.backends.*``. I do not think we can break those which > leaves us with two choices: > > A) Namespace packages - These are the ?obvious? first top to reach too, however > I can often be found saying that nobody has ever used namespace packages > without regretting it. In particular there are issues when combining > easy_install and pip. > B) Use some sort of shim to enable ``cryptography.hazmat.bindings.*`` and > ``cryptography.hazmat.backends.*`` to act as ?aliases? for something else, > either something based on setuptools entry points, or something based on > naming (ex: https://github.com/mitsuhiko/flask/blob/master/flask/ext/__init__.py) > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > More thoughts: * Can we even make it so ``pip install cryptography`` does not, by default, result in the OpenSSL backend being installed? Does this violate our backwards compatibility? pyOpenSSL in particular will likely be broken if OpenSSL isn't installed by default on all platforms. * Does the answer to the above change if we deprecate and then remove at a later date? * If OpenSSL becomes optional (either opt in or opt out) what is a "minimum" cryptography install? What, if anything, can I expect to work after I do ``pip install cryptography`` on any platform? The third one is the real stickler for me. Handling this in a case like scrypt is easy, because OpenSSL doesn't support scrypt, and ATM the only thing we have that does support scrypt is the proposed libscrypt based backend so we can easily say that scrypt is an optional interface and you have to install libscrypt for it. However if I can install cryptography without OpenSSL, then what can people depend on? Can they depend on AES being there? HMAC? CMAC? If the answer is "people can only depend on what they have installed" then likely almost everyone will simply always install (either implicitly or explicitly) cryptography-openssl because that will be the default "just make it work" option. If the decision to the third is that "you get whatever backends you have installed supports" which can be anything from "only this one tiny thing (scrypt?)" to every possible interface we support than is there a benefit to *making* people declare which backends they want to use? There could perhaps be generic named backends that would install some subset of functionality that projects could depend on when they only care about ensuring some functionality is present and not what backend provides it. Something like ``pip install cryptography[standardcrypto]`` or ``pip install cryptography-standardcrypto`` which would resolve to some subset of {openssl, commoncrypto, win32} (maybe even nss, botan, etc in the future) depending on platform. Any sort of generic named thing has the problem of how does some third party non PyCA based backend declare themselves as supporting that? How could a someone make a cryptography-nss which can implement the ``cryptography-standardcrypto`` package? Packaging has some answers to some of these questions now, and some in the future but if OpenSSL becomes optional (either opt in or opt out) we need to figure out how OpenSSL being missing or replaced by something else is handled. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cristian at ayni.io Mon Apr 28 23:00:26 2014 From: cristian at ayni.io (Cristian Salamea) Date: Mon, 28 Apr 2014 16:00:26 -0500 Subject: [Cryptography-dev] Packaging ideas In-Reply-To: References: Message-ID: <535EC16A.10106@ayni.io> El 28/04/14 11:32, Alex Gaynor escribi?: > So, before I can start talking about solutions, I have to describe the use > cases. > > 1) Some folks need a very specific backend (i.e. pyOpenSSL) and right now > it's just assumed that that will come with `pip install cryptography`. > 2) Some folks want to use cryptography as the API for non-traditional > primitives (e.g. scrypt), and `pip install cryptography` brings a lot more > than what they want. > 3) Some folks want to type the letters AES, and don't really care where > that comes from. > 4) Some folks want to type the letters "fab", and super don't care that > fabric needs a paramiko which needs an AES. > 5) Some folks want to be able `pip install something` without getting an > OpenSSL, because OpenSSL is literally the worst thing in the world. > > Invariants that must hold true: > > * `pip install cryptography` gets you a thing that can AES (and whatever > other core algorithms we think are important) > * There's a thing you can `pip install` that gets you scrypt, but not > OpenSSL > * There's a thing you can `pip install` to definitely get some OpenSSL. > > Solution: > > We break things down into a few different packages: > > * `pip install cryptography-core`: This contains all the primitives, all > the recipes, all the everything besides the backends/bindings > * `pip install cryptography-{openssl,scrypt,etc.}`: This contains the > backend/bindings for the specific thing > * `pip install cryptography`: This gets you a cryptography, and whatever > set of core backends we think are reasonable for your platform for doing > stuff like typing AES (and, god willing, then typing HMAC). Hi i am starting here and i think cryptography package could be the core, i see this library as the tool that gives me all stuff to write my new backend/binding and get cryptography-{ossl,nss}. Regards, > > Thoughts? > Alex > > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -- Cristian Salamea CTO Ayni Consulting Cooperativa de Software Libre