From newsham at LAVA.NET Fri Oct 3 01:17:20 2008 From: newsham at LAVA.NET (Tim Newsham) Date: Thu, 2 Oct 2008 13:17:20 -1000 Subject: [PYTHON-CRYPTO] M2Crypto questions Message-ID: Hi, I have several questions about M2Crypto and SSL verification. - Is there any code to load in the certificate roots from the windows certificate store? - Is it possible to validate the certificate against a set of roots after the connection has been made (ie. not using ctx.set_verify() to force verification at connection time)? - Is it possible to view the certificate chain used in the validation? - Is it possible to get a specific error about certificate validation failures that describes the cause of the validation failure? Tim Newsham http://www.thenewsh.com/~newsham/ From hjtoi at COMCAST.NET Sat Oct 4 07:22:14 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Fri, 3 Oct 2008 22:22:14 -0700 Subject: [PYTHON-CRYPTO] M2Crypto questions In-Reply-To: References: Message-ID: <48E6FD86.8010006@comcast.net> (I replied to Tim earlier privately, but the list message didn't come through.) Tim Newsham wrote: > > - Is there any code to load in the certificate roots from the > > windows certificate store? No. But you might want to find my post about where to get root certificates of some interest: http://www.heikkitoivonen.net/blog/2008/09/30/root-certificates-for-python-programs-using-python/ > > - Is it possible to validate the certificate against a set of roots > > after the connection has been made (ie. not using ctx.set_verify() > > to force verification at connection time)? I believe it will be possible with 0.19, there were some bugs/missing APIs that prevented/made it hard to do this with earlier releases (see https://bugzilla.osafoundation.org/show_bug.cgi?id=11690). I have never done this personally, though, so you'd be better off getting the details from elsewhere. You could search/ask on the openssl-users list, for example. > > - Is it possible to view the certificate chain used in the validation? Yes. One way is to set up a custom verification callback, which let's you look at the whole chain or the currently-being-tested-cert as you like. See for example the implementation in Chandler here: http://svn.osafoundation.org/chandler/trunk/chandler/parcels/osaf/framework/certstore/ssl.py (specifically, look at the verifyCallback method). > > - Is it possible to get a specific error about certificate validation > > failures that describes the cause of the validation failure? Yes, see for example the Chandler code above (look for store.get_error()). -- Heikki Toivonen - http://heikkitoivonen.net -- Heikki Toivonen - http://heikkitoivonen.net From hjtoi at COMCAST.NET Sat Oct 4 07:24:10 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Fri, 3 Oct 2008 22:24:10 -0700 Subject: [PYTHON-CRYPTO] M2Crypto 0.19beta2 Message-ID: <48E6FDFA.7090403@comcast.net> This is the second and likely last beta of the M2Crypto 0.19 release. Please give these bits a spin and report any problems. I expect the final 0.19 bits will be out Oct 5, 2008. Note: Python 2.6 shipped with optional timeout parameters to many network modules; these will not be supported in M2Crypto 0.19. I just noticed them too late... Otherwise 2.6 should work. Changes since 0.19beta1: - SSL.Connection.accept() passed wrong certificate to postConnectionCheck callback Changes: - OpenSSL OBJ_* functions wrapped by Pavel Shramov - OpenSSL ENGINE interface wrapped, providing support for smart cards, by Martin Paljak and Pavel Shramov - EVP.PKey.get_rsa() now returns RSA_pub, which fixes segmentation fault when trying to encrypt using public key from X509 certificate, by Ben Timby - httpslib.ProxyHTTPSConnection now sends the required Host header, by Karl Grindley - Use the proxied User-Agent value in CONNECT requests, by James Antill and Miloslav Trmac - Fixed m2urllib.build_opener when optional handlers were in use, affected Python 2.5 and later, by Miloslav Trmac - Reverted the incorrect GIL change done in 0.18 to m2.passphrase_callback, which caused a deadlock when called from mod_python for example. Thanks to Michal Kochel and Keith Jackson. - httpslib.HTTPSConnection now raises ValueError for illegal keyword argument - m2.pkey_write_pem[_no_cipher] changed to use the recommended (more secure) PEM_write_bio_PKCS8PrivateKey (used by PEM_write_bio_PrivateKey). - X509.load_cert, load_cert_bio, load_cert_der_string, new_stack_from_der, load_request and load_crl now raise X509Error for invalid data. Previously some of these raised a string as an error, some did not raise but caused strange errors later, for example x509.verify() would return -1. - Fixed SSL.Connection.get_socket_read_timeout and set_socket_read_timeout on 64bit platforms by adding SSL.timeout.struct_size() and using it instead of hardcoded size for socket.getsockopt - X509_Store.load_info now returns the value from the underlying m2.x509_store_load_locations call, and in case of error raises X509Error - Fixed SMIME.verify to raise the correct PKCS7_Error (used to raise SMIME_Error) when verification fails with Python 2.6 Requirements: * Python 2.3 or newer o m2urllib2 requires Python 2.4 or newer * OpenSSL 0.9.7 or newer o Some optional new features will require OpenSSL 0.9.8 or newer * SWIG 1.3.24 or newer required for building o SWIG 1.3.30 or newer may be required with Python 2.5 or newer and Python 2.4 with Py_ssize_t patches Get it while it's hot from M2Crypto homepage: http://chandlerproject.org/Projects/MeTooCrypto Or use easy_install: easy_install M2Crypto -- Heikki Toivonen - http://heikkitoivonen.net From hjtoi at COMCAST.NET Mon Oct 6 06:41:41 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Sun, 5 Oct 2008 21:41:41 -0700 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.19 Message-ID: <48E99705.5040904@comcast.net> I am happy to announce the M2Crypto 0.19 release! M2Crypto is the most complete Python wrapper for OpenSSL featuring RSA, DSA, DH, HMACs, message digests, symmetric ciphers (including AES); SSL functionality to implement clients and servers; HTTPS extensions to Python's httplib, urllib, and xmlrpclib; unforgeable HMAC'ing AuthCookies for web session management; FTP/TLS client and server; S/MIME; ZServerSSL: A HTTPS server for Zope and ZSmime: An S/MIME messenger for Zope. M2Crypto can also be used to provide SSL for Twisted. Note: Python 2.6 shipped with optional timeout parameters to many network modules; there is no support for these in M2Crypto 0.19. Otherwise 2.6 is supported. Changes: - OpenSSL OBJ_* functions wrapped by Pavel Shramov - OpenSSL ENGINE interface wrapped, providing support for smart cards, by Martin Paljak and Pavel Shramov - EVP.PKey.get_rsa() now returns RSA_pub, which fixes segmentation fault when trying to encrypt using public key from X509 certificate, by Ben Timby - httpslib.ProxyHTTPSConnection now sends the required Host header, by Karl Grindley - Use the proxied User-Agent value in CONNECT requests, by James Antill and Miloslav Trmac - Fixed m2urllib.build_opener when optional handlers were in use, affected Python 2.5 and later, by Miloslav Trmac - Reverted the incorrect GIL change done in 0.18 to m2.passphrase_callback, which caused a deadlock when called from mod_python for example. Thanks to Michal Kochel and Keith Jackson. - SSL.Connection.accept() passed wrong certificate to postConnectionCheck callback - httpslib.HTTPSConnection now raises ValueError for illegal keyword argument - m2.pkey_write_pem[_no_cipher] changed to use the recommended (more secure) PEM_write_bio_PKCS8PrivateKey (used by PEM_write_bio_PrivateKey). - X509.load_cert, load_cert_bio, load_cert_der_string, new_stack_from_der, load_request and load_crl now raise X509Error for invalid data. Previously some of these raised a string as an error, some did not raise but caused strange errors later, for example x509.verify() would return -1. - Fixed SSL.Connection.get_socket_read_timeout and set_socket_read_timeout on 64bit platforms by adding SSL.timeout.struct_size() and using it instead of hardcoded size for socket.getsockopt - X509_Store.load_info now returns the value from the underlying m2.x509_store_load_locations call, and in case of error raises X509Error - Fixed SMIME.verify to raise the correct PKCS7_Error (used to raise SMIME_Error) when verification fails with Python 2.6 Requirements: * Python 2.3 or newer o m2urllib2 requires Python 2.4 or newer * OpenSSL 0.9.7 or newer o Some optional new features will require OpenSSL 0.9.8 or newer * SWIG 1.3.24 or newer required for building o SWIG 1.3.30 or newer may be required with Python 2.5 or newer and Python 2.4 with Py_ssize_t patches Download link available from the M2Crypto homepage: http://chandlerproject.org/Projects/MeTooCrypto Or use easy_install (may not work on all systems): easy_install M2Crypto -- Heikki Toivonen - http://heikkitoivonen.net From dlitz at DLITZ.NET Sat Oct 11 02:25:44 2008 From: dlitz at DLITZ.NET (Dwayne C. Litzenberger) Date: Fri, 10 Oct 2008 20:25:44 -0400 Subject: [PYTHON-CRYPTO] PyCrypto rant thread Message-ID: <20081011002544.GA6196@rivest.dlitz.net> I'm sure there are a number of people on this list who are currently _not_ using PyCrypto. I know PyCrypto has problems, but as its new maintainer, it would be useful for me to find out exactly what problems others have had with it, so I can figure out what direction I should go with it. So, what do you hate about PyCrypto? -- Dwayne C. Litzenberger Key-signing key - 19E1 1FE8 B3CF F273 ED17 4A24 928C EC13 39C2 5CF7 Annual key (2008) - 4B2A FD82 FC7D 9E38 38D9 179F 1C11 B877 E780 4B45 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From hjtoi at COMCAST.NET Mon Oct 13 07:06:02 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Sun, 12 Oct 2008 22:06:02 -0700 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.19.1 Message-ID: <48F2D73A.2020904@comcast.net> The 0.19.1 release fixes the build when OpenSSL has been configured without EC support, thanks to Miloslav Trmac. M2Crypto is the most complete Python wrapper for OpenSSL featuring RSA, DSA, DH, HMACs, message digests, symmetric ciphers (including AES); SSL functionality to implement clients and servers; HTTPS extensions to Python's httplib, urllib, and xmlrpclib; unforgeable HMAC'ing AuthCookies for web session management; FTP/TLS client and server; S/MIME; ZServerSSL: A HTTPS server for Zope and ZSmime: An S/MIME messenger for Zope. M2Crypto can also be used to provide SSL for Twisted. Requirements: * Python 2.3 or newer o m2urllib2 requires Python 2.4 or newer * OpenSSL 0.9.7 or newer o Some optional new features will require OpenSSL 0.9.8 or newer * SWIG 1.3.24 or newer required for building o SWIG 1.3.30 or newer may be required with Python 2.5 or newer and Python 2.4 with Py_ssize_t patches Download link available from the M2Crypto homepage: http://chandlerproject.org/Projects/MeTooCrypto Or use easy_install (may not work on all systems): easy_install M2Crypto -- Heikki Toivonen - http://heikkitoivonen.net From chrisde88 at YAHOO.DE Tue Oct 14 15:41:21 2008 From: chrisde88 at YAHOO.DE (Christian) Date: Tue, 14 Oct 2008 15:41:21 +0200 Subject: [PYTHON-CRYPTO] Issues compiling M2Crypto against Py2.6 and OpenSSL 0.9.8i on Win32 with MSVC 2008 Message-ID: <48F4A181.3090007@yahoo.de> Hi, I migrated to Python 2.6 today and wanted reinstalled all third party packages, which I had before on Python 2.5.2. But with M2Crypto 0.19.1 I had some problems. 1) There are no binary build for Python 2.6, yet. So I decided to build my own ones. But that lead to more trouble. 2) There is a bug in OpenSSL 0.9.8.i, which breaks the build process. You have to add a undef to the following block in x509.h: #ifdef OPENSSL_SYS_WIN32 /* Under Win32 these are defined in wincrypt.h */ #undef X509_NAME #undef X509_CERT_PAIR #undef X509_EXTENSIONS #endif From: http://rt.openssl.org/Ticket/Display.html?id=1700&user=guest&pass=guest But I had some post building errors, when running my Python programs. It seems, that their is at least one incompatibility with Python 2.6, namely in the SSLServer. In Python 2.6 the underlying SocketServer.BaseServer has a some extra initializing code (events to stop serve_forever), which are not initialized in SSLServer (which is a hard coded copy of the constructor). My solution would be to change: self.server_address=server_address self.RequestHandlerClass=RequestHandlerClass to: SocketServer.BaseServer.__init__(self, server_address, RequestHandlerClass) which calls the base constructor, to do the standard initialization. --- Christian PS: I have uploaded a binary installer for Python 2.6 to the Wiki. Which actually have the bug in SSLServer. From info at EGENIX.COM Wed Oct 15 00:10:33 2008 From: info at EGENIX.COM (eGenix Team: M.-A. Lemburg) Date: Wed, 15 Oct 2008 00:10:33 +0200 Subject: [PYTHON-CRYPTO] Issues compiling M2Crypto against Py2.6 and OpenSSL 0.9.8i on Win32 with MSVC 2008 In-Reply-To: <48F4A181.3090007@yahoo.de> References: <48F4A181.3090007@yahoo.de> Message-ID: <48F518D9.7030107@egenix.com> On 2008-10-14 15:41, Christian wrote: > Hi, > I migrated to Python 2.6 today and wanted reinstalled all third party > packages, which I had before on Python 2.5.2. But with M2Crypto 0.19.1 I > had some problems. > 1) There are no binary build for Python 2.6, yet. So I decided to build > my own ones. > But that lead to more trouble. > 2) There is a bug in OpenSSL 0.9.8.i, which breaks the build process. > You have to add a undef to the following block in x509.h: > #ifdef OPENSSL_SYS_WIN32 > /* Under Win32 these are defined in wincrypt.h */ > #undef X509_NAME > #undef X509_CERT_PAIR > #undef X509_EXTENSIONS > #endif > From: > http://rt.openssl.org/Ticket/Display.html?id=1700&user=guest&pass=guest Here's a patch that fixes those problems: http://rt.openssl.org/Ticket/Display.html?id=1749&user=guest&pass=guest > But I had some post building errors, when running my Python programs. It > seems, that their is at least one incompatibility with Python 2.6, > namely in the SSLServer. In Python 2.6 the underlying > SocketServer.BaseServer has a some extra initializing code (events to > stop serve_forever), which are not initialized in SSLServer (which is a > hard coded copy of the constructor). My solution would be to change: > self.server_address=server_address > self.RequestHandlerClass=RequestHandlerClass > to: > SocketServer.BaseServer.__init__(self, server_address, > RequestHandlerClass) > which calls the base constructor, to do the standard initialization. If you only want to use is SSL, I'd suggest to have a look at pyOpenSSL. We have native VC9 builds available for Python 2.6: http://www.egenix.com/products/python/pyOpenSSL/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 15 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From zooko at ZOOKO.COM Thu Oct 16 15:38:08 2008 From: zooko at ZOOKO.COM (zooko) Date: Thu, 16 Oct 2008 07:38:08 -0600 Subject: [PYTHON-CRYPTO] PyCrypto rant thread In-Reply-To: <20081011002544.GA6196@rivest.dlitz.net> References: <20081011002544.GA6196@rivest.dlitz.net> Message-ID: <861988B5-DA8A-4EC9-9FE0-7FAB3A06614B@zooko.com> On Oct 10, 2008, at 18:25 PM, Dwayne C. Litzenberger wrote: > I'm sure there are a number of people on this list who are > currently _not_ using PyCrypto. I know PyCrypto has problems, but > as its new maintainer, it would be useful for me to find out > exactly what problems others have had with it, so I can figure out > what direction I should go with it. > > So, what do you hate about PyCrypto? Hey, it's not often one gets an invitation like this! I used PyCrypto for a couple of years, first for a prototype of Phil Zimmermann's Zfone project [1] and then in allmydata.org Tahoe [2]. The first thing that I didn't like was that CTR mode made a Python callback for each block, which was too slow and CPU intensive, so I wrote a patch to do CTR mode all in C code, and only with a simple incrementing counter, and I submitted that patch to AMK, who was then the ostensible maintainer of PyCrypto. The second thing that I didn't like was that AMK never replied -- PyCrypto was more or less unmaintained at that time. The third thing that I didn't like was that my encrypted phone software didn't work when I tried to make a call one day. It just so happened that on this day I was, for the first time, trying to make a call where one endpoint was on x86 CPU and the other endpoint was on amd64 CPU, although it didn't occur to me at first that this fact would be relevant. I assumed that there was yet another bug in my setup/negotiation/streaming code, and spent a long time experimenting with it before I finally proved to myself that my code was doing the right thing and that only a bug in PyCrypto's SHA-256 implementation could explain the failure. I ran PyCrypto's SHA-256 to produce the hashes of known inputs and sure enough, it was producing the wrong answers on my amd64 machine but the right ones on my x86 machine. I investigated and figured out that PyCrypto had copied the SHA-256 implementation from LibTomCrypt years earlier, and that the bug had subsequently been fixed in LibTomCrypt itself. The fourth thing that I didn't like was that my secure distributed filesystem (Tahoe) had a mysterious unit test failure one day. It turned out that the new version of PyCrypto's SHA-256 implementation, which was another, newer, copy from LibTomCrypt, had a *different* bug, one that caused incorrect output whenever the input was a certain length in bytes, modulo 64. Again, this bug had since been fixed in LibTomCrypt, but the fix had not been copied back to PyCrypto. At this point I gave up and switched away from using PyCrypto to writing my own Python package -- pycryptopp [3] -- which uses the Crypto++ library [4]. Crypto++ has many things to recommend it, especially the fact that it has thorough self-tests, so if there ever were a bug which caused incorrect output of a hash function on a new CPU architecture, or a bug which caused incorrect output of a hash function for certain input sizes, then it would be immediately detected the next time someone ran the tests, and the bug would not live long enough to worm its way into other projects. In addition to the unit tests which come with the Crypto++ library itself, I also wrote fairly thorough unit tests myself for the pycryptopp wrapper code that I wrote. I figured that even if Crypto+ + were correct, a bug in my Python wrapper code could cause the resulting pycryptopp library to be incorrect, and this paranoia turned out to be well-founded, as I've already written at least two significant bugs in my pycryptopp wrapper code, both of which were quickly detected by running the pycryptopp self-tests. (I say that I've written "at least" two significant bugs because the tests can't prove that I haven't written other bugs into the Python wrapper code or that the Crypto++ authors haven't written other bugs into the Crypto++ library itself -- the tests just caught the obvious ones -- incorrect output and segfault respectively.) So the fifth and most important thing that I didn't like about PyCrypto was: insufficiently thorough tests and insufficiently careful quality control, e.g. copying code from LibTomCrypt and then not watching the LibTomCrypt project to see if that code was subsequently discovered to be buggy. Note that Crypto++ itself has a pretty good track record of code quality. It was also the first open source crypto library to be certified as FIPS 140-2 level 1 compliant (the second was OpenSSL). Thanks for asking! I feel much better having had a Rant with my morning coffee, and now I look forward to a happy day of trying to debug that damned seg fault. :-) You can see if I succeeded by watching the pycryptopp buildbot waterfall: http://allmydata.org/buildbot-pycryptopp/waterfall?reload=60 If the tests all start passing, then the lights will all turn green. Regards, Zooko [1] http://zfoneproject.org [2] http://allmydata.org [3] http://allmydata.org/trac/pycryptopp [4] http://cryptopp.com --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month From dlitz at DLITZ.NET Mon Oct 20 01:27:03 2008 From: dlitz at DLITZ.NET (Dwayne C. Litzenberger) Date: Sun, 19 Oct 2008 19:27:03 -0400 Subject: [PYTHON-CRYPTO] PyCrypto rant thread In-Reply-To: <861988B5-DA8A-4EC9-9FE0-7FAB3A06614B@zooko.com> References: <20081011002544.GA6196@rivest.dlitz.net> <861988B5-DA8A-4EC9-9FE0-7FAB3A06614B@zooko.com> Message-ID: <20081019232703.GA31948@rivest.dlitz.net> On Thu, Oct 16, 2008 at 07:38:08AM -0600, zooko wrote: > The first thing that I didn't like was that CTR mode made a Python > callback for each block, which was too slow and CPU intensive, so I wrote > a patch to do CTR mode all in C code, and only with a simple incrementing > counter, and I submitted that patch to AMK, who was then the ostensible > maintainer of PyCrypto. The next version of PyCrypto will add the Crypto.Util.Counter module, which lets you do something like this: >>> from binascii import b2a_hex >>> from Crypto.Util import Counter >>> ctr = Counter.new(nbits=64, prefix="\xDE\xAD\xBE\xEF", suffix="\xAA\xAA\xAA\xAA", initial_value=1, little_endian=False) >>> b2a_hex(ctr()) 'deadbeef0000000000000001aaaaaaaa' >>> b2a_hex(ctr()) 'deadbeef0000000000000002aaaaaaaa' >>> ctr.next_value() 3L >>> b2a_hex(ctr()) 'deadbeef0000000000000003aaaaaaaa' As usual, you will then be able to feed the counter object to a block cipher: >>> from Crypto.Cipher import AES >>> key = "\0" * 32 >>> a = AES.new(key, AES.MODE_CTR, counter=ctr) >>> a.encrypt("\0"*16) '\xfc{\xae\x9d\xe5ch\xf0R\x02\\P6\x04\xc5\xab' The neat part is that (as of today) PyCrypto implements a shortcut that bypasses the Python method-call interface when you use these counter objects with PyCrypto's block ciphers. > The fourth thing that I didn't like was that my secure distributed > filesystem (Tahoe) had a mysterious unit test failure one day. It turned > out that the new version of PyCrypto's SHA-256 implementation, which was > another, newer, copy from LibTomCrypt, had a *different* bug, one that > caused incorrect output whenever the input was a certain length in bytes, > modulo 64. Yeah... The SHA256 "length = 55 (mod 64)" bug will be fixed in the next release of PyCrypto. It's also tested for in the new test suite (see below). > So the fifth and most important thing that I didn't like about PyCrypto > was: insufficiently thorough tests and insufficiently careful quality > control, e.g. copying code from LibTomCrypt and then not watching the > LibTomCrypt project to see if that code was subsequently discovered to be > buggy. I've written a brand new self-test suite: It's located in the Crypto.SelfTest package. I'm not completely happy with it, but it has a few key features that were missing from the old test code: - It can run outside the source tree (so distro users will be able to run the tests on their own systems) - I have cited sources for most of the test vectors. Where I couldn't find any test vectors elsewhere, I generated them by hand using another implementation (usually the openssl command). So you no longer need to wonder whether the test vectors are just a dump of PyCrypto's previous output. Also, before I publish my changes, I usually run the entire test suite against Python versions 2.1, 2.2, 2.3, 2.4, 2.5, and 2.6 on my machine (Linux/i386). I also occasionally test on Win32 and Linux/amd64. > Note that Crypto++ itself has a pretty good track record of code quality. > It was also the first open source crypto library to be certified as FIPS > 140-2 level 1 compliant (the second was OpenSSL). I don't know if PyCrypto will ever get official certification. My main concerns regarding certification are as follows: 1. The standards are often many years behind the state of the art. For example, FIPS 186-3, which finally allows DSA keys to be larger than 1024 bits, still hasn't been published in non-draft form. 2. Different standards tend to specify different algorithms for the same thing. For example, FIPS 186-2 (DSS) specifies a particular algorithm for generating primes using SHA-1, but it's specific to DSA and not very useful elsewhere. I don't want to add multiple prime-generation functions (thus increasing the risk of implementation bugs) just to pass certification. 3. From what I understand (and the situation is not at all clear to me) some ECC patents have been licensed for use by the U.S. Federal Government, but not for others to use. It sounds like the U.S. government is moving toward ECC, but I will not be adding patent-encumbered features that nobody can actually use just to pass certification. 4. Barely anyone uses "certifiable" code anyway. IIRC, either Mozilla Seamonkey or Firefox has a FIPS mode, but it's not enabled by default, and "turn on FIPS mode" has never been suggested as a workaround in any security advisory I have seen. OpenSSL's FIPS certification didn't prevent the Debian PRNG bug. Rather than worry about an unlikely certification that won't matter anyway, I would rather write a simpler, secure, state-of-the-art implementation and let others worry about maintaining certification-specific branches. As for code quality, I would like to see PyCrypto do a _lot_ better than it currently does. I also want to see improvements in the quality of *downstream* code, and I am taking steps in that direction. For example, the next version of PyCrypto will deprecate Crypto.Util.randpool.RandomPool. There's nothing wrong with the RandomPool implementation by itself. Sure, almost everyone misuses it, but that's just "user error", right? No, it's an error-prone interface, so I'm replacing it. (See "RandomPool encourages insecure code" https://bugs.launchpad.net/pycrypto/+bug/249765.) The Crypto.PublicKey package has similar problems. The RSA module doesn't do any padding, and the DSA module doesn't perform the appropriate hash before signing. I don't know about the other PublicKey modules, but I suspect they have similar problems. Crypto.PublicKey is another error-prone API, so it will eventually be replaced, too. (That won't happen until after the next release, though.) > Thanks for asking! I feel much better having had a Rant with my morning > coffee, and now I look forward to a happy day of trying to debug that > damned seg fault. :-) You can see if I succeeded by watching the > pycryptopp buildbot waterfall: > > http://allmydata.org/buildbot-pycryptopp/waterfall?reload=60 I think I need something like that. :-) Cheers, - Dwayne -- Dwayne C. Litzenberger Key-signing key - 19E1 1FE8 B3CF F273 ED17 4A24 928C EC13 39C2 5CF7 Annual key (2008) - 4B2A FD82 FC7D 9E38 38D9 179F 1C11 B877 E780 4B45 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From paul.hoffman at GMAIL.COM Mon Oct 20 17:59:44 2008 From: paul.hoffman at GMAIL.COM (Paul Hoffman) Date: Mon, 20 Oct 2008 08:59:44 -0700 Subject: [PYTHON-CRYPTO] FIPS stuff (was: Re: PyCrypto rant thread) Message-ID: <1e267dfe0810200859g60edb8l387e56dcc87af65e@mail.gmail.com> Wearing my standards-weenie hat: > 1. The standards are often many years behind the state of the art. For > example, FIPS 186-3, which finally allows DSA keys to be larger than > 1024 bits, still hasn't been published in non-draft form. The publicly-available draft of FIPS 186-3 has been published for quite a while now. The reason it is not final has everything to do with US government bureacracy, and nothing to do with any technical part being tentative. Many IETF RFCs, for example, point to drafts of FIPS documents while we wait (and wait and wait) for "real" publication. > 2. Different standards tend to specify different algorithms for the > same thing. For example, FIPS 186-2 (DSS) specifies a particular > algorithm for generating primes using SHA-1, but it's specific to DSA > and not very useful elsewhere. The prime generation code is in Appendix A of Draft FIPS 186-3. It is applicable to generating probable primes and provable primes for any system, not just for DSA. Also, in -3, it allows the use of SHA-256, -384, and -512 as well as -1. > I don't want to add multiple > prime-generation functions (thus increasing the risk of implementation > bugs) just to pass certification. Quite right. You should have just one that is usable everywhere. That is the whole point of FIPS 186-3 and NIST SP 800-90. > 3. From what I understand (and the situation is not at all clear to me) > some ECC patents have been licensed for use by the U.S. Federal > Government, but not for others to use. Not correct. You can license patents for ECC from Certicom for non-USGovt use. If you are making products for specific USGovt use, you may be able to get a free sublicense from the USGovt. > It sounds like the U.S. > government is moving toward ECC, but I will not be adding > patent-encumbered features that nobody can actually use just to pass > certification. That makes sense, but it is not relevant to getting certified. Most of the products with FIPS 140-2 certifications do not do ECC at all. > 4. Barely anyone uses "certifiable" code anyway. That is a difficult statement to prove either way without looking at the build chains for every product that is released. Also, don't forget that Microsoft's CAPI is certified. > IIRC, either Mozilla > Seamonkey or Firefox has a FIPS mode, but it's not enabled by > default, ...which shows that they *do* use the certified code, they just don't demand that users use it... > and "turn on FIPS mode" has never been suggested as a > workaround in any security advisory I have seen. OpenSSL's FIPS > certification didn't prevent the Debian PRNG bug. Both of those, while completely true, are completely orthogonal to whether or not certification is useful. They only show that certified code can be misused with the same gusto as non-certified code. > Rather than worry about an unlikely certification that won't matter anyway, > I would rather write a simpler, secure, state-of-the-art implementation and > let others worry about maintaining certification-specific branches. That sounds like a good plan. Having said that, it would great if you could re-evaluate your hesitancy to use certifiable methods from the FIPS and NIST SP 800 series in the base code itself, regardless of whether or not you or anyone intends to spend the (large amount of) time and effort to be FIPS 140 certified. --Paul Hoffman From dlitz at DLITZ.NET Mon Oct 20 22:08:33 2008 From: dlitz at DLITZ.NET (Dwayne C. Litzenberger) Date: Mon, 20 Oct 2008 16:08:33 -0400 Subject: [PYTHON-CRYPTO] FIPS stuff (was: Re: PyCrypto rant thread) In-Reply-To: <1e267dfe0810200859g60edb8l387e56dcc87af65e@mail.gmail.com> References: <1e267dfe0810200859g60edb8l387e56dcc87af65e@mail.gmail.com> Message-ID: <20081020200833.GA31910@rivest.dlitz.net> On Mon, Oct 20, 2008 at 08:59:44AM -0700, Paul Hoffman wrote: >Wearing my standards-weenie hat: > >> 1. The standards are often many years behind the state of the art. For >> example, FIPS 186-3, which finally allows DSA keys to be larger than >> 1024 bits, still hasn't been published in non-draft form. > >The publicly-available draft of FIPS 186-3 has been published for >quite a while now. The reason it is not final has everything to do >with US government bureacracy, and nothing to do with any technical >part being tentative. Many IETF RFCs, for example, point to drafts of >FIPS documents while we wait (and wait and wait) for "real" >publication. Yes, and as far as I know, any certification effort would have to target FIPS 186-2, not the FIPS 186-3 draft. That makes it difficult to implement a policy of avoiding weak algorithms, since the appropriate replacements might not be in the standard yet. DSA is a terrible signature algorithm, anyway. If you ever have a transient RNG failure while generating a DSA signature then your private key is trivially (and permanently) compromised. RSA only fails so catastrophically during key generation. Now that the patents on RSA have expired, why should anyone use DSA in new applications? >> 2. Different standards tend to specify different algorithms for the >> same thing. For example, FIPS 186-2 (DSS) specifies a particular >> algorithm for generating primes using SHA-1, but it's specific to DSA >> and not very useful elsewhere. > >The prime generation code is in Appendix A of Draft FIPS 186-3. It is >applicable to generating probable primes and provable primes for any >system, not just for DSA. Also, in -3, it allows the use of SHA-256, >-384, and -512 as well as -1. Sure, and that's great. However, PyCrypto already has a PRNG, so using the DSS prime-generation algorithm would require implementing *yet another PRNG*. The only tangible benefit of supporting the DSS prime-generation algorithm is that it allows compact representation of the private key (since you would only need to store the PRNG seed). >> I don't want to add multiple >> prime-generation functions (thus increasing the risk of implementation >> bugs) just to pass certification. > >Quite right. You should have just one that is usable everywhere. That >is the whole point of FIPS 186-3 and NIST SP 800-90. That's a very US-centric point of view. I would be surprised if the European, Russian, Japanese, and Chinese standards didn't specify different algorithms. If I'm going to care about certification, why would I limit myself to the US standards? I don't even live in the US. >> 3. From what I understand (and the situation is not at all clear to >> me) >> some ECC patents have been licensed for use by the U.S. Federal >> Government, but not for others to use. > >Not correct. You can license patents for ECC from Certicom for >non-USGovt use. If you are making products for specific USGovt use, >you may be able to get a free sublicense from the USGovt. Sorry, I'm referring specifically to the patents that the NSA licensed. As far as I can tell, the NSA's license only covers U.S. federal government use, not other uses. That's what I meant. As for being able to license patents separately from Certicom, I have no motivation to help anyone who makes the software patent problem worse, standards notwithstanding. It's often a steep uphill battle to get widespread adoption of any patent-encumbered crypto algorithm. I want to keep it that way. >> 4. Barely anyone uses "certifiable" code anyway. > >That is a difficult statement to prove either way without looking at >the build chains for every product that is released. Fair enough. What I mean is that for the packages I've seen that have an optional "FIPS mode" or "FIPS build", it seems like only a small fraction of users actually choose the certified/certifiable code over the "normal" code. >Also, don't forget that Microsoft's CAPI is certified. Is that the same CAPI that gave us the massively brain-damaged CryptGenRandom? (http://eprint.iacr.org/2007/419) That _anything_ written by Microsoft could be certified is exactly what's wrong with the certification system: Certification doesn't make an implementation secure. It doesn't even make it reasonable to believe that an implementation is secure. Certification just makes it harder to fire the moron who chose a "certified" implementation over a secure one. I recently found out about a procurement process that was looking for a full-disk encryption system that was FIPS 140 certified. Did it matter that FIPS 140 doesn't actually cover full-disk encryption? Nope, and products that weren't certified were dismissed without a second thought. >> IIRC, either Mozilla >> Seamonkey or Firefox has a FIPS mode, but it's not enabled by >> default, > >...which shows that they *do* use the certified code, they just don't >demand that users use it... It shows that the code is there, and that Mozilla is *huge*, not that the code is widely used. >> and "turn on FIPS mode" has never been suggested as a >> workaround in any security advisory I have seen. OpenSSL's FIPS >> certification didn't prevent the Debian PRNG bug. > >Both of those, while completely true, are completely orthogonal to >whether or not certification is useful. They only show that certified >code can be misused with the same gusto as non-certified code. It suggests that certification status and number of security holes are uncorrelated at best and inversely related at worst. >> Rather than worry about an unlikely certification that won't matter anyway, >> I would rather write a simpler, secure, state-of-the-art implementation and >> let others worry about maintaining certification-specific branches. > >That sounds like a good plan. Having said that, it would great if you >could re-evaluate your hesitancy to use certifiable methods from the >FIPS and NIST SP 800 series in the base code itself, regardless of >whether or not you or anyone intends to spend the (large amount of) >time and effort to be FIPS 140 certified. I treat FIPS and NIST special publications just like any other peer-reviewed source: useful, but tentative, and susceptible to errors, omissions, obsolescence and politics. I'm certainly not going to make a special effort to avoid the standards, but I have no intention of ever compromising the security of PyCrypto (or adding patent-encumbered code) just because some standard or certification process says I should. I don't care if Dual_EC_DRBG becomes a "mandatory" part of some standard; It's still not getting into my tree. -- Dwayne C. Litzenberger Key-signing key - 19E1 1FE8 B3CF F273 ED17 4A24 928C EC13 39C2 5CF7 Annual key (2008) - 4B2A FD82 FC7D 9E38 38D9 179F 1C11 B877 E780 4B45 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From paul.hoffman at GMAIL.COM Tue Oct 21 15:54:20 2008 From: paul.hoffman at GMAIL.COM (Paul Hoffman) Date: Tue, 21 Oct 2008 06:54:20 -0700 Subject: [PYTHON-CRYPTO] FIPS stuff (was: Re: PyCrypto rant thread) In-Reply-To: <20081020200833.GA31910@rivest.dlitz.net> References: <1e267dfe0810200859g60edb8l387e56dcc87af65e@mail.gmail.com> <20081020200833.GA31910@rivest.dlitz.net> Message-ID: <1e267dfe0810210654r26946474k99dc3a1bcd0e547a@mail.gmail.com> On Mon, Oct 20, 2008 at 1:08 PM, Dwayne C. Litzenberger wrote: > On Mon, Oct 20, 2008 at 08:59:44AM -0700, Paul Hoffman wrote: > Yes, and as far as I know, any certification effort would have to target > FIPS 186-2, not the FIPS 186-3 draft. That makes it difficult to implement > a policy of avoiding weak algorithms, since the appropriate replacements > might not be in the standard yet. Not true. You could implement both, as many others who have gotten certified have. You do not have to make any algorithm you consider weak part of your shipping product. > > DSA is a terrible signature algorithm, anyway. If you ever have a transient > RNG failure while generating a DSA signature then your private key is > trivially (and permanently) compromised. RSA only fails so catastrophically > during key generation. Now that the patents on RSA have expired, why should > anyone use DSA in new applications? > So, don't do DSA. No one is forcing you to. Probably no one is even asking you to. RSA is part of FIPS 186-2 and -3. You will be in good company ignoring DSA completely. At least in the VPN field, almost no one does DSA. > Sure, and that's great. However, PyCrypto already has a PRNG, so using the > DSS prime-generation algorithm would require implementing *yet another > PRNG*. The only tangible benefit of supporting the DSS prime-generation > algorithm is that it allows compact representation of the private key (since > you would only need to store the PRNG seed). The other tangible benefit of swapping your current PRNG with one that is FIPS-approved is that you will know that the PRNG you end up with is correct. That is important to some users. You would be amazed at the number of hand-rolled PRNGs that people are sure are correct that have flaws. >> Quite right. You should have just one that is usable everywhere. That >> is the whole point of FIPS 186-3 and NIST SP 800-90. > > That's a very US-centric point of view. I would be surprised if the > European, Russian, Japanese, and Chinese standards didn't specify different > algorithms. I have never heard of that, and I work with manufacturers who implement at least the first three. Do you have any evidence of differences in PRNG requirements? > If I'm going to care about certification, why would I limit > myself to the US standards? I don't even live in the US. No one said you you limit yourself to US standards. If you want to add support for NESSIE, GOST, and Camillia, feel free to. (Sorry, I don't know the name of the Chinese standard you referred to.) I don't even believe that *you* should try to get PyCrypto certified. I am proposing that you use the best known crypto practices, and don't do anything that would prevent someone else who wants to gat a PyCrytpo-based system from spending the time and effort to get their system certified, anywhere in the world. > As for being able to license patents separately from Certicom, I have no > motivation to help anyone who makes the software patent problem worse, > standards notwithstanding. It's often a steep uphill battle to get > widespread adoption of any patent-encumbered crypto algorithm. I want to > keep it that way. Note that other crypto libraries include ECC, and it seems kind of unlikely that they have licenses for them. >> That is a difficult statement to prove either way without looking at >> the build chains for every product that is released. > > Fair enough. What I mean is that for the packages I've seen that have an > optional "FIPS mode" or "FIPS build", it seems like only a small fraction of > users actually choose the certified/certifiable code over the "normal" code. You seem to think there are two different code bases in all those products. I have seen the opposite in all cases I have looked at. Where there is a difference is that the "FIPS mode", by nature, puts a boundary around the code it uses, but tht doesn't prevent the non-FIPS mode from using the same code. > It suggests that certification status and number of security holes are > uncorrelated at best and inversely related at worst. Fully agree with the first part, not at all with the second unless you have any evidence. > I treat FIPS and NIST special publications just like any other peer-reviewed > source: useful, but tentative, and susceptible to errors, omissions, > obsolescence and politics. That's a healthy attitude. > I'm certainly not going to make a special effort > to avoid the standards, Err, you said above that you weren't going to use the PRNG from FIPS 186-3, but instead use your own. > but I have no intention of ever compromising the > security of PyCrypto (or adding patent-encumbered code) just because some > standard or certification process says I should. Sounds fine. From dlitz at DLITZ.NET Tue Oct 21 18:03:26 2008 From: dlitz at DLITZ.NET (Dwayne C. Litzenberger) Date: Tue, 21 Oct 2008 12:03:26 -0400 Subject: [PYTHON-CRYPTO] FIPS stuff (was: Re: PyCrypto rant thread) In-Reply-To: <1e267dfe0810210654r26946474k99dc3a1bcd0e547a@mail.gmail.com> References: <1e267dfe0810200859g60edb8l387e56dcc87af65e@mail.gmail.com> <20081020200833.GA31910@rivest.dlitz.net> <1e267dfe0810210654r26946474k99dc3a1bcd0e547a@mail.gmail.com> Message-ID: <20081021160326.GA6205@rivest.dlitz.net> On Tue, Oct 21, 2008 at 06:54:20AM -0700, Paul Hoffman wrote: >On Mon, Oct 20, 2008 at 1:08 PM, Dwayne C. Litzenberger wrote: >> On Mon, Oct 20, 2008 at 08:59:44AM -0700, Paul Hoffman wrote: >> Yes, and as far as I know, any certification effort would have to target >> FIPS 186-2, not the FIPS 186-3 draft. That makes it difficult to implement >> a policy of avoiding weak algorithms, since the appropriate replacements >> might not be in the standard yet. > >Not true. You could implement both, as many others who have gotten >certified have. You do not have to make any algorithm you consider >weak part of your shipping product. My whole point is that I don't want to add multiple implementations where a single implementation would suffice but for the certification requirements. >> Sure, and that's great. However, PyCrypto already has a PRNG, so using >> the >> DSS prime-generation algorithm would require implementing *yet another >> PRNG*. The only tangible benefit of supporting the DSS prime-generation >> algorithm is that it allows compact representation of the private key (since >> you would only need to store the PRNG seed). > >The other tangible benefit of swapping your current PRNG with one that >is FIPS-approved is that you will know that the PRNG you end up with >is correct. There is no such guarantee of correctness. There are implementation bugs and algorithmic weaknesses to consider. If I replaced PyCrypto's current PRNG with (NIST-approved) Dual_EC_DRBG, would that be a good thing? >You would be amazed at the number of hand-rolled PRNGs that people are >sure are correct that have flaws. I would be amazed if most of them _weren't_ flawed. PyCrypto's flaws have primarily been implementation bugs and error-prone API design. My approach is to take _one_ good design and produce a good implementation, rather than taking several designs and trying to implement them all. In the case of a PRNG, I have chosen Fortuna over the DSA PRNG. Fortuna is Niels Ferguson & Bruce Schneier's algorithm, which is profusely documented in _Practical Cryptography_. As far as I know (and I might be mistaken here) the design rationale for the DSA PRNG has never been published. Users are encouraged to use Random.new() as their source of random bytes. Random.new() feeds entropy from the OS PRNG and from two different timers to Fortuna. The resulting data flow looks something like this: OS PRNG* --\ time.time() ---+----> Fortuna -> isPrime time.clock() --/ (* OS-provided RNG: /dev/urandom, CryptGenRandom, or os.urandom()) Adding the DSA PRNG will make the graph look like this: OS PRNG* --\ time.time() ---+----> Fortuna -> DSA PRNG -> isPrime time.clock() --/ (* OS-provided RNG: /dev/urandom, CryptGenRandom, or os.urandom()) If Fortuna is predictable or if isPrime doesn't work, then adding the DSA PRNG is unlikely to make things better, but a flawed implementation of the DSA PRNG could make things worse. In this example, my time would be better spent making sure Fortuna and isPrime work properly (and attempting to work around weaknesses in various OS PRNGs---argh). If you would like to argue the merits of Fortuna vs the DSA PRNG, that's perfectly fine. "You shouldn't just make up your own algorithm" is also a good argument (albeit not applicable in this case). However, "you can't get certified without the DSA PRNG" is not a convincing argument in my mind. That's really all I'm saying. >>> Quite right. You should have just one that is usable everywhere. That >>> is the whole point of FIPS 186-3 and NIST SP 800-90. >> >> That's a very US-centric point of view. I would be surprised if the >> European, Russian, Japanese, and Chinese standards didn't specify different >> algorithms. > >I have never heard of that, and I work with manufacturers who >implement at least the first three. Do you have any evidence of >differences in PRNG requirements? The PRNG was just an example. I understand that there are a wide variety of padding schemes and block cipher modes of operation. If I were implementing PyCrypto from scratch today, I might have omitted certain standardized modes of operation, like OFB and CFB mode, since they provide no benefit over modes like CBC and CTR. Similarly, if SHA-1 weren't already implemented, I wouldn't implement it today just to pass certification. As I mentioned before, one thing I am explicitly avoiding at the moment is X.509 and ASN.1. I have no idea if these are required for certification, but I suspect at least ASN.1 is. >> If I'm going to care about certification, why would I limit >> myself to the US standards? I don't even live in the US. > >No one said you you limit yourself to US standards. If you want to add >support for NESSIE, GOST, and Camillia, feel free to. (Sorry, I don't >know the name of the Chinese standard you referred to.) I don't even >believe that *you* should try to get PyCrypto certified. I am >proposing that you use the best known crypto practices, and don't do >anything that would prevent someone else who wants to gat a >PyCrytpo-based system from spending the time and effort to get their >system certified, anywhere in the world. That's exactly what I am proposing. Good. :-) >> As for being able to license patents separately from Certicom, I have no >> motivation to help anyone who makes the software patent problem worse, >> standards notwithstanding. It's often a steep uphill battle to get >> widespread adoption of any patent-encumbered crypto algorithm. I want to >> keep it that way. > >Note that other crypto libraries include ECC, and it seems kind of >unlikely that they have licenses for them. I'm referring specifically to NSA Suite B. I have heard a few people say that it's "okay to use because the NSA licensed the patents", but further investigation suggests that the patents are only licensed for narrow, U.S. federal government uses. On the other hand, I've heard that there are numerous ECC techniques that _aren't_ covered by patents. Unfortunately, there's no clarity, especially with regard to patents outside the U.S. >>> That is a difficult statement to prove either way without looking at >>> the build chains for every product that is released. >> >> Fair enough. What I mean is that for the packages I've seen that have an >> optional "FIPS mode" or "FIPS build", it seems like only a small >> fraction of >> users actually choose the certified/certifiable code over the "normal" code. > >You seem to think there are two different code bases in all those >products. I have seen the opposite in all cases I have looked at. >Where there is a difference is that the "FIPS mode", by nature, puts a >boundary around the code it uses, but tht doesn't prevent the non-FIPS >mode from using the same code. Fair enough. I have no objections to this approach. >> It suggests that certification status and number of security holes are >> uncorrelated at best and inversely related at worst. > >Fully agree with the first part, not at all with the second unless you >have any evidence. Fair enough. >> I treat FIPS and NIST special publications just like any other peer-reviewed >> source: useful, but tentative, and susceptible to errors, omissions, >> obsolescence and politics. > >That's a healthy attitude. :-) >> I'm certainly not going to make a special effort >> to avoid the standards, > >Err, you said above that you weren't going to use the PRNG from FIPS >186-3, but instead use your own. Luckily, I didn't design my own PRNG; I used Fortuna. See above. >> but I have no intention of ever compromising the >> security of PyCrypto (or adding patent-encumbered code) just because some >> standard or certification process says I should. > >Sounds fine. > Good. I should make it clear that I'm not particularly hostile to certification. With all the crap out there, I can understand why people would want a "certified" product. On the other hand, the certification standards often seem to be obsolete, and I have no intention of implementing obsolete algorithms (or allowing weak parameter choices) just to pass certification. My main complaint about the current FIPS standards is that they mostly target a 2**80 security level, which is fairly weak by modern standards. -- Dwayne C. Litzenberger Key-signing key - 19E1 1FE8 B3CF F273 ED17 4A24 928C EC13 39C2 5CF7 Annual key (2008) - 4B2A FD82 FC7D 9E38 38D9 179F 1C11 B877 E780 4B45 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From mads at KIILERICH.COM Wed Oct 22 00:07:22 2008 From: mads at KIILERICH.COM (Mads Kiilerich) Date: Wed, 22 Oct 2008 00:07:22 +0200 Subject: [PYTHON-CRYPTO] PyCrypto rant thread In-Reply-To: <20081011002544.GA6196@rivest.dlitz.net> References: <20081011002544.GA6196@rivest.dlitz.net> Message-ID: <48FE529A.2060405@kiilerich.com> Dwayne C. Litzenberger wrote, On 10/11/2008 02:25 AM: > I'm sure there are a number of people on this list who are currently > _not_ using PyCrypto. I know PyCrypto has problems, but as its new > maintainer, it would be useful for me to find out exactly what > problems others have had with it, so I can figure out what direction I > should go with it. This thread is getting to closer and closer to being a rent thread as you intended ;-) I have no strong feelings about PyCrypto or any of its alternatives. Having one defacto standard implementation of anything is good. In one way. But having several competing alternatives is also very good. In another way. I was looking for a python crypto library, and PyCrypto was one I was looking at. None of the candidates were obvious choices for me. You and others have pointed out places where PyCrypto had "room for improvement". Recognizing the problems and attacking them head on like you are doing is very good for the project. It is obvious from this thread that you have strong opinions about this project. You have good reasons for doing what you are doing, and for why your approach - and the result - is better than the alternatives. PLEASE make this explicit in the documentation and on the web page. PLEASE help us to choose PyCrypto for the right reasons - or to go for one of the alternatives if that would make all of us happier. Biased but fair comparisions would be nice. One important aspect which I had problems with and haven't seen mentioned is interoperability. It would be very helpful - both for evaluation and for debugging - to have examples of how PyCrypto can interface with and is compatible with other libraries. A real (or rather unrealistic) killer feature would be to have wrappers which implement (subsets) of the API of other libraries, so that their test suites could be run and examples compared. /Mads -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3435 bytes Desc: S/MIME Cryptographic Signature URL: From morel.jeremy at GMAIL.COM Tue Oct 28 10:35:15 2008 From: morel.jeremy at GMAIL.COM (Jeremy Morel) Date: Tue, 28 Oct 2008 10:35:15 +0100 Subject: [PYTHON-CRYPTO] M2Crypto and CAcert.org certificates Message-ID: Hi all, I am trying to use M2Crypto to authentify mails signed with a certificate delivered by CACert.org. I understand they do not use traditional public key/private key signing, but rather give you a certificate you use to sign your mails with embed the public key. Toying with M2Crypto, I was able to extract that public key, but all my attempts at verifying the signature failed. My code derives from the example available at http://sandbox.rulemaker.net/ngps/m2/howto.smime.html Here : from M2Crypto import SMIME, X509 # Load the data, verify it. p7, data = SMIME.smime_load_pkcs7('test_sign.eml') certif = X509.load_cert('root.crt') stack = X509.X509_Stack() signers = p7.get0_signers(stack) for signer in signers: pubkey = signer.get_pubkey() print signer # Instantiate an SMIME object. s = SMIME.SMIME() # Load the signer's cert. x509 = X509.X509() x509.set_pubkey(pubkey) sk = X509.X509_Stack() sk.push(x509) s.set_x509_stack(sk) st = X509.X509_Store() st.load_info('root.crt') s.set_x509_store(st) # Load the data, verify it. v = s.verify(p7) In this context, I have 2 files : * test_sign.eml is a signed email * root.crt is the root certificate from CACert.org data does contain the body of the mail, but it looks like p7 is empty. I get this error : Traceback (most recent call last): File "sign.py", line 38, in v = s.verify(p7) File "build/bdist.macosx-10.5-i386/egg/M2Crypto/SMIME.py", line 215, in verify M2Crypto.SMIME.PKCS7_Error: no content Does anyone know how to fix this ? Thanks in advance, J?r?my From hjtoi at COMCAST.NET Wed Oct 29 04:53:09 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Tue, 28 Oct 2008 20:53:09 -0700 Subject: [PYTHON-CRYPTO] M2Crypto and CAcert.org certificates In-Reply-To: References: Message-ID: <4907DE25.7090400@comcast.net> Jeremy Morel wrote: > I am trying to use M2Crypto to authentify mails signed with a certificate > delivered by CACert.org. I understand they do not use traditional public > key/private key signing, but rather give you a certificate you use to sign > your mails with embed the public key. Toying with M2Crypto, I was able to Can you send a link to where they explain this stuff? > My code derives from the example available at That is from the old M2Crypto location. M2Crypto homepage is now here: http://chandlerproject.org/Projects/MeTooCrypto and the new location for the document you were reading is http://svn.osafoundation.org/m2crypto/trunk/doc/howto.smime.html I have fixed one bug in the examples, in the Sign and Encrypt section. > certif = X509.load_cert('root.crt') You are not using certif for anything. > signers = p7.get0_signers(stack) > for signer in signers: > pubkey = signer.get_pubkey() > print signer That signer is an X509 object... > # Load the signer's cert. > x509 = X509.X509() > x509.set_pubkey(pubkey) ... so why are you creating a new X509 object here, why not use signer? If it does not work with that modification, could you send me an email signed with CAcert certificate and point me to a place where I can download their root cert? Cheers, -- Heikki Toivonen - http://heikkitoivonen.net From morel.jeremy at GMAIL.COM Wed Oct 29 08:44:44 2008 From: morel.jeremy at GMAIL.COM (=?ISO-8859-1?Q?J=E9r=E9my_Morel?=) Date: Wed, 29 Oct 2008 08:44:44 +0100 Subject: [PYTHON-CRYPTO] M2Crypto and CAcert.org certificates In-Reply-To: <4907DE25.7090400@comcast.net> References: <4907DE25.7090400@comcast.net> Message-ID: > Jeremy Morel wrote: >> I am trying to use M2Crypto to authentify mails signed with a >> certificate >> delivered by CACert.org. I understand they do not use traditional >> public >> key/private key signing, but rather give you a certificate you use >> to sign >> your mails with embed the public key. Toying with M2Crypto, I was >> able to > > Can you send a link to where they explain this stuff? Unfortunately, I can't. This explaination was given to me by a friend. But I guess you could arrive to the same conclusion by reading the CA website : http://www.cacert.org/ The main clue about that is that they do not deliver public/private key. They just give you a certificate and that's all. > > >> My code derives from the example available at > > That is from the old M2Crypto location. M2Crypto homepage is now here: > http://chandlerproject.org/Projects/MeTooCrypto and the new location > for > the document you were reading is > http://svn.osafoundation.org/m2crypto/trunk/doc/howto.smime.html > > I have fixed one bug in the examples, in the Sign and Encrypt section. > >> certif = X509.load_cert('root.crt') > > You are not using certif for anything. I know, I'm sorry. I tried to remove unused stuff from my previous attemps but some of it was still here. > > >> signers = p7.get0_signers(stack) >> for signer in signers: >> pubkey = signer.get_pubkey() >> print signer > > That signer is an X509 object... > > >> # Load the signer's cert. >> x509 = X509.X509() >> x509.set_pubkey(pubkey) > > ... so why are you creating a new X509 object here, why not use > signer? "It seemed like a good idea at the time" > > > If it does not work with that modification, could you send me an email > signed with CAcert certificate and point me to a place where I can > download their root cert? Unfortunately, it still doesn't work. I have the "no content" error. Root certificate can be found at http://www.cacert.org/index.php?id=3 and this email should be signed with my certificate. Thanks, J?r?my Morel -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2576 bytes Desc: not available URL: From acspike at GMAIL.COM Fri Oct 31 16:08:14 2008 From: acspike at GMAIL.COM (Aaron C Spike) Date: Fri, 31 Oct 2008 10:08:14 -0500 Subject: [PYTHON-CRYPTO] Are verify callbacks global in M2Crypto? Message-ID: <490B1F5E.3010008@gmail.com> Using M2Crypto 0.18.2 on Ubuntu Hardy Heron, it seems that the callbacks passed to Context.set_verify() are shared between Contexts and therefore only a single callback is used. I've created a test case to illustrate my experience. def test_verify_cb_multiple_ctx(self): pid = self.start_server(self.args) try: class VCB: def __init__(self): self.counter = 0 def __call__(self,ok,store): self.counter += 1 return 1 cb1 = VCB() cb2 = VCB() ctx1 = SSL.Context() ctx2 = SSL.Context() ctx1.set_verify(SSL.verify_peer | SSL.verify_fail_if_no_peer_cert, 9, cb1) ctx2.set_verify(SSL.verify_peer | SSL.verify_fail_if_no_peer_cert, 9, cb2) s1 = SSL.Connection(ctx1) s2 = SSL.Connection(ctx2) try: s1.connect(self.srv_addr) s1.close() s2.connect(self.srv_addr) s2.close except SSL.SSLError, e: assert 0, e finally: self.stop_server(pid) self.failIf(cb1.counter != cb2.counter) ====================================================================== FAIL: test_verify_cb_multiple_ctx (__main__.MiscSSLClientTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/test_ssl.py", line 629, in test_verify_cb_multiple_ctx self.failIf(cb1.counter != cb2.counter) AssertionError In the above example cb1.counter remains 0. My goal is to serve requests on two sockets with different requirements for the clients which connect to each. While this example is seen from the client perspective I experience the same behavior on the server. I would appreciate advice on how I can correct my code to function as I expect. Thank you, Aaron Spike