From andre.l.caron at gmail.com Thu Aug 11 13:38:42 2016 From: andre.l.caron at gmail.com (=?UTF-8?B?QW5kcsOpIENhcm9u?=) Date: Thu, 11 Aug 2016 13:38:42 -0400 Subject: [Cryptography-dev] ECDSA Interoperablity with Microsoft CNG-based peer Message-ID: Hi all, I'm dealing with a C++ client & server pair that uses ECDSA to verify the server's identity. I'm trying to write a new Python client that will exchange with the server without making any changes to the server. I've gotten quite a bit of this in place with cryptography (the Python package :-), but I'm incapable of getting the Python client to verify the signature sent by the server and I'd like to see if you can help me out. One of the problems here is that the serialization formats seem to be internal to Microsoft's CNG API. For example, the public key is the raw output of BCryptExportKey() and the signature is the raw output of BCryptSignHash(). These are Microsoft APIs, so... needless to say cryptography doesn't "just work" with these formats. I'm pretty sure I managed to nail the key format conversion as Microsoft makes an obscure reference to the format[1], but I'm still having trouble with signatures. The blob I get as output from BCryptSignHash() has 64 bytes, but signatures for the same algorithm using cryptography are usually 70-72 bytes, so I'm confused. Cryptography's ECC signature computation clearly documents the format: "The signature is formatted as DER-encoded bytes, as specified in RFC 3279." However, Microsoft doesn't seem to output record an equivalent anywhere. They're usually pretty consistent with their APIs and storage formats, so I assume some sort of storage similar to the keys where we have two 32-byte octet streams in big endian format containing the values for R and S, but I haven't had any luck with this. I also know that the DER encoding for two integer fields will normally add 6 bytes of overhead, which gets us up to 70, but there is still the occasional extra 1 or 2 bytes, so I'm obviously missing something and may not be on the right track. [1] :https://msdn.microsoft.com/library/aa375520.aspx Anyways, I managed to extract the BCrypt* function calls from the server and client into a pair of C++ program the contain only the signing and signature verification code to reproduce the flow. The total is ~30 lines of C++ code on each side, plus ~400 lines wrappers for BCrypt* calls (resource management, error handling and links to CNG API docs). I've also written a small cryptography-based Python program that tries to mimic the C++ client and I cannot get that part to run. If anyone has a few minutes to spare to give my Python code a second pair of eyeballs, I'd really appreciate it. I've saved up all of that on this Gist: https://gist.github.com/AndreLouisCaron/ab5ee411d0722a0981feceddbf5cb3d9 The gist contents are as follows: - genkeys.py: generate a public/private key pair, write to disk in Microsoft's format; - server.cpp: load secret key, compute signature, save payload & signature to disk; - client.cpp: load public key, payload & signature from disk, verify signature; - common.h: stuff shared by client.cpp & server.cpp; - client.py: same as client.cpp, but using cryptography. I also have an alternate C++ client based on OpenSSL which might be a better source of inspiration. I'll see if I can extract pars of that too as a reference since it might be easier to map to cryptography's internals. Thanks in advance, Andr? -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Fri Aug 12 07:41:10 2016 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Fri, 12 Aug 2016 07:41:10 -0400 Subject: [Cryptography-dev] ECDSA Interoperablity with Microsoft CNG-based peer In-Reply-To: References: Message-ID: https://stackoverflow.com/questions/20992760/understanding-bcryptsignhash-output-signature matches your intuition: the format out of Microsoft's function is just the two numbers concatenated together, perhaps they are little endian instead of big endian though? Alex On Thu, Aug 11, 2016 at 1:38 PM, Andr? Caron wrote: > Hi all, > > I'm dealing with a C++ client & server pair that uses ECDSA to verify the > server's identity. I'm trying to write a new Python client that will > exchange with the server without making any changes to the server. I've > gotten quite a bit of this in place with cryptography (the Python package > :-), but I'm incapable of getting the Python client to verify the signature > sent by the server and I'd like to see if you can help me out. > > One of the problems here is that the serialization formats seem to be > internal to Microsoft's CNG API. For example, the public key is the raw > output of BCryptExportKey() and the signature is the raw output of > BCryptSignHash(). These are Microsoft APIs, so... needless to say > cryptography doesn't "just work" with these formats. > > I'm pretty sure I managed to nail the key format conversion as Microsoft > makes an obscure reference to the format[1], but I'm still having trouble > with signatures. The blob I get as output from BCryptSignHash() has 64 > bytes, but signatures for the same algorithm using cryptography are usually > 70-72 bytes, so I'm confused. Cryptography's ECC signature computation > clearly documents the format: "The signature is formatted as DER-encoded > bytes, as specified in RFC 3279." However, Microsoft doesn't seem to > output record an equivalent anywhere. They're usually pretty consistent > with their APIs and storage formats, so I assume some sort of storage > similar to the keys where we have two 32-byte octet streams in big endian > format containing the values for R and S, but I haven't had any luck with > this. I also know that the DER encoding for two integer fields will > normally add 6 bytes of overhead, which gets us up to 70, but there is > still the occasional extra 1 or 2 bytes, so I'm obviously missing something > and may not be on the right track. > > [1] :https://msdn.microsoft.com/library/aa375520.aspx > > Anyways, I managed to extract the BCrypt* function calls from the server > and client into a pair of C++ program the contain only the signing and > signature verification code to reproduce the flow. The total is ~30 lines > of C++ code on each side, plus ~400 lines wrappers for BCrypt* calls > (resource management, error handling and links to CNG API docs). > > I've also written a small cryptography-based Python program that tries to > mimic the C++ client and I cannot get that part to run. > > If anyone has a few minutes to spare to give my Python code a second pair > of eyeballs, I'd really appreciate it. > > I've saved up all of that on this Gist: https://gist.github.com/ > AndreLouisCaron/ab5ee411d0722a0981feceddbf5cb3d9 > > The gist contents are as follows: > - genkeys.py: generate a public/private key pair, write to disk in > Microsoft's format; > - server.cpp: load secret key, compute signature, save payload & signature > to disk; > - client.cpp: load public key, payload & signature from disk, verify > signature; > - common.h: stuff shared by client.cpp & server.cpp; > - client.py: same as client.cpp, but using cryptography. > > I also have an alternate C++ client based on OpenSSL which might be a > better source of inspiration. I'll see if I can extract pars of that too > as a reference since it might be easier to map to cryptography's internals. > > Thanks in advance, > > Andr? > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: D1B3 ADC0 E023 8CA6 -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre.l.caron at gmail.com Tue Aug 16 11:18:02 2016 From: andre.l.caron at gmail.com (=?UTF-8?B?QW5kcsOpIENhcm9u?=) Date: Tue, 16 Aug 2016 11:18:02 -0400 Subject: [Cryptography-dev] ECDSA Interoperablity with Microsoft CNG-based peer In-Reply-To: References: Message-ID: Hi Alex, Thanks a bunch for the tip! I managed to get the signature verification step to work. Here is my code: signature = ... # 64 bytes in P1363 format. # Load signature from Microsoft's raw format to the DER-encoded # format expected by cryptography. # # Conversion steps: # - decode public numbers as 2 big-endian octet stream; # - DER-encode public numbers. def _bin2int(backend, x): backend = backend._backends[0] i = backend._lib.BN_bin2bn(x, len(x), backend._ffi.NULL) return backend._bn_to_int(i) signature = encode_dss_signature( _bin2int(backend, signature[:32]), _bin2int(backend, signature[32:]), ) I also managed to get the CGN API-based signature verification step to work when using a Cryptography-based server to sign the payload using this piece of code: signature = ... # ASN.1 # Convert signature from the DER-encoded format used by # OpenSSL into Microsoft's raw format # # Conversion steps: # - DER-decode numbers; # - encode numbers as 2 big-endian octet streams. def int2bin(backend, i): backend = backend._backends[0] i = backend._int_to_bn(i) x = backend._ffi.new( "unsigned char[]", backend._lib.BN_num_bytes(i) ) n = backend._lib.BN_bn2bin(i, x) i = backend._ffi.buffer(x)[:n] return i r, s = decode_dss_signature(signature) signature = int2bin(backend, r) + int2bin(backend, s) (I updated my gist with the C++ client & server and the Python client & server if you want to look at the rest.) I also have related snippets that allow me to convert public & private keys to and from Microsoft's format. Now, i have full interop: I can generate a keypair, share the public key, sign and verify using both Python and C++ (both directions). However, I'm relying on cryptography internals to do this, which is definitely not desirable in the medium-long term. Know of a better way to do these conversions by relying only on public APIs? Also, I guess I'm not the only person that's going to be running into this. Any interest in adding built-in support for this in cryptography? If so, I'd be willing to put some effort into a PR. Thanks, Andr? On Fri, Aug 12, 2016 at 7:41 AM, Alex Gaynor wrote: > https://stackoverflow.com/questions/20992760/understanding-bcryptsignhash- > output-signature matches your intuition: the format out of Microsoft's > function is just the two numbers concatenated together, perhaps they are > little endian instead of big endian though? > > Alex > > On Thu, Aug 11, 2016 at 1:38 PM, Andr? Caron > wrote: > >> Hi all, >> >> I'm dealing with a C++ client & server pair that uses ECDSA to verify the >> server's identity. I'm trying to write a new Python client that will >> exchange with the server without making any changes to the server. I've >> gotten quite a bit of this in place with cryptography (the Python package >> :-), but I'm incapable of getting the Python client to verify the signature >> sent by the server and I'd like to see if you can help me out. >> >> One of the problems here is that the serialization formats seem to be >> internal to Microsoft's CNG API. For example, the public key is the raw >> output of BCryptExportKey() and the signature is the raw output of >> BCryptSignHash(). These are Microsoft APIs, so... needless to say >> cryptography doesn't "just work" with these formats. >> >> I'm pretty sure I managed to nail the key format conversion as Microsoft >> makes an obscure reference to the format[1], but I'm still having trouble >> with signatures. The blob I get as output from BCryptSignHash() has 64 >> bytes, but signatures for the same algorithm using cryptography are usually >> 70-72 bytes, so I'm confused. Cryptography's ECC signature computation >> clearly documents the format: "The signature is formatted as DER-encoded >> bytes, as specified in RFC 3279." However, Microsoft doesn't seem to >> output record an equivalent anywhere. They're usually pretty consistent >> with their APIs and storage formats, so I assume some sort of storage >> similar to the keys where we have two 32-byte octet streams in big endian >> format containing the values for R and S, but I haven't had any luck with >> this. I also know that the DER encoding for two integer fields will >> normally add 6 bytes of overhead, which gets us up to 70, but there is >> still the occasional extra 1 or 2 bytes, so I'm obviously missing something >> and may not be on the right track. >> >> [1] :https://msdn.microsoft.com/library/aa375520.aspx >> >> Anyways, I managed to extract the BCrypt* function calls from the server >> and client into a pair of C++ program the contain only the signing and >> signature verification code to reproduce the flow. The total is ~30 lines >> of C++ code on each side, plus ~400 lines wrappers for BCrypt* calls >> (resource management, error handling and links to CNG API docs). >> >> I've also written a small cryptography-based Python program that tries to >> mimic the C++ client and I cannot get that part to run. >> >> If anyone has a few minutes to spare to give my Python code a second pair >> of eyeballs, I'd really appreciate it. >> >> I've saved up all of that on this Gist: https://gist.github.com/ >> AndreLouisCaron/ab5ee411d0722a0981feceddbf5cb3d9 >> >> The gist contents are as follows: >> - genkeys.py: generate a public/private key pair, write to disk in >> Microsoft's format; >> - server.cpp: load secret key, compute signature, save payload & >> signature to disk; >> - client.cpp: load public key, payload & signature from disk, verify >> signature; >> - common.h: stuff shared by client.cpp & server.cpp; >> - client.py: same as client.cpp, but using cryptography. >> >> I also have an alternate C++ client based on OpenSSL which might be a >> better source of inspiration. I'll see if I can extract pars of that too >> as a reference since it might be easier to map to cryptography's internals. >> >> Thanks in advance, >> >> Andr? >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> > > > -- > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: D1B3 ADC0 E023 8CA6 > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hines at monash.edu Wed Aug 17 20:15:02 2016 From: chris.hines at monash.edu (Chris Hines) Date: Thu, 18 Aug 2016 10:15:02 +1000 Subject: [Cryptography-dev] ssh public key processing Message-ID: Hi List, I have a question about the function cryptography.hazmat.primatives.serialization.load_ssh_public_key Basically is the function inteornded to load only the public key or is it intended that it be able to process any like out of an authorized_keys_file Source code shows that the function is prepared to strip of the key-type (eg ssh-rsa) and use it for comparison against the inner_key_type but is not prepared to strip off any options that can be passed in an authorized_keys file (For example SSH_FORCE_COMMAND or no-port-forwarding). I ask because the downstream project OpenStack Nova uses load_ssh_public_key to verify contents intended for authorized_keys is valid. Its easy enough to remove ssh options in Nova before passing to load_ssh_public_key, but I though if load_ssh_public_key already deals with the key-type header, perhaps it should also deal with the other options. I can create issues and merge requests if that is helpful, just looking for clarification on the intention (i.e. does load_ssh_public_key load contents intended for authorized_keys or just the public key part) Cheers, -- Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre.l.caron at gmail.com Wed Aug 17 20:31:37 2016 From: andre.l.caron at gmail.com (=?UTF-8?B?QW5kcsOpIENhcm9u?=) Date: Wed, 17 Aug 2016 20:31:37 -0400 Subject: [Cryptography-dev] ECDSA Interoperablity with Microsoft CNG-based peer In-Reply-To: References: Message-ID: For posterity, my two functions can be replaced with these: from cryptography.utils import int_from_bytes, int_to_bytes from cryptography.hazmat.primitives.asymmetric.utils import ( decode_dss_signature, encode_dss_signature, ) def decode_ieee_p1363_signature(data): """Decode ECDSA signature in IEEE P1363 format to ASN.1 (DER).""" return encode_dss_signature( int_from_bytes(data[:32], 'big'), int_from_bytes(data[32:], 'big'), ) def encode_ieee_p1363_signature(data): """Encode ECDSA signature in ASN.1 (DER) to IEEE P1363 format.""" r, s = decode_dss_signature(data) return int_to_bytes(r, 32) + int_to_bytes(s, 32) Andr? On Tue, Aug 16, 2016 at 11:18 AM, Andr? Caron wrote: > Hi Alex, > > Thanks a bunch for the tip! > > I managed to get the signature verification step to work. Here is my code: > > signature = ... # 64 bytes in P1363 format. > > # Load signature from Microsoft's raw format to the DER-encoded > # format expected by cryptography. > # > # Conversion steps: > # - decode public numbers as 2 big-endian octet stream; > # - DER-encode public numbers. > def _bin2int(backend, x): > backend = backend._backends[0] > i = backend._lib.BN_bin2bn(x, len(x), backend._ffi.NULL) > return backend._bn_to_int(i) > signature = encode_dss_signature( > _bin2int(backend, signature[:32]), > _bin2int(backend, signature[32:]), > ) > > I also managed to get the CGN API-based signature verification step to > work when using a Cryptography-based server to sign the payload using this > piece of code: > > signature = ... # ASN.1 > > # Convert signature from the DER-encoded format used by > # OpenSSL into Microsoft's raw format > # > # Conversion steps: > # - DER-decode numbers; > # - encode numbers as 2 big-endian octet streams. > def int2bin(backend, i): > backend = backend._backends[0] > i = backend._int_to_bn(i) > x = backend._ffi.new( > "unsigned char[]", backend._lib.BN_num_bytes(i) > ) > n = backend._lib.BN_bn2bin(i, x) > i = backend._ffi.buffer(x)[:n] > return i > r, s = decode_dss_signature(signature) > signature = int2bin(backend, r) + int2bin(backend, s) > > (I updated my gist with the C++ client & server and the Python client & > server if you want to look at the rest.) > > I also have related snippets that allow me to convert public & private > keys to and from Microsoft's format. > > Now, i have full interop: I can generate a keypair, share the public key, > sign and verify using both Python and C++ (both directions). > > However, I'm relying on cryptography internals to do this, which is > definitely not desirable in the medium-long term. Know of a better way to > do these conversions by relying only on public APIs? > > Also, I guess I'm not the only person that's going to be running into > this. Any interest in adding built-in support for this in cryptography? > If so, I'd be willing to put some effort into a PR. > > Thanks, > > Andr? > > > On Fri, Aug 12, 2016 at 7:41 AM, Alex Gaynor > wrote: > >> https://stackoverflow.com/questions/20992760/understanding- >> bcryptsignhash-output-signature matches your intuition: the format out >> of Microsoft's function is just the two numbers concatenated together, >> perhaps they are little endian instead of big endian though? >> >> Alex >> >> On Thu, Aug 11, 2016 at 1:38 PM, Andr? Caron >> wrote: >> >>> Hi all, >>> >>> I'm dealing with a C++ client & server pair that uses ECDSA to verify >>> the server's identity. I'm trying to write a new Python client that will >>> exchange with the server without making any changes to the server. I've >>> gotten quite a bit of this in place with cryptography (the Python package >>> :-), but I'm incapable of getting the Python client to verify the signature >>> sent by the server and I'd like to see if you can help me out. >>> >>> One of the problems here is that the serialization formats seem to be >>> internal to Microsoft's CNG API. For example, the public key is the raw >>> output of BCryptExportKey() and the signature is the raw output of >>> BCryptSignHash(). These are Microsoft APIs, so... needless to say >>> cryptography doesn't "just work" with these formats. >>> >>> I'm pretty sure I managed to nail the key format conversion as Microsoft >>> makes an obscure reference to the format[1], but I'm still having trouble >>> with signatures. The blob I get as output from BCryptSignHash() has 64 >>> bytes, but signatures for the same algorithm using cryptography are usually >>> 70-72 bytes, so I'm confused. Cryptography's ECC signature computation >>> clearly documents the format: "The signature is formatted as DER-encoded >>> bytes, as specified in RFC 3279." However, Microsoft doesn't seem to >>> output record an equivalent anywhere. They're usually pretty consistent >>> with their APIs and storage formats, so I assume some sort of storage >>> similar to the keys where we have two 32-byte octet streams in big endian >>> format containing the values for R and S, but I haven't had any luck with >>> this. I also know that the DER encoding for two integer fields will >>> normally add 6 bytes of overhead, which gets us up to 70, but there is >>> still the occasional extra 1 or 2 bytes, so I'm obviously missing something >>> and may not be on the right track. >>> >>> [1] :https://msdn.microsoft.com/library/aa375520.aspx >>> >>> Anyways, I managed to extract the BCrypt* function calls from the server >>> and client into a pair of C++ program the contain only the signing and >>> signature verification code to reproduce the flow. The total is ~30 lines >>> of C++ code on each side, plus ~400 lines wrappers for BCrypt* calls >>> (resource management, error handling and links to CNG API docs). >>> >>> I've also written a small cryptography-based Python program that tries >>> to mimic the C++ client and I cannot get that part to run. >>> >>> If anyone has a few minutes to spare to give my Python code a second >>> pair of eyeballs, I'd really appreciate it. >>> >>> I've saved up all of that on this Gist: https://gist.github.com/ >>> AndreLouisCaron/ab5ee411d0722a0981feceddbf5cb3d9 >>> >>> The gist contents are as follows: >>> - genkeys.py: generate a public/private key pair, write to disk in >>> Microsoft's format; >>> - server.cpp: load secret key, compute signature, save payload & >>> signature to disk; >>> - client.cpp: load public key, payload & signature from disk, verify >>> signature; >>> - common.h: stuff shared by client.cpp & server.cpp; >>> - client.py: same as client.cpp, but using cryptography. >>> >>> I also have an alternate C++ client based on OpenSSL which might be a >>> better source of inspiration. I'll see if I can extract pars of that too >>> as a reference since it might be easier to map to cryptography's internals. >>> >>> Thanks in advance, >>> >>> Andr? >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >>> >> >> >> -- >> "I disapprove of what you say, but I will defend to the death your right >> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >> "The people's good is the highest law." -- Cicero >> GPG Key fingerprint: D1B3 ADC0 E023 8CA6 >> >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Wed Aug 17 20:45:29 2016 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Wed, 17 Aug 2016 20:45:29 -0400 Subject: [Cryptography-dev] ECDSA Interoperablity with Microsoft CNG-based peer In-Reply-To: References: Message-ID: On August 16, 2016 at 11:18:16 PM, Andr? Caron (andre.l.caron at gmail.com) wrote: Hi Alex, However, I'm relying on cryptography internals to do this, which is definitely not desirable in the medium-long term. Know of a better way to do these conversions by relying only on public APIs? Not at the moment. BN conversions are strictly in the bindings. Also, I guess I'm not the only person that's going to be running into this. Any interest in adding built-in support for this in cryptography? If so, I'd be willing to put some effort into a PR. We'd like to eventually have a CNG backend, but we don't have a good story for merging coverage right now. That said, it probably should live in a separate repo while it's being worked on so getting coverage data becomes a simpler issue then. Thanks, Andr? -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Thu Aug 18 09:08:13 2016 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Thu, 18 Aug 2016 09:08:13 -0400 Subject: [Cryptography-dev] ssh public key processing In-Reply-To: References: Message-ID: Hi Chris, I don't think we've tried to specifically bound it. In general the assumption has been that the keys it loads would be OpenSSH public keys in the form that you get from an "id_rsa.pub" file (for example). What do the options look like? Are they put into the line at the end as comments? -Paul (reaperhulk) On August 18, 2016 at 8:15:16 AM, Chris Hines (chris.hines at monash.edu) wrote: Hi List, I have a question about the function cryptography.hazmat.primatives.serialization.load_ssh_public_key Basically is the function inteornded to load only the public key or is it intended that it be able to process any like out of an authorized_keys_file Source code shows that the function is prepared to strip of the key-type (eg ssh-rsa) and use it for comparison against the inner_key_type but is not prepared to strip off any options that can be passed in an authorized_keys file (For example SSH_FORCE_COMMAND or no-port-forwarding). I ask because the downstream project OpenStack Nova uses load_ssh_public_key to verify contents intended for authorized_keys is valid. Its easy enough to remove ssh options in Nova before passing to load_ssh_public_key, but I though if load_ssh_public_key already deals with the key-type header, perhaps it should also deal with the other options. I can create issues and merge requests if that is helpful, just looking for clarification on the intention (i.e. does load_ssh_public_key load contents intended for authorized_keys or just the public key part) Cheers, -- Chris _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre.l.caron at gmail.com Thu Aug 18 09:21:17 2016 From: andre.l.caron at gmail.com (=?UTF-8?B?QW5kcsOpIENhcm9u?=) Date: Thu, 18 Aug 2016 09:21:17 -0400 Subject: [Cryptography-dev] ECDSA Interoperablity with Microsoft CNG-based peer In-Reply-To: References: Message-ID: Hi Paul, > Not at the moment. BN conversions are strictly in the bindings. Actually, if you look at my previous email, these two little helpers turned out to work fine for me: from cryptography.utils import ( int_from_bytes, int_to_bytes, ) > We'd like to eventually have a CNG backend, but we don't have a good story for merging coverage right now. That said, it probably should live in a separate repo while it's being worked on so getting coverage data becomes a simpler issue then. I didn't mean to say I wanted to add a CNG backend. I meant adding conversion functions for P1363 encoding to easy compatibility with a peer that's CNG based. Here are the three things I'd be willing to work on: 1. transform from ASN.1 (DER) to P1363 2. transform from P1363 to ASN.1 (DER) 3. equivalent to "EllipticCurvePublicNumbers.from_encoded_point()", but for private numbers (to load private keys encoded as X, Y and d values in big-endian octet streams). Cheers, Andr? On Wed, Aug 17, 2016 at 8:45 PM, Paul Kehrer wrote: > On August 16, 2016 at 11:18:16 PM, Andr? Caron (andre.l.caron at gmail.com) > wrote: > > Hi Alex, > > > > > However, I'm relying on cryptography internals to do this, which is > definitely not desirable in the medium-long term. Know of a better way to > do these conversions by relying only on public APIs? > > Not at the moment. BN conversions are strictly in the bindings. > > > > Also, I guess I'm not the only person that's going to be running into > this. Any interest in adding built-in support for this in cryptography? > If so, I'd be willing to put some effort into a PR. > > We'd like to eventually have a CNG backend, but we don't have a good story > for merging coverage right now. That said, it probably should live in a > separate repo while it's being worked on so getting coverage data becomes a > simpler issue then. > > > > Thanks, > > Andr? > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.hines at monash.edu Thu Aug 18 19:04:10 2016 From: chris.hines at monash.edu (Chris Hines) Date: Fri, 19 Aug 2016 09:04:10 +1000 Subject: [Cryptography-dev] ssh public key processing In-Reply-To: References: Message-ID: Hi Paul, Options are specified in the sshd man page https://www.freebsd.org/cgi/man.cgi?sshd(8) Under the section Authorized Keys File Format. Technically options are not part of the public key (so are not covered by RFC4253) but are part of the OpenSSHD authorized_keys file format (which includes everything that can appear in an id_rsa.pub etc and extends it with the options defined in the man page). Sadly I don't think there is an RFC for the authorized_keys file (I suspect its OpenSSH specific) but I think we all assume its a de-facto standard :-) So at question is whether python-cryptography primitives support only the RFC4253 spec for public key formats or more generally processes lines from an authorized_keys file. The fix should be relatively simple. When you split the data into fields (key-type, data and comment) don't assume the first field is key-type, but instead search all fields. The particular option *I* am most concerned with is the cert-authority option (a relatively new feature in OpenSSH, but a pretty awesome one IMHO). It gets pre-pended to the key before going in authorized_keys, so a typical entry looks like: cert-authority ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMnxAxjwoCQsFJ1SC7+LJeSQCmUi8plJ9nKVmAjKlDr5Z240doRVIBr7+6veFJDPkaFzDxxa4dDn4O1ITXXVrr/JUCQbo4nF3ln4LSGWdDniJxH5uhc7PDYe2FDiiUYLrl4I+n1cB5YSOTDjxERcvXORKKGjPoyYNVuv343a5n7ygIOVTnI9VwTlxKj8gNyMm7wRg+aIJ8yiDwaDqestL9qTGMc+bb8Q0w0OJn8KZoGBYGl7LvS0QNyXsp+5J0GGEuE1c0lp1d5HpgvqRdWeWrlGY5alyQ2BchfJLbUbWQGBP/+kmVZCR022jbEo13/SznECr8ym8cXZzYg+hC1Err hines at tun This particular option brings some of the functionality of x509 certificates used for TLS communications (i.e. certifying authorities generating certificates that can be trusted becase you trust the CA) to SSH without bringing the full attack surface of x509 to ssh (at least that was the justification from the OpenSSH authors for not just using x509 :-) Other interesting options include command="..." and no-agent-forwarding,no-port-forwarding Ironically I'm not actually certain if this support should be added to python-cryptography, or if it should be added downstream in openstack nova. Technically its part of the authorized_keys file format, not part of the pub key format, and the function in question is pretty clearly named "load_ssh_public_key". On the other hand, it is clear there is some confusion in the wider community of users on the difference between a public key and a line in an authorized_keys file, and perhaps it would be a nice feature to add. Cheers, -- Chris. On 18 August 2016 at 23:08, Paul Kehrer wrote: > Hi Chris, > > I don't think we've tried to specifically bound it. In general the > assumption has been that the keys it loads would be OpenSSH public keys in > the form that you get from an "id_rsa.pub" file (for example). > > What do the options look like? Are they put into the line at the end as > comments? > > -Paul (reaperhulk) > > > On August 18, 2016 at 8:15:16 AM, Chris Hines (chris.hines at monash.edu) > wrote: > > Hi List, > I have a question about the function > cryptography.hazmat.primatives.serialization.load_ssh_public_key > > Basically is the function inteornded to load only the public key or is it > intended that it be able to process any like out of an authorized_keys_file > > Source code shows that the function is prepared to strip of the key-type > (eg ssh-rsa) and use it for comparison against the inner_key_type but is > not prepared to strip off any options that can be passed in an > authorized_keys file (For example SSH_FORCE_COMMAND or no-port-forwarding). > > I ask because the downstream project OpenStack Nova uses > load_ssh_public_key to verify contents intended for authorized_keys is > valid. Its easy enough to remove ssh options in Nova before passing to > load_ssh_public_key, but I though if load_ssh_public_key already deals with > the key-type header, perhaps it should also deal with the other options. > > I can create issues and merge requests if that is helpful, just looking > for clarification on the intention (i.e. does load_ssh_public_key load > contents intended for authorized_keys or just the public key part) > > Cheers, > -- > Chris > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronf at timeheart.net Thu Aug 18 20:41:13 2016 From: ronf at timeheart.net (Ron Frederick) Date: Thu, 18 Aug 2016 17:41:13 -0700 Subject: [Cryptography-dev] ssh public key processing In-Reply-To: References: Message-ID: <098DE40A-948D-42C0-BF77-6488951C8F0F@timeheart.net> I?ve implemented all of this in AsyncSSH, and there I chose to break out the authorized_key parsing from public key parsing. More specifically, AsyncSSH provides the following functions for reading SSH public keys: import_public_key asyncssh.import_public_key(data)[source] Import a public key This function imports a public key encoded in OpenSSH, RFC4716, or PKCS#1 or PKCS#8 DER or PEM format. Parameters: data (bytes or ASCII string) ? The data to import. Returns: An SSHKey public key read_public_key asyncssh.read_public_key(filename)[source] Read a public key from a file This function reads a public key from a file. See the function import_public_key() for information about the formats supported. Parameters: filename (str ) ? The file to read the key from. Returns: An SSHKey public key read_public_key_list asyncssh.read_public_key_list(filename)[source] Read a list of public keys from a file This function reads a list of public keys from a file. See the function import_public_key() for information about the formats supported. Parameters: filename (str ) ? The file to read the keys from. Returns: A list of SSHKey public keys Similar functions are also available for operating on SSH private keys and certificates. Then, I have a separate set of functions for operating on data in authorized_key format: import_authorized_keys asyncssh.import_authorized_keys(data)[source] Import SSH authorized keys This function imports public keys and associated options in OpenSSH authorized keys format. Parameters: data (str ) ? The key data to import. Returns: An SSHAuthorizedKeys object read_authorized_keys asyncssh.read_authorized_keys(filename)[source] Read SSH authorized keys from a file This function reads public keys and associated options in OpenSSH authorized_keys format from a file. Parameters: filename (str ) ? The file to read the keys from. Returns: An SSHAuthorizedKeys object Internally, the authorized_key functions use the public key functions, but only after stripping any prefix like ?cert-authority? or the various options for limiting when the keys can be used or what SSH actions are allowed when using that key. Then, given an SSH key and some other parameters like the client IP and certificate information the resulting SSHAuthorizedKeys object can then perform matching against all these constraints and return if a key is allowed to be used and what permissions it grants if so. On Aug 18, 2016, at 4:04 PM, Chris Hines wrote: > Hi Paul, > Options are specified in the sshd man page > > https://www.freebsd.org/cgi/man.cgi?sshd(8) > > Under the section Authorized Keys File Format. > > Technically options are not part of the public key (so are not covered by RFC4253) but are part of the OpenSSHD authorized_keys file format (which includes everything that can appear in an id_rsa.pub etc and extends it with the options defined in the man page). > > Sadly I don't think there is an RFC for the authorized_keys file (I suspect its OpenSSH specific) but I think we all assume its a de-facto standard :-) > > So at question is whether python-cryptography primitives support only the RFC4253 spec for public key formats or more generally processes lines from an authorized_keys file. > > The fix should be relatively simple. When you split the data into fields (key-type, data and comment) don't assume the first field is key-type, but instead search all fields. > > The particular option *I* am most concerned with is the cert-authority option (a relatively new feature in OpenSSH, but a pretty awesome one IMHO). It gets pre-pended to the key before going in authorized_keys, so a typical entry looks like: > > cert-authority ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMnxAxjwoCQsFJ1SC7+LJeSQCmUi8plJ9nKVmAjKlDr5Z240doRVIBr7+6veFJDPkaFzDxxa4dDn4O1ITXXVrr/JUCQbo4nF3ln4LSGWdDniJxH5uhc7PDYe2FDiiUYLrl4I+n1cB5YSOTDjxERcvXORKKGjPoyYNVuv343a5n7ygIOVTnI9VwTlxKj8gNyMm7wRg+aIJ8yiDwaDqestL9qTGMc+bb8Q0w0OJn8KZoGBYGl7LvS0QNyXsp+5J0GGEuE1c0lp1d5HpgvqRdWeWrlGY5alyQ2BchfJLbUbWQGBP/+kmVZCR022jbEo13/SznECr8ym8cXZzYg+hC1Err hines at tun > > This particular option brings some of the functionality of x509 certificates used for TLS communications (i.e. certifying authorities generating certificates that can be trusted becase you trust the CA) to SSH without bringing the full attack surface of x509 to ssh (at least that was the justification from the OpenSSH authors for not just using x509 :-) > > Other interesting options include command="..." and no-agent-forwarding,no-port-forwarding > > Ironically I'm not actually certain if this support should be added to python-cryptography, or if it should be added downstream in openstack nova. Technically its part of the authorized_keys file format, not part of the pub key format, and the function in question is pretty clearly named "load_ssh_public_key". On the other hand, it is clear there is some confusion in the wider community of users on the difference between a public key and a line in an authorized_keys file, and perhaps it would be a nice feature to add. > > Cheers, > -- > Chris. > > On 18 August 2016 at 23:08, Paul Kehrer > wrote: > Hi Chris, > > I don't think we've tried to specifically bound it. In general the assumption has been that the keys it loads would be OpenSSH public keys in the form that you get from an "id_rsa.pub" file (for example). > > What do the options look like? Are they put into the line at the end as comments? > > -Paul (reaperhulk) > > > On August 18, 2016 at 8:15:16 AM, Chris Hines (chris.hines at monash.edu ) wrote: > >> Hi List, >> I have a question about the function >> cryptography.hazmat.primatives.serialization.load_ssh_public_key >> >> Basically is the function inteornded to load only the public key or is it intended that it be able to process any like out of an authorized_keys_file >> >> Source code shows that the function is prepared to strip of the key-type (eg ssh-rsa) and use it for comparison against the inner_key_type but is not prepared to strip off any options that can be passed in an authorized_keys file (For example SSH_FORCE_COMMAND or no-port-forwarding). >> >> I ask because the downstream project OpenStack Nova uses load_ssh_public_key to verify contents intended for authorized_keys is valid. Its easy enough to remove ssh options in Nova before passing to load_ssh_public_key, but I though if load_ssh_public_key already deals with the key-type header, perhaps it should also deal with the other options. >> >> I can create issues and merge requests if that is helpful, just looking for clarification on the intention (i.e. does load_ssh_public_key load contents intended for authorized_keys or just the public key part) -- Ron Frederick ronf at timeheart.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsandin at gmail.com Mon Aug 22 12:16:59 2016 From: jsandin at gmail.com (Joel Sandin) Date: Mon, 22 Aug 2016 12:16:59 -0400 Subject: [Cryptography-dev] on how (not) to chain certs with openssl + pyopenssl Message-ID: I wanted to use pyOpenSSL to verify a certificate chain in a TLS Certificate Handshake message, and in the process stumbled upon a _disturbing_ amount of misinformation online about how to do this, both using openSSL and with pyOpenSSL. This isn't about a bug in pyOpenSSL, but the combination of a) lack of documentation and b) potentially misleading unit tests could result in serious vulnerabilities for users of the library. It's clear from what I've found online that developers are confused and may have introduced vulnerabilities into their code. I sent this to the pyOpenSSL security contact and was cleared to post this to the list to solicit feedback. FWIW there seems to be an ongoing effort to _properly_ support chain verification in pyOpenSSL: https://github.com/pyca/pyopenssl/issues/502 https://github.com/pyca/pyopenssl/pull/473 But earlier changes to pyOpenSSL claim to support some form of chain verification, and appear to be the source of confusion for developers: https://github.com/pyca/pyopenssl/pull/155 The rest is long-winded but captures some of the issues with existing documentation online as well as the pyOpenSSL verify_certificate() function for those interested. best wishes Joel Cert "Chain" Verification ------------------------- As part of a system I'm building, I'd like to independently verify the server certificate included in a TLS Certificate (0x0b) Handshake message, sent by the server after the ServerHello Handshake message. The Certificate Handshake message contains the server's certificate, intermediate certificates needed to verify the server certificate, and (optionally) the root certificate of the CA that issued the intermediate cert(s). >From the TLS RFC: https://tools.ietf.org/html/rfc5246#page-47 certificate_list This is a sequence (chain) of certificates. The sender's certificate MUST come first in the list. Each following certificate MUST directly certify the one preceding it. Because certificate validation requires that root keys be distributed independently, the self-signed certificate that specifies the root certificate authority MAY be omitted from the chain, under the assumption that the remote end must already possess it in order to validate it in any case. Lets frame the problem: My goal is to verify this chain against a root certificate that I already know about and trust. The intermediates I'm presented are certificates I may never have seen before, and may have been tampered with in transit or replaced by an attacker that is impersonating the server I'm trying to connect to. Thus _chain_ is an important term here - I need to verify the site certificate, but in the process, I need to check the validity of any intermediate certs as well. In this use case, the intermediate cert is UNTRUSTED data and nothing prevents an attacker from replacing it with a self-signed certificate (e.g. root cert) - Calling it an intermediate because thats what the server claims it is doesn't make it one. To me, this is what verifying a chain means. And IMHO, this is the 'common case' for using intermediates - there are contexts where a system directly chooses to trust an intermediate, but I do not. Unfortunately, many of the solutions posted online (including those for pyOpenSSL) mistakenly trust the intermediate and make bypassing the verification process trivial for my use case. The Command Line ---------------- Lets review some of the misinformation about the use of 'openssl verify' for this purpose, as this misinformation could be the source of future bugs if developers decide to re-implement verification based on the command-line behavior. How do we supply the chain cert(s) for our use case? The intermediates are untrusted, so the answer is to use -untrusted (check man verify) but this is not the only suggestion. Googling turns up harmful guidance on the issue. For example, the top hit: http://stackoverflow.com/questions/25482199/verify-a- certificate-chain-using-openssl-verify This thread fortunately mentions -untrusted, but also includes several contradictory recommendations*, including the following to verify UserCert.pem: openssl verify -verbose -CAfile <(cat Intermediate.pem RootCert.pem) UserCert.pem Here, the intermediate and root are passed via the -CAfile command line argument, and the certificate to verify is in UserCert.pem. Is the poster deliberately trusting the intermediate? A person asks 'Will this actually verify the intermediate cert against the root cert?' and someone responds saying yes, it does. Variations of this advice is repeated elsewhere in various forms, for example: http://superuser.com/questions/904859/why-cant-i- verify-this-certificate-chain recommending: $ cat root.pem intermediate.pem > concat.pem $ openssl verify -CAfile concat.pem john.pem john.pem: OK This discussion also seems to echo the above, depending on how enduser-example.com.chain is generated: https://raymii.org/s/tutorials/OpenSSL_command_ line_Root_and_Intermediate_CA_including_OCSP_CRL%20and_revocation.html openssl verify -CAfile enduser-certs/enduser-example.com.chain enduser-certs/enduser-example.com.crt So what happens if we follow this advice for our problem? -CAfile vs -untrusted --------------------- Lets compare the approach described above to the use of -untrusted with real certificates. When we connect to sometechcompany.com we get the server certificate sometechcompany.com.pem, the intermediate sometechcompany_ist_ca_2.pem, and the CA certificate geotrust_global_ca.pem. *After confirming that geotrust_global_ca.pem is a certificate we trust*, we use openssl verify to verify the chain (note that the RFC allows the root CA to be omitted): Both of these indicate that that the sometechcompany.com.pem is trusted: $ openssl verify -CAfile geotrust_global_ca.pem -untrusted sometechcompany_ist_ca_2.pem sometechcompany.com.pem $ openssl verify -CAfile <(cat geotrust_global_ca.pem sometechcompany_ist_ca_2.pem) sometechcompany.com.pem If we omit the chain cert, or replace it with an invalid chain certificate, verification fails (as it should): $ verify -CAfile <(cat geotrust_global_ca.pem) sometechcompany.com.pem sometechcompany.com.pem: CN = sometechcompany.com, OU = management:idms.group.105316, O = sometechcompany Inc., ST = California, C = US error 20 at 0 depth lookup:unable to get local issuer certificate $ verify -CAfile <(cat geotrust_global_ca.pem invalid_chain_cert.crt) sometechcompany.com.pem sometechcompany.com.pem: CN = sometechcompany.com, OU = management:idms.group.105316, O = sometechcompany Inc., ST = California, C = US error 20 at 0 depth lookup:unable to get local issuer certificate It's complaining about the missing chain cert, so it seems like it must be validating it against the root CA. These work too: $ openssl verify -CAfile <(cat sometechcompany_ist_ca_2.pem) sometechcompany.com.pem sometechcompany.com.pem: OK $ openssl verify -untrusted sometechcompany_ist_ca_2.pem sometechcompany.com.pem sometechcompany.com.pem: OK So on this system, openssl is using the trusted certificates for verification. Because sometechcompany_ist_ca_2.pem isn't self signed, this *is* actually "verifying" it. Unfortunately, an "intermediate" cert that is actually a root / self-signed _will be treated as a trusted CA_ when using the recommended command given above: $ openssl verify -CAfile <(cat geotrust_global_ca.pem rogue_ca.pem) fake_sometechcompany_from_rogue_ca.com.pem fake_sometechcompany_from_rogue_ca.com.pem: OK This makes it trivial for a malicious server to "impersonate" sometechcompany.com, at least from the point of view of our verification scheme. The attacker creates a root cert, sign fake_sometechcompany_from_rogue_ca.com.pem with it, and sends this root as a so-called "intermediate". As we've seen, this approach to verification will show the server cert as valid, when in fact it is not issued by any of the CAs we trust. To go back to the right solution: Specifying the intermediate as untrusted (via -untrusted) causes this ultimately-untrusted certificate to fail validation, as it should: $ openssl verify -CAfile ~/geotrust_global_ca.pem -untrusted rogue_ca.pem fake_sometechcompany_from_rogue_ca.com.pem fake_sometechcompany_from_rogue_ca.com.pem: C = US, ST = Vermont, L = Barre, O = Trusted CA Intermediate Signing Cert, CN = intermediate.trusted.ca, emailAddress = intermediate at yahoo.com error 19 at 1 depth lookup:self signed certificate in certificate chain A rogue intermediate produces the same results. Note that there are more subtleties to using verify (w.r.t. purpose), also discussed http://stackoverflow.com/questions/23304139/openssl- verify-gives-ok-for-bad-certificate-chain This link also points out another gruesome usability wart for the command line tool - openSSL ends up ignoring multiple certs in the file passed for verification, and only verifies the first cert (I saw this recommended somewhere but can't find the link). Chain verification in pyopenssl: -------------------------------- This is all a diversion though - My real goal was to use pyOpenSSL for this purpose. There's nothing in the documentation about doing this so looking at the unit tests and code (grepping chain) is the next step. I'm not the first to do so - Googling 'how to verify certificate chain in python' produces the following stackoverflow post as the first hit. The poster bases their recommendations on pyopenssl unit tests: http://stackoverflow.com/questions/30700348/how-to-validate-verify-an-x509- certificate-chain-of-trust-in-python Another hit, same recommendation: http://www.yothenberg.com/validate-x509-certificate-in-python/ This second at least calls the intermediate cert 'trusted', but that contradicts the idea of any 'chain' of verification, at least in the sense of the term for my use case. The recommended approach from these links, drawn from the unit tests, boils down to the following when verifying server_cert: store = X509Store() store.add_cert(root_cert) store.add_cert(intermediate_cert) store_ctx = X509StoreContext(store, server_cert) We see this code makes no distinction between the root_cert and the intermediate. If we look at the documentation, add_cert itself adds a *trusted* cert (maybe add_trusted_cert would be a better name?). So this is looking like the command-line example discussed above: openssl verify -verbose -CAfile <(cat Intermediate.pem RootCert.pem) UserCert.pem How NOT to do chain verification -------------------------------- Lets try it, using the same certs used above: from OpenSSL.crypto import load_certificate, load_privatekey, FILETYPE_PEM from OpenSSL.crypto import X509Store, X509StoreContext from six import u, b, binary_type, PY3 root_cert_pem = open('geotrust_global_ca.pem').read() intermediate_cert_pem = open('sometechcompany_ist_ca_2.pem').read() intermediate_server_cert_pem = open('sometechcompany.com.pem').read() rogue_ca_cert_pem = open('rogue_ca.pem').read() fake_server_cert_from_rogue_ca_pem = open('fake_sometechcompany_ from_rogue_ca.com.pem').read() root_cert = load_certificate(FILETYPE_PEM, root_cert_pem) intermediate_cert = load_certificate(FILETYPE_PEM, intermediate_cert_pem) intermediate_server_cert = load_certificate(FILETYPE_PEM, intermediate_server_cert_pem) rogue_ca_cert = load_certificate(FILETYPE_PEM, rogue_ca_cert_pem) fake_server_cert_from_rogue_ca = load_certificate(FILETYPE_PEM, fake_server_cert_from_rogue_ca_pem) # recommended approach from the links above, prints None: store = X509Store() store.add_cert(root_cert) store.add_cert(intermediate_cert) store_ctx = X509StoreContext(store, intermediate_server_cert) print(store_ctx.verify_certificate()) # BAD - attacker substitutes a rogue CA in place of the intermediate, this prints None: store = X509Store() store.add_cert(root_cert) store.add_cert(rogue_ca_cert) store_ctx = X509StoreContext(store, fake_server_cert_from_rogue_ca) print(store_ctx.verify_certificate()) # interestingly, this causes an exception to be raised, and thus the behavior is a bit better than the 'openssl verify' antipattern shown above: store = X509Store() store.add_cert(intermediate_cert) store_ctx = X509StoreContext(store, intermediate_server_cert) print(store_ctx.verify_certificate()) Again, This makes it trivial for a malicious server to "impersonate" sometechcompany.com and bypass our verification. As we saw, the attacker creates a root cert, sign fake_sometechcompany_from_rogue_ca.com.pem with it, and sends this root as a so-called "intermediate". Unit tests ---------- For completeness, here is one of the unit tests referenced by the stackoverflow post - here, as we mentioned, add_cert adds a *trusted* intermediate certificate. This is reflected in the documentation for that function, but not clear from usage, hence the confusion above. 3644 def test_valid(self): 3645 """ 3646 :py:obj:`verify_certificate` returns ``None`` when called with a 3647 certificate and valid chain. 3648 """ 3649 store = X509Store() 3650 store.add_cert(self.root_cert) 3651 store.add_cert(self.intermediate_cert) 3652 store_ctx = X509StoreContext(store, self.intermediate_server_cert) 3653 self.assertEqual(store_ctx.verify_certificate(), None) an explicitly flawed verify_chain() ----------------------------------- The ability to verify certificate chains in pyopenssl was originally discussed in the following issue (can be found using google): https://github.com/pyca/pyopenssl/issues/154 and first contributed in the following change which explicitly uses this dangerous antipattern of trusting the chain certs: https://github.com/sholsapp/pyopenssl/blob/f8b517803ecb812a2140e45e069b20 e0ec1c0389/OpenSSL/crypto.py 2307 def verify_chain(cert, trust, chain=None): 2308 """ 2309 Verify a chain of certificates. 2310 2311 :param cert: certificate to verify 2312 :param trust: certificate to trust 2313 :param chain: additional certificates needed to verify the chain 2314 :return: None if the chain is valid, raise exception otherwise 2315 """ 2316 store = X509Store() 2317 if chain: 2318 for _cert in chain: 2319 store.add_cert(_cert) 2320 store.add_cert(trust) 2321 2322 store_ctx = _lib.X509_STORE_CTX_new() 2323 _lib.X509_STORE_CTX_init(store_ctx, store._store, cert._x509, _ffi.NULL); 2324 2325 ret = _lib.X509_verify_cert(store_ctx) 2326 if ret <= 0: 2327 _raise_current_error() These changes were ultimately abandoned and verify_certificate() was added here: https://github.com/pyca/pyopenssl/pull/155 "untrusted" certs ----------------- There seem to be some references to "untrusted" certificates in open issues: https://github.com/pyca/pyopenssl/issues/502 And these changes seem to add support for chain certificates in the sense I described in my original use case: https://github.com/pyca/pyopenssl/pull/473 It will be good to call out the earlier changes mentioned above in these issues so folks are warned against using the existing verify_certificate. * Strangly, a comment in that thread actually recommends _against_ using untrusted: "-untrusted doesn't check whether certificate chain is fully valid. Please consider to pass both intermediate and root to command as -CAfile as other questions suggests. ? Envek Jun 22 at 18:06" -------------- next part -------------- An HTML attachment was scrubbed... URL: From robinfishbein at yahoo.com Tue Aug 23 00:58:04 2016 From: robinfishbein at yahoo.com (Robin Fishbein) Date: Mon, 22 Aug 2016 23:58:04 -0500 Subject: [Cryptography-dev] v1.4: _openssl has no Cryptography_STATIC_CALLBACKS References: Message-ID: Hello, I imagine this may have a trivial explanation, or perhaps indicate some quirk on my machine, but it looks as though some desired functionality isn?t available on my machine with version 1.4. I wanted to share this in case it indicates some wider issue of interest, and helps me learn a little about how cryptography works under the hood. Windows 7, 64-bit, Python 3.5.1, and I believe OpenSSL 1.0.2. I use paramiko for SFTP, and when updating paramiko (on a whim) to 2.0.2 with Anaconda, it had me update cryptography as well, to version 1.4, or maybe it was 1.3 at first ? either one experiences the issue. When attempting an SSH connect, I get a traceback ending with this: File "C:\Users\...\AppData\Local\Continuum\Anaconda3\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py", line 73, in wrapper if lib.Cryptography_STATIC_CALLBACKS: AttributeError: cffi library '_openssl' has no function, constant or global variable named 'Cryptography_STATIC_CALLBACKS' When I manually carry out some of the imports in the source... from cryptography.hazmat.bindings._openssl import ffi, lib ...and then look at what auto-complete shows for lib, I don?t see Cryptography_STATIC_CALLBACKS, which is consistent with the AttributeError. I don?t yet understand where to look for the source underneath the bootstrapped _openssl.*.pyd file. The SSH connection and all subsequent functionality work fine with the latest versions of paramiko and cffi as long as I revert cryptography to 1.0.2. -Robin Fishbein -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Tue Aug 23 09:04:06 2016 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Tue, 23 Aug 2016 09:04:06 -0400 Subject: [Cryptography-dev] v1.4: _openssl has no Cryptography_STATIC_CALLBACKS In-Reply-To: References: Message-ID: This is likely to be something related to anaconda, but I'm not sure how to go about debugging exactly why you're seeing this occur. It *looks* like it's attempting to load an old _openssl.so with a new cryptography. -Paul On August 23, 2016 at 12:58:20 PM, Robin Fishbein via Cryptography-dev ( cryptography-dev at python.org) wrote: Hello, I imagine this may have a trivial explanation, or perhaps indicate some quirk on my machine, but it looks as though some desired functionality isn?t available on my machine with version 1.4. I wanted to share this in case it indicates some wider issue of interest, and helps me learn a little about how cryptography works under the hood. Windows 7, 64-bit, Python 3.5.1, and I believe OpenSSL 1.0.2. I use paramiko for SFTP, and when updating paramiko (on a whim) to 2.0.2 with Anaconda, it had me update cryptography as well, to version 1.4, or maybe it was 1.3 at first ? either one experiences the issue. When attempting an SSH connect, I get a traceback ending with this: File "C:\Users\...\AppData\Local\Continuum\Anaconda3\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py", line 73, in wrapper if lib.Cryptography_STATIC_CALLBACKS: AttributeError: cffi library '_openssl' has no function, constant or global variable named 'Cryptography_STATIC_CALLBACKS' When I manually carry out some of the imports in the source... from cryptography.hazmat.bindings._openssl import ffi, lib ...and then look at what auto-complete shows for lib, I don?t see Cryptography_STATIC_CALLBACKS, which is consistent with the AttributeError. I don?t yet understand where to look for the source underneath the bootstrapped _openssl.*.pyd file. The SSH connection and all subsequent functionality work fine with the latest versions of paramiko and cffi as long as I revert cryptography to 1.0.2. -Robin Fishbein _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Fri Aug 26 12:02:56 2016 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Fri, 26 Aug 2016 09:02:56 -0700 Subject: [Cryptography-dev] pyca/cryptography 1.5 released Message-ID: PyCA cryptography 1.5 has been released to PyPI. cryptography is a package which provides cryptographic recipes and primitives to Python developers. Our goal is for it to be your "cryptographic standard library". We support Python 2.6-2.7, Python 3.3+, and PyPy. Changelog: * Added calculate_max_pss_salt_length for RSA PSS. * Added "one shot" DSAPrivateKey.sign andDSAPublicKey.verify methods to DSA keys. * Added "one shot" EllipticCurvePrivateKey.sign and EllipticCurvePublicKey.verify methods to ECDSA keys. * Switched back to the older callback model on Python 3.5 in order to mitigate a locking callback problem with OpenSSL <1.1.0. * x509.CertificateBuilder, x509.CertificateRevocationListBuilder, and x509.RevokedCertificateBuilder now accept timezone aware datetime objects as method arguments. * cryptography now supports OpenSSL 1.1.0 as a compilation target. As of 1.5 we are still linking 1.0.2h for the Mac and Windows wheels, but in the 1.6 release we will switch to linking 1.1.0. Thanks to all the contributors for their help on this release! -Paul Kehrer (reaperhulk) -------------- next part -------------- An HTML attachment was scrubbed... URL: