From prashanthravindran at gmail.com Wed Dec 12 09:04:44 2018 From: prashanthravindran at gmail.com (Prashanth Ravindran) Date: Wed, 12 Dec 2018 19:34:44 +0530 Subject: [Cryptography-dev] Private Set Intersection Message-ID: Hi, I am trying to implement private set intersection using the cryptography library, and I would like to take the bytes of a private key and do a hash(key_material, message). 1) How do I get the bytes for an instance of cryptography.hazmat.backends.openssl.ec._EllipticCurvePrivateKey 2) Is there any way of encrypting a message using the above private key. Thanks, Prashanth From alex.gaynor at gmail.com Wed Dec 12 17:01:31 2018 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 12 Dec 2018 17:01:31 -0500 Subject: [Cryptography-dev] Private Set Intersection In-Reply-To: References: Message-ID: 1) By get the bytes I assume you mean for the private value? In that case, key.private_numbers().private_value will give you it as an integer, then you can encode it as you like. 2) No, EC private keys do not mathetmatically support encryption. If you want to encrypt with an elliptic curve private key, you need to do implement an ECIES scheme where you perform an ECDH exchange with another key and then encrypt under a key derived from the shared secret. Alex On Wed, Dec 12, 2018 at 4:55 PM Prashanth Ravindran < prashanthravindran at gmail.com> wrote: > Hi, > > I am trying to implement private set intersection using the cryptography > library, and I would like to take the bytes of a private key and do a > hash(key_material, message). > > 1) How do I get the bytes for an instance of > cryptography.hazmat.backends.openssl.ec._EllipticCurvePrivateKey > 2) Is there any way of encrypting a message using the above private key. > > Thanks, > Prashanth > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -- All that is necessary for evil to succeed is for good people to do nothing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Thu Dec 20 00:54:37 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Thu, 20 Dec 2018 00:54:37 -0500 Subject: [Cryptography-dev] Parsing DER from PE File Message-ID: I've asked this question on Stack Overflow here: https://stackoverflow.com/q/53862702/1033217 I have compared my code to Dider Stevens's disitool here (examine the function ExtractDigitalSignature): https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py When I load that extracted file into a variable and try to parse it with cryptography, it fails. If I pipe the same file to openssl on the command line, it works. I am thinking this has to do with the number of certificates in the directory in the PE file. There can be three (cert, intermediate CA, and CA, etc). Any idea what's going on? -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Thu Dec 20 11:12:11 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Thu, 20 Dec 2018 08:12:11 -0800 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: Could you give us an example (in hex or b64 or something) so we can easily reproduce? Make sure any certs you're giving us don't contain sensitive data of course. -Paul On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: I've asked this question on Stack Overflow here: https://stackoverflow.com/q/53862702/1033217 I have compared my code to Dider Stevens's disitool here (examine the function ExtractDigitalSignature): https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py When I load that extracted file into a variable and try to parse it with cryptography, it fails. If I pipe the same file to openssl on the command line, it works. I am thinking this has to do with the number of certificates in the directory in the PE file. There can be three (cert, intermediate CA, and CA, etc). Any idea what's going on? _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Thu Dec 20 15:22:45 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Thu, 20 Dec 2018 15:22:45 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: Definitely. I've attached the DER data as extracted from the PE file using the following code: pe = pefile.PE(fname) pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) sigoff = 0 siglen = 0 for s in pe.__structures__: if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': sigoff = s.VirtualAddress siglen = s.Size pe.close() with open(fname, 'rb') as fh: fh.seek(sigoff) thesig = fh.read(siglen) with open('extracted.der', 'wb') as fh: fh.write(thesig[8:]) I've attached extracted.der as a zip file to maintain integrity as an attachment. Thanks! On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer wrote: > Could you give us an example (in hex or b64 or something) so we can easily > reproduce? Make sure any certs you're giving us don't contain sensitive > data of course. > > -Paul > > > On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > I've asked this question on Stack Overflow here: > https://stackoverflow.com/q/53862702/1033217 > > I have compared my code to Dider Stevens's disitool here (examine the > function ExtractDigitalSignature): > https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py > > When I load that extracted file into a variable and try to parse it with > cryptography, it fails. If I pipe the same file to openssl on the command > line, it works. > > I am thinking this has to do with the number of certificates in the > directory in the PE file. There can be three (cert, intermediate CA, and > CA, etc). > > Any idea what's going on? > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: extracted.zip Type: application/zip Size: 3916 bytes Desc: not available URL: From paul.l.kehrer at gmail.com Fri Dec 21 09:02:13 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Fri, 21 Dec 2018 06:02:13 -0800 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: Thanks, that's perfect. Looking at this data it's actually a PKCS7 envelope holding multiple certificates and at the moment cryptography unfortunately has no interface for parsing PKCS7. If you wouldn't mind sharing your use case directly on https://github.com/pyca/cryptography/issues/3983 then it will help me when I'm prioritizing features for upcoming releases. -Paul On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: Definitely. I've attached the DER data as extracted from the PE file using the following code: pe = pefile.PE(fname) pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) sigoff = 0 siglen = 0 for s in pe.__structures__: if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': sigoff = s.VirtualAddress siglen = s.Size pe.close() with open(fname, 'rb') as fh: fh.seek(sigoff) thesig = fh.read(siglen) with open('extracted.der', 'wb') as fh: fh.write(thesig[8:]) I've attached extracted.der as a zip file to maintain integrity as an attachment. Thanks! On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer wrote: > Could you give us an example (in hex or b64 or something) so we can easily > reproduce? Make sure any certs you're giving us don't contain sensitive > data of course. > > -Paul > > > On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > I've asked this question on Stack Overflow here: > https://stackoverflow.com/q/53862702/1033217 > > I have compared my code to Dider Stevens's disitool here (examine the > function ExtractDigitalSignature): > https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py > > When I load that extracted file into a variable and try to parse it with > cryptography, it fails. If I pipe the same file to openssl on the command > line, it works. > > I am thinking this has to do with the number of certificates in the > directory in the PE file. There can be three (cert, intermediate CA, and > CA, etc). > > Any idea what's going on? > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Fri Dec 21 11:21:41 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Fri, 21 Dec 2018 08:21:41 -0800 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: Out of curiosity, does the following code load the cert you expect? der should be the bytes of extracted.der: from cryptography.hazmat.backends.openssl.backend import backend from cryptography.hazmat.backends.openssl import x509 bio = backend._bytes_to_bio(der) pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) certs = [] for i in range(backend._lib.sk_X509_num(signers)): x509_ptr = backend._lib.sk_X509_value(signers, i) certs.append(x509._Certificate(backend, x509_ptr)) Certs will be a list of signer certificates -- in this case, just one cert in the list. Please note that this code does not manage memory correctly so it should strictly be used to test if the cert you need is being properly extracted :) -Paul (reaperhulk) On December 21, 2018 at 8:02:13 AM, Paul Kehrer (paul.l.kehrer at gmail.com) wrote: Thanks, that's perfect. Looking at this data it's actually a PKCS7 envelope holding multiple certificates and at the moment cryptography unfortunately has no interface for parsing PKCS7. If you wouldn't mind sharing your use case directly on https://github.com/pyca/cryptography/issues/3983 then it will help me when I'm prioritizing features for upcoming releases. -Paul On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: Definitely. I've attached the DER data as extracted from the PE file using the following code: pe = pefile.PE(fname) pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) sigoff = 0 siglen = 0 for s in pe.__structures__: if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': sigoff = s.VirtualAddress siglen = s.Size pe.close() with open(fname, 'rb') as fh: fh.seek(sigoff) thesig = fh.read(siglen) with open('extracted.der', 'wb') as fh: fh.write(thesig[8:]) I've attached extracted.der as a zip file to maintain integrity as an attachment. Thanks! On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer wrote: > Could you give us an example (in hex or b64 or something) so we can easily > reproduce? Make sure any certs you're giving us don't contain sensitive > data of course. > > -Paul > > > On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > I've asked this question on Stack Overflow here: > https://stackoverflow.com/q/53862702/1033217 > > I have compared my code to Dider Stevens's disitool here (examine the > function ExtractDigitalSignature): > https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py > > When I load that extracted file into a variable and try to parse it with > cryptography, it fails. If I pipe the same file to openssl on the command > line, it works. > > I am thinking this has to do with the number of certificates in the > directory in the PE file. There can be three (cert, intermediate CA, and > CA, etc). > > Any idea what's going on? > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Sun Dec 23 04:10:13 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Sun, 23 Dec 2018 04:10:13 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: I've added the use case to the issue as requested. I tried the code snippet, but the contents of signers is missing. What should that be? NameError: name 'signers' is not defined On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer wrote: > Out of curiosity, does the following code load the cert you expect? der > should be the bytes of extracted.der: > > from cryptography.hazmat.backends.openssl.backend import backend > from cryptography.hazmat.backends.openssl import x509 > > bio = backend._bytes_to_bio(der) > pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) > certs = [] > for i in range(backend._lib.sk_X509_num(signers)): > x509_ptr = backend._lib.sk_X509_value(signers, i) > certs.append(x509._Certificate(backend, x509_ptr)) > > Certs will be a list of signer certificates -- in this case, just one cert > in the list. Please note that this code does not manage memory correctly so > it should strictly be used to test if the cert you need is being properly > extracted :) > > -Paul (reaperhulk) > > > On December 21, 2018 at 8:02:13 AM, Paul Kehrer (paul.l.kehrer at gmail.com) > wrote: > > Thanks, that's perfect. Looking at this data it's actually a PKCS7 > envelope holding multiple certificates and at the moment cryptography > unfortunately has no interface for parsing PKCS7. If you wouldn't mind > sharing your use case directly on > https://github.com/pyca/cryptography/issues/3983 then it will help me > when I'm prioritizing features for upcoming releases. > > -Paul > > > On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > Definitely. I've attached the DER data as extracted from the PE file using > the following code: > > pe = pefile.PE(fname) > > pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) > sigoff = 0 > siglen = 0 > for s in pe.__structures__: > if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': > sigoff = s.VirtualAddress > siglen = s.Size > pe.close() > with open(fname, 'rb') as fh: > fh.seek(sigoff) > thesig = fh.read(siglen) > with open('extracted.der', 'wb') as fh: > fh.write(thesig[8:]) > > I've attached extracted.der as a zip file to maintain integrity as an > attachment. > > Thanks! > > On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer > wrote: > >> Could you give us an example (in hex or b64 or something) so we can >> easily reproduce? Make sure any certs you're giving us don't contain >> sensitive data of course. >> >> -Paul >> >> >> On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) >> wrote: >> >> I've asked this question on Stack Overflow here: >> https://stackoverflow.com/q/53862702/1033217 >> >> I have compared my code to Dider Stevens's disitool here (examine the >> function ExtractDigitalSignature): >> >> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >> >> When I load that extracted file into a variable and try to parse it with >> cryptography, it fails. If I pipe the same file to openssl on the command >> line, it works. >> >> I am thinking this has to do with the number of certificates in the >> directory in the PE file. There can be three (cert, intermediate CA, and >> CA, etc). >> >> Any idea what's going on? >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Sun Dec 23 04:17:27 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Sun, 23 Dec 2018 04:17:27 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: import os import pathlib import pefile target = pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') fname = str(target) totsize = os.path.getsize(target) pe = pefile.PE(fname) pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) sigoff = 0 siglen = 0 for s in pe.__structures__: if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': sigoff = s.VirtualAddress siglen = s.Size pe.close() with open(fname, 'rb') as fh: fh.seek(sigoff) thesig = fh.read(siglen) from cryptography.hazmat.backends.openssl.backend import backend from cryptography.hazmat.backends.openssl import x509 bio = backend._bytes_to_bio(thesig[8:]) pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) certs = [] for i in range(backend._lib.sk_X509_num(signers)): x509_ptr = backend._lib.sk_X509_value(signers, i) certs.append(x509._Certificate(backend, x509_ptr)) That's the exact code I'm trying to run with the provided code snippet at the end. If you want to follow along with the exact file I'm working with: hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 The password to that file is "infected" and btw: it is live malware, so please treat it accordingly. Run code on it in a safe environment for handling malware. On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons wrote: > I've added the use case to the issue as requested. I tried the code > snippet, but the contents of signers is missing. What should that be? > > NameError: name 'signers' is not defined > > On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer > wrote: > >> Out of curiosity, does the following code load the cert you expect? der >> should be the bytes of extracted.der: >> >> from cryptography.hazmat.backends.openssl.backend import backend >> from cryptography.hazmat.backends.openssl import x509 >> >> bio = backend._bytes_to_bio(der) >> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >> certs = [] >> for i in range(backend._lib.sk_X509_num(signers)): >> x509_ptr = backend._lib.sk_X509_value(signers, i) >> certs.append(x509._Certificate(backend, x509_ptr)) >> >> Certs will be a list of signer certificates -- in this case, just one >> cert in the list. Please note that this code does not manage memory >> correctly so it should strictly be used to test if the cert you need is >> being properly extracted :) >> >> -Paul (reaperhulk) >> >> >> On December 21, 2018 at 8:02:13 AM, Paul Kehrer (paul.l.kehrer at gmail.com) >> wrote: >> >> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >> envelope holding multiple certificates and at the moment cryptography >> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >> sharing your use case directly on >> https://github.com/pyca/cryptography/issues/3983 then it will help me >> when I'm prioritizing features for upcoming releases. >> >> -Paul >> >> >> On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) >> wrote: >> >> Definitely. I've attached the DER data as extracted from the PE file >> using the following code: >> >> pe = pefile.PE(fname) >> >> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >> sigoff = 0 >> siglen = 0 >> for s in pe.__structures__: >> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >> sigoff = s.VirtualAddress >> siglen = s.Size >> pe.close() >> with open(fname, 'rb') as fh: >> fh.seek(sigoff) >> thesig = fh.read(siglen) >> with open('extracted.der', 'wb') as fh: >> fh.write(thesig[8:]) >> >> I've attached extracted.der as a zip file to maintain integrity as an >> attachment. >> >> Thanks! >> >> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer >> wrote: >> >>> Could you give us an example (in hex or b64 or something) so we can >>> easily reproduce? Make sure any certs you're giving us don't contain >>> sensitive data of course. >>> >>> -Paul >>> >>> >>> On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) >>> wrote: >>> >>> I've asked this question on Stack Overflow here: >>> https://stackoverflow.com/q/53862702/1033217 >>> >>> I have compared my code to Dider Stevens's disitool here (examine the >>> function ExtractDigitalSignature): >>> >>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>> >>> When I load that extracted file into a variable and try to parse it with >>> cryptography, it fails. If I pipe the same file to openssl on the command >>> line, it works. >>> >>> I am thinking this has to do with the number of certificates in the >>> directory in the PE file. There can be three (cert, intermediate CA, and >>> CA, etc). >>> >>> Any idea what's going on? >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Sun Dec 23 19:03:50 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Sun, 23 Dec 2018 17:03:50 -0700 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: One day I will learn to run the code I write before I ask people to use it. The missing signers variable should go after the pkcs7 assignment. It looks like this: signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) With that in place and using the extracted.der you previously provided I can parse a cert, which has the following subject/issuer data: ? ? ? ? Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Code Signing CA ? ? ? ? Validity ? ? ? ? ? ? Not Before: Oct 19 00:00:00 2018 GMT ? ? ? ? ? ? Not After : Sep 25 23:59:59 2019 GMT ? ? ? ? Subject: C=GB/postalCode=WA1 1RG, ST=UK, L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, LIMITED, CN=TATIANA PUK, LIMITED I've also attached the cert. If this is what you're looking for then your use case is covered by the existing issue, although I still need to decide on an API for this. -Paul On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) wrote: import os import pathlib import pefile target = pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') fname = str(target) totsize = os.path.getsize(target) pe = pefile.PE(fname) pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) sigoff = 0 siglen = 0 for s in pe.__structures__: ? ? if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': ? ? ? ? sigoff = s.VirtualAddress ? ? ? ? siglen = s.Size pe.close() with open(fname, 'rb') as fh: ? ? fh.seek(sigoff) ? ? thesig = fh.read(siglen) from cryptography.hazmat.backends.openssl.backend import backend from cryptography.hazmat.backends.openssl import x509 bio = backend._bytes_to_bio(thesig[8:]) pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) certs = [] for i in range(backend._lib.sk_X509_num(signers)): ? ? x509_ptr = backend._lib.sk_X509_value(signers, i) ? ? certs.append(x509._Certificate(backend, x509_ptr)) That's the exact code I'm trying to run with the provided code snippet at the end. If you want to follow along with the exact file I'm working with: hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 The password to that file is "infected" and btw: it is live malware, so please treat it accordingly. Run code on it in a safe environment for handling malware. On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons wrote: I've added the use case to the issue as requested. I tried the code snippet, but the contents of signers is missing. What should that be? NameError: name 'signers' is not defined On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer wrote: Out of curiosity, does the following code load the cert you expect? der should be the bytes of extracted.der: from cryptography.hazmat.backends.openssl.backend import backend from cryptography.hazmat.backends.openssl import x509 bio = backend._bytes_to_bio(der) pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) certs = [] for i in range(backend._lib.sk_X509_num(signers)): ? ? x509_ptr = backend._lib.sk_X509_value(signers, i) ? ? certs.append(x509._Certificate(backend, x509_ptr)) Certs will be a list of signer certificates -- in this case, just one cert in the list. Please note that this code does not manage memory correctly so it should strictly be used to test if the cert you need is being properly extracted :) -Paul (reaperhulk) On December 21, 2018 at 8:02:13 AM, Paul Kehrer (paul.l.kehrer at gmail.com) wrote: Thanks, that's perfect. Looking at this data it's actually a PKCS7 envelope holding multiple certificates and at the moment cryptography unfortunately has no interface for parsing PKCS7. If you wouldn't mind sharing your use case directly on?https://github.com/pyca/cryptography/issues/3983?then it will help me when I'm prioritizing features for upcoming releases. -Paul On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: Definitely. I've attached the DER data as extracted from the PE file using the following code: pe = pefile.PE(fname) pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) sigoff = 0 siglen = 0 for s in pe.__structures__: ? ? if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': ? ? ? ? sigoff = s.VirtualAddress ? ? ? ? siglen = s.Size pe.close() with open(fname, 'rb') as fh: ? ? fh.seek(sigoff) ? ? thesig = fh.read(siglen) with open('extracted.der', 'wb') as fh: ? ? fh.write(thesig[8:]) I've attached extracted.der as a zip file to maintain integrity as an attachment. Thanks! On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer wrote: Could you give us an example (in hex or b64 or something) so we can easily reproduce? Make sure any certs you're giving us don't contain sensitive data of course. -Paul On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: I've asked this question on Stack Overflow here: https://stackoverflow.com/q/53862702/1033217 I have compared my code to Dider Stevens's disitool here (examine the function ExtractDigitalSignature): https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py When I load that extracted file into a variable and try to parse it with cryptography, it fails. If I pipe the same file to openssl on the command line, it works. I am thinking this has to do with the number of certificates in the directory in the PE file. There can be three (cert, intermediate CA, and CA, etc). Any idea what's?going on? _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cert.zip Type: application/octet-stream Size: 1900 bytes Desc: not available URL: From rsimmons0 at gmail.com Sun Dec 23 20:32:00 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Sun, 23 Dec 2018 20:32:00 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: This works great! Thanks! On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer wrote: > One day I will learn to run the code I write before I ask people to use > it. The missing signers variable should go after the pkcs7 assignment. It > looks like this: > > signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) > > With that in place and using the extracted.der you previously provided I > can parse a cert, which has the following subject/issuer data: > > Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA > Limited, CN=COMODO RSA Code Signing CA > Validity > Not Before: Oct 19 00:00:00 2018 GMT > Not After : Sep 25 23:59:59 2019 GMT > Subject: C=GB/postalCode=WA1 1RG, ST=UK, > L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, > LIMITED, CN=TATIANA PUK, LIMITED > > I've also attached the cert. If this is what you're looking for then your > use case is covered by the existing issue, although I still need to decide > on an API for this. > > -Paul > > > > On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > import os > import pathlib > import pefile > > target = > pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') > fname = str(target) > totsize = os.path.getsize(target) > pe = pefile.PE(fname) > > pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) > sigoff = 0 > siglen = 0 > for s in pe.__structures__: > if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': > sigoff = s.VirtualAddress > siglen = s.Size > pe.close() > with open(fname, 'rb') as fh: > fh.seek(sigoff) > thesig = fh.read(siglen) > > from cryptography.hazmat.backends.openssl.backend import backend > from cryptography.hazmat.backends.openssl import x509 > > bio = backend._bytes_to_bio(thesig[8:]) > pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) > certs = [] > for i in range(backend._lib.sk_X509_num(signers)): > x509_ptr = backend._lib.sk_X509_value(signers, i) > certs.append(x509._Certificate(backend, x509_ptr)) > > That's the exact code I'm trying to run with the provided code snippet at > the end. If you want to follow along with the exact file I'm working with: > hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 > > The password to that file is "infected" and btw: it is live malware, so > please treat it accordingly. Run code on it in a safe environment for > handling malware. > > On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons > wrote: > >> I've added the use case to the issue as requested. I tried the code >> snippet, but the contents of signers is missing. What should that be? >> >> NameError: name 'signers' is not defined >> >> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer >> wrote: >> >>> Out of curiosity, does the following code load the cert you expect? der >>> should be the bytes of extracted.der: >>> >>> from cryptography.hazmat.backends.openssl.backend import backend >>> from cryptography.hazmat.backends.openssl import x509 >>> >>> bio = backend._bytes_to_bio(der) >>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>> certs = [] >>> for i in range(backend._lib.sk_X509_num(signers)): >>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>> certs.append(x509._Certificate(backend, x509_ptr)) >>> >>> Certs will be a list of signer certificates -- in this case, just one >>> cert in the list. Please note that this code does not manage memory >>> correctly so it should strictly be used to test if the cert you need is >>> being properly extracted :) >>> >>> -Paul (reaperhulk) >>> >>> >>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer (paul.l.kehrer at gmail.com) >>> wrote: >>> >>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >>> envelope holding multiple certificates and at the moment cryptography >>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >>> sharing your use case directly on >>> https://github.com/pyca/cryptography/issues/3983 then it will help me >>> when I'm prioritizing features for upcoming releases. >>> >>> -Paul >>> >>> >>> On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) >>> wrote: >>> >>> Definitely. I've attached the DER data as extracted from the PE file >>> using the following code: >>> >>> pe = pefile.PE(fname) >>> >>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>> sigoff = 0 >>> siglen = 0 >>> for s in pe.__structures__: >>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>> sigoff = s.VirtualAddress >>> siglen = s.Size >>> pe.close() >>> with open(fname, 'rb') as fh: >>> fh.seek(sigoff) >>> thesig = fh.read(siglen) >>> with open('extracted.der', 'wb') as fh: >>> fh.write(thesig[8:]) >>> >>> I've attached extracted.der as a zip file to maintain integrity as an >>> attachment. >>> >>> Thanks! >>> >>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer >>> wrote: >>> >>>> Could you give us an example (in hex or b64 or something) so we can >>>> easily reproduce? Make sure any certs you're giving us don't contain >>>> sensitive data of course. >>>> >>>> -Paul >>>> >>>> >>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons ( >>>> rsimmons0 at gmail.com) wrote: >>>> >>>> I've asked this question on Stack Overflow here: >>>> https://stackoverflow.com/q/53862702/1033217 >>>> >>>> I have compared my code to Dider Stevens's disitool here (examine the >>>> function ExtractDigitalSignature): >>>> >>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>> >>>> When I load that extracted file into a variable and try to parse it >>>> with cryptography, it fails. If I pipe the same file to openssl on the >>>> command line, it works. >>>> >>>> I am thinking this has to do with the number of certificates in the >>>> directory in the PE file. There can be three (cert, intermediate CA, and >>>> CA, etc). >>>> >>>> Any idea what's going on? >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Mon Dec 24 11:50:53 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Mon, 24 Dec 2018 09:50:53 -0700 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: Message-ID: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Great! I have an idea of how to implement an API for this limited subset of pkcs7 as a utility function like the pkcs12 support we recently merged. Hopefully I or someone else can get to it soon. -Paul > On Dec 23, 2018, at 6:32 PM, Robert Simmons wrote: > > This works great! Thanks! > >> On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer wrote: >> One day I will learn to run the code I write before I ask people to use it. The missing signers variable should go after the pkcs7 assignment. It looks like this: >> >> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) >> >> With that in place and using the extracted.der you previously provided I can parse a cert, which has the following subject/issuer data: >> >> Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Code Signing CA >> Validity >> Not Before: Oct 19 00:00:00 2018 GMT >> Not After : Sep 25 23:59:59 2019 GMT >> Subject: C=GB/postalCode=WA1 1RG, ST=UK, L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, LIMITED, CN=TATIANA PUK, LIMITED >> >> I've also attached the cert. If this is what you're looking for then your use case is covered by the existing issue, although I still need to decide on an API for this. >> >> -Paul >> >> >> >>> On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) wrote: >>> >>> import os >>> import pathlib >>> import pefile >>> >>> target = pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') >>> fname = str(target) >>> totsize = os.path.getsize(target) >>> pe = pefile.PE(fname) >>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>> sigoff = 0 >>> siglen = 0 >>> for s in pe.__structures__: >>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>> sigoff = s.VirtualAddress >>> siglen = s.Size >>> pe.close() >>> with open(fname, 'rb') as fh: >>> fh.seek(sigoff) >>> thesig = fh.read(siglen) >>> >>> from cryptography.hazmat.backends.openssl.backend import backend >>> from cryptography.hazmat.backends.openssl import x509 >>> >>> bio = backend._bytes_to_bio(thesig[8:]) >>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>> certs = [] >>> for i in range(backend._lib.sk_X509_num(signers)): >>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>> certs.append(x509._Certificate(backend, x509_ptr)) >>> >>> That's the exact code I'm trying to run with the provided code snippet at the end. If you want to follow along with the exact file I'm working with: >>> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 >>> >>> The password to that file is "infected" and btw: it is live malware, so please treat it accordingly. Run code on it in a safe environment for handling malware. >>> >>>> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons wrote: >>>> I've added the use case to the issue as requested. I tried the code snippet, but the contents of signers is missing. What should that be? >>>> >>>> NameError: name 'signers' is not defined >>>> >>>>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer wrote: >>>>> Out of curiosity, does the following code load the cert you expect? der should be the bytes of extracted.der: >>>>> >>>>> from cryptography.hazmat.backends.openssl.backend import backend >>>>> from cryptography.hazmat.backends.openssl import x509 >>>>> >>>>> bio = backend._bytes_to_bio(der) >>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>>> certs = [] >>>>> for i in range(backend._lib.sk_X509_num(signers)): >>>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>>> >>>>> Certs will be a list of signer certificates -- in this case, just one cert in the list. Please note that this code does not manage memory correctly so it should strictly be used to test if the cert you need is being properly extracted :) >>>>> >>>>> -Paul (reaperhulk) >>>>> >>>>> >>>>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer (paul.l.kehrer at gmail.com) wrote: >>>>>> >>>>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 envelope holding multiple certificates and at the moment cryptography unfortunately has no interface for parsing PKCS7. If you wouldn't mind sharing your use case directly on https://github.com/pyca/cryptography/issues/3983 then it will help me when I'm prioritizing features for upcoming releases. >>>>>> >>>>>> -Paul >>>>>> >>>>>> >>>>>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: >>>>>>> >>>>>>> Definitely. I've attached the DER data as extracted from the PE file using the following code: >>>>>>> >>>>>>> pe = pefile.PE(fname) >>>>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>>>>> sigoff = 0 >>>>>>> siglen = 0 >>>>>>> for s in pe.__structures__: >>>>>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>>>>> sigoff = s.VirtualAddress >>>>>>> siglen = s.Size >>>>>>> pe.close() >>>>>>> with open(fname, 'rb') as fh: >>>>>>> fh.seek(sigoff) >>>>>>> thesig = fh.read(siglen) >>>>>>> with open('extracted.der', 'wb') as fh: >>>>>>> fh.write(thesig[8:]) >>>>>>> >>>>>>> I've attached extracted.der as a zip file to maintain integrity as an attachment. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer wrote: >>>>>>>> Could you give us an example (in hex or b64 or something) so we can easily reproduce? Make sure any certs you're giving us don't contain sensitive data of course. >>>>>>>> >>>>>>>> -Paul >>>>>>>> >>>>>>>> >>>>>>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: >>>>>>>>> >>>>>>>>> I've asked this question on Stack Overflow here: >>>>>>>>> https://stackoverflow.com/q/53862702/1033217 >>>>>>>>> >>>>>>>>> I have compared my code to Dider Stevens's disitool here (examine the function ExtractDigitalSignature): >>>>>>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>>>>>>> >>>>>>>>> When I load that extracted file into a variable and try to parse it with cryptography, it fails. If I pipe the same file to openssl on the command line, it works. >>>>>>>>> >>>>>>>>> I am thinking this has to do with the number of certificates in the directory in the PE file. There can be three (cert, intermediate CA, and CA, etc). >>>>>>>>> >>>>>>>>> Any idea what's going on? >>>>>>>>> _______________________________________________ >>>>>>>>> Cryptography-dev mailing list >>>>>>>>> Cryptography-dev at python.org >>>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>>> _______________________________________________ >>>>>>>> Cryptography-dev mailing list >>>>>>>> Cryptography-dev at python.org >>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>> _______________________________________________ >>>>>>> Cryptography-dev mailing list >>>>>>> Cryptography-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Tue Dec 25 21:41:12 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Tue, 25 Dec 2018 21:41:12 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: Thanks for the help above. However, I think I'm still missing something. When piping the DER binary data to openssl on the command line, the output appears to have three certificates in the example DER early in this thread. The code above has a list for certs, but it appears to only contain one cert at the end of the for loop. Is there a way to view the data from the other two? I've attached the output from openssl command line. On Mon, Dec 24, 2018 at 11:51 AM Paul Kehrer wrote: > Great! I have an idea of how to implement an API for this limited subset > of pkcs7 as a utility function like the pkcs12 support we recently merged. > Hopefully I or someone else can get to it soon. > > -Paul > > On Dec 23, 2018, at 6:32 PM, Robert Simmons wrote: > > This works great! Thanks! > > On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer > wrote: > >> One day I will learn to run the code I write before I ask people to use >> it. The missing signers variable should go after the pkcs7 assignment. It >> looks like this: >> >> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) >> >> With that in place and using the extracted.der you previously provided I >> can parse a cert, which has the following subject/issuer data: >> >> Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA >> Limited, CN=COMODO RSA Code Signing CA >> Validity >> Not Before: Oct 19 00:00:00 2018 GMT >> Not After : Sep 25 23:59:59 2019 GMT >> Subject: C=GB/postalCode=WA1 1RG, ST=UK, >> L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, >> LIMITED, CN=TATIANA PUK, LIMITED >> >> I've also attached the cert. If this is what you're looking for then your >> use case is covered by the existing issue, although I still need to decide >> on an API for this. >> >> -Paul >> >> >> >> On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) >> wrote: >> >> import os >> import pathlib >> import pefile >> >> target = >> pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') >> fname = str(target) >> totsize = os.path.getsize(target) >> pe = pefile.PE(fname) >> >> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >> sigoff = 0 >> siglen = 0 >> for s in pe.__structures__: >> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >> sigoff = s.VirtualAddress >> siglen = s.Size >> pe.close() >> with open(fname, 'rb') as fh: >> fh.seek(sigoff) >> thesig = fh.read(siglen) >> >> from cryptography.hazmat.backends.openssl.backend import backend >> from cryptography.hazmat.backends.openssl import x509 >> >> bio = backend._bytes_to_bio(thesig[8:]) >> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >> certs = [] >> for i in range(backend._lib.sk_X509_num(signers)): >> x509_ptr = backend._lib.sk_X509_value(signers, i) >> certs.append(x509._Certificate(backend, x509_ptr)) >> >> That's the exact code I'm trying to run with the provided code snippet at >> the end. If you want to follow along with the exact file I'm working with: >> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 >> >> The password to that file is "infected" and btw: it is live malware, so >> please treat it accordingly. Run code on it in a safe environment for >> handling malware. >> >> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons >> wrote: >> >>> I've added the use case to the issue as requested. I tried the code >>> snippet, but the contents of signers is missing. What should that be? >>> >>> NameError: name 'signers' is not defined >>> >>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer >>> wrote: >>> >>>> Out of curiosity, does the following code load the cert you expect? der >>>> should be the bytes of extracted.der: >>>> >>>> from cryptography.hazmat.backends.openssl.backend import backend >>>> from cryptography.hazmat.backends.openssl import x509 >>>> >>>> bio = backend._bytes_to_bio(der) >>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>> certs = [] >>>> for i in range(backend._lib.sk_X509_num(signers)): >>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>> >>>> Certs will be a list of signer certificates -- in this case, just one >>>> cert in the list. Please note that this code does not manage memory >>>> correctly so it should strictly be used to test if the cert you need is >>>> being properly extracted :) >>>> >>>> -Paul (reaperhulk) >>>> >>>> >>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer ( >>>> paul.l.kehrer at gmail.com) wrote: >>>> >>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >>>> envelope holding multiple certificates and at the moment cryptography >>>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >>>> sharing your use case directly on >>>> https://github.com/pyca/cryptography/issues/3983 then it will help me >>>> when I'm prioritizing features for upcoming releases. >>>> >>>> -Paul >>>> >>>> >>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons (rsimmons0 at gmail.com) >>>> wrote: >>>> >>>> Definitely. I've attached the DER data as extracted from the PE file >>>> using the following code: >>>> >>>> pe = pefile.PE(fname) >>>> >>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>> sigoff = 0 >>>> siglen = 0 >>>> for s in pe.__structures__: >>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>> sigoff = s.VirtualAddress >>>> siglen = s.Size >>>> pe.close() >>>> with open(fname, 'rb') as fh: >>>> fh.seek(sigoff) >>>> thesig = fh.read(siglen) >>>> with open('extracted.der', 'wb') as fh: >>>> fh.write(thesig[8:]) >>>> >>>> I've attached extracted.der as a zip file to maintain integrity as an >>>> attachment. >>>> >>>> Thanks! >>>> >>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer >>>> wrote: >>>> >>>>> Could you give us an example (in hex or b64 or something) so we can >>>>> easily reproduce? Make sure any certs you're giving us don't contain >>>>> sensitive data of course. >>>>> >>>>> -Paul >>>>> >>>>> >>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons ( >>>>> rsimmons0 at gmail.com) wrote: >>>>> >>>>> I've asked this question on Stack Overflow here: >>>>> https://stackoverflow.com/q/53862702/1033217 >>>>> >>>>> I have compared my code to Dider Stevens's disitool here (examine the >>>>> function ExtractDigitalSignature): >>>>> >>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>>> >>>>> When I load that extracted file into a variable and try to parse it >>>>> with cryptography, it fails. If I pipe the same file to openssl on the >>>>> command line, it works. >>>>> >>>>> I am thinking this has to do with the number of certificates in the >>>>> directory in the PE file. There can be three (cert, intermediate CA, and >>>>> CA, etc). >>>>> >>>>> Any idea what's going on? >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cert Type: application/octet-stream Size: 19920 bytes Desc: not available URL: From rsimmons0 at gmail.com Wed Dec 26 00:14:43 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Wed, 26 Dec 2018 00:14:43 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: On a side note: there is one oid in the extensions of this cert that is listed as unknown, but openssl parses it as: Netscape Cert Type: Object Signing Is this something to submit a bug for? Also, happy holidays! On Tue, Dec 25, 2018 at 9:41 PM Robert Simmons wrote: > Thanks for the help above. However, I think I'm still missing something. > When piping the DER binary data to openssl on the command line, the output > appears to have three certificates in the example DER early in this thread. > The code above has a list for certs, but it appears to only contain one > cert at the end of the for loop. Is there a way to view the data from the > other two? I've attached the output from openssl command line. > > On Mon, Dec 24, 2018 at 11:51 AM Paul Kehrer > wrote: > >> Great! I have an idea of how to implement an API for this limited subset >> of pkcs7 as a utility function like the pkcs12 support we recently merged. >> Hopefully I or someone else can get to it soon. >> >> -Paul >> >> On Dec 23, 2018, at 6:32 PM, Robert Simmons wrote: >> >> This works great! Thanks! >> >> On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer >> wrote: >> >>> One day I will learn to run the code I write before I ask people to use >>> it. The missing signers variable should go after the pkcs7 assignment. It >>> looks like this: >>> >>> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) >>> >>> With that in place and using the extracted.der you previously provided I >>> can parse a cert, which has the following subject/issuer data: >>> >>> Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA >>> Limited, CN=COMODO RSA Code Signing CA >>> Validity >>> Not Before: Oct 19 00:00:00 2018 GMT >>> Not After : Sep 25 23:59:59 2019 GMT >>> Subject: C=GB/postalCode=WA1 1RG, ST=UK, >>> L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, >>> LIMITED, CN=TATIANA PUK, LIMITED >>> >>> I've also attached the cert. If this is what you're looking for then >>> your use case is covered by the existing issue, although I still need to >>> decide on an API for this. >>> >>> -Paul >>> >>> >>> >>> On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) >>> wrote: >>> >>> import os >>> import pathlib >>> import pefile >>> >>> target = >>> pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') >>> fname = str(target) >>> totsize = os.path.getsize(target) >>> pe = pefile.PE(fname) >>> >>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>> sigoff = 0 >>> siglen = 0 >>> for s in pe.__structures__: >>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>> sigoff = s.VirtualAddress >>> siglen = s.Size >>> pe.close() >>> with open(fname, 'rb') as fh: >>> fh.seek(sigoff) >>> thesig = fh.read(siglen) >>> >>> from cryptography.hazmat.backends.openssl.backend import backend >>> from cryptography.hazmat.backends.openssl import x509 >>> >>> bio = backend._bytes_to_bio(thesig[8:]) >>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>> certs = [] >>> for i in range(backend._lib.sk_X509_num(signers)): >>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>> certs.append(x509._Certificate(backend, x509_ptr)) >>> >>> That's the exact code I'm trying to run with the provided code snippet >>> at the end. If you want to follow along with the exact file I'm working >>> with: >>> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 >>> >>> The password to that file is "infected" and btw: it is live malware, so >>> please treat it accordingly. Run code on it in a safe environment for >>> handling malware. >>> >>> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons >>> wrote: >>> >>>> I've added the use case to the issue as requested. I tried the code >>>> snippet, but the contents of signers is missing. What should that be? >>>> >>>> NameError: name 'signers' is not defined >>>> >>>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer >>>> wrote: >>>> >>>>> Out of curiosity, does the following code load the cert you expect? >>>>> der should be the bytes of extracted.der: >>>>> >>>>> from cryptography.hazmat.backends.openssl.backend import backend >>>>> from cryptography.hazmat.backends.openssl import x509 >>>>> >>>>> bio = backend._bytes_to_bio(der) >>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>>> certs = [] >>>>> for i in range(backend._lib.sk_X509_num(signers)): >>>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>>> >>>>> Certs will be a list of signer certificates -- in this case, just one >>>>> cert in the list. Please note that this code does not manage memory >>>>> correctly so it should strictly be used to test if the cert you need is >>>>> being properly extracted :) >>>>> >>>>> -Paul (reaperhulk) >>>>> >>>>> >>>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer ( >>>>> paul.l.kehrer at gmail.com) wrote: >>>>> >>>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >>>>> envelope holding multiple certificates and at the moment cryptography >>>>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >>>>> sharing your use case directly on >>>>> https://github.com/pyca/cryptography/issues/3983 then it will help me >>>>> when I'm prioritizing features for upcoming releases. >>>>> >>>>> -Paul >>>>> >>>>> >>>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons ( >>>>> rsimmons0 at gmail.com) wrote: >>>>> >>>>> Definitely. I've attached the DER data as extracted from the PE file >>>>> using the following code: >>>>> >>>>> pe = pefile.PE(fname) >>>>> >>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>>> sigoff = 0 >>>>> siglen = 0 >>>>> for s in pe.__structures__: >>>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>>> sigoff = s.VirtualAddress >>>>> siglen = s.Size >>>>> pe.close() >>>>> with open(fname, 'rb') as fh: >>>>> fh.seek(sigoff) >>>>> thesig = fh.read(siglen) >>>>> with open('extracted.der', 'wb') as fh: >>>>> fh.write(thesig[8:]) >>>>> >>>>> I've attached extracted.der as a zip file to maintain integrity as an >>>>> attachment. >>>>> >>>>> Thanks! >>>>> >>>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer >>>>> wrote: >>>>> >>>>>> Could you give us an example (in hex or b64 or something) so we can >>>>>> easily reproduce? Make sure any certs you're giving us don't contain >>>>>> sensitive data of course. >>>>>> >>>>>> -Paul >>>>>> >>>>>> >>>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons ( >>>>>> rsimmons0 at gmail.com) wrote: >>>>>> >>>>>> I've asked this question on Stack Overflow here: >>>>>> https://stackoverflow.com/q/53862702/1033217 >>>>>> >>>>>> I have compared my code to Dider Stevens's disitool here (examine the >>>>>> function ExtractDigitalSignature): >>>>>> >>>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>>>> >>>>>> When I load that extracted file into a variable and try to parse it >>>>>> with cryptography, it fails. If I pipe the same file to openssl on the >>>>>> command line, it works. >>>>>> >>>>>> I am thinking this has to do with the number of certificates in the >>>>>> directory in the PE file. There can be three (cert, intermediate CA, and >>>>>> CA, etc). >>>>>> >>>>>> Any idea what's going on? >>>>>> _______________________________________________ >>>>>> Cryptography-dev mailing list >>>>>> Cryptography-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>> >>>>>> _______________________________________________ >>>>>> Cryptography-dev mailing list >>>>>> Cryptography-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>> >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Wed Dec 26 11:26:53 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Wed, 26 Dec 2018 08:26:53 -0800 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: We haven't had anyone request support for those legacy extension types, but if you think you need it feel free to file an issue and we can discuss adding it. The data can be parsed out of the UnknownExtension type right now of course. So you need all 3 certs? Only one of them is used for signing which is why the PKCS7_get0_signers call only returns that one. To obtain the rest we'll need to de-opaque two PKCS7 structs in cryptography: PKCS7_ENVELOPE and PKCS7_SIGN_ENVELOPE. Your use case should only require SIGN_ENVELOPE de-opaqued but might as well get them both. -Paul On December 25, 2018 at 9:15:10 PM, Robert Simmons (rsimmons0 at gmail.com) wrote: On a side note: there is one oid in the extensions of this cert that is listed as unknown, but openssl parses it as: Netscape Cert Type: Object Signing Is this something to submit a bug for? Also, happy holidays! On Tue, Dec 25, 2018 at 9:41 PM Robert Simmons wrote: > Thanks for the help above. However, I think I'm still missing something. > When piping the DER binary data to openssl on the command line, the output > appears to have three certificates in the example DER early in this thread. > The code above has a list for certs, but it appears to only contain one > cert at the end of the for loop. Is there a way to view the data from the > other two? I've attached the output from openssl command line. > > On Mon, Dec 24, 2018 at 11:51 AM Paul Kehrer > wrote: > >> Great! I have an idea of how to implement an API for this limited subset >> of pkcs7 as a utility function like the pkcs12 support we recently merged. >> Hopefully I or someone else can get to it soon. >> >> -Paul >> >> On Dec 23, 2018, at 6:32 PM, Robert Simmons wrote: >> >> This works great! Thanks! >> >> On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer >> wrote: >> >>> One day I will learn to run the code I write before I ask people to use >>> it. The missing signers variable should go after the pkcs7 assignment. It >>> looks like this: >>> >>> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) >>> >>> With that in place and using the extracted.der you previously provided I >>> can parse a cert, which has the following subject/issuer data: >>> >>> Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA >>> Limited, CN=COMODO RSA Code Signing CA >>> Validity >>> Not Before: Oct 19 00:00:00 2018 GMT >>> Not After : Sep 25 23:59:59 2019 GMT >>> Subject: C=GB/postalCode=WA1 1RG, ST=UK, >>> L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, >>> LIMITED, CN=TATIANA PUK, LIMITED >>> >>> I've also attached the cert. If this is what you're looking for then >>> your use case is covered by the existing issue, although I still need to >>> decide on an API for this. >>> >>> -Paul >>> >>> >>> >>> On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) >>> wrote: >>> >>> import os >>> import pathlib >>> import pefile >>> >>> target = >>> pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') >>> fname = str(target) >>> totsize = os.path.getsize(target) >>> pe = pefile.PE(fname) >>> >>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>> sigoff = 0 >>> siglen = 0 >>> for s in pe.__structures__: >>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>> sigoff = s.VirtualAddress >>> siglen = s.Size >>> pe.close() >>> with open(fname, 'rb') as fh: >>> fh.seek(sigoff) >>> thesig = fh.read(siglen) >>> >>> from cryptography.hazmat.backends.openssl.backend import backend >>> from cryptography.hazmat.backends.openssl import x509 >>> >>> bio = backend._bytes_to_bio(thesig[8:]) >>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>> certs = [] >>> for i in range(backend._lib.sk_X509_num(signers)): >>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>> certs.append(x509._Certificate(backend, x509_ptr)) >>> >>> That's the exact code I'm trying to run with the provided code snippet >>> at the end. If you want to follow along with the exact file I'm working >>> with: >>> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 >>> >>> The password to that file is "infected" and btw: it is live malware, so >>> please treat it accordingly. Run code on it in a safe environment for >>> handling malware. >>> >>> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons >>> wrote: >>> >>>> I've added the use case to the issue as requested. I tried the code >>>> snippet, but the contents of signers is missing. What should that be? >>>> >>>> NameError: name 'signers' is not defined >>>> >>>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer >>>> wrote: >>>> >>>>> Out of curiosity, does the following code load the cert you expect? >>>>> der should be the bytes of extracted.der: >>>>> >>>>> from cryptography.hazmat.backends.openssl.backend import backend >>>>> from cryptography.hazmat.backends.openssl import x509 >>>>> >>>>> bio = backend._bytes_to_bio(der) >>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>>> certs = [] >>>>> for i in range(backend._lib.sk_X509_num(signers)): >>>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>>> >>>>> Certs will be a list of signer certificates -- in this case, just one >>>>> cert in the list. Please note that this code does not manage memory >>>>> correctly so it should strictly be used to test if the cert you need is >>>>> being properly extracted :) >>>>> >>>>> -Paul (reaperhulk) >>>>> >>>>> >>>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer ( >>>>> paul.l.kehrer at gmail.com) wrote: >>>>> >>>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >>>>> envelope holding multiple certificates and at the moment cryptography >>>>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >>>>> sharing your use case directly on >>>>> https://github.com/pyca/cryptography/issues/3983 then it will help me >>>>> when I'm prioritizing features for upcoming releases. >>>>> >>>>> -Paul >>>>> >>>>> >>>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons ( >>>>> rsimmons0 at gmail.com) wrote: >>>>> >>>>> Definitely. I've attached the DER data as extracted from the PE file >>>>> using the following code: >>>>> >>>>> pe = pefile.PE(fname) >>>>> >>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>>> sigoff = 0 >>>>> siglen = 0 >>>>> for s in pe.__structures__: >>>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>>> sigoff = s.VirtualAddress >>>>> siglen = s.Size >>>>> pe.close() >>>>> with open(fname, 'rb') as fh: >>>>> fh.seek(sigoff) >>>>> thesig = fh.read(siglen) >>>>> with open('extracted.der', 'wb') as fh: >>>>> fh.write(thesig[8:]) >>>>> >>>>> I've attached extracted.der as a zip file to maintain integrity as an >>>>> attachment. >>>>> >>>>> Thanks! >>>>> >>>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer >>>>> wrote: >>>>> >>>>>> Could you give us an example (in hex or b64 or something) so we can >>>>>> easily reproduce? Make sure any certs you're giving us don't contain >>>>>> sensitive data of course. >>>>>> >>>>>> -Paul >>>>>> >>>>>> >>>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons ( >>>>>> rsimmons0 at gmail.com) wrote: >>>>>> >>>>>> I've asked this question on Stack Overflow here: >>>>>> https://stackoverflow.com/q/53862702/1033217 >>>>>> >>>>>> I have compared my code to Dider Stevens's disitool here (examine the >>>>>> function ExtractDigitalSignature): >>>>>> >>>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>>>> >>>>>> When I load that extracted file into a variable and try to parse it >>>>>> with cryptography, it fails. If I pipe the same file to openssl on the >>>>>> command line, it works. >>>>>> >>>>>> I am thinking this has to do with the number of certificates in the >>>>>> directory in the PE file. There can be three (cert, intermediate CA, and >>>>>> CA, etc). >>>>>> >>>>>> Any idea what's going on? >>>>>> _______________________________________________ >>>>>> Cryptography-dev mailing list >>>>>> Cryptography-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>> >>>>>> _______________________________________________ >>>>>> Cryptography-dev mailing list >>>>>> Cryptography-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>> >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Sat Dec 29 00:32:24 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Sat, 29 Dec 2018 00:32:24 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: I do need all three certs. I do see what PKCS7_get0_signers does now. On Wed, Dec 26, 2018 at 11:27 AM Paul Kehrer wrote: > We haven't had anyone request support for those legacy extension types, > but if you think you need it feel free to file an issue and we can discuss > adding it. The data can be parsed out of the UnknownExtension type right > now of course. > > So you need all 3 certs? Only one of them is used for signing which is why > the PKCS7_get0_signers call only returns that one. To obtain the rest we'll > need to de-opaque two PKCS7 structs in cryptography: PKCS7_ENVELOPE > and PKCS7_SIGN_ENVELOPE. Your use case should only require SIGN_ENVELOPE > de-opaqued but might as well get them both. > > -Paul > > > > On December 25, 2018 at 9:15:10 PM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > On a side note: there is one oid in the extensions of this cert that is > listed as unknown, but openssl parses it as: > Netscape Cert Type: > Object Signing > > Is this something to submit a bug for? > > Also, happy holidays! > > On Tue, Dec 25, 2018 at 9:41 PM Robert Simmons > wrote: > >> Thanks for the help above. However, I think I'm still missing something. >> When piping the DER binary data to openssl on the command line, the output >> appears to have three certificates in the example DER early in this thread. >> The code above has a list for certs, but it appears to only contain one >> cert at the end of the for loop. Is there a way to view the data from the >> other two? I've attached the output from openssl command line. >> >> On Mon, Dec 24, 2018 at 11:51 AM Paul Kehrer >> wrote: >> >>> Great! I have an idea of how to implement an API for this limited subset >>> of pkcs7 as a utility function like the pkcs12 support we recently merged. >>> Hopefully I or someone else can get to it soon. >>> >>> -Paul >>> >>> On Dec 23, 2018, at 6:32 PM, Robert Simmons wrote: >>> >>> This works great! Thanks! >>> >>> On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer >>> wrote: >>> >>>> One day I will learn to run the code I write before I ask people to use >>>> it. The missing signers variable should go after the pkcs7 assignment. It >>>> looks like this: >>>> >>>> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) >>>> >>>> With that in place and using the extracted.der you previously provided >>>> I can parse a cert, which has the following subject/issuer data: >>>> >>>> Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA >>>> Limited, CN=COMODO RSA Code Signing CA >>>> Validity >>>> Not Before: Oct 19 00:00:00 2018 GMT >>>> Not After : Sep 25 23:59:59 2019 GMT >>>> Subject: C=GB/postalCode=WA1 1RG, ST=UK, >>>> L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, >>>> LIMITED, CN=TATIANA PUK, LIMITED >>>> >>>> I've also attached the cert. If this is what you're looking for then >>>> your use case is covered by the existing issue, although I still need to >>>> decide on an API for this. >>>> >>>> -Paul >>>> >>>> >>>> >>>> On December 23, 2018 at 2:17:54 AM, Robert Simmons (rsimmons0 at gmail.com) >>>> wrote: >>>> >>>> import os >>>> import pathlib >>>> import pefile >>>> >>>> target = >>>> pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') >>>> fname = str(target) >>>> totsize = os.path.getsize(target) >>>> pe = pefile.PE(fname) >>>> >>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>> sigoff = 0 >>>> siglen = 0 >>>> for s in pe.__structures__: >>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>> sigoff = s.VirtualAddress >>>> siglen = s.Size >>>> pe.close() >>>> with open(fname, 'rb') as fh: >>>> fh.seek(sigoff) >>>> thesig = fh.read(siglen) >>>> >>>> from cryptography.hazmat.backends.openssl.backend import backend >>>> from cryptography.hazmat.backends.openssl import x509 >>>> >>>> bio = backend._bytes_to_bio(thesig[8:]) >>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>> certs = [] >>>> for i in range(backend._lib.sk_X509_num(signers)): >>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>> >>>> That's the exact code I'm trying to run with the provided code snippet >>>> at the end. If you want to follow along with the exact file I'm working >>>> with: >>>> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 >>>> >>>> The password to that file is "infected" and btw: it is live malware, so >>>> please treat it accordingly. Run code on it in a safe environment for >>>> handling malware. >>>> >>>> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons >>>> wrote: >>>> >>>>> I've added the use case to the issue as requested. I tried the code >>>>> snippet, but the contents of signers is missing. What should that be? >>>>> >>>>> NameError: name 'signers' is not defined >>>>> >>>>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer >>>>> wrote: >>>>> >>>>>> Out of curiosity, does the following code load the cert you expect? >>>>>> der should be the bytes of extracted.der: >>>>>> >>>>>> from cryptography.hazmat.backends.openssl.backend import backend >>>>>> from cryptography.hazmat.backends.openssl import x509 >>>>>> >>>>>> bio = backend._bytes_to_bio(der) >>>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>>>> certs = [] >>>>>> for i in range(backend._lib.sk_X509_num(signers)): >>>>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>>>> >>>>>> Certs will be a list of signer certificates -- in this case, just one >>>>>> cert in the list. Please note that this code does not manage memory >>>>>> correctly so it should strictly be used to test if the cert you need is >>>>>> being properly extracted :) >>>>>> >>>>>> -Paul (reaperhulk) >>>>>> >>>>>> >>>>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer ( >>>>>> paul.l.kehrer at gmail.com) wrote: >>>>>> >>>>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >>>>>> envelope holding multiple certificates and at the moment cryptography >>>>>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >>>>>> sharing your use case directly on >>>>>> https://github.com/pyca/cryptography/issues/3983 then it will help >>>>>> me when I'm prioritizing features for upcoming releases. >>>>>> >>>>>> -Paul >>>>>> >>>>>> >>>>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons ( >>>>>> rsimmons0 at gmail.com) wrote: >>>>>> >>>>>> Definitely. I've attached the DER data as extracted from the PE file >>>>>> using the following code: >>>>>> >>>>>> pe = pefile.PE(fname) >>>>>> >>>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>>>> sigoff = 0 >>>>>> siglen = 0 >>>>>> for s in pe.__structures__: >>>>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>>>> sigoff = s.VirtualAddress >>>>>> siglen = s.Size >>>>>> pe.close() >>>>>> with open(fname, 'rb') as fh: >>>>>> fh.seek(sigoff) >>>>>> thesig = fh.read(siglen) >>>>>> with open('extracted.der', 'wb') as fh: >>>>>> fh.write(thesig[8:]) >>>>>> >>>>>> I've attached extracted.der as a zip file to maintain integrity as an >>>>>> attachment. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer >>>>>> wrote: >>>>>> >>>>>>> Could you give us an example (in hex or b64 or something) so we can >>>>>>> easily reproduce? Make sure any certs you're giving us don't contain >>>>>>> sensitive data of course. >>>>>>> >>>>>>> -Paul >>>>>>> >>>>>>> >>>>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons ( >>>>>>> rsimmons0 at gmail.com) wrote: >>>>>>> >>>>>>> I've asked this question on Stack Overflow here: >>>>>>> https://stackoverflow.com/q/53862702/1033217 >>>>>>> >>>>>>> I have compared my code to Dider Stevens's disitool here (examine >>>>>>> the function ExtractDigitalSignature): >>>>>>> >>>>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>>>>> >>>>>>> When I load that extracted file into a variable and try to parse it >>>>>>> with cryptography, it fails. If I pipe the same file to openssl on the >>>>>>> command line, it works. >>>>>>> >>>>>>> I am thinking this has to do with the number of certificates in the >>>>>>> directory in the PE file. There can be three (cert, intermediate CA, and >>>>>>> CA, etc). >>>>>>> >>>>>>> Any idea what's going on? >>>>>>> _______________________________________________ >>>>>>> Cryptography-dev mailing list >>>>>>> Cryptography-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Cryptography-dev mailing list >>>>>>> Cryptography-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>> >>>>>> _______________________________________________ >>>>>> Cryptography-dev mailing list >>>>>> Cryptography-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>> >>>>>> _______________________________________________ >>>>>> Cryptography-dev mailing list >>>>>> Cryptography-dev at python.org >>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>> >>>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Sat Dec 29 01:19:59 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Sat, 29 Dec 2018 01:19:59 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: I also am having trouble parsing the extensions of the attached cert using the code above: Traceback (most recent call last): File "./extract_sigs.py", line 65, in for extension in cert.extensions: File "/root/sigs/lib/python3.7/site-packages/cryptography/utils.py", line 162, in inner result = func(instance) File "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/x509.py", line 137, in extensions self._backend, self._x509 File "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py", line 252, in parse value = handler(backend, ext_data) File "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py", line 342, in _decode_basic_constraints return x509.BasicConstraints(ca, path_length) File "/root/sigs/lib/python3.7/site-packages/cryptography/x509/extensions.py", line 345, in __init__ raise ValueError("path_length must be None when ca is False") ValueError: path_length must be None when ca is False On Sat, Dec 29, 2018 at 12:32 AM Robert Simmons wrote: > I do need all three certs. I do see what PKCS7_get0_signers does now. > > On Wed, Dec 26, 2018 at 11:27 AM Paul Kehrer > wrote: > >> We haven't had anyone request support for those legacy extension types, >> but if you think you need it feel free to file an issue and we can discuss >> adding it. The data can be parsed out of the UnknownExtension type right >> now of course. >> >> So you need all 3 certs? Only one of them is used for signing which is >> why the PKCS7_get0_signers call only returns that one. To obtain the rest >> we'll need to de-opaque two PKCS7 structs in cryptography: PKCS7_ENVELOPE >> and PKCS7_SIGN_ENVELOPE. Your use case should only require SIGN_ENVELOPE >> de-opaqued but might as well get them both. >> >> -Paul >> >> >> >> On December 25, 2018 at 9:15:10 PM, Robert Simmons (rsimmons0 at gmail.com) >> wrote: >> >> On a side note: there is one oid in the extensions of this cert that is >> listed as unknown, but openssl parses it as: >> Netscape Cert Type: >> Object Signing >> >> Is this something to submit a bug for? >> >> Also, happy holidays! >> >> On Tue, Dec 25, 2018 at 9:41 PM Robert Simmons >> wrote: >> >>> Thanks for the help above. However, I think I'm still missing something. >>> When piping the DER binary data to openssl on the command line, the output >>> appears to have three certificates in the example DER early in this thread. >>> The code above has a list for certs, but it appears to only contain one >>> cert at the end of the for loop. Is there a way to view the data from the >>> other two? I've attached the output from openssl command line. >>> >>> On Mon, Dec 24, 2018 at 11:51 AM Paul Kehrer >>> wrote: >>> >>>> Great! I have an idea of how to implement an API for this limited >>>> subset of pkcs7 as a utility function like the pkcs12 support we recently >>>> merged. Hopefully I or someone else can get to it soon. >>>> >>>> -Paul >>>> >>>> On Dec 23, 2018, at 6:32 PM, Robert Simmons >>>> wrote: >>>> >>>> This works great! Thanks! >>>> >>>> On Sun, Dec 23, 2018 at 7:05 PM Paul Kehrer >>>> wrote: >>>> >>>>> One day I will learn to run the code I write before I ask people to >>>>> use it. The missing signers variable should go after the pkcs7 assignment. >>>>> It looks like this: >>>>> >>>>> signers = backend._lib.PKCS7_get0_signers(pkcs7, backend._ffi.NULL, 0) >>>>> >>>>> With that in place and using the extracted.der you previously provided >>>>> I can parse a cert, which has the following subject/issuer data: >>>>> >>>>> Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA >>>>> Limited, CN=COMODO RSA Code Signing CA >>>>> Validity >>>>> Not Before: Oct 19 00:00:00 2018 GMT >>>>> Not After : Sep 25 23:59:59 2019 GMT >>>>> Subject: C=GB/postalCode=WA1 1RG, ST=UK, >>>>> L=WARRINGTON/street=Brunel House, 340 Firecrest Court, O=TATIANA PUK, >>>>> LIMITED, CN=TATIANA PUK, LIMITED >>>>> >>>>> I've also attached the cert. If this is what you're looking for then >>>>> your use case is covered by the existing issue, although I still need to >>>>> decide on an API for this. >>>>> >>>>> -Paul >>>>> >>>>> >>>>> >>>>> On December 23, 2018 at 2:17:54 AM, Robert Simmons ( >>>>> rsimmons0 at gmail.com) wrote: >>>>> >>>>> import os >>>>> import pathlib >>>>> import pefile >>>>> >>>>> target = >>>>> pathlib.Path().home().joinpath('Desktop').joinpath('HWID_4_0_6YMBWX.exe') >>>>> fname = str(target) >>>>> totsize = os.path.getsize(target) >>>>> pe = pefile.PE(fname) >>>>> >>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>>> sigoff = 0 >>>>> siglen = 0 >>>>> for s in pe.__structures__: >>>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>>> sigoff = s.VirtualAddress >>>>> siglen = s.Size >>>>> pe.close() >>>>> with open(fname, 'rb') as fh: >>>>> fh.seek(sigoff) >>>>> thesig = fh.read(siglen) >>>>> >>>>> from cryptography.hazmat.backends.openssl.backend import backend >>>>> from cryptography.hazmat.backends.openssl import x509 >>>>> >>>>> bio = backend._bytes_to_bio(thesig[8:]) >>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>>> certs = [] >>>>> for i in range(backend._lib.sk_X509_num(signers)): >>>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>>> >>>>> That's the exact code I'm trying to run with the provided code snippet >>>>> at the end. If you want to follow along with the exact file I'm working >>>>> with: >>>>> hxxps://dangerous[.]link/d9b72c43-1bdd-415b-b15f-3a436b26bca8 >>>>> >>>>> The password to that file is "infected" and btw: it is live malware, >>>>> so please treat it accordingly. Run code on it in a safe environment for >>>>> handling malware. >>>>> >>>>> On Sun, Dec 23, 2018 at 4:10 AM Robert Simmons >>>>> wrote: >>>>> >>>>>> I've added the use case to the issue as requested. I tried the code >>>>>> snippet, but the contents of signers is missing. What should that be? >>>>>> >>>>>> NameError: name 'signers' is not defined >>>>>> >>>>>> On Fri, Dec 21, 2018 at 11:21 AM Paul Kehrer >>>>>> wrote: >>>>>> >>>>>>> Out of curiosity, does the following code load the cert you expect? >>>>>>> der should be the bytes of extracted.der: >>>>>>> >>>>>>> from cryptography.hazmat.backends.openssl.backend import backend >>>>>>> from cryptography.hazmat.backends.openssl import x509 >>>>>>> >>>>>>> bio = backend._bytes_to_bio(der) >>>>>>> pkcs7 = backend._lib.d2i_PKCS7_bio(bio.bio, backend._ffi.NULL) >>>>>>> certs = [] >>>>>>> for i in range(backend._lib.sk_X509_num(signers)): >>>>>>> x509_ptr = backend._lib.sk_X509_value(signers, i) >>>>>>> certs.append(x509._Certificate(backend, x509_ptr)) >>>>>>> >>>>>>> Certs will be a list of signer certificates -- in this case, just >>>>>>> one cert in the list. Please note that this code does not manage memory >>>>>>> correctly so it should strictly be used to test if the cert you need is >>>>>>> being properly extracted :) >>>>>>> >>>>>>> -Paul (reaperhulk) >>>>>>> >>>>>>> >>>>>>> On December 21, 2018 at 8:02:13 AM, Paul Kehrer ( >>>>>>> paul.l.kehrer at gmail.com) wrote: >>>>>>> >>>>>>> Thanks, that's perfect. Looking at this data it's actually a PKCS7 >>>>>>> envelope holding multiple certificates and at the moment cryptography >>>>>>> unfortunately has no interface for parsing PKCS7. If you wouldn't mind >>>>>>> sharing your use case directly on >>>>>>> https://github.com/pyca/cryptography/issues/3983 then it will help >>>>>>> me when I'm prioritizing features for upcoming releases. >>>>>>> >>>>>>> -Paul >>>>>>> >>>>>>> >>>>>>> On December 20, 2018 at 2:23:11 PM, Robert Simmons ( >>>>>>> rsimmons0 at gmail.com) wrote: >>>>>>> >>>>>>> Definitely. I've attached the DER data as extracted from the PE file >>>>>>> using the following code: >>>>>>> >>>>>>> pe = pefile.PE(fname) >>>>>>> >>>>>>> pe.parse_data_directories(directories=[pefile.DIRECTORY_ENTRY['IMAGE_DIRECTORY_ENTRY_SECURITY']]) >>>>>>> sigoff = 0 >>>>>>> siglen = 0 >>>>>>> for s in pe.__structures__: >>>>>>> if s.name == 'IMAGE_DIRECTORY_ENTRY_SECURITY': >>>>>>> sigoff = s.VirtualAddress >>>>>>> siglen = s.Size >>>>>>> pe.close() >>>>>>> with open(fname, 'rb') as fh: >>>>>>> fh.seek(sigoff) >>>>>>> thesig = fh.read(siglen) >>>>>>> with open('extracted.der', 'wb') as fh: >>>>>>> fh.write(thesig[8:]) >>>>>>> >>>>>>> I've attached extracted.der as a zip file to maintain integrity as >>>>>>> an attachment. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> On Thu, Dec 20, 2018 at 11:12 AM Paul Kehrer < >>>>>>> paul.l.kehrer at gmail.com> wrote: >>>>>>> >>>>>>>> Could you give us an example (in hex or b64 or something) so we can >>>>>>>> easily reproduce? Make sure any certs you're giving us don't contain >>>>>>>> sensitive data of course. >>>>>>>> >>>>>>>> -Paul >>>>>>>> >>>>>>>> >>>>>>>> On December 19, 2018 at 11:55:04 PM, Robert Simmons ( >>>>>>>> rsimmons0 at gmail.com) wrote: >>>>>>>> >>>>>>>> I've asked this question on Stack Overflow here: >>>>>>>> https://stackoverflow.com/q/53862702/1033217 >>>>>>>> >>>>>>>> I have compared my code to Dider Stevens's disitool here (examine >>>>>>>> the function ExtractDigitalSignature): >>>>>>>> >>>>>>>> https://github.com/DidierStevens/DidierStevensSuite/blob/master/disitool.py >>>>>>>> >>>>>>>> When I load that extracted file into a variable and try to parse it >>>>>>>> with cryptography, it fails. If I pipe the same file to openssl on the >>>>>>>> command line, it works. >>>>>>>> >>>>>>>> I am thinking this has to do with the number of certificates in the >>>>>>>> directory in the PE file. There can be three (cert, intermediate CA, and >>>>>>>> CA, etc). >>>>>>>> >>>>>>>> Any idea what's going on? >>>>>>>> _______________________________________________ >>>>>>>> Cryptography-dev mailing list >>>>>>>> Cryptography-dev at python.org >>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Cryptography-dev mailing list >>>>>>>> Cryptography-dev at python.org >>>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Cryptography-dev mailing list >>>>>>> Cryptography-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Cryptography-dev mailing list >>>>>>> Cryptography-dev at python.org >>>>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>>>> >>>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>>> _______________________________________________ >>>>> Cryptography-dev mailing list >>>>> Cryptography-dev at python.org >>>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>>> >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>>> _______________________________________________ >>>> Cryptography-dev mailing list >>>> Cryptography-dev at python.org >>>> https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: e226bb734a0781b62f2f5a27216aba2b.der Type: application/x-x509-ca-cert Size: 1584 bytes Desc: not available URL: From paul.l.kehrer at gmail.com Sat Dec 29 09:29:17 2018 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Sat, 29 Dec 2018 06:29:17 -0800 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: This is https://github.com/pyca/cryptography/issues/3856 The cert inside the PKCS7 you gave has both CA False and a path length (of 0) set. That is technically invalid according to the RFC so the parser rejects it. We've talked about how to resolve this in https://github.com/pyca/cryptography/pull/3862 but I haven't finished it because at the time I decided the more elegant solution within the context of our APIs would be to have an x509 hazmat layer that the recipe layer consumed. I could maybe be convinced that moving the logic as in 3862 is a good solution though since parsing a cert is not a validation step. Refusing to generate bad certs + (some day) having a verifier that will reject junk we've parsed is possibly a decent solution. -Paul On December 29, 2018 at 12:20:27 AM, Robert Simmons (rsimmons0 at gmail.com) wrote: I also am having trouble parsing the extensions of the attached cert using the code above: Traceback (most recent call last): File "./extract_sigs.py", line 65, in for extension in cert.extensions: File "/root/sigs/lib/python3.7/site-packages/cryptography/utils.py", line 162, in inner result = func(instance) File "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/x509.py", line 137, in extensions self._backend, self._x509 File "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py", line 252, in parse value = handler(backend, ext_data) File "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py", line 342, in _decode_basic_constraints return x509.BasicConstraints(ca, path_length) File "/root/sigs/lib/python3.7/site-packages/cryptography/x509/extensions.py", line 345, in __init__ raise ValueError("path_length must be None when ca is False") ValueError: path_length must be None when ca is False -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsimmons0 at gmail.com Mon Dec 31 02:28:20 2018 From: rsimmons0 at gmail.com (Robert Simmons) Date: Mon, 31 Dec 2018 02:28:20 -0500 Subject: [Cryptography-dev] Parsing DER from PE File In-Reply-To: References: <33A9E4A5-B3A6-4666-AEBE-1C45F02B39EF@gmail.com> Message-ID: This makes sense. Also, my goal is parsing rather than validation in the step I'm working on. It's all potentially malware, so it may have malformed data. On Sat, Dec 29, 2018, 09:29 Paul Kehrer This is https://github.com/pyca/cryptography/issues/3856 > > The cert inside the PKCS7 you gave has both CA False and a path length (of > 0) set. That is technically invalid according to the RFC so the parser > rejects it. We've talked about how to resolve this in > https://github.com/pyca/cryptography/pull/3862 but I haven't finished it > because at the time I decided the more elegant solution within the context > of our APIs would be to have an x509 hazmat layer that the recipe layer > consumed. I could maybe be convinced that moving the logic as in 3862 is a > good solution though since parsing a cert is not a validation step. > Refusing to generate bad certs + (some day) having a verifier that will > reject junk we've parsed is possibly a decent solution. > > -Paul > > > On December 29, 2018 at 12:20:27 AM, Robert Simmons (rsimmons0 at gmail.com) > wrote: > > I also am having trouble parsing the extensions of the attached cert using > the code above: > > Traceback (most recent call last): > File "./extract_sigs.py", line 65, in > for extension in cert.extensions: > File "/root/sigs/lib/python3.7/site-packages/cryptography/utils.py", > line 162, in inner > > result = func(instance) > File > "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/x509.py", > line 137, in extensions > self._backend, self._x509 > File > "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py", > line 252, in parse > value = handler(backend, ext_data) > File > "/root/sigs/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/decode_asn1.py", > line 342, in _decode_basic_constraints > return x509.BasicConstraints(ca, path_length) > File > "/root/sigs/lib/python3.7/site-packages/cryptography/x509/extensions.py", > line 345, in __init__ > raise ValueError("path_length must be None when ca is False") > ValueError: path_length must be None when ca is False > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: