From k.jackiewicz at samsung.com Thu Jul 29 11:33:08 2021 From: k.jackiewicz at samsung.com (=?ks_c_5601-1987?B?S3J6eXN6dG9mIEphY2tpZXdpY3ovU2VjdXJpdHkgKFBMVA==?= =?ks_c_5601-1987?B?KSAvU1JQT0wvRW5naW5lZXIvu++8usD8wNo=?=) Date: Thu, 29 Jul 2021 17:33:08 +0200 Subject: [Cryptography-dev] Problems with codecov & doc8 References: Message-ID: <000001d7848f$0b284fe0$2178efa0$@samsung.com> Hi, Project codecov and doc8 checks for my recent PR (https://github.com/pyca/cryptography/pull/6181) have failed and I can't find the way to make them pass. IMHO the codecov fail is unrelated to the PR. It looks like a problem with testing environment. I ran pytest with coverage report before and after my changes and the coverage ratio was unchanged. Is it possible to reproduce the codecov check locally somehow? The doc8 check fails because of too long line in CHANGELOG.rst but the line was not changed in the PR. I noticed that doc8 complains only if the offending line is not the last one on the list. I could trick it by putting the KBKDFCMAC entry before the offending line but I don't think it's the way to go. Please advise, Thanks -- Krzysiek From paul.l.kehrer at gmail.com Thu Jul 29 14:29:44 2021 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Thu, 29 Jul 2021 13:29:44 -0500 Subject: [Cryptography-dev] Problems with codecov & doc8 In-Reply-To: <000001d7848f$0b284fe0$2178efa0$@samsung.com> References: <000001d7848f$0b284fe0$2178efa0$@samsung.com> Message-ID: Unfortunately codecov commonly has issues with proper reporting. It looks like in your case the mac builders weren't successfully submitting coverage. We'd love to migrate to a more functional system, but codecov and coveralls both have...issues. We won't let that block review, although pushing a few extra commits or rebasing can resolve it since it's an inconsistent problem with codecov. As for doc8, feel free to modify the offending line. Moving the trailing punctuation (which should be a period) will likely resolve it. It's probably worth filing a bug with doc8 around this if you can come up with a small reproducer since I believe we've seen this issue in the past. -Paul On Thu, Jul 29, 2021 at 10:38 AM Krzysztof Jackiewicz/Security (PLT) /SRPOL/Engineer/???? wrote: > > Hi, > > Project codecov and doc8 checks for my recent PR > (https://github.com/pyca/cryptography/pull/6181) have failed and I can't > find the way to make them pass. > > IMHO the codecov fail is unrelated to the PR. It looks like a problem with > testing environment. I ran pytest with coverage report before and after my > changes and the coverage ratio was unchanged. Is it possible to reproduce > the codecov check locally somehow? > > The doc8 check fails because of too long line in CHANGELOG.rst but the line > was not changed in the PR. I noticed that doc8 complains only if the > offending line is not the last one on the list. I could trick it by putting > the KBKDFCMAC entry before the offending line but I don't think it's the > way to go. > > Please advise, > > Thanks > > -- > Krzysiek > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From k.jackiewicz at samsung.com Fri Jul 30 07:57:47 2021 From: k.jackiewicz at samsung.com (=?UTF-8?Q?Krzysztof_Jackiewicz/Security_=28P?= =?UTF-8?Q?LT=29_/SRPOL/Engineer/=EC=82=BC=EC=84=B1=EC=A0=84=EC=9E=90?=) Date: Fri, 30 Jul 2021 13:57:47 +0200 Subject: [Cryptography-dev] Problems with codecov & doc8 In-Reply-To: References: <000001d7848f$0b284fe0$2178efa0$@samsung.com> Message-ID: <00a401d7853a$1f419cb0$5dc4d610$@samsung.com> Thanks for suggestions. The punctuation was not an issue. Doc8 complains if the too long line contains whitespaces and is followed by another bullet list entry (if --allow-long-titles option is off, it will fail regardless of the entry order). Breaking the line in the middle of the cross-reference solves the problem for now. I fixed it in a separate commit which will hopefully solve the codecov problem too. I'll fill a bug report @doc8 later on. -- Krzysiek -----Original Message----- From: Cryptography-dev On Behalf Of Paul Kehrer Sent: Thursday, July 29, 2021 8:30 PM To: cryptography-dev at python.org Subject: Re: [Cryptography-dev] Problems with codecov & doc8 Unfortunately codecov commonly has issues with proper reporting. It looks like in your case the mac builders weren't successfully submitting coverage. We'd love to migrate to a more functional system, but codecov and coveralls both have...issues. We won't let that block review, although pushing a few extra commits or rebasing can resolve it since it's an inconsistent problem with codecov. As for doc8, feel free to modify the offending line. Moving the trailing punctuation (which should be a period) will likely resolve it. It's probably worth filing a bug with doc8 around this if you can come up with a small reproducer since I believe we've seen this issue in the past. -Paul On Thu, Jul 29, 2021 at 10:38 AM Krzysztof Jackiewicz/Security (PLT) /SRPOL/Engineer/???? wrote: > > Hi, > > Project codecov and doc8 checks for my recent PR > (https://protect2.fireeye.com/v1/url?k=5f977e63-000c46ba-5f96f52c-0cc4 > 7a3356b2-e04c6f618d9a0339&q=1&e=74ac4513-e918-4f02-99f7-77762aafea3a&u=https%3A%2F%2Fgithub.com%2Fpyca%2Fcryptography%2Fpull%2F6181) have failed and I can't find the way to make them pass. > > IMHO the codecov fail is unrelated to the PR. It looks like a problem > with testing environment. I ran pytest with coverage report before and > after my changes and the coverage ratio was unchanged. Is it possible > to reproduce the codecov check locally somehow? > > The doc8 check fails because of too long line in CHANGELOG.rst but the > line was not changed in the PR. I noticed that doc8 complains only if > the offending line is not the last one on the list. I could trick it > by putting the KBKDFCMAC entry before the offending line but I don't > think it's the way to go. > > Please advise, > > Thanks > > -- > Krzysiek > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev From dirkx at webweaving.org Sat Jul 31 07:08:57 2021 From: dirkx at webweaving.org (Dirk-Willem van Gulik) Date: Sat, 31 Jul 2021 13:08:57 +0200 Subject: [Cryptography-dev] X509_STORE_set_purpose() missing Message-ID: Could it be that somehow in (in the latest build) - X509_STORE_set_purpose and associated #defines are missing ? In below - things work fine up until lib.X509_STORE_set_purpose() - but that calls gives me a: AttributeError: cffi library '_openssl' has no function, constant or global variable named 'X509_STORE_set_purpose' With kind regards, Dw # Create the pkcs7 object pkcs7_object = lib.d2i_PKCS7_bio(bio.bio, ffi.NULL) # We're not passing any untrusted certificates, the chain should # complete, up to, but not including the CA cert, in the CMS package. # other = lib.sk_X509_new_null() binding._openssl_assert(lib, other != ffi.NULL) # We are prividing exactly one certificate - that of the certificate # authority - as trusted. It has to be signed by this national root. # store = lib.X509_STORE_new() lib.X509_STORE_add_cert(store, certificate._x509) # type: ignore # As we're using certifcates somewhat off-label; we need to relax # the purpose verification. This is the equivalent of the -purpose any # flag in: # openssl smime -verify -inform DER -content payload.raw \ # -CAfile ca.pem -in signature.p7 -purpose any lib.X509_STORE_set_purpose(store, 7) # X509_PURPOSE_ANY -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Sat Jul 31 08:27:14 2021 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Sat, 31 Jul 2021 07:27:14 -0500 Subject: [Cryptography-dev] X509_STORE_set_purpose() missing In-Reply-To: References: Message-ID: Searching our history I don't believe we've ever bound X509_STORE_set_purpose. Did this work in a previous version of cryptography and has only recently stopped? In general, cryptography does not bind all of OpenSSL, only the functions, macros, and constants we need to expose our APIs. We have one consumer that we officially support which uses the bindings directly (pyOpenSSL), but otherwise we consider the bindings to be private API surface and will add/remove as needed to support various versions of OpenSSL. If cryptography is lacking public APIs for your use case please consider filing an issue and helping design/implement those APIs with us. Years of experience with maintaining our bindings across dozens of OpenSSL versions and various forks has taught us that we can't reliably support random bindings we don't use ourselves. -Paul On Sat, Jul 31, 2021 at 6:38 AM Dirk-Willem van Gulik wrote: > > Could it be that somehow in (in the latest build) - X509_STORE_set_purpose and associated #defines are missing ? > > In below - things work fine up until lib.X509_STORE_set_purpose() - but that calls gives me a: > > AttributeError: cffi library '_openssl' has no function, constant or global variable named 'X509_STORE_set_purpose' > > With kind regards, > > Dw > > # Create the pkcs7 object > pkcs7_object = lib.d2i_PKCS7_bio(bio.bio, ffi.NULL) > > # We're not passing any untrusted certificates, the chain should > # complete, up to, but not including the CA cert, in the CMS package. > # > other = lib.sk_X509_new_null() > binding._openssl_assert(lib, other != ffi.NULL) > > # We are prividing exactly one certificate - that of the certificate > # authority - as trusted. It has to be signed by this national root. > # > store = lib.X509_STORE_new() > lib.X509_STORE_add_cert(store, certificate._x509) # type: ignore > > # As we're using certifcates somewhat off-label; we need to relax > # the purpose verification. This is the equivalent of the -purpose any > # flag in: > # openssl smime -verify -inform DER -content payload.raw \ > # -CAfile ca.pem -in signature.p7 -purpose any > lib.X509_STORE_set_purpose(store, 7) # X509_PURPOSE_ANY > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From dirkx at webweaving.org Sat Jul 31 11:47:35 2021 From: dirkx at webweaving.org (Dirk-Willem van Gulik) Date: Sat, 31 Jul 2021 17:47:35 +0200 Subject: [Cryptography-dev] X509_STORE_set_purpose() missing In-Reply-To: References: Message-ID: No trouble - filed as https://github.com/pyca/pyopenssl/issues/1031 I guess that given how common this is - the easiest may actually be to have an extra flag to verify - with the purpose (or all the flags). As that covers most cases. Dw. > On 31 Jul 2021, at 14:27, Paul Kehrer wrote: > > Searching our history I don't believe we've ever bound > X509_STORE_set_purpose. Did this work in a previous version of > cryptography and has only recently stopped? > > In general, cryptography does not bind all of OpenSSL, only the > functions, macros, and constants we need to expose our APIs. We have > one consumer that we officially support which uses the bindings > directly (pyOpenSSL), but otherwise we consider the bindings to be > private API surface and will add/remove as needed to support various > versions of OpenSSL. > > If cryptography is lacking public APIs for your use case please > consider filing an issue and helping design/implement those APIs with > us. Years of experience with maintaining our bindings across dozens of > OpenSSL versions and various forks has taught us that we can't > reliably support random bindings we don't use ourselves. > > -Paul > > On Sat, Jul 31, 2021 at 6:38 AM Dirk-Willem van Gulik > wrote: >> >> Could it be that somehow in (in the latest build) - X509_STORE_set_purpose and associated #defines are missing ? >> >> In below - things work fine up until lib.X509_STORE_set_purpose() - but that calls gives me a: >> >> AttributeError: cffi library '_openssl' has no function, constant or global variable named 'X509_STORE_set_purpose' >> >> With kind regards, >> >> Dw >> >> # Create the pkcs7 object >> pkcs7_object = lib.d2i_PKCS7_bio(bio.bio, ffi.NULL) >> >> # We're not passing any untrusted certificates, the chain should >> # complete, up to, but not including the CA cert, in the CMS package. >> # >> other = lib.sk_X509_new_null() >> binding._openssl_assert(lib, other != ffi.NULL) >> >> # We are prividing exactly one certificate - that of the certificate >> # authority - as trusted. It has to be signed by this national root. >> # >> store = lib.X509_STORE_new() >> lib.X509_STORE_add_cert(store, certificate._x509) # type: ignore >> >> # As we're using certifcates somewhat off-label; we need to relax >> # the purpose verification. This is the equivalent of the -purpose any >> # flag in: >> # openssl smime -verify -inform DER -content payload.raw \ >> # -CAfile ca.pem -in signature.p7 -purpose any >> lib.X509_STORE_set_purpose(store, 7) # X509_PURPOSE_ANY >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev >