From Brian.Matthews at vaisala.com Fri Jan 8 12:32:15 2021 From: Brian.Matthews at vaisala.com (Matthews Brian) Date: Fri, 8 Jan 2021 17:32:15 +0000 Subject: [Cryptography-dev] Heap error in _openssl.pyd Message-ID: Hi, We are using pyOpenSSL 20.0.1 with cryptography 3.3.1 in Python 2.7.17 x64. This is running on Microsoft Windows 2019. The pyOpenSSL is being used as part of a twisted Web implementation, handling TLS HTTPS communications. We are experiencing random crashes in the _openssl.pyd module. On some sites it is once per month, on other sites it is once per day. The crash dump shows a HEAP error originating in the _openssl.pyd module during TLS negotiation. 1. Has anyone else experienced this 2. Are the symbol pdb files available for the _openssl.pyd object 3. What VS studio version are the wheels for 2.7 built with Thanks Brian. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Fri Jan 8 12:54:01 2021 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Fri, 8 Jan 2021 11:54:01 -0600 Subject: [Cryptography-dev] Heap error in _openssl.pyd In-Reply-To: References: Message-ID: We don't have the symbol files, but it was built with VS 2010 (scripts found here: https://github.com/pyca/infra/tree/master/windows/openssl) -Paul On Fri, Jan 8, 2021 at 11:36 AM Matthews Brian wrote: > > Hi, > > > > We are using pyOpenSSL 20.0.1 with cryptography 3.3.1 in Python 2.7.17 x64. This is running on Microsoft Windows 2019. > > > > The pyOpenSSL is being used as part of a twisted Web implementation, handling TLS HTTPS communications. > > > > We are experiencing random crashes in the _openssl.pyd module. On some sites it is once per month, on other sites it is once per day. > > > > The crash dump shows a HEAP error originating in the _openssl.pyd module during TLS negotiation. > > > > Has anyone else experienced this > Are the symbol pdb files available for the _openssl.pyd object > What VS studio version are the wheels for 2.7 built with > > > > Thanks > > > > Brian. > > > > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From Brian.Matthews at vaisala.com Fri Jan 8 13:51:47 2021 From: Brian.Matthews at vaisala.com (Matthews Brian) Date: Fri, 8 Jan 2021 18:51:47 +0000 Subject: [Cryptography-dev] Heap error in _openssl.pyd In-Reply-To: References: Message-ID: Ok thanks. As the rest of Python2.7 is built with VS2008, I will try rebuilding with that. Brian -----Original Message----- From: Cryptography-dev On Behalf Of Paul Kehrer Sent: January 8, 2021 9:54 AM To: cryptography-dev at python.org Subject: Re: [Cryptography-dev] Heap error in _openssl.pyd We don't have the symbol files, but it was built with VS 2010 (scripts found here: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpyca%2Finfra%2Ftree%2Fmaster%2Fwindows%2Fopenssl&data=04%7C01%7Cbrian.matthews%40vaisala.com%7C9568a9cebccd4386a7ee08d8b3fe6d05%7C6d7393e041f54c2e9b124c2be5da5c57%7C0%7C0%7C637457252632399827%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=G8fvEPiFnlraSXitzLryVi8uwraKL0a2iIgE9efC0Fo%3D&reserved=0) -Paul On Fri, Jan 8, 2021 at 11:36 AM Matthews Brian wrote: > > Hi, > > > > We are using pyOpenSSL 20.0.1 with cryptography 3.3.1 in Python 2.7.17 x64. This is running on Microsoft Windows 2019. > > > > The pyOpenSSL is being used as part of a twisted Web implementation, handling TLS HTTPS communications. > > > > We are experiencing random crashes in the _openssl.pyd module. On some sites it is once per month, on other sites it is once per day. > > > > The crash dump shows a HEAP error originating in the _openssl.pyd module during TLS negotiation. > > > > Has anyone else experienced this > Are the symbol pdb files available for the _openssl.pyd object What VS > studio version are the wheels for 2.7 built with > > > > Thanks > > > > Brian. > > > > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail > .python.org%2Fmailman%2Flistinfo%2Fcryptography-dev&data=04%7C01%7 > Cbrian.matthews%40vaisala.com%7C9568a9cebccd4386a7ee08d8b3fe6d05%7C6d7 > 393e041f54c2e9b124c2be5da5c57%7C0%7C0%7C637457252632409829%7CUnknown%7 > CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV > CI6Mn0%3D%7C1000&sdata=sxYsgSL4BRuHbrJfZxlvVQU5Zh6zEC7epyRoU0FOhJ8 > %3D&reserved=0 _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fcryptography-dev&data=04%7C01%7Cbrian.matthews%40vaisala.com%7C9568a9cebccd4386a7ee08d8b3fe6d05%7C6d7393e041f54c2e9b124c2be5da5c57%7C0%7C0%7C637457252632409829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sxYsgSL4BRuHbrJfZxlvVQU5Zh6zEC7epyRoU0FOhJ8%3D&reserved=0 From barry.scott at forcepoint.com Tue Jan 12 04:42:51 2021 From: barry.scott at forcepoint.com (Barry Scott) Date: Tue, 12 Jan 2021 09:42:51 +0000 Subject: [Cryptography-dev] Heap error in _openssl.pyd In-Reply-To: References: Message-ID: <4292434.LvFx2qVVIh@fpbarry> This is the correct version to use that Microsoft makes available. "Microsoft Visual C++ Compiler for Python 2.7" https://www.microsoft.com/en-gb/download/details.aspx?id=44266 Your heap crashes are expected if you do not use the above compiler and its associated runtime. Barry On Friday, 8 January 2021 18:51:47 GMT Matthews Brian wrote: > Ok thanks. As the rest of Python2.7 is built with VS2008, I will try rebuilding with that. > > Brian > > -----Original Message----- > From: Cryptography-dev On Behalf Of Paul Kehrer > Sent: January 8, 2021 9:54 AM > To: cryptography-dev at python.org > Subject: Re: [Cryptography-dev] Heap error in _openssl.pyd > > We don't have the symbol files, but it was built with VS 2010 (scripts found here: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpyca%2Finfra%2Ftree%2Fmaster%2Fwindows%2Fopenssl&data=04%7C01%7Cbrian.matthews%40vaisala.com%7C9568a9cebccd4386a7ee08d8b3fe6d05%7C6d7393e041f54c2e9b124c2be5da5c57%7C0%7C0%7C637457252632399827%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=G8fvEPiFnlraSXitzLryVi8uwraKL0a2iIgE9efC0Fo%3D&reserved=0) > > -Paul > > On Fri, Jan 8, 2021 at 11:36 AM Matthews Brian wrote: > > > > Hi, > > > > > > > > We are using pyOpenSSL 20.0.1 with cryptography 3.3.1 in Python 2.7.17 x64. This is running on Microsoft Windows 2019. > > > > > > > > The pyOpenSSL is being used as part of a twisted Web implementation, handling TLS HTTPS communications. > > > > > > > > We are experiencing random crashes in the _openssl.pyd module. On some sites it is once per month, on other sites it is once per day. > > > > > > > > The crash dump shows a HEAP error originating in the _openssl.pyd module during TLS negotiation. > > > > > > > > Has anyone else experienced this > > Are the symbol pdb files available for the _openssl.pyd object What VS > > studio version are the wheels for 2.7 built with > > > > > > > > Thanks > > > > > > > > Brian. > > > > > > > > > > > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail > > .python.org%2Fmailman%2Flistinfo%2Fcryptography-dev&data=04%7C01%7 > > Cbrian.matthews%40vaisala.com%7C9568a9cebccd4386a7ee08d8b3fe6d05%7C6d7 > > 393e041f54c2e9b124c2be5da5c57%7C0%7C0%7C637457252632409829%7CUnknown%7 > > CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV > > CI6Mn0%3D%7C1000&sdata=sxYsgSL4BRuHbrJfZxlvVQU5Zh6zEC7epyRoU0FOhJ8 > > %3D&reserved=0 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fcryptography-dev&data=04%7C01%7Cbrian.matthews%40vaisala.com%7C9568a9cebccd4386a7ee08d8b3fe6d05%7C6d7393e041f54c2e9b124c2be5da5c57%7C0%7C0%7C637457252632409829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sxYsgSL4BRuHbrJfZxlvVQU5Zh6zEC7epyRoU0FOhJ8%3D&reserved=0 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > From michael at stroeder.com Tue Jan 12 10:52:01 2021 From: michael at stroeder.com (=?UTF-8?Q?Michael_Str=c3=b6der?=) Date: Tue, 12 Jan 2021 16:52:01 +0100 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: Message-ID: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> On 12/22/20 8:43 PM, Alex Gaynor wrote: > As we previewed in August [0] we're planning to incorporate Rust code > into pyca/cryptography. IMHO this will make life of distro packagers more miserable especially on non-x86 platforms. Ciao, Michael. From alex.gaynor at gmail.com Tue Jan 12 11:33:37 2021 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 12 Jan 2021 11:33:37 -0500 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> Message-ID: Do you package pyca/cryptography for a distro? Which one? We've been in close contact with the Debian and RHEL/Fedora maintainers of the pyca/cryptography package, and have incorporated feedback from them. Is there something we could be doing to improve the situation? We're happy to put in the work here, but we won't accept "don't use Rust" as an answer. Alex On Tue, Jan 12, 2021 at 11:29 AM Michael Str?der via Cryptography-dev wrote: > > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > As we previewed in August [0] we're planning to incorporate Rust code > > into pyca/cryptography. > > IMHO this will make life of distro packagers more miserable especially > on non-x86 platforms. > > Ciao, Michael. > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -- All that is necessary for evil to succeed is for good people to do nothing. From barry.scott at forcepoint.com Tue Jan 12 12:01:36 2021 From: barry.scott at forcepoint.com (Barry Scott) Date: Tue, 12 Jan 2021 17:01:36 +0000 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> Message-ID: <2453147.Lt9SDvczpP@fpbarry> On Tuesday, 12 January 2021 15:52:01 GMT Michael Str?der via Cryptography-dev wrote: > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > As we previewed in August [0] we're planning to incorporate Rust code > > into pyca/cryptography. > > IMHO this will make life of distro packagers more miserable especially > on non-x86 platforms. I was also concerned by that new toolset dependency. Will this build on Centos 8 with the version of rust that is packaged there? Barry From alex.gaynor at gmail.com Tue Jan 12 12:23:10 2021 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 12 Jan 2021 12:23:10 -0500 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: <2453147.Lt9SDvczpP@fpbarry> References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> Message-ID: Running `yum install rust` in a CentOS8 docker container seems to get me rustc 1.45.2, and as our docs say, 1.45.0 will be the initial minimum version (https://cryptography.io/en/latest/installation.html#rust). As ever, our wheels (which are how the vast majority of our users install pyca/cryptography) will not require any compiler or build toolchain on user's machines. Alex On Tue, Jan 12, 2021 at 12:17 PM Barry Scott wrote: > > On Tuesday, 12 January 2021 15:52:01 GMT Michael Str?der via Cryptography-dev wrote: > > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > > As we previewed in August [0] we're planning to incorporate Rust code > > > into pyca/cryptography. > > > > IMHO this will make life of distro packagers more miserable especially > > on non-x86 platforms. > > I was also concerned by that new toolset dependency. > > Will this build on Centos 8 with the version of rust that is packaged there? > > Barry > > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -- All that is necessary for evil to succeed is for good people to do nothing. From michael at stroeder.com Tue Jan 12 12:56:20 2021 From: michael at stroeder.com (=?UTF-8?Q?Michael_Str=c3=b6der?=) Date: Tue, 12 Jan 2021 18:56:20 +0100 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> Message-ID: <7ed223ef-113d-45e3-0c66-8ebf9d4bb2a5@stroeder.com> On 1/12/21 5:33 PM, Alex Gaynor wrote: > Do you package pyca/cryptography for a distro? Sometimes I update the openSUSE package [1] but I'm not the package maintainer. Also I don't have detailed knowledge about current state of rust tool-chain for openSUSE. But I have doubts that all the possible hardware platforms are supported. Ciao, Michael. [1] https://build.opensuse.org/package/show/devel:languages:python/python-cryptography From michael at stroeder.com Tue Jan 12 12:57:57 2021 From: michael at stroeder.com (=?UTF-8?Q?Michael_Str=c3=b6der?=) Date: Tue, 12 Jan 2021 18:57:57 +0100 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> Message-ID: <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> On 1/12/21 6:23 PM, Alex Gaynor wrote: > As ever, our wheels (which are how the vast majority of our users > install pyca/cryptography) will not require any compiler or build > toolchain on user's machines. And you will provide wheels for armv6, S/390 etc.? Ciao, Michael. From alex.gaynor at gmail.com Tue Jan 12 13:05:34 2021 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 12 Jan 2021 13:05:34 -0500 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> Message-ID: At the moment we provide wheels for: - x86_64, arm64 Linux - x86_64 Darwin - x86_64, x86_32 Windows The primary constraint on our ability to add new wheel platforms is our ability to have reliable, performant, CI for them. We will not distribute wheels for any platform we can't test against. If you're aware of a CI provider that meets those needs for armv6, S/390, etc. we'd be eager to consider them. At any rate, those platforms are all supported (at various tiers) by upstream Rust: https://doc.rust-lang.org/beta/rustc/platform-support.html so anyone able to build cryptography for armv6 or S/390 now should be able to install the rust toolchain and then build. As I said at the top, we're happy to do the work to make this as smooth a transition as practicable (and indeed, we've done work to improve setuptools-rust, add abi3 support to pyo3, make Rust available in RTD, etc.), but we're not simply going to stop these efforts: Language level memory safety is not negotiable. Alex On Tue, Jan 12, 2021 at 1:00 PM Michael Str?der via Cryptography-dev wrote: > > On 1/12/21 6:23 PM, Alex Gaynor wrote: > > As ever, our wheels (which are how the vast majority of our users > > install pyca/cryptography) will not require any compiler or build > > toolchain on user's machines. > > And you will provide wheels for armv6, S/390 etc.? > > Ciao, Michael. > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -- All that is necessary for evil to succeed is for good people to do nothing. From hs at ox.cx Wed Jan 13 02:48:53 2021 From: hs at ox.cx (Hynek Schlawack) Date: Wed, 13 Jan 2021 08:48:53 +0100 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> Message-ID: > On 12. Jan 2021, at 19:05, Alex Gaynor wrote: > > At the moment we provide wheels for: > > - x86_64, arm64 Linux This is interesting, I?ve been told ( et ff) that there?s no arm64 wheel standard yet and thus arm64 wheels are likely to cause problems. Have there been any new development in that regard? Maybe a discussion I can read up? From alex.gaynor at gmail.com Wed Jan 13 08:37:15 2021 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 13 Jan 2021 08:37:15 -0500 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> Message-ID: https://www.python.org/dev/peps/pep-0599/ :-) Alex On Wed, Jan 13, 2021 at 2:49 AM Hynek Schlawack wrote: > > > > > On 12. Jan 2021, at 19:05, Alex Gaynor wrote: > > > > At the moment we provide wheels for: > > > > - x86_64, arm64 Linux > > This is interesting, I?ve been told ( et ff) that there?s no arm64 wheel standard yet and thus arm64 wheels are likely to cause problems. > > Have there been any new development in that regard? Maybe a discussion I can read up? > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -- All that is necessary for evil to succeed is for good people to do nothing. From alex.gaynor at gmail.com Wed Jan 13 09:47:37 2021 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 13 Jan 2021 09:47:37 -0500 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: <3407119.R56niFO833@fpbarry> References: <2453147.Lt9SDvczpP@fpbarry> <3407119.R56niFO833@fpbarry> Message-ID: I'm glad to hear some folks are actually auditing 3rd party sources. Once you install a rust toolchain, you'll be able to build pyca/cryptography the same as ever. To repeat: if there's some action we can be taking to make this migration smoother, we're happy to consider it. But what we won't do is simply stop trying to drop C. Alex On Wed, Jan 13, 2021 at 9:45 AM Barry Scott wrote: > > On Tuesday, 12 January 2021 17:23:10 GMT Alex Gaynor wrote: > > Running `yum install rust` in a CentOS8 docker container seems to get > > me rustc 1.45.2, and as our docs say, 1.45.0 will be the initial > > minimum version > > (https://cryptography.io/en/latest/installation.html#rust). > > > > As ever, our wheels (which are how the vast majority of our users > > install pyca/cryptography) will not require any compiler or build > > toolchain on user's machines. > > But in the enterprise space its a no-no to use the wheels you build. > > I get your source, audit it and build that for use in our environment. > > Barry > > > > > Alex > > > > On Tue, Jan 12, 2021 at 12:17 PM Barry Scott wrote: > > > > > > On Tuesday, 12 January 2021 15:52:01 GMT Michael Str?der via Cryptography-dev wrote: > > > > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > > > > As we previewed in August [0] we're planning to incorporate Rust code > > > > > into pyca/cryptography. > > > > > > > > IMHO this will make life of distro packagers more miserable especially > > > > on non-x86 platforms. > > > > > > I was also concerned by that new toolset dependency. > > > > > > Will this build on Centos 8 with the version of rust that is packaged there? > > > > > > Barry > > > > > > > > > > > > _______________________________________________ > > > Cryptography-dev mailing list > > > Cryptography-dev at python.org > > > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > > > > > > > > -- All that is necessary for evil to succeed is for good people to do nothing. From barry.scott at forcepoint.com Wed Jan 13 09:51:25 2021 From: barry.scott at forcepoint.com (Barry Scott) Date: Wed, 13 Jan 2021 14:51:25 +0000 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <3407119.R56niFO833@fpbarry> Message-ID: <1820978.taCxCBeP46@fpbarry> On Wednesday, 13 January 2021 14:47:37 GMT Alex Gaynor wrote: > I'm glad to hear some folks are actually auditing 3rd party sources. > > Once you install a rust toolchain, you'll be able to build > pyca/cryptography the same as ever. > > To repeat: if there's some action we can be taking to make this > migration smoother, we're happy to consider it. But what we won't do > is simply stop trying to drop C. I'm all for better code. I just wanted to point out that you have user that do not show up in PyPI stats. Barry > > Alex > > On Wed, Jan 13, 2021 at 9:45 AM Barry Scott wrote: > > > > On Tuesday, 12 January 2021 17:23:10 GMT Alex Gaynor wrote: > > > Running `yum install rust` in a CentOS8 docker container seems to get > > > me rustc 1.45.2, and as our docs say, 1.45.0 will be the initial > > > minimum version > > > (https://cryptography.io/en/latest/installation.html#rust). > > > > > > As ever, our wheels (which are how the vast majority of our users > > > install pyca/cryptography) will not require any compiler or build > > > toolchain on user's machines. > > > > But in the enterprise space its a no-no to use the wheels you build. > > > > I get your source, audit it and build that for use in our environment. > > > > Barry > > > > > > > > Alex > > > > > > On Tue, Jan 12, 2021 at 12:17 PM Barry Scott wrote: > > > > > > > > On Tuesday, 12 January 2021 15:52:01 GMT Michael Str?der via Cryptography-dev wrote: > > > > > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > > > > > As we previewed in August [0] we're planning to incorporate Rust code > > > > > > into pyca/cryptography. > > > > > > > > > > IMHO this will make life of distro packagers more miserable especially > > > > > on non-x86 platforms. > > > > > > > > I was also concerned by that new toolset dependency. > > > > > > > > Will this build on Centos 8 with the version of rust that is packaged there? > > > > > > > > Barry > > > > > > > > > > > > > > > > _______________________________________________ > > > > Cryptography-dev mailing list > > > > Cryptography-dev at python.org > > > > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > > > > > > > > > > > > > > > > > > > From barry.scott at forcepoint.com Wed Jan 13 09:44:59 2021 From: barry.scott at forcepoint.com (Barry Scott) Date: Wed, 13 Jan 2021 14:44:59 +0000 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <2453147.Lt9SDvczpP@fpbarry> Message-ID: <3407119.R56niFO833@fpbarry> On Tuesday, 12 January 2021 17:23:10 GMT Alex Gaynor wrote: > Running `yum install rust` in a CentOS8 docker container seems to get > me rustc 1.45.2, and as our docs say, 1.45.0 will be the initial > minimum version > (https://cryptography.io/en/latest/installation.html#rust). > > As ever, our wheels (which are how the vast majority of our users > install pyca/cryptography) will not require any compiler or build > toolchain on user's machines. But in the enterprise space its a no-no to use the wheels you build. I get your source, audit it and build that for use in our environment. Barry > > Alex > > On Tue, Jan 12, 2021 at 12:17 PM Barry Scott wrote: > > > > On Tuesday, 12 January 2021 15:52:01 GMT Michael Str?der via Cryptography-dev wrote: > > > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > > > As we previewed in August [0] we're planning to incorporate Rust code > > > > into pyca/cryptography. > > > > > > IMHO this will make life of distro packagers more miserable especially > > > on non-x86 platforms. > > > > I was also concerned by that new toolset dependency. > > > > Will this build on Centos 8 with the version of rust that is packaged there? > > > > Barry > > > > > > > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > From paul.l.kehrer at gmail.com Wed Jan 13 11:55:53 2021 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Wed, 13 Jan 2021 10:55:53 -0600 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: <1820978.taCxCBeP46@fpbarry> References: <3407119.R56niFO833@fpbarry> <1820978.taCxCBeP46@fpbarry> Message-ID: We're definitely aware that there are a variety of consumers that aren't visible in the limited metrics available to us. Maintaining the status quo is always the "best" path in the short term for our users, as it removes the need to make alterations of any kind. However, that stance is incompatible with moving the broader ecosystem towards better solutions, so the reality is that we need to balance our desire to have a maintainable system and improve the project with myriad other needs. As a packager of our project I would expect, when encountering news of a change in the build system, that you would clone the current repository, experiment with it in your environment, and let us know if there's some issue beyond simple problems like needing to install a compatible rustc. Thus far it's unclear to me what action you want this project to take in response to your statements in this thread. We believe incorporating Rust into cryptography is in the best long-term interest of the project for users and developers. For example, we can provide high performance memory safe implementations of parsers through our Rust bindings and we'll be able to implement support for some things that were previously not practical (arbitrary ASN.1 parsing is extremely unpleasant in OpenSSL). We have no intention of replacing OpenSSL itself, as it provides a variety of advantages (especially in the realm of compliance) to our users, but Rust code will augment the core over time. To minimize the pain on the subset of users that need to compile cryptography themselves we have announced this in multiple venues and have developed a plan to ensure minimal impact. We recognize that there's no truly effective means of communication with all of our users other than shipping code, so our upcoming 3.4 release allows you to entirely disable the Rust requirement *without affecting functionality in any way*. This allows packagers who haven't seen our previous announcements additional time to determine the best way to properly support a hard Rust dependency. If you have additional suggestions on how we can more effectively communicate this transition we would be happy to discuss them. The only hard statement is that we do not intend to turn back from this path because we believe it would be irresponsible to do so. A safe library must attempt to use memory safe languages where feasible, and performance is a critical feature to many of our users. -Paul Kehrer (reaperhulk) On Wed, Jan 13, 2021 at 8:51 AM Barry Scott wrote: > > On Wednesday, 13 January 2021 14:47:37 GMT Alex Gaynor wrote: > > I'm glad to hear some folks are actually auditing 3rd party sources. > > > > Once you install a rust toolchain, you'll be able to build > > pyca/cryptography the same as ever. > > > > To repeat: if there's some action we can be taking to make this > > migration smoother, we're happy to consider it. But what we won't do > > is simply stop trying to drop C. > > I'm all for better code. > > I just wanted to point out that you have user that do not show up in PyPI stats. > > Barry > > > > > > Alex > > > > On Wed, Jan 13, 2021 at 9:45 AM Barry Scott wrote: > > > > > > On Tuesday, 12 January 2021 17:23:10 GMT Alex Gaynor wrote: > > > > Running `yum install rust` in a CentOS8 docker container seems to get > > > > me rustc 1.45.2, and as our docs say, 1.45.0 will be the initial > > > > minimum version > > > > (https://cryptography.io/en/latest/installation.html#rust). > > > > > > > > As ever, our wheels (which are how the vast majority of our users > > > > install pyca/cryptography) will not require any compiler or build > > > > toolchain on user's machines. > > > > > > But in the enterprise space its a no-no to use the wheels you build. > > > > > > I get your source, audit it and build that for use in our environment. > > > > > > Barry > > > > > > > > > > > Alex > > > > > > > > On Tue, Jan 12, 2021 at 12:17 PM Barry Scott wrote: > > > > > > > > > > On Tuesday, 12 January 2021 15:52:01 GMT Michael Str?der via Cryptography-dev wrote: > > > > > > On 12/22/20 8:43 PM, Alex Gaynor wrote: > > > > > > > As we previewed in August [0] we're planning to incorporate Rust code > > > > > > > into pyca/cryptography. > > > > > > > > > > > > IMHO this will make life of distro packagers more miserable especially > > > > > > on non-x86 platforms. > > > > > > > > > > I was also concerned by that new toolset dependency. > > > > > > > > > > Will this build on Centos 8 with the version of rust that is packaged there? > > > > > > > > > > Barry > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Cryptography-dev mailing list > > > > > Cryptography-dev at python.org > > > > > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From hs at ox.cx Tue Jan 19 01:55:44 2021 From: hs at ox.cx (Hynek Schlawack) Date: Tue, 19 Jan 2021 07:55:44 +0100 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> Message-ID: My understanding from (the stalled) https://github.com/pypa/manylinux/issues/84 and Anthony?s comments is that the problem is a lack of a clear ARM command set standard/lack of wheel tags? Or am I missing something? > On 13. Jan 2021, at 14:37, Alex Gaynor wrote: > > https://www.python.org/dev/peps/pep-0599/ :-) > > Alex > > On Wed, Jan 13, 2021 at 2:49 AM Hynek Schlawack wrote: >> >> >> >>> On 12. Jan 2021, at 19:05, Alex Gaynor wrote: >>> >>> At the moment we provide wheels for: >>> >>> - x86_64, arm64 Linux >> >> This is interesting, I?ve been told ( et ff) that there?s no arm64 wheel standard yet and thus arm64 wheels are likely to cause problems. >> >> Have there been any new development in that regard? Maybe a discussion I can read up? >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > > > > -- > All that is necessary for evil to succeed is for good people to do nothing. > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From paul.l.kehrer at gmail.com Tue Jan 19 14:05:33 2021 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Tue, 19 Jan 2021 13:05:33 -0600 Subject: [Cryptography-dev] Rust in pyca/cryptography In-Reply-To: References: <233bdf04-8216-0b31-c96d-f7203d6d28a2@stroeder.com> <2453147.Lt9SDvczpP@fpbarry> <66f3a0fb-5107-a425-8308-f1d25c393ad0@stroeder.com> Message-ID: PEP 599 expanded the manylinux standard to include armv7l and aarch64. That is/was called manylinux2014, although subsequently PEP 600 defined a new glibc version based tagging system which negates the need to ship new pip versions that understand new tags. So you can ship arm wheels now (albeit only those two arm ABI variants). We ship aarch64 for cryptography right now. -Paul On Tue, Jan 19, 2021 at 12:55 AM Hynek Schlawack wrote: > > My understanding from (the stalled) https://github.com/pypa/manylinux/issues/84 and Anthony?s comments is that the problem is a lack of a clear ARM command set standard/lack of wheel tags? Or am I missing something? > > > On 13. Jan 2021, at 14:37, Alex Gaynor wrote: > > > > https://www.python.org/dev/peps/pep-0599/ :-) > > > > Alex > > > > On Wed, Jan 13, 2021 at 2:49 AM Hynek Schlawack wrote: > >> > >> > >> > >>> On 12. Jan 2021, at 19:05, Alex Gaynor wrote: > >>> > >>> At the moment we provide wheels for: > >>> > >>> - x86_64, arm64 Linux > >> > >> This is interesting, I?ve been told ( et ff) that there?s no arm64 wheel standard yet and thus arm64 wheels are likely to cause problems. > >> > >> Have there been any new development in that regard? Maybe a discussion I can read up? > >> _______________________________________________ > >> Cryptography-dev mailing list > >> Cryptography-dev at python.org > >> https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > > > > -- > > All that is necessary for evil to succeed is for good people to do nothing. > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev