From alex.gaynor at gmail.com Sat Sep 7 21:16:15 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 7 Sep 2013 12:16:15 -0700 Subject: [Cryptography-dev] Let the flood gates open! Message-ID: Hynek just merged the AES patch, so now we have some actual crypto. I think it's time for us to get moving forwards: * More ciphers / modes (particularly I think a bunch of us want to make GCM a thing) * Starting to think about primitives that aren't block ciphers Let's do it! Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jarret.raim at RACKSPACE.COM Sat Sep 7 21:18:23 2013 From: jarret.raim at RACKSPACE.COM (Jarret Raim) Date: Sat, 7 Sep 2013 19:18:23 +0000 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: References: Message-ID: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> I have a long plane flight coming up. Any low hanging fruit I can help with? Maybe something where the interface is simple / settled? Thanks, Jarret Raim Sent from my phone, please excuse the mistakes. On Sep 7, 2013, at 2:16 PM, "Alex Gaynor" wrote: > Hynek just merged the AES patch, so now we have some actual crypto. I think it's time for us to get moving forwards: > > * More ciphers / modes (particularly I think a bunch of us want to make GCM a thing) > * Starting to think about primitives that aren't block ciphers > > Let's do it! > Alex > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From alex.gaynor at gmail.com Sat Sep 7 21:19:43 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 7 Sep 2013 12:19:43 -0700 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> Message-ID: I think we're basically agreed on the API for symmetric ciphers. Good places to jump in might be: exposing more algorithms, or implementing the decrypt() pathway! Alex On Sat, Sep 7, 2013 at 12:18 PM, Jarret Raim wrote: > I have a long plane flight coming up. Any low hanging fruit I can help > with? > > Maybe something where the interface is simple / settled? > > Thanks, > Jarret Raim > > Sent from my phone, please excuse the mistakes. > > On Sep 7, 2013, at 2:16 PM, "Alex Gaynor" wrote: > > > Hynek just merged the AES patch, so now we have some actual crypto. I > think it's time for us to get moving forwards: > > > > * More ciphers / modes (particularly I think a bunch of us want to make > GCM a thing) > > * Starting to think about primitives that aren't block ciphers > > > > Let's do it! > > Alex > > > > -- > > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > > "The people's good is the highest law." -- Cicero > > GPG Key fingerprint: 125F 5C67 DFE9 4084 > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Sun Sep 8 05:42:37 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 7 Sep 2013 20:42:37 -0700 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> Message-ID: So, Twisted has an implementation of the SSH protocol, that currently uses PyCrypto. I got curious what things it needed to be ported to cryptography, here's the feature list I came up with: * Keypair generation for both RSA and DSA * General manipulation of RSA and DSA (exact nature unchecked) * Conert a long to a bytestring with its twos complement representation (out of scope) * Convert a twos complement representation of a long into a long (out of scope) * DES3 for block ciphers * Decryption for block ciphers * CTR mode * Blowfish * CAST Alex On Sat, Sep 7, 2013 at 12:19 PM, Alex Gaynor wrote: > I think we're basically agreed on the API for symmetric ciphers. Good > places to jump in might be: exposing more algorithms, or implementing the > decrypt() pathway! > > Alex > > > On Sat, Sep 7, 2013 at 12:18 PM, Jarret Raim wrote: > >> I have a long plane flight coming up. Any low hanging fruit I can help >> with? >> >> Maybe something where the interface is simple / settled? >> >> Thanks, >> Jarret Raim >> >> Sent from my phone, please excuse the mistakes. >> >> On Sep 7, 2013, at 2:16 PM, "Alex Gaynor" wrote: >> >> > Hynek just merged the AES patch, so now we have some actual crypto. I >> think it's time for us to get moving forwards: >> > >> > * More ciphers / modes (particularly I think a bunch of us want to make >> GCM a thing) >> > * Starting to think about primitives that aren't block ciphers >> > >> > Let's do it! >> > Alex >> > >> > -- >> > "I disapprove of what you say, but I will defend to the death your >> right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >> > "The people's good is the highest law." -- Cicero >> > GPG Key fingerprint: 125F 5C67 DFE9 4084 >> > _______________________________________________ >> > Cryptography-dev mailing list >> > Cryptography-dev at python.org >> > https://mail.python.org/mailman/listinfo/cryptography-dev >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > > > > -- > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Sun Sep 8 17:21:07 2013 From: donald at stufft.io (Donald Stufft) Date: Sun, 8 Sep 2013 11:21:07 -0400 Subject: [Cryptography-dev] Operating System Support Message-ID: So I don't think we ever clarified this but what Operating Systems are we planning on supporting? For me I think *nix and Windows should be supported. The other question is are we ok with some things only working on a particular OS? If say we implement bcrypt and the library for that doesn't work on Windows (And assuming we support Windows) is that Ok? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From christian at python.org Sun Sep 8 17:35:36 2013 From: christian at python.org (Christian Heimes) Date: Sun, 08 Sep 2013 17:35:36 +0200 Subject: [Cryptography-dev] Operating System Support In-Reply-To: References: Message-ID: <522C9948.5040702@python.org> Am 08.09.2013 17:21, schrieb Donald Stufft: > So I don't think we ever clarified this but what Operating Systems > are we planning on supporting? > > For me I think *nix and Windows should be supported. > > The other question is are we ok with some things only working on a > particular OS? If say we implement bcrypt and the library for that > doesn't work on Windows (And assuming we support Windows) is that > Ok? The definition is too broad. *nix is about anything that is not Windows. We should also mention supported CPU architectures. Realistic we could support: Windows Vista, Win 7, Win 8 on X86 and X86_64 Linux recent distributions on X86, X86_64 and ARM with GCC or clang. FreeBSD X86 and X86_64 Mac OS X X86_64 From alex.gaynor at gmail.com Sun Sep 8 17:36:38 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sun, 8 Sep 2013 08:36:38 -0700 Subject: [Cryptography-dev] Operating System Support In-Reply-To: <522C9948.5040702@python.org> References: <522C9948.5040702@python.org> Message-ID: I think what Christain said is a reasonable goal, in practice I think until we have a thing people actually want to use, the "stuff that works" is going to be stuff we have an active developer using or which we have a buildbot for. Alex On Sun, Sep 8, 2013 at 8:35 AM, Christian Heimes wrote: > Am 08.09.2013 17:21, schrieb Donald Stufft: > > So I don't think we ever clarified this but what Operating Systems > > are we planning on supporting? > > > > For me I think *nix and Windows should be supported. > > > > The other question is are we ok with some things only working on a > > particular OS? If say we implement bcrypt and the library for that > > doesn't work on Windows (And assuming we support Windows) is that > > Ok? > > The definition is too broad. *nix is about anything that is not > Windows. We should also mention supported CPU architectures. > > Realistic we could support: > > Windows > Vista, Win 7, Win 8 on X86 and X86_64 > > Linux > recent distributions on X86, X86_64 and ARM with GCC or clang. > > FreeBSD > X86 and X86_64 > > Mac OS X > X86_64 > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Sun Sep 8 17:38:24 2013 From: donald at stufft.io (Donald Stufft) Date: Sun, 8 Sep 2013 11:38:24 -0400 Subject: [Cryptography-dev] Operating System Support In-Reply-To: References: <522C9948.5040702@python.org> Message-ID: On Sep 8, 2013, at 11:36 AM, Alex Gaynor wrote: > I think what Christain said is a reasonable goal, in practice I think until we have a thing people actually want to use, the "stuff that works" is going to be stuff we have an active developer using or which we have a buildbot for. > > Alex > > > On Sun, Sep 8, 2013 at 8:35 AM, Christian Heimes wrote: > Am 08.09.2013 17:21, schrieb Donald Stufft: > > So I don't think we ever clarified this but what Operating Systems > > are we planning on supporting? > > > > For me I think *nix and Windows should be supported. > > > > The other question is are we ok with some things only working on a > > particular OS? If say we implement bcrypt and the library for that > > doesn't work on Windows (And assuming we support Windows) is that > > Ok? > > The definition is too broad. *nix is about anything that is not > Windows. We should also mention supported CPU architectures. > > Realistic we could support: > > Windows > Vista, Win 7, Win 8 on X86 and X86_64 > > Linux > recent distributions on X86, X86_64 and ARM with GCC or clang. > > FreeBSD > X86 and X86_64 > > Mac OS X > X86_64 > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev Well I ask because it influences decisions and because we can get buildbots :D ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hs at ox.cx Mon Sep 9 08:01:08 2013 From: hs at ox.cx (Hynek Schlawack) Date: Mon, 9 Sep 2013 08:01:08 +0200 Subject: [Cryptography-dev] Operating System Support In-Reply-To: References: Message-ID: <5429930A-BE4E-4BF8-A12E-5C84DC949720@ox.cx> Am 08.09.2013 um 17:21 schrieb Donald Stufft : > So I don't think we ever clarified this but what Operating Systems are we planning on supporting? > > For me I think *nix and Windows should be supported. > > The other question is are we ok with some things only working on a particular OS? If say we implement bcrypt and the library for that doesn't work on Windows (And assuming we support Windows) is that Ok? Additionally to what has been said, we may have a look at projects that we?d like eventually use cryptography as a backend and try to get their platforms backed. Spontaneously, I?d think of: ? pyopenssl ? passlib -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jean-paul at hybridcluster.com Mon Sep 9 13:18:34 2013 From: jean-paul at hybridcluster.com (Jean-Paul Calderone) Date: Mon, 09 Sep 2013 07:18:34 -0400 Subject: [Cryptography-dev] Operating System Support In-Reply-To: <5429930A-BE4E-4BF8-A12E-5C84DC949720@ox.cx> References: <5429930A-BE4E-4BF8-A12E-5C84DC949720@ox.cx> Message-ID: <522DAE8A.5040000@hybridcluster.com> On 09/09/2013 02:01 AM, Hynek Schlawack wrote: > Am 08.09.2013 um 17:21 schrieb Donald Stufft : > >> So I don't think we ever clarified this but what Operating Systems are we planning on supporting? >> >> For me I think *nix and Windows should be supported. >> >> The other question is are we ok with some things only working on a particular OS? If say we implement bcrypt and the library for that doesn't work on Windows (And assuming we support Windows) is that Ok? > Additionally to what has been said, we may have a look at projects that we'd like eventually use cryptography as a backend and try to get their platforms backed. > > Spontaneously, I'd think of: > > -- pyopenssl pyOpenSSL is understaffed for serious cross-platform support. I am basically maintaining it such that Twisted's use of it keeps working. So this makes pyOpenSSL's supported platform list the same as Twisted's supported platform list. This can be found along the left-hand side of http://buildbot.twistedmatrix.com/boxes-supported?branch=trunk&num_builds=10 (sorry, the page seems to be taking ages to load these days). The short list is something like "some Linuxes, some OS Xes, some Windowses, some FreeBSDs". Jean-Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From hs at ox.cx Mon Sep 9 14:04:02 2013 From: hs at ox.cx (Hynek Schlawack) Date: Mon, 9 Sep 2013 14:04:02 +0200 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> Message-ID: <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> I?d think concentrating on giving JP what he needs for pyopenssl might be less work with quicker and actually *useful* results? Am 08.09.2013 um 05:42 schrieb Alex Gaynor : > So, Twisted has an implementation of the SSH protocol, that currently uses PyCrypto. I got curious what things it needed to be ported to cryptography, here's the feature list I came up with: > > * Keypair generation for both RSA and DSA > * General manipulation of RSA and DSA (exact nature unchecked) > * Conert a long to a bytestring with its twos complement representation (out of > scope) > * Convert a twos complement representation of a long into a long (out of scope) > * DES3 for block ciphers > * Decryption for block ciphers > * CTR mode > * Blowfish > * CAST > > Alex > > > On Sat, Sep 7, 2013 at 12:19 PM, Alex Gaynor wrote: > I think we're basically agreed on the API for symmetric ciphers. Good places to jump in might be: exposing more algorithms, or implementing the decrypt() pathway! > > Alex > > > On Sat, Sep 7, 2013 at 12:18 PM, Jarret Raim wrote: > I have a long plane flight coming up. Any low hanging fruit I can help with? > > Maybe something where the interface is simple / settled? > > Thanks, > Jarret Raim > > Sent from my phone, please excuse the mistakes. > > On Sep 7, 2013, at 2:16 PM, "Alex Gaynor" wrote: > > > Hynek just merged the AES patch, so now we have some actual crypto. I think it's time for us to get moving forwards: > > > > * More ciphers / modes (particularly I think a bunch of us want to make GCM a thing) > > * Starting to think about primitives that aren't block ciphers > > > > Let's do it! > > Alex > > > > -- > > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > > "The people's good is the highest law." -- Cicero > > GPG Key fingerprint: 125F 5C67 DFE9 4084 > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From christian at python.org Mon Sep 9 14:06:28 2013 From: christian at python.org (Christian Heimes) Date: Mon, 09 Sep 2013 14:06:28 +0200 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> Message-ID: <522DB9C4.3070305@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Am 09.09.2013 14:04, schrieb Hynek Schlawack: > I?d think concentrating on giving JP what he needs for pyopenssl > might be less work with quicker and actually *useful* results? Please don't underestimate X.509, CRL and TLS/SSL. It's much, *much* more work than a couple of block ciphers and hash algorithms. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCgAGBQJSLbm6AAoJEMeIxMHUVQ1FBsIQAITleEvRx1KIXUvrLFzDB4ZS b0D/r1yKa4YD7o/uj2s/Crb41IFMTpVdZ3ePWj/0KAAGi7V/UicrayX24faFjy/w dfnK6pSsA0UfrKSr/gEG4r+vf+Yr7kZBpGzcsULuXzv5NceLnz2eRlbUEgM6V6lN tuc0uatSTK5LA65yqXa837cr5tjTLoNzXtbJzHBUqP+MsX2Cb0t+uL7vFTBMzjVR xXolnPc1ogbCtUZQZWbt4yYGW/cX/eYm+xt8g87rZuJSkScrvVDBVxhHBkXBjH0T TiAUGotaUCNa8jTxaddEeP/GXBCdlv0J5HQ6+KhFtXD6QeCjMBePyZ62LvhF9uZN esQLr/OiihZSI88wt6Zq5EZLhuUfJEbepj3bbi5w9NLFx4mgc8cnovR0yLyVOdtG 2Ekawesg4zEpUbGy6WM90wPECeFVMr3JkuPm0t6XUk6UeIWh6TOZ3VH8XqD3vvCm //gRPHtcMr+TMr6lXSE0WTw28+Xwv7wDVn1TOJtkw5rO8iJFZcomXxGE/T8Hs7mc sjBTGupvWhUSNP7LmcSu/E2E6N+V+0KAzUA0yrWeGM0tpfu4ABuOu5pLwhbTCpt4 SqulgWaekPw5/syx4dn2UWru5qhHc/qXScQIEmF7+fuYCnb9dw2duuHhtRGKfLbu 60x0PG9oqvIGzLD+MuNF =nu2k -----END PGP SIGNATURE----- From jean-paul at hybridcluster.com Mon Sep 9 14:12:01 2013 From: jean-paul at hybridcluster.com (Jean-Paul Calderone) Date: Mon, 09 Sep 2013 08:12:01 -0400 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <522DB9C4.3070305@python.org> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> <522DB9C4.3070305@python.org> Message-ID: <522DBB11.1010302@hybridcluster.com> On 09/09/2013 08:06 AM, Christian Heimes wrote: > Am 09.09.2013 14:04, schrieb Hynek Schlawack: > > I?d think concentrating on giving JP what he needs for pyopenssl > > might be less work with quicker and actually *useful* results? > > Please don't underestimate X.509, CRL and TLS/SSL. It's much, *much* > more work than a couple of block ciphers and hash algorithms. > True. On the other hand, the project that `cryptography? killed, `opentls?, already had cffi bindings for the majority of the APIs pyOpenSSL requires and there was a branch I was working on getting merged into master that provided 100% of the necessary APIs. Now that there is any code for OpenSSL bindings at all, I took a look at what might make sense as a next step to get some of the functionality pyOpenSSL needs. I was a little discouraged by the structure of the code which looks like it is much less amenable to improvement and maintenance than the code from opentls. opentls had its problems but it was nice that it tried to split the necessary cffi declarations up a bit. I'd like to see something like this happen to the structure of the openssl bindings in cryptography. Jean-Paul -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From donald at stufft.io Mon Sep 9 15:17:23 2013 From: donald at stufft.io (Donald Stufft) Date: Mon, 9 Sep 2013 09:17:23 -0400 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <522DBB11.1010302@hybridcluster.com> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> <522DB9C4.3070305@python.org> <522DBB11.1010302@hybridcluster.com> Message-ID: <0BA49D58-C3BF-401D-A324-64DD11D53EEA@stufft.io> On Sep 9, 2013, at 8:12 AM, Jean-Paul Calderone wrote: > I was a little discouraged by the structure of the > code which looks like it is much less amenable to improvement and > maintenance than the code from opentls. opentls had its problems but it > was nice that it tried to split the necessary cffi declarations up a > bit. I'd like to see something like this happen to the structure of the > openssl bindings in cryptography. I liked what I saw from opentls with how it structured the cffi bindings, I would be +1 on something similar. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From hs at ox.cx Mon Sep 9 23:09:02 2013 From: hs at ox.cx (Hynek Schlawack) Date: Mon, 9 Sep 2013 23:09:02 +0200 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <0BA49D58-C3BF-401D-A324-64DD11D53EEA@stufft.io> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> <522DB9C4.3070305@python.org> <522DBB11.1010302@hybridcluster.com> <0BA49D58-C3BF-401D-A324-64DD11D53EEA@stufft.io> Message-ID: <395D2219-B00F-48A8-84EA-B60A8BC8FC19@ox.cx> Well, let's keep this conversation going then. What are the current issues? Sent from my phone. Am 09.09.2013 um 15:17 schrieb Donald Stufft : > > On Sep 9, 2013, at 8:12 AM, Jean-Paul Calderone wrote: > >> I was a little discouraged by the structure of the >> code which looks like it is much less amenable to improvement and >> maintenance than the code from opentls. opentls had its problems but it >> was nice that it tried to split the necessary cffi declarations up a >> bit. I'd like to see something like this happen to the structure of the >> openssl bindings in cryptography. > > I liked what I saw from opentls with how it structured the cffi bindings, I would be +1 on something similar. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev From alex.gaynor at gmail.com Mon Sep 9 23:10:10 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Mon, 9 Sep 2013 14:10:10 -0700 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <395D2219-B00F-48A8-84EA-B60A8BC8FC19@ox.cx> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> <522DB9C4.3070305@python.org> <522DBB11.1010302@hybridcluster.com> <0BA49D58-C3BF-401D-A324-64DD11D53EEA@stufft.io> <395D2219-B00F-48A8-84EA-B60A8BC8FC19@ox.cx> Message-ID: +1 to a better structure for the OpenSSL bindings, I'm not sure OpenTLS has it totally right, but something more organized would be good. Alex On Mon, Sep 9, 2013 at 2:09 PM, Hynek Schlawack wrote: > Well, let's keep this conversation going then. What are the current > issues? > > Sent from my phone. > > Am 09.09.2013 um 15:17 schrieb Donald Stufft : > > > > > On Sep 9, 2013, at 8:12 AM, Jean-Paul Calderone < > jean-paul at hybridcluster.com> wrote: > > > >> I was a little discouraged by the structure of the > >> code which looks like it is much less amenable to improvement and > >> maintenance than the code from opentls. opentls had its problems but it > >> was nice that it tried to split the necessary cffi declarations up a > >> bit. I'd like to see something like this happen to the structure of the > >> openssl bindings in cryptography. > > > > I liked what I saw from opentls with how it structured the cffi > bindings, I would be +1 on something similar. > > > > ----------------- > > Donald Stufft > > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jarret.raim at RACKSPACE.COM Mon Sep 9 23:20:41 2013 From: jarret.raim at RACKSPACE.COM (Jarret Raim) Date: Mon, 9 Sep 2013 21:20:41 +0000 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: Message-ID: What about combining the two approaches. We currently have: bindings/openssl api.py Where api.py includes both the CFFI code and the API methods themselves. The main api.py stays the same, but contains only high-level, generic API calls and imports the lower level API methods from the api package. We add two new packages: Bindings/openssl/api Bindings/openssl/cffi (need better name?) The api package would allow us to break up the api definitions. Something like block.py, tls.py, hmac.py, etc. The cffi package would contain the CFFI definitions. We don't have to decompose as much as opentls did, but it would be similar. We could also then pull over most of the work that was already done in opentls. These bindings would then be imported by the api package. Just a straw man suggestion. I'm pretty new to openssl development and I'm mostly a ruby dev with a little experience in Python. Poke away. If its valuable, I could mock out something tonight / tomorrow and we can argue about it. Jarret From: Alex Gaynor > Reply-To: "cryptography-dev at python.org" > Date: Monday, September 9, 2013 4:10 PM To: "cryptography-dev at python.org" > Subject: Re: [Cryptography-dev] Let the flood gates open! +1 to a better structure for the OpenSSL bindings, I'm not sure OpenTLS has it totally right, but something more organized would be good. Alex On Mon, Sep 9, 2013 at 2:09 PM, Hynek Schlawack > wrote: Well, let's keep this conversation going then. What are the current issues? Sent from my phone. Am 09.09.2013 um 15:17 schrieb Donald Stufft >: > > On Sep 9, 2013, at 8:12 AM, Jean-Paul Calderone > wrote: > >> I was a little discouraged by the structure of the >> code which looks like it is much less amenable to improvement and >> maintenance than the code from opentls. opentls had its problems but it >> was nice that it tried to split the necessary cffi declarations up a >> bit. I'd like to see something like this happen to the structure of the >> openssl bindings in cryptography. > > I liked what I saw from opentls with how it structured the cffi bindings, I would be +1 on something similar. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev _______________________________________________ Cryptography-dev mailing list Cryptography-dev at python.org https://mail.python.org/mailman/listinfo/cryptography-dev -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-paul at hybridcluster.com Mon Sep 9 23:21:50 2013 From: jean-paul at hybridcluster.com (Jean-Paul Calderone) Date: Mon, 09 Sep 2013 17:21:50 -0400 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <395D2219-B00F-48A8-84EA-B60A8BC8FC19@ox.cx> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> <522DB9C4.3070305@python.org> <522DBB11.1010302@hybridcluster.com> <0BA49D58-C3BF-401D-A324-64DD11D53EEA@stufft.io> <395D2219-B00F-48A8-84EA-B60A8BC8FC19@ox.cx> Message-ID: <522E3BEE.8010302@hybridcluster.com> On 09/09/2013 05:09 PM, Hynek Schlawack wrote: > Well, let's keep this conversation going then. What are the current issues? All of the cffi cdef text is inline in a single string in a method in cryptography/bindings/openssl/api.py. This will be a problem (I suspect) once the string grows to the necessary several thousand lines. I suggest splitting this up by OpenSSL header file and by declaration type. For example, all of the declarations for symbols (or whatever they're called) in evp.h should go into cryptography/bindings/openssl/evp.py. They should be further split up into a few different variables: * the name of the header (or, sometimes, headers) to include * the types to define * the functions to define * extra arbitrary C code for glue api.py should assemble all of this information and make the necessary cffi API calls. Jean-Paul > Sent from my phone. > > Am 09.09.2013 um 15:17 schrieb Donald Stufft : > >> On Sep 9, 2013, at 8:12 AM, Jean-Paul Calderone wrote: >> >>> I was a little discouraged by the structure of the >>> code which looks like it is much less amenable to improvement and >>> maintenance than the code from opentls. opentls had its problems but it >>> was nice that it tried to split the necessary cffi declarations up a >>> bit. I'd like to see something like this happen to the structure of the >>> openssl bindings in cryptography. >> I liked what I saw from opentls with how it structured the cffi bindings, I would be +1 on something similar. >> >> ----------------- >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From alex.gaynor at gmail.com Mon Sep 9 23:40:32 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Mon, 9 Sep 2013 14:40:32 -0700 Subject: [Cryptography-dev] Let the flood gates open! In-Reply-To: <522E3BEE.8010302@hybridcluster.com> References: <08088693-753C-4EA4-A32F-BB0D5B697FC3@RACKSPACE.COM> <259BBB2B-E4B9-4680-8399-D6B390284FEF@ox.cx> <522DB9C4.3070305@python.org> <522DBB11.1010302@hybridcluster.com> <0BA49D58-C3BF-401D-A324-64DD11D53EEA@stufft.io> <395D2219-B00F-48A8-84EA-B60A8BC8FC19@ox.cx> <522E3BEE.8010302@hybridcluster.com> Message-ID: Sounds pretty reasonable to me, further, the implementation of functions for different modules in primitives don't need to be instance methods on an API object, they can be module functions in those modules which take an api instance as the first parameter. Alex On Mon, Sep 9, 2013 at 2:21 PM, Jean-Paul Calderone < jean-paul at hybridcluster.com> wrote: > On 09/09/2013 05:09 PM, Hynek Schlawack wrote: > > Well, let's keep this conversation going then. What are the current > issues? > > All of the cffi cdef text is inline in a single string in a method in > cryptography/bindings/openssl/api.py. > > This will be a problem (I suspect) once the string grows to the > necessary several thousand lines. > > I suggest splitting this up by OpenSSL header file and by declaration type. > > For example, all of the declarations for symbols (or whatever they're > called) in evp.h should go into cryptography/bindings/openssl/evp.py. > They should be further split up into a few different variables: > > * the name of the header (or, sometimes, headers) to include > * the types to define > * the functions to define > * extra arbitrary C code for glue > > api.py should assemble all of this information and make the necessary > cffi API calls. > > Jean-Paul > > > Sent from my phone. > > > > Am 09.09.2013 um 15:17 schrieb Donald Stufft : > > > >> On Sep 9, 2013, at 8:12 AM, Jean-Paul Calderone < > jean-paul at hybridcluster.com> wrote: > >> > >>> I was a little discouraged by the structure of the > >>> code which looks like it is much less amenable to improvement and > >>> maintenance than the code from opentls. opentls had its problems but > it > >>> was nice that it tried to split the necessary cffi declarations up a > >>> bit. I'd like to see something like this happen to the structure of > the > >>> openssl bindings in cryptography. > >> I liked what I saw from opentls with how it structured the cffi > bindings, I would be +1 on something similar. > >> > >> ----------------- > >> Donald Stufft > >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > >> > >> _______________________________________________ > >> Cryptography-dev mailing list > >> Cryptography-dev at python.org > >> https://mail.python.org/mailman/listinfo/cryptography-dev > > _______________________________________________ > > Cryptography-dev mailing list > > Cryptography-dev at python.org > > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.l.kehrer at gmail.com Wed Sep 11 00:25:48 2013 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Tue, 10 Sep 2013 17:25:48 -0500 Subject: [Cryptography-dev] CFB Modes Message-ID: I've been implementing some of the basic modes the past two days. I'm going to be putting in a PR for CFB/OFB soon, but wanted to see if people have any opinions about supporting the 1-bit and 8-bit CFB modes. There are two ways we can approach this. The easy way just means we declare two additional mode classes (CFB1 and CFB8) and those will correspond to 1-bit and 8-bit CFB modes that OpenSSL supports in addition to the standard CFB mode (which has a bit length of the block cipher's block size). The alternate way would be to create a single class: class CFB(object): def __init__(self, initialization_vector, bit_length=None): super(CFB, self).__init__() self.initialization_vector = initialization_vector self._determine_mode(bit_length) def _determine_mode(self, bit_length): if bit_length is None: self.name = 'CFB' elif int(bit_length) == 8 self.name = 'CFB8' elif int(bit_length) == 1 self.name = 'CFB1' else: raise ValueError According to NIST SP800-38A CFB can technically take an integer parameter 1 <= s <= b where b is the block length, but OpenSSL only supports 1, 8, and block length. Other backends we may choose to implement might be more flexible, but the _determine_mode method above is OpenSSL specific. Which approach do we prefer/is there another path we should consider looking at? From donald at stufft.io Wed Sep 11 00:33:06 2013 From: donald at stufft.io (Donald Stufft) Date: Tue, 10 Sep 2013 18:33:06 -0400 Subject: [Cryptography-dev] CFB Modes In-Reply-To: References: Message-ID: On Sep 10, 2013, at 6:25 PM, Paul Kehrer wrote: > I've been implementing some of the basic modes the past two days. I'm > going to be putting in a PR for CFB/OFB soon, but wanted to see if > people have any opinions about supporting the 1-bit and 8-bit CFB > modes. > > There are two ways we can approach this. The easy way just means we > declare two additional mode classes (CFB1 and CFB8) and those will > correspond to 1-bit and 8-bit CFB modes that OpenSSL supports in > addition to the standard CFB mode (which has a bit length of the block > cipher's block size). > > The alternate way would be to create a single class: > > class CFB(object): > def __init__(self, initialization_vector, bit_length=None): > super(CFB, self).__init__() > self.initialization_vector = initialization_vector > self._determine_mode(bit_length) > > def _determine_mode(self, bit_length): > if bit_length is None: > self.name = 'CFB' > elif int(bit_length) == 8 > self.name = 'CFB8' > elif int(bit_length) == 1 > self.name = 'CFB1' > else: > raise ValueError > > According to NIST SP800-38A CFB can technically take an integer > parameter 1 <= s <= b where b is the block length, but OpenSSL only > supports 1, 8, and block length. Other backends we may choose to > implement might be more flexible, but the _determine_mode method above > is OpenSSL specific. If CFB can technically take any integer between 1 and block size then we should just have it take an integer bit length. The class should not have anything OpenSSL specific inside it. Instead it should expose enough information for the OpenSSL API to determine which CFB to use. We should not write our external API to conform to OpenSSL. > > Which approach do we prefer/is there another path we should consider looking at? > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From donald at stufft.io Thu Sep 12 04:19:49 2013 From: donald at stufft.io (Donald Stufft) Date: Wed, 11 Sep 2013 22:19:49 -0400 Subject: [Cryptography-dev] "Streaming" APIs Message-ID: So one thing that's really handy to do with encryption is to be able to (de|en)crypt things without needing to load the entire thing into memory. Currently we support this in the encrypt API by doing: cipher = BlockCipher() enciphered = cipher.encrypt(block1) enciphered += cipher.encrypt(block2) enciphered += cipher.encrypt(block3) enciphered += cipher.finalize() We needed to do this because we need to be able to call finalize() before the encryption is "done". When I was messing with padding I ended up with an API that (for padding) got around the need for an explicit finalize step but instead it required passing the entire data stream into the function. However it supports generators/iterators so you can still efficiently process large datasets. This api looks something like padder = Padding() padded1 = "".join(padder.pad("1234")) padded2 = "".join(padder.pad(c for c in "1234")) However the downside of this API is that You need to call "".join() to get actual strings or you need to do some ugly hacks inside of the pad() function so it returns a string if given a string and returns a generator if given a generator. A third option is similar to dictionaries on Python 2.x where you have something like iterpad() and pad(). This could work for encryption as well so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt(). So I guess the question is how do we want to handle these streaming sorts of APIs? 1) Thing.action() + Thing.finalize() 2a) "".join(Thing.action(iterator)) 2b) Thing.action(terator OR string) - Magic return types 3) Thing.action and Thing.iter_action 4) ???? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From donald at stufft.io Fri Sep 13 20:22:57 2013 From: donald at stufft.io (Donald Stufft) Date: Fri, 13 Sep 2013 14:22:57 -0400 Subject: [Cryptography-dev] "For Humans" Message Layout Message-ID: So I've been looking at playing around with the start of the "for humans" API. I figure we need to either do a PyNaCl style "this is what the algorithms are and that's that" or a more flexible layout. I've opted to make the format itself flexible. So here's my proposal for a the message layout for an encrypted message: https://gist.github.com/dstufft/62473d7ae4b6f8b83577 I tried to make it flexible, it makes it possible for the format itself to specify things such as the algorithm, MAC, etc. The protocol is versioned, up to 65,535 versions, so if we ever need to change it we can. All in all with some assumptions as to cipher names, block size, MAC type an estimated overhead would be 107bytes on top of the encrypted data itself and the AAD. This will require 3 passes with the standard python struct module to parse (Once for the version number, once for the sizes, once for the data). I've sent this out to a few other people to get their opinions of it as well as to this list to get everyone's opinion here. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alex.gaynor at gmail.com Fri Sep 13 20:23:56 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Fri, 13 Sep 2013 11:23:56 -0700 Subject: [Cryptography-dev] "For Humans" Message Layout In-Reply-To: References: Message-ID: Probably the biggest question I have is: is there any prior art here? Otherwise we're developing a system which, for now at least, isn't interoperable with anything else. Alex On Fri, Sep 13, 2013 at 11:22 AM, Donald Stufft wrote: > So I've been looking at playing around with the start of the > "for humans" API. I figure we need to either do a PyNaCl style > "this is what the algorithms are and that's that" or a more > flexible layout. I've opted to make the format itself flexible. > > So here's my proposal for a the message layout for an > encrypted message: https://gist.github.com/dstufft/62473d7ae4b6f8b83577 > > I tried to make it flexible, it makes it possible for the format itself > to specify things such as the algorithm, MAC, etc. The protocol is > versioned, up to 65,535 versions, so if we ever need to change > it we can. > > All in all with some assumptions as to cipher names, block size, > MAC type an estimated overhead would be 107bytes on top of > the encrypted data itself and the AAD. > > This will require 3 passes with the standard python struct module > to parse (Once for the version number, once for the sizes, once > for the data). > > I've sent this out to a few other people to get their opinions of it as > well as to this list to get everyone's opinion here. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Fri Sep 13 20:35:21 2013 From: donald at stufft.io (Donald Stufft) Date: Fri, 13 Sep 2013 14:35:21 -0400 Subject: [Cryptography-dev] "For Humans" Message Layout In-Reply-To: References: Message-ID: <2AE9D47D-4481-4358-8290-EDAA89EFE8FB@stufft.io> I'm aware of KeyCzar: https://code.google.com/p/keyczar/wiki/CiphertextFormat Differences: - KeyCzar has no notion of AAD and nothing in the format for it - KeyCzar also manages keys and includes a "key hash" which identifies which key was used to encrypt the message. - KeyCzar puts the MAC data at the end of the message - KeyCzar does not offer flexible algorithms so it doesn't need any the "length" fields. (Baked in algorithms means you know the length ahead of time). On Sep 13, 2013, at 2:23 PM, Alex Gaynor wrote: > Probably the biggest question I have is: is there any prior art here? Otherwise we're developing a system which, for now at least, isn't interoperable with anything else. > > Alex > > > On Fri, Sep 13, 2013 at 11:22 AM, Donald Stufft wrote: > So I've been looking at playing around with the start of the > "for humans" API. I figure we need to either do a PyNaCl style > "this is what the algorithms are and that's that" or a more > flexible layout. I've opted to make the format itself flexible. > > So here's my proposal for a the message layout for an > encrypted message: https://gist.github.com/dstufft/62473d7ae4b6f8b83577 > > I tried to make it flexible, it makes it possible for the format itself > to specify things such as the algorithm, MAC, etc. The protocol is > versioned, up to 65,535 versions, so if we ever need to change > it we can. > > All in all with some assumptions as to cipher names, block size, > MAC type an estimated overhead would be 107bytes on top of > the encrypted data itself and the AAD. > > This will require 3 passes with the standard python struct module > to parse (Once for the version number, once for the sizes, once > for the data). > > I've sent this out to a few other people to get their opinions of it as > well as to this list to get everyone's opinion here. > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alex.gaynor at gmail.com Sat Sep 14 21:27:51 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 14 Sep 2013 12:27:51 -0700 Subject: [Cryptography-dev] "Streaming" APIs In-Reply-To: References: Message-ID: So I've thoguht about this some, I think the idea of using iterables in the API makes a ton of sense, but somehow the `x` and `iter_x` names don't sit right with me, does anyone have any better suggestions? Alex On Wed, Sep 11, 2013 at 7:19 PM, Donald Stufft wrote: > So one thing that's really handy to do with encryption is to be able to > (de|en)crypt things without needing to load the entire thing into memory. > Currently we support this in the encrypt API by doing: > > cipher = BlockCipher() > > enciphered = cipher.encrypt(block1) > enciphered += cipher.encrypt(block2) > enciphered += cipher.encrypt(block3) > enciphered += cipher.finalize() > > We needed to do this because we need to be able to call finalize() before > the encryption is "done". > > When I was messing with padding I ended up with an API that (for padding) > got around the need for an explicit finalize step but instead it required > passing the entire data stream into the function. However it supports > generators/iterators so you can still efficiently process large datasets. > > This api looks something like > > padder = Padding() > > padded1 = "".join(padder.pad("1234")) > padded2 = "".join(padder.pad(c for c in "1234")) > > However the downside of this API is that You need to call "".join() to get > actual strings or you need to do some ugly hacks inside of the pad() > function so it returns a string if given a string and returns a generator > if given a generator. > > A third option is similar to dictionaries on Python 2.x where you have > something like iterpad() and pad(). This could work for encryption as well > so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt(). > > So I guess the question is how do we want to handle these streaming sorts > of APIs? > > 1) Thing.action() + Thing.finalize() > 2a) "".join(Thing.action(iterator)) > 2b) Thing.action(terator OR string) - Magic return types > 3) Thing.action and Thing.iter_action > 4) ???? > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Sat Sep 14 21:58:24 2013 From: donald at stufft.io (Donald Stufft) Date: Sat, 14 Sep 2013 15:58:24 -0400 Subject: [Cryptography-dev] "Streaming" APIs In-Reply-To: References: Message-ID: Yea I wasn't a huge fan of the name, I stole it from iterkeys() and I couldn't think of anything better :[ On Sep 14, 2013, at 3:27 PM, Alex Gaynor wrote: > So I've thoguht about this some, I think the idea of using iterables in the API makes a ton of sense, but somehow the `x` and `iter_x` names don't sit right with me, does anyone have any better suggestions? > > Alex > > > On Wed, Sep 11, 2013 at 7:19 PM, Donald Stufft wrote: > So one thing that's really handy to do with encryption is to be able to (de|en)crypt things without needing to load the entire thing into memory. Currently we support this in the encrypt API by doing: > > cipher = BlockCipher() > > enciphered = cipher.encrypt(block1) > enciphered += cipher.encrypt(block2) > enciphered += cipher.encrypt(block3) > enciphered += cipher.finalize() > > We needed to do this because we need to be able to call finalize() before the encryption is "done". > > When I was messing with padding I ended up with an API that (for padding) got around the need for an explicit finalize step but instead it required passing the entire data stream into the function. However it supports generators/iterators so you can still efficiently process large datasets. > > This api looks something like > > padder = Padding() > > padded1 = "".join(padder.pad("1234")) > padded2 = "".join(padder.pad(c for c in "1234")) > > However the downside of this API is that You need to call "".join() to get actual strings or you need to do some ugly hacks inside of the pad() function so it returns a string if given a string and returns a generator if given a generator. > > A third option is similar to dictionaries on Python 2.x where you have something like iterpad() and pad(). This could work for encryption as well so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt(). > > So I guess the question is how do we want to handle these streaming sorts of APIs? > > 1) Thing.action() + Thing.finalize() > 2a) "".join(Thing.action(iterator)) > 2b) Thing.action(terator OR string) - Magic return types > 3) Thing.action and Thing.iter_action > 4) ???? > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alex.gaynor at gmail.com Sat Sep 14 21:59:47 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 14 Sep 2013 12:59:47 -0700 Subject: [Cryptography-dev] "Streaming" APIs In-Reply-To: References: Message-ID: I guess my other concern is that: Say you have 3 strings you want to put into an encrypted blob, there's no longer a way to do that without constructing a list or something encrypt([a, b, c]) works, but seems unnatural. Alex On Sat, Sep 14, 2013 at 12:58 PM, Donald Stufft wrote: > Yea I wasn't a huge fan of the name, I stole it from iterkeys() and I > couldn't think of anything better :[ > > On Sep 14, 2013, at 3:27 PM, Alex Gaynor wrote: > > So I've thoguht about this some, I think the idea of using iterables in > the API makes a ton of sense, but somehow the `x` and `iter_x` names don't > sit right with me, does anyone have any better suggestions? > > Alex > > > On Wed, Sep 11, 2013 at 7:19 PM, Donald Stufft wrote: > >> So one thing that's really handy to do with encryption is to be able to >> (de|en)crypt things without needing to load the entire thing into memory. >> Currently we support this in the encrypt API by doing: >> >> cipher = BlockCipher() >> >> enciphered = cipher.encrypt(block1) >> enciphered += cipher.encrypt(block2) >> enciphered += cipher.encrypt(block3) >> enciphered += cipher.finalize() >> >> We needed to do this because we need to be able to call finalize() before >> the encryption is "done". >> >> When I was messing with padding I ended up with an API that (for padding) >> got around the need for an explicit finalize step but instead it required >> passing the entire data stream into the function. However it supports >> generators/iterators so you can still efficiently process large datasets. >> >> This api looks something like >> >> padder = Padding() >> >> padded1 = "".join(padder.pad("1234")) >> padded2 = "".join(padder.pad(c for c in "1234")) >> >> However the downside of this API is that You need to call "".join() to >> get actual strings or you need to do some ugly hacks inside of the pad() >> function so it returns a string if given a string and returns a generator >> if given a generator. >> >> A third option is similar to dictionaries on Python 2.x where you have >> something like iterpad() and pad(). This could work for encryption as well >> so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt(). >> >> So I guess the question is how do we want to handle these streaming sorts >> of APIs? >> >> 1) Thing.action() + Thing.finalize() >> 2a) "".join(Thing.action(iterator)) >> 2b) Thing.action(terator OR string) - Magic return types >> 3) Thing.action and Thing.iter_action >> 4) ???? >> >> ----------------- >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 >> DCFA >> >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> > > > -- > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 > DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From donald at stufft.io Sat Sep 14 22:05:58 2013 From: donald at stufft.io (Donald Stufft) Date: Sat, 14 Sep 2013 16:05:58 -0400 Subject: [Cryptography-dev] "Streaming" APIs In-Reply-To: References: Message-ID: <5D588F06-B3E1-4FE0-B12E-8F505DF71C12@stufft.io> If we created an encryption context object we could do BlockCipher.encryptor() = EncryptionContext() # Has the .update() + .finalize() api BlockCipher.iter_encrypti() = Iter based API, uses BlockCipher.encryptor() internally BlockCipher.encrypt() = solid string only based API, uses iter_encrypt() internally I had thought about this but I ended up not bothering because the mental overhead of giving users an EncryptionContext didn't see worth it over just passing a list. On Sep 14, 2013, at 3:59 PM, Alex Gaynor wrote: > I guess my other concern is that: Say you have 3 strings you want to put into an encrypted blob, there's no longer a way to do that without constructing a list or something encrypt([a, b, c]) works, but seems unnatural. > > Alex > > > On Sat, Sep 14, 2013 at 12:58 PM, Donald Stufft wrote: > Yea I wasn't a huge fan of the name, I stole it from iterkeys() and I couldn't think of anything better :[ > > On Sep 14, 2013, at 3:27 PM, Alex Gaynor wrote: > >> So I've thoguht about this some, I think the idea of using iterables in the API makes a ton of sense, but somehow the `x` and `iter_x` names don't sit right with me, does anyone have any better suggestions? >> >> Alex >> >> >> On Wed, Sep 11, 2013 at 7:19 PM, Donald Stufft wrote: >> So one thing that's really handy to do with encryption is to be able to (de|en)crypt things without needing to load the entire thing into memory. Currently we support this in the encrypt API by doing: >> >> cipher = BlockCipher() >> >> enciphered = cipher.encrypt(block1) >> enciphered += cipher.encrypt(block2) >> enciphered += cipher.encrypt(block3) >> enciphered += cipher.finalize() >> >> We needed to do this because we need to be able to call finalize() before the encryption is "done". >> >> When I was messing with padding I ended up with an API that (for padding) got around the need for an explicit finalize step but instead it required passing the entire data stream into the function. However it supports generators/iterators so you can still efficiently process large datasets. >> >> This api looks something like >> >> padder = Padding() >> >> padded1 = "".join(padder.pad("1234")) >> padded2 = "".join(padder.pad(c for c in "1234")) >> >> However the downside of this API is that You need to call "".join() to get actual strings or you need to do some ugly hacks inside of the pad() function so it returns a string if given a string and returns a generator if given a generator. >> >> A third option is similar to dictionaries on Python 2.x where you have something like iterpad() and pad(). This could work for encryption as well so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt(). >> >> So I guess the question is how do we want to handle these streaming sorts of APIs? >> >> 1) Thing.action() + Thing.finalize() >> 2a) "".join(Thing.action(iterator)) >> 2b) Thing.action(terator OR string) - Magic return types >> 3) Thing.action and Thing.iter_action >> 4) ???? >> >> ----------------- >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA >> >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> >> >> >> -- >> "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >> "The people's good is the highest law." -- Cicero >> GPG Key fingerprint: 125F 5C67 DFE9 4084 >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From donald at stufft.io Wed Sep 18 01:18:04 2013 From: donald at stufft.io (Donald Stufft) Date: Tue, 17 Sep 2013 19:18:04 -0400 Subject: [Cryptography-dev] "Streaming" APIs In-Reply-To: <5D588F06-B3E1-4FE0-B12E-8F505DF71C12@stufft.io> References: <5D588F06-B3E1-4FE0-B12E-8F505DF71C12@stufft.io> Message-ID: On Sep 14, 2013, at 4:05 PM, Donald Stufft wrote: > If we created an encryption context object we could do > > BlockCipher.encryptor() = EncryptionContext() # Has the .update() + .finalize() api > BlockCipher.iter_encrypti() = Iter based API, uses BlockCipher.encryptor() internally > BlockCipher.encrypt() = solid string only based API, uses iter_encrypt() internally > > I had thought about this but I ended up not bothering because the mental overhead > of giving users an EncryptionContext didn't see worth it over just passing a list. > > So how bout dem iter apis. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From paul.l.kehrer at gmail.com Thu Sep 19 17:20:22 2013 From: paul.l.kehrer at gmail.com (Paul Kehrer) Date: Thu, 19 Sep 2013 10:20:22 -0500 Subject: [Cryptography-dev] "Streaming" APIs In-Reply-To: <5D588F06-B3E1-4FE0-B12E-8F505DF71C12@stufft.io> References: <5D588F06-B3E1-4FE0-B12E-8F505DF71C12@stufft.io> Message-ID: Hmm, well I'm not sure I have a great deal to add here but maybe my opinion will kickstart some more discussion: I can see an advantage to exposing an EncryptionContext -- namely that update/finalize is a traditional semantic for encryption/hashing APIs, but if we believe that the encrypt and iter_encrypt methods completely obviate the need for update/finalize let's not provide them. (I also do not have a better name than iter_*) On Sat, Sep 14, 2013 at 3:05 PM, Donald Stufft wrote: > If we created an encryption context object we could do > > BlockCipher.encryptor() = EncryptionContext() # Has the .update() + > .finalize() api > BlockCipher.iter_encrypti() = Iter based API, uses BlockCipher.encryptor() > internally > BlockCipher.encrypt() = solid string only based API, uses iter_encrypt() > internally > > I had thought about this but I ended up not bothering because the mental > overhead > of giving users an EncryptionContext didn't see worth it over just passing a > list. > > > On Sep 14, 2013, at 3:59 PM, Alex Gaynor wrote: > > I guess my other concern is that: Say you have 3 strings you want to put > into an encrypted blob, there's no longer a way to do that without > constructing a list or something encrypt([a, b, c]) works, but seems > unnatural. > > Alex > > > On Sat, Sep 14, 2013 at 12:58 PM, Donald Stufft wrote: >> >> Yea I wasn't a huge fan of the name, I stole it from iterkeys() and I >> couldn't think of anything better :[ >> >> On Sep 14, 2013, at 3:27 PM, Alex Gaynor wrote: >> >> So I've thoguht about this some, I think the idea of using iterables in >> the API makes a ton of sense, but somehow the `x` and `iter_x` names don't >> sit right with me, does anyone have any better suggestions? >> >> Alex >> >> >> On Wed, Sep 11, 2013 at 7:19 PM, Donald Stufft wrote: >>> >>> So one thing that's really handy to do with encryption is to be able to >>> (de|en)crypt things without needing to load the entire thing into memory. >>> Currently we support this in the encrypt API by doing: >>> >>> cipher = BlockCipher() >>> >>> enciphered = cipher.encrypt(block1) >>> enciphered += cipher.encrypt(block2) >>> enciphered += cipher.encrypt(block3) >>> enciphered += cipher.finalize() >>> >>> We needed to do this because we need to be able to call finalize() before >>> the encryption is "done". >>> >>> When I was messing with padding I ended up with an API that (for padding) >>> got around the need for an explicit finalize step but instead it required >>> passing the entire data stream into the function. However it supports >>> generators/iterators so you can still efficiently process large datasets. >>> >>> This api looks something like >>> >>> padder = Padding() >>> >>> padded1 = "".join(padder.pad("1234")) >>> padded2 = "".join(padder.pad(c for c in "1234")) >>> >>> However the downside of this API is that You need to call "".join() to >>> get actual strings or you need to do some ugly hacks inside of the pad() >>> function so it returns a string if given a string and returns a generator if >>> given a generator. >>> >>> A third option is similar to dictionaries on Python 2.x where you have >>> something like iterpad() and pad(). This could work for encryption as well >>> so we'd have iter_encrypt(), iter_decrypt(), encrypt() and decrypt(). >>> >>> So I guess the question is how do we want to handle these streaming sorts >>> of APIs? >>> >>> 1) Thing.action() + Thing.finalize() >>> 2a) "".join(Thing.action(iterator)) >>> 2b) Thing.action(terator OR string) - Magic return types >>> 3) Thing.action and Thing.iter_action >>> 4) ???? >>> >>> ----------------- >>> Donald Stufft >>> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 >>> DCFA >>> >>> >>> _______________________________________________ >>> Cryptography-dev mailing list >>> Cryptography-dev at python.org >>> https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> >> >> >> -- >> "I disapprove of what you say, but I will defend to the death your right >> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >> "The people's good is the highest law." -- Cicero >> GPG Key fingerprint: 125F 5C67 DFE9 4084 >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> >> >> >> ----------------- >> Donald Stufft >> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 >> DCFA >> >> >> _______________________________________________ >> Cryptography-dev mailing list >> Cryptography-dev at python.org >> https://mail.python.org/mailman/listinfo/cryptography-dev >> > > > > -- > "I disapprove of what you say, but I will defend to the death your right to > say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev > > > > ----------------- > Donald Stufft > PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA > > > _______________________________________________ > Cryptography-dev mailing list > Cryptography-dev at python.org > https://mail.python.org/mailman/listinfo/cryptography-dev >