From erinn.looneytriggs at gmail.com Wed Oct 12 00:03:43 2011 From: erinn.looneytriggs at gmail.com (Erinn Looney-Triggs) Date: Tue, 11 Oct 2011 14:03:43 -0800 Subject: [pyOpenSSL] Tracking down a change in private key format Message-ID: <4E94BD3F.1080207@gmail.com> I am trying to track down a change in private key format that exists between RHEL 5 and RHEL 6 systems and I believe that pyOpenSSL may be responsible, though I am not sure. RHEL 5 uses pyOpenSSL-0.6-2.el5 RHEL 6 uses pyOpenSSL-0.10-2.el6.x86_64 It appears that in RHEL 5 private keys were stored in OpenSSL's default PEM format, e.g. starting with: ----BEGIN RSA PRIVATE KEY----- -----END RSA PRIVATE KEY----- In RHEL 6 this seems to have changed (using the same code to call) and the format now appears to use PKCS#8 (for the code call it is an unencrypted key): -----BEGIN PRIVATE KEY----- -----END PRIVATE KEY----- >From OpenSSL's documentation it appears that PKCS#8 is still not the default, so I am unsure as to why this is the format being output. The code in question comes from certmaster: https://fedorahosted.org/certmaster/ The code (identical on RHEL 5 and RHEL 6): def make_keypair(dest=None): pkey = crypto.PKey() pkey.generate_key(crypto.TYPE_RSA, 2048) if dest: destfd = os.open(dest, os.O_RDWR|os.O_CREAT, 0600) os.write(destfd, (crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey))) os.close(destfd) return pkey Was this formatting change intentional or did it just happen due to some change in OpenSSL? Just trying to nail down how this change came about, because it ultimately lead to the syslog daemon core dumping as it tried to load the private key. Thanks, -Erinn From exarkun at twistedmatrix.com Wed Oct 12 14:59:11 2011 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 12 Oct 2011 12:59:11 -0000 Subject: [pyOpenSSL] Tracking down a change in private key format In-Reply-To: <4E94BD3F.1080207@gmail.com> References: <4E94BD3F.1080207@gmail.com> Message-ID: <20111012125911.23178.189616158.divmod.xquotient.161@localhost.localdomain> On 11 Oct, 10:03 pm, erinn.looneytriggs at gmail.com wrote: >I am trying to track down a change in private key format that exists >between RHEL 5 and RHEL 6 systems and I believe that pyOpenSSL may be >responsible, though I am not sure. > >RHEL 5 uses pyOpenSSL-0.6-2.el5 >RHEL 6 uses pyOpenSSL-0.10-2.el6.x86_64 > >It appears that in RHEL 5 private keys were stored in OpenSSL's default >PEM format, e.g. starting with: >----BEGIN RSA PRIVATE KEY----- >-----END RSA PRIVATE KEY----- > >In RHEL 6 this seems to have changed (using the same code to call) and >the format now appears to use PKCS#8 (for the code call it is an >unencrypted key): > >-----BEGIN PRIVATE KEY----- >-----END PRIVATE KEY----- >> From OpenSSL's documentation it appears that PKCS#8 is still not the >default, so I am unsure as to why this is the format being output. The >code in question comes from certmaster: >https://fedorahosted.org/certmaster/ > >The code (identical on RHEL 5 and RHEL 6): >def make_keypair(dest=None): > pkey = crypto.PKey() > pkey.generate_key(crypto.TYPE_RSA, 2048) > if dest: > destfd = os.open(dest, os.O_RDWR|os.O_CREAT, 0600) > os.write(destfd, (crypto.dump_privatekey(crypto.FILETYPE_PEM, >pkey))) > os.close(destfd) > > return pkey > >Was this formatting change intentional or did it just happen due to >some >change in OpenSSL? Just trying to nail down how this change came about, >because it ultimately lead to the syslog daemon core dumping as it >tried >to load the private key. Hiya Erinn, The formatting change was not intentional. I think that I noticed it when I was working on OpenSSL 1.0.0 compatibility. Do you know if RHEL 6 switched to OpenSSL 1.0.0 (or maybe they just back-ported the format change to their version of 0.9.8, which is the sort of thing Debian likes to do)? I didn't think much of the change when I noticed it, and I didn't expect it to cause problems like the one you're describing, or I might have tried to have pyOpenSSL force the result to be the old format. Instead, I just made the unit tests work with either format. :/ Jean-Paul From erinn.looneytriggs at gmail.com Wed Oct 12 17:41:32 2011 From: erinn.looneytriggs at gmail.com (Erinn Looney-Triggs) Date: Wed, 12 Oct 2011 07:41:32 -0800 Subject: [pyOpenSSL] Tracking down a change in private key format In-Reply-To: <20111012125911.23178.189616158.divmod.xquotient.161@localhost.localdomain> References: <4E94BD3F.1080207@gmail.com> <20111012125911.23178.189616158.divmod.xquotient.161@localhost.localdomain> Message-ID: <4E95B52C.1080005@gmail.com> On 10/12/2011 04:59 AM, exarkun at twistedmatrix.com wrote: > On 11 Oct, 10:03 pm, erinn.looneytriggs at gmail.com wrote: >> I am trying to track down a change in private key format that exists >> between RHEL 5 and RHEL 6 systems and I believe that pyOpenSSL may be >> responsible, though I am not sure. >> >> RHEL 5 uses pyOpenSSL-0.6-2.el5 >> RHEL 6 uses pyOpenSSL-0.10-2.el6.x86_64 >> >> It appears that in RHEL 5 private keys were stored in OpenSSL's default >> PEM format, e.g. starting with: >> ----BEGIN RSA PRIVATE KEY----- >> -----END RSA PRIVATE KEY----- >> >> In RHEL 6 this seems to have changed (using the same code to call) and >> the format now appears to use PKCS#8 (for the code call it is an >> unencrypted key): >> >> -----BEGIN PRIVATE KEY----- >> -----END PRIVATE KEY----- >>> From OpenSSL's documentation it appears that PKCS#8 is still not the >> default, so I am unsure as to why this is the format being output. The >> code in question comes from certmaster: >> https://fedorahosted.org/certmaster/ >> >> The code (identical on RHEL 5 and RHEL 6): >> def make_keypair(dest=None): >> pkey = crypto.PKey() >> pkey.generate_key(crypto.TYPE_RSA, 2048) >> if dest: >> destfd = os.open(dest, os.O_RDWR|os.O_CREAT, 0600) >> os.write(destfd, (crypto.dump_privatekey(crypto.FILETYPE_PEM, >> pkey))) >> os.close(destfd) >> >> return pkey >> >> Was this formatting change intentional or did it just happen due to >> some >> change in OpenSSL? Just trying to nail down how this change came about, >> because it ultimately lead to the syslog daemon core dumping as it >> tried >> to load the private key. > Hiya Erinn, > > The formatting change was not intentional. I think that I noticed it > when I was working on OpenSSL 1.0.0 compatibility. Do you know if RHEL > 6 switched to OpenSSL 1.0.0 (or maybe they just back-ported the format > change to their version of 0.9.8, which is the sort of thing Debian > likes to do)? > > I didn't think much of the change when I noticed it, and I didn't expect > it to cause problems like the one you're describing, or I might have > tried to have pyOpenSSL force the result to be the old format. Instead, > I just made the unit tests work with either format. :/ > > Jean-Paul > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2d-oct > _______________________________________________ > pyopenssl-list mailing list > pyopenssl-list at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pyopenssl-list Yeah RHEL 6 uses openssl-1.0.0-10.el6_1.4.x86_64. So is OpenSSL setting the default to PKCS#8 now? If so they probably need to update their documentation. -Erinn From carsun_c at yahoo.com Fri Oct 28 01:07:59 2011 From: carsun_c at yahoo.com (CarSun C) Date: Thu, 27 Oct 2011 16:07:59 -0700 (PDT) Subject: [pyOpenSSL] hit an error while passing python string with embedded NUL byte(s) to OpenSSL.crypto.sign method Message-ID: <1319756879.39949.YahooMailNeo@web121313.mail.ne1.yahoo.com> With pyOpenSSL 0.13, passing a python string with?embedded NUL byte(s) to crypto.sign method will generate an error saying "must be string without null bytes, not str". Initially I thought the API is meant to support C-string (NUL-terminated) only, though it isn't explicitly mentioned in the documentation, above error seems to say so. However, looking into the crypto_sign function (within "crypto.c"), that doesn't seem to be the case, as the format string being used?(enclosed below) put a "#" right after?BYTESTRING_FMT...? "O!" BYTESTRING_FMT "#s:sign" Unsurprisingly, once I modify the format string to "O!s#s:sign", signing a python string with embedded NUL byte(s) works perfectly fine. Am I over looking something? or is this a bug? Would appreciate your feedback - c Below is the signature of the sign method sign(key, data, digest) -------------- next part -------------- An HTML attachment was scrubbed... URL: From exarkun at twistedmatrix.com Fri Oct 28 05:26:38 2011 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 28 Oct 2011 03:26:38 -0000 Subject: [pyOpenSSL] hit an error while passing python string with embedded NUL byte(s) to OpenSSL.crypto.sign method In-Reply-To: <1319756879.39949.YahooMailNeo@web121313.mail.ne1.yahoo.com> References: <1319756879.39949.YahooMailNeo@web121313.mail.ne1.yahoo.com> Message-ID: <20111028032638.23178.1287035491.divmod.xquotient.747@localhost.localdomain> On 27 Oct, 11:07 pm, carsun_c at yahoo.com wrote: >With pyOpenSSL 0.13, passing a python string with?embedded NUL byte(s) >to crypto.sign method will generate an error saying "must be string >without null bytes, not str". > >Initially I thought the API is meant to support C-string (NUL- >terminated) only, though it isn't explicitly mentioned in the >documentation, above error seems to say so. However, looking into the >crypto_sign function (within "crypto.c"), that doesn't seem to be the >case, as the format string being used?(enclosed below) put a "#" right >after?BYTESTRING_FMT... > >"O!" BYTESTRING_FMT "#s:sign" > > >Unsurprisingly, once I modify the format string to "O!s#s:sign", >signing a python string with embedded NUL byte(s) works perfectly fine. > >Am I over looking something? or is this a bug? This sounds like a bug that was fixed in 0.13: https://bugs.launchpad.net/pyopenssl/+bug/653830 Can you double check that you observed the problem with that version and not an earlier version? BYTESTRING_FMT is defined as "s" on Python 2.x, so your change shouldn't have made any difference. If you were using an older version of pyOpenSSL before, though, and then switched to a custom build based on 0.13, that would explain why it started working (it would have worked without the edit, too). If this isn't the case, a minimal reproducing example would be nice, perhaps attached to a new bug report in Launchpad. Thanks, Jean-Paul From carsun_c at yahoo.com Fri Oct 28 07:41:12 2011 From: carsun_c at yahoo.com (CarSun C) Date: Thu, 27 Oct 2011 22:41:12 -0700 (PDT) Subject: [pyOpenSSL] hit an error while passing python string with embedded NUL byte(s) to OpenSSL.crypto.sign method In-Reply-To: <20111028032638.23178.1287035491.divmod.xquotient.747@localhost.localdomain> References: <1319756879.39949.YahooMailNeo@web121313.mail.ne1.yahoo.com> <20111028032638.23178.1287035491.divmod.xquotient.747@localhost.localdomain> Message-ID: <1319780472.50415.YahooMailNeo@web121305.mail.ne1.yahoo.com> I'm so sorry, you're absolutely right. 0.12 is built-in into my current environment, which I wasn't aware of until recently. You're correct about 0.13 too. Should have caught that, and should have searched the Bug history more carefully. Anyhow, really appreciate your quick response. Thanks. -C ________________________________ From: "exarkun at twistedmatrix.com" To: "pyopenssl-list at lists.sourceforge.net" Sent: Thursday, October 27, 2011 8:26 PM Subject: Re: [pyOpenSSL] hit an error while passing python string with embedded NUL byte(s) to OpenSSL.crypto.sign method On 27 Oct, 11:07 pm, carsun_c at yahoo.com wrote: > With pyOpenSSL 0.13, passing a python string with?embedded NUL byte(s) to crypto.sign method will generate an error saying "must be string without null bytes, not str". > > Initially I thought the API is meant to support C-string (NUL- terminated) only, though it isn't explicitly mentioned in the documentation, above error seems to say so. However, looking into the crypto_sign function (within "crypto.c"), that doesn't seem to be the case, as the format string being used?(enclosed below) put a "#" right after?BYTESTRING_FMT... > > "O!" BYTESTRING_FMT "#s:sign" > > > Unsurprisingly, once I modify the format string to "O!s#s:sign", signing a python string with embedded NUL byte(s) works perfectly fine. > > Am I over looking something? or is this a bug? This sounds like a bug that was fixed in 0.13: https://bugs.launchpad.net/pyopenssl/+bug/653830 Can you double check that you observed the problem with that version and not an earlier version? BYTESTRING_FMT is defined as "s" on Python 2.x, so your change shouldn't have made any difference.? If you were using an older version of pyOpenSSL before, though, and then switched to a custom build based on 0.13, that would explain why it started working (it would have worked without the edit, too). If this isn't the case, a minimal reproducing example would be nice, perhaps attached to a new bug report in Launchpad. Thanks, Jean-Paul ------------------------------------------------------------------------------ The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning at Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev _______________________________________________ pyopenssl-list mailing list pyopenssl-list at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pyopenssl-list -------------- next part -------------- An HTML attachment was scrubbed... URL: