From heikki at OSAFOUNDATION.ORG Fri Jun 2 21:14:34 2006 From: heikki at OSAFOUNDATION.ORG (Heikki Toivonen) Date: Fri, 2 Jun 2006 12:14:34 -0700 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.16beta1 Message-ID: <44808E1A.5080609@osafoundation.org> I am happy to announce the first beta of the M2Crypto 0.16 release. Please give these bits a spin and report any problems. I will be making new betas once a week (or more often if needed) until regressions are fixed. I expect the final 0.16 bits will be out by the end of June 2006. Highlights: - All known memory leaks fixed - All known regressions fixed - Added --openssl option to build command which can be used to specify where OpenSSL is installed, by Matt Rodriguez - ECDSA signatures and ECDH key agreement, requires OpenSSL 0.9.8+, by Arno Bakker - Added sha224, sha256, sha384 and sha512, by Larry Bugbee - Added serialNumber, SN, surname, GN and givenName fields to X509_Name, by Martin Paljak - And various other improvements and bugfixes, see CHANGES file Requirements: * Python 2.3 or newer * OpenSSL 0.9.7 or newer o Some optional new features will require OpenSSL 0.9.8 or newer * SWIG 1.3.24 or newer Get it while it's hot from M2Crypto homepage: http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto -- Heikki Toivonen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 251 bytes Desc: OpenPGP digital signature URL: From MKRodriguez at LBL.GOV Mon Jun 5 21:46:24 2006 From: MKRodriguez at LBL.GOV (Matt Rodriguez) Date: Mon, 5 Jun 2006 12:46:24 -0700 Subject: [PYTHON-CRYPTO] m2crypto beta release Message-ID: <44848A10.7060102@lbl.gov> I did some quick QA on the beta release of m2crypto, I found a few problems but overall I think the quality of the project has improved significantly over the last few months. 1. test_ssl.py is not working for me. The problem is the unittests for the SSLClientTestCase. All of these tests fail with a Connection refused error. I haven't looked into this any deeper than this, because I'm just doing some preliminary checks right now. The rest of the unittests work flawlessly for me. 2. There still are a bunch of compiler warnings. These just need to go away before the final release. I've noticed that SWIG-1.3.24 generates code that produces compiler warnings, these warnings go away in SWIG 1.3.25. I know I suggested that we should make 1.3.24 a requirement, but I would say that we should probably just require 1.3.25. We should have a simple check in the setup.py script to ensure that we are using an appropriate version of SWIG. Below you'll find the compiler warnings that we need to get rid of. 3. I ran valgrind on all of the unittest files except, test_ssl, test_ssl_win, and alltests, I found some leaks in test_x509, but I suspect that it is the issue is how the tests are written, and not any underlying problem in the toolkit. Below you'll find the valgrind output This is just a cursory ananlysis of the beta release. Cheers, Matt Rodriguez Here is the valgrind output. OK ==8740== ==8740== Syscall param write(buf) points to uninitialised byte(s) ==8740== at 0x1BAFE543: __write_nocancel (in /lib/tls/libc-2.3.5.so) ==8740== by 0x1BAAF1BE: _IO_file_write@@GLIBC_2.1 (in /lib/tls/libc-2.3.5.so) ==8740== by 0x1BAADBC4: new_do_write (in /lib/tls/libc-2.3.5.so) ==8740== by 0x1BAADCDE: _IO_do_write@@GLIBC_2.1 (in /lib/tls/libc-2.3.5.so) ==8740== by 0x1BAAD3E9: _IO_file_close_it@@GLIBC_2.1 (in /lib/tls/libc-2.3.5.so) ==8740== by 0x1BAA394A: fclose@@GLIBC_2.1 (in /lib/tls/libc-2.3.5.so) ==8740== by 0x1BFA4437: RAND_write_file (in /usr/lib/libcrypto.so.0.9.7) ==8740== by 0xDACCBB17: ??? ==8740== Address 0x1C172000 is not stack'd, malloc'd or (recently) free'd ==8740== ==8740== Invalid read of size 4 ==8740== at 0x1B96377F: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95F911: PyDict_Clear (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9AD005: PyImport_Cleanup (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F30: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x80485F9: main (in /usr/bin/python2.4) ==8740== Address 0x1BCC3010 is 0 bytes after a block of size 72 free'd ==8740== at 0x1B9003C3: free (in /usr/lib/valgrind/vgpreload_memcheck.so) ==8740== by 0x1BF51CEC: CRYPTO_free (in /usr/lib/libcrypto.so.0.9.7) ==8740== ==8740== Invalid read of size 4 ==8740== at 0x1B96377F: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9AC821: _PyImport_Fini (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F35: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x80485F9: main (in /usr/bin/python2.4) ==8740== Address 0x1BBBA010 is 1392 bytes inside a block of size 1536 free'd ==8740== at 0x1B9003C3: free (in /usr/lib/valgrind/vgpreload_memcheck.so) ==8740== by 0x1B9637A0: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95CF67: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95CF3E: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9AC83A: _PyImport_Fini (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F35: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x80485F9: main (in /usr/bin/python2.4) ==8740== ==8740== Invalid read of size 4 ==8740== at 0x1B96377F: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9517B6: PyInt_Fini (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F84: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x80485F9: main (in /usr/bin/python2.4) ==8740== Address 0x1BBE8010 is not stack'd, malloc'd or (recently) free'd ==8740== ==8740== Invalid read of size 4 ==8740== at 0x1B96377F: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B982EE4: _PyUnicodeUCS4_Fini (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F8E: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x80485F9: main (in /usr/bin/python2.4) ==8740== Address 0x1BCCA010 is 944 bytes inside a block of size 965 free'd ==8740== at 0x1B9003C3: free (in /usr/lib/valgrind/vgpreload_memcheck.so) ==8740== by 0x1B9637A0: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B966EE6: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95CF3E: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B93BD9E: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B96F683: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B93BDAC: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B96F683: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B93BDAC: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B96F683: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95CF3E: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B51ED: PyInterpreterState_Clear (in /usr/lib/libpython2.4.so.1.0) ==8740== ==8740== Invalid read of size 4 ==8740== at 0x1B96377F: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B93097E: PyGrammar_RemoveAccelerators (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F9C: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x80485F9: main (in /usr/bin/python2.4) ==8740== Address 0x1BBEB010 is 904 bytes inside a block of size 1536 free'd ==8740== at 0x1B9003C3: free (in /usr/lib/valgrind/vgpreload_memcheck.so) ==8740== by 0x1B9637A0: PyObject_Free (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95CF67: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B93BD9E: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B96F683: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B93BDAC: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95C8C8: (within /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B95CC0A: PyDict_SetItem (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9607B9: _PyModule_Clear (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9ACEDC: PyImport_Cleanup (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9B7F30: Py_Finalize (in /usr/lib/libpython2.4.so.1.0) ==8740== by 0x1B9BE12D: Py_Main (in /usr/lib/libpython2.4.so.1.0) ==8740== ==8740== ERROR SUMMARY: 2951762 errors from 321 contexts (suppressed: 117 from 5) ==8740== malloc/free: in use at exit: 1873288 bytes in 2319 blocks. ==8740== malloc/free: 17970 allocs, 15651 frees, 5371657 bytes allocated. ==8740== For counts of detected errors, rerun with: -v ==8740== searching for pointers to 2319 not-freed blocks. ==8740== checked 2413164 bytes. ==8740== ==8740== ==8740== 559 (44 direct, 515 indirect) bytes in 3 blocks are definitely lost in loss record 17 of 31 ==8740== at 0x1B8FF8A6: malloc (in /usr/lib/valgrind/vgpreload_memcheck.so) ==8740== by 0x1BF514AB: (within /usr/lib/libcrypto.so.0.9.7) ==8740== ==8740== LEAK SUMMARY: ==8740== definitely lost: 44 bytes in 3 blocks. ==8740== indirectly lost: 515 bytes in 10 blocks. ==8740== possibly lost: 0 bytes in 0 blocks. ==8740== still reachable: 1872729 bytes in 2306 blocks. ==8740== suppressed: 0 bytes in 0 blocks. ==8740== Reachable blocks (those to which a pointer was found) are not shown. ==8740== To see them, rerun with: --show-reachable=yes Compiler Warnings: SWIG/_m2crypto_wrap.c: In function ?rand_pseudo_bytes?: SWIG/_m2crypto_wrap.c:3444: warning: pointer targets in passing argument 1 of ?PyString_FromStringAndSize? differ in signedness SWIG/_m2crypto_wrap.c: In function ?hmac?: SWIG/_m2crypto_wrap.c:3606: warning: pointer targets in passing argument 7 of ?HMAC? differ in signedness SWIG/_m2crypto_wrap.c: In function ?bytes_to_key?: SWIG/_m2crypto_wrap.c:3646: warning: pointer targets in passing argument 1 of ?PyString_FromStringAndSize? differ in signedness SWIG/_m2crypto_wrap.c: In function ?sign_final?: SWIG/_m2crypto_wrap.c:3718: warning: pointer targets in passing argument 1 of ?PyString_FromStringAndSize? differ in signedness SWIG/_m2crypto_wrap.c: In function ?pkey_as_der?: SWIG/_m2crypto_wrap.c:3784: warning: pointer targets in passing argument 1 of ?PyString_FromStringAndSize? differ in signedness SWIG/_m2crypto_wrap.c: In function ?AES_crypt?: SWIG/_m2crypto_wrap.c:3860: warning: pointer targets in passing argument 1 of ?PyString_FromStringAndSize? differ in signedness SWIG/_m2crypto_wrap.c: In function ?dsa_sign_asn1?: SWIG/_m2crypto_wrap.c:4673: warning: pointer targets in passing argument 5 of ?DSA_sign? differ in signedness SWIG/_m2crypto_wrap.c: In function ?x509_name_set_by_nid?: SWIG/_m2crypto_wrap.c:5316: warning: pointer targets in passing argument 4 of ?X509_NAME_add_entry_by_NID? differ in signedness SWIG/_m2crypto_wrap.c: In function ?x509_name_add_entry_by_txt?: SWIG/_m2crypto_wrap.c:5321: warning: pointer targets in passing argument 2 of ?X509_NAME_add_entry_by_txt? differ in signedness SWIG/_m2crypto_wrap.c:5321: warning: pointer targets in passing argument 4 of ?X509_NAME_add_entry_by_txt? differ in signedness SWIG/_m2crypto_wrap.c: In function ?util_string_to_hex?: SWIG/_m2crypto_wrap.c:5715: warning: pointer targets in passing argument 1 of ?PyString_FromStringAndSize? differ in signedness From heikki at OSAFOUNDATION.ORG Mon Jun 5 22:36:54 2006 From: heikki at OSAFOUNDATION.ORG (Heikki Toivonen) Date: Mon, 5 Jun 2006 13:36:54 -0700 Subject: [PYTHON-CRYPTO] m2crypto beta release In-Reply-To: <44848A10.7060102@lbl.gov> References: <44848A10.7060102@lbl.gov> Message-ID: <448495E6.9090204@osafoundation.org> Matt Rodriguez wrote: > 1. test_ssl.py is not working for me. The problem is the unittests for > the SSLClientTestCase. All of these tests fail with a Connection refused > error. I haven't looked into > this any deeper than this, because I'm just doing some preliminary > checks right now. The rest of the unittests work flawlessly for me. The SSL tests are somewhat problematic. They try to launch the openssl command line test server against which the client testcases run. So if you don't have OpenSSL in your path, or have some firewall rules you might get failed results. The good thing about using that external program is that it does more "real" testing than running against an in-memory test server. But that would be a nice alternative to have. > 2. There still are a bunch of compiler warnings. These just need to go > away before the final release. I've noticed that SWIG-1.3.24 generates > code that > produces compiler warnings, these warnings go away in SWIG 1.3.25. I > know I suggested that we should make 1.3.24 a requirement, but I would > say that we should > probably just require 1.3.25. We should have a simple check in the > setup.py script to ensure that we are using an appropriate version of > SWIG. Below you'll find the > compiler warnings that we need to get rid of. I don't think the requirements can be changed this late. It's also problematic that you get different warnings based on different SWIG versions, compiler versions and OpenSSL versions. I've fixed most warnings, but it was starting to look like spending more effort on these would become harder and harder without actually buying that much. Some examples of frustrating warnings include a bunch of functions that SWIG generates but which are not used. I can work on integrating patches for the warnings, but I doubt I would be doing much work myself on these for 0.16. > 3. I ran valgrind on all of the unittest files except, test_ssl, > test_ssl_win, and alltests, I found some leaks in test_x509, but I > suspect that it is the issue is how the tests are > written, and not any underlying problem in the toolkit. Below you'll > find the valgrind output Could you explain how to run Valgrind on the tests, I am afraid I've never used Valgrind? (Well, I've used kcachegrind to view python hotshot profiles, which isn't helping much.) I'll also see if I can figure out how to read the results you posted. Thanks, -- Heikki Toivonen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 251 bytes Desc: OpenPGP digital signature URL: From MKRodriguez at LBL.GOV Mon Jun 5 23:52:40 2006 From: MKRodriguez at LBL.GOV (Matt Rodriguez) Date: Mon, 5 Jun 2006 14:52:40 -0700 Subject: [PYTHON-CRYPTO] m2crypto beta release In-Reply-To: <448495E6.9090204@osafoundation.org> References: <44848A10.7060102@lbl.gov> <448495E6.9090204@osafoundation.org> Message-ID: <4484A7A8.6010906@lbl.gov> Heikki Toivonen wrote: > Matt Rodriguez wrote: > >> 1. test_ssl.py is not working for me. The problem is the unittests for >> the SSLClientTestCase. All of these tests fail with a Connection refused >> error. I haven't looked into >> this any deeper than this, because I'm just doing some preliminary >> checks right now. The rest of the unittests work flawlessly for me. >> > > The SSL tests are somewhat problematic. They try to launch the openssl > command line test server against which the client testcases run. So if > you don't have OpenSSL in your path, or have some firewall rules you > might get failed results. > > The good thing about using that external program is that it does more > "real" testing than running against an in-memory test server. But that > would be a nice alternative to have. > > I think using the openssl should be ok, but maybe a few more checks should be made before the tests are run. For example we could check if openssl is in the path, and then maybe a simple connection test should be run first. If that fails, the test exits with an error explaining the rest of the tests will fail. >> 2. There still are a bunch of compiler warnings. These just need to go >> away before the final release. I've noticed that SWIG-1.3.24 generates >> code that >> produces compiler warnings, these warnings go away in SWIG 1.3.25. I >> know I suggested that we should make 1.3.24 a requirement, but I would >> say that we should >> probably just require 1.3.25. We should have a simple check in the >> setup.py script to ensure that we are using an appropriate version of >> SWIG. Below you'll find the >> compiler warnings that we need to get rid of. >> > > I don't think the requirements can be changed this late. > > It's also problematic that you get different warnings based on different > SWIG versions, compiler versions and OpenSSL versions. I've fixed most > warnings, but it was starting to look like spending more effort on these > would become harder and harder without actually buying that much. > > Some examples of frustrating warnings include a bunch of functions that > SWIG generates but which are not used. > > Agreed. > I can work on integrating patches for the warnings, but I doubt I would > be doing much work myself on these for 0.16. > > I think fixing all of the warnings in the SWIG generated code across all accepted versions of SWIG will be difficult with the current accepted versions of SWIG. I can accept that it's a little late to update the requirements for this project. But I think that fixing the other compiler warnings should be fairly simple, and I will submit a patch that does that. >> 3. I ran valgrind on all of the unittest files except, test_ssl, >> test_ssl_win, and alltests, I found some leaks in test_x509, but I >> suspect that it is the issue is how the tests are >> written, and not any underlying problem in the toolkit. Below you'll >> find the valgrind output >> > > Could you explain how to run Valgrind on the tests, I am afraid I've > never used Valgrind? (Well, I've used kcachegrind to view python hotshot > profiles, which isn't helping much.) > > I'll also see if I can figure out how to read the results you posted. > > Thanks, > > Valgrind is a great tool and pretty simple to use. I think you can only use it on Linux, there is a powerpc port out there, but I don't know what the status on that is. %valgrind --leak-check=yes program_name That'll give you a summary report at the end. Sometimes I use --error-limit=no when I'm working with debugging the openssl libraries, because there are a little things in openssl that valgrind complains about. Matt From bugbee at MAC.COM Wed Jun 7 11:38:14 2006 From: bugbee at MAC.COM (Larry Bugbee) Date: Wed, 7 Jun 2006 02:38:14 -0700 Subject: [PYTHON-CRYPTO] v0.16beta1 - win32 no OPENSSL_Applink fatal error In-Reply-To: <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> References: <9418DB6C0B9D434190E54A78E931C3D1015F8C87@XCH-NW-7V1.nw.nos.boeing.com> <47EA562F-FEE6-4767-8643-3DFC25500F27@mac.com> <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> Message-ID: Hi All, On Windows with MinGW, or on Windows without MinGW.... Is anyone else having fatal "no OPENSSL_Applink" run-time errors? ...running any program that uses file I/O to load/save PEM files. All the unit tests that don't do file I/O run successfully. And everything runs fine under MacOSX regardless of file I/O. I didn't know this, but it seems OpenSSL now has a requirement you call OPENSSL_Applink() once and only once from main at the start of your program. from http://www.openssl.org/support/faq.html#PROG2 "you have to compile small C snippet with compiler and/or options of your choice. The snippet gets installed as /include/openssl/applink.c and should be either added to your application project or simply #include-d in one [and only one] of your application source files. Failure to link this shim module into your application manifests itself as fatal "no OPENSSL_Applink" run-time error. An explicit reminder is due that in this situation [mixing compiler options] it is as important to add CRYPTO_malloc_init prior first call to OpenSSL." from openssl's INSTALL.W32 If you link with OpenSSL .DLLs, then you're expected to include into your application code small "shim" snippet, which provides glue between OpenSSL BIO layer and your compiler run-time. Look up OPENSSL_Applink reference page for further details. See also openssl's ms/applink.c I've tried futzing with __m2crypto.i and __init__.py NoGo. Suggestions? Tx, Larry -----Original Message----- From: Heikki Toivonen [mailto:heikki at osafoundation.org] Sent: Tuesday, June 06, 2006 11:22 AM To: Bugbee, Larry Subject: Re: MinGW > Then I ran each of the test_XXX.py files. Some worked. Only these > gave a "OPENSSL_Uplink(00D4A010,05): no OPENSSL_Applink" error. > 'test_bio_file', > 'test_dh', > 'test_dsa', > 'test_rsa', > 'test_smime', > 'test_x509' > 'test_ecdh' > 'test_ecdsa' Hmm, I really have no idea. It's good that you got progress though. I think now would be a time to post to the mailing list. From arno=py at CS.VU.NL Wed Jun 7 14:35:57 2006 From: arno=py at CS.VU.NL (Arno Bakker) Date: Wed, 7 Jun 2006 14:35:57 +0200 Subject: [PYTHON-CRYPTO] v0.16beta1 - win32 no OPENSSL_Applink fatal error In-Reply-To: References: <9418DB6C0B9D434190E54A78E931C3D1015F8C87@XCH-NW-7V1.nw.nos.boeing.com> <47EA562F-FEE6-4767-8643-3DFC25500F27@mac.com> <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> Message-ID: <20060607123557.GA5065@host.few.vu.nl> In your letter dated Wed, Jun 07, 2006 at 02:38:14AM -0700, you wrote: > > Is anyone else having fatal "no OPENSSL_Applink" run-time > errors? ...running any program that uses file I/O to load/save PEM > files. All the unit tests that don't do file I/O run successfully. > And everything runs fine under MacOSX regardless of file I/O. > Yes. The only solution I found so far is to recompile python with the applink.c included. See my earlier mails: https://listserv.surfnet.nl/scripts/wa.exe?A2=ind0604&L=python-crypto&P=1049 CU, Arno From bugbee at MAC.COM Wed Jun 7 21:59:40 2006 From: bugbee at MAC.COM (Larry Bugbee) Date: Wed, 7 Jun 2006 12:59:40 -0700 Subject: [PYTHON-CRYPTO] v0.16beta1 - win32 no OPENSSL_Applink fatal error In-Reply-To: <20060607123557.GA5065@host.few.vu.nl> References: <9418DB6C0B9D434190E54A78E931C3D1015F8C87@XCH-NW-7V1.nw.nos.boeing.com> <47EA562F-FEE6-4767-8643-3DFC25500F27@mac.com> <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> <20060607123557.GA5065@host.few.vu.nl> Message-ID: On Jun 7, 2006, at 5:35 AM, Arno Bakker wrote: > Yes. The only solution I found so far is to recompile python with > the applink.c included. See my earlier mails: > > https://listserv.surfnet.nl/scripts/wa.exe?A2=ind0604&L=python- > crypto&P=1049 It looks to me like we have a problem. Not all apps are going to conveniently remember to call OPENSSL_Applink(), nor in my opinion, should they. Some may not Soooo.... 1- Why does everything work without this stunt under MacOSX? ...or is that a false positive? 2- Is there some way to trick OpenSSL by adding something to __m2crypto.i and __init__.py so that when M2Crypto is imported, OPENSSL_Applink() gets called? (I tried but my knowledge of exactly how is limited.) 3- OPENSSL_Applink(), in applink.c, has a once flag. Do we call it from every load_key() and save_key()? 4- ??? Larry From guido at PYTHON.ORG Thu Jun 8 04:28:20 2006 From: guido at PYTHON.ORG (Guido van Rossum) Date: Wed, 7 Jun 2006 19:28:20 -0700 Subject: [PYTHON-CRYPTO] Pure-python RSA module In-Reply-To: <20060603162712.GA30774@unrealtower.org> References: <20060603162712.GA30774@unrealtower.org> Message-ID: Hallo Sybren, Sorry voor de late reactie. 't Is druk hier! Er zijn al een aantal RSA implementaties beschikbaar voor Python, alle gebaseerd op OpenSSL. Dat is natuurlijk niet zo portable als jouw puur-Python versie, maar in de praktijk is OpenSSL zeer algemeen beschikbaar, en het is natuurlijk wel sneller. Ik geloof dat er ook al een andere pure-Python crypto kit bestaat die wat overlap met jouw pakket bevat, maar helaas ben ik de details daarvan vergeten. Misschien kan Google je nog helpen. :-) Ik denk dat de beste weg om jouw code meer gebruik te geven tweedelig is: (1) in plaats van de GPL gebruik de Apache license; (2) post een aankondiging op comp.lang.python. Een andere plaats waar je waarschijnlijk wel hulp kunt krijgen is de python-crypto mailing list. is het adres -- in Nederland zelfs! Veel succes, --Guido On 6/3/06, Sybren St?vel wrote: > Hoi Guido, > > Als onderdeel van mijn informatica studie aan de Universiteit van > Amsterdam heb ik, samen met twee collega studenten, een RSA module > geschreven in Python. Het zou mij een eer zijn als deze module > onderdeel wordt van de Python library. > > De module bevat een functie om te testen of een getal een priemgetal > is, functies om RSA keys te genereren, en om te encrypten/decrypten > met die keys. De code ligt zeer dicht bij de mathematische achtergrond > van RSA. > > Op dit moment valt de module onder de GPL. Dat is natuurlijk makkelijk > aan te passen zodat het onder de Python licensie valt. > > De module is te vinden op http://www.stuvel.eu/rsa. Voor het gemak heb > ik de tarball geattached. > > Groet, > -- > Sybren St?vel > > "Gravitation cannot be held responsible for people falling in love." > -- Albert Einstein > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2.2 (GNU/Linux) > > iD8DBQFEgbhgNIPvGICzDyQRAupwAJ9OoTYVzS1AICb3TM2igR54mFSWuwCfcNK2 > 6ptR+4kHbYPvGPVjKyH2/ss= > =nnNE > -----END PGP SIGNATURE----- > > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at PYTHON.ORG Thu Jun 8 04:28:44 2006 From: guido at PYTHON.ORG (Guido van Rossum) Date: Wed, 7 Jun 2006 19:28:44 -0700 Subject: [PYTHON-CRYPTO] Pure-python RSA module In-Reply-To: References: <20060603162712.GA30774@unrealtower.org> Message-ID: Sorry for the wrong post. Please ignore. On 6/7/06, Guido van Rossum wrote: > Hallo Sybren, > > Sorry voor de late reactie. 't Is druk hier! > > Er zijn al een aantal RSA implementaties beschikbaar voor Python, alle > gebaseerd op OpenSSL. Dat is natuurlijk niet zo portable als jouw > puur-Python versie, maar in de praktijk is OpenSSL zeer algemeen > beschikbaar, en het is natuurlijk wel sneller. > > Ik geloof dat er ook al een andere pure-Python crypto kit bestaat die > wat overlap met jouw pakket bevat, maar helaas ben ik de details > daarvan vergeten. Misschien kan Google je nog helpen. :-) > > Ik denk dat de beste weg om jouw code meer gebruik te geven tweedelig > is: (1) in plaats van de GPL gebruik de Apache license; (2) post een > aankondiging op comp.lang.python. > > Een andere plaats waar je waarschijnlijk wel hulp kunt krijgen is de > python-crypto mailing list. is het adres > -- in Nederland zelfs! > > Veel succes, > > --Guido > > On 6/3/06, Sybren St?vel wrote: > > Hoi Guido, > > > > Als onderdeel van mijn informatica studie aan de Universiteit van > > Amsterdam heb ik, samen met twee collega studenten, een RSA module > > geschreven in Python. Het zou mij een eer zijn als deze module > > onderdeel wordt van de Python library. > > > > De module bevat een functie om te testen of een getal een priemgetal > > is, functies om RSA keys te genereren, en om te encrypten/decrypten > > met die keys. De code ligt zeer dicht bij de mathematische achtergrond > > van RSA. > > > > Op dit moment valt de module onder de GPL. Dat is natuurlijk makkelijk > > aan te passen zodat het onder de Python licensie valt. > > > > De module is te vinden op http://www.stuvel.eu/rsa. Voor het gemak heb > > ik de tarball geattached. > > > > Groet, > > -- > > Sybren St?vel > > > > "Gravitation cannot be held responsible for people falling in love." > > -- Albert Einstein > > > > > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG v1.4.2.2 (GNU/Linux) > > > > iD8DBQFEgbhgNIPvGICzDyQRAupwAJ9OoTYVzS1AICb3TM2igR54mFSWuwCfcNK2 > > 6ptR+4kHbYPvGPVjKyH2/ss= > > =nnNE > > -----END PGP SIGNATURE----- > > > > > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From heikki at OSAFOUNDATION.ORG Mon Jun 12 21:04:39 2006 From: heikki at OSAFOUNDATION.ORG (Heikki Toivonen) Date: Mon, 12 Jun 2006 12:04:39 -0700 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.16beta2 Message-ID: <448DBAC7.3030503@osafoundation.org> This is the second beta of the M2Crypto 0.16 release. Please give these bits a spin and report any problems. I will be making new betas once a week (or more often if needed) until regressions are fixed. I expect the final 0.16 bits will be out by the end of June 2006. Fixed since beta1: - Bug 6031, OpenSSL can be configured without EC so need to deal with it gracefully, by Miloslav Trmac. Highlights: - All known memory leaks fixed - All known regressions fixed - Added --openssl option to build command which can be used to specify where OpenSSL is installed, by Matt Rodriguez - ECDSA signatures and ECDH key agreement, requires OpenSSL 0.9.8+, by Arno Bakker - Added sha224, sha256, sha384 and sha512, by Larry Bugbee - Added serialNumber, SN, surname, GN and givenName fields to X509_Name, by Martin Paljak - And various other improvements and bugfixes, see CHANGES file Requirements: * Python 2.3 or newer * OpenSSL 0.9.7 or newer o Some optional new features will require OpenSSL 0.9.8 or newer * SWIG 1.3.24 or newer Get it while it's hot from M2Crypto homepage: http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto -- -- Heikki Toivonen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 251 bytes Desc: OpenPGP digital signature URL: From turam at MCS.ANL.GOV Mon Jun 19 22:37:54 2006 From: turam at MCS.ANL.GOV (Thomas D. Uram) Date: Mon, 19 Jun 2006 15:37:54 -0500 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.16beta2 In-Reply-To: <448DBAC7.3030503@osafoundation.org> References: <448DBAC7.3030503@osafoundation.org> Message-ID: <44970B22.9090703@mcs.anl.gov> While the Python version requirement is stated as "2.3 or newer", version 378 of M2Crypto/SSL/Cipher.py introduced Python2.4-specific code: def __iter__(self): return (self[i] for i in xrange(m2.sk_ssl_cipher_num(self.stack))) (see diff at http://viewcvs.osafoundation.org/m2crypto/trunk/M2Crypto/SSL/Cipher.py?rev=378&r1=299&r2=378) If generators are preferred for this code, the following 2.3-compatible code could be used instead: def __iter__(self): for i in xrange(m2.sk_ssl_cipher_num(self.stack)): yield self[i] Personally, I'd like to see python 2.3 compatibility preserved. Tom On 6/12/06 2:04 PM, Heikki Toivonen wrote: > This is the second beta of the M2Crypto 0.16 release. > > Please give these bits a spin and report any problems. I will be making > new betas once a week (or more often if needed) until regressions are > fixed. I expect the final 0.16 bits will be out by the end of June 2006. > > Fixed since beta1: > - Bug 6031, OpenSSL can be configured without EC so need to deal with it > gracefully, by Miloslav Trmac. > > Highlights: > - All known memory leaks fixed > - All known regressions fixed > - Added --openssl option to build command which can be used to specify > where OpenSSL is installed, by Matt Rodriguez > - ECDSA signatures and ECDH key agreement, requires OpenSSL 0.9.8+, > by Arno Bakker > - Added sha224, sha256, sha384 and sha512, by Larry Bugbee > - Added serialNumber, SN, surname, GN and givenName fields to X509_Name, > by Martin Paljak > - And various other improvements and bugfixes, see CHANGES file > > Requirements: > * Python 2.3 or newer > * OpenSSL 0.9.7 or newer > o Some optional new features will require OpenSSL 0.9.8 or newer > * SWIG 1.3.24 or newer > > Get it while it's hot from M2Crypto homepage: > http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto > > From heikki at OSAFOUNDATION.ORG Tue Jun 20 19:32:46 2006 From: heikki at OSAFOUNDATION.ORG (Heikki Toivonen) Date: Tue, 20 Jun 2006 10:32:46 -0700 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.16beta2 In-Reply-To: <44970B22.9090703@mcs.anl.gov> References: <448DBAC7.3030503@osafoundation.org> <44970B22.9090703@mcs.anl.gov> Message-ID: <4498313E.5080306@osafoundation.org> Thomas D. Uram wrote: > If generators are preferred for this code, the following 2.3-compatible > code could be used instead: > > def __iter__(self): > for i in xrange(m2.sk_ssl_cipher_num(self.stack)): > yield self[i] Thanks, fixed. Will be in the next beta/release. -- Heikki Toivonen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 251 bytes Desc: OpenPGP digital signature URL: From heikki at OSAFOUNDATION.ORG Wed Jun 21 23:07:35 2006 From: heikki at OSAFOUNDATION.ORG (Heikki Toivonen) Date: Wed, 21 Jun 2006 14:07:35 -0700 Subject: [PYTHON-CRYPTO] ANN: M2Crypto 0.16beta3 Message-ID: <4499B517.40409@osafoundation.org> This is the third (and hopefully last) beta of the M2Crypto 0.16 release. Thanks for all testers! Please give these bits a spin and report any problems. I expect the final 0.16 bits will be out by the end of June 2006. Fixed since beta2: - Do the SSL.Cipher_Stack iterator in a way that works also in Python 2.3, by Thomas D. Uram. - Ensure that we have valid certificates in proxy cert demo, by Matt Rodriguez. Fixed since beta1: - Bug 6031, OpenSSL can be configured without EC so need to deal with it gracefully, by Miloslav Trmac. Highlights: - All known memory leaks fixed - All known regressions fixed - Added --openssl option to build command which can be used to specify where OpenSSL is installed, by Matt Rodriguez - ECDSA signatures and ECDH key agreement, requires OpenSSL 0.9.8+, by Arno Bakker - Added sha224, sha256, sha384 and sha512, by Larry Bugbee - Added serialNumber, SN, surname, GN and givenName fields to X509_Name, by Martin Paljak - And various other improvements and bugfixes, see CHANGES file Requirements: * Python 2.3 or newer * OpenSSL 0.9.7 or newer o Some optional new features will require OpenSSL 0.9.8 or newer * SWIG 1.3.24 or newer Get it while it's hot from M2Crypto homepage: http://wiki.osafoundation.org/bin/view/Projects/MeTooCrypto -- Heikki Toivonen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 251 bytes Desc: OpenPGP digital signature URL: From bugbee at MAC.COM Thu Jun 22 06:15:50 2006 From: bugbee at MAC.COM (Larry Bugbee) Date: Wed, 21 Jun 2006 21:15:50 -0700 Subject: [PYTHON-CRYPTO] 0.16beta3 - problems on Windows XP Message-ID: <766D6212-A1C8-43D6-8CC3-CE0B2D8262ED@mac.com> Under Windows XP using MinGW 3.8, gcc 3.4.2, SWIG 1.3.29, Python 2.4.3, OpenSSL 0.9.8b.... Both M2Crypto and OpenSSL were built using MinGW/gcc. OpenSSL passed all its regression tests. The build and install of M2Crypto under MinGW/gcc appeared normal. The following M2Crypto tests ran fine. test_asn1.py test_authcookie.py test_bn.py test_evp.py test_bio.py Any tests that involve file I/O, such as test_bio_file.py test_dsa.py test_rsa.py test_ecdsa.py work right up until an internal test did I/O giving results like... +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ C:\work\m2crypto-0.16beta3\tests>python test_dsa.py .OPENSSL_Uplink(00B4A010,05): no OPENSSL_Applink C:\work\m2crypto-0.16beta3\tests>python test_bio_file.py OPENSSL_Uplink(00B4A010,07): no OPENSSL_Applink C:\work\m2crypto-0.16beta3\tests>python alltests.py ............OPENSSL_Uplink(00B4A010,07): no OPENSSL_Applink C:\work\m2crypto-0.16beta3\tests> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Questions: - Has anybody had success building and testing beta3 under Windows using VS 7.1 or MinGW 3.8? - Anybody else encountering "no OPENSSL_Applink" problems? - Any suggestions on how to fix? Thanks, Larry From bugbee at MAC.COM Sun Jun 25 08:17:48 2006 From: bugbee at MAC.COM (Larry Bugbee) Date: Sat, 24 Jun 2006 23:17:48 -0700 Subject: [PYTHON-CRYPTO] win32 no OPENSSL_Applink fatal error In-Reply-To: References: <9418DB6C0B9D434190E54A78E931C3D1015F8C87@XCH-NW-7V1.nw.nos.boeing.com> <47EA562F-FEE6-4767-8643-3DFC25500F27@mac.com> <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> <20060607123557.GA5065@host.few.vu.nl> Message-ID: >>> Is anyone else having fatal "no OPENSSL_Applink" run-time >>> errors? ...running any program that uses file I/O to load/save PEM >>> files. All the unit tests that don't do file I/O run successfully. >>> And everything runs fine under MacOSX regardless of file I/O. >>> >> >> Yes. The only solution I found so far is to recompile python with >> the applink.c included. See my earlier mails: >> >> https://listserv.surfnet.nl/scripts/wa.exe?A2=ind0604&L=python- >> crypto&P=1049 > > I have yet to test this.... no configure under MinGW > > I have not yet heard of anyone having success or failure compiling > M2Crypto using VS7.1, assuming a vanilla compile of Python 2.4. It > certainly doesn't work using gcc under MinGW. > > Assuming a VS 7.1 compile and test of M2Crypto would give this > error.... It may be necessary, but it is silly to be telling users > they need to recompile Python 2.4 from scratch including applink.c > just so they can run OpenSSL/M2Crypto. Please ignore the previous post... it was an unfinished draft. I am curious about two things. First, has *anybody* tested M2Crypto0.16beta3 against Windows? ...using VS 7.1? ...without compiling Python 2.4 from scratch with the applink.c include. Second, if Python 2.4 needs applink.c for OpenSSL/M2Crypto to work, is this not a mistake? (This is not a requirement to work under Linux or MacOSX.) I submit OpenSSL is causing a problem for Python wrapping for Windows and the requirement should be rethought. ...or we need to figure out how to include it as part of the SWIG step (_m2crypto.i). ...or ??? From my knothole, asking users to recompile Python just so they can use OpenSSL/M2Crypto is not right. Or am I out in left field? Thoughts? Larry From arno=py at CS.VU.NL Mon Jun 26 09:51:12 2006 From: arno=py at CS.VU.NL (Arno Bakker) Date: Mon, 26 Jun 2006 09:51:12 +0200 Subject: [PYTHON-CRYPTO] win32 no OPENSSL_Applink fatal error In-Reply-To: References: <9418DB6C0B9D434190E54A78E931C3D1015F8C87@XCH-NW-7V1.nw.nos.boeing.com> <47EA562F-FEE6-4767-8643-3DFC25500F27@mac.com> <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> <20060607123557.GA5065@host.few.vu.nl> Message-ID: <20060626075112.GB24360@host.few.vu.nl> > > First, has *anybody* tested M2Crypto0.16beta3 against > Windows? ...using VS 7.1? ...without compiling Python 2.4 from > scratch with the applink.c include. > I just did, but it gives the expected no OPENSSL_Applink error ;-( > From my knothole, asking users to recompile Python just so they can > use OpenSSL/M2Crypto is not right. Or am I out in left field? > > Thoughts? > Just a guess, but I assume this problem holds for the Python SSL module as well. So I'm hoping Guido will release a new Python 2.4 with applink.c to support OpenSSL >= 0.9.8. Any chance of that? I'm not sufficiently fluent in the VC/SWIG business to see how to fit applink.c in M2Crypto instead, so can't propose a better solution than recompiling python/py2exe. Regards, Arno From bugbee at MAC.COM Fri Jun 30 10:47:45 2006 From: bugbee at MAC.COM (Larry Bugbee) Date: Fri, 30 Jun 2006 01:47:45 -0700 Subject: [PYTHON-CRYPTO] win32 no OPENSSL_Applink fatal error - a trial In-Reply-To: <20060626075112.GB24360@host.few.vu.nl> References: <9418DB6C0B9D434190E54A78E931C3D1015F8C87@XCH-NW-7V1.nw.nos.boeing.com> <47EA562F-FEE6-4767-8643-3DFC25500F27@mac.com> <6A13B20E-B4F5-486A-ACBA-64207C1969EA@mac.com> <20060607123557.GA5065@host.few.vu.nl> <20060626075112.GB24360@host.few.vu.nl> Message-ID: <88093B5D-F2C8-4981-B1CA-CC6F2B859ECC@mac.com> >> From my knothole, asking users to recompile Python just so they can >> use OpenSSL/M2Crypto is not right. I really want to see OpenSSL/M2Crypto work on Windows without having to ask arguably the least capable to compile their Python from scratch. I've played with _m2crypto.i and __init__.py and me with no success. ...but that may because I was going at it wrong. Please check my work and I'll try any reasonable suggestion. Here is my diff file. Suggestions are needed. Clues anybody? Larry Index: M2Crypto/__init__.py =================================================================== --- M2Crypto/__init__.py (revision 454) +++ M2Crypto/__init__.py (working copy) @@ -43,3 +43,4 @@ decrypt=0 __m2crypto.lib_init() +__m2crypto.openssl_applink() Index: setup.py =================================================================== --- setup.py (revision 454) +++ setup.py (working copy) @@ -22,14 +22,15 @@ break if os.name == 'nt': - libraries = ['ssleay32', 'libeay32'] +# libraries = ['ssleay32', 'libeay32'] + libraries = ['ssl32', 'eay32'] # for MinGW option_dict = {'openssl_prefix': 'c:\\pkg'} else: libraries = ['ssl', 'crypto'] option_dict = {'openssl_prefix': '/usr'} parse_args(option_dict) - + include_dir = os.path.join(option_dict['openssl_prefix'], 'include') include_dirs = [os.path.join(os.getcwd(), 'SWIG'), include_dir] swig_opts_str = ''.join(('-I', include_dir)) @@ -100,7 +101,10 @@ m2crypto = Extension(name = '__m2crypto', - sources = ['SWIG/_m2crypto.i'], +# sources = ['SWIG/_m2crypto.i'], + sources = ['SWIG/_m2crypto.i', # for MinGW +# 'C:\work\openssl-0.9.8b/ms/applink.c', + 'applink.c'], include_dirs = include_dirs, library_dirs = library_dirs, libraries = libraries, Index: SWIG/_m2crypto.i =================================================================== --- SWIG/_m2crypto.i (revision 454) +++ SWIG/_m2crypto.i (working copy) @@ -57,3 +57,5 @@ %constant int decrypt = 0; #endif +%rename(openssl_applink) OPENSSL_Applink; +extern void ** OPENSSL_Applink(void);