From mal at egenix.com Mon Mar 16 22:02:53 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 16 Mar 2009 22:02:53 +0100 Subject: [pyOpenSSL] Patch for pyOpenSSL adding wrappers for ciphers, hashes, etc. Message-ID: <49BEBE7D.1030408@egenix.com> There's a patch available for pyOpenSSL which appears to be originating from an older pyOpenSSL branch started by Dave Cridland: http://sourceforge.net/tracker/index.php?func=detail&aid=2591835&group_id=31249&atid=401760 This is the SVN repo of Dave's branch: http://trac.dave.cridland.net/cgi-bin/trac.cgi/browser/projects/pyopenssl The patch mentions these authors: +# Copyright 2006 Dave Cridland. See COPYING for rights +# Copyright 2006 Arnaud Desmons +# Copyright 2005 Keyphrene Keyphrene contributed the wrappers for the EVP APIs (high level interface to ciphers, hashes, digests, etc). Dave Cridland has extended the SSL part of pyOpenSSL. Arnaud Desmons is not mentioned anywhere in the patch, so it's not clear what he contributed, but he appears in the Keyphrene patch for the first time: http://trac.dave.cridland.net/cgi-bin/trac.cgi/changeset/696 The license on the patch is LGPL 2.1, just like for pyOpenSSL itself. Any chance of getting at least part of this merged into pyOpenSSL ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 16 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Mon Mar 16 21:43:49 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 16 Mar 2009 21:43:49 +0100 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads Message-ID: <49BEBA05.1030107@egenix.com> It appears as if the new thread lock support in pyOpenSSL 0.8.0 is causing serious problems with Python and threads. We have observed regular core dumps using pyOpenSSL 0.8 which did not happen with 0.7. There's a patch available on SF which appears to fix the problem: http://sourceforge.net/tracker/index.php?func=detail&aid=2543118&group_id=31249&atid=401760 Any chance of getting that into a new release ? As it stands, version 0.8.0 is not really usable in multi-threaded applications. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 16 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From info at egenix.com Mon Mar 16 23:41:30 2009 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Mon, 16 Mar 2009 23:41:30 +0100 Subject: [pyOpenSSL] ANN: eGenix pyOpenSSL Distribution 0.8.1-0.9.8j-1 Message-ID: <49BED59A.6010703@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com pyOpenSSL Distribution Version 0.8.1-0.9.8j-1 An easy to install and use repackaged distribution of the pyOpenSSL Python interface for OpenSSL - available on Windows and Unix platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.8.1-0.9.8j-1-GA.html ________________________________________________________________________ INTRODUCTION The eGenix.com pyOpenSSL Distribution includes everything you need to get started with SSL in Python. It comes with an easy to use installer that includes the most recent OpenSSL library versions in pre-compiled form. pyOpenSSL is an open-source Python add-on (http://pyopenssl.sf.net/) that allows writing SSL aware networking applications as well as certificate management tools. OpenSSL is an open-source implementation of the SSL protocol (http://www.openssl.org/). For more information, please see the product page: http://www.egenix.com/products/python/pyOpenSSL/ ________________________________________________________________________ NEWS This new release of the eGenix.com pyOpenSSL Distribution fixes a serious problem in pyOpenSSL 0.8 related to threaded applications. The problem causes invalid thread states in the Python interpreter which then result in random core dumps and seg faults. The patch was provided by Maxim Sobolev on SourceForge: http://sourceforge.net/tracker/index.php?func=detail&aid=2543118&group_id=31249&atid=401760 Note that this patch has not yet been integrated into upstream pyOpenSSL. We have also fixed several compiler warnings found in the code. The version of pyOpenSSL you find in the source release has those patches applied. Binaries are available for Linux x86 and x64 as well as Windows x86 and include pyOpenSSL as well as the OpenSSL libraries. For Plone users and friends of buildout scripts, we have added pre-built binaries for Windows. They install just like the Linux versions and allow easy integration of the archives into buildout scripts. ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/pyOpenSSL/ ________________________________________________________________________ UPGRADING Before installing this version of pyOpenSSL, please make sure that you uninstall any previously installed pyOpenSSL version. Otherwise, you could end up not using the included OpenSSL libs. _______________________________________________________________________ SUPPORT Commercial support for these packages is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 16 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From dave at cridland.net Mon Mar 16 23:35:02 2009 From: dave at cridland.net (Dave Cridland) Date: Mon, 16 Mar 2009 22:35:02 +0000 Subject: [pyOpenSSL] Patch for pyOpenSSL adding wrappers for ciphers, hashes, etc. In-Reply-To: <49BEBE7D.1030408@egenix.com> References: <49BEBE7D.1030408@egenix.com> Message-ID: <7517.1237242902.389919@peirce.dave.cridland.net> On Mon Mar 16 21:02:53 2009, M.-A. Lemburg wrote: > There's a patch available for pyOpenSSL which appears to be > originating > from an older pyOpenSSL branch started by Dave Cridland: > > http://sourceforge.net/tracker/index.php?func=detail&aid=2591835&group_id=31249&atid=401760 > > Other way around, I applied it on my fork. Actually, quite a lot of stuff is not mine, the only bits that are are a few additional API points in the Connection object, and most importantly the ability to pass in non-socket, Python file protocol objects. > This is the SVN repo of Dave's branch: > > http://trac.dave.cridland.net/cgi-bin/trac.cgi/browser/projects/pyopenssl > > The patch mentions these authors: > > +# Copyright 2006 Dave Cridland. See COPYING for rights Tick. Although more properly 2006-2007 > +# Copyright 2006 Arnaud Desmons Indeed... > +# Copyright 2005 Keyphrene I applied this patch in 2006 (in two commits, because I fu... didn't quite get things right). I'm not strictly sure, therefore, that this date is right without going back. > Keyphrene contributed the wrappers for the EVP APIs (high level > interface > to ciphers, hashes, digests, etc). > > Yup. Revs 696 and 697. > Dave Cridland has extended the SSL part of pyOpenSSL. > > Yup. Basically rev 683, plus a bunch of stuff to fix bugs I put in that, plus 755 (which is sessions, and largely untested in anger). > Arnaud Desmons is not mentioned anywhere in the patch, so it's not > clear what he contributed, but he appears in the Keyphrene patch for > the first time: > > http://trac.dave.cridland.net/cgi-bin/trac.cgi/changeset/696 > > Yes, because he hadn't added himself in, and I noticed only then. His patch is 695. Keyphrene.com's work was published on their website, I merely noticed it and incorporated it into my code for the convenience of having all these patches in one place. Arnaud's patch was sent to this list. > The license on the patch is LGPL 2.1, just like for pyOpenSSL > itself. > > For my changes, I'll BSD, or whatever else people want. I'm not a particular fan of this work being LGPL in the first place, but it's all too late to change now. > Any chance of getting at least part of this merged into pyOpenSSL ? It'd be nice - I still use this fork, actually, mainly out of lethargy - but it mostly dates from a time when I needed to do byte-counting on TLS sessions, and all sorts, as part of my work. I don't have such a pressing need now. If there's interest in the code, I'll cheerfully rebase it onto the current tree. Hmmm, I came over all git then, but you know what I mean. Dave. -- Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at dave.cridland.net - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ - http://dave.cridland.net/ Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade From exarkun at divmod.com Tue Mar 17 00:35:46 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 16 Mar 2009 18:35:46 -0500 Subject: [pyOpenSSL] Please use Launchpad for pyOpenSSL issues In-Reply-To: 0 Message-ID: <20090316233546.12853.199218807.divmod.quotient.24134@henry.divmod.com> Hi All, I'd like to migrate all of the pyOpenSSL development tools off of SourceForge and onto Launchpad. The canonical source repository is already hosted there (and has been for some time) in bzr (which means anyone can make a branch to hack on, rather than having to juggle patches). I've also been using it for issue tracking. Since the only way that I know of to prevent new tickets from being filed on SourceForge is by disabling access to the tracker entirely (which prevents anyone from even *looking* at existing tickets), I've left the SourceForge tracker active. However, Launchpad is where I go to look for pyOpenSSL issues. Launchpad also has a user interface that doesn't make me want to kill myself. So, I'd greatly appreciate it if new issues were filed there. :) Thanks, Jean-Paul From exarkun at divmod.com Tue Mar 17 00:28:05 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 16 Mar 2009 18:28:05 -0500 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <49BEBA05.1030107@egenix.com> Message-ID: <20090316232805.12853.492943161.divmod.quotient.24126@henry.divmod.com> On Mon, 16 Mar 2009 21:43:49 +0100, "M.-A. Lemburg" wrote: >It appears as if the new thread lock support in pyOpenSSL 0.8.0 is >causing serious problems with Python and threads. We have observed >regular core dumps using pyOpenSSL 0.8 which did not happen with >0.7. Ouch. >There's a patch available on SF which appears to fix the problem: > >http://sourceforge.net/tracker/index.php?func=detail&aid=2543118&group_id=31249&atid=401760 > >Any chance of getting that into a new release ? A quick skim of the patch leads me to believe it largely reverts one of the bug fixes I made in 0.8.0 which was causing thread-related crashes in 0.7.0. https://bugs.launchpad.net/pyopenssl/+bug/262381 includes some discussion of that issue. In the light of my final comment on that ticket, it may be that the new failure is much worse than the old failure. What would make things much easier for me is if someone could provide a test case which demonstrates the current bug. The patch which "fixes" the current problem looks simple enough (it's mostly noise - it could be 10 lines long instead of a hundred), but since I think it re-introduces another thread bug, I probably would prefer /not/ to apply it exactly, but to work out a solution which preserves the old fix and eliminates the new bug. If a unit test is too hard, then something along the lines of leakcheck/thread-crash.py would also be useful. If that's too hard, an explanation of why the current code is wrong would help. >As it stands, version 0.8.0 is not really usable in multi-threaded >applications. It seems that 0.7.0 wasn't really either. And I suspect that without various vendor-supplied patches that didn't make it upstream until 0.8.0, neither were any of the releases before that one either. :/ Having to revisit this part of pyOpenSSL again, I'm tempted to go for a solution I previously ruled out for performance reasons - explicitly tag each OpenSSL object wrapper (for those OpenSSL structures which can only be used in one thread) with a thread id and before each operation, check to make sure the current thread matches that thread id. This is similar to the approach taken by SQLite3, I think. It should provide the most robust protection against thread issues, but at the cost of a thread check per method. I'm hoping to have 0.9.0 out soon. Before this issue was raised, I was only blocked on some issues building Windows packages. I suppose it would be good to include some resolution to this issue in the next release though. If anyone will be at PyCon and wants to help out, let me know. Jean-Paul From exarkun at divmod.com Tue Mar 17 00:29:27 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 16 Mar 2009 18:29:27 -0500 Subject: [pyOpenSSL] potential memory leak In-Reply-To: <49A41090.3090803@coresecurity.com> Message-ID: <20090316232927.12853.96361409.divmod.quotient.24129@henry.divmod.com> On Tue, 24 Feb 2009 13:21:52 -0200, Adrian Manrique wrote: >I found a potential memory leak in flush_error_queue at util.c using Py_DECREF >macro with a function call inside. Since macros expand its argument n times >( as Py_DECREF does, where n > 1 ) this call could be generated n times. In >this case error_queue_to_list allocates a PyList object and transfer the >ownership of the pointer to the caller function. Hi Adrian, Thanks for the patch. So that I don't lose track of it, could you file a ticket on Launchpad and attach it there? If you can provide a test or a short script (in the style of those in the leakcheck directory in the repository) which covers this case, that would also help me out a lot. Thanks, Jean-Paul From mal at egenix.com Tue Mar 17 12:07:09 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 17 Mar 2009 12:07:09 +0100 Subject: [pyOpenSSL] Please use Launchpad for pyOpenSSL issues In-Reply-To: <20090316233546.12853.199218807.divmod.quotient.24134@henry.divmod.com> References: <20090316233546.12853.199218807.divmod.quotient.24134@henry.divmod.com> Message-ID: <49BF845D.70703@egenix.com> On 2009-03-17 00:35, Jean-Paul Calderone wrote: > Hi All, > > I'd like to migrate all of the pyOpenSSL development tools off of > SourceForge and onto Launchpad. The canonical source repository > is already hosted there (and has been for some time) in bzr (which > means anyone can make a branch to hack on, rather than having to > juggle patches). I've also been using it for issue tracking. Since > the only way that I know of to prevent new tickets from being filed > on SourceForge is by disabling access to the tracker entirely (which > prevents anyone from even *looking* at existing tickets), I've left > the SourceForge tracker active. However, Launchpad is where I go > to look for pyOpenSSL issues. Launchpad also has a user interface > that doesn't make me want to kill myself. So, I'd greatly appreciate > it if new issues were filed there. :) I'd suggest to add a note to the SF page, just like it was done for the Python SF project page: http://sourceforge.net/projects/python/ and then move the few existing patches over to launchpad (if they are still relevant). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Tue Mar 17 12:36:48 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 17 Mar 2009 12:36:48 +0100 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <20090316232805.12853.492943161.divmod.quotient.24126@henry.divmod.com> References: <20090316232805.12853.492943161.divmod.quotient.24126@henry.divmod.com> Message-ID: <49BF8B50.1070405@egenix.com> On 2009-03-17 00:28, Jean-Paul Calderone wrote: > On Mon, 16 Mar 2009 21:43:49 +0100, "M.-A. Lemburg" wrote: >> It appears as if the new thread lock support in pyOpenSSL 0.8.0 is >> causing serious problems with Python and threads. We have observed >> regular core dumps using pyOpenSSL 0.8 which did not happen with >> 0.7. > > Ouch. > >> There's a patch available on SF which appears to fix the problem: >> >> http://sourceforge.net/tracker/index.php?func=detail&aid=2543118&group_id=31249&atid=401760 >> >> Any chance of getting that into a new release ? > > A quick skim of the patch leads me to believe it largely reverts one of > the bug fixes I made in 0.8.0 which was causing thread-related crashes > in 0.7.0. https://bugs.launchpad.net/pyopenssl/+bug/262381 includes > some discussion of that issue. In the light of my final comment on > that ticket, it may be that the new failure is much worse than the old > failure. The old code wasn't really protecting the programmer against core dumps caused by misusing OpenSSL (e.g. sharing SSL connections between threads), but it also did not get in the way of multi-threaded programs which did implement the correct way of not sharing connections. However, with the 0.8.0 approach, SSL connections started getting in the way of perfectly fine working applications. At least in our application we also had the problem of not being able to reproduce the problem easily - it occurred seemingly random and the core dumps did not relate to the SSL code either. > What would make things much easier for me is if someone could provide a > test case which demonstrates the current bug. The patch which "fixes" > the current problem looks simple enough (it's mostly noise - it could > be 10 lines long instead of a hundred), but since I think it re-introduces > another thread bug, I probably would prefer /not/ to apply it exactly, > but to work out a solution which preserves the old fix and eliminates > the new bug. > > If a unit test is too hard, then something along the lines of > leakcheck/thread-crash.py would also be useful. > > If that's too hard, an explanation of why the current code is wrong > would help. I think you should not try to unroll the Py_BEGIN_ALLOW_THREADS ...Do some blocking I/O operation... Py_END_ALLOW_THREADS blocks and just use those macros in the code. Storing the thread state in some application data object is not a good approach, since there's nothing preventing some other thread from restoring the thread state saved by the thread that just released the lock. By contrast, the above macros will add a local stack variable and store the thread state there, so it's not possible for another thread to accidentally restore a thread state that it doesn't own. See http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock IMHO, programmers should just be made aware of the fact that OpenSSL connections should not be shared among threads - just like the DB-API strongly suggests to not share database connections between threads, but doesn't try to prevent this. >> As it stands, version 0.8.0 is not really usable in multi-threaded >> applications. > > It seems that 0.7.0 wasn't really either. And I suspect that without > various vendor-supplied patches that didn't make it upstream until > 0.8.0, neither were any of the releases before that one either. :/ See above. 0.7 worked just fine for applications using the correct implementation approach. > Having to revisit this part of pyOpenSSL again, I'm tempted to go for > a solution I previously ruled out for performance reasons - explicitly > tag each OpenSSL object wrapper (for those OpenSSL structures which > can only be used in one thread) with a thread id and before each > operation, check to make sure the current thread matches that thread id. > This is similar to the approach taken by SQLite3, I think. It should > provide the most robust protection against thread issues, but at the > cost of a thread check per method. I wouldn't go down that road... most Python C extensions simply use the above macro pair around blocking operations and don't bother with any further tricks to work around issues in the underlying C library code. > I'm hoping to have 0.9.0 out soon. Before this issue was raised, I > was only blocked on some issues building Windows packages. I suppose > it would be good to include some resolution to this issue in the > next release though. If anyone will be at PyCon and wants to help out, > let me know. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From exarkun at divmod.com Tue Mar 17 15:50:12 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 17 Mar 2009 09:50:12 -0500 Subject: [pyOpenSSL] Please use Launchpad for pyOpenSSL issues In-Reply-To: <49BF845D.70703@egenix.com> Message-ID: <20090317145012.12853.197549701.divmod.quotient.24384@henry.divmod.com> On Tue, 17 Mar 2009 12:07:09 +0100, "M.-A. Lemburg" wrote: >On 2009-03-17 00:35, Jean-Paul Calderone wrote: >> Hi All, >> >> I'd like to migrate all of the pyOpenSSL development tools off of >> SourceForge and onto Launchpad. The canonical source repository >> is already hosted there (and has been for some time) in bzr (which >> means anyone can make a branch to hack on, rather than having to >> juggle patches). I've also been using it for issue tracking. Since >> the only way that I know of to prevent new tickets from being filed >> on SourceForge is by disabling access to the tracker entirely (which >> prevents anyone from even *looking* at existing tickets), I've left >> the SourceForge tracker active. However, Launchpad is where I go >> to look for pyOpenSSL issues. Launchpad also has a user interface >> that doesn't make me want to kill myself. So, I'd greatly appreciate >> it if new issues were filed there. :) > >I'd suggest to add a note to the SF page, just like it was >done for the Python SF project page: > >http://sourceforge.net/projects/python/ > >and then move the few existing patches over to launchpad (if they >are still relevant). > Yes, this is a good idea. I've tried adding a note to the tracker pages (the bug tracker has had one for a long time now, but it was only normal sized, I just made it bigger). I didn't realize the other trackers had to be configured separately. And I don't know how to add a message to the main page. :/ Jean-Paul From exarkun at divmod.com Tue Mar 17 15:59:48 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 17 Mar 2009 09:59:48 -0500 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <49BF8B50.1070405@egenix.com> Message-ID: <20090317145948.12853.1524816542.divmod.quotient.24388@henry.divmod.com> On Tue, 17 Mar 2009 12:36:48 +0100, "M.-A. Lemburg" wrote: >On 2009-03-17 00:28, Jean-Paul Calderone wrote: >> On Mon, 16 Mar 2009 21:43:49 +0100, "M.-A. Lemburg" wrote: >>> It appears as if the new thread lock support in pyOpenSSL 0.8.0 is >>> causing serious problems with Python and threads. We have observed >>> regular core dumps using pyOpenSSL 0.8 which did not happen with >>> 0.7. >> >> Ouch. >> >>> There's a patch available on SF which appears to fix the problem: >>> >>> http://sourceforge.net/tracker/index.php?func=detail&aid=2543118&group_id=31249&atid=401760 >>> >>> Any chance of getting that into a new release ? >> >> A quick skim of the patch leads me to believe it largely reverts one of >> the bug fixes I made in 0.8.0 which was causing thread-related crashes >> in 0.7.0. https://bugs.launchpad.net/pyopenssl/+bug/262381 includes >> some discussion of that issue. In the light of my final comment on >> that ticket, it may be that the new failure is much worse than the old >> failure. > >The old code wasn't really protecting the programmer against >core dumps caused by misusing OpenSSL (e.g. sharing SSL connections >between threads), but it also did not get in the way of multi-threaded >programs which did implement the correct way of not sharing connections. > >However, with the 0.8.0 approach, SSL connections started getting >in the way of perfectly fine working applications. At least in our >application we also had the problem of not being able to reproduce >the problem easily - it occurred seemingly random and the core dumps >did not relate to the SSL code either. Isn't this just how it goes with threading bugs? ;) You're right, though, it's worse now than it was. > [snip] > >I think you should not try to unroll the > >Py_BEGIN_ALLOW_THREADS >...Do some blocking I/O operation... >Py_END_ALLOW_THREADS > >blocks and just use those macros in the code. pyOpenSSL has always done this. It's necessary if Python-defined callbacks are going to be supported, because the GIL must be re-acquired in the callback. This means thread state structured needs to be available in the callback, which in turn means Py_BEGIN_ALLOW_THREADS isn't usable. It's possible that PyGILState_Ensure and PyGILState_Release would be useful here, but I'm not sure. I need to investigate further. >Storing the thread state in some application data object is not a >good approach, since there's nothing preventing some other thread >from restoring the thread state saved by the thread that just >released the lock. > >By contrast, the above macros will add a local stack variable and >store the thread state there, so it's not possible for another >thread to accidentally restore a thread state that it doesn't >own. Right. That's why I changed pyOpenSSL /from/ doing that (prior to 0.8) to using thread local storage instead (0.8). However, it seems there's some bug in the thread local storage version of the locking scheme. >See >http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock > >IMHO, programmers should just be made aware of the fact that OpenSSL >connections should not be shared among threads - just like the DB-API >strongly suggests to not share database connections between threads, >but doesn't try to prevent this. > >>> As it stands, version 0.8.0 is not really usable in multi-threaded >>> applications. >> >> It seems that 0.7.0 wasn't really either. And I suspect that without >> various vendor-supplied patches that didn't make it upstream until >> 0.8.0, neither were any of the releases before that one either. :/ > >See above. > >0.7 worked just fine for applications using the correct >implementation approach. Right. I meant that 0.6 and earlier would crash randomly when pyOpenSSL APIs were being used from more than one thread. 0.7 is probably the most stable version of pyOpenSSL wrt threading. >> Having to revisit this part of pyOpenSSL again, I'm tempted to go for >> a solution I previously ruled out for performance reasons - explicitly >> tag each OpenSSL object wrapper (for those OpenSSL structures which >> can only be used in one thread) with a thread id and before each >> operation, check to make sure the current thread matches that thread id. >> This is similar to the approach taken by SQLite3, I think. It should >> provide the most robust protection against thread issues, but at the >> cost of a thread check per method. > >I wouldn't go down that road... most Python C extensions simply use the >above macro pair around blocking operations and don't bother with >any further tricks to work around issues in the underlying C library >code. I'll think about that. I'd prefer for there to be no way to crash the interpreter using pyOpenSSL, though. Probably which decision I make will depend in part on what the actual problem with the thread local storage approach in 0.8 is. Jean-Paul From mal at egenix.com Tue Mar 17 19:57:09 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 17 Mar 2009 19:57:09 +0100 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <20090317145948.12853.1524816542.divmod.quotient.24388@henry.divmod.com> References: <20090317145948.12853.1524816542.divmod.quotient.24388@henry.divmod.com> Message-ID: <49BFF285.50206@egenix.com> On 2009-03-17 15:59, Jean-Paul Calderone wrote: > On Tue, 17 Mar 2009 12:36:48 +0100, "M.-A. Lemburg" wrote: >> I think you should not try to unroll the >> >> Py_BEGIN_ALLOW_THREADS >> ...Do some blocking I/O operation... >> Py_END_ALLOW_THREADS >> >> blocks and just use those macros in the code. > > pyOpenSSL has always done this. It's necessary if Python-defined > callbacks are going to be supported, because the GIL must be re-acquired > in the callback. This means thread state structured needs to be available > in the callback, which in turn means Py_BEGIN_ALLOW_THREADS isn't usable. > > It's possible that PyGILState_Ensure and PyGILState_Release would be > useful here, but I'm not sure. I need to investigate further. Ah, callbacks... evil stuff :-) You're right. Callbacks don't work with the above approach. >> Storing the thread state in some application data object is not a >> good approach, since there's nothing preventing some other thread >>from restoring the thread state saved by the thread that just >> released the lock. >> >> By contrast, the above macros will add a local stack variable and >> store the thread state there, so it's not possible for another >> thread to accidentally restore a thread state that it doesn't >> own. > > Right. That's why I changed pyOpenSSL /from/ doing that (prior to 0.8) > to using thread local storage instead (0.8). However, it seems there's > some bug in the thread local storage version of the locking scheme. Perhaps it's related to this notice of caution in thread.c: """ Use PyThread_set_key_value(thekey, value) to associate void* value with thekey in the current thread. Each thread has a distinct mapping of thekey to a void* value. Caution: if the current thread already has a mapping for thekey, value is ignored. """ If value is ignored, MY_BEGIN_ALLOW_THREADS() won't save the current thread state, but leave the old one in place. It's also possible for this API to fail in case of a memory issue, but the macro doesn't check for this, so the application may end up not storing anything at all. MY_END_ALLOW_THREADS() needs to remove the stored TLS value in order to make room for the next MY_BEGIN_ALLOW_THREADS() call. Note that I'm not sure using TLS is a good idea: the implementations are highly platform dependent and may have undocumented/unwanted side-effects. IMHO, it's safer to store the thread id together with the tstate itself in the context and then check whether the thread id matches before restoring the state. This has the additional benefit of making some form of error raising possible in order to inform the programmer of the obvious mistake in using the connections from multiple threads, even if it's just a plain fprintf(stderr, ...). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 17 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From exarkun at divmod.com Tue Mar 17 20:07:26 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 17 Mar 2009 14:07:26 -0500 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <49BFF285.50206@egenix.com> Message-ID: <20090317190726.12853.811649851.divmod.quotient.24479@henry.divmod.com> On Tue, 17 Mar 2009 19:57:09 +0100, "M.-A. Lemburg" wrote: >On 2009-03-17 15:59, Jean-Paul Calderone wrote: > [snip] > >Perhaps it's related to this notice of caution in thread.c: > >""" >Use PyThread_set_key_value(thekey, value) to associate void* value with >thekey in the current thread. Each thread has a distinct mapping of thekey >to a void* value. Caution: if the current thread already has a mapping >for thekey, value is ignored. >""" Oh, boy. Yes, that could be it. I assumed it was more like setting a key in a Python dictionary. >If value is ignored, MY_BEGIN_ALLOW_THREADS() won't save the current >thread state, but leave the old one in place. So only the first MY_BEGIN_ALLOW_THREADS per thread does anything. That could break things, I guess. :) > >It's also possible for this API to fail in case of a memory issue, >but the macro doesn't check for this, so the application may end >up not storing anything at all. So the macros should do error checking as well (assuming it makes sense to continue to use TLS at all). >MY_END_ALLOW_THREADS() needs to remove the stored TLS value in order >to make room for the next MY_BEGIN_ALLOW_THREADS() call. > >Note that I'm not sure using TLS is a good idea: the implementations >are highly platform dependent and may have undocumented/unwanted >side-effects. Can you give some examples? The only thing I'm aware of is that performance may differ wildly from platform to platform, but I thought it was pretty good on all mainstream platforms these days. >IMHO, it's safer to store the thread id together with the tstate >itself in the context and then check whether the thread id matches >before restoring the state. > >This has the additional benefit of making some form of error >raising possible in order to inform the programmer of the obvious >mistake in using the connections from multiple threads, >even if it's just a plain fprintf(stderr, ...). That sounds essentially like the idea I mentioned in my earlier reply (the one I compared to SQLite). I see how all of that logic could be wrapped up in the MY_BEGIN/END_ALLOW_THREADS macro now, so I'm even more inclined to go that route now. Tests or scripts that produce failures would still be very helpful. Clearly the test suite doesn't exercise the buggy codepaths currently, and it seems neither does thread-crash.py (probably because it chokes so quickly due to its misuse of Connections in multiple threads). If I can reproduce the problem others are seeing, I'll be able to tell if a particular change actually fixes it. Otherwise I'm just guessing. :) Jean-Paul From gmoreira at gmail.com Tue Mar 17 22:19:54 2009 From: gmoreira at gmail.com (Gustavo Moreira) Date: Tue, 17 Mar 2009 18:19:54 -0300 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <20090317190726.12853.811649851.divmod.quotient.24479@henry.divmod.com> References: <49BFF285.50206@egenix.com> <20090317190726.12853.811649851.divmod.quotient.24479@henry.divmod.com> Message-ID: I could reproduce the crash. This happens only (at least in my case) in Win32 platform, I tried it on linux and does not occur. Just add a couple of certificates in the test-crash-server.py, class handler_class, method __init__. Run test-crash-server.py and then test-crash-client.py, and in seconds python crashes!. Gustavo. On 03/17/2009, Jean-Paul Calderone wrote: > > On Tue, 17 Mar 2009 19:57:09 +0100, "M.-A. Lemburg" > wrote: > >On 2009-03-17 15:59, Jean-Paul Calderone wrote: > > > [snip] > > > > >Perhaps it's related to this notice of caution in thread.c: > > > >""" > >Use PyThread_set_key_value(thekey, value) to associate void* value with > >thekey in the current thread. Each thread has a distinct mapping of > thekey > >to a void* value. Caution: if the current thread already has a mapping > >for thekey, value is ignored. > >""" > > > Oh, boy. Yes, that could be it. I assumed it was more like setting a key > in a Python dictionary. > > > >If value is ignored, MY_BEGIN_ALLOW_THREADS() won't save the current > >thread state, but leave the old one in place. > > > So only the first MY_BEGIN_ALLOW_THREADS per thread does anything. That > could break things, I guess. :) > > > > > >It's also possible for this API to fail in case of a memory issue, > >but the macro doesn't check for this, so the application may end > >up not storing anything at all. > > > So the macros should do error checking as well (assuming it makes sense > to continue to use TLS at all). > > > >MY_END_ALLOW_THREADS() needs to remove the stored TLS value in order > >to make room for the next MY_BEGIN_ALLOW_THREADS() call. > > > >Note that I'm not sure using TLS is a good idea: the implementations > >are highly platform dependent and may have undocumented/unwanted > >side-effects. > > > Can you give some examples? The only thing I'm aware of is that > performance may differ wildly from platform to platform, but I thought > it was pretty good on all mainstream platforms these days. > > > >IMHO, it's safer to store the thread id together with the tstate > >itself in the context and then check whether the thread id matches > >before restoring the state. > > > >This has the additional benefit of making some form of error > >raising possible in order to inform the programmer of the obvious > >mistake in using the connections from multiple threads, > >even if it's just a plain fprintf(stderr, ...). > > > That sounds essentially like the idea I mentioned in my earlier reply > (the one I compared to SQLite). I see how all of that logic could be > wrapped up in the MY_BEGIN/END_ALLOW_THREADS macro now, so I'm even > more inclined to go that route now. > > Tests or scripts that produce failures would still be very helpful. > Clearly the test suite doesn't exercise the buggy codepaths currently, > and it seems neither does thread-crash.py (probably because it chokes > so quickly due to its misuse of Connections in multiple threads). > > If I can reproduce the problem others are seeing, I'll be able to tell > if a particular change actually fixes it. Otherwise I'm just guessing. :) > > > Jean-Paul > > > > ------------------------------------------------------------------------------ > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and > easily build your RIAs with Flex Builder, the Eclipse(TM)based development > software that enables intelligent coding and step-through debugging. > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com > _______________________________________________ > pyopenssl-list mailing list > pyopenssl-list at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pyopenssl-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test-crash-client.py Type: text/x-python Size: 594 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test-crash-server.py Type: text/x-python Size: 1793 bytes Desc: not available URL: From exarkun at divmod.com Wed Mar 18 14:36:45 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 18 Mar 2009 08:36:45 -0500 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: Message-ID: <20090318133645.12853.1246663977.divmod.quotient.24871@henry.divmod.com> On Tue, 17 Mar 2009 18:19:54 -0300, Gustavo Moreira wrote: >I could reproduce the crash. This happens only (at least in my case) in >Win32 platform, I tried it on linux and does not occur. >Just add a couple of certificates in the test-crash-server.py, class >handler_class, method __init__. Run test-crash-server.py and then >test-crash-client.py, and in seconds python crashes!. > Hi Gustavo, This is great. I can reproduce the crash with this client/server pair. Thanks very much for providing them. :) I've filed a bug on Launchpad and attached them to it: https://bugs.launchpad.net/pyopenssl/+bug/344815 I'll try to resolve this soon and include it in the 0.9 release. Jean-Paul From info at egenix.com Wed Mar 18 15:35:44 2009 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Wed, 18 Mar 2009 15:35:44 +0100 Subject: [pyOpenSSL] ANN: eGenix pyOpenSSL Distribution 0.8.1-0.9.8j now also for Mac OS X Message-ID: <49C106C0.8020007@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com pyOpenSSL Distribution Version 0.8.1-0.9.8j An easy to install and use repackaged distribution of the pyOpenSSL Python interface for OpenSSL - available on Windows, Mac OS X and Unix platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.8.1-0.9.8j-1-GA.html ________________________________________________________________________ INTRODUCTION The eGenix.com pyOpenSSL Distribution includes everything you need to get started with SSL in Python. It comes with an easy to use installer that includes the most recent OpenSSL library versions in pre-compiled form. pyOpenSSL is an open-source Python add-on (http://pyopenssl.sf.net/) that allows writing SSL aware networking applications as well as certificate management tools. OpenSSL is an open-source implementation of the SSL protocol (http://www.openssl.org/). For more information, please see the product page: http://www.egenix.com/products/python/pyOpenSSL/ ________________________________________________________________________ NEWS This new release of the eGenix.com pyOpenSSL Distribution fixes a serious problem in pyOpenSSL 0.8 related to threaded applications. The problem causes invalid thread states in the Python interpreter which then result in random core dumps and seg faults. The patch was provided by Maxim Sobolev on SourceForge: http://sourceforge.net/tracker/index.php?func=detail&aid=2543118&group_id=31249&atid=401760 Note that this patch has not yet been integrated into upstream pyOpenSSL. We have also fixed several compiler warnings found in the code. The version of pyOpenSSL you find in the source release has those patches applied. Binaries are available for Linux x86 and x64 as well as Windows x86 and Mac OS X PPC/Intel. They include pyOpenSSL and the necessary OpenSSL libraries. For Plone users and friends of buildout scripts, we have added pre-built binaries for Windows. They install just like the Linux versions and allow easy integration of the archives into buildout scripts. ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/pyOpenSSL/ ________________________________________________________________________ UPGRADING Before installing this version of pyOpenSSL, please make sure that you uninstall any previously installed pyOpenSSL version. Otherwise, you could end up not using the included OpenSSL libs. _______________________________________________________________________ SUPPORT Commercial support for these packages is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 18 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From adrian at coresecurity.com Wed Mar 18 15:52:09 2009 From: adrian at coresecurity.com (Adrian Manrique) Date: Wed, 18 Mar 2009 11:52:09 -0300 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <20090318133645.12853.1246663977.divmod.quotient.24871@henry.divmod.com> References: <20090318133645.12853.1246663977.divmod.quotient.24871@henry.divmod.com> Message-ID: <49C10A99.3000808@coresecurity.com> Jean-Paul Calderone wrote: > On Tue, 17 Mar 2009 18:19:54 -0300, Gustavo Moreira wrote: > >> I could reproduce the crash. This happens only (at least in my case) in >> Win32 platform, I tried it on linux and does not occur. >> Just add a couple of certificates in the test-crash-server.py, class >> handler_class, method __init__. Run test-crash-server.py and then >> test-crash-client.py, and in seconds python crashes!. >> >> Hi again :) just for the record, we've found with Gustavo that this bug is sort of platform dependent. In our case, local thread dictionary is implemented with a linked list, using a key/threadId as key of the dictionary ( you can see it in thread.c from your python source code ). The problem appears when thread Ids are reused as it happens on win32 platform. The testcase Gustavo sent, creates a thread per connection on the server side, which produces at some point a thread with id used by some previous death thread. On linux platform thread ids have less probability of being reused, that's why the bug didn't trigger very quickly. cheers a/ From mal at egenix.com Wed Mar 18 21:38:49 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 18 Mar 2009 21:38:49 +0100 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <49BFF285.50206@egenix.com> References: <20090317145948.12853.1524816542.divmod.quotient.24388@henry.divmod.com> <49BFF285.50206@egenix.com> Message-ID: <49C15BD9.30204@egenix.com> On 2009-03-17 19:57, M.-A. Lemburg wrote: >>> Storing the thread state in some application data object is not a >>> good approach, since there's nothing preventing some other thread >> >from restoring the thread state saved by the thread that just >>> released the lock. >>> >>> By contrast, the above macros will add a local stack variable and >>> store the thread state there, so it's not possible for another >>> thread to accidentally restore a thread state that it doesn't >>> own. >> Right. That's why I changed pyOpenSSL /from/ doing that (prior to 0.8) >> to using thread local storage instead (0.8). However, it seems there's >> some bug in the thread local storage version of the locking scheme. > > Perhaps it's related to this notice of caution in thread.c: > > """ > Use PyThread_set_key_value(thekey, value) to associate void* value with > thekey in the current thread. Each thread has a distinct mapping of thekey > to a void* value. Caution: if the current thread already has a mapping > for thekey, value is ignored. > """ > > If value is ignored, MY_BEGIN_ALLOW_THREADS() won't save the current > thread state, but leave the old one in place. > > It's also possible for this API to fail in case of a memory issue, > but the macro doesn't check for this, so the application may end > up not storing anything at all. > > MY_END_ALLOW_THREADS() needs to remove the stored TLS value in order > to make room for the next MY_BEGIN_ALLOW_THREADS() call. > > Note that I'm not sure using TLS is a good idea: the implementations > are highly platform dependent and may have undocumented/unwanted > side-effects. > > IMHO, it's safer to store the thread id together with the tstate > itself in the context and then check whether the thread id matches > before restoring the state. > > This has the additional benefit of making some form of error > raising possible in order to inform the programmer of the obvious > mistake in using the connections from multiple threads, > even if it's just a plain fprintf(stderr, ...). For better or worse, we have found that even the patch on SF doesn't fix the problem in all cases: While we no longer get segfaults on the second SSL request to a server application using the patched pyOpenSSL 0.8.0, we still get immediate segfaults if two clients try to access the server at the same time. This time around the problem shows up on Windows. Only going back to pyOpenSSL 0.7 solved both issues. I guess the reason for this is that pyOpenSSL 0.7 stores the thread state in the connection rather than the context as does the SF patch. Do you have any information on whether the context in OpenSSL is thread-safe ? The FAQ entry for OpenSSL only mentions connections: http://www.openssl.org/support/faq.html#PROG1 Note that it also mentions two callbacks that have to be initialized for multi-threaded applications: CRYPTO_set_locking_callback() and CRYPTO_set_id_callback() which pyOpenSSL does not set. Unfortunately, the man page explaining these is not very helpful: http://www.openssl.org/docs/crypto/threads.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 18 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From exarkun at divmod.com Wed Mar 18 21:54:26 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 18 Mar 2009 15:54:26 -0500 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <49C15BD9.30204@egenix.com> Message-ID: <20090318205426.12853.2086949527.divmod.quotient.25004@henry.divmod.com> On Wed, 18 Mar 2009 21:38:49 +0100, "M.-A. Lemburg" wrote: >On 2009-03-17 19:57, M.-A. Lemburg wrote: > > [snip] > >For better or worse, we have found that even the patch on SF doesn't >fix the problem in all cases: > >While we no longer get segfaults on the second SSL request to >a server application using the patched pyOpenSSL 0.8.0, we still >get immediate segfaults if two clients try to access the server >at the same time. This time around the problem shows up on Windows. > >Only going back to pyOpenSSL 0.7 solved both issues. > >I guess the reason for this is that pyOpenSSL 0.7 stores the >thread state in the connection rather than the context as does >the SF patch. > >Do you have any information on whether the context in OpenSSL is >thread-safe ? > >The FAQ entry for OpenSSL only mentions connections: >http://www.openssl.org/support/faq.html#PROG1 > >Note that it also mentions two callbacks that have to be initialized >for multi-threaded applications: CRYPTO_set_locking_callback() and >CRYPTO_set_id_callback() which pyOpenSSL does not set. > >Unfortunately, the man page explaining these is not very helpful: > >http://www.openssl.org/docs/crypto/threads.html You've linked to all the documentation I'm aware of. My interpretation of it has been that it means Context objects are threadsafe, as long as you initialize OpenSSL's threading layer (which pyOpenSSL does in 0.8.0 but not in any previous version!). Probably reading the implementation is the only way to be really sure (though, of course, a future release of OpenSSL might invalidate anything learned using that approach). Jean-Paul From mal at egenix.com Wed Mar 18 22:28:34 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 18 Mar 2009 22:28:34 +0100 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <20090318205426.12853.2086949527.divmod.quotient.25004@henry.divmod.com> References: <20090318205426.12853.2086949527.divmod.quotient.25004@henry.divmod.com> Message-ID: <49C16782.10300@egenix.com> On 2009-03-18 21:54, Jean-Paul Calderone wrote: > On Wed, 18 Mar 2009 21:38:49 +0100, "M.-A. Lemburg" wrote: >> Do you have any information on whether the context in OpenSSL is >> thread-safe ? >> >> The FAQ entry for OpenSSL only mentions connections: >> http://www.openssl.org/support/faq.html#PROG1 >> >> Note that it also mentions two callbacks that have to be initialized >> for multi-threaded applications: CRYPTO_set_locking_callback() and >> CRYPTO_set_id_callback() which pyOpenSSL does not set. >> >> Unfortunately, the man page explaining these is not very helpful: >> >> http://www.openssl.org/docs/crypto/threads.html > > You've linked to all the documentation I'm aware of. My interpretation > of it has been that it means Context objects are threadsafe, as long as > you initialize OpenSSL's threading layer (which pyOpenSSL does in 0.8.0 > but not in any previous version!). Ah sorry, I grepped the wrong pyOpenSSL version. Good to know that 0.8.0 does setup the OpenSSL threading support ! > Probably reading the implementation > is the only way to be really sure (though, of course, a future release > of OpenSSL might invalidate anything learned using that approach). True. They do have a tendency to change internals frequently... even in sub-patch-level releases. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 18 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Thu Mar 19 11:37:56 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 19 Mar 2009 11:37:56 +0100 Subject: [pyOpenSSL] Patch for pyOpenSSL adding wrappers for ciphers, hashes, etc. In-Reply-To: <7517.1237242902.389919@peirce.dave.cridland.net> References: <49BEBE7D.1030408@egenix.com> <7517.1237242902.389919@peirce.dave.cridland.net> Message-ID: <49C22084.90004@egenix.com> On 2009-03-16 23:35, Dave Cridland wrote: > On Mon Mar 16 21:02:53 2009, M.-A. Lemburg wrote: >> There's a patch available for pyOpenSSL which appears to be >> originating >> from an older pyOpenSSL branch started by Dave Cridland: >> >> http://sourceforge.net/tracker/index.php?func=detail&aid=2591835&group_id=31249&atid=401760 >> >> > Other way around, I applied it on my fork. > > Actually, quite a lot of stuff is not mine, the only bits that are > are a few additional API points in the Connection object, and most > importantly the ability to pass in non-socket, Python file protocol > objects. > > [snip] > >> The license on the patch is LGPL 2.1, just like for pyOpenSSL >> itself. >> >> > For my changes, I'll BSD, or whatever else people want. I'm not a > particular fan of this work being LGPL in the first place, but it's > all too late to change now. > > >> Any chance of getting at least part of this merged into pyOpenSSL ? > > It'd be nice - I still use this fork, actually, mainly out of > lethargy - but it mostly dates from a time when I needed to do > byte-counting on TLS sessions, and all sorts, as part of my work. I > don't have such a pressing need now. > > If there's interest in the code, I'll cheerfully rebase it onto the > current tree. Thanks for the detailed explanation of where the new code originated. I think that's important to have if the patch should end up in the main pyOpenSSL distribution. I think there is a lot of useful stuff in the patch. I'd especially be interested in the EVP work done by Keyphrene.com since currently the only way access those APIs in Python appears to be M2Crypto with its long history of memory leaks and reference count bugs (not sure what todays situation is like, but a few years ago, it was a minefield). By contrast, pyOpenSSL has demonstrated to be a really stable platform to work with, so if those EVP patches are of the same quality, we'd much rather like to use those. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 19 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From dave at cridland.net Thu Mar 19 12:07:03 2009 From: dave at cridland.net (Dave Cridland) Date: Thu, 19 Mar 2009 11:07:03 +0000 Subject: [pyOpenSSL] Patch for pyOpenSSL adding wrappers for ciphers, hashes, etc. In-Reply-To: <49C22084.90004@egenix.com> References: <49BEBE7D.1030408@egenix.com> <7517.1237242902.389919@peirce.dave.cridland.net> <49C22084.90004@egenix.com> Message-ID: <7951.1237460823.776165@peirce.dave.cridland.net> On Thu Mar 19 10:37:56 2009, M.-A. Lemburg wrote: > Thanks for the detailed explanation of where the new code > originated. > I think that's important to have if the patch should end up in the > main pyOpenSSL distribution. > > Certainly. > I think there is a lot of useful stuff in the patch. I'd especially > be interested in the EVP work done by Keyphrene.com since currently > the only way access those APIs in Python appears to be M2Crypto with > its long history of memory leaks and reference count bugs (not sure > what todays situation is like, but a few years ago, it was a > minefield). > > Keyphrene.com's code I think I merged by diffing their "Org.keyphrene" package. You do have to dig down quite a bit, sadly, to find out it's the same codebase underneath, merged with various other projects. Close examination reveals it includes "mxTidy", for instance, although the copyright notice is only presevred in the mxTidy.c file. (I assume it *is* compatible with LGPL, right?) Dave. -- Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at dave.cridland.net - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ - http://dave.cridland.net/ Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade From mal at egenix.com Thu Mar 19 12:58:54 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 19 Mar 2009 12:58:54 +0100 Subject: [pyOpenSSL] Patch for pyOpenSSL adding wrappers for ciphers, hashes, etc. In-Reply-To: <7951.1237460823.776165@peirce.dave.cridland.net> References: <49BEBE7D.1030408@egenix.com> <7517.1237242902.389919@peirce.dave.cridland.net> <49C22084.90004@egenix.com> <7951.1237460823.776165@peirce.dave.cridland.net> Message-ID: <49C2337E.9000605@egenix.com> On 2009-03-19 12:07, Dave Cridland wrote: > On Thu Mar 19 10:37:56 2009, M.-A. Lemburg wrote: > Keyphrene.com's code I think I merged by diffing their "Org.keyphrene" > package. You do have to dig down quite a bit, sadly, to find out it's > the same codebase underneath, merged with various other projects. They do appear to provide a separate download for this as well: http://www.keyphrene.com/products/pyOpenSSL-extended/index.php?lng=en > Close examination reveals it includes "mxTidy", for instance, although > the copyright notice is only presevred in the mxTidy.c file. (I assume > it *is* compatible with LGPL, right?) mxTidy is under the eGenix Public License (a Python style license), the original HTML Tidy under a BSD-style license. Both are compatible with the LGPL, AFAIK. However, they also both require that the licenses be kept with the source code. If that's not the case, then they are in violation of the licenses. Looking at their archive http://www.keyphrene.com/products/org.keyphrene/?lng=en they don't provide any license or documentation files whatsoever, so I guess we need to have a friendly word with them :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 19 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Thu Mar 19 13:05:17 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 19 Mar 2009 13:05:17 +0100 Subject: [pyOpenSSL] Patch for pyOpenSSL adding wrappers for ciphers, hashes, etc. In-Reply-To: <49C2337E.9000605@egenix.com> References: <49BEBE7D.1030408@egenix.com> <7517.1237242902.389919@peirce.dave.cridland.net> <49C22084.90004@egenix.com> <7951.1237460823.776165@peirce.dave.cridland.net> <49C2337E.9000605@egenix.com> Message-ID: <49C234FD.6060608@egenix.com> On 2009-03-19 12:58, M.-A. Lemburg wrote: > On 2009-03-19 12:07, Dave Cridland wrote: >> On Thu Mar 19 10:37:56 2009, M.-A. Lemburg wrote: >> Keyphrene.com's code I think I merged by diffing their "Org.keyphrene" >> package. You do have to dig down quite a bit, sadly, to find out it's >> the same codebase underneath, merged with various other projects. > > They do appear to provide a separate download for this as well: > > http://www.keyphrene.com/products/pyOpenSSL-extended/index.php?lng=en ... in addition to wrapping it up as "ssl4py": http://www.keyphrene.com/products/4py/index.php?lng=en without any notice about the origin of the software. >> Close examination reveals it includes "mxTidy", for instance, although >> the copyright notice is only presevred in the mxTidy.c file. (I assume >> it *is* compatible with LGPL, right?) > > mxTidy is under the eGenix Public License (a Python style license), > the original HTML Tidy under a BSD-style license. Both are compatible > with the LGPL, AFAIK. However, they also both require that the licenses > be kept with the source code. If that's not the case, then they are in > violation of the licenses. > > Looking at their archive > > http://www.keyphrene.com/products/org.keyphrene/?lng=en > > they don't provide any license or documentation files whatsoever, > so I guess we need to have a friendly word with them :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 19 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From exarkun at divmod.com Sun Mar 22 17:32:17 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sun, 22 Mar 2009 11:32:17 -0500 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <49C16782.10300@egenix.com> Message-ID: <20090322163217.12853.723591180.divmod.quotient.26557@henry.divmod.com> Hello all, I just pushed a branch to launchpad which may address the 0.8 thread-related crashes. The bug, along with a reproduction script, is described here: https://bugs.launchpad.net/pyopenssl/+bug/344815 The branch, along with checkout instructions, which fixes the problem is here: https://code.launchpad.net/~exarkun/pyopenssl/thread-crash If you're not in to bzr, then you can get just the revision which includes the fix here: http://bazaar.launchpad.net/~exarkun/pyopenssl/thread-crash/revision/98 Note that the branch is from current trunk tip (ie, the latest development version). It is not a branch of the 0.8.0 release. In my testing, the reproduction scripts attached to the ticket produce a server-side segfault after running for between 2 and 10 seconds. It is likely that the bad behavior will trigger less frequently if run on a single core machine, but it may still trigger. With the branch linked above, the scripts run until they exhaust the available IP address space (each connection allocates a port which then goes into TIME_WAIT, so eventually there are no more ports free until the TIME_wAIT timeout expires). I can run the client multiple times without the server crashing. This is a good sign. Any additional testing anyone can do would be greatly appreciated. Thanks, Jean-Paul From mal at egenix.com Wed Mar 25 00:24:29 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 25 Mar 2009 00:24:29 +0100 Subject: [pyOpenSSL] Python core dumps when using pyOpenSSL 0.8.0 and threads In-Reply-To: <20090322163217.12853.723591180.divmod.quotient.26557@henry.divmod.com> References: <20090322163217.12853.723591180.divmod.quotient.26557@henry.divmod.com> Message-ID: <49C96BAD.4070602@egenix.com> On 2009-03-22 17:32, Jean-Paul Calderone wrote: > Hello all, > > I just pushed a branch to launchpad which may address the 0.8 thread-related > crashes. > > The bug, along with a reproduction script, is described here: > > https://bugs.launchpad.net/pyopenssl/+bug/344815 > > The branch, along with checkout instructions, which fixes the problem is here: > > https://code.launchpad.net/~exarkun/pyopenssl/thread-crash > > If you're not in to bzr, then you can get just the revision which includes > the fix here: > > http://bazaar.launchpad.net/~exarkun/pyopenssl/thread-crash/revision/98 This doesn't appear to cover all cases, esp. not if you have an application that uses recursion in callbacks. The patch causes an already existing thread state to get overwritten by another one. If the application starts returning from a deeper nesting, this will cause a wrong thread state to get restored (actually, the last one will get restored multiple times). To solve this, the thread state would have to be managed on a stack or recursive callbacks would have to be prohibited by checking whether there already is a saved thread state and then raising an exception instead of overwriting the state with a new one. The latter solution appears to be a lot easier to implement and much safer overall. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2009-03-19: Released mxODBC.Connect 1.0.1 http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/