From reitz at INWORKS.DE  Mon Mar 15 13:24:39 2004
From: reitz at INWORKS.DE (Andre Reitz)
Date: Mon, 15 Mar 2004 13:24:39 +0100
Subject: [PYTHON-CRYPTO] Memoryleak in SSL.Connection
Message-ID: <20040315132439.0637874c.reitz@inworks.de>

Hy all,

There is a memoryleak if you use
the function setblocking..... of SSL.Connection

e.g.:
  self._write_bio
  self._read_bio
...are bound instance methods, which referr to the Connection object

it seems that because of this backreference,
the cyclic garbagecollection would be needed to free
the Connection object,
but because of the __del__ method in Connection, it can 
never be freed.

This is fatal if used as a Server:) 
We have exactly this problem:
 When the Server runs one Week, the server
 consumes half of the memory.




Greetings, Andre'
And thank you very much in advance.






...
    sendall = send = write = _write_bio
    recv = read  = _read_bio

    def setblocking(self, mode):
        """Set this connection's underlying socket to _mode_."""
        self.socket.setblocking(mode)
        if mode:
            self.send = self.write = self._write_bio
            self.recv = self.read = self._read_bio
        else:
            self.send = self.write = self._write_nbio
            self.recv = self.read = self._read_nbio


Perhaps this should be done like this.....

    def setblocking(self,mode):
        self.blockingmode=mode
    ...
    def send(self,...):
        if self.blockingmode: return self._write_bio(...)
        return self._write_nbio(...)



Does anybody have a better idea?



-- 
__________________________________________________________________________

Als Technologieunternehmen konzipieren und entwickeln wir ma?geschneiderte Feedback- und
Monitoring-Systeme - wie beispielsweise L?sungen f?r Beschwerde- und Ideenmanagement.
Mit dem Inquery? Survey Server bieten wir eine der leistungsf?higsten Standardl?sungen f?r
Online-Umfragen mit dem Schwerpunkt auf der Messung von Kundenzufriedenheit an.
__________________________________________________________________________


Inworks GmbH
Andre Reitz, Leiter Entwicklung
H?rvelsinger Weg 39, 89081 Ulm, Germany
Tel +49 (0) 731 / 93807-21
Fax +49(0)731/93807-18
Internet: http://www.inworks.de



From ngps at POST1.COM  Mon Mar 15 13:48:24 2004
From: ngps at POST1.COM (Ng Pheng Siong)
Date: Mon, 15 Mar 2004 20:48:24 +0800
Subject: [PYTHON-CRYPTO] Memoryleak in SSL.Connection
In-Reply-To: <20040315132439.0637874c.reitz@inworks.de>
References: <20040315132439.0637874c.reitz@inworks.de>
Message-ID: <20040315124824.GB1262@vista.netmemetic.com>

On Mon, Mar 15, 2004 at 01:24:39PM +0100, Andre Reitz wrote:
> Perhaps this should be done like this.....
>
>     def setblocking(self,mode):
>         self.blockingmode=mode
>     ...
>     def send(self,...):
>         if self.blockingmode: return self._write_bio(...)
>         return self._write_nbio(...)
>
> Does anybody have a better idea?

Your idea sounds good enough! ;-)

Does it arrest the leak?

Cheers.

--
Ng Pheng Siong <ngps at netmemetic.com>

http://firewall.rulemaker.net -+- Firewall Change Management & Version Control
http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL



From reitz at INWORKS.DE  Mon Mar 15 14:51:55 2004
From: reitz at INWORKS.DE (Andre Reitz)
Date: Mon, 15 Mar 2004 14:51:55 +0100
Subject: [PYTHON-CRYPTO] Memoryleak in SSL.Connection
In-Reply-To: <20040315124519.GA1262@vista.netmemetic.com>
References: <20040315132439.0637874c.reitz@inworks.de>
            <20040315124519.GA1262@vista.netmemetic.com>
Message-ID: <20040315145155.1d36d5d8.reitz@inworks.de>

On Mon, 15 Mar 2004 20:45:19 +0800
Ng Pheng Siong <ngps at netmemetic.com> wrote:

> On Mon, Mar 15, 2004 at 01:24:39PM +0100, Andre Reitz wrote:
> > Perhaps this should be done like this.....
> > 
> >     def setblocking(self,mode):
> >         self.blockingmode=mode
> >     ...
> >     def send(self,...):
> >         if self.blockingmode: return self._write_bio(...)
> >         return self._write_nbio(...)
> > 
> > Does anybody have a better idea?
> 
> Your idea sounds good enough! ;-)
> 
> Does it arrest the leak?
> 

I didnt really try...
courrently I use this as a workaround:

self._connection=M2Crypto.SSL.Connection(...)
....
....
try                  : del self._connection.send
except AttributeError: pass
try                  : del self._connection.recv
except AttributeError: pass
try                  : del self._connection.write
except AttributeError: pass
try                  : del self._connection.read
except AttributeError: pass

this helps....

> -- 
> Ng Pheng Siong <ngps at netmemetic.com> 
> 
> http://firewall.rulemaker.net -+- Firewall Change Management & Version Control
> http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL


-- 
__________________________________________________________________________

Als Technologieunternehmen konzipieren und entwickeln wir ma?geschneiderte Feedback- und
Monitoring-Systeme - wie beispielsweise L?sungen f?r Beschwerde- und Ideenmanagement.
Mit dem Inquery? Survey Server bieten wir eine der leistungsf?higsten Standardl?sungen f?r
Online-Umfragen mit dem Schwerpunkt auf der Messung von Kundenzufriedenheit an.
__________________________________________________________________________


Inworks GmbH
Andre Reitz, Leiter Entwicklung
H?rvelsinger Weg 39, 89081 Ulm, Germany
Tel +49 (0) 731 / 93807-21
Fax +49(0)731/93807-18
Internet: http://www.inworks.de



From creslin_hawk at YAHOO.COM  Tue Mar 16 22:19:01 2004
From: creslin_hawk at YAHOO.COM (Dan Holstad)
Date: Tue, 16 Mar 2004 13:19:01 -0800
Subject: [PYTHON-CRYPTO] import error
Message-ID: <20040316211901.68176.qmail@web13124.mail.yahoo.com>


I am getting the following error when trying to use the M2Crypto module:

python alltests.py

Traceback (most recent call last):

File "alltests.py", line 6, in ?

from M2Crypto import Rand

File "/usr/local/lib/python2.3/site-packages/M2Crypto/__init__.py", line 7, in ?

import __m2crypto

ImportError: dynamic module does not define init function (init__m2crypto)



System: Red Hat 9.0

SWIG Version 1.3.17u-20030522-1858

Python 2.3


Do you Yahoo!?
Yahoo! Mail - More reliable, more storage, less spam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-crypto/attachments/20040316/d6f3267f/attachment.html>

From demirg at ITU.EDU.TR  Fri Mar 19 10:02:05 2004
From: demirg at ITU.EDU.TR (Gokhan Demir)
Date: Fri, 19 Mar 2004 11:02:05 +0200
Subject: [PYTHON-CRYPTO] m2crypto doc
Message-ID: <1079686925.9309.7.camel@go96156.yurtlar.itu.edu.tr>

Hi,

I do not know if this mail group is still used effectively. I hope it is
:)

I am trying to implement some basic (Bit commitment, Fair coin flips)
and advanced (Secure Election) cryptography protocols using m2crypto.

The main problem here is that there is not enough doc or any manual
about using m2crypto. (at least I was not able to find any) Does anyone
know any doc about m2crypto or should I use mxCrypto which seems to have
some doc?

Gokhan Demir



From ngps at POST1.COM  Tue Mar 30 05:37:56 2004
From: ngps at POST1.COM (Ng Pheng Siong)
Date: Tue, 30 Mar 2004 11:37:56 +0800
Subject: [PYTHON-CRYPTO] m2crypto doc
In-Reply-To: <1079686925.9309.7.camel@go96156.yurtlar.itu.edu.tr>
References: <1079686925.9309.7.camel@go96156.yurtlar.itu.edu.tr>
Message-ID: <20040330033756.GE626@vista.netmemetic.com>

On Fri, Mar 19, 2004 at 11:02:05AM +0200, Gokhan Demir wrote:
> I am trying to implement some basic (Bit commitment, Fair coin flips)
> and advanced (Secure Election) cryptography protocols using m2crypto.
>
> The main problem here is that there is not enough doc or any manual
> about using m2crypto. (at least I was not able to find any) Does anyone
> know any doc about m2crypto or should I use mxCrypto which seems to have
> some doc?

Hi,

Yeah, M2Crypto is weak in docu. I've just released 0.13, which has
epydoc-generated API docs. Still not much, because the APIs aren't
documented well, but it's a start. Thanks to Roger Binns for deciding for
me which doc-generating tool to use. ;-)

What protocols do you have in mind to implement?

Cheers.

--
Ng Pheng Siong <ngps at netmemetic.com>

http://firewall.rulemaker.net -+- Firewall Change Management & Version Control
http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL



From ngps at POST1.COM  Tue Mar 30 05:42:11 2004
From: ngps at POST1.COM (Ng Pheng Siong)
Date: Tue, 30 Mar 2004 11:42:11 +0800
Subject: [PYTHON-CRYPTO] [Announce] M2Crypto 0.13
Message-ID: <20040330034211.GF626@vista.netmemetic.com>

Hi,

M2Crypto 0.13 is now available.

 Changes since 0.12/0.11
-------------------------
- Patches from Artur Frysiak <wiget at pld-linux.org>. Thanks Artur.
  = Allow using a passphrase callback in class SMIME.
  = Added method get0_signers to class PKCS7, which retrieves signers'
    certificates from a PKCS7 blob.
  = Added methods as_pem and save_pem to class X509.
  = Added file version.py.
  = Allow SSL.Context.load_verify_locations to accept both 'cafile' and
    'capath'.
- Fixed BIO.read() not reading until EOF. Thanks to Egil Muller
  <redhog at redhog.org> for suggestion.
- Honour 'mode' parameter in SSL.Connection.makefile. Thanks again to Egil
  Muller.
- Roger Binns contributed epydoc-generated docs for M2Crypto. Thanks Roger.
- Peter Teniz contributed patches to create X.509 requests and certificates.
  Thanks Peter.
- Updated Medusa to 0.54.
- Make various OpenSSL bignum functions (written long ago) available to Python.


As usual, find it here:

    http://sandbox.rulemaker.net/ngps/m2/


Feedback is appreciated. Cheers.

--
Ng Pheng Siong <ngps at netmemetic.com>

http://firewall.rulemaker.net -+- Firewall Change Management & Version Control
http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL



From michael at ELYT.COM  Tue Mar 30 09:25:36 2004
From: michael at ELYT.COM (Michael Dunstan)
Date: Tue, 30 Mar 2004 19:25:36 +1200
Subject: [PYTHON-CRYPTO] Memoryleak in SSL.Connection
Message-ID: <703FA3DD-821B-11D8-A5A3-000393B53F20@elyt.com>

I have seen the same leak when running ZServerSSL. Had to restart the
server every few days. Until the following change was made for
setblocking:

      def setblocking(self, mode):
          """Set this connection's underlying socket to _mode_."""
          self.socket.setblocking(mode)
          if mode:
-            self.send = self.write = self._write_bio
+            self.send = self.write = Connection._write_bio
_            self.recv = self.read = self._read_bio
+            self.recv = self.read = Connection._read_bio
          else:
-            self.send = self.write = self._write_nbio
+            self.send = self.write = Connection._write_nbio
-            self.recv = self.read = self._read_nbio
+            self.recv = self.read = Connection._read_nbio

This does have different meaning but I don't think that affects the
functionality of the class in this case.

Since applying the patch we have not had any memory problems for some
time. About 8 months now.

Also while debugging the leak found something that may be of interest.
It turns out that IE does not like the headers that zope creates for
304 (Not modified) responses for images. The result is that IE drops
the connection and creates a new one after each 304 it gets for an
image. Each time this happens I see something like the following in
zope event logs:

2003-05-13T10:40:18 INFO(0) ZServer recv: closing channel
<ZServer.HTTPS_Server.zhttps_channel connected 127.0.0.1:1187 at
0x2852014 channel#: 204 requests:> unexpected eof

(At the time that was m2crypto-0.09, Python 2.1.3, Zope 2.6.0.)

Naturally a site that uses images will serve up a lot of 304's and so
you see a lot of these messages for those users using IE. Also you
see a lot more memory usage in Zope without the above fix :-)

It is quite a simple matter to correct the headers used for the image
304's in Zope. There is even an issue in the zope collector about this:
http://collector.zope.org/Zope/544. Looks like this incorrect handling
of the headers in zope is intentional to support caching in some
relic version of apache configured as a proxy server. In our case apache
is clearly not part of the equation (the transaction is encrypted
between
zope and the browser) so it is safe to correct the headers. This is
simply
of applying a patch of the following form to lib/python/OFS/Image.py:

                  if last_mod > 0 and last_mod <= mod_since:
                      # Set header values since apache caching will
return Content-Length
                      # of 0 in response if size is not set here
                      RESPONSE.setHeader('Last-Modified',
rfc1123_date(self._p_mtime))
-                    RESPONSE.setHeader('Content-Type',
self.content_type)
-                    RESPONSE.setHeader('Content-Length', self.size)
                      RESPONSE.setHeader('Accept-Ranges', 'bytes')
                      self.ZCacheable_set(None)
                      RESPONSE.setStatus(304)
                      return ''

(That's bound to have wrapped and looks a little ugly.)

Michael



From ngps at POST1.COM  Tue Mar 30 17:55:21 2004
From: ngps at POST1.COM (Ng Pheng Siong)
Date: Tue, 30 Mar 2004 23:55:21 +0800
Subject: [PYTHON-CRYPTO] Memoryleak in SSL.Connection
In-Reply-To: <703FA3DD-821B-11D8-A5A3-000393B53F20@elyt.com>
References: <703FA3DD-821B-11D8-A5A3-000393B53F20@elyt.com>
Message-ID: <20040330155521.GB446@vista.netmemetic.com>

On Tue, Mar 30, 2004 at 07:25:36PM +1200, Michael Dunstan wrote:
> I have seen the same leak when running ZServerSSL. Had to restart the
> server every few days. Until the following change was made for
> setblocking:
>
> -            self.send = self.write = self._write_bio
> +            self.send = self.write = Connection._write_bio
> _            self.recv = self.read = self._read_bio
> +            self.recv = self.read = Connection._read_bio
>          else:
> -            self.send = self.write = self._write_nbio
> +            self.send = self.write = Connection._write_nbio
> -            self.recv = self.read = self._read_nbio
> +            self.recv = self.read = Connection._read_nbio
>
> Since applying the patch we have not had any memory problems for some
> time. About 8 months now.

Great! Thanks for the patch Michael.

Now to roll out 0.13p1!

> It is quite a simple matter to correct the headers used for the image
> 304's in Zope. There is even an issue in the zope collector about this:
> http://collector.zope.org/Zope/544. Looks like this incorrect handling
> of the headers in zope is intentional to support caching in some
> relic version of apache configured as a proxy server.

Bad Zope. No biscuit for Zope.


Cheers.

--
Ng Pheng Siong <ngps at netmemetic.com>

http://firewall.rulemaker.net -+- Firewall Change Management & Version Control
http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL



From m24me at KATEHOK.HOMEIP.NET  Tue Mar 30 21:26:52 2004
From: m24me at KATEHOK.HOMEIP.NET (Igor Belyi)
Date: Tue, 30 Mar 2004 21:26:52 +0200
Subject: [PYTHON-CRYPTO] getattr problem in DSA.py
Message-ID: <LISTSERV%2004033021265229@NIC.SURFNET.NL>

Hello, I think there is a problem in DSA.py file distributed in M2Crypto which
prevents from accessing ['p', 'q', 'g', 'pub', 'priv'] fields of the DSA
object.

In the method '__getattr__(self, name)' of the 'class DSA' the line:

method = getattr(m2, 'm2.dsa_get_%s' % (name,))

should acctually be:

method = getattr(m2, 'dsa_get_%s' % (name,))

Sincerely,
Igor



From demirg at ITU.EDU.TR  Wed Mar 31 00:41:50 2004
From: demirg at ITU.EDU.TR (Gokhan Demir)
Date: Wed, 31 Mar 2004 01:41:50 +0300
Subject: [PYTHON-CRYPTO] m2crypto doc
In-Reply-To: <20040330033756.GE626@vista.netmemetic.com>
References: <1079686925.9309.7.camel@go96156.yurtlar.itu.edu.tr>
            <20040330033756.GE626@vista.netmemetic.com>
Message-ID: <1080686509.3313.10.camel@go96156.yurtlar.itu.edu.tr>

> Hi,
>
> Yeah, M2Crypto is weak in docu. I've just released 0.13, which has
> epydoc-generated API docs. Still not much, because the APIs aren't
> documented well, but it's a start. Thanks to Roger Binns for deciding for
> me which doc-generating tool to use. ;-)

I will look at the new release, special thanks to Roger Binns from me,
too ;)

> What protocols do you have in mind to implement?

As the beginning
 Secret Splitting
 Time Stamping
 Bit Commitment
 Fair Coin Flips
then,
 Zero-Knowledge Proofs
 Blind Signatures
and at last
 Secure Elections

At least I hope so.

I tried to install mxCrypto which is an extension to pycrypt and but
while compiling it requested two files whose names are "rc5.h" and
"idea.h". I found them from the internet and placed to include directory
of openssl. Then it complained about another thing which I can't
remember now.

> Cheers.

Thanks.



From ngps at POST1.COM  Wed Mar 31 06:39:57 2004
From: ngps at POST1.COM (Ng Pheng Siong)
Date: Wed, 31 Mar 2004 12:39:57 +0800
Subject: [PYTHON-CRYPTO] getattr problem in DSA.py
In-Reply-To: <LISTSERV%2004033021265229@NIC.SURFNET.NL>
References: <LISTSERV%2004033021265229@NIC.SURFNET.NL>
Message-ID: <20040331043957.GD284@vista.netmemetic.com>

On Tue, Mar 30, 2004 at 09:26:52PM +0200, Igor Belyi wrote:
> In the method '__getattr__(self, name)' of the 'class DSA' the line:
> method = getattr(m2, 'm2.dsa_get_%s' % (name,))
> should acctually be:
> method = getattr(m2, 'dsa_get_%s' % (name,))

Thanks, Igor.

Cheers.

--
Ng Pheng Siong <ngps at netmemetic.com>

http://firewall.rulemaker.net -+- Firewall Change Management & Version Control
http://sandbox.rulemaker.net/ngps -+- Open Source Python Crypto & SSL