[ python-Bugs-1376775 ] Memory leak in the email package

[SourceForge.net]

Bugs item #1376775, was opened at 2005-12-08 17:03
Message generated for change (Comment added) made by ken668
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1376775&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: ken668 (ken668)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: Memory leak in the email package

Initial Comment:
memory leak in email.message_from_string. This is what
I did to create a leak.

You used the attached file, memleak.eml.

f = open("memleak.eml")
buffer = f.read()
f.close()

# now buffer has the email string
msg = email.message_from_string(buffer)
msg = None # this should free the memory but it doesn't

# The memory that is used in msg is not completely free


----------------------------------------------------------------------

>Comment By: ken668 (ken668)
Date: 2005-12-20 14:11

Message:
Logged In: YES 
user_id=1400763

I assume the new email package has cyclic references in it.
My program has garabage collection disabled because of
performance issues. That means the email package will cause
memory leak in my program. I will use the old package
instead. Thanks for the help.



----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2005-12-20 11:03

Message:
Logged In: YES 
user_id=38376

Nope.

If you do msg=None, you'll release a reference to the
message object, and all direct, non-cyclic members.
Any cyclic references held by the message object (or
created by the email parser) will be removed at a later
time, by the garbage collector.

And even if you add an explicit gc.collect() call to
your program, that won't remove the contents of the
*buffer* variable.

(If you run the program in a loop *without* an explicit
gc.collect() call, the process will appear to grow for
a while, since you're creating objects faster than the
GC can remove them.  Add gc.collect() to the loop, and
the memory use will be rock solid.)

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-20 10:37

Message:
Logged In: YES 
user_id=1400763

May be you did not look at my test program more closely.
Please look at the statement msg=None. The should removed
all references of the object. The memory will definely grow
if you repeat the operation.  Why don't you run the
email.message_from_string in a loop and see whether the
memory grows. Thanks.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2005-12-20 08:44

Message:
Logged In: YES 
user_id=38376

This is not a leak (a leak means that the process will
grow if you *repeat* the operation, not that things are
unexpectedly left in the object memory).

And it's not a bug, either; the "copy" of the message
that's returned by get_objects() is the contents of the
buffer variable in the test program.

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-14 22:44

Message:
Logged In: YES 
user_id=1400763

Yes, it is a bug. My mistake was when I gave the details, I
should have mentioned gc.get_objects(), not gc.garbage.
gc.get_objects() showed the leaks not gc.garbage.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-12-14 21:56

Message:
Logged In: YES 
user_id=33168

Do you still believe there is a problem or can this report
be closed?

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-12 08:00

Message:
Logged In: YES 
user_id=1400763

My mistake. I had also called gc.get_objects() too. It was
gc.get_objects() that outputed the content of the email
message. It was not gc.garbage. Sorry about that.

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-12 07:59

Message:
Logged In: YES 
user_id=1400763

My mistake. I had also called gc.get_objects() too. It was
gc.get_objects() that outputed the content of the email
message. It was not gc.garbage. Sorry about that.

----------------------------------------------------------------------

Comment By: ken668 (ken668)
Date: 2005-12-12 07:53

Message:
Logged In: YES 
user_id=1400763

I added these three lines after the line "msg=None"

import gc
print "gc.collect()\n\n", gc.collect()
print "gc.garbage\n\n", gc.garbage

If you pipe the output of gc.garbage to a file, you will see
the email message you just sent are still be in the memory.

Everytime I call email.message_from_string, a copy of the
message will be kept in the memory even after I set the
returned value to None.

I tried the same thing with email package email-2.5.tar.gz,
that memory was freed right after I set the variable "msg"
to None.

Thanks

Ken



----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-12-11 12:13

Message:
Logged In: YES 
user_id=33168

What causes you to believe this is a memory leak?  I ran
this under valgrind and it doesn't report any leaks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1376775&group_id=5470