[Patches] [ python-Patches-439364 ] GC for frames and generators
noreply@sourceforge.net
noreply@sourceforge.net
Mon, 09 Jul 2001 14:08:22 -0700
Patches item #439364, was opened at 2001-07-07 15:32
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=439364&group_id=5470
Category: core (C code)
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Neil Schemenauer (nascheme)
Assigned to: Tim Peters (tim_one)
Summary: GC for frames and generators
Initial Comment:
For me, test_generators leaks, with or without this
patch. The number of objects tracked by the GC does
not increase but memory usage does. There must be
other objects creating cycles that are not tracked by
the GC.
I haven't profiled the effect this change. I have
another patch in the works that should minimize the
performance hit.
Please examine frame_traverse and frame_clear closely.
I'm not 100% sure that I am chasing all the references
necessary to find cycles or that I am dropping enough
to break them.
----------------------------------------------------------------------
>Comment By: Tim Peters (tim_one)
Date: 2001-07-09 14:08
Message:
Logged In: YES
user_id=31435
The attached fibleak.py is a self-contained massive leaker.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2001-07-09 13:48
Message:
Logged In: YES
user_id=31435
+ Performance hit is about a 2.5% slowdown. More than I
hoped, but less than I feared <wink>.
+ The leak after the patch is solely due to fun_tests.
Comment out the other lines in the __test__ dict, and on my
box it leaks about half a megabyte per second then -- very
dramatic, and impossible to ascribe to anything but a leak.
+ I don't know what causes the leak. Offhand my first
guess would be unchased pointers inside traceback objects.
Indeed, the first thing I'd *try* is adding tracebackobject
to GC too just to see whether that made the leak go away
(much easier than thinking about it <0.9 wink>).
----------------------------------------------------------------------
Comment By: Neil Schemenauer (nascheme)
Date: 2001-07-09 07:11
Message:
Logged In: YES
user_id=35752
Your right, it doesn't leak before the patch. The process
size goes up slowly be eventually stops increasing. After
the patch the size continues going up slowly. :-( I'll
try to find what is still leaking.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2001-07-08 21:40
Message:
Logged In: YES
user_id=31435
Thanks, Neil! I've been too overwhelmed with merging descr-
branch code to look at this until now.
Clarification, please: when you say "test_generators
leaks, with or without this patch", what exactly do you
mean by "leak"?
It does not leak for me before the patch (haven't yet tried
it with the patch), and this is what I mean-- exactly --
by "does not leak": I change a stock CVS build by
changing "if 0:" to "if 1:" in test_generators.py's
test_main, then let it run and run and run.
Now on Win98SE, the heap space *does* grow very slowly and
steadily, until it reaches about 3600Kb. Sometimes a
little more, sometimes a little less, depending on what
else I'm doing at the time. It takes about 5 minutes of
CPU time on a 866MHz box to reach that. But then it
*stays* at this point forever after (well, after another 10
minutes of CPU time it hasn't budged -- Win98SE probably
can't run forever <wink>).
This is presumably because Win98SE malloc is lazy about
coalescing free()'d memory until it nears a 4Mb boundary
(at which point it starts rearranging VM address space--
because 4Mb is all the address space the system allocates
to the initial heap --and it's very reluctant to do that).
So if you're seeing a very slow "leak" (are you?), you
*may* just be seeing a platform malloc reluctant to
defragment free()'d memory.
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=439364&group_id=5470