bugs in `gc.get_referents()'

[Michael Hudson]

Zooko <zooko at zooko.com> writes:

> [Would this be more appropriate for the Python developers' list?]

You're looking for

    http://sourceforge.net/tracker/?group_id=5470&atid=105470

> Hello Pythonistas, Pythonics and Pythonifiers!
> 
> I'm using Python 2.2b2 as compiled and packaged by debian.
> 
> I've been using `gc.get_objects()' and `gc.get_referents()' (since renamed 
> `gc.get_referrers()') to find out how my app Mojo Nation[1] uses memory.
> 
> (Thank you to the implementors of those functions!  It has been a
> great help!)
> 
> There are two apparent bugs: first, the return value from get_referents() 
> *always* contains a number of entries that do not refer to the object in 
> question, and it is always the same entries.

Wild, wild stab in the dark: try running gc.collect() first.

It's hard to see how this could be happening from staring at the code.
It's not very complicated.

[...] 
> The second bug is that about once every few hours of use,
> get_referents() seg faults.  I think it might be more likely to do
> that when I have a larger collection of objects in memory -- maybe
> the time to traverse the larger set of objects reveals a race
> condition?

It's possible, I'd hazard.  You could try surrounding your calls to
calls to gc.enable/gc.disable.

What do the stack traces show?

> I'm running gc.get_referents() in an interactive interpreter while
> several threads are simultaneously working and mutating the data!

I *doubt* threads are running during the call to gc.get_referrers.
Could be wrong.

> Also some of those threads are using native code which might be
> failing to DECREF properly.

If that's the case, then you've got problems all your own, no?

> I would be rather interested in digging into the implementation of
> get_referents() myself, but first I'd like to hear from the
> Pythonistas whether these bugs are surprising or unsurprising,

Well, the functions are pretty new.

> whether the code has been changed since 2.2b2,

Only the name changes, I think.

> if there is any sort of "starting point" for digging into it that
> they recommend, etc.

$ less Modules/gcmodule.c

I think.

Cheers,
M.

-- 
  We did requirements and task analysis, iterative design, and user
  testing. You'd almost think programming languages were an interface
  between people and computers.                    -- Steven Pemberton
          (one of the designers of Python's direct ancestor ABC)