[Python-Dev] Re: [Patches] Reference cycle collection for Python

[Tim Peters]

Very briefly:

[Guido]
> ...
> Today, Eric proposed to do away with Neil's hash table altogether --
> as long as we're wasting memory, we might as well add 3 fields to each
> container object rather than allocating the same amount in a separate
> hash table.  Eric expects that this will run faster, although this
> obviously needs to be tried.

No, it doesn't <wink>:  it will run faster.

> Container types are: dict, list, tuple, class, instance; plus
> potentially user-defined container types such as kjbuckets.  I
> have a feeling that function objects should also be considered
> container types, because of the cycle involving globals.

Note that the list-migrating steps you sketch later are basically the same
as (but hairier than) the ones JimF and I worked out for M&S-on-RC a few
years ago, right down to using appending to effect a breadth-first traversal
without requiring recursion -- except M&S doesn't have to bother accounting
for sources of refcounts.  Since *this* scheme does more work per item per
scan, to be as fast in the end it has to touch less stuff than M&S.  But the
more kinds of types you track, the more stuff this scheme will have to
chase.

The tradeoffs are complicated & unclear, so I'll just raise an uncomfortable
meta-point <wink>:  you balked at M&S the last time around because of the
apparent need for two link fields + a bit or two per object of a "chaseable
type".  If that's no longer perceived as being a showstopper, M&S should be
reconsidered too.

I happen to be a fan of both approaches <wink>.  The worst part of M&S-on-RC
(== the one I never had a good answer for) is that a non-cooperating
extension type E can't be chased, hence objects reachable only from objects
of type E never get marked, so are vulnerable to bogus collection.  In the
Neil/Toby scheme, objects of type E merely act as  sources of "external"
references, so the scheme fails safe (in the sense of never doing a bogus
collection due to non-cooperating types).

Hmm ... if both approaches converge on keeping a list of all chaseable
objects, and being careful of uncoopoerating types, maybe the only real
difference in the end is whether the root set is given explicitly (as in
traditional M&S) or inferred indirectly (but where "root set" has a
different meaning in the scheme you sketched).

> ...
> In our case, we may need a type-specific "clear" function for containers
> in the type object.

I think definitely, yes.

full-speed-sideways<wink>-ly y'rs  - tim