[ python-Bugs-1377858 ] segfaults when using __del__ and weakrefs

[SourceForge.net]

Bugs item #1377858, was opened at 2005-12-10 13:20
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1377858&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: Carl Friedrich Bolz (cfbolz)
Assigned to: Michael Hudson (mwh)
Summary: segfaults when using __del__ and weakrefs

Initial Comment:
You can segfault Python by creating a weakref to an
object in its __del__ method, storing it somewhere and
then trying to dereference the weakref afterwards. the
attached file shows the described behaviour.

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2007-01-17 10:38

Message:
Logged In: YES 
user_id=357491
Originator: NO

I have just been waiting on someone to do a final code review on it.  As
soon as someone else signs off I will commit it.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-01-16 23:02

Message:
Logged In: YES 
user_id=33168
Originator: NO

Brett, Michael, Armin, can we get this patch checked in for 2.5.1?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-08-19 21:31

Message:
Logged In: YES 
user_id=357491

After finally figuring out where *list was made NULL (and
adding a comment about it where it occurs), I added a test
to test_weakref.py .  Didn't try to tackle classic classes.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-08-12 04:31

Message:
Logged In: YES 
user_id=4771

The clear_weakref(*list) only clears the first
weakref to the object.  You need a while loop
in your patch. (attached proposed fix)

Now we're left with fixing the same bug in
old-style classes (surprize surprize), and
turning the crasher into a test.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 10:36

Message:
Logged In: YES 
user_id=357491

So after staring at this crasher it seemed to me to be that
clearing the new weakrefs w/o calling their finalizers after
calling the object's finalizer was the best solution.  I
couldn't think of any other good way to communicate to the
new weakrefs that the object they refer to was no longer
viable memory without doing clear_weakref() work by hand.

Attached is a patch to do this.  Michael, can you have a look?

----------------------------------------------------------------------

Comment By: Georg Brandl (birkenfeld)
Date: 2006-01-10 11:29

Message:
Logged In: YES 
user_id=1188172

Added to outstanding_crashes.py.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2006-01-09 03:58

Message:
Logged In: YES 
user_id=6656

Hmm, maybe the referenced mayhem is more to do with clearing __dict__ than

calling __del__.  What breaks if we do things in this order:

1. call __del__
2. clear weakrefs
3. clear __dict__

?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2006-01-09 03:54

Message:
Logged In: YES 
user_id=6656

Hmm, I was kind of hoping this report would get more attention.

The problem is obvious if you read typeobject.c around line 660: the
weakref 
list is cleared before __del__ is called, so any weakrefs added during the

execution of __del__ are never informed of the object's death.  One fix
for this 
would be to clear the weakref list _after_ calling __del__ but that led to
other 
mayhem in ways I haven't boethered to understand <wink> (see SF bug 
#742911).  I guess we could just clear out any weakrefs created in __del__

without calling their callbacks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1377858&group_id=5470