A couple garbage collector questions

[Daniel Berlin]

On 25 Apr 2001, Hannah Schroeter wrote:

> Hello!
>
> In article <mailman.987747988.15494.python-list at python.org>,
> Daniel Berlin  <dan at www.cgsoftware.com> wrote:
>
>
> >[...]
>
> >As for data on the other side, i've never seen non-atomic reference
> >counting exact a huge performance penalty, in a well-designed application.
> >INCREF is a macro that expands to (with depythonified equivalent names) :
> >object->refcount++.
> >DECREF is a macro that expands to (with depythonified equivalent names):
> >if  (object->refcount--) free(object)
>
> And in a threaded environment, that is in fact:
> INCREF ->
> 	lock(refcnt_mutex)
> 	object->refcount++
> 	unlock(refcnt_mutex)
>
> DECREF ->
> 	lock(refcnt_mutex)
> 	object->refcount--
> 	if (! object->refcount)
> 		free(object)
> 	unlock(refcnt_mutex)
>

Except, you never bothered to look before saying this.
Because in python, it's not.

> The refcnt_mutex can either be global (much contention) or per refcnt
> (memory overhead for lock datastructure in addition to the refcnt
> itself). Or it could be per ((size_t)object) >> A_FEW_BITS pages,
> or anything similar.
>
> >I'm pretty curious as to how you think this will exact a huge performance
> >cost.
>
> >Cache locality isn't hurt, why the heck would you decref an object you
> >hadn't just used? Why the heck would you incref an object you weren't
> >about to use?
>
> Objects you fill into collections/remove out of them? The actual reference
> to the object instance data may be arbitrarily near/far away from the refcnt
> operations.

>
> >So where is the huge performance cost i'm paying?
>
> See, one assignment
> 	x = y
> is in general in fact an
> 	if (x != nil)
> 		DECREF(x)
> 	if (y != nil)
> 		INCREF(y)
> 	x = y
>
> So, expanded, that is:
> 	if (x != nil) {
> 		lock(x->refcnt_lock)
> 		x->refcnt --
> 		unlock(x->refcnt_lock)
> 		if (! x->refcnt)
> 			free(x)
> 	}
> 	if (y != nil) {
> 		lock(y->refcnt_lock)
> 		y->refcnt++
> 		unlock(y->refcnt_lock)
> 	}
> 	x = y
>
> On real GC's, it is just:
> 	x = y
>
> No lock overhead and potential lock contention, and even in the single
> threaded case (no locks), you still have a significant instruction overhead.
>
You don't, because the reference counting is not atomic on python. Well,
it's not atomic in and of itself. It's atomic because we guarantee only
one python thread running at once, on a given interpreter.

So you don't have the above.

No offense, but please *look* at the damn code before you claim what it
does.

Reference counting in python is not atomic because of locks around every
refcnt, and thus, you don't pay that cost.

> Except if  you achieve to eliminate most INCREF/DECREFs through intelligent
> global dataflow analysis.
>
> Kind regards,
>
> Hannah.
> --
> http://mail.python.org/mailman/listinfo/python-list
>