Please explain the meaning of 'stealing' a ref

[Alex Martelli]

Christos TZOTZIOY Georgiou wrote:
   ...
>>Yes, you have to incref IF you want to keep owning that reference.
> 
> So, "owning a reference" means simply that some code did an incref and
> is responsible to decref, right?  This is my first request for

"Owning a reference" means "I'll dispose of that reference when I'm
done with it".  As per:

http://www.python.org/doc/current/api/refcountDetails.html

"""
The reference count behavior of functions in the Python/C API is best
explained in terms of ownership of references. Note that we talk of owning
references, never of owning objects; objects are always shared! When a
function owns a reference, it has to dispose of it properly -- either by
passing ownership on (usually to its caller) or by calling Py_DECREF() or
Py_XDECREF(). When a function passes ownership of a reference on to its
caller, the caller is said to receive a new reference. When no ownership is
transferred, the caller is said to borrow the reference. Nothing needs to
be done for a borrowed reference. 
"""

I really don't know how we could have expressed this any more clearly;
if you can indicate what is hard for you to understand here, and how
we could make it easier, I'd be really greateful!

> clarification, although I understand that so far I was confusing "owning
> a reference" and "owning an object".

Ditto: I don't know how we could try to dispell this common doubt
any more clearly than by saying "we talk of owning references, never
of owning objects; objects are always shared!".


>>> decref afterwards yourself.  What's the reason?  Efficiency for
>>
>>You'll decref when you don't want to own that reference any more,
>>and you incref'd it, but if you do that soon, why bother incref'ing?-)
> 
> I'd do that because I did not understand what "stealing a reference"
> means; therefore I translated "stealing" = decref, so I presumed that I
> should issue an incref in advance in order to make sure the object
> passed to the function does not cease to exist before the function
> returns.  I assumed based on unclear (to me) terms.

Hmmm -- wasn't the example on the same URL I just quoted, right after
the 2nd paragrap, clear enough in terms of no incref being needed?
Can you suggest a way in which we could make it clearer to you?


> I understand that.  So, a function "stealing" a reference means the
> function takes away from the caller the responsibility for decref'ing
> the object, and "stealing a reference" does not translate into some C
> code, right?  This is my second request for clarification.

Right.  There is no specific C code to which you can point and say
"see, right here, the reference is being stolen".  If you looked at
the source for e.g. PyTuple_SetItem you might notice it *doesn't*
incref (nor of course decref) its 3rd argument -- it copies that
PyObject* somewhere _and that's it_ -- contrasted with the "normal"
case of e.g. PySequence_SetItem which copies the PyObject* s/where
AND incref's it.  So the "stealing" is the copying-somewhere plus
the LACK of the incref that would normally go with copying, in terms
of C source.


>>> Also, "borrowing" and "stealing" are the same thing?  I just think that
>>
>>No!  You get a borrowed reference when you get a reference but are not
>>transferred the ownership of it.  That's a pretty widespread use regarding
>>references that are returned to objects that surely pre-existed.  You
>>must then incref in the relatively rare case you want to keep hold of
>>that reference for the longer term.
> 
> So a function "borrowing" a reference means that the function does not
> incref or decref an object which was passed to it, right?  That is the
> third request.

The function will incref the reference passed to if IF it's copying
it somewhere (and will then arrange for that copy to be decref'd later
when that reference is not needed any more).  If the function is
just using the reference and is done with it when it returns, then
the function doesn't bother incref'ing it.  The caller keeps ownership
if it originally had it (i.e. unless it was borrowing from somewhere
else, then ownership remains with whoever ultimately had it), there
is no ownership transfer for that reference.

> I assume that most functions of the python API "borrow" references?

Yes, and so do all functions you'll be writing to be callable from
Python -- it's the normal case.

> Thanks for your time.

You're welcome!  Now I'd like, if possible, to enhance the docs so
that other people's future doubts might be similarly helped -- that's
the reason I'm asking you for advice on this!

If you could spend some time to visit safari.oreilly.com -- subscribe
and be sure to cancel before 2 weeks so you don't have to pay! -- and
check the penultimate chapter of Python in a Nutshell, where I've
tried an alternate "concise mixed reference/tutorial" tack to the
whole issue of extending Python, I'd be particularly grateful of any
feedback about that, too... thanks!


Alex