Finding the instance reference of an object

[Joe Strout]

First, I want to thank everyone for your patience -- I think we're  
making progress towards a consensus.

On Nov 6, 2008, at 8:48 PM, Steven D'Aprano wrote:

>> that's
>> pretty much the whole point: that variables in Python don't contain
>> objects, but merely contain references to objects that are actually
>> stored somewhere else (i.e. on the heap).
>
> You're wrong, Python variables don't contain *anything*. Python  
> variables
> are names in a namespace.

I think we're saying the same thing.  What's a name?  It's a string of  
characters used to refer to something.  That which refers to something  
is a reference.  The thing it refers to is a referent.  I was trying  
to avoid saying "the value of an object reference is a reference to an  
object" since that seems tautological and you don't like my use of the  
word "value," but I see you don't like "contains" either.

Maybe we can try something even wordier: a variable in Python is, by  
some means we're not specifying, associated with an object.  (This is  
what I mean when I say it "refers" to the object.)  Can we agree that  
far?

Now, when you pass a variable into a method, the formal parameter gets  
associated with same object the actual parameter was associated with.   
I like to say that the object reference gets copied into the formal  
parameter, since that's a nice, simple, clear, and standard way of  
describing it.  I think you object to this way of saying it.  But are  
we at least in agreement that this is what happens?

> But putting that aside, consider the Python
> code "x = 1". Which statement would you agree with?
>
> (A) The value of x is 1.

Only speaking loosely (which we can get away with because numbers are  
immutable types, as pointed out in the last section of [1]).

> (B) The value of x is an implementation-specific thing which is
> determined at runtime. At the level of the Python virtual machine, the
> value of x is arbitrary and can't be determined.

Hmm, this might be true to somebody working at the implementation  
level, but I think we're all agreed that that's not the level of this  
discussion.  What's relevant here is how the language actually  
behaves, as observable by tests written in that language.

> If you answer (A), then your claim that Python is call-by-value is  
> false.

Correct.

> If you answer (B), then your claim that Python is call-by-value is  
> true
> but pointless, obtuse and obfuscatory.

Correct again.  My answer is:

(C) The value of x is a reference to an immutable object with the  
value of 1.  (That's too wordy for casual conversation so we might  
casually reduce this to (A), as long as we all understand that (A) is  
not actually true.  It's a harmless fiction as long as the object is  
immutable; it becomes important when we're dealing with mutable  
objects.)

>> This is explicitly stated in
>> the Python docs [1], yet many here seem to want to deny it.
>
>> [1] http://www.python.org/doc/2.5.2/ext/refcounts.html
>
> You have a mysterious and strange meaning of the word "explicitly".  
> Would
> you care to quote what you imagine is this explicit claim?

A few samples: "The chosen method is called reference counting. The  
principle is simple: every object contains a counter, which is  
incremented when a reference to the object is stored somewhere, and  
which is decremented when a reference to it is deleted. When the  
counter reaches zero, the last reference to the object has been  
deleted and the object is freed.   ...Python uses the traditional  
reference counting implementation..."

This seems like a point we really shouldn't need to argue.  Do you  
really want to defend the claim that Python does not use references?

> Yes, you are right, Python does not offer pass by reference. The
> canonical test for "call by reference" behaviour is to write a  
> function
> that does this:
>
> x = 1
> y = 2
> swap(x, y)
> assert x == 2 and y == 1
>
> If you can write such a function, your language may be call-by- 
> reference.
> If you can't, it definitely isn't c-b-r. You can't write such a  
> function
> in standard Python, so Python isn't c-b-r.

Whew!  That's a relief.  A week ago (or more?), it certainly sounded  
like some here were claiming that Python is c-b-r (usually followed by  
some extended hemming and hawing and except-for-ing to explain why you  
couldn't do the above).

> The canonical test for "call by value" semantics is if you can write a
> function like this:
>
> x = [1]  # an object that supports mutation
> mutate(x)
> assert x == [1]
>
> If mutations to an argument in a function are *not* reflected in the
> caller's scope, then your language may be call-by-value. But if  
> mutations
> are visible to the caller, then your language is definitely not c-b-v.

Aha.  So, in your view, neither C, nor C++, nor Java, nor VB.NET are c- 
b-v, since all of those support passing an object reference into a  
function, and using that reference to mutate the object.

Your view is at odds with the standard definition, though; in fact I'm  
pretty sure we could dig up C and Java specs that explicitly spell out  
their c-b-v semantics, and RB and VB.NET pretty clearly mean "ByVal"  
to indicate by-value in those languages.

The canonical test of c-b-v is whether a *reassignment* of the formal  
parameter is visible to the caller.  Simply using the parameter for  
something (such as dereferencing it to find and change data that lives  
on the heap) doesn't prove anything at all about how the parameter was  
passed.

> Python is neither call-by-reference nor call-by-value.

That can be true only if at least one of the following is true:

1. Python's semantics are different from C/C++ (restricted to  
pointers), Java, and RB/VB.NET; or
2. C/C++ (restricted to pointers), Java, and RB/VB.NET are not call-by- 
value.

I asked you before which of these you believed to be the case, so we  
could focus on that, but I must have missed your reply.  Can you  
please clarify?

 From your above c-b-v test, I guess you would argue point 2, that  
none of those languages are c-b-v.  If so, then we can proceed to  
examine that in more detail.  Is that right?

>> That would indeed be nonsense.  But it's also not what I'm saying.  
>> See
>> [2] again for a detailed discussion and examples.  Call-by-value and
>> call-by-reference are quite distinct.
>
> And also a false dichotomy.

I've never claimed these are the only options; just that they're the  
only ones actually used in any of the languages under discussion.  If  
you think Python uses call by name, call by need, call by macro  
expansion, or something else at [2], please do say which one.  "Call  
by object", as far as I can tell, is just a made-up term for call-by- 
value when the value is an object reference.  (And I'm reasonably OK  
with that as long as we're all agreed that that is what it means.)

>>> "Calling by value" is not a useful definition of Pythons behaviour.
>>
>> It really is, though.  You have to know how the formal parameter  
>> relates
>> to the actual parameter.  Is it a copy of it, or an alias of it?
>
> And by definition, "call by value" means that the parameter is a  
> copy. So
> if you pass a ten megabyte data structure to a function using call-by-
> value semantics, the entire ten megabyte structure is copied.

Right.  And if (as is more sensible) you pass a reference to a ten MB  
data structure to a function using call-by-value, then the reference  
is copied.

> Since this does not happen in Python, Python is not a call-by-value
> language. End of story.

So your claim is that any language that includes references (which is  
all OOP languages, as far as I'm aware), is not call-by-value?

>> Without knowing that, you don't know what assignments to the formal
>> parameter will do, or even what sort of arguments are valid. Answer:
>> it's a copy of it.
>
> Lies, all lies. Python doesn't copy variables unless you explicitly  
> ask
> for a copy.

Hmm, I'm struggling to understand why you would say this.  Perhaps you  
mean that Python doesn't copy *objects* unless you explicitly ask for  
a copy.  That's certainly true.  But it does copy references in many  
circumstances, including in assignment statements, and parameter  
passing.

> That some implementations of Python choose to copy pointers
> rather than move around arbitrarily large blocks of memory instead  
> is an
> implementation detail. It's an optimization and irrelevant to the
> semantics of argument passing in Python.

I agree that under the hood, there are probably other ways to get the  
same behavior.  What's important is to know whether the formal  
parameter is an alias of the actual parameter, or its own independent  
local variable that (let me try to say it more like your way here)  
happens to be initially associated with the same referent as the  
actual parameter.  This obviously has behavioral consequences, as you  
showed above.

A concise way to describe the behavior of Python and other languages  
is to simply say: the object reference is copied into the formal  
parameter.

>> Assignments don't affect the actual parameter at
>> all.  This is exactly what "call by value" means.
>
> Nonsense. I don't know where you get your definitions from, but it  
> isn't
> a definition anyone coming from a background in C, Pascal or Fortran
> would agree with.

Well I can trivially refute that by counterexample: I come from a  
background in C, Pascal, and FORTRAN, and I agree with it.

As for where I get my definitions from, I draw from several sources:

1. Dead-tree textbooks
2. Wikipedia [2] (and yes, I know that has to be taken with a grain of  
salt, but it's so darned convenient)
3. My wife, who is a computer science professor and does compiler  
research
4. http://javadude.com/articles/passbyvalue.htm (a brief but excellent  
article)
5. Observations of the "ByVal" (default) mode in RB and VB.NET
6. My own experience implementing the RB compiler (not that  
implementation details matter, but it forced me to think very  
carefully about references and parameter passing for a very long time)

Not that I'm trying to argue from authority; I'm trying to argue from  
logic.  I suspect, though, that your last comment gets to the crux of  
the matter, and reinforces my guess above: you don't think c-b-v means  
what most people think it means.  Indeed, you don't think any of the  
languages shown at [1] are, in fact, c-b-v languages.  If so, then we  
should focus on that and see if we can find a definitive answer.

Best,
- Joe

[1] http://www.strout.net/info/coding/valref/
[2] http://en.wikipedia.org/wiki/Evaluation_strategy