Pass-by-reference : Could a C#-like approach work in Python?

[Stephen Horne]

I get to the reason I'm rapidly changing my mind on this idea at the
end of this post, so you may want to read the last few paragraphs
first, but...


On Wed, 10 Sep 2003 13:04:01 -0700, Erik Max Francis <max at alcyone.com>
wrote:

>Stephen Horne wrote:
>
>> Its not an issue of whether the same effect can be achieved in Python.
>> It's more an issue of whether the method used expresses the
>> programmers intentions properly.
>
>I agree.  And with your suggestion, now every single function call can
>potentially change meaning.  Given Python's dynamicism, you may not even
>know if this particular function call exhibits the behavior.  Now
>Python's nice, predictable behavior is up for grabs.

I don't agree with this.

If the caller has to specify the behaviour in the syntax they use for
the call AS WELL AS the definition (in both cases using the 'ref'
modifiers) then the call mechanism is 100% explicit and predictable.
If, given Pythons dynamism, you have an function object of the wrong
type (and therefore one which does not support the call mechanism you
are trying to use) then it will fail - just as attempting to do a
conventional function call on a non-function will fail. 

That is, suppose I define a function...

  def Call_This (p_Function) :
    x = 1
    p_Function (ref x)

This function doesn't know what type of object it will be called with
- it *should* be a function with a reference argument but it might be
a conventional function, it might be a function with the wrong number
of arguments, or it might not be a function at all.

But it doesn't really matter because I have explicitly specified how I
want to call that object - if the object doesn't support that call
mechanism it simply fails.

So if i write...

  def Example_Fn_1 (p) :
    None

  Call_This (Example_Fn_1)

then an exception will be raised because I provided the wrong type of
function as a parameter. But this is not really any different to what
we should already expect if I write...

  Call_This ("oops, this isn't a function")

>If you want to rebinding outside the function, a far better way of
>"expressing the programmer's intentions properly," in my opinion, is to
>make the behavior as explicit as possible:
>
>	class Container:
>	    def __init__(self, x=None): self.x = x
>	    def get(self): return self.x
>	    def set(self, x): self.x = x
>	    def increment(self): self.x += 1 # for example
>
>	def Inc(c):
>	    container.increment()
>
>	container = Container(1)
>	Inc(container)
>	print container.get()

Of course this handles the increment example and it handles most cases
quite well, but there are many cases where this is inappropriate.
Putting an object into a mutable wrapper *purely* as a way to allow a
function to rebind the object it needs to rebind is a bad idea - it
means creating extra housekeeping code that (1) obscures the real
application logic, and (2) creates extra opportunities for bugs.

But of course my putting it that way is bogus - in reality, most data
would be in mutable objects (class instances) anyway in real world
programs, and the use of return values (including returning a tuple)
can handle returning any number of output values.

In other words, the following *is* bad...

  variable = Wrapper (variable)
  Call_Function (variable)
  variable = Wrapper.Unwrap ()

But realistically, that wouldn't happen. Rather, you'd have...

  variable = Call_Function (variable)

And so we get back to Peter Ottens point - Python doesn't need
call-by-reference (or rather call-by-reference-to-reference) because
it has another way of providing output values - and significantly, one
which allows multiple values to be conveniently returned.

As a fan of many functional techniques and as someone who
instinctively distrusts side-effects, I actually like this. But a
little voice keeps whispering to me "yes, but not always".

Part of it is of course that I still feel it is a wart that, when
passing a mutable object as a parameter, I can rebind any internal
component of that object and it will be visible to the caller - but if
I rebind the object as a whole the change doesn't affect the caller.
Of course we've been through this before, though, and I doubt you want
to revisit the arguments any more than I do.

The most compelling example I have is essentially the side-effect
thing from my reply to Peter Ottens post. As an attempt to express it
more clearly, there are times when you set up an object purely so that
you can store it into a container...

  obj = Obj_Type ()

  obj.Do_Setup_Stuff_1 ()
  obj.Do_Setup_Stuff_2 ()
  obj.Do_Setup_Stuff_3 ()

  container.Add (obj)

In many cases, the Add function will not take a copy because it seems
pointless. The caller set up the object purely to store it in the
container, and in most cases is going to discard its reference to the
object immediately afterwards. But then there's always the possibility
that the caller might carry on using that object. For instance...

  obj = Obj_Type ()

  obj.Do_Setup_Stuff_1 ()
  obj.Do_Setup_Stuff_2 ()
  obj.Do_Setup_Stuff_3 ()

  container.Add (obj)

  obj.Do_Setup_Stuff_3_Slightly_Differently ()

  container.Add (obj)

This probably wouldn't do what was expected - the 'slightly
differently' call changed an object that is also being referenced from
inside the container - an accidental side-effect.

If a 'ref' parameter is used and the 'Add' function rebinds it to
'None', however, we get...

  obj = Obj_Type ()

  obj.Do_Setup_Stuff_1 ()
  obj.Do_Setup_Stuff_2 ()
  obj.Do_Setup_Stuff_3 ()

  container.Add (ref obj)

  obj.Do_Setup_Stuff_3_Slightly_Differently ()
    #  exception here tells that something has gone wrong

  container.Add (ref obj)

Which quickly gets bugfixed to...

  obj = Obj_Type ()

  obj.Do_Setup_Stuff_1 ()
  obj.Do_Setup_Stuff_2 ()
  obj.Do_Setup_Stuff_3 ()

  container.Add_Copy (obj)

  obj.Do_Setup_Stuff_3_Slightly_Differently ()
    #  no exception as Add_Copy doesn't rebind the parameter

  container.Add (ref obj)


Then again, even in this case there are other ways. For instance...

  builder = Builder ()

  builder.Do_Setup_Stuff_1 ()
  builder.Do_Setup_Stuff_2 ()
  builder.Do_Setup_Stuff_3 ()

  container.Add (builder.Build ())

  builder.Do_Setup_Stuff_3_Slightly_Differently ()

  container.Add (builder.Build ())

In this case, the builder class can set a 'used' flag after the
'Build' call which tells it that it needs to make a copy before
applying further changes. In that way, both unnecessary copies and
accidental side-effects are avoided while keeping the intention clear
(especially as this is basically one of the classic design patterns
from Gamma et al - though I may be confusing Builder with Factory
Method) and with no changes to Python.

So my 'most compelling example' isn't compelling at all. Damn.

I've been thinking on this for some hours now and still not come up
with a more compelling example (or at least not one which can't be
better handled a different way) so I guess the idea is a dud.

Bet you never expected me to admit that so easy ;-)