[Python-ideas] proto-PEP: Fixing Non-constant Default Arguments

[Jan Kanis]

On Tue, 30 Jan 2007 08:09:37 +0100, Chris Rebert <cvrebert at gmail.com>  
wrote:

> I also reworded the sentences using 'shall'. I don't really get why that  
> matters, but what the heck.

Well, it doesn't. Just makes it nicer to read. (As long as the text is  
still clear, that is.)


> Jan Kanis wrote:
>> Well, I (obviously) like the idea, but your pep misses some important  
>> points (and some not so important).
>>  The important one is about the scoping of variables in default  
>> expressions. The pep says nothing about them.
>> If I read the pep correctly, any variables in default expressions are  
>> handled the same as variables in the body of a function. This means the  
>> compiler decides if they should be local, lexical or global. If there  
>> is an assignment to a variable, the compiler makes it a local, else it  
>> finds the right enclosing scope (lexical or global). In the current  
>> python, this works fine:
>>   >>> a = 123
>>  >>> def foo(b=a):
>>           a = 2
>>           print a, b
>>   >>> foo()
>>  2 123
>>  >>> a = 42
>>  >>> foo()
>>  2 123
>>  In the pep, the a in the default expression would be handled just like  
>> any other a in the function body, which means it wil become a _local_  
>> variable. Calling the function would then result in an  
>> UnboundLocalError. Just like this in current python:
>>   >>> a = 123
>>  >>> def foo(b=None):
>>           b = a if b==None else b
>>           a = 2
>>           print a
>>   >>> foo()
>>   Traceback (most recent call last):
>>    File "<pyshell#22>", line 1, in <module>
>>      foo()
>>    File "<pyshell#21>", line 2, in foo
>>      b = a if b==None else b
>>  UnboundLocalError: local variable 'a' referenced before assignment
>>  The solution, I think, as I wrote in my previous messages, is to have  
>> the compiler explicitly make variables in default expressions lexical  
>> or global variables.
>> This would still break the foo() in my example above, because you can't  
>> assign to a lexical variable, or to a global that isn't declared  
>> global. Therefore I think the compiler should distinguish between the a  
>> in the default expression and the a in the function body, and treat  
>> them as two different variables. You can think about it as if the  
>> compiler silently renames one of them (without this rename being  
>> visible to python code.  AFAIK the bytecode is stack based and closure  
>> vars are put in some kind of anonymous cell, which means they both  
>> don't actually have a name anyway.)


Also, I've been thinking about the part that I suggested about having  
python distinguish between lexically scoped variables in default  
expressions and local variables that may have the same name.
To clarify my proposal, a demonstration:

Current python:

a = 4
b = 5
def foo(x = [b]*a):
	x = copy.deepcopy(x)  # one possible workaround to have the same list on  
every call
	b = 'bla'         # no name clash of this b and b in default arg expr
	x += [b]
	print x

foo()
  # prints [5, 5, 5, 5, 'bla']
foo()
  # prints [5, 5, 5, 5, 'bla']

proposed semantics:

a = 4
b = 5
def foo(x = [b]*a):
	 # deepcopy() no longer nescessary
	b = 'bla'   # no name clash of this b and b in default arg expr, the  
compiler distinguishes them
	x += [b]
	print x

foo()
  # prints [5, 5, 5, 5, 'bla']
foo()
  # prints [5, 5, 5, 5, 'bla']


This would work the same as this code in the current python. Imagine it as  
the compiler silently transforming the above code to this code:

a = 4
b = 5
def foo(x = _undefined):
	if x == _undefined:
		x = [b]*a
	_b = 'bla'
	x += [_b]
	print x

foo()
  # prints [5, 5, 5, 5, 'bla']
foo()
  # prints [5, 5, 5, 5, 'bla']

note: the use of deepcopy assumes that whatever is passed to foo is  
deepcopyable. With the new semantics this function can be used without  
these kind of assumptions.

However, I've been thinking about weather it is a good idea to have python  
distinguish between the local and lexical 'b' in the above code, and do  
the what you can think of as implicit renaming. Making the distinction  
makes the whole proposal more backward compatible, however I think that if  
backward compatibility were not an issue, it would be best not to make the  
distinction and just have the programmer choose a different name for the  
local variable. This would improve readability as there are no two  
different variables with the same name in the same piece of code, and  
decrease language complexity as there is no rule that's best explained in  
terms of variable name rewriting.
Not making the distinction does cause the default arguments proposal to be  
less backward compatible, but I don't know if that is really a big  
problem. I assume this pep wil be a py3k pep anyway, so there are going to  
be incompatibilities anyway. Also, the compiler is perfectly able to see  
at compile time if a variable used in a default expression and as a local,  
so it is able to raise a very descriptive error.
All in all, I think this part of my proposal is not such a good idea,  
unless there's going to be some kind of transitional implementation of  
this pep in py 2.x

meta-note: I hope it is clear what I'm trying to say. Basically, I'm  
clarifying, arguing against, and retracting a part of my own proposal I  
made in an earlier mail.


- Jan