Values and objects

[Steven D'Aprano]

On Sun, 11 May 2014 09:18:34 +1000, Chris Angelico wrote:

> On Sun, May 11, 2014 at 5:10 AM, Ethan Furman <ethan at stoneleaf.us>
> wrote:
>> And if you don't like that argument (although it is a perfectly sound
>> and correct argument), think of the module name space:
>>
>>
>> ret = spam
>> spam = 23
>>
>> will net you a simple NameError, because spam has not yet been created.
> 
> What about this, though:
> 
> ret = int
> int = 23
>
> That will *not* net you a NameError, because 'int' exists in an outer
> scope (builtins). You can create a new module-scope variable and it will
> immediately begin to shadow a builtin; you can delete that variable and
> it will immediately cease to shadow that builtin. 

Yes. That's part of documented language behaviour to do with scoping 
search paths. See below.


> That's the difference
> I'm talking about. With function-local variables, they all have to exist
> (as other responses confirmed, that *is* a language guarantee), even
> though some of them aren't bound to anything yet.

Define "exist". 

Just because a slot exists waiting for a variable, doesn't mean that the 
variable which would use that slot exists. Like dicts and lists, function 
locals() are over-allocated: CPython pre-allocates slots for locals based 
on compile-time information, even if those locals are never bound to and 
therefore don't exist:

def demo():
    if False:
        x = 23
    print x  # Fails

You're trying to argue that because there is a slot for x, the variable x 
exists. But I think that is the wrong definition of "exists". If you have 
a list L = [], would you say that L[4] exists because the array is over-
allocated and already has (at least) four slots waiting to be used, but 
L[100] doesn't because it isn't over-allocated that much? What if the pre-
allocation rules change? What if an implementation decides not to bother 
pre-allocating empty lists?

What if an implementation pre-allocates a random number of array slots? 
Would you be comfortable saying that there is a 25% chance that L[4] 
exists and a 1% chance that L[100] exists?

In both these cases, array slots in a list or dict, and locals, we risk 
mistaking implementation details for language features. I think you 
actually are making this error when it comes to locals.

When talking about the implementation, it is relevant to discuss the 
existence of slots. But when talking about the language, about features 
visible from Python code such as variables (a.k.a. name bindings), local 
slots don't matter. Jython and IronPython don't (so far as I know) use 
them, and locals() happens to be writable. Here's an example from 
IronPython 2.6:

>>> def test():
...     if False: spam = None
...     d = locals()
...     d['spam'] = 23
...     return spam
...
>>> test()
23


And a similar example from Jython 2.5:

>>> def test():
...     locals()['spam'] = 42
...     return spam
...     spam = None
...
>>> test()
42


Neither example works in CPython, you get an UnboundLocalError, but 
whether or not locals() is writable is is *explicitly* documented as an 
implementation detail. How implementations store local variables (in a 
dictionary like globals, slots, in the Cloud) is up to them, so long as 
the implementation follows the scoping rules.

Python determines whether a variable is local or not at compile-time:

(1) If a function contains a binding operation (assignment; import; del) 
on that variable, then the variable is defined as a local unless 
otherwise declared as a global or nonlocal (Python 3 only).

(2) Otherwise it is not a local.

(3) Outside of a function, it is a global whether declared as such or not.

(4) Variables (names) don't exist until they have a value bound to them.

I guess you want to challenge #4, and say that local variables exist as 
soon as the slot that holds them exists. I think that is wrong. And 
besides, even CPython 2 doesn't always use slots for locals: consider 
what happens when you use import * inside a function. And try running 
this function in both 2.7 and 3.3 and see if you can explain the 
difference:

def test():
    if False: x = None
    exec("x = 1")
    return x


But I digress.

Name look-ups for locals look only in the local scope. Name lookups for 
everything else search the chain nonlocals:globals:builtins.

Name bindings (assignments, del, imports) for locals write only to the 
local scope. For globals they write only to the global scope and for 
nonlocals they write to the nearest enclosing nonlocal scope that already 
defines that variable.



-- 
Steven D'Aprano
http://import-that.dreamwidth.org/