Python handles globals badly.

[Steven D'Aprano]

On Sat, 12 Sep 2015 01:03 am, Ian Kelly wrote:

> On Fri, Sep 11, 2015 at 2:42 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
[...]
>> Almost. If it's never assigned within the function, then it is looked up
>> according to the non-local scoping rules:
>>
>> - closures and enclosing functions (if any);
>> - globals;
>> - builtins;
>>
>> in that order.
> 
> I excluded non-locals intentionally, but if you want to be pedantic
> about it, then that's still not quite right. Non-locals are indeed
> identified by the compiler and compiled with the
> LOAD_DEREF/STORE_DEREF opcodes (rather than the _GLOBAL and _FAST
> variants used by globals and locals, respectively). 

Ah, nice, yes I forgot about that, thanks for the correction.


> The compiler 
> doesn't make any such distinction between globals and builtins
> however, as that can only be determined at run-time.
> 
>> There's also a bunch of specialised and complicated rules for what
>> happens if you make a star import ("from module import *") inside a
>> function, or call eval or exec without specifying a namespace. Both of
>> these things are now illegal in Python 3.
> 
> Huh?
> 
>>>> exec("x = 42")
>>>> x
> 42
>>>> exec("x = 43", None, None)
>>>> x
> 43
> 
> That's in Python 3.4.0. Maybe I don't understand what you mean by
> "without specifying a namespace".

Inside a function star imports are illegal in Python 3:

py> def f():
...     from math import *
...
  File "<stdin>", line 1
SyntaxError: import * only allowed at module level


My recollection was incorrect about exec. You can still exec inside a
function, but it may have no effect:

py> def f():
...     x = 1
...     exec("x = 2")
...     return x
...
py> f()
1

You can specify locals, but it doesn't help:

py> def f():
...     x = 1
...     exec("x = 2", globals(), locals())
...     return x
...
py> f()
1


However, in Python 2, Python tried hard to make exec work:

py> def f():
...     x = 1
...     exec("x = 2")
...     return x
...
py> f()
2


I don't recall all the details, but in Python 2 functions could use two
different schemes for local variables: the regular, optimized one using
memory slots, and a dict-based one that came into play with exec. So we
have this:

py> def g():
...     a = 1
...     exec("b = 2")
...     return (a, b)
...
py> dis.dis(g)
  2           0 LOAD_CONST               1 (1)
              3 STORE_FAST               0 (a)

  3           6 LOAD_CONST               2 ('b = 2')
              9 LOAD_CONST               0 (None)
             12 DUP_TOP
             13 EXEC_STMT

  4          14 LOAD_FAST                0 (a)
             17 LOAD_NAME                0 (b)
             20 BUILD_TUPLE              2
             23 RETURN_VALUE



`a` is a regular, optimized local looked up with LOAD_FAST; but `b` gets the
same old LOAD_NAME used for globals and built-ins.


In Python 3.3, that same function uses LOAD_GLOBAL for `b`, even though the
variable does actually exist:


py> dis.dis(g)
  2           0 LOAD_CONST               1 (1)
              3 STORE_FAST               0 (a)

  3           6 LOAD_GLOBAL              0 (exec)
              9 LOAD_CONST               2 ('b = 2')
             12 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             15 POP_TOP

  4          16 LOAD_FAST                0 (a)
             19 LOAD_GLOBAL              1 (b)
             22 BUILD_TUPLE              2
             25 RETURN_VALUE
py> g()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in g
NameError: global name 'b' is not defined



The conclusion I draw from all this is that the rules governing local
variables in Python are a mess :-)


-- 
Steven