Early binding as an option

[Chris Angelico]

On Wed, Aug 3, 2011 at 11:16 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> Chris Angelico wrote:
>> Of course; that's a different issue altogether. No, I'm talking about
>> the way a tight loop will involve repeated lookups for the same name.
>
> It's not really a different issue. The "standard" approach for performing
> early binding in Python is by binding a global name to a local name at
> function definition time.

Thanks for the detailed response!

My concept was that this wouldn't affect the dynamism significantly,
and that if the function's caller wanted to change the definition of
'len', s/he could still do so; the "snapshotting" would occur as the
function begins executing. This would make things rather safer, as
there's a limited window during which this could affect things.

> In CPython at least, local lookups are faster
> than globals: locals are stored in a fixed array, and the function knows
> the numeric offset of each variable.

Ah! I was not aware of this, and thought that locals were a dictionary
too. Of course, it makes a lot of sense. In that case, the classic
"grab it as a local" isn't just loading down the locals dictionary
with more names and thus slower lookups.

>        Do not supply the 'int', 'default', and 'maxwidth' arguments.
>        """

I love the way Python doesn't stop you from doing stupid things. This
is an invitation to do something... hmm.
>>> random.randrange(5,10,1,int=lambda x: (print(x),int(x))[1])

> But, are you *sure* that name lookups are *really* the bottleneck? Name
> lookups are pretty fast. If you want them faster, turn them into a local
> variable. It's not clear to me that syntax, or a pragma directive, or some
> other form of magic, is better than an explicit assignment:

No, I'm not sure. Unfortunately I have no convenient way to compare;
all I can do is compare with a completely different language, which is
of course NEVER going to be fair. The only actual data I have on the
subject is the perfect-numbers search the other day; Pike managed the
same task in a fraction of the time that Python spent on it. Pike has
a single integer type that quietly switches from small to large at
2**32, with a noticeable performance change. This is the same as
Python 2, but could explain some of the Python 3 penalty. There's only
two obvious possibilities (there may be others, of course): firstly,
that the actual name lookup is expensive; and secondly, that Pike is
able to optimize integer arithmetic by knowing that the value in
question is an int, and it will never be anything else. Since I was
under the impression that local variables were stored in a hash table,
I concluded that the first option was reasonably likely, and tried to
get more data.

>> So... Would this potentially produce wrong results? Would it be of any
>> use, or would its benefit be only valued in times when the whole
>> function needs to be redone in C?
>
> You really would need to demonstrate that the bottleneck in useful code (as
> opposed to toy examples) was the name lookups.

And especially, prove that the same can't be done with local variables.

So which is the better idiom?

def func(x):
   _len = len  # bind at function call time
   for i in x:
       _len(i)  # lookups of _len will be faster than len

or:

def func(x):
   len = len  # localize len
   for i in x:
       len(i)  # use it exactly as you otherwise would

In other words, is it "Explicit is better than implicit" or "Simple is
better than complex"? Since this does absolutely exactly the same
thing as the original, I'd be inclined to go with the second form; a
single line at the top of the function, with an explanatory comment
perhaps, and the rest of the code is unaware of the optimization. But
this has the potential to be insanely confusing if anything goes
wrong, because down in the body of the function there's no clue that
anything was changed.

Unfortunately I can't offer any real-world use cases or examples. This
is purely a thought experiment at the moment.

Chris Angelico