Speed quirk: redundant line gives six-fold speedup

[Jack Diederich]

On Thu, Aug 25, 2005 at 04:44:24PM +0000, Mark Dickinson wrote:
> I have a simple 192-line Python script that begins with the line:
> 
> dummy0 = 47
> 
> The script runs in less than 2.5 seconds.  The variable dummy0 is never
> referenced again, directly or indirectly, by the rest of the script.
> 
> Here's the surprise: if I remove or comment out this first line, the
> script takes more than 15 seconds to run.  So it appears that adding a
> redundant line produces a spectacular six-fold increase in speed!
> 
> (Actually, I had to add 29 dummy lines at the beginning of the code to
> get the speed increase; if any one of these lines is removed the
> running time reverts to around 15 seconds again.)
> 
> Questions:
> 
> (1) Can anyone else reproduce this behaviour, or is it just some quirk
>     of my setup?

I get the same thing.

> (2) Any possible explanations?  Is there some optimization that kicks
>     in at a certain number of lines, or at a certain length of
>     bytecode?

It seems to be related to the number of globals.  I get the "fast"
version with 30 to 120 globals and the "slow" version with less than 30
or more than 130.  It actually gets even slower for higher numbers
of globals.

Here is a snippet to adjust the number of globals

for (i) in range(100):
  globals()['dummy%d' % (i)] = 1

> (3) If (2), is there some way to force the optimization, so that I can
>     get the speed increase without having to add the extra lines?

Yes, module level globals have bad lookup times compared to function
local names.  If you refactor your code to pass around the data
currently at the global module level you should see times at least
as fast as the current 'fast' one.

That said, I'm very surprised that the lookup times jump around so much.
Your code does bazillions of namespace lookups, so a small difference
in lookup times is getting multiplied into some really big numbers.

-jackdied