Most efficient way to write data out to a text file?

[Bjorn Pettersen]

> From: Dave Kuhlman [mailto:dkuhlman at rexx.com] 
> 
> candiazoo at attbi.com wrote:
> 
> [snip]
> 
> Thank you for posting this.  Very helpful.
> 
> Could we have a bit of evaluation and explanation please.  I'm 
> suspicious about whether some of these changes actually made a 
> difference.  I'd like a little understanding before I start 
> changing my Python coding style.

[snip optimization involving removing free vars, and string concat]

Looking up a name in globals or builtin is very expensive compared to
looking it up in local scope (use the dis module to see the opcodes
involved -- anything that has LOAD_GLOBAL in it is potentially slow).
Most experienced Python programmers will therefore use something like:

  def foo(arg1, arg2, func=MyModule.func):
      ...

after they have profiled the program and found foo to be a bottleneck.
Similarly, looking up an attribute of an object involves searching its
__dict__. If you save the returned value in a local, you basically get a
fast "load from array" operation instead.

When it comes to string concatenation it is an O(n**2) operation, so
it's usually better to do:

  def foo():
      result = []
      append = result.append
      for i in xrange(100000):
          append(`i`)	# `i` is faster than repr(i)
      return ''.join(result)

I would say the string concat optimization is always a good idea, the
global->local optimization is ok as long as it doesn't interfer with
readability -- other people will surely disagree <wink>.

-- bjorn