speedy Python strings?

[Josiah Carlson]

> Another possibility is to buffer things in a list, using append to add 
> to the end and pop(0) to pop off the front (a roll-your-own queue).  
> Lists do not suffer from the quadratic behavor that string addition does.

Deleting from the front of a list results in every item being shifted up
a single entry.  While this is insignificant for short lists, it is
significant for long lists:

>>> def del_from_front(size):
...     a = range(size)
...     t = time.time()
...     while a: a.pop(0)
...     return time.time()-t
...
>>> del_from_front(1000)
0.016000032424926758
>>> del_from_front(10000)
0.85899996757507324
>>> del_from_front(20000)
3.2970001697540283
>>> def del_from_end(size):
...     a = range(size)
...     t = time.time()
...     while a: a.pop()
...     return time.time()-t
...
>>> del_from_end(10000)
0.062999963760375977
>>> del_from_end(100000)
0.51600003242492676
>>> del_from_end(1000000)
5.4530000686645508



In terms of buffering as the parent post asks, file IO is already
buffered by the underlying libraries and operating system.  Another
layer of Python buffering doesn't seem to make much sense.  I would
suggest writing off the buffering exercise as a learning experience.

 - Josiah