[Python-Dev] Usage of += on strings in loops in stdlib

[Steven D'Aprano]

On 13/02/13 20:09, Chris Withers wrote:
> On 12/02/2013 21:03, Maciej Fijalkowski wrote:
>> We recently encountered a performance issue in stdlib for pypy. It
>> turned out that someone commited a performance "fix" that uses += for
>> strings instead of "".join() that was there before.
>
> That's... interesting.
>
> I fixed a performance bug in httplib some years ago by doing the exact opposite; += -> ''.join(). In that case, it changed downloading a file from 20 minutes to 3 seconds. That was likely on Python 2.5.


I remember it well.

http://mail.python.org/pipermail/python-dev/2009-August/091125.html

I frequently link to this thread as an example of just how bad repeated string concatenation can be, how painful it can be to debug, and how even when the optimization is fast on one system, it may fail and be slow on another system.



-- 
Steven