string concatentation

[Tony Rentschler]

In comp.lang.python, you wrote:
>----- Original Message -----
>From: "Tony Rentschler" <tonyr at walterr.bellatlantic.net>
>Subject: string concatentation
>
>
>> I'm writing a program that reads individual lines of text from stdin,
>which
>> is filled by piping the output from another program into mine. I need
>> to collect the individual lines I've read into one large string. My first
>> thought was to do something like this: big_string = big_string +
>new_string.
>>
>> In practice, will this be inefficient? That is, will Python malloc a new
>> big_string each time I add a line? Is there another way to handle this
>sort
>> of situation, say with a pre-allocated buffer of some kind?
>
>Desperately inefficient.  Yes, you get a new malloc() and a free() also
>(which
>may be deferred but will eventually happen).  The most common idiom for
>handling
>many lines of input is like so:
>
># code is 1.5.2 compatible, update to 2.0/2.1 if you really care
>
>import string, sys
>
>line_list = []
>input_line = sys.stdin.readline()
>
>while input_line:
>    line_list.append(input_line)
>    input_line = sys.stdin.readline()
>
>entire_file = string.join(line_list, "")
>
>This will get the final result you are wanting... but if reading the file
>into
>a string is ALL you want to do (that is, you aren't doing something ELSE in
>that
>loop), do this:
>
>entire_file = sys.stdin.read()
>
>One malloc(), basically.
>

Thanks, the list idiom is just what I was looking for. I'm new
to Python and am still learning how best to accomplish certain tasks. In
this case, I can't read() the whole file in one fell swoop because the
upstream program sends it line by line. If the list method is more efficient,
does that then mean that Python lists allocate more space than required as
items are added so as to grow efficiently?