Copying data between file-like objects

[Fredrik Lundh]

Thomas Lotze wrote:

>> if you don't know how large f2 can be, use shutil.copyfileobj:
>>
>>     >>> help(shutil.copyfileobj)
>>     Help on function copyfileobj in module shutil:
>>
>>     copyfileobj(fsrc, fdst, length=16384)
>>         copy data from file-like object fsrc to file-like object fdst
>
> This sounds like what I was looking for. Thanks for the pointer.
> However, the following doesn't seem like anything is being copied:
>
>>>> from StringIO import StringIO
>>>> from shutil import copyfileobj
>>>> s = StringIO()
>>>> s2 = StringIO()
>>>> s.write('asdf')
>>>> copyfileobj(s, s2)
>>>> s2.getvalue()

copyfileobj copies from the current location, and write leaves the file
pointer at the end of the file.  a s.seek(0) before the copy fixes that.

>> to copy stringio objects, you can use f1 = StringIO(f2.getvalue()).
>
> But this should have the same problem as using read(): a string will be
> created on the way which contains all the content.

getvalue() returns the contents of the f2 file as a string, and f1 will use that
string as the buffer.  there's no extra copying.

> Because I want to manipulate a copy of the data and be able to compare it
> to the original afterwards.

why not just use a plain string (or a list of strings)?  your focus on StringIO sounds
like a leftover from some C library you've been using in an earlier life ;-)

> Another thing I'd like to do is copy parts of a StringIO object's content
> to another object. This doesn't seem possible with any shutil method. Any
> idea on that?

use a plain string and slicing.  (if you insist on using StringIO, use seek and read)

> What one can really wonder, I admit, is why the difference between holding
> data two or three times in memory matters that much, especially if the
> latter is only for a short time. But as I'm going to use the code that
> handles the long string as a core component to some application, I'd like
> to make it behave as well as possible.

use plain strings, so you know what you're doing.

</F>