[Patches] [ python-Patches-951915 ] fix bug in StringIO.truncate - length not changed

[SourceForge.net]

Patches item #951915, was opened at 2004-05-11 09:07
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=951915&group_id=5470

Category: Library (Lib)
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 6
Submitted By: David Fraser (davidfraser)
Assigned to: Raymond Hettinger (rhettinger)
Summary: fix bug in StringIO.truncate - length not changed

Initial Comment:
If truncate() is called on a StringIO object, the
length is not changed, so that seek(0, 2) calls will go
beyond the correct end of the file.
This patch adds a line to update the length, and a test
to the test method that checks that it works.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2004-12-20 18:57

Message:
Logged In: YES 
user_id=80475

Applied as Lib/StringIO.py 1.38 and 1.37.2.1

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2004-12-20 07:55

Message:
Logged In: YES 
user_id=4771

The patch should be reviewed by someone and then applied if no further issue is discovered, I guess.

----------------------------------------------------------------------

Comment By: David Fraser (davidfraser)
Date: 2004-12-20 04:58

Message:
Logged In: YES 
user_id=221678

Armin, are there any problems with your refined patch that
you are aware of? Once that is applied the other issues
could be looked at.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2004-12-20 04:25

Message:
Logged In: YES 
user_id=4771

The current implementation already delays concatenation and typically only uses a single ''.join() to build the final buffer, so it's reasonably efficient.

What I was musing about in my previous post is a way to entierely avoid the copy overhead and let all operations deal directly with a list of strings as the basic data structure (or maybe a tree of strings, or some more advanced structure).  Repeated writes of very small strings should probably still be consolidated into larger strings (e.g. of a few KBs) but no single huge string (or array) needs to be built at all.

But this is just musing aloud, and this thread is about a bug in StringIO.py, so we should probably focus on fixing it...

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-06-09 13:16

Message:
Logged In: YES 
user_id=80475

To avoid repeated string concatenation, the underlying data 
structure could be changed to an array.array object with a 
character typecode.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2004-05-12 11:59

Message:
Logged In: YES 
user_id=4771

David, it seems to me that f.truncate(huge_value) would incorrectly set f.len to huge_value with your patch.  Here is another patch fixing this and the other details I mentioned.  I also put the new test into test_StringIO.py instead.  Perhaps we should remove the if __name__=='__main__' bit, although it is nice as a quick example.

This makes me wonder if there is any reason left for which cStringIOs aren't subclassable, or if we care.

Alternatively, it makes me wonder if there wouldn't be a more efficient implementation of StringIO.py that would entierely avoid concatenating large strings, or if we care.  This might make StringIO at least as efficient as cStringIO for some cases, e.g. when writing a lot of strings a few kb each, by avoiding the copy overhead.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2004-05-12 10:34

Message:
Logged In: YES 
user_id=4771

StringIO.py needs to be reviewed.  I could spot several other (though more minor) problems in a couple of minutes.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=951915&group_id=5470