[ python-Bugs-844561 ] codecs.open().readlines(sizehint) bug

[SourceForge.net]

Bugs item #844561, was opened at 2003-11-18 18:22
Message generated for change (Comment added) made by lemburg
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=844561&group_id=5470

Category: Unicode
Group: Python 2.2
Status: Open
Resolution: None
>Priority: 3
Submitted By: Jeff Epler (jepler)
Assigned to: M.-A. Lemburg (lemburg)
Summary: codecs.open().readlines(sizehint) bug

Initial Comment:
codecs.open().readlines(sizehint) can return truncated
lines.  The attached script, which uses
readlines(sizehint) to count the number of lines in a
file, demonstrates the problem.  Correct output would
be 1000 in both cases, but different values are
returned depending on sizehint because of the truncated
lines.

----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2004-02-26 00:04

Message:
Logged In: YES 
user_id=38388

It's hard to say whether this is a bug or not. The sizehint
argument is not well documented and the way you use it
does not look a proper way to use it.

>From the docs:
""""
f the optional sizehint argument is present, instead of
reading up to EOF, whole lines totalling approximately
sizehint bytes (possibly after rounding up to an internal
buffer size) are read. 
""""

In your example the underlying open() implementation 
seems to round up the sizehint value to include the whole
line, while the codec.open() version will only read sizehint
bytes without any rounding (see the codecs.py 
implementation).


----------------------------------------------------------------------

Comment By: Jeff Epler (jepler)
Date: 2003-11-18 18:28

Message:
Logged In: YES 
user_id=2772

The script triggers the assertion error using at least
python 2.3.2 (locally compiled) and python 2.2.2 (redhat 9 RPM)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=844561&group_id=5470