[Python-bugs-list] [ python-Bugs-603724 ] setting file buffer size is unreliable

[noreply@sourceforge.net]

Bugs item #603724, was opened at 2002-09-02 22:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=603724&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeremy Yallop (yallop)
Assigned to: Nobody/Anonymous (nobody)
Summary: setting file buffer size is unreliable

Initial Comment:
The description of open() (ie file()) says:

   "If the buffering argument is given, 0 means
unbuffered, 1 means line buffered, and larger numbers
specify the buffer size."

PyFile_SetBufSize() passes the requested buffer size on
to setvbuf(), with NULL as setvbuf()'s second
parameter.  The C89 standard doesn't guarantee any
change to the buffer size when the second parameter is
NULL, and some stdio implementations (legitimately)
ignore the size parmater in such circumstances.  C99's
gives more guidelines, but nothing that can be relied upon:

   "If buf is not a null pointer, the array it points
to may be used instead of a buffer allocated by the
setvbuf function and the argument size     specifies
the size of the array; otherwise, size may determine
the size of  buffer allocated by the setvbuf function."
(7.19.5.6)

(What good is "may" to anyone?)

The result of all this is that

  fd = open('file', 'w', 8)

will not have the desired (and documented) effect
(flushing the output buffer every 8 characters) on some
platforms, so either the documentation of open() or the
code (PyFile_SetBufSize()) should be fixed.

The same problems exist with setbuf() as well.

----------------------------------------------------------------------

>Comment By: Jeremy Hylton (jhylton)
Date: 2002-09-03 22:35

Message:
Logged In: YES 
user_id=31392

The C99 rationale has a helpful comment on the subject of
setvbuf():

  C90 was not clear about what, if anything, the size
argument means
  when buf is a null pointer.  Existing practice is mixed: some
  implementations ignore it completely, other
implementations use it as
  guidance for determining the size of the buffer allocated
by setvbuf.
  C9X gives warning that size might be ignored in this case,
so portable
  programs must be sure to supply a reasonable value.

I'd be mostly inclined to change the documentation to say
that the buffer is under control of the C library and that
Python does whatever setvbuf() does.  Or however we write
the C99 weasel words in Python docstring-ease.

Thankfully, Linux and Windows both have sane setvbuf()
implementations.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=603724&group_id=5470