Buffering of sys.stdout and sys.stderr in python3 (and documentation)

[Terry Reedy]

On 12/9/2011 2:32 PM, Geoff Bache wrote:
> Hi all,
>
> Short version:
>
> I'm a bit confused in general as to the changes between python2 and
> python3 regarding how standard output and standard error do buffering.
> A few things seem to have changed and I've failed to find any
> documentation of how and why. Also, the meaning of "python -u" seems
> to have changed and the docs don't seem to reflect the new behaviour
> (and I can't find any docs about the change either)...
>
> Long version:
>
>> From rude experiment it seems that:
> 1) In Python 2.x, standard error was always unbuffered while standard
> output was buffered by default.  In python3, both are buffered. In
> both cases, "buffered" means line-buffered when writing to the console
> and not line-buffered when redirected to files.
> 2) In Python 2.x, the "-u" flag meant everything was totally
> unbuffered. In Python 3.x, it means that both stdout and stderr are
> line-buffered also when redirected to files.
>
> Are either of these changes documented anywhere? (1) seems important :
> it can lead to not seeing exception printouts, if stderr is redirected
> to a file and the program is subsequently terminated with SIGTERM. I
> just wasted quite a bit of time due to this situation...
>
> This is what the Python 3 docs have to say about the -u flag:
>
> "Force the binary layer of the stdin, stdout and stderr streams (which
> is available as their buffer attribute) to be unbuffered. The text I/O
> layer will still be line-buffered."
>
> The "still" seems misleading to me, as it is only relevant if writing
> to the console. It would be useful to contrast the behaviour with and
> without "-u" when writing to files I would say.

The difference from 2.x should be in What's New in 3.0, except that the 
new i/o module is in 2.6, so it was not exactly new. You might be able 
to find more in http://python.org/dev/peps/pep-3116/

You *should* be able to find sufficient info in the 3.x docs. If, after 
you get other responses (or not), you think the docs need upgrading, 
open an issue on the tracker at bugs.python.org with suggestions as 
specific as possible, including changed or new lines of text based on 
your experience and experiments.

-- 
Terry Jan Reedy