Unbuffered stderr in Python 3

[George Trojan]

-------- Forwarded Message --------
Subject: 	Re: Unbuffered stderr in Python 3
Date: 	Tue, 03 Nov 2015 18:03:51 +0000
From: 	George Trojan <george.trojan at noaa.gov>
To: 	python-list at python.org



On 11/03/2015 05:00 PM, python-list-request at python.org wrote:
> On Mon, 02 Nov 2015 18:52:55 +1100, Steven D'Aprano wrote:
>
>> In Python 2, stderr is unbuffered.
>>
>> In most other environments (the shell, C...) stderr is unbuffered.
>>
>> It is usually considered a bad, bad thing for stderr to be buffered. What
>> happens if your application is killed before the buffer fills up? The
>> errors in the buffer will be lost.
>>
>> So how come Python 3 has line buffered stderr? And more importantly, how
>> can I turn buffering off?
> It's probably related to the fact that std{in,out,err} are Unicode
> streams.
>
> 	> type(sys.stderr)
> 	<class '_io.TextIOWrapper'>
> 	> type(sys.stderr.buffer)
> 	<class '_io.BufferedWriter'>
> 	> type(sys.stderr.buffer.raw)
> 	<class '_io.FileIO'>
>
> It appears that you can turn it off with:
>
> 	sys.stderr = io.TextIOWrapper(sys.stderr.buffer.raw)
> or:
> 	sys.stderr = io.TextIOWrapper(sys.stderr.detach().detach())
>
> This results in a sys.stderr which appears to work and whose
> .line_buffering property is False.
>
>
This does set line buffering, but does not change the behaviour:

(devenv-3.4.1) dilbert at gtrojan> cat x.py
import sys
import time
if sys.version>'3':
     import io
     sys.stderr = io.TextIOWrapper(sys.stderr.detach().detach())
     #sys.stderr = io.TextIOWrapper(sys.stderr.buffer.raw)
     print(sys.stderr.line_buffering)
sys.stderr.write('a')
time.sleep(10)

This is python2.7.5. a is printed before ^C.

(devenv-3.4.1) dilbert at gtrojan> /bin/python x.py
a^CTraceback (most recent call last):

Here buffer is flushed on close, after typing ^C.//

(devenv-3.4.1) dilbert at gtrojan> python x.py
False
^CaTraceback (most recent call last):

George

Found it. write_through must be set to True.

(devenv-3.4.1) dilbert at gtrojan> cat x.py
import sys
import time
if sys.version>'3':
     import io
     sys.stderr = io.TextIOWrapper(sys.stderr.detach().detach(), 
write_through=True)
     #sys.stderr = io.TextIOWrapper(sys.stderr.buffer.raw)
     print(sys.stderr.line_buffering)
sys.stderr.write('a')
time.sleep(10)
(devenv-3.4.1) dilbert at gtrojan> python x.py
False
a^CTraceback (most recent call last):

/

/