sys.stdout vs. sys.stderr

[Mitchell L Model]

On Jan 11, 2010, at 1:47 PM Nobody <nobody at nowhere.com> wrote:

>
>> On Mon, 11 Jan 2010 10:09:36 +0100, Martin v. Loewis wrote:
>>
>>> In Python 3.1 is there any difference in the buffering behavior of  
>>> the
>>> initial sys.stdout and sys.stderr streams?
>>
>> No.
>>
>>> Were they different at some earlier point in Python's evolution?
>>
>> That depends on the operating system. These used to be whatever the
>> C library set up as stdout and stderr. Typically, they were buffered
>> in the same way.
>
> On Unix, stdout will be line buffered if it is associated with a tty
> and fully buffered otherwise, while stderr is always unbuffered.
>
> On Windows, stdout and stderr are unbuffered if they refer to a  
> character
> device, fully buffered otherwise (Windows doesn't have line buffering;
> setvbuf(_IOLBF) is equivalent to setvbuf(_IOFBF)).
>
> ANSI C says:
>
> As initially opened, the standard error stream is not fully  
> buffered; the
> standard input and standard output streams are fully buffered if and  
> only
> if the  stream can be determined not to refer to an interactive  
> device.
>

I don't want to get into a quibble fight here, but I need to reraise  
this issue.
[I teach and write and want to make sure I get this right. I already  
have an
incorrect paragraph about this in my Bioinformatics Programming Using  
Python
book.] The key question here is line buffering vs full buffering. In  
Unix (at least
in an OS X Terminal), the following code prints a number every two  
seconds
in Python 2:

	>>> for n in range(5):
	. . .  		print >> sys.stderr, n,     # final , to not send newline
	. . .		time.sleep(2)

However, in Python 3, similar code does not print the numbers until  
the whole
thing finishes (again, running from the terminal).

	>>> for n in range(5):
	. . .  		print(n, file=sys.stderr, end='')
	. . .		time.sleep(2)

So it appears that in a Unix terminal window, Python 2 does not line- 
buffer stderr
whereas Python 3 does. That's what tripped me up. While developing and  
debugging
code, I often print periods on a line as some loop progresses  
(sometimes every Nth
time around, for some reasonable N) just to know the pace of execution  
and that
the program is still doing something. In doing that recently in Python  
3 I discovered that
I either had to leave out the end='' or do sys.stderr.flush() after  
every print, which
amounts to the same thing.

This was a big surprise, after many, many years of C, C++,
Java, and Python programming -- I have always thought of stderr as  
completely
unbuffered in languages that have it. Doesn't mean some languages line- 
buffer
stderr on some platforms, just pointing out an assumption I've lived  
with for a very
long time that tripped me up writing a note about using stderr in  
Python 3 without
actually demonstrating the code and therefore not catching my error.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100309/8c4c64a9/attachment.html>