Reading from sys.stdin reads the whole file in

[Naoki INADA]

I recommend Python 3.

On Python 2, iterating lines without buffering is slow, tricky and ugly.




for line in iter(sys.stdin.readline(), ''):

    print line
—
Sent from Mailbox

On Wed, Aug 27, 2014 at 3:03 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Wed, Aug 27, 2014 at 3:19 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> When I pipe one to the other, I expect each line to be printed as they
>> arrive, but instead they all queue up and happen at once:
> You're seeing two different problems here. One is the flushing of
> stdout in out.py, as Marko mentioned, but it's easily proven that
> that's not the whole issue. Compare "python out.py" and "python
> out.py|cat" - the latter will demonstrate whether or not it's getting
> flushed properly (the former, where stdout is a tty, will always flush
> correctly).
> But even with that sorted, iterating over stdin has issues in Python
> 2. Here's a tweaked version of your files (note that I cut the sleeps
> to 2 seconds, but the effect is the same):
> rosuav at sikorsky:~$ cat out.py
> import time
> print("Hello...",flush=True)
> time.sleep(2)
> print("World!",flush=True)
> time.sleep(2)
> print("Goodbye!",flush=True)
> rosuav at sikorsky:~$ cat slurp.py
> from __future__ import print_function
> import sys
> import time
> for line in sys.stdin:
>         print(time.ctime(), line)
> rosuav at sikorsky:~$ python3 out.py|python slurp.py
> Wed Aug 27 16:00:16 2014 Hello...
> Wed Aug 27 16:00:16 2014 World!
> Wed Aug 27 16:00:16 2014 Goodbye!
> rosuav at sikorsky:~$ python3 out.py|python3 slurp.py
> Wed Aug 27 16:00:19 2014 Hello...
> Wed Aug 27 16:00:21 2014 World!
> Wed Aug 27 16:00:23 2014 Goodbye!
> rosuav at sikorsky:~$
> With a Py2 consumer, there's still buffering happening. With a Py3
> consumer, it works correctly. How to control the Py2 buffering,
> though, I don't know.
> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20140826/735afa05/attachment.html>