[issue20501] fileinput module will read whole file into memory when using fileinput.hook_encoded

[Zachary Ware]

Zachary Ware added the comment:

The new test passes on Windows with the whole patch applied, but fails without the changes to fileinput.py.  Is this change meant to fix behavior, or just the memory usage issue?

Just for completeness, here's the failure output (with unpatched fileinput.py):

======================================================================
FAIL: test_modes (__main__.Test_hook_encoded)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "P:\ath\to\2.7\cpython\lib\test\test_fileinput.py", line 235, in test_modes
    check('r', [u'A\n', u'B\r\n', u'C\rD\u20ac'])
  File "P:\ath\to\2.7\cpython\lib\test\test_fileinput.py", line 233, in check
    self.assertEqual(lines, expected_lines)
AssertionError: Lists differ: [u'A\n', u'B\r\n', u'C\r', u'D... != [u'A\n', u'B\r\n', u'C\rD\u20a...

First differing element 2:
C
D\u20ac

First list contains 1 additional elements.
First extra element 3:
D\u20ac

- [u'A\n', u'B\r\n', u'C\r', u'D\u20ac']
?                         -----

+ [u'A\n', u'B\r\n', u'C\rD\u20ac']

----------------------------------------------------------------------

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20501>
_______________________________________