getting fileinput to do errors='ignore' or 'replace'?

[Serhiy Storchaka]

On 04.12.15 00:26, Oscar Benjamin wrote:
> On 3 Dec 2015 16:50, "Terry Reedy" <tjreedy at udel.edu> wrote:
>> fileinput is an ancient module that predates iterators (and generators)
> and context managers. Since by 2.7 open files are both context managers and
> line iterators, you can easily write your own multi-file line iteration
> that does exactly what you want.  At minimum:
>>
>> for file in files:
>>      with codecs.open(file, errors='ignore') as f
>>      # did not look up signature,
>>          for line in f:
>>              do_stuff(line)
>
> The above is fine but...
>
>> To make this reusable, wrap in 'def filelines(files):' and replace
> 'do_stuff(line)' with 'yield line'.
>
> That doesn't work entirely correctly as you end up yielding from inside a
> with statement. If the user of your generator function doesn't fully
> consume the generator then whichever file is currently open is not
> guaranteed to be closed.

You can convert the generator to context manager and use it in the with 
statement to guarantee closing.

with contextlib.closing(filelines(files)) as f:
     for line in f:
         ...