Readlines returns non ASCII character

[Ian Kelly]

On Wed, Sep 23, 2015 at 6:47 AM, SANKAR . <shankarphy at gmail.com> wrote:
> Hi all,
>
> I am not a expert programmer but I have to extract information from a large
> file.
>  I used  codecs.open(..) with UTF16 encoding to read this file. It could
> read all the lines in the file but returns with the non Ascii characters.
> Below are 5 sample lines. How do I avoid having this non Ascii items. Is
> there a better way to read this?

I suspect that what you want is not "non-ASCII" but just to read the
file without all the mojibake, which is likely an indication that
you're using the wrong encoding.

Do you know that UTF-16 is actually the encoding of the file?

Based on the spaces that appear between adjacent characters, I would
guess that this is probably in a 32-bit encoding, perhaps UTF-32. On
the other hand, the repeated 0x00ff 0x00fe 0x00ff are very curious; I
don't see how that could be valid UTF-32. Are you sure that this is a
text file and not some propietary binary data format?