Movie (MPAA) ratings and Python?

[Dan Stromberg]

On Wed, Dec 11, 2013 at 5:01 PM, Ned Batchelder <ned at nedbatchelder.com>wrote:

> On 12/11/13 6:39 PM, Dan Stromberg wrote:
>
>>
>> On Wed, Dec 11, 2013 at 3:24 PM, Steven D'Aprano
>> <steve+comp.lang.python at pearwood.info
>> <mailto:steve+comp.lang.python at pearwood.info>> wrote:
>>
>>     On Wed, 11 Dec 2013 15:07:35 -0800, Dan Stromberg wrote:
>>
>>      >  $ chardet mpaa-ratings-reasons.list
>>      > mpaa-ratings-reasons.list: windows-1255 (confidence: 0.97)
>>      >
>>      > I'm aware that chardet is playing guessing games, though one
>>     would hope
>>      > it would guess well most of the time, and give a reasonable
>>     confidence
>>      > rating.
>>
>>     What reason do you have for thinking that Windows-1255 isn't a
>>     reasonable
>>     guess? If the bulk of the text is Latin-1 except perhaps for one or
>> two
>>     Hebrew characters (or what chardet thinks are Hebrew characters), it
>> may
>>     actually be a reasonable guess.
>>
>>
>> I get a traceback if I try to read the file as Windows-1255.  I don't
>> get a traceback if I read it as ISO-8859-1.
>>
>>     If it is a poor guess, perhaps you ought to report it to the chardet
>>     maintainers as a good example of a poor guess.
>>
>> I was considering that, and may do so.
>>
>> I've also been wondering if ISO-8859-1 is just an octet-oriented codec,
>> so it'll read about anything.  There are clearly non-7-bit-ASCII
>> characters in the file that look like line noise in an mrxvt.
>>
>
> Both ISO-8859-1 and Windows-1255 are octet-oriented, I don't see why one
> would raise an exception when the other didn't.  Unless the exception isn't
> on the decode, but instead on your attempt to output the result. Can you
> show the full traceback you're seeing?
>

$ ./movie-ratings
Traceback (most recent call last):
  File "./movie-ratings", line 85, in <module>
    main()
  File "./movie-ratings", line 68, in main
    ratings =
get_ratings('/home/dstromberg/src/home-svn/movie-ratings/trunk/mpaa-ratings-reasons.list')
  File "./movie-ratings", line 52, in get_ratings
    for line in ratings_file:
  File "/usr/local/cpython-3.3/lib/python3.3/encodings/cp1255.py", line 23,
in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0xfc in position
1225: character maps to <undefined>

BTW, other than satisfying our respective curiosities, I consider this
project finished.  It's probably not getting ratings for my entire movie
collection, but it is getting them for a significant fraction, which is all
I was really looking for.  Now I know which ones are rated PG, so I can
decide whether to let my 8 year old watch them.

This is with cpython-3.3.

Thanks.  ^_^
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20131211/10c53819/attachment.html>