Python 2.5 and Unicode on MacOS

[Peter Otten]

Michael J. Fromberger wrote:

> I'm not sure whether this is a bug, or a feature that I do not fully
> understand.  I would therefore appreciate some advice, if you have any,
> on the following problem:
> 
> I have just installed Python 2.5 (release) under MacOS 10.4.7 ("Tiger").
> Unfortunately, the unit test "test_unicodefile.py" (in Lib/tests/)
> exhibits a warning when invoked, as shown here:
> 
> [Lib/test] % python2.5 test_unicode_file.py
> test_directories (__main__.TestUnicodeFiles) ... ok
> test_unicode_file.py:103: UnicodeWarning: Unicode equal comparison
> failed to convert both arguments to Unicode - interpreting them as being
> unequal
>   filename1==filename2
> test_equivalent_files (__main__.TestUnicodeFiles) ... ok
> test_single_files (__main__.TestUnicodeFiles) ... ok
> 
> ----------------------------------------------------------------------
> Ran 3 tests in 0.252s
> 
> As you can see, the unit tests appear to pass, but I suspect there is a
> hidden problem, as a subsequent build of the latest version of
> BeautifulSoup now fails its unit tests, citing the same warning.  I
> can't tell whether this is a problem with the way BeautifulSoup is doing
> things, or whether there is a more general problem with Unicode support
> in the MacOS build of Python 2.5.
> 
> For reference, I have done a framework build of Python, and it seems to
> work fine for everything else I have tried.  Do you have any idea what
> might be causing this trouble?  I'd be grateful for your insights.

This Problem is not Mac-specific. Should u"ö" == "ö" evaluate to True, or
False, or should it raise an exception? 

I would favor always raising an exception, but the current behavior is the
result of an extensive discussion on python-dev (that I did not follow
completely) -- google for "unicode hell" and "dicts are broken". 

My personal conclusion is not to mix unicode and non-ascii str. I hope I'm
safe then...

Peter