[Pythonmac-SIG] Opening files with Japanese characters on Tiger

[James Kelly]

Hi all,

I ran into an issue recently opening files with Japanese characters on
Tiger.  Essentially I am reading from a utf-8 file containing lists of file
names.  Most are completely in English, a few contain Japanese characters in
the filename.  I am opening the files for a binary write.  Unfortunately
when I do this, and the filename contains Japanese characters, I get an
exception of type exceptions.IOError, with the specific exception being an
'invalid mode: wb.'

I've tried changing the encoding of the string in various ways, but as far
as I know, I should be passing in a utf-8 string to open, and the string is
already in utf-8.

This does work on Leopard, however.  I don't actually think it's a Python
issue directly though, as we're using py2app, so the python being used is
the same in (Python 2.5.0 from pythonmac.org) testing this on both Tiger and
Leopard.  There seems to be some underlying interaction with the file system
that is not going well on Tiger.  Any ideas?  Googling has really only
turned up this:

http://www.selenic.com/mercurial/bts/issue481

Which seems to be a similar issue.  Perhaps my google-foo is weak.  :)

At any rate, a work around has been instituted, the open command is in a
try/except block, and if the exception is thrown, we just write the file to
a temporary English file, then do a rename to the Japanese filename.  It
works, but it would be nice to not have to do that.

Any ideas appreciated, and I may end up at Filesystem-Dev, I just figured I
would start here.  Thanks!

Jim