[Python-Dev] Adding the 'path' module (was Re: Some RFE for review)

[Neil Hodgson]

Thomas Heller:

> OTOH, I once had a bug report from a py2exe user who complained that the
> program didn't start when installed in a path with japanese characters
> on it.  I tried this out, the bug existed (and still exists), but I was
> astonished how many programs behaved the same: On a PC with english
> language settings, you cannot start WinZip or Acrobat Reader (to give
> just some examples) on a .zip or .pdf file contained in such a
> directory.

   Much of the time these sorts of bugs don't make themselves too hard
to live with because  most non-ASCII names that any user encounters
are still in the user's locale and so get mapped by Windows. It can be
a lot of work supporting wide file names. I have just added wide file
name support to my editor, SciTE, for the second time and am about to
rip it out again as it complicates too much code for too few
beneficiaries. (I want one executable for both Windows NT+ and 9x, so
wide file names has to be a runtime choice leading to maybe 50 new
branches in the code).

   If returning a mixture of unicode and narrow strings from
os.listdir is the right thing to do then maybe it better for sys.argv
and os.environ to also be mixtures. In patch #1231336 I added parallel
attributes, sys.argvu and os.environu to hold unicode versions of this
information. The alternative, placing unicode items in the existing
attributes minimises API size.

   One question here is whether unicode items should be added only
when the element is outside the user's locale (the CP_ACP code page)
or whenever the item is outside ASCII. The former is more similar to
existing behaviour but the latter is safer as it makes it harder to
implicitly treat the data as being in an incorrect encoding.

   Neil