[issue9992] Command line arguments are not correctly decodediflocale and fileystem encodingsaredifferent

Mon Oct 11 16:32:34 CEST 2010

Ronald Oussoren <ronaldoussoren at mac.com> added the comment:

On 09 Oct, 2010,at 02:07 PM, Antoine Pitrou <report at bugs.python.org> wrote:

Antoine Pitrou <pitrou at free.fr> added the comment:

> For the command line, it would mean that we 
> introduced a new encoding: "command line encoding", which will be utf-8 on 
> OSX.

Or more generally "environment encoding", if it's also used for env
vars. This could solve the subprocess issue neatly.
Â 

Note that the command-line and environment encoding on OSX is generally UTF-8, even if that is not always reflected in the locale settings.

On recent OSX releases LANG will be set to a UTF-8 aware locale ("en_US.UTF-8" on my machine) when you start a shell using Terminal.app.

The correct locale environment variables are AFAIK not set in two important situations: on OSX 10.4 and when running code from an application bundle, in both cases the environment/command-line encoding should be treated as UTF-8.

There is one reason for not wanting to assume that the encoding is always UTF-8: the user might access the system from a non-UTF8 terminal (such as when logging in with an SSH session from a system not using UTF-8, or using an alternate terminal application). IMHO these are minor enough use-cases that we could just enforce that the encoding is UTF-8 on OSX.Â 

That would ensure that the filesystem encoding and environment/command-line encoding are consistent and we'd no longer run into the problem that triggered this issue.

Ronald

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9992>
_______________________________________

----------
title: Command line arguments are not correctly decodediflocale and fileystem encodingsaredifferent -> Command line arguments are not correctly decodediflocale	and fileystem encodingsaredifferent
Added file: http://bugs.python.org/file19184/unnamed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9992>
_______________________________________
-------------- next part --------------
<html><body><div><br><br>On 09 Oct, 2010,at 02:07 PM, Antoine Pitrou &lt;report at bugs.python.org&gt; wrote:<br><br></div><div><blockquote type="cite"><div><div class="_stretch"><br>
Antoine Pitrou &lt;pitrou at free.fr&gt; added the comment:<br>
<br>
&gt; For the command line, it would mean that we <br>
&gt; introduced a new encoding: "command line encoding", which will be utf-8 on <br>
&gt; OSX.<br>
<br>
Or more generally "environment encoding", if it's also used for env<br>
vars. This could solve the subprocess issue neatly.</div></div></blockquote><span>&nbsp;</span></div><div><br></div><div><span></span>Note that the command-line and environment encoding on OSX is generally UTF-8, even if that is not always reflected in the locale settings.</div><div><br></div><div>On recent OSX releases LANG will be set to a UTF-8 aware locale ("en_US.UTF-8" on my machine) when you start a shell using Terminal.app.</div><div><br></div><div>The correct locale environment variables are AFAIK not set in two important situations: on OSX 10.4 and when running code from an application bundle, in both cases the environment/command-line encoding should be treated as UTF-8.</div><div><br></div><div>There is one reason for not wanting to assume that the encoding is always UTF-8: the user might access the system from a non-UTF8 terminal (such as when logging in with an SSH session from a system not using UTF-8, or using an alternate terminal application). IMHO these are
  minor enough use-cases that we could just enforce that the encoding is UTF-8 on OSX.&nbsp;</div><div><br></div><div>That would ensure that the filesystem encoding and environment/command-line encoding are consistent and we'd no longer run into the problem that triggered this issue.</div><div><br></div><div>Ronald</div><div><blockquote type="cite"><div><div class="_stretch"><br>
<br>
----------<br>
<br>
_______________________________________<br>
Python tracker &lt;report at bugs.python.org&gt;<br>
&lt;<a href="http://bugs.python.org/issue9992" _mce_href="http://bugs.pythonorg/issue9992">http://bugs.python.org/issue9992</a>&gt;<br>
_______________________________________<br>
</div></div></blockquote></div></body></html>