[New-bugs-announce] [issue10051] distutils fail to install unicode-encoded files with POSIX locale

[Michał Górny]

New submission from Michał Górny <mgorny at gentoo.org>:

I was reported an issue where the user was unable to install my package [1] using python3.1. I was able to guess the issue comes from locale being set to POSIX and reproduce it.

It seems that the reasons are utf8-encoded characters in the installed script file. All the project files carry encoding declaration as per PEP-0263. They install fine with python2 but fail both with 3.1 and 3.2.

I think the build system should be locale-agnostic, and work correctly even if LC_ALL=C. If it's necessary to use text mode for scripts, I think a similar encoding guessing method should be used as in python itself.

I'm attaching a complete build log (for multiple versions of Python). Below I'm repasting the backtrace:

Traceback (most recent call last):
  File "setup.py", line 35, in <module>
    'Topic :: System :: Installation/Setup'
  File "/usr/lib64/python3.1/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/usr/lib64/python3.1/distutils/dist.py", line 919, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.1/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.1/distutils/command/build.py", line 128, in run
    self.run_command(cmd_name)
  File "/usr/lib64/python3.1/distutils/cmd.py", line 315, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python3.1/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.1/distutils/command/build_scripts.py", line 51, in run
    self.copy_scripts()
  File "/usr/lib64/python3.1/distutils/command/build_scripts.py", line 83, in copy_scripts
    first_line = f.readline()
  File "/usr/lib64/python3.1/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 127: ordinal not in range(128)

[1] http://github.com/mgorny/smart-live-rebuild

----------
assignee: tarek
components: Distutils
files: build.log
messages: 118188
nosy: eric.araujo, mgorny, tarek
priority: normal
severity: normal
status: open
title: distutils fail to install unicode-encoded files with POSIX locale
type: behavior
versions: Python 3.1, Python 3.2
Added file: http://bugs.python.org/file19159/build.log

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10051>
_______________________________________